Madden (2013) APA Handbook of Behavior Analysis (v.1)

INDEX BOOKS GROUPS
Perpetuando impressões!
APA Handbook of
Behavior
Analysis
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
APA Handbooks in Psychology
APA Handbook of
Behavior
Analysis
volume 1
Methods and Principles
Gregory J. Madden, Editor-in-Chief

William V. Dube, Timothy D. Hackenberg,
Gregory P. Hanley, and Kennon A. Lattal, Associate Editors
American Psychological Association • Washington, DC
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Copyright © 2013 by the American Psychological Association. All rights reserved. Except as permitted
under the United States Copyright Act of 1976, no part of this publication may be reproduced or distributed
in any form or by any means, including, but not limited to, the process of scanning and digitization,
or stored in a database or retrieval system, without the prior written permission of the publisher.
Published by
American Psychological Association
750 First Street, NE
Washington, DC 20002-4242
www.apa.org
To order
APA Order Department
P.O. Box 92984
Washington, DC 20090-2984
Tel: (800) 374-2721; Direct: (202) 336-5510
Fax: (202) 336-5502; TDD/TTY: (202) 336-6123
Online: www.apa.org/pubs/books/
E-mail: order@apa.org
In the U.K., Europe, Africa, and the Middle East, copies may be ordered from
American Psychological Association
3 Henrietta Street
Covent Garden, London
WC2E 8LU England
American Psychological Association Staff

Gary R. VandenBos, PhD, Publisher
Julia Frank-McNeil, Senior Director, APA Books
Theodore J. Baroody, Director, Reference, APA Books
Kristen Knight, Project Editor, APA Books
Typeset in Berkeley by Cenveo Publisher Services, Columbia, MD
Printer: United Book Press, Baltimore, MD

Cover Designer: Naylor Design, Washington, DC
Library of Congress Cataloging-in-Publication Data
APA handbook of behavior analysis / Gregory J. Madden, editor-in-chief ;

William V. Dube . . . [et al.], associate editors.
v. cm. — (APA handbooks in psychology)
Includes bibliographical references and index.
Contents: v. 1. Methods and principles — v. 2. Translating principles into practice.
ISBN 978-1-4338-1111-1 — ISBN 1-4338-1111-1
1. Behaviorism (Psychology) 2. Behaviorism (Psychology)—Handbooks, manuals, etc.
I. Madden, Gregory J. (Gregory Jude) II. Dube, William V. III. American Psychological Association.
IV. Title: Handbook of behavior analysis.
BF199.A67 2013
150—dc23
2011050720
British Library Cataloguing-in-Publication Data

A CIP record is available from the British Library.
Printed in the United States of America

First Edition
DOI: 10.1037/13937-000
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Contents
Volume 1: Methods and Principles
Editorial Board �� vii

About the Editor-in-Chief�� ix
Contributors�� xi
Foreword�� xv
Series Preface�� xix
Introduction �� xxi
Part I. Overview�� 1
Chapter 1. Single-Case Research Methods: An Overview�� 3
Iver H. Iversen
Chapter 2. The Five Pillars of the Experimental Analysis of Behavior�� 33
Kennon A. Lattal
Chapter 3. Translational Research in Behavior Analysis�� 65
William V. Dube
Chapter 4. Applied Behavior Analysis�� 81
Dorothea C. Lerman, Brian A. Iwata, and Gregory P. Hanley
Part II. Single-Case Research Designs�� 105

Chapter 5. Single-Case Experimental Designs �� 107
Michael Perone and Daniel E. Hursh
Chapter 6. Observation and Measurement in Behavior Analysis �� 127
Raymond G. Miltenberger and Timothy M. Weil
Chapter 7. Generality and Generalization of Research Findings �� 151
Marc N. Branch and Henry S. Pennypacker
Chapter 8. Single-Case Research Designs and the Scientist-Practitioner
Ideal in Applied Psychology �� 177
Neville M. Blampied
Chapter 9. Visual Analysis in Single-Case Research�� 199
Jason C. Bourret and Cynthia J. Pietras
Chapter 10. Quantitative Description of Environment–Behavior Relations�� 219
Jesse Dallery and Paul L. Soto
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Contents
Chapter 11. Time-Series Statistical Analysis of Single-Case Data�� 251

Jeffrey J. Borckardt, Michael R. Nash, Wendy Balliet,
Sarah Galloway, and Alok Madan
Chapter 12. New Methods for Sequential Behavior Analysis �� 267
Peter C. M. Molenaar and Tamara Goode
Part III. The Experimental Analysis of Behavior�� 281

Chapter 13. Pavlovian Conditioning�� 283
K. Matthew Lattal
Chapter 14. The Allocation of Operant Behavior�� 307
Randolph C. Grace and Andrew D. Hucks
Chapter 15. Behavioral Neuroscience�� 339
David W. Schaal
Chapter 16. Stimulus Control and Stimulus Class Formation�� 361
Peter J. Urcuioli
Chapter 17. Attention and Conditioned Reinforcement�� 387
Timothy A. Shahan
Chapter 18. Remembering and Forgetting�� 411
K. Geoffrey White
Chapter 19. The Logic and Illogic of Human Reasoning �� 439
Edmund Fantino and Stephanie Stolarz-Fantino
Chapter 20. Self-Control and Altruism�� 463
Matthew L. Locey, Bryan A. Jones, and Howard Rachlin
Chapter 21. Behavior in Relation to Aversive Events:
Punishment and Negative Reinforcement�� 483
Philip N. Hineline and Jesús Rosales-Ruiz
Chapter 22. Operant Variability�� 513
Allen Neuringer and Greg Jensen
Chapter 23. Behavioral Pharmacology �� 547
Gail Winger and James H. Woods
vi
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Editorial Board
EDITOR-IN-CHIEF
Gregory J. Madden, PhD, Associate Professor, Department of Psychology, Utah State
University, Logan
ASSOCIATE EDITORS
William V. Dube, PhD, Associate Professor, Eunice Kennedy Shriver Intellectual
and Developmental Disabilities Research Center and Department of Psychiatry,
University of Massachusetts Medical School, Shrewsbury
Timothy D. Hackenberg, PhD, Professor, Department of Psychology, Reed College,
Portland, OR
Gregory P. Hanley, PhD, Associate Professor, Department of Psychology, Western New
England University, Springfield, MA
Kennon A. Lattal, PhD, Centennial Professor, Department of Psychology, West Virginia
University, Morgantown
HANDBOOK EDITORIAL MANAGER

Kristen Knight, American Psychological Association, Washington, DC
vii
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
About the Editor-in-Chief
Gregory J. Madden, PhD, is a professor in the Department of Psychology at Utah State

University, Logan. He earned a master’s degree in behavior analysis from the University of
North Texas in 1992 and a doctorate in psychology from West Virginia University in 1995.
After completing a 3-year National Institutes of Health–funded postdoctoral research fellow-
ship at the University of Vermont, Dr. Madden held faculty appointments at the University
of Wisconsin–Eau Claire and the University of Kansas before joining the faculty at Utah
State University. Dr. Madden’s research falls under the umbrella of behavioral economics,
with an emphasis on impulsive choice and health decision making. He is the author or coau-
thor of many of the seminal scientific articles in the study of delayed-reward discounting and
its relation to addictions. His research in this area is supported by grants from the National
Institutes of Health (National Institute on Drug Abuse). He also holds grants from the U.S.
Department of Agriculture. The latter grants support behavioral economic approaches to
influencing dietary choices made by children in school cafeterias. Dr. Madden coedited,
with Warren K. Bickel, Impulsivity: The Behavioral and Neurological Science of Discounting
(American Psychological Association) and currently serves as the editor of the Journal of the
Experimental Analysis of Behavior, established in 1958 and the flagship journal of basic
research in behavior analysis. Dr. Madden has served on several important decision-making
bodies (e.g., the Executive Council of the Association for Behavior Analysis International)
and is the recipient of several teaching honors, including being selected as the 2011
G. Stanley Hall lecturer for American Psychological Association Division 2 (Society for the
Teaching of Psychology).
ix
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Contributors
William B. Abernathy, PhD, Department of Psychology, Southeastern Louisiana University,

Hammond, and Aubrey Daniels International, Atlanta, GA
William H. Ahearn, PhD, BCBA-D, The New England Center for Children, Western New
England University, Southborough, MA
Wendy Balliet, PhD, Department of Psychiatry and Behavioral Sciences, Medical University
of South Carolina, Charleston
Dan Bernstein, PhD, Center for Teaching Excellence, University of Kansas, Lawrence
Anthony Biglan, PhD, Oregon Research Institute, Eugene
Neville M. Blampied, MSc, Department of Psychology, University of Canterbury,
Christchurch, New Zealand
Richard R. Bootzin, PhD, Department of Psychology, University of Arizona, Tucson
Jeffrey J. Borckardt, PhD, Department of Psychiatry and Behavioral Sciences, Medical
University of South Carolina, Charleston
John C. Borrero, PhD, Department of Psychology, University of Maryland, Baltimore County
Jason C. Bourret, PhD, BCBA-D, The New England Center for Children, Western New
England University, Southborough, MA
Marc N. Branch, PhD, Department of Psychology, University of Florida, Gainesville
Christopher E. Bullock, PhD, Department of Behavioral Psychology, Kennedy Krieger
Institute, and Department of Pediatrics, The Johns Hopkins University School of
Medicine, Baltimore, MD
James E. Carr, PhD, BCBA-D, Behavior Analyst Certification Board, Tallahassee, FL
A. Charles Catania, PhD, Department of Psychology, University of Maryland, Baltimore County
Philip N. Chase, PhD, Cambridge Center for Behavioral Studies, Manchester, MA
Jesse Dallery, PhD, Department of Psychology, University of Florida, Gainesville
Edward J. Daly III, PhD, Department of Educational Psychology, University of
Nebraska–Lincoln
Iser G. DeLeon, PhD, Department of Behavioral Psychology, Kennedy Krieger Institute, and
the Department of Psychiatry and Behavioral Sciences, The Johns Hopkins University
School of Medicine, Baltimore, MD
Michael J. Dougher, PhD, Department of Psychology, University of New Mexico,
Albuquerque
Claudia L. Dozier, PhD, BCBA, Department of Applied Behavioral Science, University of
Kansas, Lawrence
xi
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Contributors
William V. Dube, PhD, Eunice Kennedy Shriver Intellectual and Developmental Disabilities
Research Center and Department of Psychiatry, University of Massachusetts Medical
School, Shrewsbury
Rogelio Escobar, PhD, Faculty of Psychology, National Autonomous University of Mexico,
Mexico City
Edmund Fantino, PhD, Department of Psychology, University of California San Diego
Monica T. Francisco, MA, Department of Psychology, Utah State University, Logan
Sarah Galloway, PhD, Department of Psychiatry and Behavioral Sciences, Medical
Sigrid S. Glenn, PhD, Department of Behavior Analysis, University of North Texas, Denton
Tamara Goode, MA, Department of Human Development and Family Studies, The
Pennsylvania State University, University Park
Randolph C. Grace, PhD, Department of Psychology, University of Canterbury,
Christchurch, New Zealand
Paul M. Guinther, PhD, Portland Psychotherapy Clinic, Research, and Training Center,
Portland, OR
Timothy D. Hackenberg, PhD, Department of Psychology, Reed College, Portland, OR
Louis P. Hagopian, PhD, Department of Behavioral Psychology, Kennedy Krieger Institute,
and the Department of Psychiatry and Behavioral Sciences, The Johns Hopkins University
Gregory P. Hanley, PhD, Department of Psychology, Western New England University,
Springfield, MA
Steven C. Hayes, PhD, Department of Psychology, University of Nevada, Reno
Sarah H. Heil, PhD, Departments of Psychiatry and Psychology, University of Vermont, Burlington
Stephen T. Higgins, PhD, Departments of Psychiatry and Psychology, University of
Vermont, Burlington
Philip N. Hineline, PhD, Department of Psychology, Temple University, Philadelphia, PA
Andrew D. Hucks, Doctoral Candidate, Department of Psychology, University of
Canterbury, Christchurch, New Zealand
Daniel E. Hursh, PhD, Department of Technology, Learning, and Culture, West Virginia
University, Morgantown
Steven R. Hursh, PhD, Institutes for Behavior Resources, Baltimore, MD
Iver H. Iversen, PhD, Department of Psychology, University of North Florida, Jacksonville;
and Oslo and Akershus University College of Applied Sciences, Oslo, Norway
Brian A. Iwata, PhD, Department of Psychology, University of Florida, Gainesville
Eric A. Jacobs, PhD, Department of Psychology, Southern Illinois University Carbondale
Greg Jensen, Doctoral Candidate, Department of Psychology, Columbia University, New
York, NY
Brooke A. Jones, MA, Department of Applied Behavioral Science, University of Kansas,
Lawrence
Bryan A. Jones, PhD, Department of Psychology, Kent State University, Ashtabula, OH
Sara Kupzyk, Doctoral Candidate, Department of Educational Psychology, University of
Nebraska–Lincoln
Patricia F. Kurtz, PhD, Department of Behavioral Psychology, Kennedy Krieger Institute,
and the Department of Psychiatry and Behavioral Sciences, The Johns Hopkins University
Kennon A. Lattal, PhD, Department of Psychology, West Virginia University, Morgantown
xii
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Contributors
K. Matthew Lattal, PhD, Department of Behavioral Neuroscience, Oregon Health & Science
University, Portland
Dorothea C. Lerman, PhD, BCB-A, School of Human Sciences and Humanities, University
of Houston–Clear Lake, Houston, TX
Michael E. Levin, MA, Department of Psychology, University of Nevada, Reno
Michael A. Lind, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, OH
Matthew L. Locey, PhD, Department of Psychology, Stony Brook University, Stony
Brook, NY
Alok Madan, PhD, MPH, Department of Psychiatry and Behavioral Sciences, Medical
Gregory J. Madden, PhD, Department of Psychology, Utah State University, Logan
R. Mark Mathews, PhD, Faculty of Health Sciences, University of Sydney, Sydney, Australia
William J. McIlvane, PhD, Department of Psychiatry, University of Massachusetts Medical
School, Worcester
Caio F. Miguel, PhD, BCBA-D, Department of Psychology, California State University,
Sacramento
Raymond G. Miltenberger, PhD, BCBA, Department of Child and Family Studies,
University of South Florida, Tampa
Peter C. M. Molenaar, PhD, Department of Human Development and Family Studies, The
Pennsylvania State University, University Park
Michael R. Nash, PhD, Department of Psychology, University of Tennessee, Knoxville
Nancy A. Neef, PhD, School of Physical Activity and Educational Services, The Ohio State
University, Columbus
Allen Neuringer, PhD, Psychology Department, Reed College, Portland, OR
John A. Nevin, PhD, Department of Psychology, University of New Hampshire, Durham
M. Christopher Newland, PhD, Department of Psychology, Auburn University, Auburn, AL
Henry S. Pennypacker, PhD, Department of Psychology, University of Florida, Gainesville
Michael Perone, PhD, Department of Psychology, West Virginia University, Morgantown
Christopher J. Perrin, PhD, School of Physical Activity and Educational Services, The Ohio
State University, Columbus
Cynthia J. Pietras, PhD, Department of Psychology, Western Michigan University,
Kalamazoo
Howard Rachlin, PhD, Department of Psychology, Stony Brook University, Stony Brook, NY
Griffin W. Rooker, PhD, BCBA, Kennedy Krieger Institute, and the Department of
Psychiatry and Behavioral Sciences, The Johns Hopkins University School of Medicine,
Baltimore, MD
Jesús Rosales-Ruiz, PhD, Department of Behavior Analysis, University of North Texas, Denton
David W. Schaal, PhD, Accuray, Inc., Sunnyvale, CA
Timothy A. Shahan, PhD, Department of Psychology, Utah State University, Logan
Murray Sidman, PhD, Sarasota, FL
Stacey C. Sigmon, PhD, Departments of Psychiatry and Psychology, University of Vermont,
Burlington
Paul L. Soto, PhD, Division of Behavioral Biology, Department of Psychiatry and Behavioral
Sciences, Johns Hopkins University, Baltimore, MD
Ralph Spiga, PhD, Department of Psychiatry and Behavioral Science, Temple University
School of Medicine, Philadelphia, PA
Claire St. Peter, PhD, Department of Psychology, West Virginia University, Morgantown
xiii
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Contributors
Stephanie Stolarz-Fantino, PhD, Department of Psychology, University of California San

Diego
Travis Thompson, PhD, Special Education Program, Department of Educational Psychology,
University of Minnesota, Minneapolis
Jeffrey H. Tiger, PhD, BCBA-D, Psychology Department, University of Wisconsin–Milwaukee
Jane Turner, MClinPsych, University of Sydney, Sydney, Australia
Peter J. Urcuioli, PhD, Department of Psychological Sciences, Purdue University, West
Lafayette, IN
Roger Vilardaga, PhD, Department of Psychiatry and Behavioral Sciences, University of
Washington, Seattle
Timothy R. Vollmer, PhD, Psychology Department, University of Florida, Gainesville
David P. Wacker, PhD, Center for Disabilities and Development, University of Iowa, Iowa
City
Timothy M. Weil, PhD, BCBA, Department of Child and Family Studies, University of
South Florida, Tampa
K. Geoffrey White, PhD, Department of Psychology, University of Otago, Dunedin, New
Zealand
Gail Winger, PhD, Department of Pharmacology, University of Michigan Medical School,
Ann Arbor
James H. Woods, PhD, Department of Psychology and Department of Pharmacology,
University of Michigan, Ann Arbor
xiv
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Foreword
I was startled recently to realize that when I first became acquainted with B. F. Skinner’s pio-
neering work, the field of behavior analysis did not exist. I and others thought we were going
to contribute to psychology through research on what Skinner had called operant behavior.1
Psychology, however, proved inhospitable. Intense opposition made us realize eventually
that we were involved in something so different that it would constitute a new field of study.
We started the Journal of the Experimental Analysis of Behavior, which is probably where the
name behavior analysis came from, and this journal is now in its 54th year of publication.
Many related journals followed, with perhaps the most widely read being the Journal of
Applied Behavior Analysis. Our largest organization, the Association for Behavior Analysis
International, has grown steadily, with more than 5,000 members from more than 40 countries
attending its most recent annual convention.
The American Psychological Association’s sponsorship of this handbook might appear to
represent a reversal of long-standing and deep philosophical, theoretical, and methodologi-
cal antagonisms. No such reversal, however, has taken place. This new recognition probably
simply reflects the historical datum that behavior analysis did originate within psychology,
thereby justifying a belated acknowledgment of behavior analysis as a successful offshoot
from psychological roots. In spite of the many university departments of behavior analysis
that are now springing up, it is true that many common interests relate the two fields. This
publication should not, however, lull students into thinking that psychology and behavior
analysis are the same.
It has been said, partly in jest, that psychology presents what everybody knows in a lan-
guage nobody understands. Readers of this handbook should not expect to find traditional
psychological concerns just being expressed in unfamiliar language. The differences that
have caused psychology to reject behavior analysis are real, and for many who experience
excitement when nontraditional ways of thinking prove productive, behavior analysis pro-
vides a never-ending source of both intellectual challenge and satisfaction. In the small space
allotted, perhaps I can outline a few major distinctions between the two disciplines that I
have found stimulating. The chapters of this handbook will fill out the story; students who
are seeking not familiarity with standard ways of looking at behavior but who find excite-
ment in unconventional thinking will experience much stimulation here.
For example, most psychologists continue to hold psychology to be the science of mind
and not the science of behavior. They consider behavior—which behavior analysts define as
Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. New York, NY: Appleton-Century-Crofts.
1
xv
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Foreword
what one does in relation to one’s environment—to be important only as a reflection of the
inner person. Philosophers, the general public, and most psychologists object to behavior ana-
lysts’ rejection of the inner person as a determiner of behavior. For many, that inner person is
the brain and nervous system. For others, it is in the genes or the endocrine system. For still
others, everything we as people do is important only as a reflection of particular classes of
behavior—personal values, belief systems, emotional states, customs, and so on—that are
themselves held to be basic determiners of the rest of our behavior. Finally, many people con-
sider behavior to be directed by a hypothetical entity such as the will, the ego, or the personal-
ity. For behavior analysts, however, behavior is important in its own right; the laws of nature
apply directly to what we do. Those laws undoubtedly include variables located in the ner-
vous system, the genes, the culture, and elsewhere. However interesting they are in their own
right, such variables exert their effects on behavior through laws that describe behavior itself.
Behavior analysts dispense with the myth of the inner person as the creator of behavior. Both
philosophically and empirically, we are what we do, and when and where we do it.
Philosophically, excitement arises from the recognition that what we do is just as subject
to natural law as is the behavior of any of our internal parts or of objects in our inanimate
environment. Does this imply that we have no control over our own destinies, that we are
nothing but machines? Among the most powerful sources of behavioral control are our indi-
vidual behavioral histories, in each of us a web of such complexity that it tempts us to give
up and claim independence. Just as the complexity of evolution causes many to claim an all-
powerful creator instead, the complexity of individual behavioral histories causes many to
claim control by an inner self. Contributors will reveal many of those historically generated
complexities and will point out where to seek more. Rather than needing to claim myself as
prepotent, I have always found it exciting to recognize that I am at one with the rest of the
universe and to help discover the laws that relate me to everything else.
Behavior analysis has always differed methodologically from psychology, largely because
of its emphasis on the observed behavior of individuals rather than the hypothetical behavior
that the average of a group represents. Skinner’s original work was technically elegant in
showing how to follow orderliness in the behavior of individuals, moment by moment, with-
out having to wait until one could average the behavior of many individuals.2 Unfortunately,
most psychologists have never experienced the thrill of discovery that individual-subject
methodology produces, with the generality of experimental results established not by statis-
tics but by replication.3
This methodological emphasis on the behavior of individuals is responsible for the wide-
spread success of applied behavior analysis. Although statistical evaluation of group data is
valuable in establishing the practicality of behavioral change techniques, both the discovery
and the application of new techniques require observations of individuals. Experimental and
applied behavior analysis spring from the same methodological roots. It will not be difficult
for readers of this handbook to see why traditional clinical psychologists, educators, and
management consultants find the remarkable spillover from the laboratory to the clinic,
classroom, and industrial management environments threatening.
An overriding behavior-analytic principle, fostered by both its philosophy and its meth-
odology and not espoused by any branch of psychology, is that reinforcement contingencies,
Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforcement. New York, NY: Appleton-Century-Crofts.
2
Sidman, M. (1988). Tactics of scientific research: Evaluating experimental data in psychology. Boston, MA: Authors Cooperative.
3
(Original work published 1960)
xvi
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Foreword
if–then relations, constitute its basic units of analysis: If, under particular environmental cir-
cumstances, a defined behavior occurs, then the behavior will produce particular conse-
quences. Such contingencies range from the two-term units that are found only in the
laboratory, where experimenters use them to construct behavior of exquisite complexity, to
the N-term units that characterize behavioral adaptation to the complexities of everyday life.
The technology involved in the construction and investigation of ever more complex indi-
vidual behavior is breathtaking. Do not skip the methodological details you will find
throughout this handbook; those details give the experimenter, clinician, educator, and
others their power both to analyze existing behavior and to construct new behavior.
Finally, an aspect of behavior analysis that I believe has been responsible for much of its
productivity is its view of the role of theory. The framing of hypotheses is often taken as a
defining characteristic of science. Neither I nor many of the contributors to this handbook
think so. How about the framing of answerable questions as a defining characteristic of sci-
ence? This subtle difference in approach is responsible for the pioneering spirit and the thrill
of discovery that characterizes behavior analysts’ reactions to the consequences of their own
behavior. Discovering something new provides a different kind of satisfaction than does the
confirmation of one’s own suppositions. The pages of this handbook form a record of cre-
ativity that is often considered to characterize the arts, not the sciences. Engagement in
behavior-analytic research and practice opens up a vista of reinforcers that is rarely to be
found in psychological research or practice.
Murray Sidman, PhD
xvii
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Series Preface
The APA Handbook of Behavior Analysis is the seventh publication to be released in the American
Psychological Association’s latest reference line, the APA Handbooks in Psychology™ series.
The series comprises multiple two- and three-volume sets focused on core subfields. Addi-
tionally, some single-volume handbooks on highly focused content areas within core sub-
fields will be released in coming years.
Each handbook set is formulated primarily to address the reference interests and needs of
researchers, clinicians, and practitioners in psychology and allied behavioral fields. A second-
ary purpose is to meet the needs of professionals in pertinent complementary fields (i.e., by
content area), be they corporate executives and human resources personnel; physicians, psy-
chiatrists, and other health personnel; teachers and school administrators; cultural diversity
and pastoral counselors; legal professionals; and so forth. Finally, the entire series is geared
to graduate students in psychology who require well-organized, detailed supplementary
texts, not only for “filling in” their own specialty areas but also for gaining sound familiarity
with other established specialties and emerging trends across the breadth of psychology.
Under the direction of small and select editorial boards consisting of top scholars in the
field, with chapters authored by both senior and rising researchers and practitioners, each
reference set is committed to a steady focus on best science and best practice. Coverage
focuses on what is currently known in the particular subject area (including basic historical
reviews) and the identification of the most pertinent sources of information in both core and
evolving literature. Volumes and chapters alike pinpoint practical issues; probe unresolved
and controversial topics; and present future theoretical, research, and practice trends.
The editors provide clear guidance to the “dialogue” among chapters, with internal cross-
referencing, demonstrating a robust integration of topics that leads the user to a clearer
understanding of the complex interrelationships within each field.
With the imprimatur of the largest scientific and professional organization represent-
ing psychology in the United States and the largest association of psychologists in the
world, with content edited and authored by some of its most respected members, the APA
Handbooks in Psychology series will be the indispensable and authoritative reference resource
to turn to for researchers, instructors, practitioners, and field leaders alike.
Gary R. VandenBos, PhD

APA Publisher
xix
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
Behavior analysis is at once a field, a discipline, and a science that takes an empirical
approach to understanding and influencing the behavior of individuals. Behavior analysts
may be found conducting (a) research on basic behavioral processes with nonhuman and
human subjects; (b) translational research designed to extend basic research to analyses,
syntheses, and applied research that address socially important human behavior; (c) inter-
vention research based on the basic and translational research to improve the human condi-
tion from gestation to gerontology (e.g., smoking by pregnant mothers; regular, special, and
higher education; clinical conditions; organizational behavior management; public health);
and (d) conceptual analyses (e.g., history, theory, philosophy). In the first three domains,
behavior analysis begins with the replicable behavioral processes and products of individu-
als, even if it ends with the behavioral processes and products of populations.
This emphasis on the behavior of individuals has its roots in the basic research laborato-
ries of Ivan Pavlov (1849–1936) and B. F. Skinner (1904–1990). Their research proved for-
mative in the founding and evolution of behavior analysis as it, and the research that it
inspired, demonstrated lawful, orderly relations among behavior and its antecedents and
consequences. For example, Pavlov demonstrated an elegant understanding of conditioned
salivary reflexes when he turned them on and off, much as one world a light switch, by pre-
senting and removing conditioned stimuli that had in previous sessions reliably preceded the
delivery of food. Attracted by the ability of these Pavlovian (classical) and operant condition-
ing principles to reliably produce stable baselines in the behavior of intact individual organ-
isms, biobehavioral researchers interested in the effects of drugs, brain lesions, stress, illness,
and so forth adopted (and adapted) Pavlov’s and Skinner’s research preparations, a practice
that continues in modern pharmacology, toxicology, and neuroscience laboratories today.
The replicability of basic processes and principles of Pavlovian and operant conditioning
made them staples of laboratory courses at universities around the world. A generation of
psychologists completed experiments in the systematic analysis (e.g., conditioning, extinc-
tion) of the behavior of intact participants (e.g., rats, pigeons, people) with tangible out-
comes (e.g., learning).
The discovery of processes and principles by which the behavior of individuals could be
profoundly and reliably influenced was translated quickly into interventions addressing
socially important human behavior. Pioneers such as John B. Watson (1878–1958), Florence
Mateer (1887–1961), Mary Cover Jones (1896–1987), and O. Hobart Mowrer (1907–1982)
began using Pavlovian conditioning principles to understand and treat human anxieties and
xxi
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
phobias. In the decades following, operant principles were discovered and analyzed and then
used by the first generation of behavior analysts to interpret socially important human
behavior. In Principles of Psychology, Keller and Schoenfeld (1950) integrated Skinner’s
(1938) fundamental discoveries with those of contemporary learning theorists and applied
these principles to a wide variety of domains, including personality, cooperation and con-
flict, and Freudian concepts. In Science and Human Behavior, Skinner (1953) used Pavlovian
and operant conditioning processes and principles to provide an internally consistent con-
ceptual analysis of everyday human behavior. Thus, although students were shaping the
behavior of rats and pigeons in laboratory courses, they were often concurrently considering
their extension and application to topics ranging from cognition to problems of self-control,
from economics to religion, and from psychotherapy to the intentional design of a culture.
This second generation of behavior analysts produced an explosion of basic and applied
research that necessitated the founding of two journals. In 1958, the Journal of the Experimen-
tal Analysis of Behavior began publishing basic laboratory research that expanded, extended,
and clarified the foundational principles of behavior analysis, an activity it continues today.
Issue after issue was packed with demonstrations of lawful, orderly, and replicable behavior
change observed within individual organisms. Several contributors to this journal soon began
to demonstrate the utility of these processes and principles in applied settings. The breadth
of applications and success was staggering, necessitating a new journal, the Journal of Applied
Behavior Analysis, which began publication in 1968. Nearly 40 years later, and on the 100th
anniversary of the birth of B. F. Skinner, the president of the Association for Psychological
Science, Henry L. Roediger, concisely described the impact of behavior analysis:
We know how to alleviate or eliminate phobias through extinction-based thera-
pies; we know the power of a token economy in regulating behavior on a men-
tal ward; we can reduce problematic behaviors and increase the probability of
desired behaviors by judiciously providing and withholding reinforcements.
Even for problems that cognitively oriented psychologists study, behavioristic
therapies are the treatments of choice. For an autistic child, Lovaas’s behavioristic
techniques provide the greatest (indeed, so far the only) hope. (Theory of mind
debates about autism are fine, but not if you want therapies and treatment—go to
behaviorism.) Similarly, for stuttering and aphasia, as interesting as their analysis
by psycholinguists may be, the treatments come largely from the behaviorists’
labs. In the field of neurobiology of learning, the central paradigm is classical
conditioning and the main theoretical model is the Rescorla–Wagner model. And
behavioristic analyses exist in self-management programs, in industry (Organi-
zational Behavior Management), in sports, in parenting guides, and of course in
animal training programs for pets and for zoos. Anywhere that prediction and
control of overt behavior is critical, one finds behavioristic analyses at work. In
sum . . . although most psychologists don’t know it, behaviorism still is alive and
thriving. (Roediger, 2004, p. 41)
Thus, two defining features of behavior analysis are (a) its focus on the behavior of the
individual and (b) the production of research findings that often have practical utility. A
third defining feature is the use of single-case research designs. Again, the behavior analyst’s
focus on individual behavior requires a means by which to evaluate within-subject effects of
independent variables without statistical averaging across subjects (Sidman, 1960). Today,
myriad single-case designs are available for the variety of questions faced by basic and
xxii
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
applied behavior analysts. Most of these designs emphasize steady-state behavioral assess-
ments and within-subject replication of experimental effects. These designs are ideally suited
to the goals of the practicing behavior analyst: determining the long-term efficacy of an inter-
vention on the behavior of individuals. As articulated by Hayes, Barlow, and Nelson-Gray
(1999) and by Blampied (Chapter 8, this volume), the scientist-practitioner model might be
more widely adopted and successfully implemented if the research methods taught in gradu-
ate programs were compatible with the demands placed on practitioners—to evaluate the
extent to which interventions make a difference in the lives of individuals. Single-case
research designs are well suited to this task and are the default among behavior analysts.
A final defining feature of behavior analysis is that as a field, it is careful about what qual-
ifies as an explanation of behavior. Specifically, behavior analysts reject reified constructs.
Reification often occurs when a temporally extended pattern of individual behavior that
might be described as, for example, impulsive is subsequently explained as being caused by
impulsivity. Citing the research of evolutionary psychologists, Dawkins (2006) suggested
that this tendency to reify human traits may have improved the fitness of individuals who
could use this heuristic to predict and adapt to the behavior of others. However, predicting
behavior is different than explaining it and, more important, influencing it. To accomplish
the latter, behavior analysts need to know the determinants of behavior, and these determi-
nants cannot be reified behavioral observations (e.g., Bechtel, 2005). The scientifically defen-
sible causes of a temporally extended pattern of impulsive action are to be found in the
individual’s genetics, current biological state, current environmental context, past learning
about environment–behavior relations, and neurochemical processes and changes underly-
ing them (e.g., Anderson & Woolverton, 2005; Anokhin, Golosheykin, Grant, & Heath,
2011; Odum, 2011; Schweitzer & Sulzer-Azaroff, 1988; Winstanley, Theobald, Dalley, Car-
dinal, & Robbins, 2006). The first and in the preceding sentence is important to the behavior
analyst because these variables are so intertwined that to omit one from the equation would
result in an incomplete explanation (e.g., Schaal, 2005; Skinner, 1989).
Of course, Pavlov and Skinner did not possess expertise in genetics, biological systems,
behavior analysis, pharmacology, and neuroscience. Thus, their principles of behavior were
(and are) incomplete, a fact that fuels behavioral research today. Contemporary behavior-
analytic theorists build mathematical models that rarely include variables beyond the charac-
teristics of antecedent stimulus and consequent events (see Chapter 10, this volume, and
Volume 2, Chapter 7, this handbook). Does this mean that these equations are incomplete?
Yes. Does it mean they are wrong? It depends on what the researcher or the practitioner is
tasked with accomplishing. If the equations are to explain all behavior of all organisms under
all conditions, then they are hopelessly flawed. However, if they are used to make novel pre-
dictions about the outcomes of experiments and interventions that manipulate their quanti-
tative parameters, then the equations may prove to have great utility. The necessity of a
complete explanation of behavior depends on the task at hand; the behavioral principles enu-
merated throughout this handbook are incomplete, but their practical utility is demonstrated
every day by those researchers and practitioners mentioned by Roediger (2004).
For practical reasons, then, the training of behavior analysts focuses primarily on ontoge-
netic learning processes (i.e., how biological organisms, possessing inherited behavioral
tendencies, come to interact adaptively with their social and physical environments; see
Chapter 2, this volume) and research methods that will allow the further elucidation of these
processes in individual organisms (see Chapter 1, this volume). Thus, with an incomplete
understanding of the determinants of behavior, behavior analysts strive to fill in the gaps in
xxiii
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
people’s understanding and to use the most reliable components of the current understand-
ing to take effective action in bettering the human condition.
In the APA Handbook of Behavior Analysis, the associate editors and I have endeavored to
represent the field comprehensively while embracing and expanding the translational tradi-
tion of behavior analysis. The latter has been accomplished by, where possible, combining
teams of basic and applied researchers to collaboratively author chapters. As a result, the
chapters in this two-volume set are particularly well suited to students of behavior who are
interested in translating contemporary research findings into procedures that will signifi-
cantly improve the lives of individuals.
The handbook is composed of two volumes and five parts. Part I begins with chapters
providing overviews of the four subsequent parts that span Volumes 1 and 2. In the first
overview chapter (Chapter 1, this volume), Iversen provides a brief (and fascinating) his-
tory of the development of single-case research designs, followed by a concise overview of
the logic behind and procedures used in the most common of these designs. The chapter is
rounded out with an insightful consideration of the pros and cons of the variety of data
analysis techniques that may be used with these designs. In his overview chapter (Chapter 2,
this volume), Kennon A. Lattal discusses the five pillars of the experimental analysis of
behavior: (a) the inductive single-case research methods used by behavior analysts, (b) rein-
forcement, (c) punishment, (d) control by stimuli correlated with reinforcers and punish-
ers, and (e) contextual and stimulus control. The latter four pillars outline basic principles
of ontogenetic learning—how consequences and stimuli correlated with them influence
behavior. Chapters 1 and 2 (and the remaining chapters in Volume 1) are well suited to stu-
dents undertaking graduate coursework in either single-case research designs or basic prin-
ciples of behavior analysis.
The remaining two overview chapters (Chapters 3 and 4, this volume) correspond to the
content of Volume 2. Chapter 3, by Dube, discusses the modern push by funding organiza-
tions such as the National Institutes of Health to translate basic research findings into appli-
cations that benefit human health and well-being. Dube notes that these translational
activities were an early development in behavior analysis and discusses how research in
applied settings has often produced a reverse translation in which applied research findings
are fed back into basic research activities. In Chapter 4, Lerman, Iwata, and Hanley discuss
the historical roots of applied behavior analysis, noting its unique characteristics within
behavior analysis. Their overview outlines the near-universal characteristics of applied
behavioral research when the goals of the intervention are response acquisition, mainte-
nance, generalization, and, in the case of problem behavior, suppression.
The remaining chapters in Volume 1 are divided into two parts. Part II provides compre-
hensive coverage of single-case research designs and data analysis techniques. In Chapter 5,
Perone and Hursh expand significantly on the coverage of single-case designs provided by
Iversen in Chapter 1. In Chapter 6, Miltenberger and Weil discuss the variety of methods
used by behavior analysts to observe and accurately record behavior in real time in the vari-
ety of laboratory and applied settings in which operant behavior occurs. In Chapter 7,
Branch and Pennypacker discuss the logic of single-case research designs in relation to the
group designs more frequently used in psychology. They make a compelling argument that
building a stable behavioral science requires that the reliability and generality of its theories
be established through direct replication (i.e., demonstrating that the same experimental
manipulation will reliably and repeatedly affect individual behavior) and systematic replica-
tion (i.e., establishing that the same experimental manipulation will reliably affect behavior
xxiv
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
in different settings and with different individuals). In Chapter 8, Blampied takes the logic
developed in the preceding three chapters and argues forcefully that single-case research
designs are ideally suited to the needs of scientist-practitioners in psychology and the behav-
ioral sciences.
The remaining chapters in Part II of Volume 1 tackle data analysis issues. In Chapter 9,
Bourret and Pietras discuss the variety of ways in which behavior analysts will graph time-
series data and conduct visual analyses to evaluate the stability of a baseline and determine
whether an experimental variable has systematically affected behavior. In Chapter 10, Dal-
lery and Soto provide a user-friendly overview of the quantitative methods frequently used
by behavior analysts involved in testing mathematical models of behavior (i.e., equations
that make precise predictions about behavior on the basis of the values of experimentally
manipulated parameters in the model). Readers who have been reluctant to tackle articles
that include models such as the matching law or delay discounting will want to study this
chapter. In Chapter 11, Borckardt, Nash, Balliet, Galloway, and Madan discuss the variety of
inferential statistical methods that have thus far been developed to aid in the analysis of
time-series data. Finally, Molenaar and Goode (Chapter 12) provide an introduction to
dynamic systems models that may be used to describe sequential dependencies in time-series
data and to model precisely anticipated effects of experimental parameters.
The coverage of single-case research designs and data analysis techniques prepares the
reader for Part III of Volume 1, which includes chapters covering a range of contemporary
research areas in the experimental analysis of behavior. Chapter 13 by K. Matthew Lattal
provides an excellent overview of contemporary theories of Pavlovian conditioning. Behav-
ior analysts tend to place more emphasis on operant than on Pavlovian conditioning pro-
cesses (as does this handbook), so readers will not want to overlook this important chapter.
In Chapter 14, Grace and Hucks tackle one of the most important topics in behavior analy-
sis: an understanding of choice. They provide approachable overviews of contemporary
quantitative models of choice, including the matching law, delay-reduction theory, contex-
tual choice model, and hyperbolic value-added model. In Chapter 15, Schaal provides an
overview of some of the most important advances in behavioral neuroscience, including the
neurological correlates of conditioned stimuli and reinforcement. Chapter 16 by Urcuioli
provides a primer on stimulus control and a systematic treatment of how different stimuli
come to be associated with categories or classes. This chapter is important for those inter-
ested in teaching discriminations and classification skills and is a must-read before these
more applied issues are tackled by McIlvane in Volume 2, Chapter 6, of this handbook.
The next four chapters in Volume 1 tackle topics that those outside of behavior analysis
may consider outside the realm of behavioral analysis. In Chapter 17, Shahan uses quantita-
tive models (several previously introduced by Grace & Hucks in Chapter 14) to understand
attentional processes. In Chapter 18, White outlines behavioral procedures for studying
remembering and forgetting as well as three contemporary behavior theories of these phe-
nomena. Irrational decision making is discussed in the context of behavioral economics and
game theory in Chapter 19 by Fantino and Stolarz-Fantino. Finally, in Chapter 20, Locey,
Jones, and Rachlin use (once again) equations introduced by Grace and Hucks in Chapter 4
to understand impulsivity, self-control, and altruism.
The final three chapters of Volume 1 provide a prelude to the translational research intro-
duced in Volume 2. In Chapter 21, Hineline and Rosales-Ruiz note that aversive events are a
part of the natural world in which species have evolved and live contemporarily. Thus, the
effects of this class of environmental events on ontogenetic behavior must not be ignored.
xxv
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
Their chapter summarizes the understanding of punishment and negative reinforcement and
provides a thoughtful consideration of the ethics surrounding these procedures. In Chapter 22,
Neuringer and Jensen discuss the importance of variation within evolving systems (such as a
pattern or repertoire of behavior) and provide extensive empirical evidence that variation is
a dimension of behavior that may be selected by consequences. They extend their findings to
translational topics such as teaching new behaviors, problem solving, creativity, and psycho-
pathology (to name a few). Finally, Chapter 23, by Winger and Woods, provides an over-
view of how a wide variety of commonly prescribed drugs, abused drugs, or both affect
specific aspects of operant and Pavlovian behavior. In addition to providing an approachable
summary of pharmacokinetics, these authors discuss drugs as antecedent and consequent
stimuli—the latter providing a widely used model of human drug seeking, drug taking, and
drug abuse.
Volume 2 of the handbook consists of two parts: Translational Research in Behavior Anal-
ysis and Applied and Clinical Issues. In Chapter 1, Guinther and Dougher discuss how basic
behavioral processes and principles have been incorporated into the variety of therapies used
by behavior analysts working in outpatient clinical settings. The chapter provides a discus-
sion (complementary to that in Chapter 3, this volume) of how the direction of translation
was not one way. Translating basic principles discovered in nonhuman laboratories was use-
ful to a point when working with verbally competent humans whose disorder involved mal-
adaptive cognition. The arrow of translation then bent as several clinical behavior analysts
(e.g., Brownstein, Dougher, Hayes) returned to the human laboratory to explore cognition as
a dependent variable. The result was a new approach to behavior therapy in which the deter-
minants and stimulus functions of cognition are targeted for change, an important break
from the mainstream view that disordered cognition is the determinant of the clinical
patient’s presenting problem (see Volume 2, Chapter 18, by Levin, Hayes, & Vilardaga).
In Chapter 2, Thompson provides several examples of biobehavioral research that have led
to important translations in areas ranging from substance abuse to the treatment of self-injury
among individuals with intellectual and developmental disabilities (IDDs). Thompson con-
tends that neural processes operating in parallel with overt operant behavior (and operating in
the absence of overt operant behavior, as is presumably the case in sensory preconditioning)
constitute a fourth term to be added to the three-term contingency (antecedent–behavior–
consequence) that must be understood to achieve a complete and, in some cases, practical
account of behavior.
Chapter 3, by DeLeon, Bullock, and Catania, translates well-established and more contem-
porary basic research findings in the area of reinforcement processes (e.g., schedules of rein-
forcement) to designing effective instructional contexts for typically developing children and
individuals with IDDs. The chapter includes an empirically informed discussion of the oft-
raised concerns about reinforcement’s negative effects on intrinsic motivation. In Chapter 4,
Kennon A. Lattal, St. Peter Pipkin, and Escobar discuss basic research in the areas of extinc-
tion, resurgence of previously reinforced operants, and reinstatement of extinguished operants.
These authors discuss how these findings have been, or could be, translated to effective meth-
ods of decreasing problem behavior. Response persistence in the face of disruptors such as
extinction is discussed by Nevin and Wacker (Chapter 5). These authors provide an approach-
able overview of behavioral momentum theory and its translational implications for applied
settings in which the goal is the persistence of adaptive behavior into nontherapeutic settings.
In Chapter 6, McIlvane discusses how the principles covered in Chapters 16 and 17 (this
volume) have been translated to developing intricate instructional programs designed to
xxvi
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
help the learner make simple and complex discriminations (e.g., same–different). The
author provides an overview of one of the most important of complex discriminations:
stimulus–stimulus relations in which the members of a stimulus class have no common
features (e.g., vocal words, printed words, and the objects to which they correspond). In
Chapter 7, Jacobs, Borrero, and Vollmer build on several chapters (e.g., Chapters 10 and 14,
this volume) to consider how quantitative models of behavior (e.g., the matching law, forag-
ing, and signal detection theory) have been, or could be, translated to applied settings (e.g.,
treatment of severe problem behavior, behavior of human data collectors in applied research).
The final three chapters in Volume 2, Part I, consider translations to behavior of substan-
tial social significance—addictions, toxic substances, and public health. In Chapter 8, Hursh,
Madden, Spiga, DeLeon, and Francisco provide an overview of how behavioral economic
concepts and measures have augmented behavioral processes and principles to provide
insights into reinforcer value, reinforcer interactions (substitutes and complements), and
new applications in the areas of addiction and in promoting adaptive behavior among indi-
viduals with IDDs. In Chapter 9, Newland provides an introduction to behavioral toxicol-
ogy, along with a discussion of how behavioral research findings have influenced and will
continue to influence public policy. Finally, in Chapter 10, Biglan and Glenn discuss how
operant contingencies may select cultural practices and how adopting a nurturing positive
reinforcement approach might enhance public health.
Part II of Volume 2 tackles a broad range of applied and clinical issues. Much of Part II is
organized by population or diagnosis to allow the authors to describe how behavioral princi-
ples have been used in, adapted to, and expanded to specific applied and clinical settings.
Chapter 11, by Kurtz and Lind, discusses one of the clinical areas in which applied behavior
analysis represents a widespread treatment approach: in working with individuals with IDDs.
Chapter 12, by Ahearn and Tiger, considers another area: autism. Both chapters provide excel-
lent coverage of the empirical evidence for specific techniques designed to address the unique
challenges of individuals with these diagnoses. In Chapter 13, Carr and Miguel discuss how a
behavior-analytic approach to verbal behavior has proven effective in understanding and over-
coming the language challenges of individuals with autism and IDDs. In Chapter 14, Hago-
pian, Dozier, Rooker, and Jones discuss effective methods of ameliorating the severe problem
behavior often observed in these populations. Chapter 15, by Neef, Perrin, and Madden,
expands the scope of diagnoses to individuals with attention-deficit/hyperactivity disorder.
Chapter 16, by Daly and Kupzyk, discusses behavioral approaches to teaching reading
using many of the principles discussed in prior chapters (e.g., Volume 2, Chapter 6); empiri-
cal evidence supporting many of these approaches is considered. Behavioral approaches to
the prevention and treatment of pediatric and adult sleep disorders are discussed in Chapter 17
by Blampied and Bootzin.
In Chapter 18, Levin, Hayes, and Vilardaga pick up where Guinther and Dougher (Vol-
ume 2, Chapter 1) left off—discussing the two-way translational process between basic labo-
ratory and clinical settings. The results of basic laboratory research conducted by clinically
oriented behavior analysts led Hayes and others to what Levin et al. refer to as a functional
contextual approach to cognition and language, leading ultimately to a new approach to psy-
chotherapy: acceptance and commitment therapy. An overview of this therapy is integrated
with the relational frame theory on which it is based, and the extant empirical evidence
informing the therapy is reviewed.
The final four chapters outline the extension of behavior-analytic principles to four very
different contexts. In Chapter 19, Higgins, Heil, and Sigmon provide an introduction to
xxvii
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
arranging reinforcement contingencies as a means of addressing addictions. Extensive

empirical evidence is reviewed, demonstrating that this behavior-analytic approach is highly
effective in producing drug abstinence across a wide variety of substance use disorders.
Chapter 20, by Abernathy, discusses the variety of ways in which behavioral principles have
proven useful in addressing human performance issues in workplace settings. In Chapter 21,
Bernstein and Chase provide an overview of behavioral approaches to teaching in institutes
of higher learning. Their chapter contains a thoughtful discussion of what colleges and uni-
versities value in learning outcomes and how behavioral approaches to teaching have yet to
adequately give this class of consumers what it wants. Finally, in Chapter 22, Turner and
Mathews discuss an area of future growth for behavior analysis: behavioral gerontology. Sev-
eral successful applications are discussed, as are areas for future research.
The breadth of topics covered in this two-volume handbook underscores the success
of the two-way translational research strategy adopted by behavior analysts: (a) Identify
experimental manipulations that reliably affect the behavior of individual nonhumans
under controlled laboratory conditions; (b) evaluate the extent to which the functional
equivalent of these manipulations influences the behavior of individual humans; (c) when
necessary, identify additional principles to account for uniquely human behavior (e.g.,
language and related cognition); and (d) translate these internally consistent, systematic
principles to address socially significant issues that, at their core, involve human
behavior.
Behavior analysts faced with one of these socially significant issues will conceptualize the
issue from a functional perspective; that is, they will work from the assumption that some of
the variance in the behavior of an individual (that not accounted for by genetic variables,
environmental toxins, etc.) is lawfully related to regularities of the interaction between
behavior and the individual’s environment. By experimentally manipulating components of
this environment, the behavior analyst may garner evidence for and against environment–
behavior relations specific to the behavior of interest. When these relations are understood,
the behavior analyst is in a better position to engineer the environment so that more adap-
tive behavior may emerge. This strategy has been successfully applied to the myriad specific
diagnoses, populations, and contexts covered in Volume 2 and to many more that even a
two-volume handbook could not contain. As new or refined principles of behavior are dis-
covered in the basic research laboratory and through bidirectional translational research,
behavior analysts anticipate that the specifics of the environment–engineer process will
change (sciences do not stagnate, and they produce new technologies). What will not change
is the functional perspective; behavior serves a function, and understanding that function
has and will continue to yield useful technologies of behavior.
Gregory J. Madden
Editor-in-Chief
References
Anderson, K. G., & Woolverton, W. L. (2005). Effects of clomipramine on self-control choice
in Lewis and Fischer 344 rats. Pharmacology, Biochemistry, and Behavior, 80, 387–393.
doi:10.1016/j.pbb.2004.11.015
Anokhin, A. P., Golosheykin, S., Grant, J. D., & Heath, A. C. (2011). Heritability of delay discounting
in adolescence: A longitudinal twin study. Behavior Genetics, 41, 175–183. doi:10.1007/s10519-
010-9384-7
xxviii
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Introduction
Bechtel, W. (2005). The challenge of characterizing operations in the mechanisms underlying behav-
ior. Journal of the Experimental Analysis of Behavior, 84, 313–325. doi:10.1901/jeab.2005.103-04
Dawkins, R. (2006). The God delusion. Boston, MA: Houghton Mifflin.
Hayes, S. C., Barlow, D. H., & Nelson-Gray, R. O. (1999). The scientist practitioner (2nd ed.).
Needham Heights, MA: Allyn & Bacon.
Keller, F. S., & Schoenfeld, W. N. (1950). Principles of psychology. New York, NY: Appleton-Century-
Crofts.
Odum, A. L. (2011). Delay discounting: Trait variable? Behavioural Processes, 87, 1–9. doi:10.1016/j.
beproc.2011.02.007
Roediger, H. L. (2004, March). What happened to behaviorism? APS Observer, 17, 40–42.
Schaal, D. W. (2005). Naming our concerns about neuroscience: A review of Bennett and Hacker’s
philosophical foundations of neuroscience. Journal of the Experimental Analysis of Behavior, 84,
683–692. doi:10.1901/jeab.2005.83-05
Schweitzer, J. B., & Sulzer-Azaroff, B. (1988). Self-control: Teaching tolerance for delay in impulsive chil-
dren. Journal of the Experimental Analysis of Behavior, 50, 173–186. doi:10.1901/jeab.1988.50-173
Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. New York,
NY: Basic Books.
Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. Oxford, England: Appleton-
Century.
Skinner, B. F. (1953). Science and human behavior. Toronto, Ontario, Canada: Collier-Macmillan.
Skinner, B. F. (1989). Origins of cognitive thought. American Psychologist, 44, 13–18.
doi:10.1037/0003-066X.44.1.13
Winstanley, C. A., Theobald, D. E., Dalley, J. W., Cardinal, R. N., & Robbins, T. W. (2006). Double
dissociation between serotonergic and dopaminergic modulation of medial prefrontal cortex and
orbitofrontal cortex during a test of impulsive choice. Cerebral Cortex, 16, 106–114. doi:10.1093/
cercor/bhi088
xxix
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Part I
Overview
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 1
Single-Case Research Methods:

An Overview
Iver H. Iversen
My experiments had indeed gone well. Research Methods Using Single Subjects
I was getting data from a single rat that The single-case research method involves repeated
were more orderly and reproducible than measures of one individual’s behavior before, dur-
the averages of large groups in mazes ing, and often after an experimental, educational,
and discrimination boxes, and a few or therapeutic intervention. Data are collected
principles seemed to be covering a lot of repeatedly over several observation periods or ses-
ground. (Skinner, 1979, p. 114) sions in what is customarily called a time series. The
Replication is the essence of believability. objective is to change a behavior to determine the
(Baer, Wolf, & Risley, 1968, p. 95) variables that control that behavior. When environ-
mental variables have been found that reliably
Single-case research methods refer to a vast collection
change behavior, the method can be used to control
of procedures for conducting behavioral research with
behavior. The investigator, educator, or therapist
individual subjects. Such methods are used in basic
can make the behavior start and stop and change
research and for improving behavioral problems with
frequency or duration at a specific time and place.
educational and therapeutic interventions. Analyses
Therefore, single-case methods provide a tool for a
and interpretations of data collected with research
science of behavior at the level of the individual sub-
methods for individual subjects have developed into
ject. When an educator or therapist can effectively
procedures that are considerably different from those
control the client’s behavior, then methods exist for
used in research with groups of subjects. In this chap-
helping that client acquire new behavior or over-
ter, I provide an overview of designs, analyses, and
come a problem with existing behavior. This ability
interpretations of research results and treatment out-
to help a client by using methods to control the cli-
comes using single-case research methods.
ent’s behavior is exactly what often creates contro-
versy around the use of such methods. The methods
Background and History
for controlling behavior and for helping the client
The case in single-case research methods essentially are the same, but the verbs control and help are not
refers to a unit of analysis for an individual, a few synonymous. To control behavior means to be able
people in a group, or a large group with varying to change behavior reliably, and in this context con-
membership. Single-case research methods should trol has a technical meaning originating from the
be contrasted with single-case studies that ordinarily laboratory. The word control, however, also has
consist of anecdotal narrations of what happened to political and societal meanings related to authorita-
a given person. No treatments or manipulations of tive restrictions of behavior for the individual. The
experimental variables take place in case studies. intended helping function of an applied behavior
I thank Dominik Guess and Wendon Henton for valuable comments on earlier versions of this chapter.
DOI: 10.1037/13937-001
APA Handbook of Behavior Analysis: Vol. 1. Methods and Principles, G. J. Madden (Editor-in-Chief)
3
Copyright © 2013 by the American Psychological Association. All rights reserved.
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
science is the opposite—to establish enrichment and number of data points for each individual because of
expansion of the individual’s behavioral repertoire, the repeated observations of that individual’s behav-
not to restrict it. This complex issue has lead to ior. Therefore, it is incorrect and misleading to refer
several considerations regarding the ethics involved to the single-case research method as an N = 1
in helping a client. For example, Skinner (1978) design, falsely implying that only one data point has
argued that a client who needs help to obtain essen- been collected and that the researcher is trying to
tial goods for survival should have a right to acquire promote a finding based on a single data point.
a behavior that can provide the goods rather than
merely be provided the goods regardless of any Glimpses Into the History of
behavior. Similarly, Van Houten et al. (1988) Single-Case Research Methods
stated that I. P. Pavlov’s (1927) Conditioned Reflexes had a
major influence on B. F. Skinner’s decision to study
individuals who are recipients . . . of
psychology and on his choice of research methodol-
treatment designed to change their
ogy (Catania & Laties, 1999; Iversen, 1992; Skinner,
behavior have the right to a therapeutic
1966). Skinner was impressed with Pavlov’s precise,
environment, services whose overriding
quantitative measures of behavior in one organism
goal is personal welfare, treatment by a
at a time. Thus, Skinner (1956) once wrote, “I had
competent behavior analyst, programs
the clue from Pavlov: Control your conditions and
that teach functional skills, behavioral
you will see order” (p. 223). Pavlov described in
assessment and ongoing evaluation, and
great detail control conditions and recording of indi-
the most effective treatment procedures
vidual drops of saliva at a given time of day for a
available. (p. 381)
single animal. Figure 1.1 shows an early version of a
Because of the overall success of single-case single-case research method used by Pavlov (1928).
research methods, a plethora of articles, book chap- The dog, Krasavets, had previously been condi-
ters, and entire books devoted to the topic have tioned to a positive tone (positive stimulus, or S+)
appeared over the past 40 years. Recent publications with food powder. Over nine trials, Pavlov alter-
in this vast literature illustrate the wide use of nated the S+ with a negative tone (negative stimu-
single-case research methods: Specific topics are lus, or S−) that had not been conditioned to
basic methodology (Barlow, Nock, & Hersen, 2009; food. The S+ elicited several drops of saliva from
J. M. Johnston & Pennypacker, 2009), educational two glands, and the S− elicited no saliva. The
research (Kennedy, 2005; Sulzer-Azaroff & Mayer,
1991), health sciences (Morgan & Morgan, 2009),
clinical and applied settings (Kazdin, 2011), com-
munity settings (O’Neill, McDonnell, Billingsly, &
Jenson, 2010), and medicine (Janosky, Leininger,
Hoerger, & Libkuman, 2009).
The term single-case research methods is synony-
mous with a variety of related terms, the most com-
mon of which are single-subject designs and N = 1
designs. The last term should be avoided because it
is misleading. N = 1 obviously means the use of Figure 1.1. Illustration of an early single-case
research method. Detailed data for a single dog
only one subject. However, N has an entirely differ- from Pavlov’s experiments on classical conditioning.
ent meaning in statistics, where it stands for the Positive tone is followed by food; negative tone is not.
number of data points collected, not for the number From Lectures on Conditioned Reflexes: Twenty-Five
of subjects. In customary group research, each Years of Objective Study of the Higher Nervous Activity
(Behaviour) of Animals (Vol. 1, p. 173), by I. P. Pavlov,
subject contributes exactly one data point. In con- 1928, New York, NY: International Publishers. In the
trast, single-case research methods generate a high public domain.
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Single-Case Research Methods
a lternation was irregular by design so that Pavlov (1865/1957). Nearly 150 years ago, and long before
could examine what happened to the conditioned Loeb, Pavlov, Thorndike, Watson, and Skinner, he
reflex to S+ after several presentations of S−; articulated in clear terms the need for what he called
indeed, the elicitation was reduced, as can be seen comparative experiments with individual, intact ani-
for S+ at 2:32. Thus, at the level of the individual mals. When using several animals for comparison,
dog, Pavlov first demonstrated a baseline of reliable he realized that the errors in his data stemmed from
elicitation of saliva to S+ and then demonstrated variations from animal to animal, and he wrote that
that repeated, successive presentations of S− inhib- “to remove this source of error, I was forced to
ited the flow of saliva on the next presentation make the whole experiment on the same animal . . .
of S+. because in this respect two frogs are not always
Pavlov’s work in Russia and contemporary work comparable” (p. 183). Bernard’s work appeared in
in Europe by Wundt on perception and by Weber Russian, and his work had a major influence on
and Fechner on psychophysics grew from physiol- Russian physiologists while Pavlov was a young
ogy, in which the customary method was to investi- researcher. Of particular historical significance, Pav-
gate the effects of independent variables on individual lov’s main professor, Botkin, had been a student of
organisms (Boring, 1929). In Europe, Ebbinghaus Bernard’s (see Paré, 1990), and Pavlov expressed the
(1885/1913) became famous for his memory studies greatest admiration for Bernard’s experimental
using a single subject (himself). In the United States, approaches (Todes, 2002). Indeed, according to
Thorndike’s (1911) research on the law of effect Wood (2004), Pavlov was an apostle of Bernard’s.
with individual cats quickly became well-known As for single-case methods with humans, Watson
(Boakes, 1984). Apparently, Thorndike was ahead of and Rayner (1920) and Jones (1924) demonstrated
Pavlov by a few years. Thus, Pavlov (1928) wrote, conditioning and extinction of fear reactions in
infants (i.e., “little Albert” and “little Peter”). The
Some years after the beginning of the
methods were crude, and the data were qualitative
work with our new method I learned
descriptions with scant operational definitions.
that somewhat similar experiments on
However, multiple training conditions, repetition of
animals had been performed in America,
test conditions, and tests for transfer made these
and indeed not by physiologists but by
studies influential in psychology (Harris, 1979).
psychologists. Thereupon I studied in
Thorndike (1927) reported an impressive laboratory
more detail the American publications,
study with adults that serves as a very early model of
and now I must acknowledge that the
the A-B-A design (see below). Subjects received
honour of having made the first steps
instructions such as “Draw an x-inch line,” where x
along this path belongs to E. L. Thorn-
was 3, 4, 5, or 6. Such instructions were presented
dike. By two or three years his experi-
in random order, and, being blindfolded, the sub-
ments preceded ours. (pp. 39–40)
jects never saw what they drew. The consequence,
Pavlov’s work apparently did not appear in English or effect, of drawing was the experimenter saying
until a paper in Science (Pavlov, 1906) and the either “right” or “wrong.” Each subject had one
translation of Conditioned Reflexes (Pavlov, 1927), early session with no effect delivered, then seven
long after Thorndike had finished his early animal training sessions with an effect, and then one more
research. In addition, Jacque Loeb, educated in the late session without an effect. Figure 1.2 shows
German tradition of physiology of individual ani- average data for all subjects and individual data for
mals (e.g., Loeb, 1900), had a major influence on two subjects. All subjects improved accuracy of line
Pavlov, Thorndike, Watson, and Skinner (Greens- drawing during training but dropped in accuracy
pan & Baars, 2005). when the effect was removed in the late test; 16 of
At the risk of digressing too far into the his- 24 subjects had a gain compared with their scores
tory of psychology, I cannot help mentioning the in the early test (e. g., Subject 29), whereas eight
work of the French physiologist Claude Bernard subjects had no gain and instead a drop (e.g.,
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
S ubject 42) compared with the early test. Thorndike

concluded that the effect was responsible for the
gain in line-drawing accuracy. In the area of motor
learning and control, this experiment is considered
a classic method for demonstration of the effects
of feedback on performance and learning (e.g.,
Schmidt & Lee, 2005).
With the advent of Skinner’s (1938) rigorous
experimental methods featuring operationally
defined measures of behavior and highly controlled
conditions for individual animal subjects, single-
case research methods developed rapidly and laid
the foundation for successful application to humans,
which began around the 1950s. For examples of
early behavior research and therapy using humans
(i.e., 1950–1965), see the collections of articles in
Eysenck (1960) and Ullman and Krasner (1965). In
addition, Wolpe (1958) developed single-case meth-
ods for treatment of phobias in humans.
The methods of behavior analysis for individual
subjects were laid out clearly in influential texts by
Keller and Schoenfeld (1950), Skinner (1953), and
Bijou and Baer (1961). The Journal of the Experimen-
tal Analysis of Behavior was founded in 1958 and
published experiments based on single-case meth-
odology using both human and nonhuman subjects.
When the Journal of Applied Behavior Analysis
appeared in 1968, single-case research methods
were further established as important research tools
that were not for animals only. For a more thorough
history of behavior analysis and single-case research
methods, see Barlow et al. (2009), Kazdin (2011),
and Blampied (1999).
Scientific Method
One important method, namely that of comparing
collected data from at least two different conditions,
Figure 1.2. Example of a historically early A-B-A is common across different research disciplines.
design for all subjects (top) and for two subjects Reaching this point has not come easy. The history
(middle and bottom). Twenty-four blindfolded human
subjects drew lines. Percentage correct expresses how of science is full of anecdotes that illustrate the
many lines were drawn within criterion length (3, 4, 5 struggle a particular scientist had in convincing con-
or 6 inches). In early and late tests (Thorndike’s terms), temporary scholars of new findings. Boorstin (1983)
no consequence was presented to the subjects, whereas
related how Galileo had to go through painstaking
during training subjects were presented with the experi-
menter’s saying “right” or “wrong” depending on their steps of comparing different conditions to demon-
performance. Data from Thorndike (1927). strate that through his telescope one could in fact
see how things at a great distance looked. In about
1650, Galileo first aimed the telescope at buildings
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
and other earthly objects and then made a drawing demonstration by Pascal of a scientific principle as
of what he saw. Then he walked to the location seen well as of a method of testing.
in the telescope and made a drawing of what he saw Although these anecdotes are amusing several
there. He compared such drawings over and over to hundred years later for their extreme and cumber-
demonstrate that the telescope worked. Using this some methods of testing, the practicing scientist of
method, Galileo could prepare his audience for an today still on occasion faces opposition to conclu-
important inductive step. When the telescope was sions from experiments. Thus, behavior analysts
aimed at objects in the sky, the drawing made of sometimes find themselves in a position in which it
what one saw through the telescope would reflect is difficult to convince psychologists with different
the structure of what existed at the faraway location. training that changes in contingencies of reinforce-
As is well-known from the history of science, many ment can bring about large and robust behavior
of Galileo’s contemporaries said that what they saw changes. For example, I once presented to col-
in the telescope was inside the telescope and not far leagues in different areas of psychology some data
away. Yet, his method of repeated comparisons of that demonstrated very reliable and precise stimulus
different viewing conditions was an important step control of behavior in rats. A colleague objected that
in convincing at least some scientists of his time that such mechanistic, on–off control of behavior is not
a telescope was an instrument that could be used to psychology. Others in the audience were suspicious
view distant, unreachable objects. In a review of the of the extremely low variability in the data, which
history of experimental control, Boring (1954) gave was believed to have come from an equipment error
a related example of how Pascal in 1648 tested and not from the method of controlling behavior.
whether a new instrument could measure baromet- The method of comparing two or more series
ric pressure. When a glass tube closed at one end of readings with the same measurement instrument
was filled with mercury and then inverted with the under two or more different conditions is the
open end immersed in a cup of mercury, a vacuum hallmark of the scientific method (Boring, 1954).
would form at the closed end of the tube. Pascal had Whether the research design is a between-groups
the idea that the weight of air pressing on the mer- or a within-subject comparison of the effects of a
cury in the open cup influenced the height of the manipulation of a variable on behavior, the shared
column of mercury. Hence, the length of the vac- element is always the comparisons of different con-
uum at the top of the tube should change at a higher ditions using the same measurement (see also Baer,
altitude, where the weight of air was supposedly less 1993).
than on the ground. Pascal sent family members
with two such instruments made of glass tubes,
Designs for Single-Case
cups, and mercury to take the measurements. Read-
Research Methods
ings were first made for both instruments at the foot
of a mountain. Then one person stayed with one A variety of designs have been developed for
instrument and took readings throughout the day. research with animals and humans. Because this
Another person carried the second instrument up chapter is an overview, I can only describe the most
the mountain and took measurements at different common designs. Chapter 5 of this volume covers
locations; when back at the foot of the mountain, designs in considerably more detail. Experimental
measurements were taken again with both instruments. designs are specific to the problem the researcher
Clearly, the essence of the method is the comparison seeks to investigate. However, an experimental
between one condition at the foot of the mountain, design is also like an interface between the
the control condition, and the elevated conditions. researcher and the subject because information
Because the control readings stayed the same passes in both directions. An essential aspect of
throughout the day, whereas the elevated condition single-case research methods is the interaction
readings changed, proof existed that the instrument between subject and researcher, educator, or thera-
worked as intended. This experiment was a real-life pist. Pavlov, Thorndike, and Skinner, as originators
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
of the single-case method, modified their apparatus A-B Design

and experimental procedures for individual animals The essence of single-case research methods is the
depending on how the animal reacted to the experi- comparison of repeated measures of the same behav-
mental procedures (Skinner, 1956). The scientist ior for the same individual for at least two different
modifies the subject’s behavior by experimental experimental, educational, or treatment conditions.
manipulations that, when reliable and replicable, Figure 1.3 shows a schematic of an A-B design,
constitute the foodstuff of science; equally as using hypothetical data. Phase A is repeated baseline
important for the process of discovery is the sub- recording of behavior across several observation
ject’s influence on the researcher’s behavior. Sidman periods (sessions) before any intervention takes
(1960) described how the researcher needs to “lis- place. The intervention is a change in the individu-
ten” to how the data change in accordance with al’s environment, also across several sessions, usu-
experimental manipulations. To discover laws of ally in the form of a change in how reinforcement is
behavior, the experimenter needs to adjust methods provided contingent on behavior. The data in Phase
depending on how they affect the subject’s behav- A serve as a comparison with behavior changes in
ior. In basic research, the experimental situation is Phase B. However, the comparison is logically from
often a dynamic exchange between researcher and an extension of data from Phase A to Phase B. Data
subject that may lead to novel discoveries and in Phase A are used to forecast or predict what
designs. In applied behavior analysis, as successful would have happened during the next sessions had
experiments and treatments are replicated over the intervention in Phase B not been introduced. If
time, designs tend to become relatively fixed and the data in Phase A are without trends and of low
standardized. The strength of standardization is a variability and are taken across several sessions,
high degree of internal validity and ease in commu- then the experienced researcher or therapist predicts
nicating the design. However, the weakness of stan- and assumes that this behavior will continue at that
dardization is that researchers or therapists may level if no interruptions in the environment occur.
primarily investigate problems that fit the standard Thus, the dotted line in Phase B in Figure 1.3 indi-
designs (Sidman, 1981). Nonetheless, the standard cates the projected trend of the behavior from Phase
designs covered here serve as basic building blocks A to Phase B. The effect on behavior of introducing
in both basic research and application. My main the change in Phase B is thus evaluated against
focus is the underlying logic of each of the covered the backdrop of the projected data from Phase A.
designs. Because forecast behavior cannot be measured, the
Figure 1.3. A-B design using hypothetical data.
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
difference between Phase A and Phase B data consti- the A-B design by itself is uncommon in clinical and
tutes the experimental effect, as indicated by the educational situations. However, the A-B design is
bracket. The validity of the statement that the inter- very common in laboratory demonstrations, espe-
vention in Phase B caused the change in behavior cially with animal subjects, where confounding vari-
from A to B thus depends not only on the change of ables can be controlled experimentally.
behavior in Phase B but also on how good the pre-
diction is that the behavior would have remained A-B-A Design
the same had the intervention not been introduced To demonstrate clear experimental control while
in Phase B. reducing the potential influence of confounding
Ideally, the behavior should not change in Phase variables, the experimenter can supplement the A-B
B unless the intervention took place. The change design with a withdrawal of the change in Phase B
should not occur by chance. Because other changes and a return to the conditions of Phase A, as a back-
in the environment might take place at the same to-baseline control. This design is therefore often
time as the intervention is introduced, the researcher called a withdrawal design. When the baseline mea-
or therapist can never be sure whether the interven- sured in the first Phase A can be recovered in the
tion or some other factor produces the behavior second Phase A, after the intervention has changed
change. Such unintended factors are customarily the behavior in Phase B, then the researcher has
called confounding variables. For example, a child shown control of behavior and demonstrated high
may have an undesirable low rate of smiling in the internal validity. The behavior is changed by the
baseline phase (A). Then when social reinforcement intervention and changed back when the interven-
is explicitly made contingent on smiling in the inter- tion is removed. The researcher therefore has full
vention phase (B), the rate of smiling may increase control over the behavior of the individual subject
above that in the baseline (A). If the sessions stretch or client, which means that the behavior can be
out over several weeks, with one session each day, changed at the researcher’s or therapist’s discretion.
then a host of other changes could happen at the Figure 1.4 shows a cumulative record of lever press-
same time as the intervention. For example, a parent ing for a single rat during baseline, acquisition, and
could return from a trip away from home, the child extinction of lever pressing within one session. The
could recover from the flu, a bully in class may be rapid increase in lever pressing when it produces
away for a few weeks, and so on. Such other factors food contrasts with the very low rate of lever press-
could have made the rate of smiling increase by ing during the baseline phase when lever pressing
themselves or in addition to the effect of providing did not produce food. The rapid decline in lever
social reinforcement for smiling. For additional con- pressing when reinforcement is withdrawn demon-
siderations regarding the role of confounding vari- strates the control reinforcement had over the
ables, see Kazdin (1973). Because of the potential behavior. The experienced investigator comes to
difficulty in controlling such confounding variables, learn that when such clear control is obtained with
Figure 1.4. A-B-A design: a cumulative record of lever pressing by a single rat. A = baseline with
response-independent reinforcement. B = continuous reinforcement; each lever press produced reinforce-
ment. Second A = extinction. Data from Iversen (2010).
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
one individual, then it will also be found with other, control over the behavior, they desire the behavior
similar individuals. change in Phase B to not remain intact when the
In clinical and educational applications, the intervention is removed.
A-B-A design is useful to show that an intervention
is effective and that a given behavior change is under A-B-A-B Design
the therapist’s control and not caused by other fac- To resolve some of these difficulties, an additional
tors. However, a critical problem arises with this phase can be added to the A-B-A design in which the
design regarding clinical or educational significance intervention (B) is repeated, thereby forming an A-B-
for the participant and for the caregivers. If the ther- A-B design, often called a reversal-replication design
apist follows an A-B-A design and changes behavior or a double-replication design. Figure 1.5 (top) shows
from a low level in Phase A to a high level in Phase B an example (R. G. Fox, Copeland, Harris, Rieth, &
and then back again to a low level in the second Hall, 1975) in which the number of math problems
Phase A to demonstrate control, in the end there completed by one eighth-grade underachieving stu-
is no gain for the client or for the caregivers. The dent changed when the teacher paid attention to her
educator or therapist can hardly convince the care- when she worked on the assignments. The number
givers that it is a great step forward to know that of problems completed shows a gradual and very
the behavior can now be brought back to the same large change in the treatment condition. Then, when
level as when the participant came in for treatment. the baseline was reinstated, the number dropped to
The A-B-A design has shown gain in control of the baseline levels only to increase again when the treat-
behavior but has produced no clinical or educational ment was reintroduced. Because the number of prob-
gain. Therefore, the A-B-A design is not useful as a lems completed is high in both treatment (B) phases
stand-alone treatment in clinical or educational and low in both baseline (A) phases, the data dem-
situations. onstrate control by the intervention of reinforcing
A concern with the A-B-A design is that behavior completion of math problems in a single individual
that changed in Phase B may not always show a child. This design satisfies both the need to demon-
reversal back to the level seen in the first Phase A strate control of behavior and the need to help the
when the intervention is withdrawn. For example, client because the behavior is at the improved level
for motor skill acquisition such as cycling or walk- when the treatment ends after the last Phase B. This
ing, the acquired behavior may not drop back to the off–on–off–on logic of the A-B-A-B design mimics
baseline level when reinforcement is removed in the daily situations in which people determine whether
second Phase A. The individual has acquired a new something works or not by turning it on and off a
skill in Phase B that may lead to new contingent few times. The A-B-A-B design shows that control of
reinforcers that were not within reach before the behavior can be repeated for the same individual—
skill was acquired. that is, intrasubject replication. This ability of the
When behavior does not return to baseline level researcher to replicate the behavioral change in a
after the intervention is removed, the possibility also single individual provides a tremendous source of
exists that the behavior change was produced by or knowledge about the conditions that control behav-
aided by a confounding variable and not by the ior. Such results are of practical significance for
intervention. The educator and therapist therefore teachers and family or caregivers because as Baer
face a dilemma. They want to help the client acquire et al. (1968) succinctly stated, “Replication is the
a new behavior that is lacking or to suppress an essence of believability” (p. 95).
existing unwanted behavior. They also want, how- The A-B-A-B design is customarily shown as a
ever, to be able to communicate to other therapists time series with several sessions in each phase. The
and educators that they have developed a method to underlying off–on–off–on logic also exists in other
control the behavior of interest. To help the client, procedures in which two conditions alternate sev-
they desire the behavior change in Phase B to remain eral times within a session, as in multiple sched-
intact when the intervention is removed. To show ules (e.g., two or more schedules of reinforcement
10
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 1.5. Top: A-B-A-B design with follow-up (post checks). From “A Computerized System for Selecting
Responsive Teaching Studies, Catalogued Along Twenty-Eight Important Dimensions, by R. G. Fox, R. E.
Copeland, J. W. Harris, H. J. Rieth, and R. V. Hall, in E. Ramp and G. Semb (Eds.), Behavior Analysis: Areas of
Research and Application (p. 131), 1975, Englewood Cliffs, NJ: Prentice-Hall, Inc. Copyright 1975 by Prentice-Hall,
Inc. Reprinted with permission. Bottom: Illustration of a within-session repeated AB * N design. Event record
showing onset of discriminative stimulus, first pen; response, second pen; reinforcement, third pen. Data are for
one rat after discrimination training showing perfect discrimination performance. Data from Iversen (2010).
alternate after a few minutes, each under a separate on occasional test trials in which a stimulus is
stimulus). Such A-B-A-B-A-B-A-B . . . or A-B * N altered, such as, for example, in the determination
designs demonstrate powerful, repeated behavior of stimulus generalization. With various modifica-
control. The A-B * N design can also be imple- tions, the repeated A-B changes within a session also
mented at the moment-to-moment level. Figure 1.5, form the basis of methods used in research and edu-
bottom, shows a sample of an event record in which cation, such as the matching-to-sample procedure
a rat promptly presses a lever each time a light turns (see Discrete-Trial Designs section, below).
on and almost never presses it when the light is off.
The A-B * N design can itself serve as a baseline for Multiple-Baseline Designs
other designs. For example, this discrimination pro- A popular expansion of the A-B design is the
cedure can serve as a baseline for assessment of the multiple-baseline design. This design is used when
effects of other factors such as food deprivation, reversing the behavior back to the baseline level is
stimulus factors, drugs, and so forth. The method of not desirable from an educational or therapeutic
repeated A-B * N changes within a session can also perspective. After an undesirable behavior has been
serve as a baseline for comparison with the outcome changed to a desirable level, educators, therapists,
11
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
and caregivers are reluctant to bring the behavior

back to its former undesirable level to prove that it
was the intervention that changed the behavior to
the acceptable level. Indeed, it may be considered
unethical to force the removal of a desirable behav-
ior acquired by a person with disability. With the
multiple-baseline design, the possible influence of
confounding variables is not assessed by withdraw-
ing the intervention condition as with the A-B-A and
A-B-A-B designs. Instead, data are compared with
simultaneously running baselines for other behav-
iors, situations, or individuals. The multiple-
baseline design probably had its formal origin in
Baer et al. (1968). Multiple-baseline designs are not
ordinarily used with animal subjects because rever-
sals to baseline and replications are not undesirable
or unethical. Figure 1.6 illustrates with hypothetical
data the underlying logic of multiple-baseline
designs. Two children in the same environment
have the same behavioral deficit. In the top display,
the target behavior is recorded concurrently for
both children for several sessions as a baseline
before intervention. For Peter, the intervention
begins at Session 16 and continues for 10 sessions,
and the target behavior shows a clear increase com-
pared with baseline. For Allen, the behavior is still
recorded throughout the intervention phase for
Peter, but no intervention is scheduled for Allen.
However, Allen also shows a similar large increase
in the target behavior. Faced with such data, one
Figure 1.6. Confounding variable in a multiple-
would be forced to conclude either that the behav-
baseline design. Top two graphs: Allen’s baseline behav-
ior change for Peter may not have been the result of ior changes during the intervention for Peter, suggesting
the intervention but of other factors in the environ- the influence of a confounding variable during inter-
ment or that Allen imitated Peter’s new behavior vention. Bottom two graphs: Allen’s behavior does not
change during the intervention for Peter but changes
change. Such data never find their way to publica- when his intervention starts, suggesting control by the
tion because they do not demonstrate a clear effect intervention. Data are hypothetical.
of the intervention for him. The bottom display
shows, also with hypothetical data, the more r eplication of the treatment effect. The benefit of this
customary published data pattern from such a design is that when faced with such data, the thera-
multiple-baseline design across subjects. The pist or educator can fairly safely conclude that the
baseline for Allen continues undisturbed, whereas intervention caused the behavior change. An addi-
Peter’s behavior changes during the intervention. tional benefit for the client is that no forced with-
When the same intervention is then also introduced drawal of treatment (return to baseline) occurs,
for Allen, his behavior shows an increase similar which would have ruined any educational or thera-
to that of Peter. The data show that the intended peutic gain for that client.
behavior change occurs only when the intervention Figure 1.7 shows an empirical example of teach-
is introduced. The data also show successful ing words to an 11-year-old boy with autism using a
12
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 1.7. Multiple-baseline design across behaviors for one child. From “Increasing
Spontaneous Language in Three Autistic Children,” by J. L. Matson, J. A. Sevin, D.
Fridley, and S. R. Love, 1990, Journal of Applied Behavior Analysis, 23, p. 231. Copyright
1990 by the Society for the Experimental Analysis of Behavior, Inc., Lawrence, KS.
Reprinted with permission.
multiple-baseline design across behaviors (Matson, against possible confounding variables that might
Sevin, Fridley, & Love, 1990). Baselines of three occur simultaneously with the introduction of the
words were recorded concurrently. Saying “please” intervention for the first subject; similarly, the base-
was taught first while the baseline was continued line for the third subject guards against possible
for the other two target words. Saying “thank you” confounding variables associated with the introduc-
was then taught while the baseline was continued tion of the intervention for the first and second
for the last target word, which was the last to be subjects.
taught. The data show that teaching one word made The multiple-baseline design offers several levels
saying that word increase in frequency, whereas say- of comparison of effects and no effects of the inter-
ing the other words did not increase in frequency. In vention (Iversen, 2012). Figure 1.8 illustrates, with
addition, the data show that saying a given word hypothetical data, the many comparisons that can be
did not increase in frequency until teaching for that made in multiple-baseline designs, in this case a
particular word started. Thus, the data generated by multiple-baseline design across subjects. For exam-
this design demonstrate clear control over the target ple, with four clients, the baselines of the same
behaviors. In addition, the data show that the con- behavior are recorded concurrently for all clients,
trol was established by the teaching methods and and the intervention is introduced successively
not by other confounding variables. across clients. Each client’s baseline serves as a
The logic of the multiple-baseline designs dic- comparison for the effect of the intervention on that
tates that the baselines run in parallel (concur- client (as indicated with dotted arrows at A, B, C,
rently). For example, with a multiple-baseline and D). The intervention phase for one client is
design across subjects, one would not run the first compared with the baseline for the next client
subject in January, the next subject in February, and (reading from the top of the chart) to determine
the third subject in March. The point of the design whether possible confounding variables might pro-
is that the baseline for the second subject guards duce concomitant changes in the behavior of other
13
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
Figure 1.8. Logic of the multiple-baseline across-subjects design.

A, B, C, and D arrows refer to change in behavior during the inter-
vention compared with the baseline for each individual; a, b, c, d, e,
and f arrows refer to the absence of a change for one individual when
the intervention has an effect at the same time for another individual.
Data are hypothetical.
clients when the behavior is made to change for the tion, there are six demonstrations of the absence of
previous client. Thus, the dotted lines (marked a, b, the possible effect of confounding variables because
c, d, e, and f ) indicate the possible assessments of the behavior score does not change for one client
effects of confounding variables. In this hypothetical when the intervention takes effect for another client.
example, there are four demonstrations of an inter- Ideally, to provide maximal internal validity, the
vention effect because the behavior score increases intervention should produce a change in behavior
each time the intervention is introduced. In addi- for the client for whom the intervention takes place
14
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
and should have no effect on the behavior of the a two-way interaction between experimenter and
other clients. Notice that the data for the client in subject and therapist and client, respectively.
the bottom display provides the most information,
with a change when the intervention takes place for Gradual or Sudden Changes in A-B, A-B-A,
that client and three comparisons during the base- A-B-A-B, and Multiple-Baseline Designs
line to introduction of interventions for the other The literature review for this chapter revealed that
clients but without any change on the baseline for two distinctly different patterns of behavior change
the last client. appear to be associated with the A-B, A-B-A, A-B-A-B,
Baer (1975) noted that multiple-baseline designs and multiple-baseline designs. Figure 1.9 exemplifies
offer an opportunity for systematic replication this issue for the A-B-A-B design using hypothetical
(across subjects, responses, or situations) but does data so as not to highlight or critique specific stud-
not offer an opportunity for direct replication for the ies. In the top graph, the behavior change is a grad-
same subject, response, or situation (i.e., there is no ual increase in the B phase and a gradual decrease in
return to baseline or subsequent return to interven- the second A phase with a return to baseline condi-
tion). In essence, the technique is repeated, but tions, and the last B phase also shows a gradual
control over behavior is not. Thus, an essential com- increase in the behavior, as in the first B phase. In
ponent of functional behavior analysis is lost with the bottom graph, the behavior in the first B phase
the multiple-baseline design, yet the technique is shifts up abruptly as soon as the intervention takes
well suited for applied behavior analysis. Successful,
systematic replications of procedures across behav-
iors, subjects, and situations and across laboratories,
classrooms, and clinics over time offer important
evidence that the designs are indeed responsible for
the behavior changes.
In general, multiple-baseline designs are con-
ducted across behaviors or situations for the same
individual or across individuals (with the same
behavior measured for all individuals). The designs
have also been used for groups of individuals. Multiple-
baseline designs are appealing to educators and
therapists and have become so popular that they are
often presented in textbooks as the golden example
of modern behavior analysis tools. However, these
designs do not quite live up to the original formula-
tion of single-case research methods’ being an inter-
action between the investigator and the subject. The
investigator should be able to change the procedure
as needed depending on how the subject’s behavior
changes as a function of the investigator’s experi-
mental manipulations. To be successful, applied
behavior analysis should guard against becoming a
discipline in which participants are pushed through
a rigid protocol with a predetermined number of Figure 1.9. Two different patterns of data in A-B-A-B
sessions and fixed set of conditions regardless of designs. The top display shows an example of gradual
how their behavior changes. In a recent interview, acquisition in both B phases and gradual extinction in
the second A phase. The bottom display shows abrupt
Sidman (as cited in Holth, 2010) pointed out that to changes in the B phases and in return to the A phase.
be effective, both basic and applied research requires Data are hypothetical.
15
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
place, stays at the same level, and then just as evaluate the frequency and root of the different
abruptly shifts down to the same level as in the first behavior patterns in these designs.
A phase with a similar abrupt shift up in the last B
phase. With animals, data customarily look as illus-
Alternating-Treatments or
trated in the top graph, because phase B is usually
Multielement Designs
contingent reinforcement and the change in Phase B
A variant of the A-B-A-B design is a more random
can be considered behavior acquisition. Similarly,
alternation of two or more components of interven-
when reinforcement is removed in the second A
tion or conditions for research. For example, the
phase, behavior will ordinarily extinguish gradually
effects of three doses of a drug on a behavioral
across sessions. In studies with humans, however,
baseline of operant conditioning for an individual
both of these data patterns appear in the literature,
may be compared across sessions with the dose
often without comment or clarification. When data
selected randomly for each session. Thus, a possible
show a large, abrupt change in the very first session
sequence of conditions might be B-A-A-C-B-C-C-A-
of intervention and an equally large and abrupt
B-A-C-B, and so on. This design allows for random
change back to baseline in the first session of with-
presentation of each condition and can assess
drawal, the participant either made contact with the
sequential effects in addition to determining the
changed reinforcement contingency immediately or
effect of several levels of an independent variable.
responded immediately to an instruction regarding
The basic design has its origin in Sidman (1960)
the change, such as “From now on you have to do x
under the label multielement manipulation and has
to earn the reinforcer” or “From now on you will no
since been labeled multielement design or alternating
longer earn the reinforcer when you do x.” Thus, the
treatments. This design is somewhat similar to the
two different behavior patterns observed in the A-B-
design used in functional assessment (see the sec-
A-B design with human clients, gradual versus
tion Functional Assessment later in this chapter).
abrupt changes, could conceivably reflect whether
However, an important difference is that in func-
the behavior was under control by contingencies of
tional assessment, each condition involves assess-
reinforcement or by discriminative stimuli (instruc-
ment of existing behavior under familiar conditions,
tion). Skinner (1969) drew a distinction between
whereas with the alternating-treatments design, each
contingency-shaped behavior and rule-governed
condition is an intervention seeking to change
behavior. Behavior that is rule governed already
behavior. Alternating-treatments designs are consid-
exists in the individual’s repertoire and is therefore
erably more complex than what can be covered here
switched on and off by the instructions, and the
(see, e.g., Barlow et al., 2009).
experiment or treatment becomes an exercise
in stimulus control, or rule following. However,
contingency-shaped behavior may not exist before Changing-Criterion Designs
an intervention intended to control the behavior, When a behavior needs to be changed drastically in
and the experiment or treatment becomes a demon- topography, duration, or frequency, an effective
stration or study of acquisition. Thus, with human approach is to change the criterion for reinforce-
participants, two fundamentally different behavioral ment in small steps. This approach is essentially the
processes may underlie the different patterns of method of shaping by successive approximation
behavior seen with the use of the A-B-A-B design (Cooper, Heron, & Heward, 2007). Concrete, mea-
(and also with the A-B, A-B-A, and multiple-baseline surable criteria are applied in a stepwise fashion in
designs). Unfortunately, authors do not always accordance with behavioral changes. Each step
explain the procedure carefully enough that one can serves as a baseline for comparison to the next step.
determine whether the participant was instructed Withdrawals or reversals are rare because the goal is
about changes in procedure when a new phase was to establish a drastic behavior change. The method
initiated (see also Kazdin, 1973). Perhaps future can be characterized as an A-B-C-D-E-F-G . . .
systematic examinations of existing literature can design, although it is rarely written that way. For
16
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
example, as a laboratory demonstration, the the number of circles increased as well. The objec-
duration of lever holding by a rat may be increased tive was to enable a topography change from touch-
in small steps of first 200 milliseconds across ses- lift to a continuous finger motion across circles, as
sions, then 500 milliseconds, then 1 second, and so in touch-drag. The lower diagram in Figure 1.10
forth. Eventually, the rat may steadily hold the lever shows the development of drawing for one subject
down for an extended period of time, say up to 10 for one of the four trial types on the monitor.
seconds or longer if the final criterion is set at 10 Touch-lift is indicated by a dot and touch-drag
seconds (e.g., Brenagan & Iversen, 2012). The changing- (touch and then move the finger while it still has
criterion design may also be used in the form of contact with the monitor) is indicated by a thick line
stimulus-shaping methods in educational research that connects successive circles. The figure shows
in which stimuli are faded in or faded out or modi- the interplay between procedure and results. The
fied in topography. For example, the spoken word circles come closer together, the number of circles
house is taught when a stimulus, say a schematic of increases, and the chimpanzee’s behavior changes
a house, is presented to a child. Over time the sche- accordingly from touch-lift to touch-drag. Small
matic is modified in many small steps into the specks of not lifting the finger between consecutive
printed word HOUSE; at each step, the correct circles initially appear in Session 13, and the first
response to the stimulus remains the spoken word full sweep across all circles without lifting appears
house. A variety of other schematics of objects, each already in Session 14 and is fully developed by Ses-
with its own separate spoken word, are similarly sion 16. The time to complete each trial (vertical
modified into printed words over time. Eventually, a lines) shortens as the topography changed from
situation is created in which the child may produce touch-lift to touch-drag. Eventually, the chimpanzee
spoken words that correspond to the printed words swept over the circles in one movement for all four
(e.g., Cooper et al., 2007). McDougall, Hawkins, trial types. In the remaining steps, the stimuli were
Brady, and Jenkins (2006) described various imple- changed by fading techniques across and within ses-
mentations of the changing-criterion design in edu- sions from an array of circles to just two dots, one
cation and suggested that the changing-criterion where drawing should start and one where it should
design can profitably be combined with A-B-A-B or end. An additional aspect of the method was that the
multiple-baseline designs to establish optimal chimpanzees were also taught to end the trials them-
designs for individuals in need of large-scale behav- selves by pressing a trial termination key, which was
ior changes. introduced in Step 6. Thereby, the end of the drawn
Actual laboratory experiments and educational trace (lifting of the finger) came under control by
or clinical interventions using variations of changing- the stimuli on the monitor and not by delivery of
criterion designs are customarily highly complex reinforcement. The final performance was a highly
mixtures of different methods. For example, to gen- precise drawing behavior under visual guidance by
erate visual guidance of motor behavior in chimpan- the stimuli on the screen. Such stimulus control of
zees, Iversen and Matsuzawa (1996) introduced an complex motor performance was acquired with a
automated training protocol to bring line drawing completely automated method over a span of about
with a finger on a touch screen under stimulus con- 100 sessions (3–4 weeks) entirely by continuously
trol. Finger movement left “electronic ink” on the rearranging reinforcement contingencies and
screen surface. The top-left diagram in Figure 1.10 stimulus-fading procedures in small steps without
shows a sketch of the 10-step training procedure. the use of verbal instruction. In the final perfor-
Each session presented in mixed order four trial mance, the chimpanzees would look at the dots, aim
types as four orientations of an array of circles (as one finger at the start dot, rapidly move the finger
indicated). The chimpanzees had to touch the cir- across the monitor, lift the finger at the second dot,
cles to produce reinforcement. Stimuli changed and then press the trial termination key to produce
across and within sessions. Thus, within Steps 2 and reinforcement, all in less than 1 second for each
3, the circles were moved closer across trials, and trial. The top right graph in Figure 1.10 shows the
17
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
Figure 1.10. Top left: Schematic of the experimental procedure. Top right: Frequency plot of angles of
drawing, as illustrated in the top images. Bottom: Data are from one trial type, and all trials of that type are
shown in six successive sessions. Number of circles and distance between circles changed within and across ses-
sions. A dot indicates touch-lift, and a black line indicates touch-drag. From “Visually Guided Drawing in the
Chimpanzee (Pan Troglodytes),” by I. H. Iversen and T. Matsuzawa, 1996, Japanese Psychological Research, 38,
pp. 128, 131, 133. Copyright 1996 by John Wiley & Sons, Inc. Reprinted with permission.
resulting control of the angle of the drawn trace for time training the chimpanzees and added that they
each trial type. could “obviously” already draw because such
When behavior control techniques are success- smooth and precise motor performance could not at
ful and produce very reliable and smooth perfor- all have been acquired by simple shaping. Such
mance, spectators of the final performance may not comments may be somewhat amusing to behavior
quite believe that the subjects could at some point analysts. Yet, they are made by professionals in
in time not do this. In fact, a renowned develop- other areas of psychology and reveal a disturbing
mental psychologist happened to visit the labora- lack of understanding of and respect for effective
tory while the drawing experiment was ongoing. behavior control techniques developed with the use
On seeing one of the chimpanzees draw line after of single-case research methods. Unfortunately, the
line smoothly and without hesitation (in Step 10), commentaries also reveal a failure by behavior ana-
he exclaimed that the investigators had wasted their lysts to promote understanding about behavior
18
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
analysis, even to professionals in other areas of Time Scales of Single-Case Research

psychology. Methods
The time scale can vary considerably for single-case
Discrete-Trial Designs research methods. Laboratory demonstrations with
A discrete trial is the presentation of some stimu- animals using the A-B-A design to show acquisition
lus material and implementation of a response– and extinction of a simple operant can usually be
reinforcer contingency that applies only in the pres- accomplished in a matter of 20 to 30 minutes (e.g.,
ence of that stimulus. The discrete-trial design has Figure 1.4). Educational interventions using A-B-A-
long been a standard procedure for use with animals B designs may last for weeks. Therapeutic interven-
in all sorts of experiments within and outside of tions for children with autism spectrum disorder
behavior analysis. Complex experiments and educa- may last a few years, with multiple changes in
tional projects, for example, involving conditional designs within this time period (e.g., Lovaas, 1987).
discriminations (e.g., matching to sample), are often Green, Brennan, and Fein (2002), for example,
based on discrete-trial designs (see Volume 2, Chap- described a behavior analysis treatment project for
ter 6, this handbook). one toddler with autism, Catherine. Treatment con-
Historically, the discrete-trial method has tinued for 3 years with gradually increasing com-
become almost the hallmark of applied behavior plexity, beginning with instruction in home settings,
analysis, especially for its use in education and treat- through other settings, and eventually to regular
ment of children with intellectual disabilities (Baer, preschool settings with minimal instruction. Figure
2005). For example, the method may be as simple as 1.11 shows the chronological order of skill intro-
presenting a picture of an animal, and the response duction for the 1st year. The design is a gradual
that produces the reinforcer in this trial is the spo- introduction of skill complexity in which previ-
ken name of the animal on the picture; in another ously acquired skills serve as prerequisites for new
trial, the picture may be of another animal, and the skills; the logic of the design is similar to that of the
reinforced response is the name of that animal. changing-criterion design, mentioned earlier, except
Thus, the method is useful for teaching which that the criterion change is across topographically
stimuli (verbal or pictorial) should control which different behaviors and situations. There is no base-
responses and when. Loosely speaking, the discrete- line for each skill other than the therapist’s knowl-
trial method teaches when a response is permitted edge that a given skill was absent or not sufficiently
and when it is not. The method may be used in a developed before it was explicitly targeted for acqui-
very informal way, as in training when normal activ- sition treatment. There are no withdrawals because,
ities should and should not occur. The method may for such real-life therapeutic interventions, they
also be presented very formally in an automated would force removal of an acquired, desirable skill.
arrangement, as in the above example with chim- Effective withdrawals may even be impossible for
panzees. In applied behavior analysis, the discrete- many of the acquired skills such as eye contact, imi-
trial method is useful for teaching single units of tation, and speech. Instead, continued progress
behavior such as acquisition of nouns but is less revealed that the overall program of intervention
useful for teaching sequential behaviors, such as was successful. Besides, the study replicated previ-
brushing teeth (Steege & Mace, 2007). Apparently, ous similar studies. Green et al. concluded that
the very term discrete-trial teaching has of late, out- “over the course of 3 years of intense, comprehen-
side the field of applied behavior analysis, come to sive treatment, Catherine progressed from exhibit-
be considered as a simple procedure for simple ing substantial delays in multiple skill domains to
behaviors. Attempts have been made, therefore, to functioning at or above her age level in all domains”
present applied behavior analysis to audiences out- (p. 97).
side of behavior analysis as a considerably richer set Multiple-baseline designs can on occasion last
of methods than just discrete-trial methods (Ghezzi, for years. D. K. Fox, Hopkins, and Anger (1987),
2007; Steege & Mace, 2007). for example, introduced a token reinforcement
19
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
Figure 1.11. Timeline for teaching various skills to a single

child in an intense behavior analysis program. Arrows indicate that
instruction continued beyond the time period indicated here. From
“Intensive Behavioral Treatment for a Toddler at High Risk for
Autism,” by G. Green, L. C. Brennan, and D. Fein, 2002, Behavior
Modification, 26, p. 82. Copyright 2002 by Sage Publications.
Reprinted with permission.
rogram for safety behaviors in two open-pit mines.

p single-case research methods, the case may not nec-
Concurrent baseline recordings of safety were essarily be one individual but a group of individuals—
recorded for both mines. After 2 years in one mine, and in this case, the group may even change
the contingencies were changed for that mine, and members over time.
the baseline was continued for the other mine for The length of time a project lasts is not an
another 3 years before the contingencies were essential aspect of single-case research designs. The
changed for that mine, too. For this 15-year project, essential aspects are that data consist of repeated
at each mine the contingencies were applied at the observations or recordings and that such data are
level of both the individual worker and teams of compared across different experimental or treatment
workers. This example serves as a reminder that for conditions.
20
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Using Single-Case Designs for from the goal of providing impetus for intervention,
Assessment with publications presenting results from assess-
ment alone without actual follow-up intervention.
A multitude of methods have been developed to
Thus, the reader is left wondering whether the
assess behavior. The general methodology is similar
causes of the problem behavior revealed by the
to single-case designs, which can be used in testing
assessment package were also causes that could be
which specific stimuli control a given client’s
manipulated and whether such manipulations
behavior.
would in fact improve the problem behavior.
Functional Assessment Functional assessment serves as a useful clini-
Functional assessment seeks to ascertain the imme- cal and educational tool to determine possible
diate causes of problem behavior by identifying immediate causes of problem behavior. How-
antecedent events that initiate the behavior and con- ever, reliability of assessment outcome cannot be
sequent events that reinforce and maintain the fully evaluated without a direct link between the
behavior. The goal of assessment is to determine the inferred causes of problem behavior in assessment
most appropriate intervention for the given individ- and the outcome of subsequent intervention
ual. For example, a child exhibiting self-injurious using manipulations of these inferred causes for
behavior may be placed in several different situa- the same client.
tions to determine which situations and possible
consequences of the behavior in those situations Assessment Using Discrete-Trial
affect the behavior frequency. The situations may be Procedures
(a) alone, (b) alone with toys, (c) with one parent, Many children and adults with intellectual disorders
(d) with the other parent, (e) with a sibling, and have not acquired expressive language, and commu-
(f) with a teacher. The child is exposed to one situa- nication with them can be difficult or impossible. A
tion each session with situations alternating across method called facilitated communication claims to
sessions. For determination of reliability, each situa- enable communication with such clients by having a
tion is usually presented more than once. If the specially trained person, the facilitator, hold the cli-
problem behavior in this example occurs most fre- ent’s hand or arm while the client types with one
quently when either parent is present, and it is finger on a keyboard. Because some typed messages
observed that the parent interacts with the child have expressed advanced language use without the
when the problem behavior occurs, then the infer- clients ever having shown other evidence of such
ence is drawn that the problem behavior may be language, facilitated communication has been ques-
maintained by parental interaction (i.e., positive rein- tioned as a means of authentic communication. The
forcement from attention) and that the parent’s entry is question is whether the facilitator rather than the
an antecedent for the behavior (i.e., parental entrance client could be the author of the typed messages. To
is an immediate cause of initiation of the behavior). test for such a possibility, several investigators have
The therapist will then design an intervention based used single-case research methods to examine stim-
on the information collected during assessment. ulus control of typing (e.g., Montee, Miltenberger,
Causes of the problem behavior are inferred from func- & Wittrock, 1995; Wheeler, Jacobson, Paglieri, &
tional assessment methods, because no experimental Schwartz, 1993). The most common test is to first
analysis is performed in which the causes are manip- present the client and the facilitator with a series of
ulated systematically to determine how they influ- pictures in discrete trials and ask the client to type
ence behavior. Functional assessment of problem the name of the object shown on the picture. If the
behaviors has become very prevalent; special issues client types the correct object names, then the
and books have been published on how to conduct method is changed by adding test probes on some
the assessments (e.g., Dunlap & Kincaid, 2001; Neef trials. On those test trials, the pictures are arranged
& Iwata, 1994). Recently, functional assessment has such that the facilitator sees one picture, and the
appeared to have taken on a life of its own, separate client sees another picture; an important added
21
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
methodological feature is that the client cannot see

the picture the facilitator sees, and vice versa. These
studies have shown with overwhelming reliability
that the clients type the name of the picture that the
facilitators see; if the clients see a different picture,
then the clients do not type the name of the picture
they see but instead type the name of the picture the
facilitator sees. Thus, these studies have demon-
strated, client by client, that it is the facilitator who
types the messages and that the pictures do not con-
trol correct typing by the clients—the clients cannot
communicate the names of the pictures they see.
For commentaries and reviews of this literature,
see Green (2005); Green and Shane (1994); and
Jacobson, Mulick, and Schwartz (1995). As Green
(2005) pointed out, the continued false beliefs by
facilitators, family, caregivers, and news media that
clients actually communicate with facilitated com-
munication may in fact deprive them of an opportu-
nity to receive effective, scientifically validated
treatment.
Real-Life Assessment Using Multistage

Reversal-Replication Designs
Applications of single-case research methods with
clients in real-life situations away from clinic or
school may not always follow a pure, prearranged
formula. Contextual factors, such as varying client Figure 1.12. Top: Schematic of the events in a single
health and family situations, are additional consider- trial. The first 1.5 seconds is an observation period (in
this case, the presession instruction was to always select
ations. In such cases, the consistent replication of the noun of a noun–verb choice; new words appeared
behavior patterns in similar conditions across multi- on each trial, and the correct position varied randomly
ple sessions becomes the indicator of whether a from trial to trial). A 0.5-second baseline of electroen-
given treatment or assessment has the intended out- cephalogram (EEG) is then recorded, followed by an
active phase in which the patient can control the cursor
come. For example, completely paralyzed patients (ball) on the screen for 3 seconds. Bottom: Data from
with amyotrophic lateral sclerosis were trained to one patient with amyotrophic lateral sclerosis from one
communicate using only their brainwaves (via elec- training day with several successive tasks. T = task; T1 =
simple target; T5 = noun–verbs; T7 = color matching;
troencephalogram) to control the movement of a
T8 = addition or subtraction matching. From “A Brain–
cursor on a computer screen (Birbaumer et al., Computer Interface Tool to Assess Cognitive Functions
1999). Letters were placed on the screen, and the in Completely Paralyzed Patients With Amyotrophic
patient could move the cursor toward them and Lateral Sclerosis,” by I. H. Iversen, N. Ghanayim, A.
Kübler, N. Neumann, N. Birbaumer, and J. Kaiser, 2008,
thereby spell words to communicate. In additional Clinical Neurophysiology, 119, pp. 2217, 2220. Copyright
experiments, abilities to distinguish verbs from 2008 by Elsevier. Reprinted with permission.
nouns, odd from even numbers, and consonants
from vowels and to perform simple computations trial. The patient gets online visual feedback from
were assessed in a matching-to-sample–type task the electroencephalogram in the form of cursor
(Iversen et al., 2008). The top part of Figure 1.12 movement. If the cursor reaches the correct stimu-
shows a schematic of the events during one 5-second lus, then a smiley face appears on the screen. The
22
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
patients lived in their private homes with constant electrode, equipment failure, or the lack of patient
care and were assessed intermittently over several cooperation. That the patient scored high again on
sessions spanning a few hours once or twice each the simple task in the last A phase, immediately after
week. The ideal scenario was always to test patients the session in the D phase, demonstrates that the
several times on a very simple task, such as steering patient had some difficulties discriminating the
the cursor to a filled box versus an open box, to stimuli presented in Task 8 in the D phase (i.e.,
make sure that the electrodes were attached cor- addition and subtraction, such as 3 + 5 or 7 − 2).
rectly, equipment worked, and the patient could still Among the many findings is that several warm-up
use the electroencephalogram feedback to control sessions were necessary at the beginning of the day
the cursor. Once the patient reached at least 85% before the patient reached the usual 85% correct on
correct on a simple task, then tasks with assessment the simple task. Several such training days with this
stimuli (e.g., nouns and verbs) were presented for very ill, speechless and motionless patient demon-
one or several sessions. strated that the patient had some deficits in basic
If the neurodegenerative amyotrophic lateral skills. Thus, the data showed that the patient, a for-
sclerosis destroys the patient’s ability to discriminate mer banker, now had problems with very simple
words or numbers, then the patient should show addition and subtraction.
a deficit in such tasks compared with when the This example illustrates the use of single-case
patient can solve a simple task. To determine research methods in complex living situations with
whether a patient has a potential deficit in a given patients with extreme disability. To extract mean-
skill, such as odd–even discrimination, then it is ingful data from research with such patients in a
necessary to know that the patient can still do the varying environment, it is necessary to know repeat-
simple task of moving the cursor to the correct tar- edly that the complex recording and control equip-
get when the target is just a filled box. Thus, to ment works as intended and that patients’ basic
interpret the data, it was necessary to present the skills are continuously intact because no verbal
simple task several times at the beginning of, during, communication from the patient is possible (i.e., the
and end of a given day the patient was visited by the patient cannot tell the trainer that he is tired or that
testing team. It proved challenging at times to con- something else may be the matter). Indeed, both
vince family members that it was necessary to repeat trainers and family members had to be instructed in
the simple tasks several times because, as the family some detail, and often, as to why it was necessary to
members said, “You already know that he can do repeat the simple tasks several times on each visit
that, so why do you keep wasting time repeating it?” to the patient’s home. The multiphase replication
The bottom part of Figure 1.12 shows the results for design with repeated presentation of phases of sim-
each of 16 consecutive sessions for a single test day ple tasks in alternation with more complex tasks is a
for one patient. The task numbers on the x-axis refer necessary component of single-case research or test-
to the type of training task, with Task 1 (T1) being ing methods applied to complex living situations in
the simplest, which was repeated several times. The which communication may be compromised or
overall design forms a phase sequence of A-B-C-A- impossible.
C-D-A, in which the A phases present the simple
task and serve as a baseline control, and the B, C,
Data Analysis Methods
and D phases present more complex test material.
The A phases thus serve as a control for equipment A fundamental aspect of single-case designs is that
integrity and for the patient’s basic ability to move behavior is recorded repeatedly and under the same
the cursor to a correct target. For example, had the set of methods and definitions across different
training day ended after the session in Phase D, it experimental, educational, or treatment conditions.
would have been difficult to conclude that the Chapter 6 of this volume covers these issues; see
patient had a deficit in this task because the deteri- also J. M. Johnston and Pennypacker (2009) and
orating percentage correct could reflect a loose Cooper et al. (2007).
23
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
Data Recording Methods ioral data is Parsonson and Baer (1978), which
Automated recording techniques (customarily used covers analysis of data from basic single-case
with animal subjects) require frequent verification research designs.
that monitoring devices (switches, photocells, Fundamental issues in visual analysis are eval-
touch screens, etc.) record behavior as intended and uations of baseline stability and baseline trends.
that these monitoring devices are calibrated cor- Baselines should ideally vary little, and experi-
rectly. With observational methods, calibration of menters should analyze any conditions responsi-
criteria for response occurrence used by different ble for unexplained variation (Sidman, 1960).
observers is a crucial issue, and intraobserver as Data from an intervention cannot always be inter-
well as interobserver agreement calculations are preted if variability from the baseline carries over
essential for objective recording of behavior (see into the intervention phase. Trends in baselines
Chapter 6, this volume). Equally essential is intra- can be problematic if they are in the direction of
and interpersonnel consistency in the methods of the expected experimental or therapeutic effect.
delivering consequences to clients in educational For example, if a baseline rate of behavior gradu-
and clinical settings. However, reports of agreement ally increases over several observation periods,
scores do not ordinarily include information about and behavior increases further during the inter-
how consistently personnel follow described vention, then it can be difficult or impossible to
methods. determine whether the intervention was responsi-
Common measures of behavior are frequency ble for the increase (e.g., Cooper et al., 2007). The
of occurrence (often converted to response rate), expressions appropriate baseline and inappropriate
response duration, and response topography (e.g., baseline have appeared in the literature to empha-
Barlow et al., 2009). A given behavior can also be size this issue. An appropriate baseline is either a
analyzed in terms of its placement in time as a flat baseline without a trend or a baseline with a
behavior pattern or its placement among other trend in the direction opposite to the expected
behaviors, as in analyses of the sequential properties effect. For example, if the rate of self-injury in a
of behavior (e.g., Henton & Iversen, 1978; Iversen, child is steadily increasing in baseline, and the
1991; see also Chapter 12, this volume). intervention is expected to decrease the behavior,
then an increasing baseline is appropriate for
Visual Data Analysis Methods intervention because the behavior is expected
Behavior analysis has a long tradition (beginning to decrease in the intervention even against an
with Pavlov, Thorndike, and Skinner) of focusing increasing baseline. There is certainly no rationale
on nonstatistical, visual analyses of data from sin- in waiting for self-injury to stabilize before inter-
gle subjects (Iversen, 1991). In Chapter 9 of this vention starts. However, an inappropriate baseline
volume, Bourret and Pietras describe a variety of is one that has a trend in the same direction as the
methods of visual data analysis. Such analyses are expected outcome of the intervention (see also
now fairly standard and are covered in most text- Cooper et al., 2007).
books on single-case research designs (e.g., Kennedy, Patterns of data are important in interpretations
2005; Morgan & Morgan, 2009) and in textbooks of behavior changes, in addition to descriptive sta-
on behavior analysis in general (e.g., Cooper et al., tistical evaluations. For example, the two trends
2007). previously noted here in data for A-B, A-B-A, A-B-A-
Visual analyses of data are not particular to B, and multiple-baseline designs (i.e., Figure 1.9)
experimental or applied behavior analysis and per- would not be apparent had data been presented only
meate all sciences and other forms of communica- as averages for each phase of recorded behavior.
tion, as exemplified in the books by Tufte (e.g., Plotting data for each session captures trends, vari-
1983, 1990; see also Iversen, 1988) on analyzing ability, and intervention outcomes. Without such
and presenting visual information. Within behavior data, the investigator easily misses important infor-
analysis, a classic text on visual analysis of behav- mation that might lead to subsequent changes in
24
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
procedure. In fact, successful interaction between be ruled out). If a t test is applied, however, compar-
researcher and subject depends on visual data ing the baseline data with the treatment data, the
analysis concomitant with project progression (e.g., standard deviation for the B phase may be very large
Sidman, 1960; Skinner, 1938) because data range from the baseline level to the
highest rate when treatment is most effective. The
Statistical Data Analysis Methods result may be a nonsignificant effect. Besides, the
Statistical analyses of behavioral data in both basic data in the B phase showing a gradual acquisition
research and application have been controversial may not be normally distributed and may not be
since Skinner’s (1938) The Behavior of Organisms, independent measures (i.e., as a series of increasing
which was devoid of traditional statistical analyses. values, data on session n influence data on session
Behavior analysis gathers information about the n + 1); the result is that the t test is not valid. How-
behavior of individual subjects, whereas the tradi- ever, the failure of a common statistical test to show
tional statistical approach gathers information about an effect certainly does not mean that such data are
groups of subjects and offers no information about unimportant or that there is no effect of the experi-
data from individual subjects. However, visual anal- mental manipulation. Single-case research methods
yses of behavioral data are not always sufficient, and are not designed for hypothesis testing and inferen-
statistical methods can be used to supplement the tial statistics but for analysis of behavior of the indi-
analysis. Several authors have analyzed the ongoing vidual subject and for development of methods that
controversy regarding use of statistics in behavior can serve to help individuals acquire appropriate
analysis and psychology (e.g., Barlow et al., 2009; behavior. The assumptions of inferential statistics,
Kratochwill & Brody, 1978; see Chapter 7, this vol- meant for between-group comparisons, with inde-
ume). Chapters 11 and 12 of this volume provide pendent observations, random selection of subjects,
information on new statistical techniques, which random allocation to treatment, and random treat-
may prove useful for behavior analysts conducting ment onset and offset are obviously not fulfilled in
time-series single-case research designs. single-case research methods. Eventually, however,
A major issue with the use of statistics in behav- statistical tests appropriate for single-case methods
ior analysis is the treatment of variability in data. may evolve from further developments in analyses
Visual analyses take variability as informative data of interrupted time-series data (e.g., Barlow et al.,
that can prompt an experimental analysis of the 2009; Crosbie, 1993; see Chapters 11 and 12, this
sources of this variability (e.g., Sidman, 1960). With volume).
statistical analysis, however, variability is artificially Aggregation from individual data points through
compressed to a more manageable single number averages and standard deviations to statistical tests
(variance or standard deviation) without analysis of to p values and to the final binary statement “yes or
the source of the variability. Thus, visual analysis no” is, of course, common in all sciences, including
and statistical analysis can be seen as antithetical psychology. Quantitative reduction of complex
tools to uncover causes of variability in the behavior behavior patterns to yes or no eases communication
of the individual subject. of theories and ideas through publications and pre-
Consider, for example, the use of a simple t test sentations, in which actual data from individual sub-
for evaluation of data from two successive phases of jects or individual trials may be omitted. In contrast,
an A-B design (e.g., Figure 1.3) as an illustration of a focus on data linked more directly to experimental
the problems one may face with the use of a statisti- manipulations can lead to demonstrations of stun-
cal test created for entirely different experimental ning control and prediction of behavior at the
designs. If the B phase shows acquisition of behavior moment-to-moment level for the individual subject
at a gradually increasing rate compared with the (e.g., Henton & Iversen, 1978; Sidman, 1960),
baseline in the A phase, the behavior analyst has no which is much closer to everyday interactions
problems visually identifying a large effect of the between behavior and environment. In daily life,
manipulation (given that confounding variables can people respond promptly to single instances of
25
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
interpersonal and environmental cues. Such stimu- autism. One group of children (n = 19) with the
lus control of behavior is the essence of interhuman diagnosis of autism received intensive behavior anal-
communication and conduct. By demonstrating ysis procedures (40 hours/week) for 2 years, and
control of behavior at this level for the individual another group (n = 19) with the same diagnosis did
subject, behavior analysis can be both a science of not receive the same treatment. The differences
behavior and a tool for educational and therapeutic between the groups were vast; for example, 49% of
interventions. the children in the treatment group showed signifi-
cant gains in IQ and other measures of behavioral
Quantitative Descriptions of Behavior– functioning compared with the other group. This
Environment Relations study helped generate respect among parents and
Behavioral data from single-case designs have professionals for behavioral methods in treatment
invited quantitative descriptions of the relationships of children with autism. Since the Lovaas (1987)
between behavior and environmental variables. Such study, several other studies have similarly compared
descriptions vary considerably from issue to issue two groups of children (sometimes randomly
and attract general interest in both basic research assigned) with similar levels of autism spectrum dis-
(Shull, 1991) and application (see Chapter 10, this order, in which one group received intensive behav-
volume). ior analysis treatment for 1 or several years, and the
other group received less treatment or no treatment
other than what is ordinarily provided by the child’s
Single-Case Designs and Group
community (so-called “eclectic” treatment). For
Studies Compared
example, Dawson et al. (2009) recently reported one
“Operant methods make their own use of Grand such study with 48 children (18–30 months old)
Numbers; instead of studying a thousand rats for diagnosed with autism spectrum disorder in a ran-
one hour each, or a hundred rats for ten hours each, domized controlled trial. Statistical procedures dem-
the investigator is likely to study one rat for a thou- onstrated significant gains in a variety of behavioral
sand hours” (Skinner, 1969, p. 112). Research measures for the group that received the treatment.
involving comparisons of groups of subjects, in For this study and the Lovaas study, each child went
which each group is exposed once to one level of a through complex and intense procedures based on
manipulation, is rare in behavior analysis, in partic- single-case research methods for about 2 years. The
ular with animal subjects. group comparisons mainly used standard measures
Yet, group research or studies with a large num- of the children’s performances. Enough group
ber of participants sometimes have relevance for studies have, in fact, been conducted that several
behavior analysis. For example, in a recent interview meta-analyses of the efficacy of behavior analysis
(Holth, 2010), Murray Sidman remarked that large- treatment of children with autism spectrum disorder
scale implementation of behavior analysis tech- have now been performed on the basis of such
niques may require prior demonstration of the studies.
effectiveness of those techniques in large popula- Group comparison methods have also increased
tions. Thus, studies using randomization and con- the visibility of behavior analysis techniques in areas
trol groups may be necessary for promulgation of of application. For example, Taub et al. (2006) used
effective behavior control techniques. A positive behavioral techniques in treatment of patients with
outcome of a group study may not make behavior paralysis of arms or legs resulting from stroke or
analysts know more about control of an individual brain injury. With the less affected or “normal”
client’s behavior, yet such an outcome may nonethe- limb restrained to prevent its use, 21 patients who
less make more people know about behavior analy- received massed practice of the affected limb (6 hours/
sis. For example, the widely cited group study by day for 10 consecutive weekdays with shaping of
Lovaas (1987) generated broad interest in behavior movement and establishment of stimulus control
analysis methods for treatment of children with of movement using social reinforcement) were
26
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
c ompared with a matched control group of 20 informing the general public, therapists, scientists in
patients who received customary, standard physical other areas, educators, politicians, and so forth
therapy and general support. Treatment patients about behavior analysis techniques should, how-
showed huge and clinically significant gains in ever, be balanced with a concern for ethical treat-
motor control of the affected arm, whereas patients ment of the participants involved. When a behavior
in the control group showed no such gains. A simi- analyst, based on experience with previous results,
lar example is provided in research by Horne et al. knows full well that a particular treatment using
(2004). These investigators had first obtained reli- single-case methodology has proven to be effective
able single-case data that reinforcement techniques over and over, then it is indeed an ethical dilemma
could be used effectively to increase the consump- to knowingly split a population of children in need
tion of fruits and vegetables among schoolchildren. of that treatment into two groups for comparison of
To promote the findings, Horne et al. conducted a treatment versus no treatment. Children who are in
large group study with 749 schoolchildren. The a control group for a few years for a valid statistical
children were split into an experimental group and a comparison with the treatment group may thus be
control group. A baseline of fruit and vegetable con- deprived of an opportunity for known, effective
sumption was taken first. Then at lunchtime, fruit treatment.
and vegetable consumption was encouraged by hav- Several so-called randomized controlled group
ing the children in the experimental group watch studies and meta-analyses of such studies have been
video adventures featuring heroic peers (the Food conducted over the past few decades to determine
Dudes) who enjoy eating fruits and vegetables, and whether applied behavior modification actually
the children received reinforcers for eating fruit and works (e.g., Spreckley & Boyd, 2009). For elaborate
vegetables. Children in the control group had free comments and critiques of some meta-analysis stud-
access to fruit and vegetables. Compared with the ies of applied behavior analysis methods, see Kim-
children in the control group, fruit and vegetable ball (2009). These time-consuming studies, with a
consumption was significantly higher among the presumed target audience of policymakers and
children in the experimental group. On the basis of insurance companies, would clearly not have been
such data, this program has now been implemented undertaken unless countless studies using single-
on a large scale in all schools in Ireland and in other case methods had already demonstrated that the
places in England (Lowe, 2010). behavior of an individual child can be modified and
Such group-comparison studies, published in sustained with appropriate management of rein-
journals that do not ordinarily publish studies using forcement contingencies and stimulus control tech-
single-case research methods, may be helpful in pro- niques. The sheer mass of already existing studies
moting general knowledge about behavior analysis based on single-case methodology with successful
to a much wider audience. In addition, when effec- outcomes, for literally thousands of individuals
tive, evidence-based behavior-analytic treatments across a variety of behavior problems and treatment
become broadly recognized from publications with a settings, poses the question, “How many more ran-
large number of participants and in renowned jour- domized controlled group studies and subsequent
nals, then granting agencies, insurance companies, meta-analyses are necessary before single-case meth-
journalists, and maybe even university administrators ods can be accepted in general as effective in
and politicians start to pay attention to the findings. treatment?”
For behavior analysts, group studies may not seem The use of inferential statistics as a method of
to add much basic knowledge beyond what is proof leads to the very odd situation that such meta-
already known from studies using single-case analyses may explicitly exclude the results from
research methods. Publication of group studies may, application of single-case methods with individual
however, be a tactic for promotion of basic, impor- clients (e.g., Spreckley & Boyd, 2009), even though
tant, and effective behavioral techniques beyond the the overall purpose of the meta-analyses is to decide
readership of behavior analysts. The need for whether such methods work for individual clients.
27
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
Profound misunderstandings of what can be evidence-based medicine, placed single-case

accomplished by single-case research methods in research designs with randomization of treatment
general can on occasion be heard among pedagogues highest in their hierarchy of strength of evidence for
and critics of applied behavior analysis interventions treatment decisions.
for children with developmental disorders. The Single-case research designs feature an important
argument is that the trained behavior would have component of replication of behavioral phenomena
developed anyway given enough time (without for a single individual (J. M. Johnston & Penny-
training). For example, Spreckley and Boyd (2009) packer, 2009; Sidman, 1960). With direct replica-
stated that “what is too often forgotten is that the tion, the intervention is repeated, and when behavior
overwhelming majority of children with [autism changes on each replication, the experimenter or
spectrum disorder] change over time as part of their therapist has identified one of the variables that con-
development as opposed to change resulting from trol the behavior. With systematic replication, the
an intervention” (p. 343). Commentaries such as same subject or different subjects or species may
these present a negation of the immense accumula- be exposed to a variation of an original procedure
tion of experimental and applied hard evidence that (Sidman, 1960), and a successful outcome fosters the
individual behavior can indeed be effectively and accumulation of knowledge (see also Chapter 7, this
reliably changed with the use of behavior control volume). Sidman’s (1971) original demonstration of
techniques. Yet such comments persist and slow the stimulus equivalence stands out as a golden example
application. Some behavior analysts appropriately of the scientific value of replication as a method of
react when professionals make such claims without proof because the remarkable results using a single
supporting data (e.g., Kimball, 2009; Morris, 2009). child, in a carefully designed experiment, have been
Group studies may have their place in behavior replicated numerous times and thereby spurred
analysis when intervention should not be withdrawn development of a whole new field of research and
because of the social significance of the target behavior. application.
Baer (1975) suggested combining methods of multiple- Behavior analysts are sometimes weary of designs
baseline designs and group comparisons, in which that compare groups of subjects because no behav-
one group of subjects first serves as a comparison to ioral phenomena are established at the level of the
another group who receives intervention; later, the individual subject with such designs. However, to
comparison group also receives the same intervention. promote behavioral phenomena using single-case
research designs, comparisons of experiments are
common, and one thereby inevitably compares
Conclusion
groups of subjects across experiments, often across
Behavior analysts have developed designs and tech- species as well. Indeed, such comparisons often
niques that can increase or decrease target behavior prove the generality of the basic principles discov-
on a certain occasion and time. These methods serve ered with one small set of subjects. Besides, estab-
as tools for an experimental analysis of behavior. lishing the efficiency of a behavioral procedure to be
The same tools are used in applied behavior analysis implemented on a large population scale may
to modify and maintain behavior for the purpose require certain types of controlled group studies.
of helping a client. Single-case research designs offer Group designs and multiple-baseline designs can
a wide range of methods, and this overview has also be profitably combined to produce powerful
merely scratched the surface. Because of their accu- and socially relevant effects on a large scale (e.g.,
mulated successes, single-case designs are now D. K. Fox et al., 1987).
being adopted in areas outside of behavior analysis With wide implementations in psychology, edu-
such as medicine (Janosky et al., 2009), occupational cation, medicine, and rehabilitation, single-case
therapy (M. V. Johnston & Smith, 2010), and pain methodology is now firmly established as a viable
management (Onghena & Edgington, 2005). means for discovery as well as for application of
Indeed, Guyatt et al. (2000), in their review of basic behavioral mechanisms. Of late, behavior
28
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
anagement interventions based on single-case

m Bijou, S. W., & Baer, D. M. (1961). Child development
methodology followed up with efficiency determina- I: A systematic and empirical theory. New York, NY:
Appleton-Century-Crofts. doi:10.1037/11139-000
tions from population studies have successfully
Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen,
demonstrated how the science of the individual can I., Kotchoubey, B., Kübler, A., . . . Flor, H. (1999).
be a science for all. A spelling device for the paralysed. Nature, 398,
297–298. doi:10.1038/18581
Individual prediction is of tremendous
Blampied, N. M. (1999). A legacy neglected: Restating the
importance, so long as the organism is to case for single-case research in cognitive behavior
be treated scientifically. (Skinner, 1938, therapy. Behaviour Change, 16, 89–104. doi:10.1375/
p. 444) bech.16.2.89
Boakes, R. (1984). From Darwin to behaviorism:
Man has at his disposal yet another pow- Psychology and the minds of animals. New York, NY:
erful resource—natural science with its Cambridge University Press.
strictly objective methods. This science, Boorstin, D. J. (1983). The discoverers: A history of man’s
as we all know, is making big headway search to know his world and himself. New York, NY:
every day. The facts and considerations Vintage Books.
I have placed before you are one of the Boring, E. G. (1929). A history of experimental psychology.
numerous attempts to employ—in study- New York, NY: Appleton-Century-Crofts.
ing the mechanism of the highest vital Boring, E. G. (1954). The nature and history of experi-
manifestations in the dog, the represen- mental control. American Journal of Psychology, 67,
573–589. doi:10.2307/1418483
tative of the animal kingdom which is
Brenagan, W., & Iversen, I. H. (2012). Methods to differ-
man’s best friend—a consistent, purely entially reinforce response duration in rats. Manuscript
scientific method of thinking. (From in preparation.
Pavlov’s acceptance speech on receiving Catania, A. C., & Laties, V. G. (1999). Pavlov and
the Nobel Prize in 1904; Pavlov, 1955, Skinner: Two lives in science. Journal of the
p. 148) Experimental Analysis of Behavior, 72, 455–461.
doi:10.1901/jeab.1999.72-455
References Cooper, J. O., Heron, T. E., & Heward, W. L. (2007).

Applied behavior analysis (2nd ed.). Upper Saddle
Baer, D. M. (1975). In the beginning there was the River, NJ: Pearson Education.
response. In E. Ramp & G. Semb (Eds.), Behavior
analysis: Areas of research and application (pp. 16–30). Crosbie, J. (1993). Interrupted time-series analysis with
Englewood Cliffs, NJ: Prentice-Hall. brief single-subject data. Journal of Consulting and
Clinical Psychology, 61, 966–974. doi:10.1037/0022-
Baer, D. M. (1993). Advising as if for research productiv- 006X.61.6.966
ity. Clinical Psychologist, 46, 106–109.
Dawson, G., Rogers, S., Munson, J., Smith, M., Winter,
Baer, D. M. (2005). Letters to a lawyer. In W. L. Heward, J., Greenson, J., . . . Varley, J. (2009). Randomized,
T. E. Heron, N. A. Neff, S. M. Peterson, D. M. controlled trial of an intervention for toddlers with
Sainato, G. Cartledge, . . . J. C. Dardig (Eds.), Focus autism: The early start Denver model. Pediatrics, 125,
on behavior analysis in education: Achievements, chal- e17–e23. doi:10.1542/peds.2009-958
lenges, and opportunities (pp. 3–30). Upper Saddle
River, NJ: Pearson. Dunlap, G., & Kincaid, D. (2001). The widening world of
functional assessment: Comments on four manuals
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some and beyond. Journal of Applied Behavior Analysis, 34,
current dimensions of applied behavior analysis. 365–377. doi:10.1901/jaba.2001.34-365
Journal of Applied Behavior Analysis, 1, 91–97.
doi:10.1901/jaba.1968.1-91 Ebbinghaus, H. (1913). Memory (H. A. Rueger & C. E.
Bussenius, Trans.). New York, NY: Teachers College.
Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single (Original work published 1885)
case experimental designs: Strategies for studying
behavior change (3rd ed.). New York, NY: Pearson Eysenck, H. J. (1960). Behavior therapy and the neuroses.
Education. New York, NY: Pergamon Press.
Bernard, C. (1957). An introduction to the study of experi- Fox, D. K., Hopkins, B. L., & Anger, W. K. (1987). The
mental medicine. New York, NY: Dover. (Original long-term effects of a token economy on safety
work published 1865) performance in open-pit mining. Journal of Applied
29
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
Behavior Analysis, 20, 215–224. doi:10.1901/ Iversen, I. H. (1991). Methods of analyzing behavior
jaba.1987.20-215 patterns. In I. H. Iversen & K. A. Lattal (Eds.),
Techniques in the behavioral and neural sciences:
Fox, R. G., Copeland, R. E., Harris, J. W., Rieth, H. J., & Experimental analysis of behavior, Part 2 (pp. 193–242).
Hall, R. V. (1975). A computerized system for select- New York, NY: Elsevier.
ing responsive teaching studies, catalogued along
twenty-eight important dimensions. In E. Ramp & Iversen, I. H. (1992). Skinner’s early research: From
G. Semb (Eds.), Behavior analysis: Areas of research reflexology to operant conditioning. American
and application (pp. 124–158). Englewood Cliffs, NJ: Psychologist, 47, 1318–1328. doi:10.1037/0003-
Prentice-Hall. 066X.47.11.1318
Ghezzi, P. M. (2007). Discrete trials teaching. Psychology Iversen, I. H. (2010). [Laboratory demonstration of acqui-
in the Schools, 44, 667–679. doi:10.1002/pits.20256 sition of operant behavior]. Unpublished raw data.
Green, G. (2005). Division fellow, Gina Green, reacts to Iversen, I. H. (2012). Tutorial: Multiple baseline designs.
CNN program “Autism is a World” which focuses Manuscript in preparation.
on facilitated communication. Psychology in Mental Iversen, I. H., Ghanayim, N., Kübler, A., Neumann, N.,
Retardation and Developmental Disabilities, 31, 7–10. Birbaumer, N., & Kaiser, J. (2008). A brain-
Green, G., Brennan, L. C., & Fein, D. (2002). Intensive computer interface tool to assess cognitive functions
behavioral treatment for a toddler at high risk in completely paralyzed patients with amyotrophic
for autism. Behavior Modification, 26, 69–102. lateral sclerosis. Clinical Neurophysiology, 119,
doi:10.1177/0145445502026001005 2214–2223. doi:10.1016/j.clinph.2008.07.001
Green, G., & Shane, H. C. (1994). Science, reason, and Iversen, I. H., & Matsuzawa, T. (1996). Visually guided
facilitated communication. Journal of the Association drawing in the chimpanzee (Pan troglodytes).
for Persons with Severe Handicaps, 19, 151–172. Japanese Psychological Research, 38, 126–135.
doi:10.1111/j.1468-5884.1996.tb00017.x
Greenspan, R. J., & Baars, B. J. (2005). Consciousness
Jacobson, J. W., Mulick, J. W., & Schwartz, A. A. (1995).
eclipsed: Jacques Loeb, Ivan P. Pavlov, and the rise
A history of facilitated communication: Science, pseu-
of reductionistic biology after 1900. Consciousness
doscience, and antiscience. American Psychologist, 50,
and Cognition, 14, 219–230. doi:10.1016/j.concog.
750–765. doi:10.1037/0003-066X.50.9.750
2004.09.004
Janosky, J. E., Leininger, S. L., Hoerger, M. P., &
Guyatt, G. H., Haynes, R. B., Jaeschke, R. Z., Cook, D. J., Libkuman, T. M. (2009). Single subject designs
Green, L., Naylor, C. D., . . . Richardson, W. S. in biomedicine. New York, NY: Springer-Verlag.
(2000). Users’ guides to the medical literature: XXV. doi:10.1007/978-90-481-2444-2
Evidence-based medicine: Principles for applying the
users’ guides to patient care. JAMA, 284, 1290–1296. Johnston, J. M., & Pennypacker, H. S. (2009). Strategies
doi:10.1001/jama.284.10.1290 and tactics in behavioral research (3rd ed.). New York,
NY: Routledge.
Harris, B. (1979). Whatever happened to little Albert?
American Psychologist, 34, 151–160. doi:10.1037/ Johnston, M. V., & Smith, R. O. (2010). Single subject
0003-066X.34.2.151 design: Current methodologies and future directions.
OTJR: Occupation,Participation and Health, 30, 4–10.
Henton, W. W., & Iversen, I. H. (1978). Classical condi- doi:10.3928/15394492-20091214-02
tioning and operant conditioning: A response pattern
analysis. New York, NY: Springer-Verlag. Jones, M. C. (1924). A laboratory study of fear: The case
of Peter. Pedagogical Seminary, 31, 308–315. doi:10.
Holth, P. (2010). A research pioneer’s wisdom: An inter- 1080/08856559.1924.9944851
view with Dr. Murray Sidman. European Journal of
Behavior Analysis, 11, 181–198. Kazdin, A. E. (1973). Methodological and assessment
considerations in evaluating reinforcement pro-
Horne, P. J., Tapper, K., Lowe, C. F., Hardman, C. A., grams in applied settings. Journal of Applied Behavior
Jackson, M. C., & Woolner, J. (2004). Increasing Analysis, 6, 517–531. doi:10.1901/jaba.1973.6-517
children’s fruit and vegetable consumption: A peer-
modeling and rewards-based intervention. European Kazdin, A. E. (2011). Single-case research designs: Methods
Journal of Clinical Nutrition, 58, 1649–1660. for clinical and applied settings (2nd ed.). New York,
doi:10.1038/sj.ejcn.1602024 NY: Oxford University Press.
Keller, F. S., & Schoenfeld, W. N. (1950). Principles of
Iversen, I. H. (1988). Tactics of graphic design: A
psychology. New York, NY: Appleton-Century-
review of Tufte’s The Visual Display of Quantitative
Crofts.
Information [Book review]. Journal of the
Experimental Analysis of Behavior, 49, 171–189. Kennedy, C. H. (2005). Single-case designs for educational
doi:10.1901/jeab.1988.49-171 research. New York, NY: Pearson Allyn & Bacon.
30
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kimball, J. W. (2009). Comments on Spreckley and Boyd (Ed.), Single-subject research: Strategies for evaluating
(2009). Science in Autism Treatment, 6, 3–19. change (pp. 101–165). New York, NY: Academic
Press.
Kratochwill, T. R., & Brody, G. H. (1978). Single sub-
ject designs: A perspective on the controversy Pavlov, I. P. (1906). The scientific investigation of the
over employing statistical inference and impli- psychical faculties or processes in higher animals.
cations for research and training in behavior Science, 24, 613–619. doi:10.1126/science.24.620.613
modification. Behavior Modification, 2, 291–307.
Pavlov, I. P. (1927). Conditioned reflexes (G. V. Anrep,
doi:10.1177/014544557823001
Trans.). London, England: Oxford University Press.
Loeb, J. (1900). Comparative physiology of the brain and
Pavlov, I. P. (1928). Lectures on conditioned reflexes:
comparative psychology. New York, NY: Putnam.
Twenty-five years of objective study of the higher nervous
doi:10.5962/bhl.title.1896
activity (behaviour) of animals (Vol. 1). New York, NY:
Lovaas, O. I. (1987). Behavioral treatment and normal International Publishers. doi:10.1037/11081-000
educational and intellectual functioning in young Pavlov, I. P. (1955). Nobel speech delivered in Stockholm
autistic children. Journal of Consulting and Clinical on December 12, 1904. In K. S. Koshtoyants (Ed.),
Psychology, 55, 3–9. doi:10.1037/0022-006X.55.1.3 I. P. Pavlov: Selected works (pp. 129–148). Honolulu,
Lowe, F. (2010, September). Can behavior analysis change HI: University Press of the Pacific.
the world? Paper presented at the Ninth International Schmidt, R. A., & Lee, T. D. (2005). Motor control and
Congress on Behavior Studies, Crete, Greece. learning: A behavioral emphasis. Champaign, IL:
Matson, J. L., Sevin, J. A., Fridley, D., & Love, S. R. Human Kinetics.
(1990). Increasing spontaneous language in three Shull, R. L. (1991). Mathematical description of operant
autistic children. Journal of Applied Behavior Analysis, behavior: An introduction. In I. H. Iversen & K. A.
23, 227–233. doi:10.1901/jaba.1990.23-227 Lattal (Eds.), Experimental analysis of behavior
McDougall, D., Hawkins, J., Brady, M., & Jenkins, A. (Vol. 2, pp. 243–282). New York, NY: Elsevier.
(2006). Recent innovations in the changing criterion Sidman, M. (1960). Tactics of scientific research:
design: Implications for research and practice in spe- Evaluating experimental data in psychology. New
cial education. Journal of Special Education, 40, 2–15. York, NY: Basic Books.
doi:10.1177/00224669060400010101
Sidman, M. (1971). Reading and auditory-visual equiva-
Montee, B. B., Miltenberger, R. G., & Wittrock, D. lences. Journal of Speech and Hearing Research, 14,
(1995). An experimental analysis of facilitated com- 5–13.
munication. Journal of Applied Behavior Analysis, 28,
189–200. doi:10.1901/jaba.1995.28-189 Sidman, M. (1981). Remarks. Behaviorism, 9, 127–129.
Morgan, D. L., & Morgan, R. K. (2009). Single-case Skinner, B. F. (1938). The behavior of organisms: An exper-
research methods for the behavioral and health sciences. imental analysis. New York, NY: Appleton-Century-
Los Angeles, CA: Sage. Crofts.
Morris, E. K. (2009). A case study in the misrepresenta- Skinner, B. F. (1953). Science and human behavior. New
tion of applied behavior analysis in autism: The York, NY: Macmillan.
Gernsbacher lectures. Behavior Analyst, 32, 205–240. Skinner, B. F. (1956). A case history in scientific method.
Neef, N. A., & Iwata, B. A. (1994). Current research on American Psychologist, 11, 221–233. doi:10.1037/
functional analysis methodologies: An introduction. h0047662
Journal of Applied Behavior Analysis, 27, 211–214. Skinner, B. F. (1966). Some responses to the stimulus
doi:10.1901/jaba.1994.27-211 “Pavlov.”Conditional Reflex, 1, 74–78.
O’Neill, R. E., McDonnell, J., Billingsly, F., & Jenson, W. Skinner, B. F. (1969). Contingencies of reinforcement. New
(2010). Single case designs in educational and commu- York, NY: Appleton-Century-Crofts.
nity settings. New York, NY: Merrill.
Skinner, B. F. (1978). The ethics of helping people. In
Onghena, P., & Edgington, E. S. (2005). Customization B. F. Skinner (Ed.), Reflections on behaviorism and
of pain treatment: Single-case design and society (pp. 33–47). Englewood Cliffs, NJ: Prentice
analysis. Clinical Journal of Pain, 21, 56–68. Hall.
doi:10.1097/00002508-200501000-00007
Skinner, B. F. (1979). The shaping of a behaviorist. New
Paré, W. P. (1990). Pavlov as a psychophysiological York, NY: Knopf.
scientist. Brain Research Bulletin, 24, 643–649.
doi:10.1016/0361-9230(90)90002-H Spreckley, M., & Boyd, R. (2009). Efficacy of applied
behavioral intervention in preschool children with
Parsonson, B. S., & Baer, D. M. (1978). The analysis and autism for improving cognitive, language, and adap-
presentation of graphic data. In T. R. Kratochwill tive behavior: A systematic review and meta analysis.
31
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Iver H. Iversen
Journal of Pediatrics, 154, 338–344. doi:10.1016/j. Tufte, E. R. (1990). Envisioning information. Cheshire,
jpeds.2008.09.012 CT: Graphics Press.
Steege, M. W., & Mace, F. C. (2007). Applied behavior Ullman, L. P., & Krasner, L. (1965). Case studies in behav-
analysis: Beyond discrete trial teaching. Psychology in ior modification. New York, NY: Holt, Rinehart &
the Schools, 44, 91–99. doi:10.1002/pits.20208 Winston.
Sulzer-Azaroff, B., & Mayer, G. R. (1991). Behavior Van Houten, R., Axelrod, S., Baily, J. S., Favell, J. F.,
analysis for lasting change. Fort Worth, TX: Harcourt Foxx, R. M., Iwata, B. A., & Lovaas, O. I. (1988).
Brace. The right to effective behavioral treatment. Journal of
Applied Behavior Analysis, 21, 381–384. doi:10.1901/
Taub, E., Uswatte, G., King, D. K., Morris, D., Crago, jaba.1988.21-381
J. E., & Chatterjee, A. (2006). A placebo-controlled
trial of constraint induced movement therapy for Watson, J. B., & Rayner, R. (1920). Conditioned emo-
upper extremity after stroke. Stroke, 37, 1045–1049. tional reactions. Journal of Experimental Psychology,
doi:10.1161/01.STR.0000206463.66461.97 3, 1–14. doi:10.1037/h0069608
Thorndike, E. L. (1911). Animal intelligence. New York, Wheeler, D. L., Jacobson, J. W., Paglieri, R. A., &
NY: Macmillan. Schwartz, A. A. (1993). An experimental assessment
of facilitated communication. Mental Retardation, 31,
Thorndike, E. L. (1927). The law of effect. American Journal 49–59.
of Psychology, 39, 212–222. doi:10.2307/1415413
Wolpe, J. (1958). Psychotherapy by reciprocal inhibition.
Todes, D. P. (2002). Pavlov’s physiological factory: Stanford, CA: Stanford University Press.
Experiment, interpretation, laboratory enterprise.
Baltimore, MD: Johns Hopkins University Press. Wood, J. D. (2004). The first Nobel prize for inte-
grated systems physiology: Ivan Petrovich Pavlov,
Tufte, E. R. (1983). The visual display of quantitative infor- 1904. Physiology, 19, 326–330. doi:10.1152/
mation. Cheshire, CT: Graphics Press. physiol.00034.2004
32
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 2
The Five Pillars of the

Experimental Analysis of
Behavior
Kennon A. Lattal
“What is the experimental analysis of behavior?” control are not possible in the absence of the rein-
Skinner (1966) famously asked in an address to Divi- forcement that maintains the behavior being pun-
sion 25 of the American Psychological Association, ished or under control of other stimuli; stimuli
now the Division for Behavior Analysis (then the correlated with reinforcers and punishers also have
Division for the Experimental Analysis of Behavior). their effects only in the context of reinforcement and
His answer included a set of methods and a subject are also closely related to other, more direct stimulus
matter, both of which originated with his research control processes; and punishment affects both rein-
and conceptual analyses that began in the 1930s. forcement and stimulus control.
Since those early days, what began as operant condi-
tioning has evolved from its humble laboratory and
Pillar 1: Research Methods
nonhuman animal origins to encompass the breadth
of contemporary psychology. This handbook, which Research methods in TEAB are more than a set of
is itself testimony to the preceding observation, is techniques for collecting and analyzing data. They
the impetus for revisiting Skinner’s question. The certainly enable those activities, but, more impor-
developments described in each of the handbook’s tant, they reflect the basic epistemological stance of
chapters are predicated on a few fundamental princi- behavior analysis: The determinants of behavior are
ples, considered here as the pillars that constitute the to be found in the interactions between individuals
foundation of the experimental analysis of behavior and their environment. This stance led to the adop-
(TEAB). These five pillars—research methods, rein- tion and evolution of methods and concepts that
forcement, punishment, control by stimuli correlated emphasize the analysis of functional relations
with reinforcers and punishers, and contextual and between features of that environment and the behav-
stimulus control—are the subject of this chapter. ior of individual organisms. Skinner (1956) put it
Together, they provide the conceptual and empirical as follows:
framework for understanding the ways in which
environmental events interact with behavior. The We are within reach of a science of the
pillars, although discussed separately from one individual. This will be achieved not
another here for didactic purposes, are inextricably by resorting to some special theory of
linked: The methods of TEAB are pervasive in inves- knowledge in which intuition or under-
tigating the other pillars; punishment and stimulus standing takes the place of observation
This chapter is dedicated to Stephen B. Kendall, who, as my first instructor and, later, mentor in the experimental analysis of behavior, provided an
environment that allowed me to learn about the experimental analysis of behavior by experimenting.
I thank Karen Anderson, Liz Kyonka, Jack Marr, Mike Perone, and Claire St. Peter for helpful discussions on specific topics reviewed in this
chapter, and Rogelio Escobar and Carlos Cançado for their valuable comments on an earlier version of the chapter.
DOI: 10.1037/13937-002
33
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
and analysis, but through an increasing TEAB involve the repeated observation of the tar-
grasp of relevant conditions to produce geted response over multiple sessions until an
order in the individual case. (p. 95) appropriate level of stability is achieved. Without an
appropriate design and an appropriate degree of sta-
Single-Case Procedures and Designs bility in the baseline, attributing the changes in the
Two distinguishing features of research methods in dependent variable to the independent variable is
TEAB are emphases on what Bachrach (1960) called not possible.
the informal theoretical approach and on examining Decisions about the criteria for the baseline begin
the effects of independent variables on well-defined with a definition of the response. Skinner (1966)
responses of individual subjects. Skinner’s (1956) noted that “an emphasis on rate of occurrence of
review of his early research defined Bachrach’s label. repeated instances of an operant distinguishes the
The inductive tradition allows free rein in isolating experimental analysis of behavior” (p. 213). Unless
the variables of which behavior is a function, unen- the response is systematically measured and repeat-
cumbered by many of the shoulds, oughts, and able, stability will be difficult, if not impossible, to
inflexibilities of research designs derived from infer- achieve. The dimensions of baseline stability are the
ential statistical research methods (Michael, 1974; amount of variability or bounce in the data and the
see Chapters 5 and 7, this volume). extent of trends. Baseline stability criteria are typi-
The essence of the second feature, single-case cally established relative to the anticipated effects of
experimental designs, is that by investigating the the independent variable. Thus, if a large effect of
effects of the independent variable on individual the independent variable is anticipated, more varia-
subjects, each subject serves as its own control. tion in the baseline is acceptable because, presum-
Thus, effects of independent variables are compared, ably, the effect will be outside the baseline range.
within individual subjects, with a baseline on which Similarly, if a strong downward trend is expected
the independent variable is absent (or present at when the independent variable is introduced, then
some other value). This methodological approach to an upward trend in the baseline data is more accept-
analyzing the subject matter of psychology can be able than if the independent variable were expected
contrasted with an approach based on the inferential to increase the rate of responding.
statistical analysis of the data generated across dif- Some circumstances may not seem to lend them-
ferent groups of subjects exposed to the presence or selves readily to single-case designs. An example is
absence or different values of the independent vari- that in which behavioral effects cannot be reversed,
able (Michael, 1974; see Chapters 7 and 8, this vol- and thus baselines cannot be recovered. Sidman
ume). A single-case analysis precludes a major (1960), however, observed that even in such cases,
source of variation inherent in all group designs: “the use of separate groups destroys the continuity
that variation resulting from between-subjects com- of cause and effect that characterizes an irreversible
parisons. It also minimizes other so-called threats to behavioral process” (p. 53). In the case of irrevers-
internal validity such as those associated with statis- ible effects, creative designs in the single-case tradi-
tical regression toward the mean and subject selection have been used to circumvent the problem.
tion biases (cf. Kazdin, 1982). Boren and Devine (1968), for example, investigated
Three central features of single-case research are the repeated acquisition of behavioral chains by
selecting an appropriate design, establishing base- arranging a task in which monkeys learned a
line performance, and selecting the number of subsequence of 10 responses. Once that pattern was sta-
jects to study. The most basic design involves first ble, the pattern was changed, and the monkeys had
establishing a baseline, A; then introducing the inde- to learn a new sequence. This procedure allowed the
pendent variable, B; and finally returning to the study of repeated acquisition of behavioral chains in
baseline. From this basic A-B-A design, many varia- individual subjects across a relatively long period of
tions spring (Kazdin, 1982; see Chapter 5, this vol- time. In other cases either in which baselines are
ume). Within-subject designs in the tradition of unlikely to be reversed or in which it is ethically
34
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
The Five Pillars of the Experimental Analysis of Behavior
questionable to do so, a multiple-baseline design method. When individual subjects’ behavior is

often can be used (Baer, Wolf, & Risley, 1968; see studied, it is not surprising that the same value of a
Chapter 5, this volume). variable may have different effects across subjects.
Selecting the number of subjects is based on both For example, a relatively brief delay of reinforce-
practical concerns and experimenter judgment. A ment may markedly reduce the responding of one
few studies in the Journal of the Experimental Analy- subject and not change the responding of another.
sis of Behavior were truly single subject in that they Rather than averaging the two, the tactic in TEAB
involved only one subject (e.g., de Lorge, 1971), but is to conduct a parametric analysis of delay dura-
most have involved more. Decisions about numbers tion with both subjects, to search for orderly and
of subjects interact with decisions about the design, qualitatively similar functional relations across
types of independent variables being studied, and subjects even though, on the basis of intersubject
range of values of these variables. Between-subjects comparisons, individuals may respond differently
direct replications and both between- and within- at any particular value. The achievement of these
subject systematic replications involving different qualitatively similar functional relations contrib-
values of the independent variable increase the gen- utes to the generality of the effect. The establish-
erality of the findings (see Chapter 7, this volume). ment of experimental control through the methods
Individual-subject research has emphasized exper- described in this section is a major theme of TEAB,
imental control over variability, as contrasted with exemplified in each of the other, empirical pillars
group designs in which important sources of variabil- of TEAB. Before reviewing those empirical pillars,
ity are isolated statistically at the conclusion of the however, examining how critical features of the
experiment. Sidman (1960) noted that “acceptance of environment are defined and used in TEAB is
variability as unavoidable or, in some sense as repre- important.
sentative of ‘the real world’ is a philosophy that leads
to the ignoring of relevant factors” (p. 152). He Defining Environmental Events
methodically laid out the tactics of minimizing vari- In the laboratory, responses often are defined opera-
ability in experimental situations and identified sev- tionally, as a matter of convenience, in terms of,
eral sources of such variability. Between-subjects for example, a switch closure. In principle, how-
variability already has been discussed. Another major ever, they are defined functionally, in terms of their
source of variability discussed by Sidman is that effects. Baer (1981) observed that although every
resulting from weak experimental control. Inferential response or class of responses has form or structure,
statistical analysis is sometimes used to supplement operant behavior has no necessary structure. Rather,
experimental analysis. Such analysis is not needed if its structure is determined by environmental
baseline and manipulation of response distributions circumstance:
do not overlap, but sometimes they do, and in these
cases some behavior analysts have argued for their The structure of operant behavior is lim-
inclusion (see, e.g., Baron [1999] and particularly ited primarily by our ability to arrange
Davison [1999] on the potential utility of nonpara- the environment into contingencies with
metric statistics in within-subject designs in which that behavior; to the extent that we can
baseline and intervention distributions overlap). Oth- wield the environment more and more
ers (e.g., Michael, 1974), however, have noted that completely, to that extent behavior has
statistical analysis of group data draws the focus away less and less necessary structure. This is
from an experimental analysis of effects demonstrable tantamount to saying that it is mainly our
in individual subjects, removes the experimenter current relatively low level of technologi-
from the data, and substitutes statistical control for cal control over the environment that
experimental control. seems to leave behavior with apparent
A final point with respect to single-case designs necessary structure and that such a limi-
relates to the previous discussion of the inductive tation is trivial. (Baer, 1981, p. 220).
35
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
Functional definitions in TEAB originated with eliminated. Responding then was maintained when
Skinner’s (1935) concept of the operant. With that its only consequence was to deliver an electric shock
analysis, he organized fluid, unique individual that had previously been avoided.
responses into integrated units or classes— As the research cited by Morse and Kelleher
operants—whereby all the unique members have (1977) suggests, the type of event is less important
the same effect on the environment. Stimuli were than its behavioral effect (which in turn depends on
similarly organized into functional classes on the the organism’s history of interaction with the event).
basis of similarity of environmental (behavioral) Events that increase or maintain responding when
effect. He also conceptualized reinforcement not in made dependent on the response are categorized as
terms of its forms or features, but functionally, in reinforcers, and events that suppress or eliminate
terms of its effects on responses. Thus, any environ- responding are categorized as punishers. Associating
mental event can function, in principle, as a rein- a valence, positive or negative, with these function-
forcer or as a punisher, or as neither, depending on ally defined reinforcers and punishers is conven-
how it affects behavior. On the question of circular- tional. The valence describes the operation whereby
ity, Skinner (1938) simply noted that the behavioral effect of the consequence occurs, that
is, whether the event is added to or subtracted from
a reinforcing stimulus is defined as such
the environment, which yields a 2 × 2 contingency
by its power to produce the resulting
table in which valence is shown as a function of
change. There is no circularity about
behavioral change (maintain or increase in the case
this; some stimuli are found to produce
of reinforcement, and decrease or eliminate in the
the change, others not, and they are clas-
case of punishment).
sified as reinforcing and non-reinforcing
Despite widespread adoption of this categoriza-
accordingly. (p. 62)
tion system, the use of valences has been criticized
Another aspect of the contextual basis of reinforcers on the grounds that they are arbitrary and ambigu-
and punishers is what Keller and Schoenfeld (1950; ous (Baron & Galizio, 2005; Michael, 1975). Baron
cf. Michael, 1982) called the establishing operation. and Galizio (2005) cited an experiment in which
The establishing operation delineates the conditions “rats kept in a cold chamber would press a lever that
necessary for some event or activity to function as a turned on a heat lamp” (p. 87) to make the point
reinforcer or punisher. In most laboratory research, that in such cases it indeed is difficult to separate
establishing a reinforcer involves restricted access, cold removal and heat presentation. Presenting
be it, for example, to food or to periods free of elec- food, it has been argued, may be tantamount to
tric shock delivery. That is, reinforcers and punish- removing (or at least reducing) deprivation and
ers have to be established by constructing a specific removing electric shock may be tantamount to pre-
context or history. Morse and Kelleher (1977) sum- senting a shock-free period (cf. Verhave, 1962).
marized several experiments in which they sug- Although the Michael (1975) and Baron and Galizio
gested that electric shock delivery sufficient to position falls on sympathetic ears (e.g., Marr, 2006),
maintain avoidance behavior was established as a the distinction continues. The continued use of
positive reinforcer. This establishment was accom- the positive–negative distinction is a commentary
plished by creating particular kinds of behavioral on its utility in the general verbal community of
histories. McKearney (1972), for example, first behavior analysts as well as in application and
maintained responding by a free-operant shock- teaching. Despite potential ambiguities in some cir-
avoidance schedule and then concurrently super cumstances, the operations are clear—events that
imposed a schedule of similar shocks delivered experimenters present and remove are sufficiently
independently of responding. This schedule was in straightforward to allow description. The ques-
turn replaced by response-dependent shocks sched- tion of valences, as with any question in a science,
uled at the same rate as the previously response- should have an empirical answer. Because the jury is
independent ones, and the avoidance schedule was still out on this question, the long-enduring practice
36
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
of identifying valences on the basis of experimental techniques for establishing operant responses, the
operations is retained in this chapter. success of the instructions depends on their preci-
sion. An alternative form of instructional control is
to physically guide the response (e.g., Gibson,
Pillar 2: Reinforcement
1966). Such guided practice may be considered a
An organism behaves in the context of an environ- form of imitation, although imitation as a more gen-
ment in which other events are constantly occur- eral technique of establishing operant behavior does
ring, some as a result of its responses, and others not involve direct physical contact with the learner.
independent of its responses. One outcome of some The gold-standard technique for establishing an
of these interactions is that the response becomes operant response is the differential reinforcement of
more likely than it would be in their absence. Such successive approximations, or shaping. Discovered
an outcome is particularly effective when a depen- by Skinner in the 1940s, shaping involves immedi-
dency exists between such environmental events ately reinforcing successively closer approximations
and behavior, a two-term contingency involving to the target response (e.g., Eckerman, Hienz, Stern,
responding and what will come to function as a & Kowlowitz, 1980; Pear & Legris, 1987). A sophis-
reinforcer. This process of reinforcement is funda- ticated analysis of shaping is that of Platt (1973),
mental in understanding behavior and is thus a who extensively studied the shaping of interresponse
pillar of TEAB. Of the four empirical pillars, rein- times (IRTs; the time between successive responses).
forcement may be considered the most basic because Baron (1991) also described a procedure for shaping
the other three cannot exist in the absence of rein- responding under a shock-avoidance contingency.
forcement. Each of the other pillars adds another Shaping is part of any organism’s day-to-day interac-
element to reinforced responding. tions with its environment. Whether one is hammer-
ing a nail or learning a new computer program, the
Establishing a Response natural and immediate consequences of an action
As noted, to establish an operant response, a rein- play a critical role in determining whether a given
forcer must be established and the target response response will be eliminated, repeated, or modified.
specified precisely so that it is distinguished from Some researchers have suggested that shaping
other response forms. Several techniques may then occurs when established reinforcers occur indepen-
be used to bring about the target response. One is to dently of responding. Skinner (1948; see also
simply wait until it occurs (e.g., Neuringer, 1970); Neuringer, 1970), for example, provided food-
however, the target response may never occur with- deprived pigeons with brief access to food at 15-s
out more direct intervention. Some responses can be intervals. Each pigeon developed repetitive stereo-
elicited or evoked through a technique known collo- typed responses. Skinner attributed the outcome to
quially as “baiting the operandum.” Spreading a accidental temporal contiguity between the response
little peanut butter on a lever, for example, evokes and food delivery. His interpretation, however, was
considerable exploration by the rat of the lever, challenged by Staddon and Simmelhag (1971) and
typically resulting in its depression, which then can Timberlake and Lucas (1985), who attributed the
be reinforced conventionally. The difficulty is that resulting behavior to biological and ecological pro-
such baiting sometimes results in atypical response cesses rather than reinforcement. Nonetheless, the
topographies that later can be problematic. A partic- notion of superstitious behavior resulting from acci-
ularly effective technique related to baiting is to dental pairings of response and reinforcer remains
elicit the response through a Pavlovian conditioning an important methodological and interpretational
procedure known as autoshaping (Brown & Jenkins, concept in TEAB (e.g., the changeover delay used
1968). Once elicited, the response then can be rein- ubiquitously in concurrent schedules is predicated
forced. With humans, instructions are often an effi- on its value in eliminating the adventitious rein-
cient means of establishing an operant response forcement of changing between concurrently avail-
(see Rules and Instructions section). As with all able operanda).
37
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
Positive Reinforcement former arrangement is described as differential-

Positive reinforcement is the development or mainte- reinforcement-of-low-rate (DRL), or an IRT > t,
nance of a response resulting from the response- schedule, and the latter as a differential-reinforcement-
dependent, time-limited presentation of a stimulus of-high-rate, or an IRT < t, schedule. The latter in par-
or event (i.e., a positive reinforcer). ticular often is arranged such that the first IRT < t after
the passage of a variable period of time is reinforced.
Schedules of positive reinforcement. A schedule is The various individual schedules can be com-
a prescription for arranging reinforcers in relation to bined to yield more complex arrangements, suited
time and responses (Zeiler, 1984). The simplest such for the analysis of particular behavioral processes.
arrangement is to deliver reinforcers independently The taxonomic details of such schedules are beyond
of responding. Zeiler (1968), for example, first the scope of this chapter (see Ferster & Skinner,
stabilized key pecking of pigeons on fixed-interval 1957; Lattal, 1991). Several of them, however, are
(FI) or variable-interval (VI) schedules. Then, the described in other sections of this chapter in the
response–reinforcer dependency was eliminated so context of particular behavioral processes.
that reinforcers were delivered independently of key Schedules of reinforcement are important in
pecking at the end of fixed or variable time periods. TEAB because they provide useful baselines for the
This elimination generally reduced response rates, analysis of other behavioral phenomena. Their
but the patterns of responding continued to be importance, however, goes much further than this.
determined by the temporal distribution of reinforc- The ways in which consequences are scheduled are
ers: Fixed-time schedules yielded positively acceler- fundamental in determining behavior. This point
ated responding across the interfood intervals, and resonates with the earlier Baer (1981) quotation in
variable-time (VT) schedules yielded more evenly the Defining Environmental Events section about
distributed responding across those intervals. behavioral structure. The very form of behavior is
Zeiler’s (1968) experiment underlines the impor- a function of the organism’s history, of the ways
tance of response–reinforcer dependency in schedule- in which reinforcement has been arranged—
maintained responding. This dependency has been scheduled—in the past as well as in the present.
implemented in two ways in reinforcement schedules.
In ratio schedules, either a fixed or a variable number Parameters of positive reinforcement. The sched-
of responses is the sole requirement for reinforce- ules described in the previous section have their
ment. In interval schedules (as distinguished from effects on behavior as a function of the parameters
time schedules, in which the response–reinforcer of the reinforcers that they arrange. Four widely
dependency is absent), a single response after a fixed studied parameters of reinforcement are dependency,
or variable time period is the requirement for rein- rate, delay, and amount.
forcement. Each of these four schedules—fixed ratio The importance of the response–reinforcer depen-
(FR), variable ratio (VR), FI, and VI—control well- dency in response maintenance has been described in
known characteristic response patterns. In addition, the preceding section. Its significance is underscored
the distribution of reinforcers in VI and VR schedules, by subsequent experiments showing that variations
respectively, affect the latency to the first response in the frequency with which this dependency is
after a reinforcer and, with the VI schedule, the distri- imposed or omitted modulate response rates (e.g.,
bution of responses across the interreinforcer interval Lattal, 1974; Lattal & Bryan, 1976). In addition, add-
(Blakely & Schlinger, 1988; Catania & Reynolds, ing response-independent reinforcers when respond-
1968; Lund, 1976). ing is maintained under different schedules changes
Other arrangements derive from these basic both rates and patterns of responding (e.g., Lattal &
schedules. For example, reinforcing a sequence of Bryan, 1976; Lattal, Freeman, & Critchfield, 1989).
two responses separated from one another by a rela- Reinforcement rate is varied on interval sched-
tively long or a relatively short time period results in, ules by changing the interreinforcer interval and on
respectively, low and high rates of responding. The ratio schedules by varying the response requirement.
38
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
The effects of reinforcement rate depend on what is response-dependent, time-limited removal of some
measured (e.g., response rate, latency to the first stimulus or event (i.e., a negative reinforcer).
response after a reinforcer). Generally speaking, pos-
Schedules of negative reinforcement. Schedules
itively decelerated hyperbolic functions describe the
of negative reinforcement involve contingencies in
relation between response rate and reinforcement
which situations are either terminated or postponed
rate (Blakely & Schlinger, 1988; Catania & Reyn-
as a consequence of the response. The prototypical
olds, 1968; Felton & Lyon, 1966; but see the Behav-
stimulus used as the negative reinforcer in labora-
ioral Economics section later in this chapter—if the
tory investigations of negative reinforcement is
economy is closed, a different relation may hold).
electrical stimulation, because of both its reliability
Delaying a reinforcer from the response that pro-
and its specifiability in physical terms, although
duces it generally decreases response rates as a func-
examples of negative reinforcement involving other
tion of the delay duration (whether the delay is
types of events abound.
accompanied by a stimulus change) and the sched-
ule on which it is imposed (Lattal, 2010). The Escape. Responding according to some sched-
effects of the delay also may be separated from the ule intermittently terminates the delivery of elec-
inevitable changes in reinforcement rate and distri- tric shocks for short periods. Azrin, Holz, Hake,
bution that accompany the introduction of a delay and Allyon (1963) delivered to squirrel monkeys
of reinforcement (Lattal, 1987). response-independent shocks according to a VT
Amount of reinforcement includes both its form schedule. A fixed number of lever presses suspended
and its quantity. In terms of form, some reinforcers shock delivery and changed the stimulus condi-
are substitutable for one another to differing degrees tions (turning on a tone and dimming the chamber
(e.g., root beer and lemon-lime soda), whereas other lights) for a fixed time period. Lever pressing was
qualitatively different reinforcers do not substitute a function of both the duration of the time out and
for one another, but may be complementary. Two shock intensity, but the data were in the form of
complementary reinforcers covary with one another cumulative records, precluding a quantitative analy-
(e.g., food and water). Reinforcers that vary in consis of the functional relations between responding
centration (e.g., a 10% sucrose solution vs. a 50% and these variables. This and an earlier experiment
sucrose solution), magnitude (one vs. six food pel- on VI escape schedules with rats (Dinsmoor, 1962)
lets), or duration (1 s versus 6 s of food access) often are among the few studies reporting the effects of
have variable effects on behavior (see review by schedules of negative reinforcement based on shock
Bonem & Crossman, 1988), with some investigators termination, thereby limiting the generality of the
(e.g., Blakely & Schlinger, 1988) reporting system- findings.
atic differences as a function of duration, but others One problem with using escape from electric
not (Bonem & Crossman, 1988). One variable that shock is that shock can elicit responses, such as
affects these different outcomes is whether the quan- freezing or emotional reactions, that are incompati-
titatively different reinforcers are arranged across ble with the operant escape response. An alternative
successive conditions or within individual sessions method of studying escape that circumvents this
(Catania, 1963). DeGrandpre, Bickel, Hughes, problem is a timeout from the avoidance procedure
Layng, and Badger (1993) suggested that reinforcer first described by Verhave (1962). Perone and Gal-
amount effects are better predicted by taking into izio (1987, Experiment 1) trained rats to lever press
account both other reinforcement parameters and when this response postponed the delivery of sched-
the schedule requirements (see Volume 2, Chapter 8, uled shocks. At the same time, a multiple schedule
this handbook). was in effect for a second lever in the chamber.
During one of the multiple-schedule components,
Negative Reinforcement pressing the second lever produced timeouts from
Negative reinforcement is the development or avoidance (i.e., escape from the avoidance contin-
maintenance of a response resulting from the gency and the stimuli associated with it) according
39
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
to a VI schedule. Presses on the second lever had no Parameters of negative reinforcement. Two
effect during the other multiple-schedule compo- variables that affect the rate of responding under
nent (escape extinction). During the VI escape com- schedules of negative reinforcement are the param-
ponent, responding on the second lever was of eters (e.g., type, frequency [S-S interval in the case
moderate rate and constant over time, but it was of free-operant avoidance], intensity, duration) of
infrequent in the escape-extinction component. the stimulus that is to be escaped or avoided and
Other parameters of timeout from avoidance largely the duration of the period of stimulus avoidance
have been unexplored. or elimination yielded by each response (e.g., the
R-S interval in Sidman avoidance). Leander (1973)
Avoidance. The difference between escape and found that response rates on free-operant avoid-
avoidance is one of degree rather than kind. When ance schedules were an increasing function of the
the escape procedure is conceptualized convention- interaction between electric shock intensity and
ally as allowing response-produced termination of duration (cf. Das Graças de Souza, de Moraes, &
a currently present stimulus, avoidance procedures Todorov, 1984). Shock frequency can be manipu-
allow responses to preclude, cancel, or postpone lated by changing either the S-S or the R-S interval.
stimuli that, in the absence of the response, will With other parameters of the avoidance contin-
occur. The presentation of an electric shock, for gency held constant, Sidman (1953) showed that
example, is preceded by a warning stimulus, dur- response rates increased with shorter S-S intervals.
ing which a response terminates the stimulus and Response rates also vary inversely with the dura-
cancels the impending shock. Thus, there is escape tion of the R-S interval during free-operant avoid-
from a stimulus associated with a negative reinforcer ance, such that shorter R-S intervals control higher
as well as avoidance of the negative reinforcer itself. response rates than longer R-S intervals (Sidman,
Although a good bit of research has been conducted 1953). Furthermore, Logue and de Villiers (1978)
on discriminated avoidance (in which a stimulus used concurrent variable-cycle avoidance schedules
change precedes an impending negative reinforcer, to show that response rates on operanda associated
e.g., Hoffman, 1966), avoidance unaccompanied by with these schedules were proportional to the fre-
stimulus change is more commonly investigated in quency of scheduled shocks (rate of negative rein-
TEAB. forcement) arranged on the two alternatives. That is,
Free-operant avoidance, sometimes labeled non- more frequently scheduled shocks controlled higher
discriminated or unsignaled avoidance, is character- response rates than did less frequently scheduled
ized procedurally by the absence of an exteroceptive shocks.
stimulus change after the response that postpones or
deletes a forthcoming stimulus, such as electric Extinction
shock. The original free-operant avoidance proce- Extinction is functionally a reduction or elimination
dure was described by Sidman (1953) and often of responding brought about in either of two general
bears his name. Each response postponed for a fixed operations: by removing the positive or negative rein-
period an otherwise-scheduled electric shock. If a forcer or by rendering the reinforcer ineffective by
shock was delivered, subsequent shocks were deliv- eliminating the establishing operation. The former is
ered at fixed intervals until the response occurred. described hereafter as conventional extinction, because
These two temporal parameters, labeled, respec- these operations are the ones more commonly used
tively, the response–shock (R-S) and shock–shock in TEAB when analyzing extinction. With positive
(S-S) intervals together determine response rates. reinforcement, the latter is accomplished by either
Deletion and fixed- and variable-cycle avoidance providing continuous access to the reinforcer (satia-
schedules arrange the cancellation of otherwise tion) or by removing the response–reinforcer depen-
unsignaled, scheduled shocks as a function of dency (Rescorla & Skucy, 1969; see Schedules of
responding, with effects on responding similar to Positive Reinforcement section earlier in this chapter
those of Sidman avoidance (see Baron, 1991). for the effects of this operation on responding). With
40
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
negative reinforcement, extinction is accomplished analyses of the reinforcement of operant behavior.

by making the negative reinforcer inescapable. All begin with description. Many involve quantita-
The rapidity of extinction depends both on the tive analysis and extrapolation through modeling.
organism’s history of reinforcement and probably Others, although also quantitative in the sense
(although experimental analyses are lacking) on of reducing measurement down to numerical repre-
which of the aforementioned procedures are used to sentation, are less abstract, remaining closer to
arrange extinction (Shnidman, 1968). Herrnstein observed functional relations. Each has been suc-
(1969) suggested that the speed of extinction of cessful in accounting for numerous aspects of
avoidance is related to the discriminability of the behavioral phenomena. Each also has limitations.
extinction contingency, an observation that holds as None has achieved universal acceptance. The result
well in the case of positive reinforcement. Extinc- is that instead of representing a progression with
tion effects are rarely permanent once reinforcement one framework leading to another, these frame-
is reinstated. Permanent effects of extinction are works together make up a web of interrelated obser-
likely the result of the alternative reinforcement of vations, each contributing something to the general
other responses while extinction of the targeted understanding of how reinforcement has its effects
response is in effect. on behavior. The following sections provide an
Extinction also can generate other responses, overview of some of these contributions to this web.
some of which may be generalized or induced from
the extinguished response itself and others of which Levels of influence. Thorndike (1911) observed
depend on other stimuli in the environment in that
which extinction occurs (see Volume 2, Chapter 4,
of several responses made to the same
this handbook). Some instances of such behavior are
situation, those which are accompanied
described as schedule induced and are perhaps more
or closely followed by satisfaction to the
accurately labeled extinction induced because they
animal will, other things being equal, be
typically occur during those parts of a schedule
more firmly connected with the situation,
associated with nonreinforcement (local extinction).
so that, when it recurs, they will be more
For example, such responding is observed during
likely to recur. (p. 244)
the period after reinforcement under FR or FI sched-
ules, in which the probability of a reinforcer is zero. The temporal relation between a response and the
Azrin, Hutchinson, and Hake (1966; see also Kup- reinforcer that follows has been a hallmark of the
fer, Allen, & Malagodi, 2008) found that pigeons reinforcement process. Skinner’s (1948) “supersti-
attack conspecifics when a previously reinforced key tion” demonstration underscored the importance of
peck response is extinguished. Another example of this relation by suggesting that even in the absence
the generative effects of extinction is resurgence. If a of a programmed consequence of responding, the
response is reinforced and then extinguished while a environment will strengthen whatever response
second response is concurrently reinforced, extin- occurs contiguously with the reinforcer. Ferster and
guishing that second response leads to a resurgence Skinner (1957) frequently assigned response–
of the first response. The effect occurs whether the reinforcer temporal contiguity a primary role in
second response is or is not extinguished before accounting for the effects of reinforcement
concurrently reinforcing the second, and the effect schedules.
depends on parameters of both the first and the sec- To say that all were not content with response–
ond reinforced response (e.g., Bruzek, Thompson, reinforcer temporal contiguity as the central mecha-
& Peters, 2009; Lieving & Lattal, 2003). nism for reinforcement is an understatement. In a
seminal experiment, Herrnstein and Hineline
Frameworks (1966) exposed rats to a schedule consisting of two
Different frameworks have evolved that summarize response-independent shock distributions, one fre-
and integrate the empirical findings deriving from quent, the other less so. Each session started in the
41
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
frequent-shock distribution, and a lever press consumption, or use. Premack (1959) proposed
shifted the distribution of shocks to the leaner one, reinforcement to be access to a (relatively) preferred
at which the rat remained until a shock was deliv- activity, such as eating. For Premack, the first step
ered. At this point, the frequent shock distribution in assessing reinforcement was to create a prefer-
was reinstated and remained in effect until the next ence hierarchy. Next, highly preferred activities were
response, at which point the above-described cycle restricted and made accessible contingent on engage-
repeated. Responses did not eliminate shocks; they ment in a nonpreferred activity, with the outcome that
only reduced shock frequency. Furthermore, the low-probability response increased in frequency.
because shocks were distributed randomly in time, Timberlake and Allison (1974) suggested that the
temporal discriminations were precluded. Herrn- Premack principle was a corollary of a more general
stein and Hineline reasoned that if responding were response deprivation principle whereby any response
maintained under this schedule, it would be because constrained below its baseline level can function
of an aggregated effect of reductions in shock fre- as a reinforcer for another response that allows the
quency over noninstantaneous time periods (cf. constrained response to rise to its baseline level.
Sidman, 1966). Consistent with Herrnstein and Premack’s analysis foreshadowed other conceptualiza-
Hineline’s findings, Baum’s (1973) description of the tions of reinforcement contingencies in terms of con-
correlation-base law of effect remains a cogent sum- straints on behavioral output (e.g., Staddon, 1979).
mary of a molar framework for reinforcement (see
Choice and matching. Perhaps the contribution to
also Baum, 1989; Williams [1983] critiqued the cor-
modern behavior analysis with the greatest impact is
relational framework).
the matching law (Herrnstein, 1970; see Chapter 10,
TEAB makes frequent reference to level of analy-
this volume). The matching law is both a summary
sis. This phrase refers to both the description of data
of a number of empirical reinforcement effects and a
and the framework for accounting for those data.
framework for integrating those effects. Herrnstein’s
Molecular descriptions are of individual or groups of
(1961) original proposal was a simple quantitative
responses, often in relation to reinforcement. Molar
statement that relative responding is distributed
descriptions are of aggregated responses across time
proportionally among concurrently available alter-
and the allocation of time to differing activities.
natives as a function of the relative reinforcement
Molecular accounts of reinforcement effects empha-
proportions associated with each alternative. He
size the role of events occurring at the time of rein-
and others thereafter developed it into its more gen-
forcement (e.g., Peele, Casey, & Silberberg, 1984),
eralized form, expressed by Baum (1974; see also
and molar accounts emphasize the role of aggregated
Staddon, 1968; McDowell, 1989) as
effects of reinforcers integrated over noninstanta-
r 
a
neous time periods (e.g., Baum, 1989). Proponents R1
of each framework have at various times claimed pri- = b 1  ,
R2  r2 
macy in accounting for the effects of reinforcement,
but the isolation of an irrefutable single mechanism where R1 and R2 are response rates to the two alter-
at one level or another seems remote. The issue is natives and r1 and r2 are reinforcement rates associ-
not unlike others concerning levels of analysis in ated with those alternatives. The parameters a and b
other disciplines, for example, punctuated equilib- are indices of, respectively, sensitivity (also some-
rium versus continuous evolution and wave and par- times labeled the discriminability of the alternatives)
ticle theories of light. The “resolution” of the molar and bias (e.g., a preexisting preference for one oper-
versus molecular issue may ultimately be pragmatic: andum over the other). When restated in logarith-
The appropriate level is that at which behavior is mic form, a and b describe the slope and intercept of
predicted and controlled for the purposes at hand. a straight line fitted to the plot of the two ratios on
either axis of a graph. A rather typical, but not uni-
Relational reinforcement theory. Reinforcers versal, finding in many experiments is undermatching,
necessarily involve activities related to their access, that is, preferences for the richer alternative are less
42
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
than is predicted by a strict proportionality between reinforcement occurs only if the target response does
response and reinforcement ratios. This undermatch- not occur for the specified period.
ing and overmatching (a greater-than-predicted pref- Nevin (1974; see Volume 2, Chapter 5, this
erence for the richer alternative, and bias) were the handbook) conceptualized response strength as
impetus for developing the generalized matching law. resistance to change of a response when competing
One of Herrnstein’s (1970) insights was that all reinforcement contingencies impinge on that
behavior should be considered in the framework of response. The relation between response strength
choice. Even when only one response alternative is and resistance to change originated early in the psy-
being measured, there is a choice between that chology of learning (e.g., Hull, 1943), but Nevin
response and engaging in other behavior. This expanded it to schedule-maintained responding. He
observation led to two further conclusions: The total arranged multiple schedules of reinforcement in
amount of behavior in a situation is constant or which the parameters of the reinforcer—rate, mag-
fixed (but see McDowell, 1986); it is simply distrib- nitude, and delay, in different experiments—differed
uted differently depending on the circumstances, in the two components. The reinforcer maintaining
and there are unmeasured sources of reinforcement the response was made less effective by, in different
(originally labeled ro, but later re). Thus, deviations conditions, removing it (extinction), prefeeding the
from the strict proportionality rule, such as under- food-restricted animals (satiation), or providing
matching, are taken by some to reflect changes in re response-independent reinforcers at different rates
as well as perhaps bias or sensitivity changes. during the chamber blackout that separated the two
Davison and Tustin (1978) considered the components. Each of these disrupting operations
matching law in the broader context of decision the- had similar disruptive (response rate–lowering)
ory, in particular, signal detection theory (D. M. effects such that the responding maintained by more
Green & Swets, 1966). Originally developed to dis- frequent, larger, or less delayed reinforcers was less
tinguish sensitivity and bias effects in psychophysi- reduced than was responding in the other compo-
cal data, Davison and Tustin used signal detection nent in which the reinforcers were less frequent,
theory to describe the discriminative function of shorter, or more delayed.
reinforcement in maintaining operant behavior. In considering the resistance of behavior to
Their analysis thus describes the mathematical sepa- change, Nevin, Mandell, and Atak (1983) proposed
ration of the biasing and discriminative stimulus a behavioral analogy to physical momentum, that is,
effects of the reinforcer. More generally, the analy- the product of mass and velocity:
sis of choice has been extended across TEAB, from
When responding occurs at the same rate
simple reinforcement schedules to foraging, social
in two different schedule components,
behavior, and applied behavior analysis (see
but one is less affected by an external
Volume 2, Chapter 7, this handbook).
variable than is the other, we suggest that
the performance exhibiting greater resis-
Response strength and behavioral momentum.
tance to change be construed as having
Reinforcement makes a response more likely, and
greater mass. (p. 50)
this increase in response probability or rate has con-
ventionally been taken by many as evidence of the Nevin et al. have shown that reinforcement rate,
strength of that response. One difficulty with such regardless of the contingency between responses
a view of response strength is that response rate is and reinforcers, is a primary determinant of momen-
not determined simply by whether the response has tum: More frequently reinforced responses are more
been reinforced but by how the contingencies by resistant to change. For example, Nevin, Tota,
which the response is reinforced are arranged. Thus, Torquato, and Shull (1990, Experiment 1) found
the absence of the target response may index its that responding maintained by a combination of
strength in the case of a differential-reinforcement- response-dependent and response-independent food
of-other-behavior (DRO) schedule, in which deliveries (a VI schedule + a VT schedule) was
43
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
more resistant to disruption than was responding directly compared the effects of reinforcement rate
maintained by response-dependent food delivery on VI schedule performance in open and closed
only. This was the case because the reinforcement economies. In the open economy, sessions were ter-
rate in the VI + VT component was higher, even minated before the pigeon earned its daily food
though response rate was lower in this component. allotment; postsession feeding was provided so the
Nevin et al. (1990) observed that “as a result of Pav- animal was maintained at a target weight. In the
lovian contingencies, nonspecific effects underlying closed economy, the pigeons earned all of their food
Pavlovian contingencies resulting from the associa- by key pecking. The functions relating response rate
tion of discriminative stimuli with different rates of to reinforcement rate differed for the two economic
reinforcement may have nonspecific effects that contexts. In the open economy, response rates
‘arouse or motivate operant behavior’” (p. 374). decreased with decreasing reinforcement rate,
Another consideration in behavioral momentum whereas in the closed economy, response rates
may be response rate. Different reinforcement rates increased with decreasing reinforcement rate. Such
result in different response rates. When reinforce- findings suggest that the functional relations that
ment rates are held constant and response rates are obtain between reinforcement parameters and
varied, the lower response rates are often more resis- behavior are not universal, but depend on the con-
tant to change (Blackman, 1968; Lattal, 1989). text in which they occur.
Consumption also differs as a function of the
Behavioral economics. Skinner (1953) observed reinforcer context. The interaction between rein-
that “statements about goods, money, prices, wages, forcers lies on a continuum. At one extreme, one
and so on, are often made without mentioning reinforcer is just as effective as another in maintain-
human behavior directly, and many important gen- ing behavior (perfect substitutes). At the other
eralizations in economics appear to be relatively extreme, reinforcers do not substitute for one
independent of the behavior of the individual” another at all. Instead, as consumption of one
(p. 398). The chasm described by Skinner has long increases (decreases) with price changes, so does the
since been bridged, both in TEAB (Madden, 2000; other despite its price being unchanged. Such a rela-
see Volume 2, Chapter 8, this handbook) and in the tion reveals that the reinforcers function as perfect
discipline of economics. This bridging has come complements. Between the extremes, reinforcers
about through the mutual concern of the two disci- substitute for one another to varying degrees (for a
plines with the behavior of consumption of goods review, see L. Green & Freed, 1998).
and services as a function of environmental circum- Behavioral–economic analyses have shown that
stances. The behavioral–economic framework has qualitatively different reinforcers may vary differen-
been a useful heuristic for generating experimental tially in the extent to which they sustain behavior in
analyses that have expanded the understanding of the context of different environmental challenges,
reinforcement in several ways. often expressed as cost in economic analyses. Some
In a token economy on a psychiatric ward reinforcers continue to sustain behavior even as
(Allyon & Azrin, 1968), for example, consumption cost, measured, for example, by the number of
of a nominal reinforcer was reduced if the same item responses required for reinforcement, increases, and
was available at a lower cost, or no cost, elsewhere. the behavior sustained by others diminishes with
Thus, for example, if visitors brought desirable food such increased cost. This difference in response sus-
items onto the ward from outside, demand for those tainability across increasing cost distinguishes
food items within the token economy diminished. inelastic (fixed sustainability regardless of cost)
Hursh (1980) captured the essential features of this from elastic (sustainability varies with cost)
scenario by distinguishing open economies, in which reinforcers.
items are available from multiple sources, from Other ways of increasing the cost of a reinforcer
closed economies, in which those items are only are by increasing the delay between the reinforcer
available in a defined context. Hall and Lattal (1990) and the response that produced it or by reinforcing a
44
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
response with decreasing probability. These two dynamics.” The imposed contingency is not nec-
techniques of changing reinforcer cost have been essarily the effective one because that imposed
combined with different magnitudes of reinforce- contingency constantly interacts with responding,
ment to yield what first was called a self-control par- and the resulting dynamic is what is ultimately
adigm (Rachlin & Green, 1972) but now generally responsible for behavioral modulation and control.
is described as delay (or probability, as appropriate) This distinction was captured by Zeiler (1977b),
discounting. A choice is arranged between a small or who distinguished between direct and indirect
a large (or a less or more probable) reinforcer, deliv- variables operating in reinforcement schedules. In
ered immediately after the choice response. Not sur- a ratio schedule, for example, the response require-
prisingly, the larger (or more probable) reinforcer ment, n, is a direct, specified variable (e.g., as in FR
almost always is selected. Next, a delay is imposed n), but responding takes time, so as one varies the
between the response and delivery of the larger rein- ratio requirement, one is also indirectly varying the
forcer (or the probability of the larger reinforcer is time between reinforcer deliveries. Another way of
decreased), and the magnitude of the small, immedi- expressing this relation is by the feedback function
ate (or small but sure thing) reinforcer is varied sys- of a ratio schedule: Reinforcement rate is determined
tematically until an indifference point is reached at directly by response rate. The feedback function is a
which either choice is equally as likely (e.g., Mazur, quantitative description of the contingency specify-
1986; Richards, Mitchell, de Wit, & Seiden, 1997). ing the dynamic interplay between responding and
Using the delay or probability discounting proce- reinforcement.
dure, a function can be created relating indifference Using the methods developed by quantitative
points to the changing cost. Steep discounting of analysis of dynamical systems, a few behavior ana-
delayed reinforcers correlates with addictions such lysts have explored some properties of reinforce-
as substance use disorder and pathological gambling ment contingencies (see Marr, 1992). For example,
(see Madden & Bickel, 2010, for a review). in an elegant analysis Palya (1992) examined the
A final, cautionary note that applies to economic dynamic structure among successive IRTs in interval
concepts and terms as well as more generally to ver- schedules, and Hoyert (1992) applied nonlinear
bal labels commonly used in TEAB, such as contrast dynamical systems (chaos) theory in an attempt to
or even reinforcement: Terms such as elastic or describe the cyclical interval-to-interval changes
inelastic or complementary or substitutable are in response output that characterize steady-state
descriptive labels, not explanations. Reinforcers do FI schedule performance. Indeed, much of the
not have their effects because they are inelastic, sub- response variability that is often regarded as a nui-
stitutable, or discounted. sance to be controlled (e.g., Sidman, 1960) may,
from a dynamic systems perspective, be the inevita-
Behavior dynamics. The emphasis in TEAB on ble outcome of the dynamic nature of any reinforce-
steady-state performance sometimes obscures the ment contingency.
centrality of the environment–behavior dynamic
that characterizes virtually every contingency of Reinforcement in biological context. Skinner,
reinforcement. Dynamics implies change, and (1981; see also Donahoe, 2003; Staddon &
change is most immediately apparent when behavior Simmelhag, 1971) noted the parallels between the
is in transition, as in the acquisition of a response selection of traits in evolutionary time and the
previously in the repertoire in only primitive form, selection of behavior over the organism’s lifetime
transitions from one set of reinforcement conditions (i.e., ontogeny). Phylogeny underlies the selection
to another, or behavioral change from reinforcement of behavior by reinforcement in at least two ways.
to extinction. Steady-state performance, however, First, certain kinds of events may come to func-
also reveals a dynamic system. As Marr (personal tion as reinforcers or punishers in part because
communication, November 2010) observed, “[All] of phylogeny. Food, water, and drugs of various
contingencies engender and manifest systems sorts, for example, may function as reinforcers at
45
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
least in part because of their relation to the organ- fact that natural selection works on what it has to
ism’s evolved physiology. It is both retrograde and work with, not on ideals” (p. 420) and
false, however, to suggest that reinforcers reduce
optimizing means to do the best conceiv-
to physiological needs. Reinforcers have been more
able. However, natural selection need
productively viewed functionally; however, the
not maximize returns. . . . What selec-
organism’s phylogeny certainly cannot be ignored
tion must do is follow a satisficing prin-
in discussions of reinforcement (see also Breland &
ciple. . . . To satisfice means to do well
Breland, 1961). Second, the mere fact that organ-
enough to get by, not necessarily to do
ism’s behavior is determined to a considerable
the best possible. (p. 420)
extent by consequences is prima facie evidence
of evolutionary processes at work. Thus, Skinner Zeiler thus concluded that optimization in fact may
(1981) proposed that some responses are selected be rare in natural settings. He distinguished evolu-
(reinforced) and therefore tend to recur, and those tionary and immediate function of behavior, noting
that are not selected or are selected against tend that the former relates to the fitness enhancement of
to disappear from the repertoire. The research of behavior and the latter to its more immediate effects.
Neuringer (2002; see Chapter 22, this volume) In his view, optimal foraging theory errs in using the
on variability as an operant sheds additional light methods of immediate function to address questions
on the interplay between behavioral variation and of evolutionary function.
behavioral selection.
The analysis of foraging also has been a focal
Pillar 3: Punishment
point of research attempting to place reinforce-
ment in biological perspective. Foraging, whether An outcome of some interactions between an organ-
it is for food, mates, or other commodities such as ism’s responses and environmental events is that the
new spring dresses, involves choice. Lea (1979; responses become less likely than they would be in
see also Fantino, 1991) described an operant the absence of those events. As with reinforcement,
model for foraging based on concurrent chained this outcome is particularly effective when there is a
schedules (see Response-Dependent Stimuli Cor- dependency between such environmental events
related With Previously Established Reinforcers and behavior, a two-term contingency involving
section later in this chapter) in which foraging is responding and what comes to function as a pun-
viewed as consisting of several elements, begin- isher. Such a process of punishment constitutes the
ning with search and ending in consumption third pillar of TEAB.
(including buying the new spring dress). Such an
approach holds out the possibility of integrating Positive Punishment
ecology and TEAB (Fantino, 1991). Positive punishment is the suppression or elimination
A related integrative approach is found in paral- of a response resulting from the response-dependent,
lels between foraging in natural environments and time-limited presentation of a stimulus or event (i.e.,
choice as described by the matching law and its vari- a negative reinforcer). In the laboratory, electric
ants (see Choice and Matching section earlier in this shock is a prototypical positive punisher because, at
chapter). Optimal foraging theory, for example, pos- the parameters used, it produces no injury to the
its that organisms select those alternatives in such a organism, it is precisely initiated and terminated,
way that costs and benefits of the alternatives are and it is easily specified in physical terms (e.g., its
weighed in determining choices (as contrasted, e.g., intensity, frequency, and duration). Furthermore,
with maximizing theory, which posits that choices parameters of shock can be selected that minimize
are made such that reinforcement opportunities are sensitization (overreactivity to a stimulus) and habit-
maximum). Optimal foraging theory is not, how- uation (adaptation or underreactivity to a stimulus).
ever, without its critics. Zeiler (1992), for example, Punishment always is investigated in the context
has suggested that “optimality theory ignores the of reinforcement because responding must be
46
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
maintained before it can be punished. As a result, responding at the same rate. Punishers that are more
the effects of punishers always are relative to the intense (e.g., higher amperage in the case of electric
prevailing reinforcement conditions. Perhaps the shock) and more frequent have greater suppressive
most important of these is the schedule of reinforce- effects, assuming the conditions of reinforcement
ment. Punishment exaggerates postreinforcement are held constant (see Azrin & Holz, 1966, for a
pausing on FR and FI schedules, and it decreases review). The effects of punisher duration are com-
response rates on VI schedules (Azrin & Holz, plicated by the fact that longer duration punishers
1966). Because responding on DRL schedules is rel- may adventitiously reinforce responses contiguous
atively inefficient (i.e., responses are frequently with their offset, thereby potentially confounding
made before the IRT > t criterion has elapsed), by the effect of the response-dependent presentation of
suppressing responding punishment actually the punisher.
increases the rate of reinforcement. Even so, pigeons
will escape from punishment of DRL responding to Negative Punishment
a situation in which the DRL schedule is in effect Negative punishment is the suppression or elimina-
without punishment (Azrin, Hake, Holz, & tion of a response resulting from the response-
Hutchinson, 1965). Although most investigations of dependent, time-limited removal of a stimulus or
punishment have been conducted using baselines event. Both negative punishment and conventional
involving positive reinforcement, negative reinforce- extinction involve removing the opportunity for
ment schedules also are effective baselines for the reinforcement. Negative punishment differs from
study of punishment (e.g., Lattal & Griffin, 1972). extinction in three critical ways. In conventional
Punishment effects vary as a function of parame- extinction, the removal of the opportunity for rein-
ters of both the reinforcer and the punisher. With forcement occurs independently of responding, is
respect to reinforcement, punishment is less effec- relatively permanent (or at least indefinite), and is
tive when the organism is more deprived of the rein- not correlated with a stimulus change. In negative
forcer (Azrin, Holz, & Hake, 1963). The effects of punishment, the removal of the opportunity for
reinforcement rate on the efficacy of punishment are reinforcement is response dependent, time limited,
less clear. Church and Raymond (1967) reported and sometimes (but not necessarily) correlated with
that punishment efficacy increased as the rate of a distinct stimulus. These latter three characteristics
reinforcement decreased. When, however, Holz are shared by three procedures: DRO schedules
(1968) punished responding on each of two concur- (sometimes also called differential reinforcement of
rently available VI schedules arranging different pausing or omission training), timeout from positive
rates of reinforcement, the functions relating pun- reinforcement, and response cost.
ishment intensity and the percentage of response Under a DRO schedule, reinforcers depend on the
reduction from a no-punishment baseline were vir- nonoccurrence of the target response for a predeter-
tually identical. Holz’s results suggest a similar rela- mined interval. Responses during the interreinforcer
tive effect of punishment independent of the rate of interval produce no stimulus change, but each
reinforcement. Perhaps other tests, such as those response resets the interreinforcer interval. Despite
suggested by behavioral momentum theory (see the label of reinforcement, the response-dependent,
Response Strength and Behavioral Momentum sec- time-limited removal of the opportunity for rein-
tion earlier in this chapter) could prove useful in forcement typically results in substantial, if not total,
resolving these seemingly different results. response suppression, that is, punishment. With
Parameters of punishment include its immediacy DROs, both the amount of time that each response
with respect to the target response, intensity, dura- delays the reinforcer and the time between successive
tion, and frequency. Azrin (1956) showed that pun- reinforcers in the absence of intervening responding
ishers dependent on a response were more can be varied. Neither parameter seems to make
suppressive of responding than were otherwise much difference once responding is reduced. They
equivalent punishers delivered independently of may, however, affect the speed with which responding
47
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
is reduced and the recovery after termination of the timeout, can entail a concurrent loss of reinforce-
contingency (Uhl & Garcia, 1969). ment as responding is suppressed. Pietras and Hack-
As with other punishment procedures, a DRO enberg (2005), however, showed that response cost
contingency may be superimposed on a reinforce- has a direct suppressive effect on responding, inde-
ment schedule maintaining responding (Zeiler, pendent of changes in reinforcement rate.
1976, 1977a), allowing examination of the effects of
punishers, positive or negative, on steady-state Frameworks
responding. Lattal and Boyer (1980), for example, Punishment has been conceptualized by different
reinforced key pecking according to an FI 5-min investigators as either a primary or a secondary
schedule. At the same time, reinforcers were avail- process.
able according to a VI schedule for pauses in pecking
Punishment as a primary or direct process.
of x−s or more. Pecking thus postponed any rein-
Thorndike (1911) proposed that punishment effects
forcers that were made available under the VI sched-
are equivalent and parallel to those of reinforce-
ule. No systematic relation was obtained between
ment but opposite in direction. Thus, the response-
required pause duration and response rate. With a
strengthening effects defining reinforcement were
constant 5-s pause required for reinforcement of not
mirror-image effects of the response-suppressing
pecking, however, the rate of key pecking was a neg-
effects defining punishment. Schuster and Rachlin
ative function of the frequency of DRO reinforce-
(1968) suggested three examples: (a) the suppres-
ment. That is, the more often a key peck postponed
sive and facilitative effects of following a conditioned
food delivery, the lower the response rates were and
stimulus (CS) with, respectively, shock or food;
thus the greater the punishment effect was.
(b) mirror-image stimulus generalization gradients
Timeouts are similar to DROs in that, when used
around stimuli associated with reinforcement and
as punishers, they occur as response-dependent, rel-
punishment; and (c) similar indifference when
atively short-term periods of nonreinforcement.
given a choice of response-dependent and response-
They differ from DROs because the periods of non-
independent food or between response-dependent
reinforcement are not necessarily resetting with suc-
shock and response-independent shock (Shuster
cessive responses, and they are accompanied by a
& Rachlin, 1968). Although (c) has held up to
stimulus change. Timeout effects are relative to the
experimental analysis (Brinker & Treadway, 1975;
prevailing conditions of reinforcement. As was
Moore & Fantino, 1975; Schuster & Rachlin, 1968),
described earlier, periods of timeout from negative
(a) and (b) have proven more difficult to confirm.
reinforcement function as reinforcers, as do periods
For example, precise comparisons of generalization
of timeout from extinction (e.g., Azrin, 1961).
gradients based on punishment and reinforcement
Timeouts from situations correlated with reinforce-
are challenging to interpret because of the complexi-
ment, however, suppress responding when they are
ties of equating the food and shock stimuli on which
response dependent (Kaufman & Baron, 1968).
the gradients are based. The research described in
With response cost, each response or some por-
the Response-Independent Stimuli Correlated With
tion of responses, depending on the punishment
Reinforcers and Punishers section later in this chap-
schedule, results in the immediate loss of reinforc-
ter illustrates the complexities of interpreting (a).
ers, or some portion of reinforcers. Response cost is
most commonly used in laboratory and applied set- Punishment as a secondary or indirect process.
tings in which humans earn points or tokens accord- Considered a secondary process, the response sup-
ing to some schedule of reinforcement. Weiner pression obtained when responding is punished
(1962), for example, subtracted one previously comes about indirectly as the result of negative rein-
earned point for each response made by adult forcement of other responses. Thus, punishment is
human participants earning points by responding interpreted as a two-stage (factor) process whereby,
under VI schedules. The effect was considerable first, the stimulus becomes aversive and, second,
suppression of responding. Response cost, similar to responses that result in its avoidance are then
48
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
negatively reinforced. Hence, as unpunished r einforcement, onto which the stimuli and their cor-
responses are reinforced because they escape or related events are superimposed. In the first such
avoid punishers, punished ones decrease, resulting study, Estes and Skinner (1941) trained rats’ lever
in what appears as target-response suppression pressing on an FI food reinforcement schedule and
(cf. Arbuckle & Lattal, 1987). periodically imposed a 3-min tone (a warning stimu-
A variation of punishment as a secondary process lus or conditional stimulus [CS] followed by a brief
is the competitive suppressive view that responding electric shock). Both the CS and the shock occurred
decreases because punishment degrades or devalues independently of responding, and the FI continued
the reinforcer (e.g., Deluty, 1976). Thus, the sup- to operate during the CS (otherwise, the response
pressive effect of punishment is seen as an indirect would simply extinguish during the CS). Lever
effect of a less potent reinforcer for punished pressing was suppressed during the tone relative to
responses, thereby increasing the potency of rein- no-tone periods. This conditioned suppression effect
forcers for nonpunished responses. Contrary to occurs under a variety of parameters of the rein-
Deluty (1976), Farley’s (1980) results, however, sup- forcement schedule, the stimulus at the end of the
ported a direct suppressive interpretation of punish- warning stimulus, and the warning stimulus itself
ment. Critchfield, Paletz, MacAleese, and Newland (see Blackman, 1977, for a review).
(2003) compared the direct and competitive suppres- When warning stimuli that precede an unavoid-
sion interpretations of punishment. Using human able shock are superimposed during avoidance-
subjects in a task in which responding was reinforced maintained responding (see the earlier Avoidance
with points and punished by the loss of a portion of section), responses during the CS may either
those same points, a quantitative model based on the increase or decrease relative to those during the
direct suppression interpretation yielded better fits to no-CS periods. The effect appears to depend on
the data. Furthermore, Rasmussen and Newland whether the shock at the end of the CS period is dis-
(2008) suggested that the negative law of effect may criminable from those used to maintain avoidance.
not be symmetrical. Using a procedure similar to If the same shocks are used, responding is facilitated
Critchfield et al.’s, they showed that single punishers during the CS; if the shocks are distinct, the out-
subtract more value than single reinforcers add. come is often suppression during the warning stim-
ulus (Blackman, 1977).
A similar arrangement has been studied with
Pillar 4: Control by Stimuli
positive reinforcement–maintained responding, in
Correlated with Reinforcers
which a reinforcer is delivered instead of a shock at
and Punishers
the end of the CS. The effects of reinforcers at the
Reinforcers and punishers often are presented in the end of the CS that are the same as or different from
context of other stimuli that are initially without dis- that arranged by the baseline schedule have been
cernable effect on behavior. Over time and with con- investigated. Azrin and Hake (1969) labeled this
tinued correlation with established reinforcers or procedure positive conditioned suppression when they
punishers, these other events come to have behavioral found that VI-maintained lever pressing of rats dur-
effects similar to the events with which they have been ing the CS was generally suppressed relative to the
correlated. Such behavioral control by these other no-CS periods. Similar suppression occurred
events is what places them as the fourth pillar of TEAB. whether the event at the end of the CS was the same
as or different from the reinforcer used to maintain
Response-Independent Stimuli responding. LoLordo (1971), however, found that
Correlated With Previously Established pigeons’ responding increased when the CS ended
Reinforcers and Punishers with the same reinforcer arranged by the back-
In the typical operant arrangement for studying ground schedule, a result he related to autoshaping
the effects of conditioned stimuli, responding is (Brown & Jenkins, 1968). Facilitation or suppres-
maintained according to some schedule of sion during a CS followed by a positive reinforcer
49
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
seems to depend on both CS duration and the egative punishment is of particular interest
n
schedule of reinforcement. For example, Kelly because it occurred despite the fact that stimuli
(1973) observed both suppression and facilitation preceding timeouts typically facilitate rather than
during a CS as a function of whether the baseline suppress responding. Thus, the results cannot be
schedule was DRL or VR. interpreted in terms of the red key light serving as a
discriminative stimulus for a lower rate of reinforce-
Response-Dependent Stimuli Correlated ment (as they could be in Hake and Azrin’s
With Previously Established Punishers experiment).
Despite being commonplace in everyday life, condi-
tioned punishment has only rarely been studied in Response-Dependent Stimuli Correlated
the laboratory. In an investigation of positive condi- With Previously Established Reinforcers
tioned punishment, Hake and Azrin (1965) first Stimuli correlated with an already-established rein-
established responding on a VI 120-s schedule of forcer maintain responses that produce them (Wil-
positive reinforcement in the presence of a white liams, 1994). This is the case for stimuli correlated
key light. The key-light color irregularly changed with either the termination or postponement of a
from white to red for 30 s. Reinforcement continued negative reinforcer (Siegel & Milby, 1969) or with
to be arranged according to the VI 120-s schedule dur- the onset of a positive reinforcer. In the latter case,
ing the red key light; however, at the end of the red- early research on conditioned reinforcement used
key-light period, a 500-ms electric shock was chained schedules (e.g., Kelleher & Gollub, 1962),
delivered independently of responding. This shock higher order schedules (Kelleher, 1966), and a two–
suppressed but did not eliminate responding when response-key procedure (Zimmerman, 1969). Inter-
the key light was red. To this conditioned suppres- pretational limitations of each of these methods
sion procedure, Hake and Azrin then added the fol- (Williams, 1994) led to the use of two other proce-
lowing contingency. When the white key light was dures in the analysis of conditioned reinforcement.
on, each response produced a 500-ms flash of the The observing–response procedure (Wyckoff,
red key light. Responding in the presence of the 1952) involves first establishing a discrimination
white key light was suppressed. When white-key- between two stimuli by using a multiple schedule in
light responses produced a 500-ms flash of a green which one stimulus is correlated with a schedule of
key light, when green previously had not been reinforcement and the other with extinction (or a
paired with shock, responding during the white key schedule arranging a different reinforcement rate).
light was not suppressed. In addition, the degree of Once discriminative control is established, a third
response suppression was a function of the intensity stimulus is correlated with both components to
of the shock after the red key light. yield a mixed schedule. An observing response on a
A parallel demonstration of negative conditioned second operandum converts the mixed schedule to
punishment based on stimuli correlated with the a multiple schedule for a short interval (typically
onset of a timeout was conducted by Gibson (1968). 10–30 s with pigeons). Observing responses are
Gibson used Hake and Azrin’s (1965) procedure, maintained on the second operandum. These
except that instead of terminating the red key light responses, of nonhumans at least, are maintained
with an electric shock, it terminated with a timeout primarily, if not exclusively, by the positive stimulus
during which the chamber was dark and reinforce- (S+) and not by the stimulus correlated with extinc-
ment was precluded. The effect was to facilitate key tion (see Fantino & Silberberg, 2010; Perone &
pecking by pigeons during the red key light relative Baron, 1980; for an alternative interpretation of the
to rates during the white key light. When, however, role of the negative stimulus [or S−], see Escobar &
responses during the white key light produced Bruner, 2009). The general interpretation has been
500-ms flashes of the red key light, as in Hake and that the stimulus correlated with reinforcement
Azrin, responding during the white key light was functions as a conditioned reinforcer maintaining
suppressed. This demonstration of conditioned the observing response.
50
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Fantino (1969, 1977) proposed that stimuli func- the operandum leading to the VI 1-min terminal
tion as conditioned reinforcers to the extent that link predicts an exclusive preference for this alter-
they represent a reduction in the delay (time) to native, a prediction confirmed by experimental
reinforcement relative to the delay operative in their analysis.
absence. Consider the observing procedure described Before leaving the topic of conditioned reinforce-
earlier, assuming that the two components alternate ment, it should be noted that there is not uniform
randomly every 2 min. Observing responses con- agreement as to the significance of conditioned rein-
vert, for 15-s periods, a mixed VI 2-min extinction forcement (and, by extrapolation, conditioned pun-
schedule to a multiple VI 2-min extinction schedule. ishment) as a concept. Although it has strong
Because components are 2 min each, the mixed- proponents (Dinsmoor, 1983; Fantino, 1977; Wil-
schedule stimulus is correlated with a 4-min period liams, 1994), among its critics are Davison and
(delay) between successive reinforcers. In the pres- Baum (2006), who suggested that the concept had
ence of the stimulus correlated with the VI sched- outlived its usefulness and that conditioned rein-
ule, the delay between reinforcers is 2 min, a 50% forcement is more usefully considered in terms of
reduction in delay time relative to that in the discriminative stimulus control. This suggestion
mixed-schedule stimulus. Fantino’s delay reduction harkens back to an earlier one that conditioned rein-
hypothesis asserts that this signaled reduction in forcers must first be established as discriminative
delay to reinforcement maintains observing stimuli (e.g., Keller & Schoenfeld, 1950), but Davi-
responses. son and Baum called for the abandonment of the
In other tests of the delay reduction hypothesis, concept (see also Chapter 17, this volume).
concurrent chained schedules have been used (in a
chained schedule, distinct stimuli accompany the
Pillar 5: Contextual and Stimulus
different links). In such tests, two concurrently
Control
available identical VI schedules serve as the initial
links of the chained schedules. The equivalent VI Interactions between responding and reinforcers
initial links ensure that either terminal link is and punishers occur in broader environmental con-
accessed approximately equally as often. The termi- texts, both distal or historical and proximal or con-
nal links are mutually exclusive: The first initial temporary. These contexts define what is sometimes
link requirement met leads to its terminal link and called antecedent control of behavior. The term is
simultaneously cancels the alternative chained something of a misnomer because under such cir-
schedule for that cycle. When the terminal link cumstances, the behavior is controlled by the two-
requirement is met, the response is reinforced, and term contingency in the context of the third,
the concurrent initial links recur. Thus, for exam- antecedent event. Such joint control by reinforce-
ple, if the two terminal links are VI 1 min and VI 5 ment contingencies in context is what constitutes
min and the initial links are both VI 1 min, the the final pillar of TEAB.
average time to reinforcement achieved by respond-
ing on both alternatives is 3.5 min (0.5 min in the Behavioral History
initial link + 3 min in the terminal link). Responding Perhaps the broadest context for the two-term con-
exclusively on the operandum leading to the VI 1-min tingency is historical. Although the research is not
terminal link produces a reinforcer on average every extensive, several experiments have examined with
2 min, a reinforcement delay reduction of 1.5 min some precision functional relations between past
from the average for responding on both. Respond- experiences and present behavior. These analyses
ing exclusively on the operandum leading to the VI began with the work of Weiner (e.g., 1969), who
5-min terminal link produces a reinforcer once every showed that different individuals responded differ-
6 min, yielding a reinforcement delay increase of ently on contemporary reinforcement schedules as
2.5 min relative to the average for responding on a function of their previous experience responding
both. The greater delay reduction for responding on on other schedules. Freeman and Lattal (1992)
51
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
investigated the effects of different histories under each response again was punished, but reinforce-
stimulus control within individual pigeons. The ment was not reinstated. Because of its prior correla-
pigeons were first trained on FR and DRL sched- tion with reinforcement, punishment functioned as
ules, equated for reinforcement rate and designed an S+, thereby resulting in considerable responding,
to generate disparate response rate, in the presence at least in the short term. This result underlines the
of distinct stimuli. When subsequently exposed to functional definition of discriminative stimuli: They
FI schedules in the presence of both stimuli, they are defined in terms of their effect, not by their form.
responded for many sessions at higher rates in the Two prerequisites for stimulus control are (a)
presence of the stimuli previously associated with that the stimuli be different from both the absence
the FR (high response rate) schedule. This effect of stimulation (i.e., above the absolute threshold)
not only replicated Weiner’s human research, but it and that they be discriminable from one another
showed within individual subjects that an organ- (i.e., above the difference threshold) and (b) that the
ism’s behavioral history could be controlled by the stimuli be correlated with different reinforcement or
stimuli with which that history was correlated. punishment contingencies. Thresholds are in part
Other experiments have elaborated such behavioral physiological and phylogenic. Human responding,
history effects. Ono (2004), for example, showed for example, cannot be brought under control of
how preferences for forced versus free choices were visual stimuli outside the range of physical detection
determined by the organism’s past experiences with of the human eye. Signal detection theory (D. M.
the two alternatives. Green & Swets, 1966) posits that the discriminable
dimension of a stimulus can be separated from the
Discriminative Stimulus Control reinforcement contingencies that bias choices in
By correlating distinct stimuli with different condi- assessments of threshold measurements. Research-
tions of reinforcement or punishment, responding ers in TEAB have used signal detection methods to
typically comes to be controlled by those stimuli. not only complement other methods of assessing
Such discriminative stimulus control occurs when control by conventional sensory modality stimuli
experimental variations in the stimuli lead to corre- (i.e., visual and auditory stimuli; cf. Nevin, 1969)
lated variations in behavior. Discriminative stimulus but also to isolate the discriminative and reinforcing
control can be established with both reinforcement properties of a host of other environmental events,
and punishment. including reinforcement contingencies themselves
The prototypical example of discriminative stim- (e.g., Davison & Tustin, 1978; Lattal, 1979).
ulus control is one in which responding is rein- Another issue that receives considerable attention
forced in the presence of one stimulus, the S+ or SD, in the analysis of stimulus control is attention (see
and not in the presence of another, the S− or SΔ. Chapter 17, this volume). Although from some per-
Stimulus control, however, can involve different spectives, attention is considered a prerequisite to
stimuli correlated with different conditions of rein- stimulus control, in TEAB attention is stimulus con-
forcement, in which case there would be two posi- trol (e.g., Ray, 1969). The behavioral index of atten-
tive stimuli, or it can involve conditions in which tion is whether the organism is responding to the
punishment is present or absent in the presence of nominal stimuli being presented. Thus, attending to
different stimuli. a stimulus means responding in its presence and not
The lack of overriding importance of the form of in its absence, and such differential responding also
the stimulus in establishing positive discriminative defines stimulus control. According to this analysis,
stimulus control was illustrated by Holz and Azrin an instructor does not get the class’s attention to
(1961). They first punished each of a pigeon’s key start the day’s activities; responding to the day’s
responses otherwise maintained by a VI schedule of activities is what having the class’s attention means.
reinforcement. Then, both punishment and rein- Correlating stimuli with different reinforcement
forcement were discontinued, allowing responding or punishment contingencies establishes discrimi-
to drop to low, but nonzero, levels. At that point, native stimulus control. It sometimes is labeled
52
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
discrimination, although care is taken in TEAB to test of stimulus generalization, typically conducted
ensure that the term describes an environment– in the absence of reinforcement, lines differing in
behavior relation and not an action initiated by the degree of tilt are presented in mixed order, and
organism. Discriminations have been established in responding to each is recorded. The result is a gradi-
two ways. The conventional technique is simply to ent, with responding relatively high in the presence
expose the organism to the discrimination task until of stimuli most like the S+ in training and lowest in
the behavior comes under the control of the differ- the presence of stimuli most like the S−. The shape
ent discriminative stimuli. Terrace (1963) reported of the gradient indexes stimulus generalization. A
differences in the number of responses made to a flat gradient suggests all stimuli are responded to
stimulus correlated with extinction (S−) as a func- similarly, that is, significant generalization or mini-
tion of how the discrimination was trained, specifi- mal discrimination. A steep gradient (that drops
cally, how the S− and correlated period of sharply between the S+ and the next-most-similar
nonreinforcement was introduced. The typical, sud- stimuli, e.g.) indicates that the stimuli differentially
den introduction of the S− after responding had control responding, that is, significant discrimina-
been well established in the presence of another tion or minimal generalization. The peak, that is, the
stimulus resulted in many (unreinforced) responses highest point, of the gradient, is often not at the
during the S− presentations, responses that Terrace original training stimulus but rather shifted to the
labeled as errors. Introducing the S− in a different next stimulus in the direction opposite the S−. This
way changes the behavior it controls. Terrace intro- peak shift, as it is labeled, has been suggested to
duced the S− simultaneously with the commence- reflect aversive properties of the S− in that it did not
ment of S+ training, but at low intensity (the occur when discriminations were trained without
S− was a colored light transilluminating the errors (Terrace, 1966).
response key, initially for very brief time periods. Stimulus generalization gradients also can be
Over successive sessions, both the intensity and the observed around the S−. Their assessment poses a
duration of the S− were increased gradually as a difficulty if the S+ and S− are on the same contin-
function of the pigeon’s behavior in the presence of uum: The gradients around both the S+ and the S−
the S−. This procedure yielded few responses to the are confounded by the fact that movement away
S− throughout training and during the steady-state from the S− constitutes movement toward the S+
S+–S− discriminative performance. Terrace (1966) and vice versa. The solution is to use as the S+ and
suggested that the S− functioned differently when S− orthogonal stimuli, that is, stimuli that are on
established with, as opposed to without, errors; different stimulus dimensions, for example, color
however, it was unclear whether the fading proce- and line tilt. This way, changes away from, for exam-
dure or the absence of responses to the S− was ple, the line tilt correlated with S−, are not changes
responsible for these differences. Subsequent toward the key color correlated with the S+. Inhibi-
research qualified some of Terrace’s suggestions tory generalization gradients typically are V shaped,
(e.g., Rilling, 1977). with the lowest responding in the presence of the
S− and increasing with increasing disparity between
Stimulus Generalization the test stimulus and the S−. These gradients, some-
Stimulus generalization refers to changes or grada- times labeled inhibitory generalization gradients, have
tions in responding as a function of changes or been interpreted to indicate that extinction, or non-
gradations in the stimulus with which the reinforce- reinforcement, involves the learning of other behav-
ment or punishment contingency originally was cor- ior rather than simply eliminating nonreinforced
related. In a typical procedure, a discrimination is responding (Hearst, Besley, & Farthing, 1970).
established (generalization gradients are more reli-
able when a discriminative training procedure is Conditional Stimulus Control
used) between S+ (e.g., a horizontal line projected The three-term contingency discussed in the preced-
on a response key) and S− (e.g., a vertical line). In a ing sections can itself be brought under stimulus
53
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
control, giving rise to a four-term contingency. of these gradients is a matter of interpretation.

Here, the stimuli defining the three-term contin- Those who are more cognitively oriented consider
gency are conditional on another, superordinate set the gradients to reflect changes in memory, whereas
of stimuli, defining the fourth term. Conditional those favoring a behavior-analytic interpretation
stimulus control has been studied widely using a generally describe them using action terms such as
procedure sometimes called matching to sample (see remembering or forgetting.
Sidman & Tailby, 1982). The procedure consists of The conditional discrimination procedure
three-element trials separated from one another by involving both delayed presentations of choice com-
an ITI. In a typical arrangement, a pigeon is con- ponents and the use of complex visual arrays as
fronted by a three–response-key array. In the pres- samples has given rise in part to the study of animal
ence of a preparatory stimulus, a response turns on a cognition, which has in turn led to often unfounded,
sample stimulus, say a red or green key light, each and sometimes inexplicable, speculations about cog-
with a probability of 0.5 on a given trial. A response nitive mechanisms underlying conditional stimulus
to the transilluminated key turns on the two side control (and other behavioral phenomena) in non-
stimuli (comparison component), one red and the human animals. The conceptual issues related to the
other green. A peck to the key colored the same as interpretation of behavioral processes involving
the sample stimulus results in food access for 3 s, stimulus control in terms of memory or other cogni-
whereas a peck to the other key terminates the trial. tive mechanisms is beyond the scope of this chapter.
After an intertrial interval, the cycle repeats. Thus, Branch (1977), Watkins (1990), and many others
the red and green lights in the comparison compo- have offered perspectives on these issues that are
nent can be either an S+ or an S− conditional on consistent with TEAB.
the stimulus in the sample component. The percent-
age of choices of colors corresponding to the sample Temporal Discriminative Stimulus
increases with exposure, reaching an asymptote near Control of Responding and Timing
100% correct. Variations on the procedure include Every schedule of reinforcement, positive or nega-
(a) turning off the sample light during the choice tive, involves time. It is a direct variable in interval
component (zero-delay matching), (b) using sample schedules and an indirect one in ratio schedules.
and choice stimuli that differ in dimension (sym- Early research on FI schedules suggested that the
bolic matching to sample), (c) using topographically passage of time functioned as an S+ (e.g., Dews,
different responses to the different sample stimuli 1970). The discriminative properties of time were
(e.g., a key peck to one and a treadle press to the also borne out in conditional discrimination experi-
other), (d) using qualitatively different reinforcers ments in which the reinforced response was condi-
for correct responses to either of the stimuli (differ- tional on the passage of one time interval versus
ential outcomes procedure), and (e) imposing delays another (e.g., Stubbs, 1968) and in experiments
between the response to the sample and onset of the involving the peak interval procedure, in which
choice component (delayed matching to sample; see occasional reinforcers are deleted from a series of
MacKay, 1991, for a review). FIs to reveal where responding peaks before waning.
There are myriad possibilities for sample stimuli. Research on temporal control in turn has given rise
Everything from simple colors to astonishingly com- to different quantitative theories of timing, notably
plex visual arrays has been used to establish condi- scalar expectancy theory (Gibbon, 1977) and the
tional stimulus control of responding. A particularly behavioral theory of timing (Killeen & Fetterman,
fruitful area of research involving conditional stimu- 1988). Both theories integrate significant amounts of
lus control is that of delayed matching to sample data generated using the aforementioned proce-
(see Chapter 18, this volume). Generally speaking, dures, and both have had considerable heuristic
choice accuracy declines as delays increase. Indiffer- value. The behavioral theory of timing focuses more
ence between the choices is reached with pigeons directly on environmental and behavioral events in
at around 30-s delays. The appropriate description accounting for the discriminative properties of time.
54
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Concept Learning requires a demonstration of three properties: reflex-

One definition of a concept is in terms of stimulus ivity, symmetry, and transitivity. Reflexivity is estab-
control. A concept may be said to exist when a simi- lished by showing, in the absence of reinforcement,
lar response is controlled by common elements of generalized identity matching (i.e., selecting the
otherwise dissimilar stimuli. Human concepts often comparison stimulus that is identical to the sample).
are verbal in nature, for example, abstract configura- In normally developing humans, the tests for sym-
tions of stimuli that evoke words such as love, metry and transitivity often are combined. One such
esoteric, liberating, and so forth. Despite their com- test consists of teaching the relation between A and
plexity, concepts are considered in TEAB to be on a B and that between A and C, using the conditional
continuum with other types of stimulus control of discrimination procedure described previously (e.g.,
behavior. The classic demonstration of stimulus given Sample A, select B from among the available
control of responding by an abstract stimulus was comparison stimuli). In subsequent no-feedback test
that of Herrnstein, Loveland, and Cable (1976). An trials, if C is selected after a B sample and B is
S+–S− discrimination was established using a mul- selected after a C sample, then these emergent
tiple schedule in which responses in one component (untrained) transitive relations require that the
were reinforced according to a VI 30-s schedule and trained A–B and A–C relations be symmetric (B–A
extinguished in the other. The S+ in each of three and C–A, respectively).
experiments was, respectively, one of a variety of Stimulus equivalence suggests a mechanism
slides (more than 1,500 different ones—half positive whereby new stimulus relations can develop in the
and half negative in terms of containing the concept absence of direct reinforcement. Sidman (1986) sug-
under study—were used in each of the three experi- gested that these emergent relations could address
ments) that were pictures of trees, water, or a partic- criticisms of the inflexibility of a behavior-analytic
ular person. In each experiment, the S− was the approach that relies on direct reinforcement to
absence of these features in an otherwise parallel set establish new responses. In addition, if different sets
of slides. Response rates were higher in the presence of equivalence relations are themselves equated
of the concept under investigation than in its through training a connecting relation (Sidman,
absence. The basic results of Herrnstein et al. have Kirk, & Wilson-Morris, 1985), then the number of
been replicated systematically many times, using a equivalence relations established without training
variety of types of visual stimuli (e.g., Wasserman, increases exponentially. As Sidman observed, both
Young, & Peissig, 2002). of these outcomes of stimulus equivalence are
The general topic of concepts and concept learn- important advancements in accounting for the
ing as instances of stimulus control has been acquisition of verbal behavior within a behavior-
approached in a different, but equally fruitful way analytic framework. By expanding the analysis of
by Sidman (e.g., 1986; Sidman & Tailby, 1982). stimulus equivalence to a five-term contingency,
Consider three groups of unrelated stimuli, A, B, Sidman also attempted to account for meaning in
and C, presented on a computer screen. Different context (see Volume 2, Chapters 1, 6, and 18, this
patterns make up A; different shapes, B; and non- handbook).
sense syllables, C. The question posed by Sidman
and Tailby (1982) was how these structurally differ- Rules and Instructions
ent groups of stimuli might all come to control simi- An important source of discriminative stimulus con-
lar responses to them, that is, become equivalent to trol of human behavior is verbal behavior (Skinner,
one another—to function as a stimulus controlling 1957). The analysis of this discriminative function
the same response; that is, as a concept. of verbal behavior has most frequently taken the
Sidman and Tailby (1982) turned to mathematics form in TEAB of an analysis of how rules and
for a definition of equivalence and to the conditional instructions (the two terms are used interchangeably
discrimination procedure (outlined earlier) for its here, but see also Catania [1998] for suggestions
analysis. An equivalence relation in mathematics concerning the rules for describing such control of
55
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
behavior) act in concert with contingencies to con- functioned as a discriminative stimulus controlling
trol human behavior. The control by instructions or responding under the two schedules.
rules is widely, but not universally, considered a
type of discriminative control over responding (e.g.,
The Five Pillars Redux
Blakely & Schlinger, 1987).
In human interactions, instructions may be spo- Methods, reinforcement, punishment, control by
ken, written, or both. The effectiveness of instruc- stimuli correlated with reinforcers and punishers,
tions in controlling behavior varies in part as a and contextual and stimulus control—these are the
function of their specificity and their congruency five pillars of TEAB, the foundation on which the
with the contingencies to which they refer. Galizio analyses and findings described in other chapters of
(1979), for example, elegantly showed how congru- this handbook are constructed. A review of these
ent instructions complement contingencies and pillars balanced several factors with one another.
incongruent ones can lead to ignoring the instruc- The first was differing views as to what is fundamen-
tion. Incongruence does not universally have this tal. As with any science, inconsistencies in findings
effect, however. In some experiments, inaccurate and differences in interpretation are commonplace.
instructions have been found to exert control over They are, however, the fodder for further growth
responding at the expense of the actual contingen- in the science. The second was depth versus breadth
cies in effect. of the topics. The relative space devoted to the four
Galizio’s (1979) results, however, may be more a pillars representing empirical findings in TEAB
qualification than the rule in explicating the role of reflects, more or less, the relative research activity
instructions in controlling human behavior. In many making up each of those pillars. Each pillar is, of
situations in which explicit losses are not incurred course, deeper than can be developed within the
for following rules, humans often behave in stereo- space constraints assigned. Important material was
typed ways that suggest they are following either an truncated to attain the breadth of coverage expected
instruction or their interpretation of an instruction. of an overview. The third was classic and contempo-
This outcome is not surprising given the long extra- rary research. Both have a role in defining foundations;
experimental history of reinforced rule following. the former lay the groundwork for contemporary
Even in the absence of explicit rules, some observers developments, which in turn herald the future
have postulated that humans construct their own of TEAB.
rules. Such an analysis, however, is a quagmire— Finally, the metaphorical nature of the five pil-
once private events such as self-generated rules are lars needs to be taken a step further, for these are
postulated to control responding in some situations, not pillars of stone. Rather, the material of these pil-
it becomes difficult to exclude their possible role in lars is organic, subject to the same contingencies
every situation. Nonetheless, in one study of how that they seek to describe. Research areas and prob-
self-generated rules might control behavior, Catania, lems come and go for a host of reasons. They are
Matthews, and Shimoff (1982) had college students subject to the vicissitudes of life: Researchers come
respond on two buttons; one reinforced high rate into their own, move, change, retire, or die (physi-
responding, and the other reinforced lower rate cally or metaphorically; sometimes both, but some-
responding. The task was interrupted from time to times at different times); agency funding and
time, and the students were asked to guess (by com- university priorities change. Dead ends are reached.
pleting a series of structured sentences) what the Marvelous discoveries captivate entire generations
schedule requirements were on the buttons. Stating of scientists, or maybe just one scientist. Changes in
the correct rule was shaped by reinforcing approxi- TEAB will change both the content and, over time,
mations to the correct description with points. perhaps the very pillars themselves. Indeed, it is
Of interest was the relation between the shaped highly likely that the research described in this
rule and responding in accord with the schedule in handbook eventually will rewrite this chapter. As
effect on either button. In general, the shaped rules TEAB and the pillars that support it continue to
56
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
evolve, TEAB will contribute even more to an Baron, A. (1991). Avoidance and punishment. In
understanding of the behavior of organisms. I. Iversen & K. A. Lattal (Eds.), Experimental analysis
of behavior: Part1 (pp. 173–217). Amsterdam, the
Netherlands: Elsevier.
References Baron, A. (1999). Statistical inference in behavior analy-
Allyon, T., & Azrin, N. H. (1968). The token economy. sis: Friend or foe? Behavior Analyst, 22, 83–85.
New York, NY: Appleton-Century-Crofts. Baron, A., & Galizio, M. (2005). Positive and negative
Arbuckle, J. L., & Lattal, K. A. (1987). A role for negative reinforcement: Should the distinction be preserved?
reinforcement of response omission in punishment? Behavior Analyst, 28, 85–98.
Journal of the Experimental Analysis of Behavior, 48, Baum, W. M. (1973). The correlation-based law of effect.
407–416. doi:10.1901/jeab.1987.48-407 Journal of the Experimental Analysis of Behavior, 20,
Azrin, N. H. (1956). Some effects of two intermittent 137–153. doi:10.1901/jeab.1973.20-137
schedules of immediate and non-immediate punish- Baum, W. M. (1974). On two types of deviation from the
ment. Journal of Psychology, 42, 3–21. doi:10.1080/00 matching law: Bias and undermatching. Journal of
223980.1956.9713020 the Experimental Analysis of Behavior, 22, 231–242.
Azrin, N. H. (1961). Time-out from positive reinforce- doi:10.1901/jeab.1974.22-231
ment. Science, 133, 382–383. doi:10.1126/science. Baum, W. M. (1989). Quantitative description and molar
133.3450.382 description of the environment. Behavior Analyst, 12,
Azrin, N. H., & Hake, D. F. (1969). Positive conditioned 167–176.
suppression: Conditioned suppression using positive Blackman, D. (1968). Conditioned suppression or
reinforcers as the unconditioned stimuli. Journal of facilitation as a function of the behavioral baseline.
the Experimental Analysis of Behavior, 12, 167–173. Journal of the Experimental Analysis of Behavior, 11,
doi:10.1901/jeab.1969.12-167 53–61. doi:10.1901/jeab.1968.11-53
Azrin, N. H., Hake, D. F., Holz, W. C., & Hutchinson, Blackman, D. E. (1977). Conditioned suppression and the
R. R. (1965). Motivational aspects of escape from effects of classical conditioning on operant behavior.
punishment. Journal of the Experimental Analysis of In W. K. Honig & J. E. R. Staddon (Eds.), Handbook
Behavior, 8, 31–44. doi:10.1901/jeab.1965.8-31 of operant behavior (pp. 340–363). New York, NY:
Azrin, N. H., & Holz, W. C. (1966). Punishment. In W. Prentice Hall.
K. Honig (Ed.), Operantbehavior: Areas of research Blakely, E., & Schlinger, H. (1987). Rules: Function-
and application (pp. 380–447). New York, NY: altering contingency-specifying stimuli. Behavior
Appleton-Century-Crofts. Analyst, 10, 183–187.
Azrin, N. H., Holz, W. C., & Hake, D. F. (1963). Fixed- Blakely, E., & Schlinger, H. (1988). Determinants of
ratio punishment. Journal of the Experimental pausing under variable-ratio schedules: Reinforcer
Analysis of Behavior, 6, 141–148. doi:10.1901/ magnitude, ratio size, and schedule configuration.
jeab.1963.6-141 Journal of the Experimental Analysis of Behavior, 50,
Azrin, N. H., Holz, W. C., Hake, D. F., & Allyon, T. 65–73. doi:10.1901/jeab.1988.50-65
(1963). Fixed-ratio escape reinforcement. Journal of Bonem, M., & Crossman, E. K. (1988). Elucidating
the Experimental Analysis of Behavior, 6, 449–456. the effects of reinforcer magnitude. Psychological
doi:10.1901/jeab.1963.6-449 Bulletin, 104, 348–362. doi:10.1037/0033-
Azrin, N. H., Hutchinson, R. R., & Hake, D. F. (1966). 2909.104.3.348
Extinction-induced aggression. Journal of the Boren, J. J., & Devine, D. D. (1968). The repeated acquisi-
Experimental Analysis of Behavior, 9, 191–204. tion of behavioral chains. Journal of the Experimental
doi:10.1901/jeab.1966.9-191 Analysis of Behavior, 11, 651–660. doi:10.1901/
Bachrach, A. (1960). Psychological research: An introduc- jeab.1968.11-651
tion. New York, NY: Random House. Branch, M. N. (1977). On the role of “memory” in behav-
Baer, D. M. (1981). The imposition of structure on behavior analysis. Journal of the Experimental Analysis of
ior and the demolition of behavioral structures. In D. Behavior, 28, 171–179. doi:10.1901/jeab.1977.28-171
J. Bernstein (Ed.), Response structure and organization Breland, K., & Breland, M. (1961). The misbehavior
(pp. 217–254). Lincoln: University of Nebraska Press. of organisms. American Psychologist, 16, 681–684.
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some doi:10.1037/h0040090
current dimensions of applied behavior analysis. Brinker, R. P., & Treadway, J. T. (1975). Preference and
Journal of Applied Behavior Analysis, 1, 91–97. discrimination between response-dependent and
doi:10.1901/jaba.1968.1-91 response-independent schedules of reinforcement.
57
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
Journal of the Experimental Analysis of Behavior, 24, analyzing effects of reinforcer magnitude. Journal of
73–77. doi:10.1901/jeab.1975.24-73 the Experimental Analysis of Behavior, 60, 641–666.
Brown, P. L., & Jenkins, H. M. (1968). Autoshaping doi:10.1901/jeab.1993.60-641
the pigeon’s key-peck. Journal of the Experimental de Lorge, J. (1971). The effects of brief stimuli presented
Analysis of Behavior, 11, 1–8. doi:10.1901/ under a multiple schedule of second-order schedules.
jeab.1968.11-1 Journal of the Experimental Analysis of Behavior, 15,
Bruzek, J. L., Thompson, R. H., & Peters, L. C. (2009). 19–25. doi:10.1901/jeab.1971.15-19
Resurgence of infant caregiving responses. Journal of Deluty, M. Z. (1976). Choice and the rate of punishment
the Experimental Analysis of Behavior, 92, 327–343. in concurrent schedules. Journal of the Experimental
doi:10.1901/jeab.2009-92-327 Analysis of Behavior, 25, 75–80. doi:10.1901/
Catania, A. C. (1963). Concurrent performances: A jeab.1976.25-75
baseline for the study of reinforcement magnitude. Dews, P. B. (1970). The theory of fixed-interval respond-
Journal of the Experimental Analysis of Behavior, 6, ing. In W. N. Schoenfeld (Ed.), The theory of rein-
299–300. doi:10.1901/jeab.1963.6-299 forcement schedules (pp. 43–61). New York, NY:
Catania, A. C. (1998). The taxonomy of verbal behavior. Appleton-Century-Crofts.
In K. A. Lattal & M. Perone (Eds.), Handbook of Dinsmoor, J. A. (1962). Variable-interval escape from stim-
research methods in human operant behavior uli accompanied by shocks. Journal of the Experimental
(pp. 405–433). New York, NY: Plenum Press. Analysis of Behavior, 5, 41–47. doi:10.1901/jeab.
Catania, A. C., Matthews, B. A., & Shimoff, E. (1982). 1962.5-41
Instructed versus shaped human verbal behavior: Dinsmoor, J. A. (1983). Observing and conditioned rein-
Interactions with nonverbal responding. Journal of forcement. Behavioral and Brain Sciences, 6, 693–728.
the Experimental Analysis of Behavior, 38, 233–248. doi:10.1017/S0140525X00017969
doi:10.1901/jeab.1982.38-233
Donahoe, J. W. (2003). Selectionism. In K. A. Lattal &
Catania, A. C., & Reynolds, G. S. (1968). A quantitative P. N. Chase (Eds.), Behavior theory and philosophy
analysis of responding maintained by interval sched- (pp. 103–128). New York, NY: Kluwer Academic.
ules of reinforcement. Journal of the Experimental
Analysis of Behavior, 11, 327–383. doi:10.1901/ Eckerman, D. A., Hienz, R. D., Stern, S., & Kowlowitz, V.
jeab.1968.11-s327 (1980). Shaping the location of a pigeon’s peck:
Effect of rate and size of shaping steps. Journal of
Church, R. M., & Raymond, G. A. (1967). Influence of
the Experimental Analysis of Behavior, 33, 299–310.
the schedule of positive reinforcement on punished
doi:10.1901/jeab.1980.33-299
behavior. Journal of Comparative and Physiological
Psychology, 63, 329–332. doi:10.1037/h0024382 Escobar, R., & Bruner, C. A. (2009). Observing responses
Critchfield, T. S., Paletz, E. M., MacAleese, K. R., & and serial stimuli: Searching for the reinforcing prop-
Newland, M. C. (2003). Punishment in human erties of the S−. Journal of the Experimental Analysis of
choice: Direct or competitive suppression? Journal Behavior, 92, 215–231. doi:10.1901/jeab.2009.92-215
of the Experimental Analysis of Behavior, 80, 1–27. Estes, W. K., & Skinner, B. F. (1941). Some quantita-
doi:10.1901/jeab.2003.80-1 tive properties of anxiety. Journal of Experimental
Das Graças de Souza, D. D., de Moraes, A. B. A., & Psychology, 29, 390–400. doi:10.1037/h0062283
Todorov, J. C. (1984). Shock intensity and signaled Fantino, E. (1969). Conditioned reinforcement, choice,
avoidance responding. Journal of the Experimental and the psychological distance to reward. In
Analysis of Behavior, 42, 67–74. doi:10.1901/ D. P. Hendry (Ed.), Conditioned reinforcement
jeab.1984.42-67 (pp. 163–191). Homewood, IL: Dorsey Press.
Davison, M. (1999). Statistical inference in behavior anal- Fantino, E. (1977). Conditioned reinforcement: Choice
ysis: Having my cake and eating it. Behavior Analyst, and information. In W. K. Honig & J. E. R. Staddon
22, 99–103. (Eds.), Handbook of operant behavior (pp. 313–339).
Davison, M., & Baum, W. M. (2006). Do conditional rein- New York, NY: Prentice Hall.
forcers count? Journal of the Experimental Analysis of Fantino, E. (1991). Behavioral ecology. In I. Iversen &
Behavior, 86, 269–283. doi:10.1901/jeab.2006.56-05 K. A. Lattal (Eds.), Experimental analysis of behavior:
Davison, M. C., & Tustin, R. D. (1978). The relation Part 2 (pp. 117–153). Amsterdam, the Netherlands:
between the generalized matching law and signal Elsevier.
detection theory. Journal of the Experimental Analysis of Fantino, E., & Silberberg, A. (2010). Revisiting the role of
Behavior, 29, 331–336. doi:10.1901/jeab.1978.29-331 bad news in maintaining human observing behavior.
DeGrandpre, R. J., Bickel, W. K., Hughes, J. R., Layng, M. P., Journal of the Experimental Analysis of Behavior, 93,
& Badger, G. (1993). Unit price as a useful metric in 157–170. doi:10.1901/jeab.2010.93-157
58
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Farley, J. (1980). Reinforcement and punishment effects Herrnstein, R. J., & Hineline, P. N. (1966). Negative
in concurrent schedules: A test of two models. reinforcement as shock-frequency reduction. Journal
Journal of the Experimental Analysis of Behavior, 33, of the Experimental Analysis of Behavior, 9, 421–430.
311–326. doi:10.1901/jeab.1980.33-311 doi:10.1901/jeab.1966.9-421
Felton, M., & Lyon, D. O. (1966). The post-reinforcement Herrnstein, R. J., Loveland, D. H., & Cable, C. (1976).
pause. Journal of the Experimental Analysis of Behavior, Natural concepts in pigeons. Journal of Experimental
9, 131–134. doi:10.1901/jeab.1966.9-131 Psychology: Animal Behavior Processes, 2, 285–302.
doi:10.1037/0097-7403.2.4.285
Ferster, C. B., & Skinner, B. F. (1957). Schedules of rein-
forcement. New York, NY: Appleton-Century-Crofts. Hoffman, H. S. (1966). The analysis of discriminated
doi:10.1037/10627-000 avoidance. In W. K. Honig (Ed.), Operant behavior:
Areas of research and application (pp. 499–530). New
Freeman, T. J., & Lattal, K. A. (1992). Stimulus control
York, NY: Appleton-Century-Crofts.
of behavioral history. Journal of the Experimental
Analysis of Behavior, 57, 5–15. doi:10.1901/jeab. Holz, W. C. (1968). Punishment and rate of positive
1992.57-5 reinforcement. Journal of the Experimental Analysis of
Behavior, 11, 285–292. doi:10.1901/jeab.1968.11-285
Galizio, M. (1979). Contingency-shaped and rule-
governed behavior: Instructional control of human Holz, W. C., & Azrin, N. H. (1961). Discriminative prop-
loss avoidance. Journal of the Experimental Analysis of erties of punishment. Journal of the Experimental
Behavior, 31, 53–70. doi:10.1901/jeab.1979.31-53 Analysis of Behavior, 4, 225–232. doi:10.1901/
jeab.1961.4-225
Gibbon, J. (1977). Scalar expectancy theory and Weber’s
law in animal timing. Psychological Review, 84, Hoyert, M. S. (1992). Order and chaos in fixed-interval
279–325. doi:10.1037/0033-295X.84.3.279 schedules of reinforcement. Journal of the Experimental
Analysis of Behavior, 57, 339–363. doi:10.1901/
Gibson, D. A. (1966). A quick and simple method for jeab.1992.57-339
magazine training the pigeon. Perceptual and Motor
Skills, 23, 1230. doi:10.2466/pms.1966.23.3f.1230 Hull, C. L. (1943). Principles of psychology. New York,
NY: Appleton-Century-Crofts.
Gibson, D. A. (1968). Conditioned punishment by stimuli sig-
nalling time out from positive reinforcement. Unpublished Hursh, S. R. (1980). Economic concepts for the analysis
doctoral dissertation, University of Alabama, Tuscaloosa. of behavior. Journal of the Experimental Analysis of
Green, D. M., & Swets, J. A. (1966). Signal detection the-
ory and psychophysics. New York, NY: Wiley. Kaufman, A., & Baron, A. (1968). Suppression of behav-
ior by timeout punishment when suppression
Green, L., & Freed, D. E. (1998). Behavioral econom- results in loss of positive reinforcement. Journal of
ics. In W. O’Donohue (Ed.), Learning and behavior the Experimental Analysis of Behavior, 11, 595–607.
therapy (pp. 274–300). Needham Heights, MA: Allyn doi:10.1901/jeab.1968.11-595
& Bacon.
Kazdin, A. E. (1982). Single-case research designs: Methods
Hake, D. F., & Azrin, N. H. (1965). Conditioned punish- for clinical and applied settings. New York, NY:
ment. Journal of the Experimental Analysis of Behavior, Oxford University Press.
8, 279–293. doi:10.1901/jeab.1965.8-279
Kelleher, R. T. (1966). Chaining and conditioned rein-
Hall, G. A., & Lattal, K. A. (1990). Variable-interval forcement. In W. K. Honig (Ed.), Operant behavior:
schedule performance under open and closed econo- Areas of research and application (pp. 160–212). New
mies. Journal of the Experimental Analysis of Behavior, York, NY: Appleton-Century-Crofts.
54, 13–22. doi:10.1901/jeab.1990.54-13
Kelleher, R. T., & Gollub, L. R. (1962). A review of con-
Hearst, E., Besley, S., & Farthing, G. W. (1970). ditioned reinforcement. Journal of the Experimental
Inhibition and the stimulus control of operant behav- Analysis of Behavior, 5, 543–597. doi:10.1901/
ior. Journal of the Experimental Analysis of Behavior, jeab.1962.5-s543
14, 373–409. doi:10.1901/jeab.1970.14-s373
Keller, F. S., & Schoenfeld, W. N. (1950). Principles of
Herrnstein, R. J. (1961). Relative and absolute strength of psychology. New York, NY: Appleton-Century-Crofts.
response as a function of frequency of reinforcement.
Kelly, D. D. (1973). Suppression of random-ratio and
Journal of the Experimental Analysis of Behavior, 4,
acceleration of temporally spaced responding by
267–272. doi:10.1901/jeab.1961.4-267
the same prereward stimulus in monkeys. Journal of
Herrnstein, R. J. (1969). Method and theory in the study the Experimental Analysis of Behavior, 20, 363–373.
of avoidance. Psychological Review, 76, 49–69. doi:10.1901/jeab.1973.20-363
Herrnstein, R. J. (1970). On the law of effect. Journal of Killeen, P. R., & Fetterman, J. G. (1988). A behavioral
the Experimental Analysis of Behavior, 13, 243–266. theory of timing. Psychological Review, 95, 274–295.
doi:10.1901/jeab.1970.13-243 doi:10.1037/0033-295X.95.2.274
59
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
Kupfer, A. S., Allen, R., & Malagodi, E. F. (2008). Leander, J. D. (1973). Shock intensity and duration inter-
Induced attack during fixed-ratio and matched- actions on free-operant avoidance behavior. Journal
time schedules of food presentation. Journal of of the Experimental Analysis of Behavior, 19, 481–490.
the Experimental Analysis of Behavior, 89, 31–48. doi:10.1901/jeab.1973.19-481
doi:10.1901/jeab.2008.89-31
Lieving, G. A., & Lattal, K. A. (2003). Recency, repeat-
Lattal, K. A. (1974). Combinations of response- ability, and reinforcer retrenchment: An experimen-
reinforcer dependence and independence. Journal of tal analysis of resurgence. Journal of the Experimental
the Experimental Analysis of Behavior, 22, 357–362. Analysis of Behavior, 80, 217–233. doi:10.1901/
doi:10.1901/jeab.1974.22-357 jeab.2003.80-217
Lattal, K. A. (1975). Reinforcement contingencies as Logue, A. W., & de Villiers, P. A. (1978). Matching in
discriminative stimuli. Journal of the Experimental concurrent variable-interval avoidance schedules.
Analysis of Behavior, 23, 241–246. doi:10.1901/ Journal of the Experimental Analysis of Behavior, 29,
jeab.1975.23-241 61–66. doi:10.1901/jeab.1978.29-61
Lattal, K. A. (1979). Reinforcement contingencies as dis- LoLordo, V. M. (1971). Facilitation of food-reinforced
criminative stimuli: II. Effects of changes in stimulus responding by a signal for response-independent
probability. Journal of the Experimental Analysis of food. Journal of the Experimental Analysis of Behavior,
Behavior, 31, 51–22. 15, 49–55. doi:10.1901/jeab.1971.15-49
Lattal, K. A. (1987). Considerations in the experi- Lund, C. A. (1976). Effects of variations in the temporal
mental analysis of reinforcement delay. In M. L. distribution of reinforcements on interval schedule
Commons, J. Mazur, J. A. Nevin, & H. Rachlin performance. Journal of the Experimental Analysis of
(Eds.), Quantitative studies of operant behavior: The Behavior, 26, 155–164. doi:10.1901/jeab.1976.26-155
effect of delay and of intervening events on reinforce-
Mackay, H. A. (1991). Conditional stimulus control. In I.
ment value (pp. 107–123). New York, NY: Erlbaum.
Iversen & K. A. Lattal (Eds.), Experimental analysis
Lattal, K. A. (1989). Contingencies on response rate and of behavior: Part 1 (pp. 301–350). Amsterdam, the
resistance to change. Learning and Motivation, 20, Netherlands: Elsevier.
191–203. doi:10.1016/0023-9690(89)90017-9
Madden, G. J. (2000). A behavioral economics primer. In
Lattal, K. A. (1991). Scheduling positive reinforcers. In I. W. Bickel & R. K. Vuchinich (Eds.), Reframing health
H. Iversen & K. A. Lattal (Eds.), Experimental analy- behavior change with behavioral economics (pp. 3–26).
sis of behavior: Part I (pp. 87–134). Amsterdam, the Mahwah, NJ: Erlbaum.
Netherlands: Elsevier.
Madden, G. J., & Bickel, W. K. (Eds.). (2010). Impulsivity:
Lattal, K. A. (2010). Delayed reinforcement of operant The behavioral and neurological science of discounting.
behavior. Journal of the Experimental Analysis of Washington, DC: American Psychological Association.
Behavior, 93, 129–139. doi:10.1901/jeab.2010.93-129 doi:10.1037/12069-000
Lattal, K. A., & Boyer, S. S. (1980). Alternative reinforce- Marr, M. J. (1992). Behavior dynamics: One perspective.
ment effects on fixed-interval responding. Journal of Journal of the Experimental Analysis of Behavior, 57,
the Experimental Analysis of Behavior, 34, 285–296. 249–266. doi:10.1901/jeab.1992.57-249
doi:10.1901/jeab.1980.34-285
Marr, M. J. (2006). Through the looking glass: Symmetry in
Lattal, K. A., & Bryan, A. J. (1976). Effects of concurrent behavioral principles? Behavior Analyst, 29, 125–128.
response-independent reinforcement on fixed-
Mazur, J. E. (1986). Choice between single and mul-
interval schedule performance. Journal of the
tiple delayed reinforcers. Journal of the Experimental
Experimental Analysis of Behavior, 26, 495–504.
doi:10.1901/jeab.1976.26-495
jeab.1986.46-67
Lattal, K. A., Freeman, T. J., & Critchfield, T. (1989).
McDowell, J. J. (1986). On the falsifiability of match-
Dependency location in interval schedules of rein-
ing theory. Journal of the Experimental Analysis of
forcement. Journal of the Experimental Analysis of
McDowell, J. J. (1989). Two modern developments in
Lattal, K. A., & Griffin, M. A. (1972). Punishment con-
matching theory. Behavior Analyst, 12, 153–166.
trast during free-operant avoidance. Journal of the
Experimental Analysis of Behavior, 18, 509–516. McKearney, J. W. (1972). Maintenance and suppression
doi:10.1901/jeab.1972.18-509 of responding under schedules of electric shock
presentation. Journal of the Experimental Analysis of
Lea, S. E. G. (1986). Foraging and reinforcement
schedules in the pigeon: Optimal and non-optimal
aspects of choice. Animal Behaviour, 34, 1759–1768. Michael, J. (1974). Statistical inference for individual
doi:10.1016/S0003-3472(86)80262-7 organism research: Mixed blessing or curse? Journal
60
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
of Applied Behavior Analysis, 7, 647–653. doi:10.1901/ Peele, D. B., Casey, J., & Silberberg, A. (1984). Primacy
jaba.1974.7-647 of interresponse time reinforcement in account-
ing for rate differences under variable-ratio and
Michael, J. (1975). Positive and negative reinforcement:
variable-interval schedules. Journal of Experimental
A distinction that is no longer necessary; or a better
Psychology: Animal Behavior Processes, 10, 149–167.
way to talk about bad things. Behaviorism, 3, 33–44.
doi:10.1037/0097-7403.10.2.149
Michael, J. (1982). Distinguishing between discrimina-
Perone, M., & Baron, A. (1980). Reinforcement of
tive and motivational functions of stimuli. Journal of
human observing behavior by a stimulus correlated
with extinction or increased effort. Journal of the
doi:10.1901/jeab.1982.37-149
Moore, J., & Fantino, E. (1975). Choice and response doi:10.1901/jeab.1980.34-239
contingencies. Journal of the Experimental Analysis
of Behavior, 23, 339–347. doi:10.1901/jeab.1975. Perone, M., & Galizio, M. (1987). Variable-interval
23-339 schedules of time out from avoidance. Journal of
Morse, W. H., & Kelleher, R. T. (1977). Determinants of doi:10.1901/jeab.1987.47-97
reinforcement and punishment. In W. K. Honig &
J. E. R. Staddon (Eds.), Handbook of operant behavior Pietras, C. J., & Hackenberg, T. D. (2005). Response-
(pp. 98–124). New York, NY: Prentice Hall. cost punishment via token loss with pigeons.
Behavioural Processes, 69, 343–356. doi:10.1016/j.
Neuringer, A. J. (1970). Superstitious key pecking after beproc.2005.02.026
three peck-produced reinforcements. Journal of the
Experimental Analysis of Behavior, 13, 127–134. Platt, J. R. (1973). Percentile reinforcement: Paradigms
doi:10.1901/jeab.1970.13-127 for experimental analysis of response shaping. In
G. H. Bower (Ed.), The psychology of learning and
Neuringer, A. (2002). Operant variability: Evidence, motivation: Advances in theory and research (Vol. 7,
functions, and theory. Psychonomic Bulletin and pp. 271–296). New York, NY: Academic Press.
Review, 9, 672–705. doi:10.3758/BF03196324
Premack, D. (1959). Toward empirical behavior laws:
Nevin, J. A. (1969). Signal detection theory and operant 1. Positive reinforcement. Psychological Review, 66,
behavior: A review of David M. Green and John A. 219–233. doi:10.1037/h0040891
Swets’Signal detection theory and psychophysics.
Journal of the Experimental Analysis of Behavior, 12, Rachlin, H., & Green, L. (1972). Commitment, choice
475–480. doi:10.1901/jeab.1969.12-475 and self-control. Journal of the Experimental Analysis
of Behavior, 17, 15–22. doi:10.1901/jeab.1972.17-15
Nevin, J. A. (1974). Response strength in multiple sched-
ules. Journal of the Experimental Analysis of Behavior, Rasmussen, E. B, & Newlin, C. (2008). Asymmetry of
21, 389–408. doi:10.1901/jeab.1974.21-389 reinforcement and punishment in human choice.
Nevin, J. A., Mandell, C., & Atak, J. R. (1983). The 157–167. doi:10.1901/jeab.2008.89-157
analysis of behavioral momentum. Journal of the
Experimental Analysis of Behavior, 39, 49–59. Ray, B. A. (1969). Selective attention: The effects of com-
doi:10.1901/jeab.1983.39-49 bining stimuli which control incompatible behavior.
Nevin, J. A., Tota, M. E., Torquato, R. D., & Shull, R. L. 539–550. doi:10.1901/jeab.1969.12-539
(1990). Alternative reinforcement increases resis-
tance to change: Pavlovian or operant contingencies? Rescorla, R. A., & Skucy, J. C. (1969). Effect of response-
Journal of the Experimental Analysis of Behavior, 53, independent reinforcers during extinction. Journal
359–379. doi:10.1901/jeab.1990.53-359 of Comparative and Physiological Psychology, 67,
381–389. doi:10.1037/h0026793
Ono, K. (2004). Effects of experience on prefer-
ence between forced and free choice. Journal of Richards, J. B., Mitchell, S. H., de Wit, H., & Seiden, L. S.
the Experimental Analysis of Behavior, 81, 27–37. (1997). Determination of discount functions in rats
doi:10.1901/jeab.2004.81-27 with an adjusting-amount procedure. Journal of
Palya, W. L. (1992). Dynamics in the fine structure doi:10.1901/jeab.1997.67-353
of schedule-controlled behavior. Journal of the
Experimental Analysis of Behavior, 57, 267–287. Rilling, M. (1977). Stimulus control and inhibitory
doi:10.1901/jeab.1992.57-267 processes. In W. K. Honing & J. E. R. Staddon
(Eds.), Handbook of operant behavior (pp. 432–480).
Pear, J. J., & Legris, J. A. (1987). Shaping by automated
Englewood Cliffs, NJ: Prentice-Hall.
tracking of an arbitrary operant response. Journal of
the Experimental Analysis of Behavior, 47, 241–247. Schuster, R., & Rachlin, H. (1968). Indifference between
doi:10.1901/jeab.1987.47-241 punishment and free shock: Evidence for the negative
61
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kennon A. Lattal
law of effect. Journal of the Experimental Analysis of Staddon, J. E. R. (1968). Spaced responding and choice: A
Behavior, 11, 777–786. doi:10.1901/jeab.1968.11-777 preliminary analysis. Journal of the Experimental Analysis
Shnidman, S. R. (1968). Extinction of Sidman avoid-
ance behavior. Journal of the Experimental Analysis of Staddon, J. E. R. (1979). Operant behavior as adaptation
Behavior, 11, 153–156. doi:10.1901/jeab.1968.11-153 to constraint. Journal of Experimental Psychology:
General, 108, 48–67. doi:10.1037/0096-3445.108.1.48
Sidman, M. (1953). Two temporal parameters of the
maintenance of avoidance behavior by the white rat. Staddon, J. E. R., & Simmelhag, V. (1971). The “supersti-
Journal of Comparative and Physiological Psychology, tion” experiment: A re-examination of its implications
46, 253–261. doi:10.1037/h0060730 for the principles of adaptive behavior. Psychological
Review, 78, 3–43. doi:10.1037/h0030305
Sidman, M. (1960). Tactics of scientific research. New
York, NY: Basic Books. Stubbs, A. (1968). The discrimination of stimulus dura-
tion by pigeons. Journal of the Experimental Analysis of
Sidman, M. (1966). Avoidance behavior. In W. Honig
(Ed.), Operant behavior: Areas of research and appli-
cation (pp. 448–498). New York, NY: Appleton- Terrace, H. S. (1963). Discrimination learning with and
Century-Crofts. without “errors.” Journal of the Experimental Analysis
Sidman, M. (1986). Functional analysis of emergent ver-
bal classes. In T. Thompson & M. D. Zeiler (Eds.), Terrace, H. S. (1966). Stimulus control. In W. Honig
Analysis and integration of behavioral units (pp. 213–245). (Ed.), Operant behavior: Areas of research and appli-
Hillsdale, NJ: Erlbaum. cation (pp. 271–344). New York, NY: Appleton-
Century-Crofts.
Sidman, M., Kirk, B., & Wilson-Morris, M. (1985).
Six-member stimulus classes generated by Thorndike, E. L. (1911). Animal intelligence. New York,
conditional-discrimination procedures. Journal of NY: Macmillan.
the Experimental Analysis of Behavior, 43, 21–42. Timberlake, W., & Allison, J. (1974). Response depri-
doi:10.1901/jeab.1985.43-21 vation: An empirical approach to instrumental
Sidman, M., & Tailby, W. (1982). Conditional discrimi- performance. Psychological Review, 81, 146–164.
nation vs. matching to sample: An expansion of the doi:10.1037/h0036101
testing paradigm. Journal of the Experimental Analysis Timberlake, W., & Lucas, G. A. (1985). The basis of
of Behavior, 37, 5–22. doi:10.1901/jeab.1982.37-5 superstitious behavior: Chance contingency, stimu-
Siegel, P. S., & Milby, J. B. (1969). Secondary reinforce- lus substitution, or appetitive behavior? Journal of
ment in relation to shock termination: Second chap- the Experimental Analysis of Behavior, 44, 279–299.
ter. Psychological Bulletin, 72, 146–156. doi:10.1037/ doi:10.1901/jeab.1985.44-279
h0027781 Uhl, C. N., & Garcia, E. E. (1969). Comparison of omis-
Skinner, B. F. (1935). The generic nature of the concepts of sion with extinction in response elimination in rats.
stimulus and response. Journal of General Psychology, Journal of Comparative and Physiological Psychology,
12, 40–65. doi:10.1080/00221309.1935.9920087 69, 554–562. doi:10.1037/h0028243
Skinner, B. F. (1938). Behavior of organisms. New York, Verhave, T. (1962). The functional properties of a
NY: Appleton-Century-Crofts. time out from an avoidance schedule. Journal of
Skinner, B. F. (1948). “Superstition” in the pigeon. doi:10.1901/jeab.1962.5-391
Journal of Experimental Psychology, 38, 168–172.
doi:10.1037/h0055873 Wasserman, E. A., Young, M. E., & Peissig, J. J. (2002).
Brief presentations are sufficient for pigeons to
Skinner, B. F. (1953). Science and human behavior. New discriminate arrays of same and different stimuli.
York, NY: Macmillan. Journal of the Experimental Analysis of Behavior, 78,
Skinner, B. F. (1956). A case history in scientific method. 365–373. doi:10.1901/jeab.2002.78-365
American Psychologist, 11, 221–233. doi:10.1037/ Watkins, M. J. (1990). Mediationism and the obfusca-
h0047662 tion of memory. American Psychologist, 45, 328–335.
Skinner, B. F. (1957). Verbal behavior. New York, NY: doi:10.1037/0003-066X.45.3.328
Appleton-Century-Crofts. doi:10.1037/11256-000 Weiner, H. (1962). Some effects of response cost upon
Skinner, B. F. (1966). What is the experimental analysis human operant behavior. Journal of the Experimental
of behavior? Journal of the Experimental Analysis of Analysis of Behavior, 5, 201–208. doi:10.1901/
Behavior, 9, 213–218. doi:10.1901/jeab.1966.9-213 jeab.1962.5-201
Skinner, B. F. (1981). Selection by consequences. Science, Weiner, H. (1969). Conditioning history and the control
213, 501–504. doi:10.1126/science.7244649 of human avoidance and escape responding. Journal
62
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
of the Experimental Analysis of Behavior, 12, 1039–1043. Zeiler, M. D. (1977a). Elimination of reinforced behavior:
doi:10.1901/jeab.1969.12-1039 Intermittent schedules of not-responding. Journal
of the Experimental Analysis of Behavior, 27, 23–32.
Williams, B. A. (1983). Revising the principle of rein- doi:10.1901/jeab.1977.27-23
forcement. Behaviorism, 11, 63–88.
Zeiler, M. D. (1977b). Schedules of reinforcement:
Williams, B. A. (1994). Conditioned reinforcement: The controlling variables. In W. K. Honig & J. E. R.
Experimental and theoretical issues. Behavior Staddon (Eds.), Handbook of operant behavior
Analyst, 17, 261–285. (pp. 201–232). New York, NY: Prentice Hall.
Wyckoff, L. B. (1952). The role of observing responses Zeiler, M. D. (1984). Reinforcement schedules: The
in discrimination learning. Psychological Review, 59, sleeping giant. Journal of the Experimental Analysis of
431–442. doi:10.1037/h0053932 Behavior, 42, 485–493. doi:10.1901/jeab.1984.42-485
Zeiler, M. D. (1968). Fixed and variable schedules of Zeiler, M. D. (1992). On immediate function. Journal of
response-independent reinforcement. Journal of the Experimental Analysis of Behavior, 57, 417–427.
doi:10.1901/jeab.1968.11-405
Zimmerman, J. (1969). Meanwhile . . . back at the key:
Zeiler, M. D. (1976). Positive reinforcement and the Maintenance of behavior by conditioned reinforcement
elimination of reinforced responses. Journal of and response-independent primary reinforcement. In
the Experimental Analysis of Behavior, 26, 37–44. D. P. Hendry (Ed.), Conditioned reinforcement
doi:10.1901/jeab.1976.26-37 (pp. 91–124). Homewood, IL: Dorsey Press.
63
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 3
Translational Research in
Behavior Analysis
William V. Dube
As the 21st century began, the National Institutes of produce new drugs, devices, and treatments, which
Health delineated a road map for accelerating bio- has been described as
medical research progress with increased attention
from bench to bedside, from Petri dish
to more quickly translating basic research into
to people, from animal to human. This
human studies and then into tests and treatments
is what has been typically meant by
that improve clinical practice with direct benefits to
translational research. Scientists work at
patients (Zerhouni, 2003). A consensus definition of
the molecular, then cellular level to test
the term translational research adopted by several of
“basic research,” then proceed to appli-
the institutes and other organizations is “the process
cations for animals, and on to humans.
of applying ideas, insights, and discoveries gener-
(upFRONT, 2006, p. 8)
ated through basic scientific inquiry to the treatment
or prevention of human disease” (World Health The end point for this first stage of translational
Organization, 2004, p. 141). The National Institutes research (often referred to as T1) is the production
of Health recognized that a reengineering effort was of a promising new treatment with clinical and com-
needed to support the development of translational mercial potential.
science. The National Institutes of Health estab- The second stage of translational research (T2)
lished the Clinical and Translational Science Awards addresses the gap between basic science and clinical
Consortium in October 2006 to assist institutions in medicine by improving access to treatment, systems
creating integrated academic homes for multi- and of care, point-of-care decision support tools, and so
interdisciplinary research teams to apply new forth. “The ‘laboratory’ for T2 research is the commu-
knowledge and techniques to patient care. This con- nity and ambulatory care settings, where population-
sortium began with 12 academic health centers based interventions . . . bring the results of T1
located throughout the nation, has increased to 55 research to the public . . . [an] ‘implementation sci-
centers as of this writing, and is expected to expand ence’ of fielding and evaluating interventions in real
to approximately 60 institutions by 2012. world settings” (Woolf, 2008, p. 211). Westfall, Mold,
Woolf (2008) noted that two definitions of trans- and Fagnan (2007) noted that the design of imple-
lational research exist. The term most commonly mentation systems requires a different skill set than
refers to the bench-to-bedside activity of using the that of the typical practicing physician. For this rea-
knowledge gained from basic biological sciences to son, they proposed that the final stage of translation
Preparation of this chapter was supported in part by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, primar-
ily Grants HD004147, HD046666, and HD055456. The contents of this chapter are solely the responsibility of the author and do not necessarily repre-
sent the official views of the National Institute of Child Health and Human Development. I acknowledge the truly impressive efforts of the authors of
the chapters I discuss. It was a pleasure working with them.
DOI: 10.1037/13937-003
65
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
William V. Dube
involves a distinct third step (T3) with a focus on Critchfield (2010). One question that arises when
solving the problems encountered by primary care considering the basic–applied distinction is whether
physicians as they attempt to incorporate new discov- the two classifications provide sufficient description
eries into clinical practice. The distinction between of activity in the field. Is a useful distinction able to
T2 and T3 is that between translation to patients and be made between translational and applied behavior
translation to practice. To this end, Mold and Peter- analysis? McIlvane et al. (2011) pointed out several
son (2005) described emerging “primary care dimensions along which the two areas differ. For
practice-based research networks [that] are challeng- example, the participants in applied research are
ing traditional distinctions between research and qual- selected because they have ongoing behavioral
ity improvement” (p. S12). To summarize the issues and will derive immediate-term benefit from
biomedical perspective, T1 is the translation from participation, usually as the resolution of a clinical
bench to bedside, or dish to human; T2, from bedside problem. In contrast, the participants in transla-
to community; and T3, from dissemination to practice. tional research are selected because they are repre-
To most behavior analysts, this process sounds sentatives of clinical or social groups; they may
very familiar. From the beginning, modern behavior receive some immediate benefit from participation,
analysis has sought not only to treat behavior as the but the primary goal of the research is a longer term
subject matter of a natural science, but also to “apply search for processes, principles, and procedures that
[the methods of science] to human affairs” (Skinner, apply beyond the individual case. Other distinctions
1953, p. 5). The goal of translational research in bio- include differences in typical research environ-
medical science is the treatment or prevention of ments, publication outlets, and funding models.
human disease. From the perspective of behavior At the T1 stage of translational behavior analysis,
analysis, human disease encompasses a range of mal- an important goal is to provide support for applica-
adaptive behavior. Examples include educational tions by validating underlying principles. Every
failures; behavior with high risk for health problems chapter in Volume 2, Part I, of this handbook
such as substance abuse or unsafe sex; disruptive, includes relevant T1 research findings that docu-
aggressive, and self-injurious behavior in individuals ment continuity in underlying behavioral processes
with developmental disabilities; and unhappiness along the continuum from the behavioral bench to
and loss of functioning in daily life related to depres- bedside. Why is this important? One reason is that
sion, anxiety, or similar conditions. As behavior ana- behavioral interventions do not always produce the
lysts look at human behavioral diseases, the Petri expected results, and a complete understanding of
dish is the pigeon in the operant chamber and simi- relevant underlying processes may improve the cli-
lar arrangements; the bedside includes the human nician’s or teacher’s ability to make informed proce-
operant laboratory, analogue environments of func- dural adjustments. In terms that Overmier and
tional analysis (Iwata, Dorsey, Slifer, Bauman, & Burke (1992) used to describe animal models for
Richman, 1982/1994), and so forth; and practice set- human diseases, a translational research pathway
tings abound for behavior analytically inspired treat- builds confidence that the relation between a set of
ments in schools, clinics, businesses, and virtually basic research findings and a set of applied proce-
anywhere where significant human activity occurs. dures is one of true material equivalence (i.e.,
The two volumes of this handbook describe much of homology of behavioral processes) rather than mere
this basic, translational, and applied research. conceptual equivalence (behavioral similarity).
I adapt one example from Volume 2, Chapter 7,
this handbook, by Jacobs, Borrero, and Vollmer. Sup-
Translational Behavior Analysis
pose that the results of a reinforcer preference test for a
Basic and applied behavior-analytic research have student receiving special education services showed
proceeded in parallel for many years, with varying that Reinforcer A was highly preferred to several others.
degrees of cross-fertilization; for a recent summary Yet when Reinforcer A was provided for completing
of the basic–applied interface, see Mace and increasing numbers of arithmetic problems, it became
66
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Translational Research in Behavior Analysis
ineffective at sustaining the behavior. A teacher might Behavior analysis has also addressed issues anal-
not even consider trying other reinforcers that were ogous to those of the T2 and T3 stages of biomedical
less preferred, based on the preference assessment translational research, and the chapters in Volume 2
results. An understanding of translational research in of this handbook include descriptions of these
behavioral economics, however, might lead the teacher efforts. Particular emphasis on T2 and T3 stages is
to question whether the problem was related to elastic- found in Volume 2, Chapters 1 and 18, this hand-
ity of demand—the degree of decrease in the value of a book. Volume 2, Chapter 10, this handbook, with
reinforcer as its price (the amount of work required to its emphasis on the application of known behavioral
obtain it) increases (for additional information, see the principles to bring about population-wide changes
discussion of essential value in Volume 2, Chapter 8, in behavior and cultural evolution, is a particularly
this handbook). The typical reinforcer assessment good example of the T3 stage of behavior analysis.
evaluates relative preference at very low behavioral
prices; the required response is usually merely reach-
Bidirectional Translation
ing or pointing. Translational research in behavioral
economics may suggest a reevaluation of relative rein- As McIlvane et al. (2011) noted, translational behavior
forcer preferences with response requirements that analysis need not be thought of as a one-way street.
more closely match those of the arithmetic task. Rein- Topics, methods, and strategies for basic research may
forcers less preferred than Reinforcer A at low prices be selected expressly to foster translation, and transla-
may become more preferred at higher prices. tional goals may be influenced by the practical realities
A second example may be drawn from Volume 2, of application research and intervention practice. To
Chapter 6, this handbook, on discrimination learn- list a few examples, in Volume 2, Chapter 5, this
ing and stimulus control. Relatively early research handbook, Nevin and Wacker explicitly call for
on stimulus equivalence called into question reverse translation from application to basic research
whether language ability was necessary to show pos- to provide an empirical base to aid in interpretation of
itive results on equivalence tests (e.g., Devany, factors influencing long-term treatment outcomes for
Hayes, & Nelson, 1986; for an introduction to stim- problem behavior. One impetus for the research in
ulus equivalence, see Chapter 16, this volume and stimulus control that McIlvane describes in Volume 2,
Volume 2, Chapter 1, this handbook). The transla- Chapter 6, this handbook, comes from the problems
tional research reviewed in Volume 2, Chapter 6, encountered in special education classrooms in the
this handbook, by McIlvane has shown, however, course of discrete-trials teaching. In their chapter on
that the typical conditional discrimination proce- acceptance and commitment therapy (ACT; Volume
dures used in equivalence research may engender 2, Chapter 18, this handbook), Levin et al. move from
unsuspected forms of stimulus control that meet the the clinic to the human operant laboratory for their
reinforcement contingencies (and thus produce high evaluations of the impact of various ACT components.
accuracy scores during initial training) but are nev- This approach allows them to evaluate the compo-
ertheless incompatible with equivalence relations nents with nonclinical populations and using labora-
because they do not require discrimination of all of tory measures such as task persistence or recovery
the stimuli presented (for details, see Volume 2, from mood inductions. The reader will find other
Chapter 6, this handbook). The susceptibility to examples in Volume 2, Part I, of the handbook.
these unsuspected forms of stimulus control may be
more prevalent in individuals at lower developmen-
Overview of Volume 2, Part I:
tal levels. With procedural enhancements and varia-
Translational Research in
tions designed to reveal or control for these
Behavior Analysis
possibilities, equivalence relations have been shown
in humans with minimal language (D. Carr, Wilkin- Arranging Reinforcement Contingencies
son, Blackman, & McIlvane, 2000; Lionello-DeNolf, In Volume 2, Chapter 3, this handbook, DeLeon,
McIlvane, Canovas, & Barros, 2008). Bullock, and Catania describe findings from basic
67
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
William V. Dube
and translational research that can inform the design and the behavioral requirements of the reinforcement
of reinforcement systems in applied settings. The contingency (e.g., response effort) are treated as the
chapter is divided into four major sections. In the price of that commodity. Changes in price may affect
first section, The Contingencies: Which Schedule demand for a commodity (typically determined by
Should We Use? DeLeon et al. consider reinforce- measuring consumption), and DeLeon et al. describe
ment schedules and include descriptions of the basic ways in which the relations between effort and rein-
ratio and interval schedules and their characteristics forcer effectiveness may have implications for rein-
as well as several other types of schedules useful in forcer use and selection in applied settings. DeLeon
application. These schedules include differential et al. conclude a discussion of a couple of tools for
reinforcement, in which contingencies may specify the applied researcher and clinician: a decision tree
certain defined response classes or low or high rates (Figure 3.1) to guide the selection of reinforcers for
of responding; response-independent schedules; and applied settings and a list of suggestions for trouble-
concurrent schedules, which operate simultaneously shooting in situations in which arranged reinforce-
and usually independently for two or more different ment contingencies do not have the desired effect.
response classes.
In the second section, The Response: What Operant Extinction: Elimination and
Response Should We Reinforce? DeLeon et al. focus Generation of Behavior
on the response and include discussions of research Procedurally, extinction refers to the discontinuation
using percentile schedules to shape response forms of reinforcement for a response. In Volume 2, Chap-
(e.g., increasing the duration of working on a task), ter 4, this handbook, Lattal, St. Peter, and Escobar
lag schedules to increase response diversity and vari- review research on extinction and its effects in both
ability, and embedding reinforcement contingencies the elimination and the generation of behavior.
within prompted response sequences. They also Those readers unfamiliar with behavior analysis may
include an interesting treatment of the punished-by- at first be surprised that extinction procedures and
rewards issue, in which some critics of behavior response generation are related, and Lattal et al. pro-
analysis have claimed that the use of extrinsic rein- vide an interesting introduction to this relation.
forcers reduces creativity and destroys intrinsic All of the chapters in Volume 2, Part I, of this
motivation (e.g., Kohn, 1993), and a summary of handbook describe both basic research in non–
research findings that refute this notion. human animal laboratories and applied research in
In the third section, The Reinforcer: Which Rein- clinical settings. Volume 2, Chapter 4, this hand-
forcer Should We Choose? DeLeon et al. consider book features a very tight integration of the two,
the clinician’s choice of which reinforcer to use. often in the same paragraph.
They include thorough discussions of research on The chapter begins with a brief review of the his-
preference assessment procedures and the design tory of the study of extinction, from the early 20th
and implementation of token reinforcement sys- century through Skinner’s early work. This review is
tems. Thus far, the translational research on token followed by clear procedural definitions of relevant
exchange schedules has indicated that token rein- terms and an introduction to the types of functional
forcers may maintain their effectiveness when rein- outcomes that may result from procedural extinc-
forcement is delayed better than the directly tion, including both the reduction in the response
consumable reinforcers for which they are that no longer generates reinforcement and its
exchanged, which is identified as a topic for future response-generating or response-inducing effects on
research in applied settings. other classes of behavior.
In the fourth section, Reinforcer Effectiveness and The remainder of the chapter is divided into two
the Relativity of Reinforcement, DeLeon et al. con- sections, one on the eliminative effects of extinction
sider the role of context in reinforcer effectiveness and one on the generative effects. In the former sec-
and include an introduction to behavioral econom- tion, Lattal et al. review the interactions of extinc-
ics, in which reinforcers are treated as commodities tion with schedules of reinforcement and other
68
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
parameters of reinforcement, the effects of repeated analyzing stimulus control in discrimination learn-
exposures to extinction, and the effects of sudden ing. His approach makes the topic accessible in part
versus gradual introduction; the latter section is rel- because it incorporates principles of programmed
evant to the gradual reduction in reinforcement fre- instruction, including a careful analysis and presen-
quency (“schedule thinning”) that is an important tation of prerequisites for the reader and a system-
component of many clinical treatment interventions. atic progression from simple to complex. This
In this section, Lattal et al. also review research on chapter will be of interest to students and more
several response-elimination procedures with high experienced readers alike, including many who do
relevance to applied behavior analysis. These proce- not consider themselves to be behavior analysts. In
dures include differential reinforcement for other fact, one of McIlvane’s explicit goals is to provide a
behavior, response-produced time outs (i.e., a sig- reference for students of cognitive neuroscience who
naled period in which a reinforcer is not available), use discrimination procedures as behavioral prepa-
and procedures that remove the response–reinforcer rations to study correlations between behavior and
dependency, often termed noncontingent reinforce- underlying neurobiological processes. His message
ment in applied work. In this section in particular, is that even seemingly straightforward procedures
Lattal et al.’s close integration of basic and applied may bring unsuspected analytical complexities, and
research provides a valuable resource for the failure to address these complexities will introduce
clinician. unmeasured sources of variability in the data.
The section on generative effects of extinction McIlvane adopts Sidman’s (1986) analytical units
begins by reviewing research on the extinction burst, of behavior analysis as a consistent framework for
a period of increased response rate that sometimes— the discussion. The initial section describes simple
but not always—follows the onset of extinction. discrimination procedures in terms of three-term
Most research on generative effects is reviewed in analytical units that correspond to the three terms of
terms of increased variability in the topography of the operant contingency: antecedent stimulus,
the target response, increased rate of responses that behavior, and consequence. When conditional dis-
were previously unreinforced but topographically crimination, which requires relational stimulus con-
related to the reinforced response (e.g., behavior that trol by two (or more) stimuli (e.g., sample and
did not meet a defined response criterion), and comparison stimuli in a matching-to-sample proce-
schedule-induced behavior, which includes behavior dure) is considered, the analytic unit is expanded to
topographically unrelated to the target response that four terms that include two antecedent stimuli. At
occurs during the periods of extinction in intermit- each level of analysis, McIlvane carefully explains
tent reinforcement schedules. Research with humans and illustrates analyses in terms of select versus
has documented schedule-induced behavior, includ- reject stimulus control (e.g., as exerted by the cor-
ing drinking and stereotypy, and schedule-induced rect and incorrect choices in a discrimination task)
responding has been suggested as a mechanism and configural versus relational stimulus control
related to drug taking and smoking. In this section, (e.g., as exerted by two or more stimuli as a unitary
Lattal et al. also review research on resurgence, compound in the former and by independent stimu-
which is the recurrence of previously reinforced lus elements in the latter).
responding when a more recently reinforced Notably, McIlvane always makes a careful dis-
response is extinguished and the recovery of behav- tinction between the terms of the procedure, as
ior after periods of extinction by the presentation of defined by the experimenter, and the actual analyti-
discriminative, contextual, or reinforcing stimuli. cal units of behavior that those procedures engender
in the organism. One of the most important points
Simple and Complex Discrimination of the chapter is that these two sets of specifications
Learning need not always correspond. I use an oversimplified
In Volume 2, Chapter 6, this handbook, McIlvane example here for brevity: A special education
systematically lays out the complexities involved in teacher may assume that accurate performance on a
69
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
William V. Dube
matching-to-sample task in which printed-word Nevin and Wacker open with a discussion of the
samples BALL and BAT are matched to correspond- concept of response strength and basic research,
ing pictures indicates that two word–picture rela- showing that response rate and persistence (resis-
tions have been learned. As McIlvane points out, tance to change) are independent aspects of behavior.
however, the student may have learned to select the Response rate is determined by response–reinforcer
ball picture when the sample was BALL and reject contingencies (schedules of reinforcement), and
the ball picture when the sample was BAT, a perfor- resistance to change is determined by the stimulus–
mance that does not include any stimulus control at reinforcer relation—that is, the characteristic
all by the specific features of the bat picture. If per- reinforcer rate within a given stimulus situation—
formance that depends on control by those features independent of response contingencies. This latter
of the bat picture is poor in subsequent testing for point, that persistence is determined by the stimulus–
more advanced relational learning (e.g., stimulus reinforcer (Pavlovian) relation and independent of
equivalence), the teacher may erroneously conclude response–reinforcer (operant) contingencies, has
that the student is not capable of the more advanced important implications for applied behavior analy-
performance, when in fact the problem was the sis, and Nevin and Wacker include a clear explana-
teacher’s failure to fully analyze the stimulus– tion of the basic research supporting it (e.g., Nevin,
control basis for the initial training. McIlvane dis- Tota, Torquato, & Shull, 1990).
cusses situations such as this one in terms of a lack Why is the distinction between rate-governing
of coherence between the teacher’s (or experiment- and persistence-governing environmental relations
er’s) assumptions about stimulus control and the important? One answer is that applied behavior
actual controlling stimuli and relations. analysis has developed several successful approaches
The final sections of McIlvane’s chapter include a for the treatment of problem behavior in popula-
brief description of the current state of behavior- tions with developmental limitations. In such popu-
analytic theory on the acquisition of stimulus con- lations, verbal behavior is often unreliable, and so
trol via differential reinforcement and some ideas the interventions depend on direct manipulation of
about how theory might be advanced. This discus- reinforcement contingencies. Procedures such as
sion touches on the issue of improving the coher- differential reinforcement of other behavior, differ-
ence between the desired stimulus control (by the ential reinforcement of alternative behavior, and
experimenter, teacher, etc.) and the stimulus control response-independent reinforcer deliveries (often
that is actually captured by the contingencies. McIl- termed noncontingent reinforcement) are all designed
vane concludes the chapter with a consideration of to reduce the frequency of a problem behavior, and
two of the current areas of translational research in they all accomplish it by increasing the overall rate
stimulus control: stimulus control shaping (e.g., by of reinforcement in treatment settings. A wealth of
gradual stimulus change procedures) and the analy- applied research has shown that these procedures
sis of relational learning processes as seen, for exam- have been broadly effective in reducing the rates of
ple, in stimulus equivalence. problem behavior. Evidence, however, including
some presented in Nevin and Wacker’s chapter, has
Response Strength and Behavioral shown that they may do so at the cost of increasing
Persistence the longer term persistence of the problem behavior
Behavioral momentum theory makes an analogy when the inevitable challenges to treatment (e.g.,
between behavior and physical momentum: brief periods of extinction) are encountered over
Response rate is analogous to the velocity of a mov- time (e.g., Mace et al., 2010). That longer term per-
ing body, and an independent aspect of behavior sistence may be manifested in a posttreatment resur-
analogous to inertial mass is inferred from the per- gence of problem behavior.
sistence of response rate to disruption by some chal- Long-term maintenance of treatment gains is a
lenge analogous to an external force applied to a relatively understudied area in applied behavior
moving body. In Volume 2, Chapter 5, this handbook, analysis, and the latter portion of Nevin and Wacker’s
70
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
chapter outlines a new approach to the issue, are in applied research and practice. As Commons
inspired by behavioral momentum theory. In cur- (2001) has noted, however, quantitative analysis “is
rent practice, maintenance is often defined in terms not primarily a matter of fitting arbitrary functions
of continuing the desired behavior change over time to data points. Rather, each parameter and variable
and under the prevailing conditions of treatment. in a model represents part of a process that has
Nevin and Wacker propose that this step toward theoretical, empirical, and applied interpretations”
maintenance is necessary but not sufficient. They (p. 275). Throughout the chapter, Jacobs et al. dis-
redefine maintenance as the persistence of treatment cuss many translational research studies that illus-
gains when confronted by changes in antecedent trate the relationships between the analyses’
stimuli (people, tasks, prompts, etc.) and the conse- parameters and variables and events in socially
quences of behavior: “Rather than focusing almost relevant human behavior, both in and outside of
exclusively on behavior occurring under stable the laboratory.
treatment conditions, researchers should also con- The first of two major sections focuses on the
sider how various treatment conditions produce or matching law, which relates the relative allocation of
inhibit persistence during challenges to the treat- behavior between and among concurrently available
ment” (p. 124). Nevin and Wacker present a longitu- options to the relative value of the obtained rein-
dinal analysis of the effects of extinction challenges forcement for those behavioral options, where value
to treatment over an average of 35 weeks for eight is the product of rate, quality, amount (magnitude),
children. For example, after differential reinforce- and immediacy (delay). One of the major contribu-
ment of alternative behavior had replaced one child’s tions of the matching law is that it puts behavior in
destructive behavior with appropriate requesting, the context—the behaving organism always has a choice
requests were ignored during occasional brief extinc- between engaging in some behavior of interest to the
tion periods. The data from these extinction chal- behavior analyst or doing something else. Seen from
lenges show gradually increasing persistence of the perspective of the behavior analyst, therefore,
adaptive behavior and decreasing destructive behav- there is also a choice: To increase the behavior of
ior over a 7-month intervention period. interest, one may increase relative reinforcement
Nevin and Wacker conclude the chapter with contingent on that behavior or decrease reinforce-
some suggestions for further research, which ment available for alternative behavior, or both.
include some interesting reverse translation possi- The matching law section opens with a brief
bilities for the basic research laboratory to examine account of the development of the mathematical
strengthening and weakening of behavioral persis- model, accompanied by text relating the mathemati-
tence over long time courses and the effects of varia- cal terms to aspects of behavior. Helpful figures
tion in the stimulus situation. Another goal for illustrate the effects of changing the values of the
further research is to determine the extent to which terms in the corresponding equations. This intro-
the underlying behavioral processes for “high-p” duction is followed by a review of research showing
procedures (Mace et al., 1988), used in applied the applicability of the matching law to such diverse
research to increase compliance, can be related to human activities as choice between conversational
those of behavioral momentum theory. partners among college students, between academic
tasks among students with academic delays,
Translational Applications of between problem and appropriate behavior among
Quantitative Choice Models children with intellectual and developmental dis-
Jacobs, Borrero, and Vollmer’s goal in Volume 2, abilities, and even between 2- and 3-point shots
Chapter 7, this handbook is “to foster appreciation among basketball players. Areas identified for fur-
of quantitative analyses of behavior for readers ther research on choice include the effects of delay
whose primary interests are in applied research and to reinforcement for responses that closely follow a
practice” (p. 165). Quantitative analysis may at first shift from one behavioral option to another (change-
seem a bit opaque to some whose primary interests over delay), delay to reinforcement for responses
71
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
William V. Dube
during extended periods of behavior, and analytic describe how a common interest in value and choice
tools to better account for ratio- versus interval-like in the fields of economics and behavioral psychol-
aspects of reinforcement schedules in uncontrolled ogy provide a context for (a) the extension of micro-
environments. economic theory to the consumption of reinforcers
The second major section of the chapter covers in organisms and (b) the application of operant con-
temporal discounting, which describes the relation- ditioning principles to the economics of demand for
ships between the impact of a reinforcer and the commodities.
amount of time that will pass before that reinforcer I highly recommend the overview of behavioral
is obtained. This research area is relevant to issues economic concepts, regardless of the reader’s level
involving human impulsivity and self-control and of familiarity with the topics. There is something
maladaptive choices to engage in behavior that pro- here for everyone. The material is presented in a
duces a relatively smaller immediate reinforcer (e.g., clear and balanced discussion that covers areas of
second helping of chocolate cake) at the cost of fore- consistency and agreement as well as apparent
going a relatively larger delayed reinforcer (weight exceptions and thus possible issues for further
control for better health and appearance). The text research. The discussion is illustrated with examples
and accompanying figure explain how temporal dis- from both basic and translational research. Among
counting is well described by a hyperbolic function, the most important concepts reviewed are demand,
how this function helps to account for impulsive value, and discounting. As noted earlier, behavioral
choice of the smaller, more immediate reinforcer, economics treats reinforcers as commodities and the
and how the methods can be used to determine a behavior required to obtain a commodity as its
quantitative value describing the degree to which price. Of primary interest is how changes in price
the individual will discount a reinforcer (i.e., how affect demand. Demand is measured in terms of con-
steeply the value of a reinforcer will decrease as the sumption, and Hursh et al. discuss the distinction
delay to obtain it increases). Jacobs et al. then go on between total consumption as a fundamental depen-
to review research in assessment of impulsivity in dent variable and other common behavior-analytic
adults and in children, both those who are typically dependent variables such as response rate.
developing and those with developmental disabili- As price increases, consumption decreases.
ties. Research on strategies for decreasing impulsiv- Hursh et al. describe the essential value of a com-
ity is also reviewed. The section concludes with a modity (reinforcer) in terms of the rate of change in
discussion of the potential for future research to consumption as price increases, that is, “an organ-
develop discounting assessments that could identify ism’s defense of consumption in the face of con-
developmental markers and risk for problems asso- straint. Commodities that are most vociferously
ciated with impulse control. defended are the most essential” (pp. 196–197). In
The remainder of the chapter provides brief over- the Quantifying Value From Demand Curve Data
views of several other quantitative approaches and section, Hursh et al. discuss methods for obtaining
models relevant to translational research in behavior quantitative estimates of value based on rate of
analysis: behavioral economics (addressed in Volume change in consumption or, alternatively, the price
2, Chapter 8, this handbook), behavioral ecology, point that supports peak responding. Also covered
behavioral momentum (addressed in Volume 2, are ways in which the availability and price of alter-
Chapter 5, this handbook), and behavioral detection nate commodities may affect essential value.
models based on signal detection theory. Each of When the delivery of a reinforcer is delayed, its
these sections provides helpful introductory material. value decreases in relation to the duration of the
delay; that is, the value is discounted. Quantitative
Behavioral Economics analyses of discounting have revealed a very interest-
The first half of Volume 2, Chapter 8, this handbook ing and significant difference between the discount-
includes an introduction and overview of several ing function predicted by normative economic theory
important concepts. In the introduction, Hursh et al. and that actually obtained by the experimental
72
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
analysis of behavior. The former is exponential, in (Although not discussed in the chapter, this
which a reinforcer is devalued at a constant rate over approach seems at least conceptually similar to the
time, and the latter is hyperbolic, in which the deval- bundling of distributed social reinforcers in 12-step
uation rate is more extreme in the immediate future programs.) Research on reward bundling has yet to
than in the more distant future. This exponential ver- be accomplished in applied settings. A third
sus hyperbolic difference is illustrated in the top por- approach, training in skills related to executive
tion of Hursh et al.’s Figure 8.9. Because of the functioning (problem solving, strategic planning,
acceleration in value as a reinforcer becomes more working memory), is also in the earliest stages of
immediate, choice may suddenly shift from a larger– translational research.
later reinforcer (e.g., losing weight for better health In the final section of the chapter, Translating
and appearance) to a smaller–sooner one (e.g., an Behavioral Economic Principles to Other Applied
imminent piece of apple pie). Included in the chapter Settings, Hursh et al. describe translational research
is a very accessible discussion of how the discovery with individuals who have autism, intellectual and
of hyperbolic temporal discounting provides a scien- developmental disabilities, or both. Hursh et al.
tifically grounded explanation for seemingly irratio- cover the assessment of value in such populations
nal choices and preference reversals. and also describe some approaches to scheduling
The second half of the chapter reviews transla- reinforcers in ways that may help to maintain
tional research in behavioral economics with atten- demand as price rises. The section concludes with
tion to areas with potential for further research and some behavioral economic considerations of the
development. Much of the translational activity thus marketplace in which problem behavior occurs and
far has been related to analyses and treatment for some issues to be considered when introducing
addictions. One set of questions asks whether analy- therapeutic reinforcers into this marketplace.
ses of demand characteristics and essential value can
be used to predict response to treatment interven- Applied Behavior Analysis and
tions by behavioral therapy, response to treatment Neuroscience
by medication, and the transition from inpatient to In Volume 2, Chapter 2, this handbook, Travis
outpatient treatment. Thompson examines the pathway from basic labo-
In the Translating Delay Discounting section, ratory research to application “at the interface of
Hursh et al. look at some potentially promising behavior analysis and neuroscience” (p. 33). The
intervention approaches that address the person first section of the chapter, Behavior Analysis in
with addiction’s discounting of the long-term bene- Physiological Psychology, focuses on behavioral
fits that would result from recovery. One approach pharmacology, with an emphasis on drug addic-
is to teach tolerance for delay; Hursh et al. judge this tion. The historical material discussed in this sec-
approach to be at the proof-of-concept stage, with tion includes eyewitness accounts of the early work
much work remaining to be done. Another approach with nonhuman primates from one of the pioneers
has been termed reward bundling (e.g., Ainslie, in the field. There is an emerging understanding
2001): Rather than a choice between two isolated that addiction involves not only the biochemical
reinforcers (one smaller–sooner and one larger– and physiological effects of drugs but also their
later), the choice is between two series (bundles) of interactions with reinforcement contingencies. The
reinforcers distributed in time. The theory is that translational path proceeds to the merger of behav-
the bundles will affect choice in relation to the sums ioral pharmacology and applied behavior analysis
of the discounted values of all reinforcers in the in the treatment of substance abuse, and the chap-
series. That is, the difference in value of the bundles ter includes a review of some recent research in
will be greater than the difference in value of two this area.
single reinforcers, and this greater disparity will more In the Establishing Operations: Food Motivation
than offset the immediacy of the first reward in the section, Thompson examines research on the effects
smaller–sooner series (see Hursh et al.’s Figure 8.14). of neuropeptide Y, which increases food intake but
73
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
William V. Dube
apparently via an underlying mechanism distinct high-priority chemicals whose structure suggests
from that of food deprivation. In this section, significant potential for toxicity and for which com-
Thompson reviews functional magnetic resonance mercial use exceeds 500 tons per year, fewer than
imaging (MRI) research of brain activity associated one third have undergone any neurotoxicity testing.
with food motivation and evidence that typical pat- Newland goes on to outline behavioral neurotoxico-
terns of neural activity begin in childhood. Also dis- logical testing methods and analysis criteria using
cussed is research comparing pre- and postmeal both operant and Pavlovian contingencies. These
activation while viewing food pictures in typical methods are illustrated with examples that include
control participants and individuals with Prader- high levels of exposure to manganese (e.g., in unsafe
Willi syndrome, the most common known genetic mining conditions), solvents, ozone, electrical fields,
cause of life-threatening obesity in children. The and others. The value of individual subject data is
research points to distinct neural mechanisms asso- illustrated with Cory-Slechta’s (1986) results on the
ciated with food intake in Prader-Willi syndrome. effects of developmental exposure to lead on fixed-
In three of the remaining sections, Thompson interval responding (see Newland’s Figure 9.3 and
reviews translational research in three areas: accompanying text).
(a) treatments for some forms of self-injurious Given the nature of the subject matter, toxicity,
behavior in developmental disabilities that are testing in nonhuman animals is a necessity. One of
maintained by the release of endogenous opioids; the important contributions of this chapter is New-
(b) changes in the motor regions of the brain associ- land’s Human Testing section, in which he explores
ated with constraint-induced rehabilitation, in problems with and solutions for comparing the
which operant contingencies are used to encourage results of studies with nonhuman laboratory animals
people with stroke to use affected limbs; and with those conducted with human participants. One
(c) functional MRI studies of the effects of exposure solution to this problem is to develop tests that can
to operant contingencies of reinforcement on subse- be used among both populations. This approach is
quent viewing of discriminative stimuli and mem- particularly useful when the test results in humans
bers of stimulus equivalence classes. correlate well with more general measures of func-
Thompson also includes an intriguing section on tioning; for example, the correlation of performance
a possible relation between synaptogenesis and suc- on an incremental repeated acquisition test with that
cess in early intensive behavioral intervention (EIBI) on a standardized intelligence (IQ) test, described
in autism. Although EIBI has been very successful in by Newland. Another approach involves the elimi-
some cases, the gains are rather modest for approxi- nation or minimization of verbal instruction with
mately half of treated children. Thompson reviews humans, which also facilitates comparisons among a
evidence that synapse formation can be activity diverse array of human populations and cultural
dependent and raises the question of whether indi- groups (e.g., migrant laborers with occupational
vidual differences in synaptogenesis in response to exposure to pesticides).
the operant reinforcement contingencies of EIBI— In the Mechanisms and Interventions section,
differences that may have a genetic basis—could be Newland considers deficits in motor function and
related to the ultimate degree of success. This area the role of reinforcement contingencies in recovery
seems to be a very fertile one for future research. of function (a topic Thompson also addresses in his
chapter). Research on the development of tolerance
Environmental Health and Behavior to neurotoxicants and adjustment to impairment is
Analysis also described as well as disruption of behavioral
Newland begins his chapter (Volume 2, Chapter 9, allocation in behavioral choice procedures (which
this handbook) with a few sobering facts. For exam- are described in more detail in Volume 2, Chapter 7,
ple, of the tens of thousands of chemicals in pro- this handbook) and behavioral flexibility as mea-
duction, only a small fraction have been subjected sured by discrimination reversal procedures after
to rigorous tests for neurotoxicity. Even among gestational exposure to lead or methylmercury.
74
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Newland goes on to describe how these effects may of private events (such as thoughts and feelings) as
be understood in terms of distortion in the impact of behavior itself rather than as causes of behavior and
reinforcers and disrupted dopamine function. the functional (as opposed to formal) analysis of
In the remainder of the chapter, Newland pro- verbal behavior as operant behavior controlled by its
vides an education in the scientific and organiza- audience-mediated consequences. This part of the
tional problems that must be solved to conduct an chapter is divided into three sections: Rule Gover-
evidence-based risk assessment. He describes a pro- nance, Equivalence Relations and the Transfer of
cess for deriving estimates of tolerable human expo- Stimulus Functions, and Other Stimulus Relations
sures from controlled laboratory studies of animals and the Transformation of Functions. Rule gover-
and from epidemiological studies of exposed nance is the study of interactions in behavioral con-
humans, in an open and transparent manner. To trol between antecedent stimuli in the form of verbal
help meet the need to advance the pace of testing, a rules or instructions that describe contingencies
current emphasis in the area of in vitro testing (for- (including self-generated) and the actual conse-
mation of proteins, activity of cells, etc.) is on “high- quences for behavior; the rules may or may not
throughput” techniques for rapidly identifying and accurately reflect the contingencies. The implica-
characterizing neurotoxicity in large-scale efforts. tions for CBA of self-generated rules at odds with
An important challenge identified for the next gen- the actual contingencies of daily life seem evident.
eration of behavioral toxicologists is the develop- Stimulus equivalence refers to stimulus–stimulus
ment of meaningful high-throughput behavioral relations of mutual substitutability (for a basic
tests; as Newland notes, “One cannot reproduce the research review, see Chapter 16, this volume). For
[fixed-interval] schedule in a dish” (p. 245). example, the spoken word dog and the printed word
DOG may be related by equivalence if both control
From Behavioral Research to Clinical pointing to a picture of a dog (but see Volume 2,
Therapy Chapter 6, this handbook, for some of the complexi-
Clinical behavior analysis (CBA) refers to that ties of a thorough analysis). In the Equivalence Rela-
branch of applied behavior analysis that addresses tions and the Transfer of Stimulus Functions
what are commonly known as mental health issues section, Guinther and Dougher point out that the
in verbally competent adult outpatients: anxiety dis- most clinically interesting feature of stimulus equiv-
orders, depression, adjustment disorders, and so alence is transfer of function: Stimulus functions
forth. In Volume 2, Chapter 1, this handbook, acquired by direct conditioning with one stimulus
Guinther and Dougher provide an overview of the will also be manifested by other equivalent stimuli
historical development of CBA and describe transla- in the absence of direct experience. Equivalence
tional research in relation to specific CBA therapies. relations and transfer of function in verbal stimuli
The historical overview begins with the behavior help to explain why words can provoke emotional
therapy movement of the 1950s and 1960s and the responses, and the research reviewed in this section
rise to prominence of cognitive–behavioral therapy documents how stimuli can come to elicit emotions
in the 1970s, including a discussion of why the or evoke avoidance behavior via equivalence rela-
mentalistic aspects of cognitive behavior therapy tions and in the absence of any direct conditioning.
removed it from behavior analysis. Goldiamond’s The subsequent section, Other Stimulus Rela-
(1974/2002) constructional approach is credited as tions and the Transformation of Functions, on rela-
the first fully articulated behavior-analytic outpa- tional frame theory (RFT) provides an exceptionally
tient therapy, and the development of the more clear description of the relation between stimulus
modern CBA therapies over the next 30 years is equivalence and RFT. RFT expands the study of
briefly noted. stimulus–stimulus relations beyond those of equiva-
The Translational Research Relevant to CBA sec- lence to include opposition, more than–less than,
tion is preceded by a thoughtful introduction to the before–after, and many others. The research
conceptual basis of CBA in the Skinnerian analyses reviewed has indicated that the transfer of functions
75
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
William V. Dube
seen in stimulus equivalence may be modified depression; and a brief introduction to ACT for the
accordingly by other types of stimulus–stimulus treatment of a wide variety of quality-of-life issues
relations and is thus termed transformation of func- (brief because ACT is the subject of Volume 2,
tion. For example, when initial training established Chapter 18, this handbook).
more-than–less-than relations among stimuli such
that A < B < C, and B subsequently predicted elec- Acceptance and Commitment Therapy
tric shock in a Pavlovian conditioning procedure, In Volume 2, Chapter 18, this handbook, Levin,
experimental participants showed smaller skin con- Hayes, and Vilardaga present an in-depth look at
ductance changes to A and larger skin conductance ACT, which is arguably on the leading edge of clini-
changes to C than to B, even though there was no cal behavior analysis. ACT developed in concert
direct conditioning with the A and C stimuli with RFT, and in the introductory sections of the
(Dougher, Hamilton, Fink, & Harrington, 2007). chapter, Levin et al. present a detailed account of its
The section that follows, Verbal Behavior and the development. They describe functional contextual-
Clinical Relevance of RFT, considers the wide- ism, the philosophical foundation of RFT and ACT,
ranging implications for clinical relevance of RFT along with a related development strategy termed
applied to verbal behavior. contextual behavioral science. A key aspect of this
In the remainder of the chapter, Guinther and approach that seems distinct from mainstream
Dougher review the major CBA therapies in current behavior analysis is described as “user-friendly anal-
practice. In the CBA Therapies section, they begin yses . . . that practitioners can access without fully
by discussing Goldiamond’s (2002) constructional grasping the details of the basic account, while a
approach, which is foundational in both historical smaller number of scientists work out technical
and conceptual senses. As an editorial note, I think accounts” (p. 462).
that the importance of Goldiamond’s contributions The unifying conceptual system for ACT at an
to applied behavior analysis in general cannot be applied level is the psychological flexibility model,
overemphasized. To list a few examples, the diag- which is based on six key processes, each of which
nostic technique of descriptive analysis (e.g., McCo- is defined and explained: defusion interventions,
mas, Vollmer, & Kennedy, 2009), treatment acceptance interventions, contact with the present
interventions such as functional communication moment, self-as-context (perspective taking), values
training that focus on building appropriate reper- (motivational context), and committed action. A
toires that produce the same class of reinforcers as case example is presented that illustrates how these
problem behavior (e.g., E. G. Carr & Durand, processes are applied to case conceptualization and
1985), and the focus on repertoire building rather treatment in ACT, with the goal of significantly
than symptom reduction in ACT (e.g., Hayes, Stro- improved psychological flexibility.
sahl, & Wilson, 1999) owe conceptual debts to In the next section of the chapter, Expanded Use
Goldiamond. of Methodologies to Test the Theoretical Model and
The subsections describing the therapies are very Treatment Technology, Levin et al. describe a body
clearly labeled, and the reader is referred to of evidence supporting the efficacy of ACT. As
Guinther and Douglas’s chapter itself for clear intro- guided by the contextual behavioral science
ductions to each one and related efficacy research. approach, the treatment technologies and underly-
These therapies include functional analytic psycho- ing theoretical models have been evaluated by a
therapy, which is helpful for improving the relation- variety of methodologies, including group designs.
ship between therapist and patient; integrative Levin et al. describe microcomponent studies that
behavioral couples therapy for couples involved in are typically small scale and laboratory based and
romantic relationships; dialectical behavior therapy often use nonclinical populations and focus on
for the treatment of borderline personality disorder “broadly relevant features of behavior such as task
and other severe problems; behavioral activation persistence and recovery from mood inductions”
therapy, with a primary emphasis on treatment of (p. 470). The results of more than 40 such
76
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
microcomponent studies “have suggested that many within the social system of a school, one product of
of these [ACT] components are psychologically educational interlocking behavioral contingencies is
active” (p. 470). Also described is research examin- students’ academic repertoires. The term metacontin-
ing the processes of change, and results in this area gencies describes the contingent relations between
show that ACT affects theoretically specified pro- interlocking behavioral contingencies and their
cesses and that changes in these processes predict products and the consequent actions of the external
changes in treatment outcomes. One of the most environment on the social system.
salient characteristic of ACT is its broad applicabil- Biglan and Glenn argue that when the cumulative
ity. This section on evaluative research concludes effects of macrobehavior are detrimental to society
with a summary of research showing ACT’s impact and human well-being, the macrobehavior should be
on an impressive array of problem areas including treated as a behavioral public health problem. Exam-
depression, anxiety, psychosis, chronic pain, sub- ples of such detrimental macrobehaviors include
stance use, coping with chronic illness, weight loss substance abuse, academic failure, and crime.
and maintenance, burnout, sports performance, and Because research from several disciplines has related
others. a small set of non-nurturing conditions to problems
In the final section of the chapter, Effectiveness, of public health and behavior, Biglan and Glenn pro-
Dissemination, and Training Research, Levin et al. pose an approach that focuses on increasing the
describe the active role of contextual behavioral sci- prevalence of nurturing environments. They describe
ence researchers in studying issues related to the ways to promote such environments that include
effective implementation of ACT. The results of this (among others) minimizing toxic and aversive con-
research have shown that training in ACT improves ditions, reinforcing prosocial behavior, and setting
patient outcomes, and these results are also shaping limits on opportunities for problem behavior.
the development of methods for training clinicians Biglan and Glenn then present two extended
in the intervention approach. examples of planned interventions that operate at
the macrobehavioral level to make changes in social
Prosocial Behavior and Environment systems by promoting nurturing environments. The
in a Public Health Framework first is the schoolwide Positive Behavior Supports
In Volume 2, Chapter 10, this handbook, Biglan and (PBS) program, which as of 2010 had been adopted
Glenn propose a framework to relate known behav- by more than 9,500 U.S. schools. They delineate the
ioral principles as they affect individuals to the cul- foundational constructs of PBS and emphasize the
tural practices of society as a whole, for the purpose importance of multilevel organizational support at
of making general improvements in human well- state, district, and school levels. From a macrobe-
being. In the opening section of the chapter, Behav- havioral perspective, they point out the need for
ior Analysis and Cultural Change, they define research on the specific factors that influence the
macrobehavior as the similar but independent behav- spread of PBS.
ior of many individuals that has a cumulative effect The second example of an organized effort to
on the environment. A fundamental difference change macrobehavior is the tobacco control move-
between behavior and macrobehavior is that the for- ment. This section includes an absorbing look at
mer describes a lineage of recurring responses of an the interlocking behavioral contingencies of the
individual that is shaped and maintained by its con- tobacco industry as a social system and how tobacco
sequences over time. In contrast, macrobehavior is marketing resulted in smoking by approximately
an aggregate of the operant behavior of many indi- half of men and one third of women by the middle
viduals and not controlled by consequences at the of the 20th century. Biglan and Glenn also describe
macrobehavioral level. the activities of the tobacco control movement,
The behavioral components of a social system are resulting in a macrobehavioral change that reduced
described as interlocking behavioral contingencies smoking behavior by approximately half during the
that produce a measurable outcome; for example, second half of the century.
77
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
William V. Dube
Biglan and Glenn conclude with a review of and language-disabled children. Journal of the
evidence-based programs and practices that may be Experimental Analysis of Behavior, 46, 243–257.
doi:10.1901/jeab.1986.46-243
marshaled to produce system-level changes in pub-
lic health and related policies. In their words, Dougher, M. J., Hamilton, D. A., Fink, B. C., &
Harrington, J. (2007). Transformation of the discrim-
The next challenge is to develop an inative and eliciting functions of generalized rela-
tional stimuli. Journal of the Experimental Analysis of
empirical science of intentional cultural Behavior, 88, 179–197. doi:10.1901/jeab.2007.45-05
evolution. . . . We argue that this effort
Goldiamond, I. (2002). Toward a constructional
will strengthen with a focus on (a) how approach to social problems: Ethical and constitu-
to influence the spread of a behavior by tional issues raised by applied behavior analysis.
increasing the incidence of reinforcement Behavior and Social Issues, 11, 108–197. (Original
for that behavior and (b) how to alter the work published 1974)
metacontingencies for organizations that Hayes, S. C., Strosahl, K., & Wilson, K. G. (1999).
Acceptance and commitment therapy: An experiential
select practices contributing to a more
approach to behavior change. New York, NY: Guilford
nurturing society. (p. 270) Press.
Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., &
Summary Richman, G. S. (1994). Toward a functional analysis
of self-injury. Journal of Applied Behavior Analysis, 27,
During his term as editor of the Journal of Applied 197–209. (Original work published 1982)
Behavior Analysis, Wacker (1996) pointed out the Kohn, A. (1993). Punished by rewards. Boston, MA:
“need for studies that bridge basic and applied Houghton Mifflin.
research” (p. 11). The research reviewed in the Lionello-DeNolf, K. M., McIlvane, W. J., Canovas, S. D. G.,
chapters I have discussed is certainly responsive to & Barros, R. S. (2008). Reversal learning set and
functional equivalence in children with and without
that need. It encompasses the T1, T2, and T3 range autism. Psychological Record, 58, 15–36.
of translations described in biomedical science, and
Mace, F. C., & Critchfield, T. S. (2010). Translational
the bridge has firm abutments in both the basic and research in behavior analysis: Historical tradi-
the applied research literatures. tions and imperative for the future. Journal of the
doi:10.1901/jeab.2010.93-293
References
Mace, F. C., Hock, M. L., Lalli, J. S., West, P. J., Belfiore,
Ainslie, G. (2001). Breakdown of will. Cambridge,
P., Pinter, E., & Brown, D. K. (1988). Behavioral
England: Cambridge University Press.
momentum in the treatment of noncompliance.
Carr, D., Wilkinson, K. M., Blackman, D., & McIlvane, Journal of Applied Behavior Analysis, 21, 123–141.
W. J. (2000). Equivalence classes in individu- doi:10.1901/jaba.1988.21-123
als with minimal verbal repertoires. Journal of the
Experimental Analysis of Behavior, 74, 101–114. Mace, F. C., McComas, J. J., Mauro, B. C., Progar, P.
doi:10.1901/jeab.2000.74-101 R., Taylor, B., Ervin, R., & Zangrillo, A. N. (2010).
Differential reinforcement of alternative behavior
Carr, E. G., & Durand, V. M. (1985). Reducing behavior increases resistance to extinction: Clinical demon-
problems through functional communication train- stration, animal modeling, and clinical test of one
ing. Journal of Applied Behavior Analysis, 18, 111–126. solution. Journal of the Experimental Analysis of
doi:10.1901/jaba.1985.18-111 Behavior, 93, 349–367. doi:10.1901/jeab.2010.93-349
Commons, M. L. (2001). A short history of the society for McComas, J. J., Vollmer, T., & Kennedy, C. (2009).
quantitative analyses of behavior. Behavior Analyst Descriptive analysis: Quantification and examina-
Today, 2, 275–279. tion of behavior–environment interactions. Journal of
Cory-Slechta, D. A. (1986). Vulnerability to lead at later Applied Behavior Analysis, 42, 411–412. doi:10.1901/
developmental stages. In N. Krasgenor, D. B. Gray, jaba.2009.42-411
& T. Thompson (Eds.), Advances in behavioral phar- McIlvane, W. J., Dube, W. V., Lionello-DeNolf, K. M.,
macology: Developmental behavioral pharmacology Serna, R. W., Barros, R. S., & Galvão, O. F. (2011).
(pp. 151–168). Hillsdale, NJ: Erlbaum. Some current dimensions of translational behavior
Devany, J. M., Hayes, S. C., & Nelson, R. O. (1986). analysis: From laboratory research to intervention
Equivalence class formation in language-able for persons with autism spectrum disorders. In
78
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
E. A. Mayville & J. A. Mulick (Eds.), Behavioral foun- Skinner, B. F. (1953). Science and human behavior. New
dations of effective autism treatment (pp. 155–181). York, NY: Macmillan.
Cornwall-on-Hudson, NY: Sloan.
upFRONT. (2006). Translational science. Philadelphia:
Mold, J. W., & Peterson, K. A. (2005). Primary care Penn Nursing, University of Pennsylvania. Retrieved
practice-based research networks: Working at the from http://www.nursing.upenn.edu/about/
interface between research and quality improvement. Documents/UpFront_8.30.pdf
Annals of Family Medicine, 3(Suppl. 1), S12–S20.
Wacker, D. P. (1996). Behavior analysis research in JABA:
doi:10.1370/afm.303 A need for studies that bridge basic and applied
Nevin, J. A., Tota, M. E., Torquato, R. D., & Shull, R. L. research. Experimental Analysis of Human Behavior
(1990). Alternative reinforcement increases resis- Bulletin, 14, 11–14.
tance to change: Pavlovian or operant contingencies? Westfall, J. M., Mold, J., & Fagnan, L. (2007). Practice-
Journal of the Experimental Analysis of Behavior, 53, based research—“Blue highways” on the NIH road-
359–379. doi:10.1901/jeab.1990.53-359 map. JAMA, 297, 403–406. doi:10.1001/jama.297.4.403
Overmier, J. B., & Burke, P. D. (1992). Animal models Woolf, S. H. (2008). The meaning of translational
of human pathology: A quarter century of behavioral research and why it matters. JAMA, 299, 211–213.
research. Washington, DC: American Psychological doi:10.1001/jama.2007.26
Association.
World Health Organization. (2004). World report on knowl-
Sidman, M. (1986). Functional analysis of emergent
edge for better health. Geneva, Switzerland: Author.
verbal classes. In T. Thompson & M. D. Zeiler
(Eds.), Analysis and integration of behavioral units Zerhouni, E. (2003). The NIH roadmap. Science, 302,
(pp. 213–245). Hillsdale, NJ: Erlbaum. 63–72. doi:10.1126/science.1091867
79
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 4
Applied Behavior Analysis

Dorothea C. Lerman, Brian A. Iwata, and Gregory P. Hanley
Applied behavior analysis (ABA) differs from other process because the borders separating basic, trans-
areas of applied psychology in many respects, but lational, and applied research are fluid. Nevertheless,
two are especially prominent. First, ABA is not an Fuller’s (1949) study, the first published report of
eclectic enterprise, borrowing theory and method operant conditioning with a human, serves as a good
from varied persuasions; it is grounded in the the- starting point. The response that Fuller shaped—a
oretical and experimental orientation of behavior simple arm movement in an individual with pro-
analysis. Second, whereas most applied fields are found intellectual disability—had little adaptive
distinguished by their emphasis on a particular value, but it was significant in demonstrating the
clientele, problem, or setting, ABA is constrained influence of operant contingencies on human behav-
only by its principles and methods. ABA focuses ior. Although many will find the article’s similarities
on any aspect of human and sometimes nonhuman to current ABA work almost nonexistent, it is worth
behavior, regardless of who emits it or where it noting that Boyle and Greer (1983) published an
occurs, crossing professional boundaries typically extension of Fuller’s work 35 years later in JABA in
used to define clinical, educational, and organiza- which they used similar methods to shape similar
tional psychology as well as generational cohorts responses among comatose patients.
and diagnostic categories. Thus, the subject matter Soon after Fuller’s (1949) article appeared, other
of ABA is not tied to any specific area of applica- reports of human operant conditioning followed
tion. Other chapters in this handbook present during the 1950s (Azrin & Lindsley, 1956; Bijou,
summaries of applied research organized along 1955; Gewirtz & Baer, 1958; Lindsley, 1956). At the
more traditional lines. In this chapter, we empha- end of that decade, Ayllon and Michael (1959) pub-
size ABA’s distinctive features and summarize its lished what many consider the first example of ABA
major themes on the basis of the behavioral pro- because it contained multiple single-case analyses
cesses of interest. of different interventions (extinction, differential
reinforcement, avoidance training, noncontingent
reinforcement) with a range of target behaviors
Origins of Applied Behavior
(excessive visitation to the nursing station, aggres-
Analysis
sion, refusal to self-feed, and hoarding) exhibited by
The official beginning of the field of ABA is easy to psychiatric patients. Similar reports were published
pinpoint because the term was coined in 1968 when throughout the 1960s (see Kazdin, 1978, for a more
the Journal of Applied Behavior Analysis (JABA) was extensive discussion) in various journals, including
founded. However, that date represents the culmina- the Journal of the Experimental Analysis of Behavior.
tion of events initiated many years prior. Tracing The board of directors of the Journal of the Experi-
the emergence of ABA before 1968 is an arbitrary mental Analysis of Behavior, recognizing the need for
DOI: 10.1037/13937-004
81
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Lerman, Iwata, and Hanley
a journal devoted to applications of operant condi- Distinctive Features of Applied

tioning, initiated the publication of JABA in 1968. Behavior Analysis
The first issue of JABA contained an article
Methodological differences between ABA and other
that essentially defined the field. D. M. Baer,
therapeutic endeavors are so numerous that discus-
Wolf, and Risley (1968) proposed seven defining
sion is beyond the scope of this chapter; it would
characteristics—dimensions—of ABA:
also be partially redundant with a description of the
1. Applied: The focus is on problems of social experimental analysis of behavior contained in
importance. Chapter 2 of this volume. Instead, we describe here
2. Behavioral: Dependent variables reflect direct some features of applied research that distinguish it
measurement of the behaviors of interest. from basic laboratory work.
3. Analytic: Demonstrations of behavior change
include convincing evidence of experimental Nature of the Response (Dependent
control, favoring repeated measures of individual Variable)
behavior and replication of treatment effects. Basic research focuses primarily on the fundamental
4. Technological: Procedural descriptions specify learning process. As a result, structural aspects of a
operational features of intervention for all rel- response are usually of secondary concern, and
evant responses. dependent variables often consist of arbitrary, dis-
5. Conceptually systematic: Techniques used to crete responses such as a rat’s lever presses or a
change behavior are related to the basic prin- pigeon’s key pecks. The response per se assumes
ciples from which they are derived. greater importance in applied research because the
6. Effectiveness: The extent of behavior change is goal of intervention is usually to change some topo-
sufficient to be of practical value. graphical aspect of behavior. In fact, much of ABA
7. Generality: Effects of intervention strategies research involves establishing a new response that is
can be programmed across time, behavior, and not in an individual’s repertoire or modifying a cur-
setting. rent response topography from one that is deficient,
socially unacceptable, or even dangerous to one that
These dimensions have served as useful guides in is deemed more appropriate by the larger commu-
planning and evaluating ABA research since the nity. For example, eating can be accomplished in
field’s inception. It is a mistake to assume, however, any number of ways, but the form that eating takes
that any particular study or even most of those pub- may be a determinant of social acceptability (O’Brien
lished in JABA or similar journals illustrates each of & Azrin, 1972). Other examples include cadence of
the characteristics of ABA described by D. M. Baer speech (R. J. Jones & Azrin, 1969), structural aspects
et al. (1968). The most obvious example is the of composition (Brigham, Graubard, & Stans, 1972),
dimension of generality: A study with two or three and nuances of social interaction (Serna, Schumaker,
participants cannot establish the external validity of Sherman, & Sheldon, 1991). Sometimes the goal of
a therapeutic intervention; multiple, systematic rep- applied research is not to establish a specific response
lication and extension accomplish that goal (see but, rather, to increase topographical variability, as in
Chapter 7, this volume). Similarly, although applied the development of creativity (Glover & Gary, 1976;
may suggest exclusive emphasis on problematic Goetz & Baer, 1973).
human behavior (deficit or excess), a case can be Applied researchers also focus more often on the
made for studying an arbitrary response such as eye measurement of larger units of behavior because
movement (Schroeder & Holland, 1968) to identify most adaptive performances are made up of
procedures for improving attention or even nonhu- response chains, such as self-care sequences, aca-
man behavior when it is difficult to conduct certain demic work, and vocational behavior. Some chains
demonstrations with humans, as in the shaping of involve repetition of a similar topography, as in
self-injury (Schaefer, 1970). building a construction out of blocks (Bancroft,
82
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Weiss, Libby, & Ahearn, 2011); others, however, employee–customer interaction. In a similar way,
can be extremely complex. For example, Allison and contemporary research on response suppression
Ayllon (1980) evaluated a behavioral approach to attempts not only to eliminate problematic behavior
athletics coaching and measured varied behaviors but also to establish more socially appropriate
such as blocks (football), handsprings (gymnastics), alternatives.
and serves (tennis) as response chains consisting of Finally, some dependent variables in applied
five to 11 members. research are not based on direct measurement of
A third distinction between basic and applied any response but rather on some observable out-
research is the multidimensional nature of measure- come or product of a response. At the individual
ment in applied work. Basic researchers typically level, response products might consist of carbon
use response rate or response allocation as a depen- monoxide level as an index of smoking (Dallery &
dent variable because both are convenient, standard, Glenn, 2005) or pounds lost as a measure of diet or
and highly sensitive measures (Skinner, 1966; see exercise (Mann, 1972). Aggregate products such as
Chapter 10, this volume). Although they occasion- amount of litter (Bacon-Prue, Blount, Pickering, &
ally study other response dimensions such as inten- Drabman, 1980) or community water usage (Agras,
sity or force, they do so to determine how these Jacob, & Lebedeck, 1980) have also been used occa-
aspects of behavior are influenced by environmental sionally in applied research when measurement of
manipulation (Hunter & Davison, 1982; Sumpter, individual behavior is difficult or impossible.
Temple, & Foster, 1998), not because changing
these response dimensions is a goal of the research. Treatment Characteristics (Independent
Applied researchers, by contrast, will study response Variable)
allocation because, for example, decreasing Much of ABA research involves the extension of
responses allocated to drug taking is of societal well-established basic principles such as reinforce-
importance (see Volume 2, Chapter 19, this hand- ment, stimulus control, extinction, and punishment,
book). Likewise, they will study response intensity including parametric and comparative analysis. A
because it is either too low, as in inaudible speech more distinctive feature of applied work is that oper-
(Jackson & Wallace, 1974), or too high, as in noisy ational features of independent variables take on
behavior on the school bus (Greene, Bailey, & special characteristics determined by the target
Barber, 1981). Furthermore, many human perfor- behavior of interest or practical considerations dur-
mances are such that clinical improvement requires ing implementation. These characteristics include
change in more than one dimension of a response. greater emphasis on supplementary cues, procedural
For example, effective treatment of a child’s sleep variation as a means of enhancing acceptability, and
disturbance should decrease not only the frequency the use of intervention packages.
but also the duration of nighttime wakening (France &
Hudson, 1990; see Volume 2, Chapter 17, this hand- Supplementary cues. The aim of research on
book). Quantifying changes in academic performance response acquisition is to produce new behavior,
also requires multiple measures, such as the duration and efficiency is often an important consideration.
of work, the amount of work completed, and accu- Although many adaptive performances could be
racy (J. C. McGinnis, Friman, & Carlyon, 1999). shaped eventually (as they are in the nonhuman basic
In addition to focusing on multiple dimensions research laboratory) through differential reinforce-
of a response (or a response chain), applied research- ment of successive approximations, applied research-
ers often measure several different responses that, in ers typically rely heavily on supplementary methods.
combination, reflect improvement in the problem of Common procedures include simple instructions
interest. The delivery of courteous service (Johnson & (Bates, 1980), static cues such as pictures (Wacker
Fawcett, 1994), for example, requires the correct & Berg, 1983), in vivo or video modeling (Geiger,
performance of a number of responses and response LeBlanc, Dillon, & Bates, 2010), and physical prompt-
chains that differ depending on the context of the ing (R. H. Thompson, McKerchar, & Dancho, 2004).
83
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Procedural variation. Interventions evaluated in necessary versus incidental treatment components.

a research context are designed for implementation For example, the continence training program
under more naturalistic conditions. This fact makes developed by Azrin and Foxx (1971), although
procedural aspects of treatment, although perhaps highly effective and efficient, is extremely labor
incidental to their effects, potential determinants of intensive and includes procedures to increase the
adoption by parents, teachers, and other therapeu- frequency of elimination, reinforce the absence of
tic agents. As such, the applied researcher will vary toileting accidents as well as the occurrence of cor-
different components of the procedure depending rect toileting, teach dressing and undressing skills,
on the consumer of the intervention. For example, and punish toileting accidents with a series of cor-
interventions designed to teach new behaviors will rective behaviors and time out.
experiment with components of the procedure that
are anticipated to produce more efficient response Instrumentation and Design
acquisition. Social acceptability is a major consid- Measurement represents a significant challenge to
eration for response suppression procedures, and applied researchers for at least three reasons. First,
several studies have explored ways to maintain the most research is conducted outside of controlled
intervention’s efficacy while improving the proce- laboratory settings where recording of responses is
dure’s acceptability. A good example of the latter is not easily automated. Second, dependent variables
time out from positive reinforcement. Aside from are often multidimensional in nature and may con-
variations in duration (White, Nielsen, & Johnson, sist of several distinct response topographies.
1972) and schedule (Clark, Rowbury, Baer, & Baer, Finally, when interventions consist of a series of
1973), the form of time out has included seclusion procedures, the consistency of implementing inde-
in a room (Budd, Green, & Baer, 1976), physical pendent variables becomes an additional concern
holding (Rolider & Van Houten, 1985), removal of a (Peterson, Homer, & Wonderlich, 1982). As a result
ribbon worn around the neck combined with loss of of these factors, human observers collect most data
access to ongoing activities (Foxx & Shapiro, 1978), in applied research because of their versatility in
and placement on the periphery of activity so that detecting a wide range of responses. This practice
one could watch but not participate (Porterfield, introduces the potential for measurement variability
Herbert-Jackson, & Risley, 1976). that is not found in most basic research. When
designing observation procedures, the applied
Intervention packages. When the goal of interven- researcher must consider several important factors,
tion is to simply change behavior without identify- including how behavior will be categorically coded
ing the necessary components of treatment, ABA by the observers (Meany-Daboul, Roscoe, Bourret,
researchers may combine two or more distinct & Ahearn, 2007), how the observers may be most
independent variables into a package. Perhaps the efficiently trained (Wildman, Erickson, & Kent,
simplest example of such a package is a teacher’s 1975), how to compute interobserver agreement
use of praise, smiles, physical contact (a pat on the (Mudford, Taylor, & Martin, 2009), and variables
back), and feedback as consequences for appropriate that influence reliability and accuracy of measure-
student behavior. Although a component analysis ment (Kazdin, 1977; Lerman et al., 2010).
of treatment effects may be informative because it is Experimental control in basic research is typi-
unclear which teacher responses actually functioned cally achieved through a demonstration of revers-
as positive reinforcement, it may not matter much ibility (Cumming & Schoenfeld, 1959); that is,
in a practical sense because none was costly or time within-subject replication of behavior change result-
consuming. Often, an intervention package, if costly, ing from systematic application and removal of the
will be subject to a component analysis after it has independent variable. Applied researchers often use
proven effective (Yeaton & Bailey, 1983). Similarly, a similar approach with a variety of reversal-type
interventions requiring a great deal of effort to procedures such as A-B-A, B-A-B, A-B-A-B, and mul-
implement may benefit from attempts to isolate tielement designs (see Chapter 5, this volume).
84
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Occasionally, however, the effects of an applied apparently is likely to support that application when
intervention are irreversible. One situation arises it deals with socially important behaviors” (p. 91).
when the independent variable consists of instruc- Stated another way, consumers are the final judges
tion or training whose effects cannot be removed. of applied work, and the term social validity refers to
A second arises when behavior, once changed, those judgments about three aspects of ABA: (a) the
encounters other sources of control. For example, significance of goals, (b) the acceptability of proce-
a teacher may find it necessary to use prompting, dures, and (c) the importance of effects (Kazdin,
praise, and tangible reinforcers to increase a socially 1977; Wolf, 1978).
withdrawn child’s rate of peer interaction. Once A variety of methods have been used to examine
interaction increases, however, the child may obtain the social validity of applied research, including
a new source of reinforcement derived from playing evaluations of the relevance of measures by profes-
with peers, which maintains the child’s social behav- sionals (R. T. Jones, Kazdin, & Haney, 1981), rat-
ior independent of the teacher’s actions. ings of behavior change by independent judges
In both of these situations, a multiple-baseline (Bornstein, Bach, McFall, Friman, & Lyons, 1980),
design, involving staggered introduction of treat- measures of consumer satisfaction (Bourgeois,
ment across two or more baselines (representing 1990), treatment choices made by clients (Hanley,
behaviors, participants, or situations), can be used Piazza, Fisher, Contrucci, & Maglieri, 1997), and
to achieve replication without the necessity of a cost–benefit analysis of the effects of intervention
reversal. The traditional multiple-baseline design (Van Houten, Nau, & Marini, 1980). Although
involves concurrent measurement in which data are important in concept, the assessment of social valid-
collected across all baselines at the same points in ity raises many questions related to subjective mea-
calendar time. Because the logistics of clinical work surement (Hawkins, 1991), bias (Fuqua & Schwade,
or the inability to simultaneously recruit partici- 1986), and the selection of appropriate reference
pants having unusual problems limits the versatility groups as the basis for comparison (Van Houten,
of this design, a notable variation—the nonconcur- 1979) that have yet to be resolved. Perhaps for this
rent multiple baseline—has emerged in ABA reason, social validation measures continue to be
research and may often represent the design of rarely used (J. E. Carr, Austin, Britton, Kellum, &
choice. First described by Watson and Workman Bailey, 1999).
(1981) and subsequently illustrated by Isaacs,
Embry, and Baer (1982), the nonconcurrent multi-
Major Themes in Applied Research
ple baseline retains the staggered feature of interven-
tion (treatment is applied after baselines of different Response Acquisition
length); however, measurement across baselines Relying on the basic principles of learning derived
takes place at different points in time. As noted by from the experimental analysis of behavior (see
D. M. Baer et al. (1968), although this design does Chapters 2 and 3, this volume), applied behavior
not contain an explicit control for the confounding analysts have developed a technology for teaching
effects of a historical accident, differences in both new behavior and refining aspects of existing behav-
the baseline length and the temporal context of mea- ior. Much of the teaching technology has been dis-
surement make it extremely unlikely that the same covered with people diagnosed with developmental
historical accident would coincide with the imple- disabilities, in part because developing practical,
mentation of treatment on more than one baseline. age-appropriate skills is a primary treatment goal for
these individuals. Nevertheless, this same technol-
Social Validity ogy has been applied in early childhood, regular
The final characteristic of ABA that distinguishes it education, and college classrooms and in work-
from basic research is a by-product of application. places, homes, and playing fields. Independent of
As noted by D. M. Baer et al. (1968), “A society will- the context in which it is applied, ABA approaches
ing to consider a technology of its own behavior to teaching usually progress from (a) establishing
85
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
simple and observable responses; to (b) shaping is the universal first step. Specific target behaviors
more complex, independent, or fluent responses and are usually selected because they will improve a
response chains; and finally to (c) synthesizing com- person’s quality of life in both the short and the long
plex or chained responses into socially relevant rep- term by allowing the person access to new reinforc-
ertoires such as independently completing self-care ers and additional reinforcing contexts. Responses
skills (T. J. Thompson, Braam, & Fuqua, 1982), targeted with ABA procedures often allow people
reading and comprehending text (Clarfield & to live more independently or behave more effec-
Stoner, 2005; Volume 2, Chapter 16, this handbook), tively and efficiently or in more socially acceptable
solving algebraic and trigonometric problems ways. Rosales-Ruiz and Baer (1977) emphasized
(Ninness et al., 2006), playing football (J. V. Stokes, the importance of selecting behavior cusps, which
Luiselli, Reed, & Fleming, 2010), or socializing with they defined as changes in behavior that have
friends (Haring & Breen, 1992). important consequences for the person beyond the
change itself. For example, learning to walk or talk
Operant contingency. The most fundamental
provides the learner with unprecedented access to
tools of the applied behavior analyst are operant
important materials, social interactions, and sensory
contingencies. It follows that an essential aspect of
experiences that then occasion other, more com-
ABA teaching procedures is maximizing the effects
plex behavior, such as dancing or reading, allowing
of reinforcing consequences on target behaviors.
access to even richer events and interactions (Bosch
Thus, efforts are taken to ensure that the reinforcers
& Fuqua, 2001).
are valuable to the learner at the time of teaching;
these antecedent events that establish the value of Task analysis. Once selected, target behaviors are
the reinforcing consequences are referred to as estab- then examined to identify their components and
lishing operations (Laraway, Snycerski, Michael, & any relevant sequences to the components. This
Poling, 2003). Efforts are also made to use salient process is often referred to as task analysis (e.g.,
events that signal the availability of the reinforcers Cronin & Cuvo, 1979; Resnick, Wang, & Kaplan,
to the learner; when these events occasion the tar- 1973; Williams & Cuvo, 1986). The process of task
get response, they are referred to as discriminative analysis was nicely illustrated in a study by Cuvo,
stimuli. Often, experience with a properly designed Leaf, and Borakove (1978), who taught janitorial
contingency is not sufficient to generate target skills to 11 young adults with intellectual disabilities.
responses, especially those that the learner has Six general bathroom-cleaning steps were identified
never emitted previously, so careful consideration is (mirror, sink, urinal, toilet, floor, miscellaneous);
often given to prompts that will greatly increase the each was then subdivided into 13 to 56 component
probability of the target response. So that the adap- steps before training, which consisted of instructions,
tive behavior will be emitted under more natural modeling, prompting, and differential reinforcement.
conditions, efforts are taken to transfer control from Task analysis is also an important initial step when
the prompt to more naturally occurring establish- teaching almost any athletic skill. For instance, J. V.
ing operations and discriminative stimuli. These Stokes, Luiselli, Reed, and Fleming (2010) task ana-
prompt-fading procedures will yield independent lyzed offensive line-blocking skills of starting high
and generalized performances. We discuss each school varsity football players before using video and
of these universal elements of ABA approaches to auditory feedback to teach these same skills to other
response acquisition in greater detail and in the con- lineman on the team. Much early ABA research iden-
text of published empirical examples. tified aspects of task analyses that resulted in better
learning. A general finding by Williams and Cuvo
Selecting a target response. ABA procedures for
(1986) is that that greater specificity of a task analy-
developing new behavior share several common
sis results in better generalized performance.
characteristics independent of the deficit being
addressed or the population with whom it is being Performance assessment. A direct assessment of
addressed. Selection of an objective target behavior the learner’s baseline performance is conducted to
86
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
determine which components of a target skill need programs for socially important behavior (J. E. Carr,
to be taught (e.g., Lerman, Vorndran, Addison, & Nicolson, & Higbee, 2000; Fisher et al., 1992; Pace
Kuhn, 2004; T. J. Thompson et al., 1982). Teaching et al., 1985).
is then implemented for those components that the
Establishing reinforcers. Once reinforcers are
learner has not yet mastered. While assessing tack-
identified, tactics to establish and maintain their
ling performance by high school football players,
value throughout the teaching period are consid-
for example, J. V. Stokes, Luiselli, and Reed (2010)
ered. One common procedure is to reserve the
noticed that most players did not place their face
use of items delivered as consequences for correct
mask on the opponent’s chest or wrap their arms
responding to formal teaching periods. Vollmer and
around the opponent, so these specific skills were
Iwata (1991) showed that scheduling brief periods
targeted for teaching. T. J. Thompson et al. (1982)
without access to events such as food, music, or
sought to teach laundry skills (sorting, washing,
social interaction increased their effectiveness as
and drying clothes) to three young adults with intel-
reinforcers during later teaching sessions. Roane,
lectual disabilities. The skill set included 74 distinct
Call, and Falcomata (2005) found similar positive
behaviors. To enhance the efficiency of the teaching
effects of limiting access to leisure items outside of
and the evaluation of the teaching procedures, they
the teaching context, and Hanley, Iwata, and Roscoe
assessed all 74 skills in a baseline probe and then
(2006) showed that access to preferred items for
focused their teaching only on those skills that were
periods as long as 24 to 72 hours before their use
not evident during the probe.
could decrease their value. Because mere access
to preferred items can diminish their motivational
Selecting reinforcers. The success of behavior
efficacy in teaching programs, other tactics—such
change programs is heavily dependent on conse-
as varying the amount and type of reinforcers (Egel,
quences that are delivered as reinforcers, which
1981), allowing the learner opportunities to choose
are typically selected on the basis of an assessment
among different reinforcers (Fisher, Thompson,
of the learner’s preference for a range of events or
Piazza, Crosland, & Gotjen, 1997; Tiger, Hanley, &
activities. These assessments can be indirect, involv-
Hernandez, 2006), and using token reinforcers that
ing an interview of teachers or parents who attempt
can be traded in for a variety of back-up reinforcers—
to describe the learner’s preferences (Cautela &
are often arranged in ABA-based teaching programs
Kastenbaum, 1967; Cote, Thompson, Hanley, &
(Kazdin & Bootzin, 1972; Moher, Gould, Hegg, &
McKerchar, 2007; Fisher, Piazza, Bowman, & Amari,
Mahoney, 2008). For a discussion of these issues
1996), but in more cases, direct assessments are
in translational research, see Volume 2, Chapter 8,
conducted (e.g., DeLeon & Iwata, 1996; Fisher
this handbook.
et al., 1992; Pace, Ivancic, Edwards, Iwata, & Page,
1985; Sundby, Dickinson, & Michael, 1996). Direct Contingency, contiguity, and timing. Ensuring
assessments of preference usually involve the actual that reinforcers are delivered only for particular
presentation of potentially reinforcing items in one target responses during the teaching session is
of several formats, measurement of a person’s selec- another hallmark of ABA programs. When the target
tions, and brief access to an item after its selection. response is reinforced and all other responses are
Items that are selected or interacted with most extinguished, it is referred to as differential reinforce-
often are then delivered as consequences for some ment (see Vladescu & Kodak, 2010, for a recent
behavior of interest (see Volume 2, Chapter 12, this review). Differential reinforcement, and the imme-
handbook, for more detail on preference assessment diacy with which the reinforcing event is delivered
and reinforcer evaluation as they are often applied after target responses (Gleeson & Lattal, 1987;
among children diagnosed with autism). In fact, a Vollmer & Hackenberg, 2001), form strong contin-
common example of the use of arbitrary responses gencies that result in rapid learning.
in ABA is as tests for the efficacy of the preferred Some evidence has suggested that learning via
items before their selection as reinforcers in teaching reinforcement contingencies will proceed more
87
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
efficiently when the rate of learning opportunities is and prompt-fading procedures (Demchak, 1990;
kept high. For instance, Carnine (1976) showed that Gast, VanBiervliet, & Spradlin, 1979; Godby,
both correct responding and task participation of Gast, & Wolery, 1987; Odom, Chandler, Ostrosky,
typically developing children were higher when McConnell, & Reaney, 1992; Schreibman, 1975;
instructions were presented with very brief rather Wolery & Gast,1984; Wolery et al., 1992). Two gen-
than longer intertrial intervals. Similar results have eral types of prompts have been the focus of many
been reported with a wide variety of skills training applied studies. One is a response prompt, which
among participants with intellectual disabilities (e.g., involves the use of a supplementary cue to occasion
Koegel, Dunlap, & Dyer, 1980). The mechanism of a correct response, for example, providing vocal
this effect is, at present, unknown but may be the instructions, models, or physical guidance to per-
result of the inability of a low reinforcement rate to form the target behavior. These prompts are often
maintain adequate levels of participant attention. eliminated by delaying the prompt across successive
trials until the correct response occurs before (i.e.,
Gaining stimulus control. Strong contingen-
without) the prompt (Schuster, Gast, Wolery, &
cies of reinforcement not only aid in the teaching
Guiltinan, 1988). R. H. Thompson, McKerchar, and
of new responses but also make it easier to ensure
Dancho (2004) illustrated an example of response
that responding occurs in the presence of a specific
prompts by using physical prompts to teach three
stimulus condition; this correlation between ante-
infants to emit the manual signs “please” and
cedent stimuli and reinforced responding is derived
“more” in the presence of food. The prompts were
from basic research on stimulus control (Pilgrim,
gradually delayed after the visible presentation of
Jackson, & Galizio, 2000; Sidman & Stoddard,
the food until the infants were independently sign-
1967). Stimulus control is developed by differen-
ing for the food.
tially reinforcing a response in the presence of cer-
The second type of prompt is a stimulus prompt,
tain stimulus properties but not others (e.g., saying
in which some aspect of the discriminative stimulus
“dog” in the presence of the letters d-o-g and not in
is modified to more effectively occasion a correct
the presence of the letters g-o-d). The formation of
response. For instance, Duffy and Wishart (1987)
stimulus control is an important part of the develop-
taught children with Down syndrome and typically
ment of effective behavioral repertoires. Successful
developing toddlers to point to particular shapes
reading, for example, is entirely dependent on spe-
when the corresponding shape was named. Prompt-
cific responses (usually vocalizations) coming under
ing consisted of initially representing the named
control of specific letter combinations (Mueller,
shape with an object much larger than those repre-
Olmi, & Saunders, 2000; Sidman & Willson-Morris,
senting the incorrect comparison shapes. Fading
1974). Accurate responses to instructions occur
was accomplished by gradually increasing the sizes
when specific vocal or motor responses come under
of the incorrect shapes until all shapes were the
control of particular combinations of written or
same size.
spoken words (Cuvo, Davis, O’Reilly, Mooney, &
Crowley, 1992). Developing stimulus control is also
Shaping. Certain responses such as early speech
important for social behaviors like saying “please”
sounds are difficult to prompt, in which case shap-
(Halle & Holt, 1991), making a request of a person
ing may be required to initiate behavior. Shaping
who appears busy (Kuhn, Chirighin, & Zelenka,
involves slight changes in a differential reinforce-
2010), or engaging in important independent living
ment contingency such that closer approximations
skills, such as an efficient exiting response to a fire
to the target behavior are reinforced over time, and
alarm (Bannerman, Sheldon, & Sherman, 1991).
previous approximations are extinguished. Bourret,
Prompting and prompt fading. Because the Vollmer, and Rapp (2004) initially used vocal and
arrangement of contingencies per se may not be suf- model prompts to teach vocalizations to two chil-
ficient to produce new behavior, a great deal of ABA dren diagnosed with autism. The experimenters
research has been devoted to the use of prompting instructed the participants to emit target utterances
88
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
(e.g., say “tune”) and reinforced successful utter- which trials were initiated only when a child showed
ances with access to music. When children did interest in an object or topic. The key features of
not emit the target utterance, imitation of simpler their procedures were described as follows:
models was reinforced (e.g., changing say “tune” to
Whenever a child selected a preschool
say “tuh”). When the children began imitating the
play material, they were prompted and
shorter phonemes, the criterion for reinforcement
required to ask for it, first by name
was reapplied to the complete spoken word.
(noun), then by the name plus a word
Error reduction. Differential reinforcement of that described the material (adjective-
correct responses is a universal aspect of acquisi- noun combination), then by use of a
tion programs and may account for changes in the color adjective-noun combination, and
frequency of incorrect as well as correct responses. finally by requesting the material and
That is, errors may decrease simply as a result of describing how they were going to use it
extinction (Charlop & Walsh, 1986). More often, (compound sentence). As each require-
however, instructors deliver explicit consequences ment was made, the children’s general
after errors, which can include a simple statement use of that aspect of language markedly
such as “no” (Bennett & Ling, 1972), a prompt to increased. (Hart & Risley, 1974, p. 243)
engage in the correct response (Clarke, Remington,
The changing criterion for reinforcement inherent
& Light, 1986), or a remedial trial consisting of
to shaping procedures was evident in incidental
repetition of the trial on which an error was made
teaching; more significant was the fact that instruc-
(Nutter & Reid, 1978), an easier trial (Kirby &
tion (a) occurred intermittently and (b) capitalized
Holborn, 1986), or a more difficult trial (Repp &
on a child-initiated response that identified both the
Karsh, 1992). Numerous variations on the remedial
task (name the object) and the reinforcer (access to
strategy have been reported in the literature, and
the named object). These were novel and important
Worsdell et al. (2005) conducted a series of com-
features of the instructional program.
parative studies on quantitative and qualitative char-
Although many ABA-based procedures are
acteristics of remediation. Their results showed that
applied to individual learners, many situations arise
(a) multiple repetitions of the correct response were
in which individuals perform or could perform as
more effective than a single response repetition,
part of a group, which serve as the occasion for
(b) correction for every error made was superior to
implementing contingencies on group behavior.
intermittent error correction, and (c) repetition of
Group contingencies involve several arrangements in
relevant training words was slightly superior to mere
which the performance of one, some, or all members
repetition of irrelevant words. An interesting aspect
of a group determines the delivery of reinforcement.
of the data was that error correction involving pre-
Perhaps the best early examples of research on
sentation of irrelevant material also enhanced learn-
group contingencies can be found in studies con-
ing, implicating negative reinforcement (avoidance
ducted at Achievement Place, a home-style setting
of remedial trials) as a source of influence during
for predelinquent boys (Bailey, Wolf, & Phillips,
response acquisition.
1970; Fixsen, Phillips, & Wolf, 1973; Phillips, 1968;
Further variations. Aside from the many ways Phillips, Phillips, Fixsen, & Wolf, 1971; Phillips,
in which a particular aspect of the teaching situa- Phillips, Wolf, & Fixsen, 1973). For instance, Phil-
tion may be arranged, multiple components may lips et al. (1971) arranged group contingencies to
be programmed simultaneously to either increase increase promptness, room cleaning, money saving,
the efficiency of acquisition or enhance generaliza- and watching the news. In a follow-up study, Phil-
tion of the effects of the teaching. For instance, Hart lips et al. (1973) showed that a contingency in
and Risley (1968, 1974, 1975, 1980) published a which a democratically elected peer manager had
series of studies on the use of incidental teaching—a the authority both to give and to take away points
milieu-based approach to teaching language in for peers’ performances was more effective and
89
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
preferred by the adolescents than an individual con- child saying, “thanks a lot” instead of “thank you”
tingency. One of the unique features of group con- when handed a snack. Continuing with this exam-
tingencies is that they can create a context for ple, maintenance is said to occur if the child persists
unprogrammed peer-mediated contingencies rang- in saying “thank you” without the use of prompts
ing from praise and offers of assistance to criticism and continuous reinforcement. The persistence and
and sabotage. Although some of these side effects transfer of behavior change is also desirable for
have been reported (Frankosky & Sulzer-Azaroff, behaviors that have been targeted for reduction
1978; Hughes, 1979; Speltz, Shimamura, & (Shore, Iwata, Lerman, & Shirley, 1994).
McReynolds, 1982), a thorough analysis of the types Maintenance and generalization are commonly
of social interaction generated by different group treated as separate areas of concern, but they are
contingencies and their role in changing behavior necessarily intertwined. A single occurrence of a
has not been conducted. behavior in a nontraining context might constitute
generalized responding. However, behavior change
Maintenance and Generalization must persist long enough to be detected and to sat-
A behavioral technology would have limited clinical isfy clinical goals. Koegel and Rincover (1977)
value if it failed to produce durable changes in clearly illustrated this distinction between mainte-
responding. Furthermore, behavior analysts would nance and generalization. They taught children with
like performance to transfer across relevant (non- autism to follow simple instructions in a therapy set-
training) environments and perhaps even to other ting while simultaneously assessing responding in a
(untrained) ways of behaving. The term maintenance different setting, in which a novel therapist who
refers to the persistence of behavior change across did not reinforce correct responses presented the
time, and the term generalization refers to the persis- instructions. An analysis of performance across con-
tence of behavior change across settings, people, secutive instructional trials revealed the emergence
responses, and other nontraining stimuli. Mainte- of generalized responding for two of three partici-
nance and generalization are said to occur if perfor- pants. However, performance rapidly extinguished
mance persists despite the absence of ancillary in the generalization setting while being maintained
antecedents (e.g., prompts) or consequences (e.g., in the training setting in which correct responses
delivery of a token after each response) that origi- continued to produce reinforcement. It is possible
nally produced learning. For example, suppose an that stimuli associated with reinforcement in the
instructor uses model prompts and reinforcement training setting acquired exclusive control over
(e.g., praise plus a piece of candy) to teach a child to responding, that stimuli in the generalization setting
say “thank you” when the instructor hands the child became discriminative for extinction, or both. The
a snack in the kitchen. Generalization is said to processes of stimulus control and reinforcement are
occur if the child (unprompted) says “thank you” likely determinants of both maintenance and gener-
when handed (a) other preferred items (e.g., a toy), alization (Kirby & Bickel, 1988).
(b) a snack in locations other than the kitchen area,
or (c) a snack by people other than the instructor, Maintenance. Two primary approaches have been
and the response is followed by praise only, candy used in applied research to evaluate the persistence
intermittently, or no consequence at all (T. F. Stokes of behavior change over time. In the most common
& Baer, 1977).1 The changed conditions under approach, experimenters terminate the training
which “thank you” was said are typically considered condition and then briefly measure the response
examples of stimulus generalization. By contrast, after an intervening period of time. Successful
response generalization usually refers to changes in maintenance has often been reported as an outcome
responses that were not directly taught, such as the of training when this approach has been used to
This treatment of generalization deviates from that in the laboratory, where generalization is tested in extinction. The basic conceptualization of gen-
1
eralization requires that the response occur in the absence of reinforcement. The more pragmatic approach to generalization frequently taken in ABA
requires only that the response occur in the absence of the same consequence that produced the original learning.
90
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
assess maintenance (e.g., Cummings & Carr, 2009; promote generalization by preventing the develop-
Pierce & Schreibman, 1994). However, this finding ment of inappropriate or restricted stimulus control.
is somewhat surprising because few experiment- They do so by varying stimuli that are irrelevant to
ers have explicitly arranged conditions to promote the response (e.g., specific location of the learner)
durable behavior change. Furthermore, although the while maintaining relevant features of the training
authors typically delineated the conditions in effect situation (e.g., delivery of a particular instruction),
during the maintenance check (e.g., the response or by making stimuli specific to the training set-
did not produce reinforcement), they rarely pro- ting indiscriminable. In the latter case, thinning
vided information about potential determinants of the schedule of reinforcement, delaying reinforce-
maintenance during the intervening period (e.g., ment, and interspersing training and generalization
number of opportunities to engage in the response; tests may prevent the presence of the reinforcer
contingencies for responding). from acquiring a discriminative function for further
In a second, less common approach to assessing responding.
maintenance, experimenters repeatedly measure lev- Unambiguous examples of response generaliza-
els of responding after removing all sources of rein- tion are more difficult to find in the applied litera-
forcement for the response or after replacing the ture, and few studies have evaluated factors that
programmed reinforcer (e.g., candy) with a more might promote this type of generalized responding.
naturalistic reinforcer (e.g., praise). Performance per- Changes in topographical variations of the targeted
sisted in these studies only when special arrange- behavior, similar to the previous example of the stu-
ments were made to promote maintenance. These dent saying “thanks a lot” instead of “thank you,”
arrangements typically took the form of reinforce- may be more likely to occur when the targeted
ment schedule thinning (e.g., R. A. Baer, Blount, response is exposed to extinction (e.g., Duker & van
Detrich, & Stokes, 1987; Ducharme & Holborn, 1997; Lent, 1991; Goetz & Baer, 1973). Some authors
Hopkins, 1968; Kale, Kaye, Whelan, & Hopkins, have reported collateral changes in responses that
1968; Kazdin & Polster, 1973) or teaching partici- bear no physical resemblance to the behavior
pants to recruit reinforcement from others in the nat- exposed to treatment, but the mechanisms responsi-
ural environment (e.g., Seymour & Stokes, 1976). ble for these changes were unclear. Presumably, the
targeted and generalized response forms were mem-
Generalization. In a seminal article, T. F. Stokes bers of the same functional response class. In a
and Baer (1977) summarized various strategies to study by Barton and Ascione (1979), for example,
promote generalization that had appeared in the children taught to engage in vocal sharing (e.g.,
applied literature at that time. The technology of requesting to share others’ materials, inviting others
generalization described in their article has changed to share their own materials) showed increases in
very little since its publication. The most com- physical sharing (e.g., handing other children toys)
monly used ways to program stimulus generaliza- even though the experimenters did not directly
tion include (a) varying the stimulus dimension or teach those responses. Koegel and Frea (1993)
dimensions of interest during training (i.e., varying reported corresponding increases in untreated
the setting, trainers, materials), (b) ensuring that aspects of social communication, such as appropri-
the stimuli present during training are similar to ate topic content and facial expressions, after teach-
those in the generalization context, (c) thinning ing children with autism to use appropriate eye gaze
the reinforcement schedule during training, and and gestures during conversations. A similar type of
(d) arranging for the behavior to contact reinforce- generalized behavior change has also been reported
ment in the generalization context (Ducharme & when only some topographies of problem behavior
Holborn, 1997; Durand & Carr, 1991, 1992; were exposed to treatment (e.g., Lovaas & Sim-
Marzullo-Kerth, Reeve, Reeve, & Townsend, 2011; mons, 1969; Singh, Watson, & Winton, 1986).
T. F. Stokes, Baer, & Jackson, 1974). As discussed Other commonly studied forms of generalization
by Kirby and Bickel (1988), these approaches likely contain elements of both stimulus and response
91
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
generalization because different variations of the successful, the latter approach was likely responsible
trained response occur under different variations of for the inconsistent results reported with most forms
the training stimuli. For example, children who of treatment (e.g., Favell et al., 1982) and a greater
receive reinforcement for imitating specific motor reliance on punishment in both research and appli-
movements will begin to imitate novel motor move- cation (Kahng, Iwata, & Lewin, 2002; Pelios, Mor-
ments in the absence of reinforcement, an emergent ren, Tesch, & Axelrod, 1999).
skill called generalized imitation (Garcia, Baer, & Publication of a systematic method for identifying
Firestone, 1971; Young, Krantz, McClannahan, & the function or functions of problem behavior
Poulson, 1994). Other examples can be found in the (Iwata, Dorsey, Slifer, Bauman, & Richman, 1982)
research on generative language, including the gen- shifted the focus of behavior-analytic approaches to
eralized use of the plural morpheme (e.g., Guess, response suppression. Functional analysis methodol-
Sailor, Rutherford, & Baer, 1968), subject–verb ogy involves a direct test of multiple potential rein-
agreement (e.g., Lutzker & Sherman, 1974), and forcers for problem behavior, including positive
sentence structure (e.g., “I want _________”; Her- reinforcers such as attention or toys and negative
nandez, Hanley, Ingvarsson, & Tiger, 2007) and reinforcers such as escape from demands. Because of
generalization from expressive to receptive language the utility of this assessment approach, treatments
modalities (e.g., Guess & Baer, 1973). that involve terminating the reinforcement contin-
gency for problem behavior (i.e., extinction), deliver-
Response Suppression ing the maintaining reinforcer as part of differential
Treating maladaptive behavior has been a major con- reinforcement procedures, and manipulating relevant
cern of applied researchers and clinicians since the motivating operations have taken precedent in
inception of the field. Behaviors targeted for reduc- research and practice. Research has also continued to
tion have included responses that put the person evaluate the generality of the functional analysis
performing the behavior and others at risk of injury methodology across a variety of behavior problems,
(e.g., aggression, self-injury, cigarette smoking) as populations, and idiosyncratic variables (e.g., Bow-
well as those that interfere with learning or adaptive man, Fisher, Thompson, & Piazza, 1997; Hagopian,
behavior (e.g., disruption, noncompliance). From Bruzek, Bowman, & Jennett, 2007). Most recently,
the earliest research, experimenters recognized that knowledge about behavioral function has permitted
many of these behaviors are maintained by rein- more detailed analyses of the mechanisms underlying
forcement contingencies and that modifying these common treatment procedures and factors that influ-
contingencies might help alleviate the problem (e.g., ence their effectiveness.
Ayllon & Michael, 1959; Lovaas, Freitag, Gold, & Laboratory research on basic processes that
Kassorla, 1965; Wolf, Risley, & Mees, 1963). How- reduce responding provided the foundation for
ever, the field initially lacked a systematic approach behavior analysts’ current technology of treatments
to identifying the variables that maintain problem for problem behavior, including the commonly used
behavior. Although some early research focused on a procedural variations of extinction, differential rein-
range of variables that might be functionally related forcement, satiation, and punishment. In the follow-
to serious behavior disorders (e.g., E. G. Carr, New- ing sections, we provide an overview of these
som, & Binkoff, 1980; Lovaas & Simmons, 1969; response suppression procedures.
Rincover, Cook, Peoples, & Packard, 1979; Thomas,
Becker, & Armstrong, 1968), most were outcome- Extinction. Terminating the reinforcement con-
driven extensions of basic research studies in which tingency that maintains a behavior is the simplest,
the effects of differential reinforcement and punish- most direct way to suppress responding. In appli-
ment were superimposed on unknown reinforce- cation, however, extinction requires knowledge of
ment contingencies for responding (Bostow & these maintaining contingencies to ensure that the
Bailey, 1969; Burchard & Barrera, 1972; Skiba, procedural form of the intervention (e.g., withhold-
Pettigrew, & Alden, 1971). Although frequently ing attention, preventing escape from instructions)
92
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
is matched to behavioral function (e.g., maintenance extinction, extinction-induced aggression) led to the
by positive reinforcement in the form of attention or more common practice of combining extinction
negative reinforcement in the form of escape from with other treatment procedures. Research findings
instructions). have supported this practice by showing that extinc-
Early demonstrations of extinction were based tion is more effective or associated with fewer side
on hypotheses about the function of problem behav- effects when combined with differential or noncon-
ior, which were then confirmed by withholding the tingent reinforcement (E. G. Carr & Durand, 1985;
putative maintaining reinforcer. For example, Wolf, Fisher, DeLeon, Rodriguez-Catter, & Keeney, 2004;
Birnbrauer, Williams, and Lawler (1965) speculated Lerman & Iwata, 1995; Piazza, Patel, Gulotta, Sevin,
that the vomiting of a 9-year-old girl with intellec- & Layer, 2003; Steege et al., 1990; Vollmer et al.,
tual disabilities was maintained by escape from the 1998). Moreover, it appears that extinction may
classroom. The experimenters instructed the teacher often be crucial to the effectiveness of these other
to refrain from sending the student back to her dor- treatment procedures (Hagopian, Fisher, Sullivan,
mitory contingent on vomiting. The frequency of Acquisto, & LeBlanc, 1998; Mazaleski, Iwata,
vomiting decreased to zero levels across 30 days, Vollmer, Zarcone, & Smith, 1993; Zarcone, Iwata,
suggesting that the behavior was, in fact, maintained Mazaleski, & Smith, 1994).
by negative reinforcement. Lovaas and Simmons A key issue in the use of extinction is the detri-
(1969) conducted one of the earliest demonstrations mental impact of poor procedural integrity on treat-
of extinction with behavior maintained by positive ment outcomes as well as strategies to remedy this
reinforcement. The participants were two children impact. Caregivers are sometimes unwilling or
with intellectual disabilities who engaged in severe unable to completely withhold reinforcement for
self-injury. On the basis of the assumption that both problem behavior. Thus, the practical constraints of
children’s behavior was maintained by attention using extinction in applied settings have recently
from others, the children were left on their beds occasioned further research on ways to treat prob-
alone while an observer recorded instances of self- lem behavior despite continued reinforcement of the
injury from an observation room. Despite the suc- behavior.
cess in reducing the self-injury, the experimenters
concluded that extinction was an undesirable form Differential reinforcement. Interventions that
of treatment because of the initial high levels of involve delivering a reinforcer for an alternative
responding before response reduction. behavior (differential reinforcement of alterative
The development of functional analysis method- behavior [DRA]), for the absence of problem behav-
ology, as described in Volume 2, Chapter 14, this ior (differential reinforcement of other behavior
handbook, greatly facilitated the study of extinction [DRO]), and for reduced levels of problem behavior
and its procedural variations, including extinction (differential reinforcement of low rates ([DRL])
of behavior maintained by positive reinforcement remain the most common approaches to treatment.
(e.g., withholding toys; Day, Rea, Schussler, Larsen, In early treatment studies, differential reinforce-
& Johnson, 1988), extinction of behavior main- ment procedures were applied without knowledge
tained by negative reinforcement (e.g., physically of the variables maintaining the targeted behavior.
guiding compliance to prevent escape from aca- Hence, problem behavior is likely to continue to
demic demands; Iwata, Pace, Kalsher, Cowdery, & produce its maintaining reinforcer (e.g., escape from
Cataldo, 1990), and extinction of behavior main- demands) when an irrelevant reinforcer (e.g., candy)
tained by automatic reinforcement (e.g., applying is delivered when the individual met the reinforce-
protective equipment to block the putative sensory ment contingency. Although less than ideal, these
reinforcer for self-injury; Kuhn, DeLeon, Fisher, & interventions were shown to be effective in several
Wilke, 1999). studies (e.g., Allen & Harris, 1966).
Nonetheless, reports of some undesirable The use of functional reinforcers not only
effects of extinction (response bursts, resistance to increased the likelihood of success with differential
93
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
reinforcement but resulted in the development of a Motivating operations. Procedures intended to

frequently used variation of DRA called functional abolish the reinforcing effects of consequences that
communication training. With functional communi- maintain problem behavior have most commonly
cation training, the reinforcer that has maintained taken the form of response-independent delivery of
problem behavior is delivered for a communicative a reinforcer, also called noncontingent reinforcement
response (e.g., saying, “break please” to receive (e.g., Vollmer, Iwata, Zarcone, Smith, & Mazaleski,
escape) while problem behavior is extinguished. 1993). In most applications, the reinforcer that had
Other variations of DRA involve reinforcing an alter- maintained problem behavior was delivered on a
native or incompatible (noncommunicative) behav- fixed-time or variable-time schedule while problem
ior (e.g., compliance to demands, toy play). Under behavior was exposed to extinction. Other variations
DRO and DRL schedules, the person performing the of noncontingent reinforcement, however, have
behavior receives a reinforcer if problem behavior been shown to be effective, including delivery of an
does not occur, or if it has occurred less than a spec- irrelevant reinforcer (e.g., food, toys) and delivery
ified number of times, during a particular time inter- of reinforcement in the absence of extinction for
val. DRO and DRL have less clinical appeal than problem behavior (e.g., Fisher, DeLeon, Rodriguez-
DRA because no new behaviors are taught; hence, Catter & Keeney, 2004; Lalli et al., 1999; Lomas,
DRA is more commonly used in research and Fisher, & Kelley, 2010). The suppressive effects
practice. of noncontingent reinforcement also appeared to
Recent research on differential reinforcement has endure when reinforcer delivery was discontinued
focused on determinants of maintenance in applied for short periods of time (e.g., 10–15 minutes; M. A.
settings to address problems related to caregiver McGinnis, Houchins-Juárez, McDaniel, & Kennedy,
errors in implementation and transitions from inten- 2010; O’Reilly et al., 2009). Most other procedures
sive to more practical intervention (e.g., Athens & intended to abolish the reinforcing value of the con-
Vollmer, 2010; Fisher, Thompson, Hagopian, Bow- sequence have focused on modifications to aversive
man, & Krug, 2000; Hagopian, Contrucci Kuhn, stimuli that set the occasion for problem behavior.
Long, & Rush, 2005; Hanley, Iwata, & Thompson, These modifications have included reducing the
2001; Kuhn et al., 2010; Lalli et al., 1999; St. Peter frequency or pace of instructions, changing fea-
Pipkin, Vollmer, & Sloman, 2010; Vollmer, Roane, tures of tasks, embedding instructions in preferred
Ringdahl, & Marcus, 1999). This research has activities, and alternating difficult instructions with
shown the following factors to be detrimental to easier ones (e.g., Dunlap, Kern-Dunlap, Clarke, &
successful treatment outcomes: (a) failing to with- Robbins, 1991; Horner, Day, Sprague, O’Brien, &
hold reinforcement for problem behavior, (b) failing Heathfield, 1991; Kemp & Carr, 1995; Zarcone,
to deliver earned reinforcers, and (c) thinning the Iwata, Smith, Mazaleski, & Lerman, 1994). In nearly
schedule of reinforcement for appropriate behavior. all cases, these interventions were combined with
These findings have led to additional research on extinction for problem behavior.
ways to increase the success of differential reinforce-
ment despite these challenges to successful out- Punishment. A variety of procedures have been
comes. Effective strategies have included increasing effective in reducing behavior through an appar-
the quality of reinforcement for appropriate behav- ent punishment process. Research has shown that
ior when reinforcement continues to follow problem a variety of stimuli, including reprimands, physi-
behavior (Athens & Vollmer, 2010), providing cal restraint, water mist, tastes, smells, noise, and
access to alternative stimuli or activities during shock (Dorsey, Iwata, Ong, & McSween, 1980; Lalli,
schedule thinning (Fisher et al., 2000; Hagopian, Livezey, & Kates, 1996; Linscheid, Iwata, Ricketts,
Contrucci, Kuhn, Long, & Rush, 2005), and teach- Williams, & Griffin, 1990; Maglieri, DeLeon,
ing clients to respond differentially to stimuli associ- Rodriguez-Catter, & Sevin, 2000; Sajwaj, Libet, &
ated with periods of reinforcement versus extinction Agras, 1974; Stricker, Miltenberger, Garlinghouse, &
(Kuhn et al., 2010). Tulloch, 2003) can decrease problem behavior very
94
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
quickly and safely. Punishment based on the contin- problems and how behavioral processes operate
gent removal of events, including time out from pos- under more naturalistic conditions. However,
itive reinforcement and response cost, has also been despite more than 50 years of research and practice,
evaluated in the treatment of behavior disorders some essential questions remain. For example, how
(Kahng, Tarbox, & Wilke, 2001; Toole, Bowman, do behavior analysts ensure that treatment effects
Thomason, Hagopian, & Rush, 2003). Although endure over the long term? What approaches are
research on punishment has declined in recent needed to establish complex social repertoires? And
years, a substantial body of work has accumulated how do behavior analysts promote adoption of their
over the past five decades, revealing much about the technologies by those who would most benefit from
application of punishment and factors that influence them? Moreover, behavior analysts have barely
treatment outcomes. Consistent with basic findings, scratched the surface in studying some critical social
this research has indicated that punishment will problems, such as overeating, criminal behavior,
suppress behavior most effectively when the con- and schoolyard bullying. The documented success
sequence (a) is delivered immediately, (b) follows of their behavioral technologies for remediating
nearly all instances of the problem behavior, and other sorts of problems (e.g., self-injury in individu-
(c) is combined with extinction of the problem als with developmental disabilities, safety skills of
behavior (see Lerman & Vorndran, 2002, for a factory workers, reading skills of school-age chil-
review). In addition, treatment outcomes can be dren; drug use of people with addiction) suggests
enhanced by combining punishment with DRA that further research and practice into relatively
(R. H. Thompson, Iwata, Conners, & Roscoe, 1999) unexplored areas will broaden the impact and
and establishing discriminative control over the reach of ABA.
response (e.g., Maglieri et al., 2000; Piazza, Hanley, &
Fisher, 1996). Nonetheless, an insufficient amount References
of research has been conducted to develop prescrip- Agras, W. S., Jacob, R. G., & Lebedeck, M. (1980). The
tions for long-term maintenance and generalization California drought: A quasi-experimental analysis of
of punishment effects. Furthermore, although the social policy. Journal of Applied Behavior Analysis, 13,
561–570. doi:10.1901/jaba.1980.13-561
literature contains numerous reports of desirable
and undesirable side effects of punishment (e.g., Allen, K. E., & Harris, F. R. (1966). Elimination of a
child’s excessive scratching by training the mother
increases in toy play [Koegel, Firestone, Kramme, & in reinforcement procedures. Behaviour Research and
Dunlap, 1974], increases in aggression and crying Therapy, 4, 79–84. doi:10.1016/0005-7967(66)
[Hagopian & Adelinis, 2001]), no research has iden- 90046-5
tified the determinants of these outcomes. Allison, M. G., & Ayllon, R. (1980). Behavioral coaching
in the development of skills in football, gymnastics,
and tennis. Journal of Applied Behavior Analysis, 13,
Concluding Comments 297–314. doi:10.1901/jaba.1980.13-297
Athens, E. S., & Vollmer, T. R. (2010). An investigation
The ABA technologies derived from the basic princi-
of differential reinforcement of alternative behav-
ples of behavior have produced socially important ior without extinction. Journal of Applied Behavior
outcomes for many different types of people (e.g., Analysis, 43, 569–589. doi:10.1901/jaba.2010.43-569
people who abuse substances, college students, Ayllon, T., & Michael, J. (1959). The psychiatric nurse
athletes, older individuals, employees, people with as a behavioral engineer. Journal of the Experimental
intellectual disabilities), for a variety of target Analysis of Behavior, 2, 323–334. doi:10.1901/
jeab.1959.2-323
responses (e.g., literacy, smoking, sleep disturbance,
aggression, safe driving), and in a diversity of set- Azrin, N. H., & Foxx, R. M. (1971). A rapid method of
toilet training the institutionalized retarded. Journal
tings (e.g., businesses, schools, hospitals, homes). of Applied Behavior Analysis, 4, 89–99. doi:10.1901/
Research in ABA has gone beyond simple demon- jaba.1971.4-89
strations of application, generating knowledge Azrin, N. H., & Lindsley, O. R. (1956). The reinforce-
about the mechanisms that underlie common social ment of cooperation between children. Journal
95
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
of Abnormal and Social Psychology, 52, 100–102. Bosch, S., & Fuqua, R. W. (2001). Behavioral cusps:
doi:10.1037/h0042490 A model for selecting target behaviors. Journal of
Bacon-Prue, A., Blount, R., Pickering, D., & Drabman,
jaba.2001.34-123
R. (1980). An evaluation of three litter control pro-
cedures: Trash receptacles, paid workers, and the Bostow, D. E., & Bailey, J. (1969). Modification of severe
marked item techniques. Journal of Applied Behavior disruptive and aggressive behavior using brief time-
Analysis, 13, 165–170. doi:10.1901/jaba.1980.13-165 out and reinforcement procedures. Journal of Applied
Behavior Analysis, 2, 31–37. doi:10.1901/jaba.1969.2-31
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some
current dimensions of applied behavior analysis. Bourgeois, M. S. (1990). Enhancing conversation skills in
Journal of Applied Behavior Analysis, 1, 91–97. patients with Alzheimer’s disease using a prosthetic
doi:10.1901/jaba.1968.1-91 memory aid. Journal of Applied Behavior Analysis, 23,
29–42. doi:10.1901/jaba.1990.23-29
Baer, R. A., Blount, R. L., Detrich, R., & Stokes, T. F.
(1987). Using intermittent reinforcement to program Bourret, J., Vollmer, T. R., & Rapp, J. T. (2004).
maintenance of verbal/nonverbal correspondence. Evaluation of a vocal mand assessment and vocal
Journal of Applied Behavior Analysis, 20, 179–184. mand training procedures. Journal of Applied Behavior
doi:10.1901/jaba.1987.20-179 Analysis, 37, 129–144. doi:10.1901/jaba.2004.37-129
Bailey, J. S., Wolf, M. M., & Phillips, E. L. (1970). Bowman, L. G., Fisher, W. W., Thompson, R. H., &
Home-based reinforcement and the modification Piazza, C. C. (1997). On the relation of mands and
of pre-delinquents’ classroom behavior. Journal of the function of destructive behavior. Journal of
Applied Behavior Analysis, 3, 223–233. doi:10.1901/ Applied Behavior Analysis, 30, 251–265. doi:10.1901/
jaba.1970.3-223 jaba.1997.30-251
Bancroft, S. L., Weiss, J. S., Libby, M. E., & Ahern, W. H. Boyle, M. E., & Greer, R. D. (1983). Operant procedures
(2011). A comparison of procedural variations in and the comatose patient. Journal of Applied Behavior
teaching behavior chains: Manual guidance, trainer Analysis, 16, 3–12. doi:10.1901/jaba.1983.16-3
completion, and no completion of untrained steps. Brigham, T. A., Graubard, P. S., & Stans, A. (1972).
Journal of Applied Behavior Analysis, 44, 559–569. Analysis of the effects of sequential reinforcement
Bannerman, D. J., Sheldon, J. B., & Sherman, J. A. (1991). contingencies on aspects of composition. Journal of
Teaching adults with severe and profound retarda- Applied Behavior Analysis, 5, 421–429. doi:10.1901/
tion to exit their homes upon hearing the fire alarm. jaba.1972.5-421
Journal of Applied Behavior Analysis, 24, 571–577. Budd, K. S., Green, D. R., & Baer, D. M. (1976). An anal-
doi:10.1901/jaba.1991.24-571 ysis of multiple misplaced parental social contingen-
Barton, E. J., & Ascione, F. R. (1979). Sharing in pre- cies. Journal of Applied Behavior Analysis, 9, 459–470.
school children: Facilitation, stimulus generalization, doi:10.1901/jaba.1976.9-459
response generalization, and maintenance. Journal of Burchard, J. D., & Barrera, F. (1972). An analysis of
Applied Behavior Analysis, 12, 417–430. doi:10.1901/ timeout and response cost in a programmed envi-
jaba.1979.12-417 ronment. Journal of Applied Behavior Analysis, 5,
Bates, P. (1980). The effectiveness of interpersonal 271–282. doi:10.1901/jaba.1972.5-271
skills training on the social skill acquisition of Carnine, D. W. (1976). Effects of two teacher-presentation
moderately and mildly retarded adults. Journal of rates on off-task behavior, answering correctly, and
Applied Behavior Analysis, 13, 237–248. doi:10.1901/ participation. Journal of Applied Behavior Analysis, 9,
jaba.1980.13-237 199–206. doi:10.1901/jaba.1976.9-199
Bennett, C. W., & Ling, D. (1972). Teaching a complex Carr, E. G., & Durand, V. M. (1985). Reducing behavior
verbal response to a hearing-impaired girl. Journal of problems through functional communication train-
Applied Behavior Analysis, 5, 321–327. doi:10.1901/ ing. Journal of Applied Behavior Analysis, 18, 111–126.
jaba.1972.5-321 doi:10.1901/jaba.1985.18-111
Bijou, S. W. (1955). A systematic approach to an Carr, E. G., Newsom, C. D., & Binkoff, J. A. (1980).
experimental analysis of young children. Child Escape as a factor in the aggressive behavior of
Development, 26, 161–168. two retarded children. Journal of Applied Behavior
Bornstein, P. H., Bach, P. J., McFall, M. E., Friman, P. C., Analysis, 13, 101–117. doi:10.1901/jaba.1980.13-101
& Lyons, P. D. (1980). Application of a social skills Carr, J. E., Austin, J. E., Britton, L. N., Kellum, K. K., &
training program in the modification of interpersonal Bailey, J. S. (1999). An assessment of social valid-
deficits among retarded adults: A clinical replication. ity trends in applied behavior analysis. Behavioral
Journal of Applied Behavior Analysis, 13, 171–176. Interventions, 14, 223–231. doi:10.1002/(SICI)1099-
doi:10.1901/jaba.1980.13-171 078X(199910/12)14:4<223::AID-BIN37>3.0.CO;2-Y
96
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Carr, J. E., Nicolson, A. C., & Higbee, T. S. (2000). Journal of Applied Behavior Analysis, 11, 345–355.
Evaluation of a brief multiple-stimulus prefer- doi:10.1901/jaba.1978.11-345
ence assessment in a naturalistic context. Journal of Dallery, J., & Glenn, I. M. (2005). Effects of an Internet-
Applied Behavior Analysis, 33, 353–357. doi:10.1901/ based voucher reinforcement program for smoking
jaba.2000.33-353 abstinence: A feasibility study. Journal of Applied
Cautela, J. R., & Kastenbaum, R. (1967). A reinforce- Behavior Analysis, 38, 349–357. doi:10.1901/jaba.
ment survey schedule for use in therapy, training, 2005.150-04
and research. Psychological Reports, 20, 1115–1130. Day, R. M., Rea, J. A., Schussler, N. G., Larsen, S. E.,
doi:10.2466/pr0.1967.20.3c.1115 & Johnson, W. L. (1988). A functionally based
Charlop, M. H., & Walsh, M. E. (1986). Increasing autis- approach to the treatment of self-injurious behavior.
tic children’s spontaneous verbalizations of affec- Behavior Modification, 12, 565–589. doi:10.1177/
tion: An assessment of time delay and peer modeling 01454455880124005
procedures. Journal of Applied Behavior Analysis, 19, DeLeon, I. G., & Iwata, B. A. (1996). Evaluation of a
307–314. doi:10.1901/jaba.1986.19-307 multiple-stimulus presentation format for assessing
Clarfield, J., & Stoner, G. (2005). The effects of com- reinforcer preferences. Journal of Applied Behavior
puterized reading instruction on the academic per- Analysis, 29, 519–533. doi:10.1901/jaba.1996.29-519
formance of students identified with ADHD. School Demchak, M. (1990). Response prompting and fad-
Psychology Review, 34, 246–254. ing methods: A review. American Journal on Mental
Clark, H. B., Rowbury, T., Baer, A. M., & Baer, D. M. Retardation, 94, 603–615.
(1973). Timeout as a punishing stimulus in con- Dorsey, M. F., Iwata, B. A., Ong, P., & McSween, T. E.
tinuous and intermittent schedules. Journal of (1980). Treatment of self-injurious behavior using
Applied Behavior Analysis, 6, 443–455. doi:10.1901/ a water mist: Initial response suppression and gen-
jaba.1973.6-443 eralization. Journal of Applied Behavior Analysis, 13,
Clarke, S., Remington, B., & Light, P. (1986). An evalu- 343–353. doi:10.1901/jaba.1980.13-343
ation of the relationship between receptive speech Ducharme, D. E., & Holborn, S. W. (1997). Programming
skills and expressive signing. Journal of Applied generalization of social skills in preschool children
Behavior Analysis, 19, 231–239. doi:10.1901/jaba. with hearing impairments. Journal of Applied Behavior
1986.19-231 Analysis, 30, 639–651. doi:10.1901/jaba.1997.30-639
Cote, C. A., Thompson, R. H., Hanley, G. P., & Duffy, L., & Wishart, J. G. (1987). A comparison of two
McKerchar, P. M. (2007). Teacher report versus procedures for teaching discrimination skills to
direct assessment of preferences for identifying rein- Down’s syndrome and non-handicapped children.
forcers for young children. Journal of Applied Behavior British Journal of Educational Psychology, 57, 265–278.
Analysis, 40, 157–166. doi:10.1901/jaba.2007.177-05 doi:10.1111/j.2044-8279.1987.tb00856.x
Cronin, K. A., & Cuvo, A. J. (1979). Teaching mending Duker, P. C., & van Lent, C. (1991). Inducing variability
skills to mentally retarded adolescents. Journal of in communicative gestures used by severely retarded
Applied Behavior Analysis, 12, 401–406. doi:10.1901/ individuals. Journal of Applied Behavior Analysis, 24,
jaba.1979.12-401 379–386. doi:10.1901/jaba.1991.24-379
Cumming, W. W., & Schoenfeld, W. N. (1959). Some Dunlap, G., Kern-Dunlap, L., Clarke, S., & Robbins,
data on behavioral reversibility in a steady state F. R. (1991). Functional assessment, curricular
experiment. Journal of the Experimental Analysis of revision, and severe behavior problems. Journal of
Behavior, 2, 87–90. doi:10.1901/jeab.1959.2-87 Applied Behavior Analysis, 24, 387–397. doi:10.1901/
Cummings, A. R., & Carr, J. E. (2009). Evaluating jaba.1991.24-387
progress in behavioral programs for children with Durand, V. M., & Carr, E. G. (1991). Functional com-
autism spectrum disorders via continuous and dis- munication training to reduce challenging behav-
continuous measurement. Journal of Applied Behavior ior: Maintenance and application in new settings.
Analysis, 42, 57–71. doi:10.1901/jaba.2009.42-57 Journal of Applied Behavior Analysis, 24, 251–264.
Cuvo, A. J., Davis, P. K., O’Reilly, M. F., Mooney, B. doi:10.1901/jaba.1991.24-251
M., & Crowley, R. (1992). Promoting stimulus Durand, V. M., & Carr, E. G. (1992). An analysis of
control with textual prompts and performance feed- maintenance following functional communication
back for persons with mild disabilities. Journal of training. Journal of Applied Behavior Analysis, 25,
Applied Behavior Analysis, 25, 477–489. doi:10.1901/ 777–794. doi:10.1901/jaba.1992.25-777
jaba.1992.25-477
Egel, A. L. (1981). Reinforcer variation: Implications
Cuvo, A. J., Leaf, R. B., & Borakove, L. S. (1978). for motivating developmentally disabled children.
Teaching janitorial skills to the mentally retarded: Journal of Applied Behavior Analysis, 14, 345–350.
Acquisition, generalization, and maintenance. doi:10.1901/jaba.1981.14-345
97
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Favell, J. E., Azrin, N. H., Baumeister, A. A., Carr, E. R. W. Fuqua (Eds.), Research methods in applied
G., Dorsey, M. F., Forehand, R., & Solnick, J. V. behavior analysis (pp. 265–292). New York, NY:
(1982). The treatment of self-injurious behavior. Plenum Press.
Behavior Therapy, 13, 529–554. doi:10.1016/S0005-
Garcia, E., Baer, D. M., & Firestone, I. (1971). The
7894(82)80015-4
development of generalized imitation within topo-
Fisher, W. W., DeLeon, I. G., Rodriguez-Catter, V., & graphically determined boundaries. Journal of
Keeney, K. M. (2004). Enhancing the effects of Applied Behavior Analysis, 4, 101–112. doi:10.1901/
extinction on attention-maintained behavior through jaba.1971.4-101
noncontingent delivery of attention or stimuli identi- Gast, D. L., VanBiervliet, A., & Spradlin, J. E. (1979).
fied via a competing stimulus assessment. Journal of Teaching number-word equivalences: A study of
Applied Behavior Analysis, 37, 171–184. doi:10.1901/ transfer. American Journal of Mental Deficiency, 83,
jaba.2004.37-171 524–527.
Fisher, W. W., Piazza, C. C., Bowman, L. G., & Amari, A. Geiger, K. B., LeBlanc, L. A., Dillon, C. M., & Bates, S. L.
(1996). Integrating caregiver report with a direct (2010). An evaluation of preference for video and in
choice assessment to enhance reinforcer identifica- vivo modeling. Journal of Applied Behavior Analysis,
tion. American Journal on Mental Retardation, 101, 43, 279–283. doi:10.1901/jaba.2010.43-279
15–25.
Gewirtz, J. L., & Baer, D. M. (1958). The effect of brief
Fisher, W. W., Piazza, C. C., Bowman, L. G., Hagopian, social deprivation on behaviors for a social rein-
L. P., Owens, J. C., & Slevin, I. (1992). A comparison forcer. Journal of Abnormal and Social Psychology, 57,
of two approaches for identifying reinforcers for per- 165–172. doi:10.1037/h0042880
sons with severe and profound disabilities. Journal of
Applied Behavior Analysis, 25, 491–498. doi:10.1901/ Gleeson, S., & Lattal, K. A. (1987). Response-reinforcer
jaba.1992.25-491 relations and the maintenance of behavior. Journal of
Fisher, W. W., Thompson, R. H., Hagopian, L. P., doi:10.1901/jeab.1987.48-383
Bowman, L. G., & Krug, A. (2000). Facilitating tol-
erance of delayed reinforcement during functional Glover, J., & Gary, A. L. (1976). Procedures to increase
communication training. Behavior Modification, 24, some aspects of creativity. Journal of Applied Behavior
3–29. doi:10.1177/0145445500241001 Analysis, 9, 79–84. doi:10.1901/jaba.1976.9-79
Fisher, W. W., Thompson, R. H., Piazza, C. C., Crosland, Godby, S., Gast, D. L., & Wolery, M. (1987). A com-
K., & Gotjen, D. (1997). On the relative reinforc- parison of time delay and system of least prompts
ing effects of choice and differential consequences. in teaching object discrimination. Research in
Journal of Applied Behavior Analysis, 30, 423–438. Developmental Disabilities, 8, 283–305. doi:10.1016/
doi:10.1901/jaba.1997.30-423 0891-4222(87)90009-6
Fixsen, D. L., Phillips, E. L., & Wolf, M. M. (1973). Goetz, E. M., & Baer, D. M. (1973). Social control of
Achievement place: Experiments in self-government form diversity and the emergence of new forms in
with pre-delinquents. Journal of Applied Behavior children’s blockbuilding. Journal of Applied Behavior
Analysis, 6, 31–47. doi:10.1901/jaba.1973.6-31 Analysis, 6, 209–217. doi:10.1901/jaba.1973.6-209
Foxx, R. M., & Shapiro, S. T. (1978). The timeout ribbon: Greene, B. F., Bailey, J. S., & Barber, F. (1981). An analy-
A nonexclusionary timeout procedure. Journal of sis and reduction of disruptive behavior on school
Applied Behavior Analysis, 11, 125–136. doi:10.1901/ buses. Journal of Applied Behavior Analysis, 14,
jaba.1978.11-125 177–192. doi:10.1901/jaba.1981.14-177
France, K. G., & Hudson, S. M. (1990). Behavior man- Guess, D., & Baer, D. M. (1973). An analysis of indi-
agement of infant sleep disturbance. Journal of vidual differences in generalization between recep-
Applied Behavior Analysis, 23, 91–98. doi:10.1901/ tive and productive language in retarded children.
jaba.1990.23-91 Journal of Applied Behavior Analysis, 6, 311–329.
doi:10.1901/jaba.1973.6-311
Frankosky, R., & Sulzer-Azaroff, B. (1978). Individual
and group contingencies and collateral social behav- Guess, D., Sailor, W., Rutherford, G., & Baer, D. M.
iors. Behavior Therapy, 9, 313–327. doi:10.1016/ (1968). An experimental analysis of linguistic devel-
S0005-7894(78)80075-6 opment: The productive use of the plural morpheme.
Fuller, P. R. (1949). Operant conditioning of a vegetative doi:10.1901/jaba.1968.1-297
human organism. American Journal of Psychology, 62,
587–590. doi:10.2307/1418565 Hagopian, L. P., & Adelinis, J. D. (2001). Response
blocking with and without redirection for the treat-
Fuqua, R. W., & Schwade, J. (1986). Social validation ment of pica. Journal of Applied Behavior Analysis, 34,
and applied behavioral research. In A. Poling & 527–530. doi:10.1901/jaba.2001.34-527
98
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Hagopian, L. P., Bruzek, J. L., Bowman, L. G., & Jennett, Applied Behavior Analysis, 13, 407–432. doi:10.1901/
H. K. (2007). Assessment and treatment of problem jaba.1980.13-407
behavior occasioned by interruption of free-operant
Hawkins, R. P. (1991). Is social validity what we are
behavior. Journal of Applied Behavior Analysis, 40,
interested in? Argument for a functional approach.
89–103. doi:10.1901/jaba.2007.63-05
Hagopian, L. P., Contrucci Kuhn, S. A., Long, E. S., & doi:10.1901/jaba.1991.24-205
Rush, K. S. (2005). Schedule thinning following
communication training: Using competing stimuli Hernandez, E., Hanley, G. P., Ingvarsson, E. T., &
to enhance tolerance to decrements in reinforcer Tiger, J. H. (2007). A preliminary evaluation of
density. Journal of Applied Behavior Analysis, 38, the emergence of novel mand forms. Journal of
177–193. doi:10.1901/jaba.2005.43-04 Applied Behavior Analysis, 40, 137–156. doi:10.1901/
jaba.2007.96-05
Hagopian, L. P., Fisher, W. W., Sullivan, M. T., Acquisto,
J., & LeBlanc, L. A. (1998). Effectiveness of func- Hopkins, B. L. (1968). Effects of candy and social rein-
tional communication training with and without forcement, instructions, and reinforcement schedule
extinction and punishment: A summary of 21 inpa- leaning on the modification and maintenance of
tient cases. Journal of Applied Behavior Analysis, 31, smiling. Journal of Applied Behavior Analysis, 1,
211–235. doi:10.1901/jaba.1998.31-211 121–129. doi:10.1901/jaba.1968.1-121
Halle, J. W., & Holt, B. (1991). Assessing stimulus con- Horner, R. H., Day, H. M., Sprague, J. R., O’Brien, M.,
trol in natural settings: An analysis of stimuli that & Heathfield, L. T. (1991). Interspersed requests:
acquire control during training. Journal of Applied A nonaversive procedure for reducing aggres-
Behavior Analysis, 24, 579–589. doi:10.1901/jaba. sion and self-injury during instruction. Journal of
1991.24-579 Applied Behavior Analysis, 24, 265–278. doi:10.1901/
jaba.1991.24-265
Hanley, G. P., Iwata, B. A., & Roscoe, E. M. (2006).
Factors influencing the stability of preferences. Hughes, H. M. (1979). Behavior change in children at a
Journal of Applied Behavior Analysis, 39, 189–202. therapeutic summer camp as a function of feedback
doi:10.1901/jaba.2006.163-04 and individual versus group contingencies. Journal of
Abnormal Child Psychology, 7, 211–219. doi:10.1007/
Hanley, G. P., Iwata, B. A., & Thompson, R. H. (2001).
BF00918901
Reinforcement schedule thinning following treat-
ment with functional communication training. Hunter, I., & Davison, M. (1982). Independence of
Journal of Applied Behavior Analysis, 34, 17–38. response force and reinforcement rate on concurrent
doi:10.1901/jaba.2001.34-17 variable-interval schedule performance. Journal of
Hanley, G. P., Piazza, C. C., Fisher, W. W., Contrucci, the Experimental Analysis of Behavior, 37, 183–197.
S. A., & Maglieri, K. A. (1997). Evaluation of client doi:10.1901/jeab.1982.37-183
preference for function-based treatment packages. Isaacs, C. D., Embry, L. H., & Baer, D. M. (1982).
Journal of Applied Behavior Analysis, 30, 459–473. Training family therapists: An experimental analysis.
doi:10.1901/jaba.1997.30-459 Journal of Applied Behavior Analysis, 15, 505–520.
Haring, T. G., & Breen, C. G. (1992). A peer-mediated doi:10.1901/jaba.1982.15-505
social network intervention to enhance the social Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E.,
integration of persons with moderate and severe & Richman, G. S. (1982). Toward a functional
disabilities. Journal of Applied Behavior Analysis, 25, analysis of self-injury. Analysis and Intervention in
319–333. doi:10.1901/jaba.1992.25-319 Developmental Disabilities, 2, 3–20. doi:10.1016/
Hart, B. M., & Risley, T. R. (1968). Establishing use of 0270-4684(82)90003-9
descriptive adjectives in the spontaneous speech Iwata, B. A., Pace, G. M., Kalsher, M. J., Cowdery, G. E.,
of disadvantaged preschool children. Journal of & Cataldo, M. F. (1990). Experimental analysis and
Applied Behavior Analysis, 1, 109–120. doi:10.1901/ extinction of self-injurious escape behavior. Journal
jaba.1968.1-109 of Applied Behavior Analysis, 23, 11–27. doi:10.1901/
Hart, B., & Risley, T. R. (1974). Using preschool materi- jaba.1990.23-11
als to modify the language of disadvantaged children. Jackson, D. A., & Wallace, R. F. (1974). The modification
Journal of Applied Behavior Analysis, 7, 243–256. and generalization of voice loudness in a fifteen-year-
doi:10.1901/jaba.1974.7-243 old retarded girl. Journal of Applied Behavior Analysis,
Hart, B., & Risley, T. R. (1975). Incidental teaching of 7, 461–471. doi:10.1901/jaba.1974.7-461
language in the preschool. Journal of Applied Behavior Johnson, M. D., & Fawcett, S. B. (1994). Courteous ser-
Analysis, 8, 411–420. doi:10.1901/jaba.1975.8-411 vice: Its assessment and modification in a human
Hart, B., & Risley, T. R. (1980). In vivo language inter- service organization. Journal of Applied Behavior
vention: Unanticipated general effects. Journal of Analysis, 27, 145–152. doi:10.1901/jaba.1994.27-145
99
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Jones, R. J., & Azrin, N. H. (1969). Behavioral engi- Koegel, R. L., Firestone, P. B., Kramme, K. W., & Dunlap,
neering: Stuttering as a function of stimulus dura- G. (1974). Increasing spontaneous play by suppress-
tion during speech synchronization. Journal of ing self-stimulation in autistic children. Journal of
Applied Behavior Analysis, 2, 223–229. doi:10.1901/ Applied Behavior Analysis, 7, 521–528. doi:10.1901/
jaba.1969.2-223 jaba.1974.7-521
Jones, R. T., Kazdin, A. E., & Haney, J. I. (1981). Social Koegel, R. L., & Frea, W. D. (1993). Treatment of social
validation and training of emergency fire safety behavior in autism through the modification of piv-
skills for potential injury prevention and life saving. otal social skills. Journal of Applied Behavior Analysis,
Journal of Applied Behavior Analysis, 14, 249–260. 26, 369–377. doi:10.1901/jaba.1993.26-369
doi:10.1901/jaba.1981.14-249 Koegel, R. L., & Rincover, A. (1977). Research on the dif-
Kahng, S., Iwata, B. A., & Lewin, A. (2002). Behavioral ference between generalization and maintenance in
treatment of self-injury, 1964–2000. American extra-therapy responding. Journal of Applied Behavior
Journal on Mental Retardation, 107, 212–221. Analysis, 10, 1–12. doi:10.1901/jaba.1977.10-1
doi:10.1352/0895-8017(2002)107<0212:BTOSIT>
Kuhn, D. E., Chirighin, A. E., & Zelenka, K. (2010).
2.0.CO;2
Discriminated functional communication: A pro-
Kahng, S. W., Tarbox, J., & Wilke, A. E. (2001). Use of a cedural extension of functional communication
multicomponent treatment for food refusal. Journal training. Journal of Applied Behavior Analysis, 43,
of Applied Behavior Analysis, 34, 93–96. doi:10.1901/ 249–264. doi:10.1901/jaba.2010.43-249
jaba.2001.34-93
Kuhn, D. E., DeLeon, I. G., Fisher, W. W., & Wilke,
Kale, R. J., Kaye, J. H., Whelan, P. A., & Hopkins, B. L. A. E. (1999). Clarifying an ambiguous functional
(1968). The effects of reinforcement on the modi- analysis with matched and mismatched extinction
fication, maintenance, and generalization of social procedures. Journal of Applied Behavior Analysis, 32,
responses of mental patients. Journal of Applied 99–102. doi:10.1901/jaba.1999.32-99
Behavior Analysis, 1, 307–314. doi:10.1901/jaba.
Lalli, J. S., Livezey, K., & Kates, K. (1996). Functional
1968.1-307
analysis and treatment of eye poking with response
Kazdin, A. E. (1977). Artifact, bias, and complexity blocking. Journal of Applied Behavior Analysis, 29,
of assessment: The ABCs of reliability. Journal of 129–132. doi:10.1901/jaba.1996.29-129
jaba.1977.10-141 Lalli, J. S., Vollmer, T. R., Progar, P. R., Wright,
C., Borrero, J., Daniel, D., & May, W. (1999).
Kazdin, A. E. (1978). History of behavior modification. Competition between positive and negative rein-
Baltimore, MD: University Park Press. forcement in the treatment of escape behavior.
Kazdin, A. E., & Bootzin, R. R. (1972). The token econ- Journal of Applied Behavior Analysis, 32, 285–296.
omy: An evaluative review. Journal of Applied Behavior doi:10.1901/jaba.1999.32-285
Analysis, 5, 343–372. doi:10.1901/jaba.1972.5-343 Laraway, S., Snycerski, S., Michael, J., & Poling, A.
Kazdin, A. E., & Polster, R. (1973). Intermittent token (2003). Motivating operations and terms to describe
reinforcement and response maintenance in extinc- them: Some further refinements. Journal of Applied
tion. Behavior Therapy, 4, 386–391. doi:10.1016/ Behavior Analysis, 36, 407–414. doi:10.1901/jaba.
S0005-7894(73)80118-2 2003.36-407
Kemp, D. C., & Carr, E. G. (1995). Reduction of severe Lerman, D. C., & Iwata, B. A. (1995). Prevalence of the
problem behavior in community employment using extinction burst and its attenuation during treat-
an hypothesis-driven multicomponent intervention ment. Journal of Applied Behavior Analysis, 28, 93–94.
approach. Journal of the Association for Persons With doi:10.1901/jaba.1995.28-93
Severe Handicaps, 20, 229–247. Lerman, D. C., Tetreault, A., Hovanetz, A., Bellaci, E.,
Kirby, K. C., & Bickel, W. K. (1988). Toward an explicit Miller, J., Karp, H., & Toupard, A. (2010). Applying
analysis of generalization: A stimulus control inter- signal detection theory to the study of observer accu-
pretation. Behavior Analyst, 11, 115–129. racy and bias in behavioral assessment. Journal of
Kirby, K. C., & Holborn, S. W. (1986). Trained, general-
jaba.2010.43-195
ized, and collateral behavior changes of preschool
children receiving gross-motor skills training. Lerman, D. C., & Vorndran, C. (2002). On the status
Journal of Applied Behavior Analysis, 19, 283–288. of knowledge for using punishment: Implications
doi:10.1901/jaba.1986.19-283 for treating behavior disorders. Journal of Applied
Behavior Analysis, 35, 431–464. doi:10.1901/
Koegel, R. L., Dunlap, G., & Dyer, K. (1980). Intertrial
jaba.2002.35-431
interval duration and learning in autistic children.
Journal of Applied Behavior Analysis, 13, 91–99. Lerman, D. C., Vorndran, C., Addison, L., & Kuhn, S. A. C.
doi:10.1901/jaba.1980.13-91 (2004). A rapid assessment of skills in young
100
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
children with autism. Journal of Applied Behavior reinforcement for problem behavior. Journal of
Analysis, 37, 11–26. doi:10.1901/jaba.2004.37-11 Applied Behavior Analysis, 43, 119–123. doi:10.1901/
jaba.2010.43-119
Lindsley, O. R. (1956). Operant conditioning meth-
ods applied to research in chronic schizophrenia. Meany-Daboul, M. G., Roscoe, E. R., Bourret, J. C., &
Psychiatric Research Reports, 5, 118–139. Ahearn, W. A. (2007). A comparison of momentary
Linscheid, T. R., Iwata, B. A., Ricketts, R. W., Williams, time sampling and partial-interval recording for evalu-
D. E., & Griffin, J. C. (1990). Clinical evaluation of ating functional relations. Journal of Applied Behavior
the self-injurious behavior inhibiting device (SIBIS). Analysis, 40, 501–514. doi:10.1901/jaba.2007.40-501
Journal of Applied Behavior Analysis, 23, 53–78. Moher, C. A., Gould, D. D., Hegg, E., & Mahoney, A. M.
doi:10.1901/jaba.1990.23-53 (2008). Non-generalized and generalized condi-
Lomas, J. E., Fisher, W. W., & Kelley, M. E. (2010). The tioned reinforcers: Establishment and validation.
effects of variable-time delivery of food items and Behavioral Interventions, 23, 13–38. doi:10.1002/
praise on problem on problem behavior reinforced bin.253
by escape. Journal of Applied Behavior Analysis, 43, Mudford, O. C., Taylor, S. A., & Martin, N. T. (2009).
425–435. doi:10.1901/jaba.2010.43-425 Continuous recording and interobserver agreement
Lovaas, O. I., Freitag, G., Gold, V. J., & Kassorla, I. C. algorithms reported in the Journal of Applied Behavior
(1965). Experimental studies in childhood schizo- Analysis (1995–2005). Journal of Applied Behavior
phrenia: Analysis of self-destructive behavior. Analysis, 42, 165–169. doi:10.1901/jaba.2009.42-165
Journal of Experimental Child Psychology, 2, 67–84. Mueller, M. M., Olmi, D. J., & Saunders, K. J. (2000).
doi:10.1016/0022-0965(65)90016-0 Recombinative generalization of within-syllable units
Lovaas, O. I., & Simmons, J. Q. (1969). Manipulation of in prereading children. Journal of Applied Behavior
self-destruction in three retarded children. Journal of Analysis, 33, 515–531. doi:10.1901/jaba.2000.33-515
Applied Behavior Analysis, 2, 143–157. doi:10.1901/ Ninness, C., Barnes-Holmes, D., Rumph, R., McCuller,
jaba.1969.2-143 G., Ford, A. M., Payne, R., & Elliott, M. P. (2006).
Lutzker, J. R., & Sherman, J. A. (1974). Producing gener- Transformations of mathematical and stimulus
ative sentence usage by imitation and reinforcement functions. Journal of Applied Behavior Analysis, 39,
procedures. Journal of Applied Behavior Analysis, 7, 299–321. doi:10.1901/jaba.2006.139-05
447–460. doi:10.1901/jaba.1974.7-447 Nutter, D., & Reid, D. H. (1978). Teaching retarded
Maglieri, K. A., DeLeon, I. G., Rodriguez-Catter, V., & women a clothing selection skill using community
Sevin, B. M. (2000). Treatment of covert food steal- norms. Journal of Applied Behavior Analysis, 11,
ing in an individual with Prader-Willi syndrome. 475–487. doi:10.1901/jaba.1978.11-475
Journal of Applied Behavior Analysis, 33, 615–618. O’Brien, F., & Azrin, N. H. (1972). Developing proper
doi:10.1901/jaba.2000.33-615 mealtime behaviors of the institutionalized retarded.
Mann, R. A. (1972). The behavior-therapeutic use of Journal of Applied Behavior Analysis, 5, 389–399.
contingency contracting to control an adult behavior doi:10.1901/jaba.1972.5-389
problem: Weight control. Journal of Applied Behavior Odom, S. L., Chandler, L. K., Ostrosky, M., McConnell,
Analysis, 5, 99–109. doi:10.1901/jaba.1972.5-99 S. R., & Reaney, S. (1992). Fading teacher prompts
Marzullo-Kerth, D., Reeve, S. A., Reeve, K. F., & from peer-initiation interventions for young children
Townsend, D. B. (2011). Using multiple-exemplar with disabilities. Journal of Applied Behavior Analysis,
training to teach a generalized repertoire of sharing 25, 307–317. doi:10.1901/jaba.1992.25-307
to children with autism. Journal of Applied Behavior O’Reilly, M., Lang, R., Davis, T., Rispoli, M., Machalicek,
Analysis, 44, 279–294. W., Sigafoos, J., & Didden, R. (2009). A systematic
Mazaleski, J. L., Iwata, B. A., Vollmer, T. R., Zarcone, J. R., examination of different parameters of presession
& Smith, R. G. (1993). Analysis of the reinforcement exposure to tangible stimuli that maintain problem
and extinction components in DRO contingencies behavior. Journal of Applied Behavior Analysis, 42,
with self-injury. Journal of Applied Behavior Analysis, 773–783. doi:10.1901/jaba.2009.42-773
26, 143–156. doi:10.1901/jaba.1993.26-143 Pace, G. M., Ivancic, M. T., Edwards, G. L., Iwata, B. A.,
McGinnis, J. C., Friman, P. C., & Carlyon, W. D. (1999). & Page, T. J. (1985). Assessment of stimulus prefer-
The effect of token rewards on “intrinsic” motivation ence assessment and reinforcer value with profoundly
for doing math. Journal of Applied Behavior Analysis, retarded individuals. Journal of Applied Behavior
32, 375–379. doi:10.1901/jaba.1999.32-375 Analysis, 18, 249–255. doi:10.1901/jaba.1985.18-249
McGinnis, M. A., Houchins-Juarez, N., McDaniel, J. L., Pelios, L., Morren, J., Tesch, D., & Axelrod, S. (1999).
& Kennedy, C. H. (2010). Abolishing and establish- The impact of functional analysis methodology on
ing operation analyses of social attention as positive treatment choice for self-injurious and aggressive
101
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
behavior. Journal of Applied Behavior Analysis, 32, Rincover, A., Cook, R., Peoples, A., & Packard, D.
185–195. doi:10.1901/jaba.1999.32-185 (1979). Sensory extinction and sensory reinforce-
ment principles for programming multiple adaptive
Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982).
behavior change. Journal of Applied Behavior Analysis,
The integrity of independent variables in behavior
12, 221–233. doi:10.1901/jaba.1979.12-221
analysis. Journal of Applied Behavior Analysis, 15,
477–492. doi:10.1901/jaba.1982.15-477 Roane, H. S., Call, N. A., & Falcomata, T. S. (2005). A pre-
liminary analysis of adaptive responding under open
Phillips, E. L. (1968). Achievement place: Token rein-
and closed economies. Journal of Applied Behavior
forcement procedures in a home-style rehabilita-
Analysis, 38, 335–348. doi:10.1901/jaba.2005.85-04
tion setting for “pre-delinquent” boys. Journal of
Applied Behavior Analysis, 1, 213–223. doi:10.1901/ Rolider, A., & Van Houten, R. (1985). Movement suppres-
jaba.1968.1-213 sion time-out for undesirable behavior in psychotic
and severely developmentally delayed children. Journal
Phillips, E. L., Phillips, E. A., Fixsen, D. L., & Wolf, M. M. of Applied Behavior Analysis, 18, 275–288. doi:10.1901/
(1971). Achievement Place: Modification of the jaba.1985.18-275
behaviors of pre-delinquent boys within a token
economy. Journal of Applied Behavior Analysis, 4, Rosales-Ruiz, J., & Baer, D. M. (1997). Behavioral cusps:
45–59. doi:10.1901/jaba.1971.4-45 A developmental and pragmatic concept for behav-
ior analysis. Journal of Applied Behavior Analysis, 30,
Phillips, E. L., Phillips, E. A., Wolf, M. M., & Fixsen, D. L. 533–544. doi:10.1901/jaba.1997.30-533
(1973). Achievement Place: Development of the
elected manager system. Journal of Applied Behavior Sajwaj, T., Libet, J., & Agras, S. (1974). Lemon-juice
Analysis, 6, 541–561. doi:10.1901/jaba.1973.6-541 therapy: The control of life-threatening rumination
in a six-month-old infant. Journal of Applied Behavior
Piazza, C. C., Hanley, G. P., & Fisher, W. W. (1996). Analysis, 7, 557–563. doi:10.1901/jaba.1974.7-557
Functional analysis and treatment of cigarette pica.
Journal of Applied Behavior Analysis, 29, 437–450. Schaefer, H. H. (1970). Self-injurious behavior: Shaping
doi:10.1901/jaba.1996.29-437 “head-banging” in monkeys. Journal of Applied
Piazza, C. C., Patel, M. R., Gulotta, C. S., Sevin, B. M., & 1970.3-111
Layer, S. A. (2003). On the relative contributions of
positive reinforcement and escape extinction in the Schreibman, L. (1975). Effects of within-stimulus and
treatment of food refusal. Journal of Applied Behavior extra-stimulus prompting on discrimination learn-
Analysis, 36, 309–324. doi:10.1901/jaba.2003.36-309 ing in autistic children. Journal of Applied Behavior
Pierce, K. L., & Schreibman, L. (1994). Teaching daily
living skills to children with autism in unsuper- Schroeder, S. R., & Holland, J. G. (1968). Operant con-
vised settings through pictorial self-management. trol of eye movements. Journal of Applied Behavior
Journal of Applied Behavior Analysis, 27, 471–481. Analysis, 1, 161–166. doi:10.1901/jaba.1968.1-161
doi:10.1901/jaba.1994.27-471 Schuster, J. W., Gast, D. L., Wolery, M., & Guiltinan, S.
Pilgrim, C., Jackson, J., & Galizio, M. (2000). Acquisition (1988). The effectiveness of a constant time-delay
of arbitrary conditional discriminations by young procedure to teach chained responses to adolescents
normally developing children. Journal of the with mental retardation. Journal of Applied Behavior
Experimental Analysis of Behavior, 73, 177–193. Analysis, 21, 169–178. doi:10.1901/jaba.1988.21-169
doi:10.1901/jeab.2000.73-177 Serna, L. A., Schumaker, J. B., Sherman, J. A., & Sheldon,
Porterfield, J. K., Herbert-Jackson, E., & Risley, T. R. J. B. (1991). In-home generalization of social inter-
(1976). Contingent observation: An effective and actions in families of adolescents with behavior
acceptable procedure for reducing disruptive behav- problems. Journal of Applied Behavior Analysis, 24,
ior of young children in a group setting. Journal of 733–746. doi:10.1901/jaba.1991.24-733
Applied Behavior Analysis, 9, 55–64. doi:10.1901/ Seymour, F. W., & Stokes, T. F. (1976). Self-recording in
jaba.1976.9-55 training girls to increase work and evoke staff praise in
Repp, A. C., & Karsh, K. G. (1992). An analysis of a an institution for offenders. Journal of Applied Behavior
group teaching procedure for persons with devel- Analysis, 9, 41–54. doi:10.1901/jaba.1976.9-41
opmental disabilities. Journal of Applied Behavior Shore, B. A., Iwata, B. A., Lerman, D. C., & Shirley, M. J.
Analysis, 25, 701–712. doi:10.1901/jaba.1992.25-701 (1994). Assessing and programming generalized
Resnick, L. B., Wang, M. C., & Kaplan, J. (1973). Task behavioral reduction across multiple stimulus
analysis in curriculum design: A hierarchically parameters. Journal of Applied Behavior Analysis, 27,
sequenced introductory mathematics curriculum. 371–384. doi:10.1901/jaba.1994.27-371
Journal of Applied Behavior Analysis, 6, 679–709. Sidman, M., & Stoddard, L. T. (1967). The effective-
doi:10.1901/jaba.1973.6-679 ness of fading in programming a simultaneous
102
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
form discrimination for retarded children. Journal intensity with an awareness enhancement device
of the Experimental Analysis of Behavior, 10, 3–15. in the treatment of finger sucking. Education and
doi:10.1901/jeab.1967.10-3 Treatment of Children, 26, 22–29.
Sidman, M., & Willson-Morris, M. (1974). Testing for Sumpter, C., Temple, W., & Foster, T. M. (1998).
reading comprehension: A brief report on stimulus Response form, force, and number: Effects on
control. Journal of Applied Behavior Analysis, 7, concurrent-schedule performance. Journal of the
327–332. doi:10.1901/jaba.1974.7-327 Experimental Analysis of Behavior, 70, 45–68.
Singh, N. N., Watson, J. E., & Winton, A. S. (1986). doi:10.1901/jeab.1998.70-45
Treating self-injury: Water mist spray versus facial Sundby, S. M., Dickinson, A., & Michael, J. (1996).
screening or forced arm exercise. Journal of Applied Evaluation of a computer simulation to assess sub-
Behavior Analysis, 19, 403–410. doi:10.1901/jaba. ject preference for different types of incentive pay.
1986.19-403 Journal of Organizational Behavior Management, 16,
Skiba, E. A., Pettigrew, L. E., & Alden, S. E. (1971). A 45–67. doi:10.1300/J075v16n01_04
behavioral approach to the control of thumbsucking Thomas, D. R., Becker, W. C., & Armstrong, M. (1968).
in the classroom. Journal of Applied Behavior Analysis, Production and elimination of disruptive classroom
4, 121–125. doi:10.1901/jaba.1971.4-121 behavior by systematically varying teacher’s behav-
Skinner, B. F. (1966). What is the experimental analysis ior. Journal of Applied Behavior Analysis, 1, 35–45.
of behavior? Journal of the Experimental Analysis of doi:10.1901/jaba.1968.1-35
Thompson, R. H., Iwata, B. A., Conners, J., & Roscoe,
Speltz, M. L., Shimamura, J. W., & McReynolds, W. E. M. (1999). Effects of reinforcement for alternative
T. (1982). Procedural variations in group contin- behavior during punishment of self-injury. Journal of
gencies: Effects on children’s academic and social Applied Behavior Analysis, 32, 317–328. doi:10.1901/
behaviors. Journal of Applied Behavior Analysis, 15, jaba.1999.32-317
533–544. doi:10.1901/jaba.1982.15-533
Thompson, R. H., McKerchar, P. M., & Dancho, K. A.
Steege, M. W., Wacker, D. P., Cigrand, K. C., Berg, W. K., (2004). The effects of delayed physical prompts and
Novak, C. G., Reimers, T. M., & DeRaad, A. (1990). reinforcement on infant sign language acquisition.
Use of negative reinforcement in the treatment of Journal of Applied Behavior Analysis, 37, 379–383.
self-injurious behavior. Journal of Applied Behavior doi:10.1901/jaba.2004.37-379
Analysis, 23, 459–467. doi:10.1901/jaba.1990.
23-459 Thompson, T. J., Braam, S. J., & Fuqua, R. W. (1982).
Training and generalization of laundry skills: A mul-
Stokes, J. V., Luiselli, J. K., & Reed, D. D. (2010). A tiple probe evaluation with handicapped persons.
behavioral intervention for teaching tackling skills Journal of Applied Behavior Analysis, 15, 177–182.
to high school football athletes. Journal of Applied doi:10.1901/jaba.1982.15-177
2010.43-509 Tiger, J. H., Hanley, G. P., & Hernandez, E. (2006). A
further evaluation of the reinforcing value of choice.
Stokes, J. V., Luiselli, J. K., Reed, D. D., & Fleming, R. K.
(2010). Behavioral coaching to improve offensive
doi:10.1901/jaba.2006.158-04
line pass-blocking skills of high school football
athletes. Journal of Applied Behavior Analysis, 43, Toole, L. M., Bowman, L. G., Thomason, J. L., Hagopian,
463–472. doi:10.1901/jaba.2010.43-463 L. P., & Rush, K. S. (2003). Observed increases in
Stokes, T. F., & Baer, D. M. (1977). An implicit technol- positive affect during behavioral treatment. Behavioral
ogy of generalization. Journal of Applied Behavior Interventions, 18, 35–42. doi:10.1002/bin.124
Analysis, 10, 349–367. doi:10.1901/jaba.1977.10-349 Van Houten, R. (1979). Social validation: The evolution
Stokes, T. F., Baer, D. M., & Jackson, R. L. (1974). of standards of competency for target behaviors.
Programming the generalization of a greeting Journal of Applied Behavior Analysis, 12, 581–591.
response in four retarded children. Journal of doi:10.1901/jaba.1979.12-581
Applied Behavior Analysis, 7, 599–610. doi:10.1901/ Van Houten, R., Nau, P., & Marini, Z. (1980). An analy-
jaba.1974.7-599 sis of public posting in reducing speeding behavior
St. Peter Pipkin, C., Vollmer, T. R., & Sloman, K. N. on an urban highway. Journal of Applied Behavior
(2010). Effects of treatment integrity failures during Analysis, 13, 383–395. doi:10.1901/jaba.1980.13-383
differential reinforcement of alternative behavior: Vladescu, J. C., & Kodak, T. (2010). A review of recent
A translational model. Journal of Applied Behavior studies on differential reinforcement during skill
Analysis, 43, 47–70. doi:10.1901/jaba.2010.43-47 acquisition in early intervention. Journal of Applied
Stricker, J. M., Miltenberger, R. G., Garlinghouse, M., & Behavior Analysis, 43, 351–355. doi:10.1901/
Tulloch, H. E. (2003). Augmenting stimulus jaba.2010.43-351
103
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Vollmer, T. R., & Hackenberg, T. D. (2001). Reinforce Wolery, M., & Gast, D. L. (1984). Effective and efficient
ment contingencies and social reinforcement: Some procedures for the transfer of stimulus control.
reciprocal relations between basic and applied Topics in Early Childhood Special Education, 4, 52–77.
research. Journal of Applied Behavior Analysis, 34, doi:10.1177/027112148400400305
241–253. doi:10.1901/jaba.2001.34-241
Wolery, M., Holcombe, A., Cybriwsky, C., Doyle, P. M.,
Vollmer, T. R., & Iwata, B. A. (1991). Establishing Schuster, J. W., Ault, M. J., & Gast, D. L. (1992).
operations and reinforcement effects. Journal of Constant time delay with discrete responses: A
Applied Behavior Analysis, 24, 279–291. doi:10.1901/ review of effectiveness and demographic, proce-
jaba.1991.24-279 dural, and methodological parameters. Research in
Vollmer, T. R., Iwata, B. A., Zarcone, J. R., Smith, R. G., Developmental Disabilities, 13, 239–266. doi:10.1016/
& Mazaleski, J. L. (1993). The role of attention in 0891-4222(92)90028-5
the treatment of attention-maintained self-injurious Wolf, M. M. (1978). Social validity: The case for subjec-
behavior: Noncontingent reinforcement and dif- tive measurement or how applied behavior analysis is
ferential reinforcement of other behavior. Journal finding its heart. Journal of Applied Behavior Analysis,
of Applied Behavior Analysis, 26, 9–21. doi:10.1901/ 11, 203–214. doi:10.1901/jaba.1978.11-203
jaba.1993.26-9
Wolf, M. M., Birnbrauer, J. S., Williams, T., & Lawler, J.
Vollmer, T. R., Progar, P. R., Lalli, J. S., Van Camp, C. (1965). A note on apparent extinction of the vomit-
M., Sierp, B. J., Wright, C. S., & Eisenschink, K. J. ing behavior of a retarded child. In L. P. Ullmann &
(1998). Fixed-time schedules attenuate extinction- L. Krasner (Eds.), Case studies in behavior modifica-
induced phenomena in the treatment of severe aber- tion (pp. 364–366). New York, NY: Holt, Rinehart, &
rant behavior. Journal of Applied Behavior Analysis, Winston.
31, 529–542. doi:10.1901/jaba.1998.31-529
Wolf, M. M., Risley, T. R., & Mees, H. (1963).
Vollmer, T. R., Roane, H. S., Ringdahl, J. E., & Marcus, Application of operant conditioning procedures to
B. A. (1999). Evaluating treatment challenges with the behavior problems of an autistic child. Behaviour
differential reinforcement of alternative behav- Research and Therapy, 1, 305–312. doi:10.1016/0005-
ior. Journal of Applied Behavior Analysis, 32, 9–23. 7967(63)90045-7
doi:10.1901/jaba.1999.32-9
Worsdell, A. S., Iwata, B. A., Dozier, C. L., Johnson, A. D.,
Wacker, D. P., & Berg, W. K. (1983). Effects of picture Neidert, P. L., & Thomason, J. L. (2005). Analysis
prompts on the acquisition of complex vocational of response repetition as an error-correction strategy
tasks by mentally retarded individuals. Journal of during sight-word reading. Journal of Applied Behavior
Applied Behavior Analysis, 16, 417–433. doi:10.1901/ Analysis, 38, 511–527. doi:10.1901/jaba.2005.115-04
jaba.1983.16-417
Yeaton, W. H., & Bailey, J. S. (1983). Utilization analysis
Watson, P. J., & Workman, E. A. (1981). The noncon- of a pedestrian safety training program. Journal of
current multiple baseline across individuals design: Applied Behavior Analysis, 16, 203–216. doi:10.1901/
An extension of the traditional multiple baseline jaba.1983.16-203
design. Journal of Behavior Therapy and Experimental
Young, J. M., Krantz, P. J., McClannahan, L. E., &
Psychiatry, 12, 257–259.
Poulson, C. L. (1994). Generalized imitation and
White, G. D., Nielsen, G., & Johnson, S. M. (1972). response-class formation in children with autism.
Timeout duration and the suppression of deviant Journal of Applied Behavior Analysis, 27, 685–697.
behavior in children. Journal of Applied Behavior doi:10.1901/jaba.1994.27-685
Zarcone, J. R., Iwata, B. A., Mazaleski, J. L., & Smith,
Wildman, B. G., Erickson, M. T., & Kent, R. N. (1975). R. G. (1994). Momentum and extinction effects on
The effect of two training procedures on observer self-injurious escape behavior and noncompliance.
agreement and variability of behavior ratings. Child Journal of Applied Behavior Analysis, 27, 649–658.
Development, 46, 520–524. doi:10.1901/jaba.1994.27-649
Williams, G. E., & Cuvo, A. J. (1986). Training apart- Zarcone, J. R., Iwata, B. A., Smith, R. G., Mazaleski, J. L.,
ment upkeep skills to rehabilitation clients: A & Lerman, D. C. (1994). Reemergence and extinc-
comparison of task analytic strategies. Journal of tion of self-injurious escape behavior during stimu-
Applied Behavior Analysis, 19, 39–51. doi:10.1901/ lus (instructional) fading. Journal of Applied Behavior
jaba.1986.19-39 Analysis, 27, 307–316. doi:10.1901/jaba.1994.27-307
104
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 5
Single-Case Experimental
Designs
Michael Perone and Daniel E. Hursh
Single-case experimental designs are characterized “participants” were plants in fields split into plots.
by repeated measurements of an individual’s behav- The statistical methods were developed to assess the
ior, comparisons across experimental conditions significance of differences in yields of plots of plants
imposed on that individual, and assessment of the treated differently. R. A. Fisher’s (1925) Statistical
measurements’ reliability within and across the con- Methods for Research Workers set the course for the
ditions. Such designs were integral to the develop- field. Fisher began development of his methods
ment of behavioral science. Early work in the field while employed as the statistician at an agricultural
of psychology depended on the analysis of the expe- experiment station early in his career. The fact that
riences of one or a few individuals (Ebbinghaus, data on large numbers of participants tend to be
1885/1913; Thorndike, 1911; Wertheimer, 1912). normally distributed (regardless of whether the par-
The investigator identified a phenomenon (e.g., ticipants are plants, people, or other animals) led to
learning and memory, the law of effect, the phi phe- the easy adaptation of group statistical methods to
nomenon) and pursued experimental arrangements research with humans. The standard practice came
that assessed its reliability and the functional rela- to emphasize the importance of group means, differ-
tions among the pertinent variables (e.g., the rela- ences in these means, and the use of statistical tests
tion between the length of a series of nonsense to draw inferences about the likelihood that the
syllables and learning curves, recall, and retention; group differences were representative of differences
the relation between the consequences of behavior in the populations of interest (e.g., Kazdin, 1999;
and the rate of the behavior; the relation between an Perone, 1999).
observer’s distance from blinking lights and appear- Despite the rise of group statistical methods,
ance of movement). Because the research was con- single-case designs continued to be used in some
ducted on the investigators themselves (e.g., the important work because they allowed the investiga-
memory work of Ebbinghaus) or on just a few par- tor to study the details of relations among variables
ticipants (e.g., Thorndike’s cats and Wertheimer’s as expressed in the behavior of individuals (e.g.,
human observers), the experimental arrangements Bijou, 1955; Skinner, 1938; Watson, 1913), which
often involved intensive study, with numerous mea- resulted in reasonably clear demonstrations of func-
surements of behavior recorded while each individ- tional relations among the variables being studied
ual was studied under a variety of conditions. (e.g., conditioned startle responses, reinforcement,
Only after the development of statistical methods and schedules of reinforcement). Articulation of the
for analyzing aggregate data did the focus shift to necessary elements of single-case designs, notably
comparisons across groups of participants, with in Sidman’s (1960) seminal Tactics of Scientific
each group exposed to a single condition (see also Research, helped make the designs practically de
Chapter 8, this volume). In the original case, the rigueur in basic research on free-operant behavior
DOI: 10.1037/13937-005
107
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
(Baron & Perone, 1998; Johnston & Pennypacker, variables of interest. When the results of an experi-
2009; Perone, 1991). Translation of basic laboratory ment provide clear evidence that manipulation of
research for application in everyday situations the independent variable caused the changes mea-
resulted in the further development of how single- sured in the dependent variable, the experiment is
case research designs were to serve applied research- said to have internal validity. Investigators are also
ers (Baer, Wolf, & Risley, 1968, 1987; Bailey & concerned with other kinds of validity. Kazdin
Burch, 2002; Barlow, Nock, & Hersen, 2009; Mor- (1999) and Shadish, Cook, and Campbell (2002)
gan & Morgan, 2009; see Chapter 8, this volume). listed construct validity, statistical conclusion valid-
In this chapter, we describe and provide exam- ity, and external validity. Of these, external validity,
ples of the various design elements that constitute which is concerned with the generality of experi-
single-case methods. We begin by considering the mental outcomes across populations, settings, times,
fundamental requirement of any experiment— or variables, seems to draw the lion’s share of atten-
internal validity—and the kinds of obstacles to tion from methodologists. This critically important
internal validity that are most likely to be encoun- issue is addressed by Branch and Pennypacker in
tered in single-case experiments. Next, we describe this volume’s Chapter 7. Here we need only say that
a variety of designs, ranging in complexity, that are from the standpoint of experimental design, internal
commonly associated with the single-case approach. validity takes precedence because it is prerequisite
Included are designs to study irreversible or revers- to external validity. Unless an investigator can
ible changes in behavior, experimental conditions describe the functional relation between the inde-
arranged successively or simultaneously, and the pendent and dependent variables with confidence,
effects of one or more independent variables. In worrying about the generality of the relation would
each instance, we evaluate the degree to which the be premature. As Campbell and Stanley (1963) put
design can overcome obstacles to internal validity. it, “Internal validity is the basic minimum without
Some designs, for practical or ethical reasons, which an experiment is uninterpretable” (p. 5).
exclude important controls and thus compromise (For a thoughtful discussion of the interplay
internal validity, but most single-case designs are between internal and external validity, see Kazdin,
robust in promoting internal validity. A great 1999, pp. 35–38, and for a more general discussion
strength of the single-case approach is its flexibility, considering all four types of validity, see Shadish
and we describe how single-case designs can be et al., 2002, pp. 93–102.)
adjusted dynamically, over the course of an experi- Experimental designs are judged largely in terms of
ment, in response to the ongoing pattern of results. how well they promote internal validity. It may be
We go on to review the commitment of single-case helpful to think of a completed experiment as a kind of
investigators to identifying and taking command of argument in which the design and results lead to a con-
the variables that control behavior. This commit- clusion about causality. Internal validity has to do with
ment is expressed in the steady-state strategy that the persuasiveness of the argument. Consumers of the
underlies most contemporary single-case research. research—journal reviewers and editors initially—will
Finally, we describe how interparticipant replication, differ in their susceptibility to the argument, which is
a seeming departure from a single-case approach, is why editors weigh the judgments of several reviewers
needed to assess the degree to which an investigator to render a verdict on the validity of an experiment and
has succeeded in identifying and controlling relevant whether a report of it merits publication.
variables (see also Chapter 7, this volume). It may also be helpful to remember, as you read
this or any other chapter about experimental design,
that good design can only foster internal validity; it
Internal Validity of Single-Case
cannot guarantee it. Internal validity is determined
Experiments
not only by the experimental design but also by the
The essential goal of an experiment is to make experimental outcomes. Consider, for example, a
valid decisions about causal relations between the simple experiment to evaluate a treatment to reduce
108
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Single-Case Experimental Designs
smoking. The investigator begins by taking a few may be developmental in character; for example,
weeks to measure the baseline rate of smoking (e.g., with age, changes in cognitive and social develop-
in cigarettes per day). Suppose the treatment is ment could affect the efficacy of cartoons as rein-
applied, and after a few weeks smoking ceases alto- forcers. Maturational variables may also involve
gether. Finally, the treatment is withdrawn, that is, shorter term processes such as fatigue, boredom,
the investigator reinstates the baseline conditions. and hunger, and investigators should be aware of
What happens next is critical to an evaluation of the these processes even in highly controlled laboratory
experiment’s internal validity. If smoking recovers, experiments. Working in the animal laboratory,
returning to levels near those observed during the McSweeney and her colleagues (e.g., McSweeney &
initial baseline, the investigator can make a strong Roll, 1993) showed that even when the procedure is
inference about the reductive effect of the treatment held constant, response rates may change systemati-
on smoking. If smoking fails to recover, however, cally over the course of a session. There has been
the causal status of the treatment is ambiguous. It some disagreement about the responsible process
might have been the cause of a permanent reduction (the primary contenders are satiation and habitua-
in smoking, but the evidence is open to alternative tion; see McSweeney & Murphy, 2000), but from
accounts. It is possible that some other variable, the standpoint of experimental design this disagree-
operating over the course of time, is responsible for ment does not matter. What does matter is that any
the absence of smoking. Fortunately, there are ways design that compares treatment conditions arranged
to resolve the ambiguity; they are discussed later in early and late in a session may confound the condi-
the Designs for Irreversible Effects section. The gen- tions with a maturational process.
eral point remains: A final decision about internal Testing is a concern when repeated exposure to a
validity must wait until the data have been collected measurement procedure may, in itself, affect behav-
and analyzed and conclusions about the effect of the ior. Investigators who rely on verbal measures may
experimental treatment have been made. be especially concerned. It is obvious that asking a
Internal validity is fostered by designs that elimi- participant the same questions over and over could
nate or reduce the influence of extraneous variables lead to stereotyped answers, thus blocking the test’s
that could compete with the independent variable sensitivity to changes in experimental treatments. It
for control of the dependent variable. The investiga- may be less obvious that purely operant procedures
tor’s design objective is to eliminate such variables— are also susceptible to the testing threat. For exam-
famously labeled by Campbell and Stanley (1963) as ple, as rats gain experience with fixed-ratio sched-
threats to internal validity—or, if that is not possible, ules, they tend to acquire increasingly efficient
to equalize their effects across experimental condi- response topographies. Over a series of sessions
tions so that they are not confounded with the inde- with a fixed-ratio schedule, these changes in
pendent variable. Because single-case experiments responding will be confounded with the effects of
compare conditions imposed on an individual, the experimental conditions.
investigators must guard against threats that operate Instrumentation is a threat when systematic
as a function of time or repeated exposure to experi- changes or drift in a measuring device may contami-
mental treatments: history, maturation, testing, and nate the data collected over the course of a study. An
instrumentation. investigator may neglect to periodically recalibrate
History, in this context, generally refers to the the force required to activate an operandum, for
influence of factors outside the laboratory. For example, or the sensitivity of a computer touch
example, an increase in the tobacco tax during a screen may be reduced by repeated use. The instru-
smoking cessation study could contribute to a mentation threat is most likely an issue in research
smoker’s success in giving up the habit and inflate that relies on human observers to collect or code
the apparent effect of the experimental treatment. data (Chapter 6, this volume). Prudent investigators
Maturation refers to processes occurring within will carefully consider both the methods used to
the research participant. As the name implies, they train their human observers and those aspects of
109
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
their experimental protocol that may influence the Considered in the abstract, the intervention-only
consistency of the observers’ work. design would appear to be unacceptably weak in its
These four time- and experience-related threats defense against threats to internal validity. Consider
to internal validity can be addressed successfully in the idealized pattern of results in Figure 5.1. The
single-case designs by way of replication. Through- increase in behavior could be the result of the inter-
out an experiment, behavior is measured repeatedly vention, but it is also easy to imagine how it might
so that the effect of the experimental manipulation result from, say, historical or maturational factors.
can be assessed on a nearly continuous basis. Details about the procedure and the independent
Kazdin (1982) emphasized the importance of and dependent variables might lead to a more posi-
repeated measurement by calling it the fundamen- tive evaluation of the study’s validity. Suppose, for
tal requirement of single-case designs (p. 104). If example, the behavior represented in Figure 5.1 is
the behavioral measures show (a) minimal varia- correct operations of a factory machine and the
tion in value across time within each experimental intervention is some procedure for training the cor-
condition, (b) systematic differences across condi- rect operation. Suppose also that the machine is
tions, and (c) comparable values when conditions unique in both form and operation—nothing similar
are replicated, then the experimental manipulation is available outside the factory training environ-
is the most plausible causal factor. With such a pat- ment. Under these restricted circumstances, attrib-
tern of results, the influence of extraneous factors uting the improved performance to the training is
categorized as history, maturation, testing, or plausible. Still, one must admit that the conclusion
instrumentation would appear to be either elimi- is limited, and the restricted circumstances needed
nated or held constant. to support it might be rare.
The baseline intervention or A-B design improves
on the intervention-only design by adding a true
Designs
baseline phase. In the idealized results shown in
Next, we turn to some illustrative designs and con- Figure 5.2, a stable behavioral baseline is followed
sider the degree to which they are likely to be suc- by a conspicuous change that coincides with the
cessful in addressing threats to internal validity. intervention. The time course of behavioral change
in the intervention phase is similar to that shown in
Designs Without Replicated Conditions Figure 5.1 for the intervention-only design. The evi-
Two simple designs that omit replication of experi- dence of an intervention effect is strengthened in
mental conditions have appeared in the literature. the A-B design because the intervention results are
These designs do, however, involve repeated mea-
surement within a condition, allowing investigators
to rely on patterns in the results over time to assess
the possible impact of an intervention.
The intervention-only design (Moxley, 1998) is
most useful in situations in which it is unethical to
take the time to collect baseline data (as with dan-
gerous or illegal behavior) or it is not feasible (as
in instructional situations in which the yet-to-be-
taught behavior is absent from the participant’s rep-
ertoire). The data collected early in the process of
intervening serves as a kind of baseline for changes
that occur as the intervention proceeds. Changes Figure 5.1. Idealized results in an intervention-only
design. The increase in behavior over the initial val-
that are systematic, such as accelerations, decelera- ues, consistent with the goal or expected effect of the
tions, or changes in variability, are taken as evidence intervention, is taken as evidence that the intervention
of the intervention’s effectiveness. caused the increase.
110
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
attributed to the treatment. Even a long and stable

baseline is no guarantee of internal validity: To the
extent that behavioral change is delayed from the
onset of the intervention, alternative explanations
may become increasingly plausible.
In recognition of these sometimes insurmount-
able limitations of designs without replications, a
considerable range of designs has evolved that
includes replications.
Designs With Successive Conditions

A straightforward extension of the A-B design yields
Figure 5.2. Idealized results in a baseline– a major improvement in promoting internal validity:
intervention or A-B design. A stable behavioral baseline Simply reinstate the baseline condition after the
is followed by a conspicuous change coincident with the intervention—an A-B-A design. A common variation
intervention, suggesting that the intervention caused
the change. is the B-A-B design, in which the intervention is
imposed in the first phase, withdrawn in the second,
preceded by a lengthy series of measurements in and reinstated in the third. In either case, the under-
which change is absent. The causal inference—that lying logic is the same, and in both the ideal out-
the intervention is responsible for the behavioral come is for behavior to change in some systematic
change—is supported by the fact that the behavior way from the first phase to the second and then
changed only when the intervention was imple- return to initial values when the original condition
mented. More generally, an immediate change in is reinstated.
level, trend, or variability coincident with the begin- In the A-B-A-B design, the replication of the base-
ning of the intervention is taken as evidence of a line is followed by a replication of the intervention.
possible functional relation between the interven- If a change occurs in the data patterns that replicates
tion and the dependent variable. or approximates those of the first intervention
Although the A-B design is an improvement over phase, the plausibility of history, maturation, test-
the intervention-only design, it remains susceptible ing, or instrumentation effects is reduced even fur-
to history, maturation, and testing effects (and per- ther, and a compelling case can be made for the
haps also to instrumentation effects). The plausibil- intervention’s effectiveness. Put simply, the likeli-
ity of these threats to internal validity is exacerbated hood of other events being responsible for behav-
when the experimental outcomes fall short of the ioral changes is greatly reduced if the changes occur
ideal, as is often the case, especially in applied when and only when the conditions are changed.
research in which field settings may compromise The A-B-A-B design contains an initial demonstra-
experimental control of extraneous variables and tion of an effect (the first A to B change), shows that
ethical or clinical concerns may prevent the collec- the effect is likely the result of the intervention (the
tion of extended baseline data. The shorter the base- B to A change), and convinces one of that by repli-
line is, the more the A-B design comes to resemble cating the effect (the second A to B change). Figure
the intervention-only design. If the baseline mea- 5.3 illustrates a possible outcome. The hypothetical
surements are characterized by significant variabil- data in this particular example fall short of the ideal:
ity, it may be difficult to claim that any change in The initial baseline is brief, and behavior is still
behavior is clearly coincident with the treatment, changing when each of the subsequent three condi-
which is especially the case if the baseline variability tions is terminated. With such an outcome, the
is systematic. For example, if an upward trend is experiment leaves unanswered the ultimate effect of
apparent in the baseline, continuation of the trend the experimental treatment. Nevertheless, the sys-
in the intervention phase cannot with confidence be tematic changes in trend that coincide repeatedly
111
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
may want to identify the most effective way to get a

client to talk more rationally about his or her fears.
One approach may be to debunk any irrational talk;
another may be to suggest alternative rational ways
to talk about fears. The therapist can simply alter-
nate these approaches within or across sessions and
observe which approach produces more rational
talk. A teacher who wants to know whether the lat-
est approach to spelling is effective may use that
approach on some days and the old approach on
other days while assessing the students’ spelling per-
formance throughout to decide whether the latest
Figure 5.3. Hypothetical results in an A-B-A-B approach is better. This design tactic requires that
design. The experimental treatments in the two B the outcomes being assessed are likely to be sensi-
phases consistently reduce behavior, and reinstate- tive to such frequent changes and that the experi-
ment of the baseline procedure in the second A phase
increases behavior. The reversibility of the behavior ence of one intervention has only minimal impact
change in this pattern of results supports causal infer- on the effectiveness of the alternatives.
ences about the experimental treatment. Such designs are called multielement designs
(Sidman, 1960; Ulman & Sulzer-Azaroff, 1975). In
with the initiation of the intervention (B) and base- one variation on this tactic, the alternating-treatments
line (A) phases leave no doubt about the causal role design, two or more treatments are alternated rapidly
of the experimental treatment. It would be highly (Barlow et al., 2009). The operational definition of
implausible to claim that something other than the rapid depends on the experimental context and
treatment was responsible for reducing the behavior. could involve individual conditions lasting from
Many research questions call for a comparison minutes to days. For example, a therapist may,
across two or more interventions. Several design within a single session, switch back and forth from
options are available. One may use an A-B-A (or debunking irrational talk to suggesting alternative
A-B-A-B) design in which both the A and B phases rational ways to talk about a client’s fears, or a
involve an experimental treatment. If a conventional teacher may spend a week on the old approach to
baseline is desired, one may use an A-B-A-C-A design spelling before switching to the latest approach.
or perhaps an A-B-C-B design (in which A designates Figure 5.4 shows a common way to present
the conventional baseline and B and C designate the results from experiments with an alternating-
distinct interventions). In all of these designs, each treatments design. Results from the experimental
condition is imposed for a series of observations so treatments are represented by different symbols; the
that the effect of each treatment is given sufficient separation of the two functions documents the dif-
time to become evident (as in Figure 5.3). In basic ference in the treatments’ effectiveness, and more
laboratory research with rats or pigeons, it is not important, the reproducibility of the difference
unusual for a condition to be imposed for weeks of across time attests to the reliability of the effect.
daily sessions until behavior stabilizes and the When inspecting a graph such as that in Figure 5.4,
behavioral effect is replicated from one observation it is important to remember that the design involves
to the next (this topic is discussed in the Steady- a special kind of reversal, in that the behavior is ris-
State Strategy section). ing and falling across successive presentations of the
two treatments. The highly reliable character of the
Designs With Simultaneous Conditions effects of the two treatments is obscured by the
Another tactic for comparing interventions involves graphing convention: Results from like conditions
changing the conditions frequently to assess their are connected, even though the data points do not
relative impacts quickly. For example, a therapist represent successive observations.
112
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
rates can be maximized only if the participant occa-

sionally samples all of the schedules (Stubbs &
Pliskoff, 1969).
We have discussed multielement designs in the
context of comparisons across experimental treat-
ments, a design tactic that Sidman (1960) called
multielement manipulations. Multielement designs
can also be used to measure an experimental treat-
ment’s effect on two or more different response
classes or operants, a design tactic that Sidman
called multielement baselines. The idea is to arrange
the experimental circumstances to generate two or
Figure 5.4. Conventional presentation of results more behavioral baselines more or less simultane-
in an alternating-treatments design. The lines do not ously, which can be accomplished by arranging a
connect the data points in temporal sequence; rather,
they connect data points collected under like treatment multiple schedule or concurrent schedules. Once
conditions. stable baselines have been established, an experi-
mental treatment is applied to both. For example, a
In another multielement design, the experimen- multiple schedule might be arranged with contin-
tal treatments are arranged concurrently, with the gencies to engender high rates of behavior in one
participant choosing which to access (sometimes component and low rates in the other. In one or
called simultaneous-availability-of-all-conditions more experimental conditions, a drug may be
design [Browning, 1967] or, more commonly in the administered to discover whether the effect of the
basic literature, simply a concurrent schedule). In drug depends on the baseline rate (e.g., Lucki &
many cases, the goal is to assess preferences. For DeLong, 1983).
example, a therapist may ask the client which tactic Multielement designs have a major strength as
he or she wants the therapist to use during the well as a significant limitation. Their strength is in
session or the teacher may ask students which promoting internal validity. Because multielement
approach to spelling they want to use that day. A designs allow experimental treatments to be com-
problem arises, however, if the participant’s choices pared almost simultaneously (i.e., within a single
are unconstrained: One treatment may be chosen to session or pair of sessions), the influence of the
the exclusion of the other. Such an outcome may time-related threats of history, maturation, testing,
represent a strong preference, but it could also rep- and instrumentation is equalized across the condi-
resent happenstance, as when a participant selects a tions. Their limitation is that the temporal juxtapo-
particular option at the outset of an experiment and sition of the two conditions may generate different
simply sticks with it. Without adequate exposure to effects than the conditions might generate if
all of the options, it would be inappropriate to draw arranged in isolation from one another—or, put
conclusions about preference or, indeed, even to another way, the treatments may interact. The use of
consider the procedure as arranging a meaningful signals to demarcate the treatments and foster dis-
choice. Procedures have been developed to address crimination between them, as in the concurrent
this problem and ensure that the participant is regu- schedule variant, is sometimes intended to reduce
larly exposed to the available treatment conditions. the interaction. Another step is to separate the treat-
Some investigators devote portions of the experiments in time; if the treatments are not temporally
ment to forced-choice procedures that momentarily contiguous, the effect of one treatment is less likely
constrain the participant’s options to a single treat- to carry over to the next. In basic laboratory experi-
ment (e.g., Mazur, 1985). When the concurrent ments, this separation is effected by interposing time
assessment involves schedules of reinforcement, the outs between the components of a multiple sched-
schedules can be arranged so that reinforcement ule. In field experiments, the separation may arise in
113
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
the customary scheme of things—for example, when Another problem arises in cases in which a partici-
treatments are alternated across school days or pant’s experience with one treatment has an impact
across weekly therapy sessions. There is no guaran- on the effects produced by another treatment (e.g.,
tee, however, that these steps actually do prevent being taught decoding skills can result in more rapid
interacting treatments. The only sure way to allay sight word learning). If the two treatments were
this concern is to conduct additional research in compared in an alternating-treatments design, their
which each treatment is studied in isolation. effects would be obscured. The last problem is ethi-
cal rather than logistical: If the treatment effect is
Designs for Irreversible Effects beneficial (e.g., reduction in self-injurious behav-
So far, we have considered two general classes of ior), it would be undesirable to withdraw it and
experimental designs. The first consists of the return behavior to pretreatment values even if the
intervention-only and baseline–intervention (A-B) withdrawal might decisively demonstrate the treat-
designs. Although these designs may be justifiable ment’s efficacy.
under special circumstances, they are undesirable
because, in general, they provide little protection Multiple-baseline designs. One way to avoid the
against time- and experience-related threats to practical, ethical, and confounding problems of
internal validity. The second class of experimental withdrawing, reversing, or alternating treatments is
designs promotes internal validity through replica- to arrange for the replication of a treatment’s impact
tion of experimental conditions. The difference to occur across participants, behaviors, or settings.
between the two classes can be summarized this These multiple-baseline designs (Baer et al., 1968)
way: In an experiment with an A-B design, any were developed just for such situations. Data are
change observed from A to B might be related to collected under two or more independent baseline
the experimental treatment, but—depending on conditions. The baselines often come from more
the particulars of the experiment—the change than one participant, but they may also come from
might reflect the operation of maturation, history, the same participant engaged in different behaviors
testing, or instrumentation. Adding replications or from the same participant behaving in different
(e.g., in A-B-A, A-B-A-B, or multielement designs) settings. Once the baseline behavior is shown to be
tests these alternative explanations. Each replica- stable, the experimental treatment is implemented
tion, if accompanied by appropriate changes in in time-staggered fashion to one baseline (i.e., one
behavior, makes it less plausible that something participant, behavior, or setting) at a time. Adding
other than the experimental treatment could have the treatment to a second baseline is only done
caused the changes. once the impact of the treatment for the first base-
To promote internal validity, the designs in the line has become obvious. Thus, the baseline data
second class require that the participant experience for untreated participants, responses, or settings
another treatment or a return to baseline. Replicat- serves as a control for confounding variables. That
ing conditions is not always possible or desirable, is, if changes are observed when and only when
however, for several reasons. First, some treatment the treatment is applied to each of the participants,
effects are not likely to disappear simply because the responses, or settings, it is unlikely that other vari-
treatment has been discontinued (e.g., the learning ables can account for the changes. An idealized pat-
of a math fact, reading skill, or social skill that tern of results is shown in Figure 5.5.
allows the learner access to desirable items or activi- Some examples may help illustrate the three
ties). The use of an A-B-A design to assess such an common variants of the multiple-baseline design. If
irreversible outcome will yield ambiguous results: a teacher has experience suggesting that peer tutor-
When the baseline condition is replicated, the ing may help some of her students who struggle
behavior remains unchanged. It is not possible to with solving equations, that teacher may assign a
say whether the outcome is the persistent effect of peer tutor to one struggling student at a time to
the treatment or the effect of some other factor. observe whether each of the struggling students’
114
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
concurrently across more than one participant, class

of behavior, or setting. When such frequent mea-
surement is not feasible, multiple-probe designs
(Horner & Baer, 1978) are available. These designs
differ from multiple-baseline designs in that instead
of frequent measurements, only occasional probe
measurements are taken. That is, the teacher, par-
ent, or mental health worker mentioned in the
examples arranges to measure the outcomes less
often. He or she may assess the outcomes only
weekly rather than daily, even though the experi-
mental conditions (baseline or treatment) would be
implemented continuously.
Changing-criterion designs. What if the research

problem is restricted to just a single baseline—only
one participant, one class of behavior, or one
Figure 5.5. A multiple-baseline design with an setting—and it is not practical or ethical to withdraw
experimental treatment imposed in staggered tempo- or reverse treatment? We have already described two
ral fashion across three independent baselines. The ways to deal with such a situation: the intervention-
baselines could represent the behavior of different par-
ticipants, the behavior of one participant in different only design and the A-B design. We have also noted
settings, or the different behaviors of one participant. the weaknesses of these designs as regards internal
The strict coincidence between the imposition of the validity. A third option, the changing-criterion design
treatment and the appearance of behavior change allows (Hartmann & Hall, 1976), offers better protec-
the change to be attributed to the treatment.
tion against threats to internal validity. This design
equation-solving performance improves when they is well suited to the study of variables that can be
begin to work with their peer tutor and not before implemented progressively. For example, a teacher
(multiple-baseline design across participants). If a par- may use token reinforcers to help a student develop
ent has heard from other parents that developing a fluency in solving math problems. After measur-
behavior contract can be a successful means of get- ing the student’s baseline rate of problem solving,
ting their child to do their chores, that parent may the teacher may offer a token if the student’s rate is
create an initial contract that includes only one increased by, say, 10%. Each time the student’s rate
chore and then add chores to the contract one at a of problem solving stabilizes at the new criterion for
time as he or she observes that the child’s comple- reinforcement, the criterion is raised. If, as illustrated
tion of each chore becomes reliable only after it is in Figure 5.6, the student’s performance repeatedly
added to the contract (multiple-baseline design across conforms to the succession of increasingly stringent
behaviors). If a mental health worker serves a client criteria, it is possible to attribute the changes in per-
who has difficulty purchasing items, that mental formance to the changing experimental treatment.
health worker may provide modeling, guidance, and As this example implies, changing-criterion designs
reinforcement for the client’s purchasing skills at a are especially useful when the goal is to assess treat-
neighborhood convenience store, then provide the ments designed to shape skilled performances or
same treatment at the supermarket, and if successful engender novel behavior (Morgan & Morgan, 2009).
there provide the same treatment at the department
store across town (multiple-baseline design across Additional Design Options
settings). Two additional classes of single-case designs are
All of these multiple-baseline designs require the commonly used, especially in the basic experimental
feasibility of taking frequent measures more or less analysis of behavior.
115
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
strategies are commonly used. First, one or more

conditions are replicated to separate the effects of
the treatment from the effects associated with tim-
ing. For example, one could replace a deficient
A-B-C-D-E design with an A-B-C-D-E-A design or
perhaps an A-B-C-D-E-A-C design. If the rising rela-
tive response rates result from some time-related or
experiential factor, the rates should continue to rise
in the replicated conditions. If, however, the rates
revert back to the values observed in the initial A
(and C) conditions, one can safely attribute the
behavioral effects to the manipulation of relative
reinforcement rate.
The second strategy is to implement the condi-
Figure 5.6. A changing-criterion design. tions not in an ascending or descending sequence
Reinforcement is contingent on particular rates of but rather in an irregular sequence. If response rates
behavior; each time behavior adjusts to a rate criterion,
a new criterion is imposed.
rise or fall simply in relation to the temporal posi-
tion of the condition, the results may be attributed
to time-related or experiential factors. If, instead, the
Parametric designs. Experiments that compare rates are systematically related to the levels of the
several levels of a quantitative treatment are said to experimental variable (e.g., if response allocations
use parametric designs. The literature on choice (e.g., match reinforcer allocations), the most plausible
Chapter 14, this volume) abounds with such designs; explanation would identify the experimental vari-
for example, in studies of matching, a pigeon may be able as the causal factor.
exposed to a series of conditions that differ in terms The last strategy departs from a purely single-
of the distribution of food reinforcers across a pair case analysis: Different participants are exposed to
of concurrently available response keys. Across suc- the conditions in different orders. For example, one
cessive conditions, the relative rate of reinforcement participant may experience an ascending sequence
might be progressively increased (an ascending order) while another experiences a descending sequence
or decreased (a descending order), or the rates may and yet a third experiences an irregular sequence, or
be imposed in some irregular order. From a design each participant may receive a different irregular
standpoint, the issue is how to dissociate the effects order. If the behavior of all the participants shows
of the experimental variable from the maturation, the same relation to the experimental variable,
history, testing, or instrumentation. If an experiment despite the variation in the temporal order of the
arranges five relative rates in an ascending sequence, conditions, then it would again appear that the
the design might be designated an A-B-C-D-E design. experimental manipulation is responsible.
It is easy to see that the fundamental logic parallels It is beneficial to combine these strategies. For
that of the A-B design, and as such, the design is vul- example, one might arrange one or more replicated
nerable to the same threats to internal validity. If, for conditions as part of each participant’s experience,
example, relative response rates rise across the suc- while arranging different sequences of conditions
cessive conditions, the outcome may be attributed to across participants. If a systematic relation between
the experimental manipulation (response allocations the experimental manipulation and behavior is
match reinforcer allocations), but alternative expla- observed under such circumstances, the case for
nations in terms of maturation, history, testing, or attributing causality to the experimental manipula-
instrumentation may also be plausible. tion becomes compelling.
As actually implemented, however, parametric Yet another approach is to combine the paramet-
designs rarely suffer from this problem. Three ric strategy with the A-B-A reversal strategy. An
116
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
investigator might begin the experiment with a base- was 70%, 80%, or 90% of its free-feeding weight.
line Condition A (or treat the first level of the quan- Thus, the experiment had a 2 × 2 × 3 factorial
titative independent variable as a baseline) and, after design (two levels of past reinforcer × two levels of
stabilizing behavior at each successive level of the upcoming reinforcer × three levels of body weight)
quantitative variable (Conditions B, C, etc.), return and, therefore, 12 combinations of the levels of the
to the baseline condition. Thus, an A-B-C-D-E three factors.
design could be replaced with an A-B-A-C-A-D-A- The results are shown in Figure 5.7. Each panel
E-A design. The obvious disadvantage is the large represents one of the body weight conditions. Note
investment of time in repeating the baseline condi- that this factor was manipulated quantitatively in
tion. The advantage is that the effect of each treat- an ascending series (70%, 80%, 90%), with a final
ment can be evaluated relative to a fixed baseline. phase devoted to a replication of the 70% condition.
In this way, following the recommendations offered
Factorial designs. Behavior is controlled by multi- earlier in the Parametric Designs section, the experi-
ple variables at any given moment, and experiments ment disentangled any confound between time- or
may be designed to analyze such control by includ- experience-related processes and the experimental
ing all possible combinations of the levels of two or variable of body weight. Within each panel are the
more independent variables. These factorial designs median pauses, calculated over the last 10 sessions
are ubiquitous in the behavioral and biomedical sci- of each body weight condition, in each of the four
ences. They tend to be associated with group statisti- possible combinations of the other two experimental
cal traditions—indeed, a staple of graduate training variables, the past and upcoming reinforcer magni-
in psychology is to teach the statistical methods tudes. The past reinforcer is shown on the x-axis
of analysis of variance in the context of factorial and the upcoming reinforcer is shown with filled
research designs (e.g., Keppel & Wickens, 2004). (lean) and unfilled (rich) data points.
Nevertheless, the factorial strategy is by no means
restricted to group statistical approaches (Smith,
Best, Cylke, & Stubbs, 2000) and is readily used in
single-case experiments.
As an example, consider an unpublished experi-
ment (Wade-Galuska, Galuska, & Perone, 2004)
concerned with variables that affect pausing on
fixed-ratio schedules. A pigeon was trained on a
multiple-baseline schedule in which 100 pecks on a
response key produced either 2-second or 6-second
access to mixed grain. Different key colors signaled
the two schedule components, designated here as
lean (ending in 2-second access to grain) and rich
(ending in 6-second access). This arrangement
(details are available in Perone & Courtney, 1992) Figure 5.7. A factorial design to study three factors
that could affect pausing on a fixed-ratio schedule: Past
made it possible to study, on a within-session basis, schedule condition (lean [L] or rich [R], represented
the effects of two factors on the pausing that took on the x-axis), upcoming (Upc.) schedule condition
place between components: the magnitude of the (L or R, represented by filled and unfilled circles,
respectively), and body weight (expressed as a percent-
reinforcer delivered before the pause (the past rein-
age of free-feeding weight; each weight condition is
forcer, lean or rich) and the signaled magnitude of represented in a different panel). Note the replication of
the reinforcer to be delivered on completing the the 70% body weight condition (rightmost panel). The
next ratio (the upcoming reinforcer, lean or rich). results are from a single pigeon; shown are medians and
interquartile ranges of the last 10 sessions of each con-
Another factor was manipulated across successive dition. Data from Wade-Galuska, Galuska, and Perone
phases of the experiment: The pigeon’s body weight (2004).
117
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
Within each condition, pausing was a joint func- and thereby allows for more complete and general
tion of the past and upcoming schedules of rein- descriptions of functional relations between envi-
forcement. When the key color signaled that the ronment and behavior.
upcoming schedule would be rich (unfilled circles),
the past reinforcer had no apparent effect: Pausing
Flexibility of Implementation
was brief after both lean and rich schedules. When
the key color signaled that the upcoming schedule A strength of single-case research designs lies in the
would be lean (filled circles), however, the past rein- dynamics of their implementation. The examples
forcer had a major impact: Pausing was extended we have offered of various single-case designs are
after a rich schedule. In other words, the effect of merely the usual ways in which the single-case
the past reinforcer was bounded by, or depended research design strategy is used. It is important
on, the signaled magnitude of the next reinforcer. to recognize that, in practice, the designs may be
When the effect of one factor depends on the level modified in response to the pattern of results that
of another factor, the factors are said to interact. In emerges as the data are collected. Indeed, this fea-
the conventional terminology of factorial research ture of the approach is what led Skinner (1956) to
design, the interaction between the past and upcom- favor single-case designs. It is also possible to com-
ing magnitudes of reinforcement would be called a bine aspects of the basic single-case designs and
two-way interaction. even include aspects of group comparisons. This
The interaction itself depended on the level kind of flexibility can be an asset to any program of
of the body weight factor: As body weight was experimental research. It takes on special signifi-
increased, the interaction between the past and cance when the research topic is novel, when the
upcoming magnitudes of reinforcement was investigator’s ability to exert experimental control is
enhanced. This kind of finding constitutes a three- limited by ethical or logistical considerations, and
way interaction. Note also that in the final phase of when the goal is to produce an empirically validated
the experiment, replicating the 70% body weight therapeutic result for an individual.
condition reduced the interaction between the mag- It is possible that once a behavior is changed,
nitudes to the values observed in the first phase. withdrawal of the treatment or reversal of the con-
In applied research, the use of single-case facto- tingencies in an A-B-A design may not return the
rial designs can also prove beneficial. An example is behavior to baseline values. From a therapeutic or
assessment of the interaction between the type of educational standpoint, this is not a bad thing: In
directive and reinforcement contingencies as they the long run, the therapist or teacher usually wants
affect participants’ compliance with the directives the participant’s behavior to come under the control
and disruptive behavior (Richman et al., 2001). This of, and be maintained by, the consequences it auto-
three-experiment sequence first established the matically produces, so that the participant no longer
effectiveness of various forms of directives, then depends on an intervention or treatment (see Chap-
assessed their effectiveness across situations, and ter 7, this volume). However, from an experimental
finally assessed the interaction between the forms of standpoint, it is a serious problem because it leaves
the directives and targets of differential reinforce- unanswered the question of what caused the behav-
ment contingencies. All of the experiments used ior to change in the first place: Was it the experi-
multielement designs to determine the impact of the mental treatment or some process of maturation,
independent variables on the outcomes for each of history, testing, or instrumentation? When behavior
the participants. fails to revert to baseline values in an A-B-A design,
Factorial designs are prevalent in the behavioral the investigator may switch to a multiple-baseline
sciences specifically because they provide a frame- design (if data have been collected for more than
work for describing how multiple variables interact one participant, behavior, or setting). Thus, it is
to control behavior. The presence of an interaction advisable for any investigator to consider the feasi-
sheds light on the boundaries of a variable’s effect bility of establishing multiple baselines from the
118
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
beginning, in case the behavior of interest does not These comparisons and contrasts can help to isolate
return to the baseline value. the participant, behavior, and setting variables that
Multiple-baseline designs have their own set of interact with the treatment to produce the changes.
challenges requiring dynamic decision making by For example, a teacher modeling tactics for solving
the investigator. Sometimes imposing the experi- various types of math problems may see students
mental treatment on one baseline will be followed solving problems for which the solutions have yet to
by behavioral change not only in the treated baseline be modeled. If the teacher is also collecting data on
but also in the as-yet-untreated baselines. This the students’ solving of social studies problems and
might reflect the operation of maturation, history, does not observe those problems being solved until
testing, or instrumentation—in other words, it the solutions are modeled, one can make the case
might mean that the treatment is ineffective. for the general effects of modeling problem solu-
Another possibility is that the treatment really is tions. This then sets the occasion for designing
responsible for change, and the effect has spread another investigation to systematically study the fea-
across the baselines because they are not indepen- tures of the modeling of the math problem solutions
dent of one another. This threat to internal validity, to determine which features are essential for which
which Cook and Campbell (1979) called diffusion of types of problems.
treatments, can jeopardize multiple-baseline experi- If having many baselines is not feasible and the
ments under several circumstances: (a) In a multiple- investigator faces interdependence of all of the base-
baseline across-participants design, all of the lines, the possible design tactics include (a) withdraw-
participants are in the same environment and may ing the treatment or reversing the contingencies or
learn by observing the treatment being applied; (b) arranging for a changing criterion within the
(b) in a multiple-baseline across-behaviors design, treatment. The first choice depends on the probabil-
all of the responses are coming from the same parity of the behavior’s return to baseline values and
ticipant, and learning one response may facilitate the ethical appropriateness of such a tactic. The sec-
the learning of other responses; or (c) in a multiple- ond choice depends on the feasibility of incorporat-
baseline across-settings design, the same participant ing the changing criterion into the treatment and the
is responding in all of the settings, and when the sensitivity of the behavior being measured to such
response is treated and changed in one setting, it changes. Either tactic, when successful, demon-
may change in the untreated settings. The antidote, strates the functional relation between the treatment
of course, is for the investigator to select partici- and the outcomes. They both also set up the ratio-
pants, behaviors, or settings that experience and nale for studying the interdependence of the base-
logic suggest will be independent of one another. lines in a subsequent investigation. As with all
Because experience and logic do not guarantee that efforts to investigate natural phenomena, unex-
an investigator will choose independent partici- pected results help to hone understanding of the
pants, behaviors, or settings, it is advisable to select phenomena and guide further investigations.
as many baselines as is feasible so that the probabil- Other design combinations may be considered.
ity of at least some of them being independent is Withdrawing the treatment or reversing the contin-
increased. gencies in a successful multiple-baseline experiment
Interdependence of a few baselines (changes can probe for the durability of the treatment effects
occurring concurrently across those baselines) with and add another degree of replication should the
independence of other baselines (changes occurring changes not be durable. Gradually removing compo-
only when treatment is applied) in a multiple- nents of interventions to assess the importance of
baseline design can be informative. The investigator each or the intervention’s durability is another varia-
has the opportunity to inspect the similarities across tion (a partial withdrawal design; Rusch & Kazdin,
the baselines that change concurrently and the dif- 1981). Withdrawing treatment from some partici-
ferences between those baselines and the baselines pants, responses, or settings (a sequential with-
that change only when the treatment is applied. drawal design; Rusch & Kazdin, 1981) to assess the
119
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
urability of treatment effects is another variation to

d At this point, the experimental environment is
be considered depending on the focus of the rearranged to impose the next condition, again
investigation. until behavior stabilizes.
The point of all of these additional design
options is that although the research question drives Strategic Requirements
the initial selection of the elements of single-case The steady-state strategy has three requirements
design, once the data collection begins decisions (Perone, 1994):
about the next condition are driven by the patterns
1. The investigator must have sufficient control
emerging in the data being collected. Unexpected
over extraneous variables to allow behavior to
patterns can and should lead the investigator to ask
stabilize.
how best to arrange the next phase of the investiga-
2. The investigator must be able to maintain each
tion to ensure that the original or revised research
condition long enough to allow behavior to stabi-
question can be answered unambiguously.
lize; even under ideal laboratory controls, it will
take time for behavior to reach a new equilib-
Steady-State Strategy rium when conditions are changed.
3. The investigator must be able to recognize the
Behavioral experiments assess the effect of a treat-
steady state when it is achieved.
ment by comparing behavior measured during expo-
sure to the treatment with behavior measured Meeting the first two requirements is not a mat-
without the treatment or, if the experimental logic ter of experimental design; rather, the key issues are
dictates, with behavior measured during exposure to scientific understanding and resources, including
some other treatment. In a single-case experiment, time and access to participants’ behavior. The inves-
the conditions are imposed on an individual over tigator must have a reasonable idea of the extrane-
some period of time, and behavior is measured ous variables to be eliminated or held constant to
repeatedly within each condition. Inferences about allow the potential effect of the experimental vari-
the experimental treatment’s effectiveness are usu- able to become manifest. The investigator must have
ally supported by demonstrations that the difference the wherewithal to control the extraneous variables,
in behavior observed across the conditions clearly and he or she must have relatively unimpeded
exceeds any variability observed within the condi- access to the behavior of interest: An A-B-A-B
tions. The basic strategy is not unlike the one that design, for example, may require scores of sessions
underlies conventional tests of statistical inference: distributed over several months or more if behavior
The F ratio associated with the analysis of variance is to be given time to stabilize in each phase.
is formed by dividing an estimate of variance In any given area of study, initial investigations
between experimental groups by an estimate of vari- will suffer from gaps in the understanding of the
ance within the groups, and only if the between- behavioral processes at work and, consequently, of
groups variance is large relative to the within-group the variables in need of control. Persistent efforts at
variance does the investigator conclude that the experimental analysis will pay dividends in identify-
experimental treatment made a statistically signifi- ing the relevant variables and developing the means
cant difference. to control them.
The prevailing approach in single-case Persistence alone, however, cannot provide an
experiments—the steady-state strategy—is to investigator the access to behavior that may be
impose a condition until behavior is more or less needed to execute the steady-state strategy. Much
stable from one measurement (session, lesson, depends on the nature of the topic at hand and the
etc.) to the next. The idea is to fix the environ- available resources. Investigators of topics in basic
mental variables controlling behavior until the research may be in the most advantageous position,
environment–behavior relation reaches equilib- especially if they study animals. Not only are they
rium or, as Sidman (1960) put it, a steady state. able to control almost every facet of the animal’s
120
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
living arrangements (e.g., diet, housing, light–dark manipulations may not always develop according to
cycles, opportunities to engage conspecifics), they the investigator’s timetable. Sometimes there is no
also have unfettered access to the animal’s behavior. substitute for prolonged exposure to the contingen-
Sessions may be conducted daily for months without cies, and what happens in the short term may not
interruption. Such circumstances are ideal for predict what happens in the long term (for an illus-
steady-state research. tration, see Baron & Perone, 1998, pp. 50–52). In
Special problems arise when human participants applied research, logistical and ethical issues mag-
replace rats and pigeons (Baron & Perone, 1998). nify the problem of behavioral access. Participants
The typical human participant lives, works, plays, with clinically relevant repertoires may not be avail-
eats, drinks, and sleeps outside of the investigator’s able in large numbers, and the nature of their prob-
influence and is thus exposed to numerous factors lem behavior may sharply limit the duration of
that may play a role in the participant’s experimental sessions. If the research is conducted in a therapeu-
behavior (only in rare cases do human participants tic context, addressing the participant’s problem will
live in the laboratory; for an interesting example, see take priority over purely scientific considerations,
Bernstein & Ebbesen, 1978). These limitations indi- and ethical concerns about leaving problem behav-
cate a need for strong countermeasures, such as ior untreated may restrict the nature of the experi-
experimental manipulations that are “especially mental designs as well as the durations of both
forcing” (Morse & Kelleher, 1977; see also Baron & baseline and treatment conditions.
Perone, 1998, pp. 68–69) and increased exposure to The steady-state strategy works best when behav-
the laboratory environment over an extended series ior is measured repeatedly under controlled experi-
of sessions. Unfortunately, human research—when mental conditions imposed long enough for the
extended access to the participant’s behavior may be behavior to reach demonstrable states of equilib-
needed most—is when such access is most difficult rium. The pages of the Journal of the Experimental
to attain. Monetary incentives can help bring partici- Analysis of Behavior and the Journal of Applied
pants to the laboratory for repeated study, of course, Behavior Analysis attest that these challenges can be
but even well-funded investigators will find that the met. It is inevitable, however, that some experi-
number of sessions that, say, a college student will ments will fall short. In some cases, conducting
tolerate is lower than that commonly conducted in single-case experiments in the absence of steady
research with rats. To address this practical con- states will still be possible, as suggested by the hypo-
straint, some investigators arrange brief sessions, thetical outcomes depicted in Figures 5.2, 5.3, and
sometimes lasting as little as 10 minutes (e.g., 5.6. Even in these examples, however, the number
Okouchi, 2009), and schedule a series of such ses- of behavioral observations is large. We suggest,
sions each time the participant visits the laboratory. therefore, that although single-case experiments
Of course, the duration of the sessions is not the may be viable in some cases without steady states,
critical issue; rather, the question is whether one they are not likely to succeed without significant
can complete an experiment in a few hours in the access to the behavior in the form of extensive
human laboratory and compare the results to experi- repeated measurement (for a comprehensive discus-
ments that take months or years in the animal labo- sion of this issue in the context of applied research,
ratory. The answer will depend on the goals of the see Barlow et al., 2009, pp. 62–65 and 88–94, and
research as well as the investigator’s judgment about Johnston & Pennypacker, 2009, pp. 191–218).
the size of the anticipated effects and the speed of When the efforts to achieve steady states fall short,
their onset. Relatively brief experiments can be an investigator may consider the use of statistical tests
defended when they are successful in producing sta- to discriminate treatment effects from a noisy back-
ble and reproducible behavioral outcomes within ground of behavioral variability. Many arguments,
and across participants. Caution is warranted in both pro and con, have been made in this connection
planning and interpreting such experiments, however, (e.g., Ator, 1999; Baron, 1999; Branch, 1999; Crosbie,
because the behavioral effects of the experimental 1999; Davison, 1999; Kratochwill & Levin, 2010;
121
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
Perone, 1999; Shull, 1999; Smith et al., 2000; Todman absolute measures; for example, the mean rates in
& Dugard, 2001; see Chapters 7 and 11, this volume). the first three sessions and last three sessions may
We are concerned that reliance on inferential statistics differ by no more than five responses per minute.
may retard the search for effective forms of control. Not all stability criteria are expressed in quantita-
By comparison, the major advantage of the steady- tive terms. In some experiments, steady states are
state strategy is that it fosters the development of identified by visual inspection of graphed results. In
strong control. Unsystematic variability (noise or other experiments, each condition is imposed for a
bounce in the data) is addressed by reducing the fixed number of sessions (e.g., 30), and behavior in
influence of extraneous factors and increasing the the last several sessions (e.g., five) is considered rep-
influence of the independent variable. Systematic vari- resentative of the steady state.
ability (the trend that occurs in the transition between As Sidman (1960) noted, the selection of a stabil-
steady states) is addressed by holding the experimen- ity criterion depends on the nature of the experi-
tal environment constant until behavior stabilizes. Put mental question and the investigator’s judgment and
simply, the steady-state strategy demands that treat- experience. The visual stability criterion may be jus-
ment effects be clarified by improving direct control tified, for example, when the investigator’s experi-
of behavior. ence leads to the expectation of large or dramatic
changes across conditions. The fixed-time stability
Stability Criteria criterion works well when a program of research has
The final requirement of the steady-state strategy is progressed to the point at which the investigator can
that of recognizing the production of a steady state. confidently predict how many sessions will be
Various decision rules have been devised for this needed to achieve a steady state. Even the quantita-
purpose. These stability criteria are often expressed tive criteria—the relative stability criterion and the
in mathematical terms and indicate, in one way or absolute stability criterion—are specified in light of
another, the kind and amount of variation in behav- the experimental question and the investigator’s
ior that will be acceptable over a series of observa- judgment and experience. In the abstract, divorced
tions. Commonly used criteria specify (a) the from such considerations, it is impossible to say, for
number of sessions or observations to be considered example, whether a 10% relative criterion is more or
in assessing the evidence of a steady state, (b) that less stringent than a five-responses-per-minute abso-
an increasing or decreasing trend must be absent, lute criterion (for a detailed discussion of the rela-
and (c) how much bounce can be tolerated in the tionship between relative and absolute stability
behavior across sessions. If the most recent behavior criteria, see Perone, 1991, pp. 141–144).
within a condition (e.g., responding in the last six The adequacy of a stability criterion is assessed
sessions) is absent of trend and reasonably free from over the course of an experiment. A criterion is ade-
bounce, behavior is said to be stable. quate, according to Sidman (1960, p. 259), if it
Sidman (1960) provided the seminal discussion yields orderly and replicable functional relations
of stability criteria. Detailed descriptions of stability between the independent and dependent variables.
criteria, with examples from the human and animal In this connection, it is important to recognize that
literature, can be found in Baron and Perone (1998) any stability criterion, no matter how stringent, may
and Perone (1991). be met by chance, that is, in the absence of an actual
Perhaps the most important difference among steady state. However, a criterion is highly unlikely
stability criteria is in how they specify the tolerable to be repeatedly met by chance across the various
limits on bounce. Some criteria use relative mea- experimental conditions.
sures; for example, when considering the most
recent six sessions, the mean response rate in the
Once is not Enough
first three sessions and the mean in the last three
sessions may differ by no more than 10% of the Single-case designs are single because the primary
overall six-session mean. Other criteria may use unit of analysis is the behavior of the individual
122
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
organism. Treatment effects are assessed by compar- elephantine proportions. A result that can be repli-
ing the individual’s response with different levels of cated on an intraparticipant basis (i.e., from A to B,
the independent variable, and control is demon- back to A, and back again to B) cannot be dismissed
strated by two kinds of replication: (a) the stability so easily. The failure to replicate the effect in
of the individual’s behavior from one observation to another participant does not negate the original
the next under constant circumstances within a con- finding; rather, it unveils the incompleteness of
dition and (b) the stability of the change in the indi- one’s understanding of the original finding. The
vidual’s behavior from one experimental condition investigator may have erred in his or her operational
to another. definition of the independent variable, his or her
The single descriptor is misleading in that single- control of the independent variable may be defec-
case research rarely involves just one individual. tive, or the investigator may have failed to recognize
In addition to the within-participant forms of repli- other relevant variables and isolate the experiment
cation that we have emphasized throughout this from their influence. “If this proves to be the case,”
chapter, procedures are also replicated across said Sidman (1960, p. 74), “failure of [interpartici-
participants. Single-case investigators approach pant] replication will serve as a spur to further
interparticipant replication in two general ways, research rather than lead to a simple rejection of the
described by Sidman (1960) as direct replication original data.”
and systematic replication (see also Chapter 7, this Systematic replication is an attempt to replicate a
volume). functional relation under circumstances that differ
In the context of interparticipant replication, from those of the original experiment. The experi-
direct replication consists of repeating the experimental conditions might be imposed in a different
mental procedures with additional participants. A order. The range of a parametric variable might be
review of any representative sample of the literature extended. The personal characteristics of a thera-
of basic or applied behavior analysis will document peutic agent or teacher might be changed (e.g., from
that direct interparticipant replication is, for all female to male). The classification of the partici-
intents and purposes, required to establish the credi- pants might differ (e.g., pigeons might be studied
bility of single-case experimentation—even in basic instead of rats or typically developing children
laboratory research with animals, where control is at instead of those with developmental delays). New
its utmost. Why, in a science devoted to the analysis behavioral repertoires might be observed (e.g.,
of behavior in the individual organism, should be swimming instead of studying), new stimulus
this so? Interparticipant replication is needed to modalities might be activated (e.g., with auditory
show that the investigator has been successful in rather than visual stimuli), or new behavioral conse-
identifying the relevant variables and bringing them quences might be arranged (e.g., attention instead of
under satisfactory degrees of control. Whenever edibles or the postponement of a shock instead of
manipulation of an independent variable produces the presentation of a food pellet). In this way—by
the same kind of behavioral change in a new partici- replicating functional relations across a range of
pant, one grows increasingly confident that the individuals, behaviors, and operations—investigators
investigator is both manipulating the causal factor can discover the boundaries of a phenomenon and
and eliminating (or otherwise controlling) the influ- thereby reach conclusions about its generality. This
ence of extraneous factors that could obscure the issue is discussed in more detail in this volume’s
causal relation. Chapter 7.
What if the attempt at interparticipant replica- Direct replications are often considered an inte-
tion fails? Suppose, for example, that an A-B-A-B gral part of a given experiment: The report of a typi-
design produces a clear, reliable effect in one partici- cal experiment incorporates single-case results from
pant but not in another? One might be inclined to several participants, and the similarity of results
question the reality of the original result, to declare across the participants is a key feature in assessing
it a fluke. However, this would be a strategic error of the adequacy of control over the variables under
123
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
study. By comparison, systematic replications are experimental treatments or describing functional

conducted across experiments and, over the course relations, although their effectiveness is obviously
of time, across investigators, laboratories, clinics, important. There is another less practical, more the-
and so forth. When investigators refer to research oretical reason. Fundamentally, the prevalence of
topics or areas of investigation, they are commonly the approach derives from a conception of behavior
referring to collections of systematic replications as an intrinsically intraindividual phenomenon, a
that have been designed specifically to address the continuous reciprocal interaction between an organ-
limits of some behavioral phenomenon (e.g., con- ism and its environment (e.g., Skinner, 1938, 1966).
sider the literatures on choice, conditioned rein- By this reckoning, only methods that respect the
forcement, resurgence, and response-independent individual character of behavior—single-case
presentations of previously established reinforcers, methods—are valid. The point was made forcefully
so-called “noncontingent reinforcement”). by Sidman (1960), who essentially ruled that meth-
Sometimes systematic replications are a by- ods based on comparisons across individuals—as in
product, rather than the focus, of the research, as group statistical methods—fall outside the boundar-
when an investigator makes use of a previous find- ies of behavioral science.
ing to pursue a new line of inquiry. For example, the To illustrate, Sidman (1960) considered the
investigator might use fixed-interval schedules of plight of an investigator interested in the effects of
reinforcement as a baseline procedure to evaluate the number of reinforcements during acquisition of
the effects of a drug on temporally controlled behav- behavior on the subsequent extinction of the behav-
ior. Whatever may be learned about the drug’s ior. It is easy to see the severe limitation of a single-
effects, the results are likely to extend understand- case experiment that would expose an individual to
ing of fixed-interval behavior. The ubiquity of such a series of reinforcement conditions, each followed
research (Sidman [1960] called it the baseline by an extinction test. Clearly, the individual’s cumu-
method of systematic replication) is responsible for an lating experience with successive extinctions would
array of empirical generalizations—such as schedule be confounded with the effects of the reinforcement
effects—that have come to be regarded as founda- variable; one could expect extinction to proceed
tions of basic and applied behavior analysis. more rapidly with experience regardless of the num-
ber of reinforcements in the acquisition phase before
each test. An obvious solution might be to expose
Final Remarks
separate groups of individuals to the values of the
Single-case methods are well suited to the intensive reinforcement variable and combine the results of
study of behavior at the level of the individual each group’s single extinction test to construct a
organism. When properly designed and executed, function relating the number of reinforcements to
single-case experiments offer robust protection the rate of extinction. “But,” said Sidman,
against threats to internal validity. Their ability to
support causal inferences certainly matches that of the function so obtained does not rep-
the group comparison methods that dominate the resent a behavioral process. The use of
behavioral sciences, despite the small number of separate groups destroys the continuity
participants and the absence of sophisticated statisti- of cause and effect that characterizes an
cal tests. Strong methods of direct control over irreversible behavioral process. . . . If it
behavior, exemplified by the steady-state strategy, proves impossible to obtain an uncon-
obviate the need for statistical inference, and repli- taminated relation between number of
cation within and across participants attests to the reinforcements and resistance to extinc-
adequacy of the control. tion in a single subject, because of the
The prevalence of single-case designs in basic fact that successive extinctions interact
and applied behavior analysis is not simply a matter with each other, then the “pure” relation
of their effectiveness in assessing the effects of simply does not exist. The solution to
124
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
our problem is to cease trying to discover Bijou, S. W. (1955). A systematic approach to an

such a pure relation, and to direct our experimental analysis of young children. Child
Development, 26, 161–168.
research toward the study of behavior as
Branch, M. N. (1999). Statistical inference in behavior
it actually exists. . . . The [investigator]
analysis: Some things significance testing does and
should not be deceived into concluding does not do. Behavior Analyst, 22, 87–92.
that the group type of experiment in any Browning, R. M. (1967). A same-subject design for simul-
way provides a more adequately con- taneous comparison of three reinforcement contin-
trolled or more generalizable substitute gencies. Behaviour Research and Therapy, 5, 237–243.
for individual data. (p. 53) doi:10.1016/0005-7967(67)90038-1
Campbell, D. T., & Stanley, J. C. (1963). Experimental
For investigators who endorse Sidman’s (1960) and quasi-experimental designs for research. Chicago,
position, the adoption of single-case methods is not IL: Rand McNally.
a pragmatic decision. To the contrary, it is a theoret- Cook, T. D., & Campbell, D. T. (1979). Quasi-
ical commitment: Single-case methods are seen as a experimentation: Design and analysis issues for field
settings. Chicago, IL: Rand McNally.
defining feature of behavioral science (see also John-
Crosbie, J. (1999). Statistical inference in behavior analy-
ston & Pennypacker, 2009; Sidman, 1990). For
sis: Useful friend. Behavior Analyst, 22, 105–108.
these investigators, to depart from single-case meth-
Davison, M. (1999). Statistical inference in behavior anal-
ods is to change fields. By investigating functional ysis: Having my cake and eating it? Behavior Analyst,
relations via the single-case approach described in 22, 99–103.
this chapter, a thorough understanding of how and Ebbinghaus, H. (1913). Memory: A contribution to experi-
why behavior occurs can be achieved. mental psychology (H. A. Ruger & C. E. Bussenius,
Trans.). New York, NY: Columbia Teachers College.
(Original work published 1885)
References
Fisher, R. A. (1925). Statistical methods for research work-
Ator, N. A. (1999). Statistical inference in behavior analy- ers. Edinburgh, Scotland: Oliver & Boyd.
sis: Environmental determinants? Behavior Analyst,
22, 93–97. Hartmann, D. P., & Hall, R. V. (1976). The changing cri-
terion design. Journal of Applied Behavior Analysis, 9,
Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some 527–532. doi:10.1901/jaba.1976.9-527
current dimensions of applied behavior analysis.
Journal of Applied Behavior Analysis, 1, 91–97. Horner, R. D., & Baer, D. M. (1978). Multiple-probe tech-
doi:10.1901/jaba.1968.1-91 nique: A variation on the multiple baseline. Journal of
Baer, D. M., Wolf, M. M., & Risley, T. R. (1987). Some jaba.1978.11-189
still-current dimensions of applied behavior analysis.
Journal of Applied Behavior Analysis, 20, 313–327. Johnston, J. M., & Pennypacker, H. S. (2009). Strategies
doi:10.1901/jaba.1987.20-313 and tactics of behavioral research (3rd ed.). New York,
NY: Routledge.
Bailey, J. S., & Burch, M. R. (2002). Research methods in Kazdin, A. E. (1982). Single-case research designs: Methods
applied behavior analysis. Thousand Oaks, CA: Sage. for clinical and applied settings. New York, NY:
Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single Oxford University Press.
case experimental designs: Strategies for studying Kazdin, A. E. (1999). Research designs in clinical psychol-
behavior change (3rd ed.). Boston, MA: Pearson. ogy. New York, NY: Allyn & Bacon.
Baron, A. (1999). Statistical inference in behavior analy- Keppel, G., & Wickens, T. D. (2004). Design and analy-
sis: Friend or foe? Behavior Analyst, 22, 83–85. sis: A researcher’s handbook (4th ed.). Upper Saddle
Baron, A., & Perone, M. (1998). Experimental design River, NJ: Prentice-Hall.
and analysis in the laboratory study of human oper- Kratochwill, T. R., & Levin, J. R. (2010). Enhancing
ant behavior. In K. A. Lattal & M. Perone (Eds.), the scientific credibility of single-case intervention
Handbook of research methods in human operant research: Randomization to the rescue. Psychological
behavior (pp. 45–91). New York, NY: Plenum Press. Methods, 15, 124–144. doi:10.1037/a0017736
Bernstein, D. J., & Ebbesen, E. B. (1978). Reinforcement Lucki, I., & DeLong, R. E. (1983). Control rate of
and substitution in humans: A multiple-response response or reinforcement and amphetamine’s effect
analysis. Journal of the Experimental Analysis of on behavior. Journal of the Experimental Analysis of
Behavior, 30, 243–253. doi:10.1901/jeab.1978.30-243 Behavior, 40, 123–132. doi:10.1901/jeab.1983.40-123
125
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Perone and Hursh
Mazur, J. E. (1985). Probability and delay of reinforce- Rusch, F. R., & Kazdin, A. E. (1981). Toward a method-
ment as factors in discrete-trial choice. Journal of ology of withdrawal designs for the assessment of
the Experimental Analysis of Behavior, 43, 341–351. response maintenance. Journal of Applied Behavior
doi:10.1901/jeab.1985.43-341 Analysis, 14, 131–140. doi:10.1901/jaba.1981.14-131
McSweeney, F. K., & Murphy, E. S. (2000). Criticisms Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002).
of the satiety hypothesis as an explanation for Experimental and quasi-experimental designs for gener-
within-session decreases in responding. Journal of alized causal inference. Boston, MA: Houghton Mifflin.
Shull, R. L. (1999). Statistical inference in behavior
doi:10.1901/jeab.2000.74-347
analysis: Discussant’s remarks. Behavior Analyst, 22,
McSweeney, F. K., & Roll, J. M. (1993). Responding 117–121.
changes systematically within sessions during con-
Sidman, M. (1960). Tactics of scientific research:
ditioning procedures. Journal of the Experimental
Evaluating experimental data in psychology. New
York, NY: Basic Books.
jeab.1993.60-621
Sidman, M. (1990). Tactics: In reply. Behavior Analyst,
Morgan, D. L., & Morgan, R. K. (2009). Single-case
13, 187–197.
research methods for the behavioral and health sciences.
Thousand Oaks, CA: Sage. Skinner, B. F. (1938). The behavior of organisms. New
York, NY: Appleton-Century-Crofts.
Morse, W. H., & Kelleher, R. T. (1977). Determinants of
reinforcement and punishment. In W. K. Honig & Skinner, B. F. (1956). A case history in scientific method.
J. E. R. Staddon (Eds.), Handbook of operant behavior American Psychologist, 11, 221–233. doi:10.1037/
(pp. 174–200). Englewood Cliffs, NJ: Prentice-Hall. h0047662
Moxley, R. A. (1998). Treatment-only designs and stu- Skinner, B. F. (1966). What is the experimental analysis
dent self-recording as strategies for public school of behavior? Journal of the Experimental Analysis of
teachers. Education and Treatment of Children, 21, Behavior, 9, 213–218. doi:10.1901/jeab.1966.9-213
37–61. [Erratum. (1998). Education and Treatment of Smith, L. D., Best, L. A., Cylke, V. A., & Stubbs, D. A.
Children, 21, 229.] (2000). Psychology without p values: Data analysis
Okouchi, H. (2009). Response acquisition by humans with at the turn of the century. American Psychologist, 55,
delayed reinforcement. Journal of the Experimental 260–263. doi:10.1037/0003-066X.55.2.260
Analysis of Behavior, 91, 377–390. doi:10.1901/ Stubbs, D. A., & Pliskoff, S. S. (1969). Concurrent
jeab.2009.91-377 responding with fixed relative rate of reinforcement.
Perone, M. (1991). Experimental design in the analysis Journal of the Experimental Analysis of Behavior, 12,
of free-operant behavior. In I. H. Iversen & K. A. 887–895. doi:10.1901/jeab.1969.12-887
Lattal (Eds.), Techniques in the behavioral and neural
Thorndike, E. L. (1911). Animal intelligence: Experimental
sciences: Vol. 6. Experimental analysis of behavior,
studies. New York, NY: Macmillan. doi:10.5962/bhl.
Part 1 (pp. 135–171). Amsterdam, the Netherlands:
title.1201
Elsevier.
Todman, J. B., & Dugard, P. (2001). Single-case and
Perone, M. (1994). Single-subject designs and devel-
small-n experimental designs: A practical guide to ran-
opmental psychology. In S. H. Cohen & H. W.
domization tests. Mahwah, NJ: Erlbaum.
Reese (Eds.), Life-span developmental psychology:
Methodological contributions (pp. 95–118). Hillsdale, Ulman, J. D., & Sulzer-Azaroff, B. (1975). Multielement
NJ: Erlbaum. baseline design in educational research. In E. Ramp
& G. Semb (Eds.), Behavior analysis: Areas of
Perone, M. (1999). Statistical control in behavior analy-
research and application (pp. 371–391). Englewood
sis: Experimental control is better. Behavior Analyst,
Cliffs, NJ: Prentice-Hall.
22, 109–116.
Wade-Galuska, T., Galuska, C. M., & Perone, M. (2004).
Perone, M., & Courtney, K. (1992). Fixed-ratio paus-
[Pausing during signaled rich-to-lean shifts in rein-
ing: Joint effects of past reinforcer magnitude and
forcer context: Effects of cue accuracy and food
stimuli correlated with upcoming magnitude. Journal
deprivation]. Unpublished raw data.
doi:10.1901/jeab.1992.57-33 Watson, J. (1913). Psychology as the behaviorist views
Richman, D. M., Wacker, D. P., Cooper-Brown, L. J., it. Psychological Review, 20, 158–177. doi:10.1037/
Kayser, K., Crosland, K., Stephens, T. J., & Asmus, J. h0074428
(2001). Stimulus characteristics within directives: Wertheimer, M. (1912). Experimentelle studien uber das
Effects on accuracy of task completion. Journal of sehen von bewegung [Experimental studies of the
Applied Behavior Analysis, 34, 289–312. doi:10.1901/ perception of motion]. Zeitschrift fur Psychologie, 61,
jaba.2001.34-289 61–265.
126
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 6
Observation and Measurement

in Behavior Analysis
Raymond G. Miltenberger and Timothy M. Weil
You can observe a lot by watching. Behavior

—Yogi Berra, circa 1964
Regardless of whether the purpose of investigation is
Throughout its history, behavior analysis has research or practice, it is first necessary to define
focused on building an inductive science that uses behavior. Many authors define behavior in slightly
behavioral observation techniques to identify func- different terms; however, each stresses an individu-
tional behavior–environment relations such that al’s action or movement. According to Miltenberger
alteration of these relations may result in behavior (2012), behavior involves the actions of individuals,
change that is scientifically, and often socially, what people say and do. Malott and Trojan-Suarez
meaningful. Measuring behavior–environment rela- (2004) suggested that behavior is anything a dead
tions in accurate and reliable ways is thus an inte- man cannot do, again suggesting that behavior con-
gral tool in the process of analyzing and changing sists of action or movement. Cooper, Heron, and
behavior. Observable behavior has formal properties Heward (2007) said that “behavior is the activity of
or dimensions that can be measured. These behav- living organisms. Human behavior is everything
ioral dimensions include frequency, intensity, dura- people do including how they move and what they
tion, and latency (e.g., Bailey & Burch, 2002; say, think, and feel” (p. 25). Finally, Johnston and
Miltenberger, 2012). Each of these dimensions Pennypacker (1993) stated that behavior
affords the observer the opportunity to measure
is that portion of an organism’s interac-
changes in level, trend, and variability when altera-
tion with its environment that is char-
tions of environmental variables occur naturally or
acterized by detectable displacement in
are manipulated under various programmed
space through time of some part of the
conditions.
organism and that results in a measurable
Observation and measurement of behavior may
change in at least one aspect of the envi-
take many forms and involve a variety of techniques
ronment. (p. 23)
across practically any setting. In this chapter, we
discuss observation and measurement in the field of These definitions of behavior are rooted in the tradi-
behavior analysis with a focus on identifying and tional characterization of an operant as observable
measuring the target behavior, logistics of observa- action or movement that has some subsequent effect
tion, recording procedures and devices, reactivity, on (operates on) the environment (Johnston &
interobserver agreement (IOA), and ethical consid- Pennypacker, 1993, p. 25). Although the Cooper et al.
erations. The information discussed in this chapter definition of behavior includes thinking and feeling,
is relevant to both research and practice in behavior these actions are nonetheless those of an individual
analysis because observation and measurement of that can be observed and recorded. Therefore, in this
behavior is central to both endeavors. chapter we focus on observation and measurement of
DOI: 10.1037/13937-006
127
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Miltenberger and Weil
behavior that can be detected, and thus recorded, by the target behavior. For example, in research on
an observer. In some cases, the observer may be the stuttering, Wagaman, Miltenberger, and Arndorfer
individual engaging in the behavior. (1993) chose a criterion of 3% or fewer stuttered
words as an indication of treatment success on the
basis of research showing that as many as 3% of the
Selecting and Defining Target
words spoken by typical speakers were dysfluent.
Behavior
A guiding factor in the selection of a target
The first step in developing a plan for observing and behavior in applied work is its social significance
recording behavior is to select and then define (Baer, Wolf, & Risley, 1968). Behavior is targeted
appropriate target behavior. that will increase the client’s meaningful and effec-
tive interactions with the environment. One index of
Selecting Target Behavior social significance is the assessment of the social
Target behavior can be selected for a variety of over- validity of the targeted behavior (Wolf, 1978).
lapping reasons (see Kazdin, 2010). It may be useful According to Wolf (1978), one of the three levels of
but arbitrary; representative of a broader class of social validity is the degree to which society vali-
operants; the focus of intervention or educational dates the social significance of the goals of a behav-
efforts that occur in a particular setting (e.g., aca- ior change procedure. In this regard, the important
demic performance in a school setting); chosen question posed by Wolf is, “Are the specific behav-
because it causes impairment in some area of func- ioral goals really what society wants?” (p. 207). In
tioning for the individual; of particular concern to practice, assessment of the social validity of a target
the individual or significant others who seek to behavior or goal involves asking consumers for feed-
change the behavior; or chosen because it will pre- back on what behavior should be addressed, in what
vent the development of future problems (e.g., the order, and to what extent. Of course, the target
promotion of safety skills to prevent injury). When behavior selected in this way may possibly have
selecting target behavior, three general categories some secondary gain or benefit for the person pro-
are considered: behavioral deficits, behavioral viding the report and thus may or may not be in the
excesses, and problems of stimulus control. Behav- client’s best interest. The behavior analyst must be
ioral deficits are behaviors that need to increase, aware of this possibility and decide with the client,
such as desirable communicative responses for a client surrogates, treatment team members, or some
child with autism who has limited language (e.g., or all of these on the target behavior that best serves
Sundberg & Michael, 2001). Behavioral excesses the client’s interests.
are behaviors that need to decrease, such as self- Although behavior analysts are interested in the
injurious or aggressive behavior emitted by an indi- behavior of clients or research participants, they are
vidual with intellectual disability (e.g., Lerman, Iwata, also interested in the behavior of the implementers
Smith, & Vollmer, 1994). Problems of stimulus con- carrying out behavior-analytic procedures. The
trol are present when behaviors occur, but not at the degree to which individuals implement assessment
appropriate time or place or in the appropriate con- and treatment procedures as planned is referred to
text. For example, a child may learn to engage in a as implementation fidelity, procedural fidelity, or treat-
safety skill during training but fail to use it when the ment integrity (e.g., Gresham, Gansle, & Noell,
opportunity arises in the natural environment (e.g., 1993; Peterson, Homer, & Wonderlich, 1982).
Gatheridge et al., 2004; Himle, Miltenberger, Fless- Implementation fidelity is important because higher
ner, & Gatheridge, 2004). Likewise, a child with fidelity is associated with better treatment outcomes
autism may learn to label an object but be unable to (e.g., DiGennaro, Martens, & Kleinmann, 2007;
ask for the same object (e.g., failure of tact-to-mand DiGennaro, Martens, & McIntyre, 2005; DiGennaro-
transfer; Wallace, Iwata, & Hanley, 2006). Identify- Reed, Codding, Catania, & Maguire, 2010; Plavnick,
ing developmentally appropriate topographies and Ferreri, & Maupin, 2010). Implementation fidelity
levels of behavior is also important when selecting is assessed by observing and recording the behavior
128
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Observation and Measurement in Behavior Analysis
of the implementers as they observe and record the behavior analyst begins to observe instances of the
behavior of the clients or research participants and behavior, the behavioral definition may be modified
as the implementers carry out intervention proce- on the basis of those observations. Some examples of
dures. Everything discussed in this chapter applies target behavior definitions used in behavior-analytic
not only to observing and recording the behavior of research are shown in Table 6.1. Note the precise
clients or research participants, but also to measur- descriptions of behavior in these examples and the
ing the behavior of the implementers. inclusion of the appropriate context for the behavior
(e.g., unscripted verbalizations are “verbalizations
Defining the Target Behavior that were not modeled in the video script but were
The target behavior should be defined in terms that appropriate to the context of the toy”) or the neces-
are objective, clear, and complete (Kazdin, 2010). A sary timing of the behavior in relation to other
behavioral definition must include active verbs events (e.g., an acceptance occurs when “the child’s
describing the individual’s actions, that is, the topog- mouth opened . . . within 3 seconds after the food
raphy or form of the action being observed. Some item was held within 1 inch of the mouth”).
behavioral definitions may also include the environ-
mental events that precede (antecedents) or follow
Logistics of Observation
(consequences) the behavior. Behavioral definitions
cannot include category labels (e.g., aggression) or Once the target behavior is identified and defined,
appeal to internal states or characteristics (e.g., the time and place of observation must be deter-
strong willed) but rather must identify the topogra- mined and the observers identified. These logistics of
phy of the behavior. A behavioral definition should observation are not insignificant, because the choices
be easy to read and should suffice as a starting point of observation periods and observers will determine
for the observer to engage in data collection. Once a the quality of the data derived from the observations.
Table 6.1
Examples of Target Behavior Definitions From Published Articles Involving Behavior Analytic Assessment
and Treatment
Label Definition Citation

Empathy “A contextually appropriate response to a display of Schrandt, Townsend, and Poulson
affect by a doll, puppet, or person that contained (2009)
motor and vocal components (in any order) and
began within 3 s of the end of the display.” (p. 20)
Acceptance “The child’s mouth opened so that the spoon or piece of Riordan, Iwata, Finney, Wohl, and
food could be delivered within 3 s after the food item Stanley (1984)
was held within 1 in. of the mouth.” (p. 329)
Expulsion “Any amount of food (that had been in the mouth) was Riordan et al. (1984)
visible outside the mouth (Joan only) or outside
the lip and chin area (Nancy, Jerry, Holly) prior to
presentation of the next bite.” (p. 329)
Activity engagement “Facial orientation toward activity materials, appropriate Mace et al. (2009)
use of activity materials, or comments related to the
activity.” (p. 178)
Compliance “The child independently completing or initiating the Wilder, Zonneveld, Harris, Marcus, and
activity described in the instruction within 10 s.” (p. Reagan (2007)
535)
Unscripted verbalizations “Verbalizations that were not modeled in the video script MacDonald, Sacramone, Mansfield,
but were appropriate to the context of the toy [that Wiltz, and Ahern (2009)
was present].” (p. 47)
129
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Time and Place of Observations could be undesirable or unacceptable for the client
Observation periods (the time and place chosen for or significant others, such as teachers or parents.
observation) should be scheduled when the target Such delays can sometimes be circumvented by
behavior is most likely to occur (or, in the case structuring observations in an analog setting to eval-
of behavioral deficits, when the target behavior uate the effects of likely antecedent and consequent
should be occurring but is not). In some cases, the stimuli with the objective of evoking the target
target behavior occurs mostly or exclusively in the behavior. Alternatively, samples of the behavior
context of specific events (e.g., behavioral acquisi- might be collected in the natural environment at
tion training in academic sessions or athletic perfor- various times to provide a sufficient baseline that
mance), and therefore, observation sessions have to would allow making an accurate assessment of func-
occur at those times. A behavior analyst, however, tion, deciding on an appropriate intervention, or
may be interested in measuring the target behavior both. Circumstances such as availability of observers
in naturalistic settings in which the behavior is, for and the client’s availability must also be considered.
the most part, free of temporal constraints. In these A final consideration in preparing to make obser-
instances, it is possible to interview the client and vations is to select a placement within the observa-
significant people to narrow the observation win- tion environment that permits a full view of the
dow. In addition, it may be valuable to validate the person and the behavior of interest while at the
reports by collecting initial data on the occurrence same time minimizing disruptions to the client or
of the target behavior. For instance, scatterplot others in the environment. In addition, when col-
assessments, which identify at half-hour intervals lecting IOA data, it is important for both observers
throughout the day whether the behavior did not to see the same events from similar angles and dis-
occur, occurred once, or occurred multiple times, tances but simultaneously maintain their status as
may help identify the best time to schedule the independent observers. Depending on the character-
observation period (Touchette, Macdonald, & istics of the setting, issues may arise, such as walls
Langer, 1985). In instances in which reports on the or columns that impede seeing the behavior and
occurrence of the target behavior are not available or interruptions from staff or other individuals. Disrup-
when the reports are not descriptive enough, the tions in the environment are also a concern. For
behavior analyst should err on the side of caution example, children in elementary school classrooms
and conduct a scatterplot assessment or other initial are notorious for approaching and interacting with
observations to narrow the observation window. an adult while he or she is observing and recording
Identifying the times at which the target behavior the target behavior. In addition, if the target child is
is most likely to occur is desirable to capture the identified, the other children may cause disruptions
greatest number of instances. The rationale for by talking to the target child or behaving in other-
observing and recording as many instances of the wise disruptive ways. Any disruption should be
behavior as possible rests with evaluation of func- recorded so that it can be considered in accounting
tion during assessment and analysis of the effects of for variability in the data.
the independent variable during intervention. When
it is not possible to observe enough instances of the Selecting Observers
behavior across a number of observation periods to Most behavioral observations in research and prac-
establish clear relations between the behavior and tice are conducted by trained observers. Trained
specific antecedents and consequences, treatment observers may not be needed in some laboratory set-
implementation may be delayed. With behavior that tings in which permanent products are produced or
occurs at a lower rate, a longer time frame of obser- equipment records the behavior automatically.
vation may be necessary to establish functional However, in applied settings, individuals conducting
relations. behavioral observations must see the behavior as it
Although a delay to intervention after a baseline occurs so data can be collected onsite or recorded
may be acceptable in some situations, in others it for review at a later time. Individuals who could
130
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
conduct behavioral observations in the same setting for surreptitious observation of a child or a
as the client include participant observers (individu- researcher gets institutional review board approval
als who are typically present in the environment), for deception and later debriefs the participants.
nonparticipant observers (data collectors who are
Nonparticipant observers. When it is either
external to the typical workings of the environ-
impossible or undesirable to involve a participant
ment), or the client whose behavior is being
observer, nonparticipant observers who are not part
observed (self-monitoring).
of the typical environment are used. For instance,
observations in school settings may require that an
Participant observers. According to Hayes,
observer sit in an unobtrusive area of the classroom
Barlow, and Nelson-Gray (1999), participant observ-
and conduct observations of the child at various
ers may be used in situations in which a significant
times of the day. Three challenges of having non-
other or other responsible party in the setting (e.g.,
participant observers involved in data collection are
parent, teacher) is available (and trained) to col-
access, scheduling, and cost. Because observations
lect data. The primary advantage of including these
tend to occur while clients are involved in social set-
individuals as observers is that they are already in
tings, it may not be permissible to observe because
the environment, which eliminates the potential
of the potential for disruption or for reasons of
logistical problems of scheduling nonparticipant
confidentiality. In the latter case, when conducting
observers. In addition, the likelihood of the child
observations of the behavior of a single individual in
or student showing reactivity to observation is less-
a group setting such as a classroom, it is typical to
ened because the person being observed is likely to
require consent of all students in the group because
have habituated to the participant observer’s pres-
all are present during the observation. This is espe-
ence over time. A limitation when using participant
cially true when observations of minors occur.
observers is that the observers may not have time to
Because observation periods may be relatively
conduct observations because of their other respon-
short (especially in the context of research), it may
sibilities in the setting.
also be difficult to schedule an observer several
One factor to consider when arranging partici-
times a day or week to collect data for only 15 to 60
pant observation (and nonparticipant observation)
minutes. In addition, the client’s schedule may
is the possibility of surreptitious observation. With
restrict when observation may occur. Finally, a sig-
surreptitious observation, the participant observer
nificant cost may be associated with the inclusion of
would not announce or otherwise cue the partici-
skilled data collectors who may need to be hired to
pant to the fact that a particular observation session
fulfill this role. Circumventing excessive costs is
is taking place (e.g., Mowery, Miltenberger, & Weil,
possible, however, if student interns or other staff
2010). For example, in Mowery et al. (2010), gradu-
already at the site are available. In addition to the
ate students were present in a group home setting to
monetary cost of the observers, there is cost in terms
record staff behavior. However, the staff members
of time and effort to train the observers and conduct
were told that the students were there to observe
checks for IOA to ensure consistency in the data
and record client behavior as part of a class project
collected.
(this deception was approved by the institutional
review board, and participants were later debriefed). Self-monitoring. When the target behavior occurs
Surreptitious observation leads to less reactivity in in the absence of others, it may be useful to have cli-
the person being observed. For surreptitious obser- ents observe and record their own behavior. When
vation to occur ethically, the client or participant asking clients to record their own behavior, it is nec-
must consent to observation with the knowledge essary to train them as you would train any observer.
that he or she will not be told, and may not be aware Although there are examples of research using data
of, the exact time and place of observation (e.g., gathered through self-monitoring (e.g., marijuana
Wright & Miltenberger, 1987). The exception use [Twohig, Shoenberger, & Hayes, 2007]; disrup-
would be when a parent or guardian gives consent tive outbursts during athletic performances [Allen,
131
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
1998]; physical activity levels [Van Wormer, 2004]; Even with these procedures in place, clients may
binge eating [Stickney & Miltenberger, 1999]), still engage in data collection with poor fidelity or
self-monitoring is less desirable than observation make up data in an attempt to please the therapist or
by another individual because it may be unreli- researcher. Therefore, self-monitoring that lacks
able. If the target behavior occurs in the absence of verification should be avoided as a form of data col-
others, then IOA cannot be assessed. For example, lection whenever possible.
Bosch, Miltenberger, Gross, Knudson, and Brower-
Breitweiser (2008) used self-monitoring to collect Training Observers
information on instances of binge eating by young Adequate observer training is necessary to have con-
women but could not collect IOA data because binge fidence in the data. Observing and recording behav-
eating occurred only when the individual was alone. ior can be a complex endeavor in which the observer
Self-monitoring is best used when it can be com- must record, simultaneously or in rapid order, a
bined with periodic independent observations to number of response classes following a specific pro-
assess IOA. Independent observations occur when a tocol, often while attending to timing cues (see Sam-
second observer records the same behavior at the pling Procedures section). Finally, following this
same time but has no knowledge of the other routine session after session may lead to boredom
observer’s recording. Thus, the recording of both and set the occasion for observer drift. Observer drift
observers is under the stimulus control of the behav- is the loosening of the observer’s adherence to the
ior being observed and is not influenced by the behavioral definitions that are used to identify the
recording behavior of the other observer. When IOA behavioral topographies to be recorded, a decrease
is high, it might indicate that self-monitoring is in attending to specific features of the data collec-
being conducted with fidelity. It is possible, how- tion system, or both. When observer drift occurs,
ever, that self-monitoring is conducted with fidelity the accuracy and reliability of the data suffer, and
only under the conditions of another observer being faulty decisions or conclusions may result (see
present, but not when the client is alone or away Kazdin, 1977).
from the other observer. In some instances, it is pos- One way to train observers is to use behavior
sible to collect secondary data or product measures skills training (Miltenberger, 2012), which involves
that can be used to verify self-monitoring. For providing instructions and modeling, having the
instance, researchers measured expired carbon mon- observer rehearse the observation and recording
oxide samples in smoking cessation research (Brown procedures, and providing feedback immediately
et al., 2008; Raiff, Faix, Turturici, & Dallery, 2010) after the performance. Such training occurs first
and tested urine samples in research on substance with simulated occurrences of the target behavior in
abuse (Hayes et al., 2004; Wong et al., 2003). the training setting and then with actual occurrences
Given the potential unreliability of self-monitoring, of the target behavior in the natural environment.
taking steps to produce the most accurate data pos- Subsequent booster sessions can be conducted in
sible through self-monitoring is important. Such which the necessary training components are used
steps might include making a data sheet or data col- to correct problems. To maintain adequate data col-
lection device as easy to use as possible, tying data lection, it is necessary to reinforce accurate data
collection to specific times or specific activities to collection and detect and correct errors that occur.
cue the client to record his or her behavior, having Several factors can influence the fidelity of data col-
other people in the client’s environment cue the cli- lection (Kazdin, 1977). These include the quality of
ent to conduct self-monitoring, checking with the initial training, consequences delivered for the target
client frequently by phone or e-mail to see whether behavior (Harris & Ciminero, 1978), feedback from a
self-monitoring is occurring, having the client sub- supervisor for accurate data collection (Mozingo,
mit data daily via e-mail or text message, and prais- Smith, Riordan, Reiss, & Bailey, 2006), complexity
ing the client for reporting data rather than for the and predictability of the behavior being observed
level of the behavior to avoid influencing the data. (Mash & McElwee, 1974), and the mere presence of a
132
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
supervisor (Mozingo et al., 2006). With these factors Frequency. Perhaps the most common form of
in mind, strong initial training and periodic assess- continuous recording is frequency recording: count-
ment and retraining of observers are recommended for ing the number of occurrences of the target behavior
participant observers, nonparticipant observers, and in the observation period (Mozingo et al., 2006).
individuals engaging in self-monitoring. Frequency recording is most appropriate when
the behavior occurs in discrete units with fairly
consistent durations. In frequency recording, each
Recording Procedures
occurrence of the target behavior (defined by the
The procedures available for collecting data on tar- onset and offset of the behavior) is recorded in the
geted behavior are categorized as continuous record- observation period. Frequency data may be reported
ing procedures, sampling procedures, and product as total frequency—number of responses per obser-
recording. vation session—or converted to rate—number of
responses per unit of time (e.g., responses per min-
Continuous Recording ute). Total frequency would only be reported if the
Continuous recording (also called event recording) observation periods were of the same duration over
procedures involve observation and recording of time. The advantage of reporting rate is that the
each behavioral event as it occurs during the obser- measure is equivalent across observation periods of
vation period. Continuous recording will produce different durations.
the most precise measure of the behavior because Frequency recording requires the identification of
every occurrence is recorded. However, continuous a clear onset and offset of the target behavior so each
recording is also the most laborious method because instance can be counted. It has been used with a
the observer must have constant contact with the wide range of target behavior when the number of
participant’s behavior throughout the observation responses is the most important characteristic of the
period. As with all forms of data collection, continu- behavior. Examples include recording the frequency
ous recording requires the behavior analyst to first of tics (Miltenberger, Woods, & Himle, 2007), greet-
identify the dimensions on which to focus. Observ- ings (Therrien, Wilder, Rodriguez, & Wine, 2005),
ers are recommended to initially collect data on requests (Marckel, Neef, & Ferreri, 2006), and math-
multiple dimensions of the behavior (e.g., frequency ematics problems completed (Mayfield & Vollmer,
and duration) to identify the most relevant dimen- 2007). When it is difficult to discriminate the onset
sions and to then wean over the course of observa- or offset of the behavior or the behavior occurs at
tions as the analysis identifies the most important high rates such that instances of the behavior cannot
dimensions. For example, in a classroom situation be counted accurately (e.g., high-frequency tics or
involving academic performance, it may be useful to stereotypic behavior), a behavior sampling procedure
count the number of math problems completed cor- (i.e., interval or time-sample recording; see below) is
rectly, latency to initiate the task (and each prob- a more appropriate recording procedure. As we elab-
lem), and the time spent on each problem. If after orate on later, in sampling procedures the behavior is
several observations the observer finds that it takes recorded as occurring or not occurring within con-
a while for the child to initiate the task, resulting secutive or nonconsecutive intervals of time, but
in a low number of problems completed, focusing individual responses are not counted.
on measuring latency to initiate the task and fre- Four additional methods of recording frequency
quency of correct responses may be useful. Next, we are frequency-within-interval recording, real-time
describe data collection procedures related to differ- recording, cumulative frequency, and percentage of
ent dimensions of behavior. Although we discuss opportunities. Each method has advantages over a
the procedures separately, various combinations of straight frequency count.
these procedures may produce important data for
analysis that would not be apparent with a focus on Frequency-within-interval recording. One limi-
a single procedure. tation of frequency recording is that it does not
133
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
provide information on the timing of the responses Cumulative frequency. The origins of measuring
in the observation period. With frequency-within- operant behavior involved the use of an electrome-
interval recording, the frequency of the behavior is chanical data recording procedure that was designed
recorded within consecutive intervals of time to to record instances of behavior cumulatively across
indicate when the behavior occurred within the time (Skinner, 1956). Each response produced an
observation period. To conduct frequency-within- uptick in the data path as the pen moved across
interval recording, the data sheet is divided into con- the paper and the paper revolved around a drum.
secutive intervals, a timing device cues the observer The original paper records of this recording were
to the appropriate interval, and the observer records only about 6 inches wide, and thus the pen used to
each occurrence of the behavior in the appropriate record responses would, on reaching the top of the
interval. By providing information on the number of paper, reset to the bottom of the paper and continue
responses and the timing of responses, more precise recording the responses. Increasing slopes indicated
measures of IOA can be calculated. higher response rates; horizontal lines indicated an
absence of the response.
Real-time recording. Combining features of fre- This apparatus for the automatic recording of
quency and duration procedures, real-time recording cumulative frequencies is no longer used, but the
also allows the researcher to collect information usefulness of cumulative response measures persists.
on the temporal distribution of the target behavior In cumulative frequency graphs, data are displayed
over the course of an observation period (Kahng & as a function of time across the x-axis and cumula-
Iwata, 1998; Miltenberger, Rapp, & Long, 1999). tive frequency along the y-axis. The frequency of
Through use of either video playback or computers responses that occur in a given time period is added
in real time, it is possible to record the exact time of to the frequency in the previous time period. Thus,
onset and offset of each occurrence of the behavior. data presented in a cumulative record must either
For discrete momentary responses that occur for 1 maintain at a particular level (no new responses) or
second or less, the onset and offset are recorded in increase (new responses) across time but never
the same second. decrease. The use of cumulative frequency plots
Real-time recording is especially valuable when allows one to assess frequency and temporal pat-
conducting within-session analysis of behavioral terns of responding.
sequences or antecedent–behavior–consequence rela-
tions. Borrero and Borrero (2008) conducted real- Percentage of opportunities. In some cases,
time observations that included the recording of both recording the occurrence of a response in relation
target behavior and precursor behavior or events to a specific event or response opportunity is use-
related to the target behavior. These data were then ful. In such cases, the percentage of opportunities
used to construct a moment-to-moment analysis (lag- with correct responses is more important than the
sequential analysis) that provided probability values number of responses that occurred. For example, in
for the occurrence of the precursor given the target recording compliance with adult requests, the per-
behavior and of the target behavior given the precur- centage of requests to which the child responds cor-
sor. The probability of a precursor reliably increased rectly is more important than the number of correct
approximately 1 second before the emission of the responses. Ten instances of compliance are desirable
target behavior. In addition, the probability of the tar- if 10 opportunities occur. However, 10 instances
get behavior was greatest within 1 second after the of compliance are much less desirable in relation
precursor behavior or event. The real-time analysis to 30 opportunities. Other examples include the
suggested that the precursor behavior or event was a percentage of math problems completed correctly,
reliable predictor of the target behavior. Additional percentage of free throws made in a basketball game,
analysis showed that both the precursor behavior and percentage of signals detected on a radar screen dur-
the target behavior served the same function (e.g., ing a training exercise, and percentage of trials in
both led to escape from demands). which an item is labeled correctly during language
134
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
training. Considering that the number of opportuni- continues for a number of minutes, recording both
ties might vary in each of these cases, the percentage frequency and duration will reveal whether tan-
of opportunities is a more sensitive measure of the trums are occurring less often and occurring for
behavior than a simple frequency count or a rate shorter periods of time after intervention.
measure. Finally, many types of behavior targeted in
When a percentage-of-opportunities measure is applied work do not lend themselves readily to fre-
used, reporting the number of opportunities as well quency counts because they consist of (a) responses
as the percentage of correct responses is important. that occur rapidly and repetitively over extended
If the number of opportunities is substantively dif- periods of time (such as stereotypic behavior),
ferent across observations, it may affect the variabil- (b) complexes of discrete responses integrated into
ity of the data and the interpretation of the results. chains or other higher order units, or (c) both. For
For instance, if on one occasion a child is provided rapid, repetitive responses, for which each onset and
with 10 spelling words and spells eight correctly, the offset is not easily discriminated, a duration measure
result is 80% correct. The next day, if the two words can be used. In such cases, a time period in which
spelled incorrectly are retested and the child spells the behavior is absent can help the observer discrim-
one of the words correctly, the second performance inate the end of one episode and the start of the next.
result is 50% correct. These data are not comparable For target behavior consisting of multiple compo-
because the number of opportunities varied greatly, nent behaviors, the target behavior might be defined
and inappropriate conclusions could be drawn from as the entire chain, and a duration measure would
the results if only percentages were reported. In then consist of recording the time from the onset of
these instances, providing an indicator of the num- the first response in the chain to the offset of the last
ber of opportunities to respond in the graphical rep- response in the chain. Finally, in some instances
resentation of the data will assist the reader in duration is used to measure a behavior with multiple
interpreting the results. component responses when it does not make sense
to reduce the behavior to a frequency count of its
Duration. When each response lasts for an component responses. For example, duration of play
extended period of time or does not lend itself to a would be of greater interest to a parent trying to
frequency count (e.g., as in behavior such as read- increase a child’s play time than would a frequency
ing or play), it may be useful to record the duration count of the number of steps the child traveled
of the behavior, that is, the time from its onset to across the playground, went up and down a slide, or
its offset. Duration recording is desirable when the moved back and forth on a swing.
most important aspect of the behavior is the length
of time for which it occurs. For example, if the Latency. Latency is the length of time from the pre-
interest were in sustained performance or time on sentation of a discriminative stimulus to the initia-
task, duration recording is appropriate. If classroom tion of the behavior. Latency is of interest when the
teachers are concerned with sustained engagement speed of initiation of the behavior is an important
in academic activities, the observer would identify feature. For example, latency is the time from the
the length of time that engagement is desired (such sound of the starter’s pistol to the sprinter’s move-
as in reading) and collect data on the duration of ment off the starting blocks, the time it takes for
engagement to identify any discrepancy between a child to initiate a task after the teacher’s request,
the target duration and actual performance. Once a or the time it takes the wait staff at a restaurant to
discrepancy is determined to exist, programming for respond once a customer is seated.
successively longer durations could be initiated. When working with a child who does not com-
Other situations involve a combination of dura- plete math problems in the allotted time, for exam-
tion and frequency recording, as when the goal is to ple, latency indicates how long it takes the child to
decrease a young child’s tantrum behavior. If tan- initiate the task. By contrast, duration assesses how
trums occur multiple times per day and each tantrum long it takes the child to complete each problem
135
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
once he or she starts working on it. Depending on Varra, & Hayes, 2005) or other emotional responses
the child and the circumstance, one or both dimen- (Stickney & Miltenberger, 1999). In general, inten-
sions may be an important focus of assessment and sity rating scales present issues of both reliability
intervention. and validity because the ratings that might be
assigned to specific instances of behavior may be
Magnitude. On occasion, evaluating the magni- ambiguous; this is especially true when rating fear
tude or intensity of behavior is useful. One example or emotional responses because the magnitude of
of response magnitude is the force exerted (e.g., these behaviors can be rated only by the individual
muscle flexion), and another is the loudness of a engaging in the behavior.
verbal response (as measured in decibels). Although
decreases in frequency, and perhaps duration, of Sampling Procedures
undesirable behaviors such as tantrums or self- It may not always be possible to collect adequate
injury are important, a reduction in magnitude may information on the target behavior using continuous
be an important initial goal. In some cases, reduc- recording procedures. When the onset and offset of
tions in magnitude may be observed before substan- each instance of the behavior cannot be identified,
tial decreases occur on other dimensions, such as continuous recording is not possible. Likewise, the
frequency and duration. Alternatively, magnitude constraints imposed by some environments, some
may increase temporarily during an extinction burst target behaviors, or some observers may make con-
before the behavior decreases in frequency, duration, tinuous recording impossible. For example, the
or intensity. person exhibiting the target behavior might not be
Recording magnitude may be valuable when continuously in sight of the observer, the target
considering recovery from an accident or injury behavior might occur almost exclusively when the
such as a knee injury for a football player. Measure- individual is alone, or the observer might have other
ment would pertain to the ability of the affected responsibilities that compete with observation. In
muscles to exert force after rehabilitation. In these these instances, it may be desirable, and necessary,
situations, recording magnitude tends to require to collect samples of the behavior that provide an
equipment such as that used by physical therapists estimate of the behavior’s true level. Behavior-
to evaluate force. sampling procedures include interval recording and
Direct observation of response magnitude may time-sample recording. In both procedures, the
not always measure force, however. Observers can observation period is divided into smaller units of
use intensity rating scales to evaluate the magnitude time, and the observer records whether the behavior
of a response. For instance, given a scale ranging occurred in each interval.
from 1 to 10, a teacher may rate the intensity of each
occurrence of an undesirable behavior. In using rat- Interval recording. Interval recording involves
ing scales, it is important to anchor the points dividing the observation period into equal consecu-
within the scale such that two observers can agree tive intervals and recording whether the behavior
on the level of intensity given a variety of occur- occurred in each. Interval recording is different
rences of the behavior (e.g., 1 = mild whining, 10 = than frequency recording or frequency-within-
loud screaming, throwing items, and head banging). interval recording in that an interval is scored once
Although anchoring categories on a scale is consid- regardless of whether a single instance or multiple
ered valuable to decrease the variability in respond- instances of a behavior occurred during the interval.
ing across observers, the literature is not clear as to In behavior analysis research, intervals are usually
how many individual categories need be defined short—typically 10 to 15 seconds (DiGennaro et al.,
(Pedhazur & Schmelkin, 1991). Another example in 2007; Mace et al., 2009). Short intervals (usually
which magnitude can be measured with a rating less than 20 seconds) are valuable when behavior
scale is the intensity of a fear response (Milten- occurs at moderate to high frequencies or when
berger, Wright, & Fuqua, 1986; Twohig, Masuda, multiple topographies of behavior are recorded.
136
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Valuable too are shorter intervals when temporal divided by the number of observation intervals, and
correlations may yield information on antecedent the result is reported as the percentage of intervals
events and potential maintaining consequences. in which the behavior occurred. A similar process is
When interested in the relation between the target used for time-sample recording (described in the
behavior and antecedents and consequences, the next section).
observer records whether any of the three events The two types of interval recording procedures
occurred in each interval to examine the temporal are partial-interval recording and whole-interval
patterns of the behavior and its potential controlling recording. In partial-interval recording, the observer
variables (e.g., Repp & Karsh, 1994). An additional records the occurrence of the target behavior if it
condition under which shorter intervals are valuable occurred at any time within the interval. That is, the
is when an understanding of within-session tem- interval is scored if the target behavior occurred
poral distribution of the behavior is necessary. For briefly in the interval or throughout the entire inter-
example, to determine whether self-injurious behav- val. Furthermore, if the onset of the behavior occurs
ior is high in a functional analysis condition because in one interval and its offset occurs in the next, both
of an extinction burst, the behavior analyst identi- intervals are scored (e.g., Meidinger et al., 2005).
fies whether more intervals were scored early in the In whole-interval recording, the interval is scored
session than later in the session (Vollmer, Marcus, only if the target behavior occurred throughout the
Ringdahl, & Roane, 1995). Similar patterns could be entire interval. Whole-interval recording is more
discerned with cumulative frequency recording or useful with continuous behavior (e.g., play) than
real-time recording as well. with discrete or quickly occurring behavior (e.g., a
In some applied settings, however, intervals face slap). Typically, whole-interval recording is
might be much longer—perhaps 15 or 30 minutes used when a behavior occurs over longer periods of
(Aikman & Garbutt, 2003)—when behavior occurs time, as might be seen with noncompliant behavior
less frequently. Under these conditions, it may be or on-task behavior. For example, Athens, Vollmer,
difficult to draw useful correlations between ante- and St. Peter Pipkin (2007) recorded duration of on-
cedent and consequent events and the behavior as task behavior in 3-second intervals only if the
well as behavior–behavior relations. Such limita- behavior was present for the entire interval.
tions notwithstanding, longer intervals are typically
used for the convenience of data collectors (often Time-sample recording. In time-sample record-
participant observers) who can engage in other ing, the observation period is divided into intervals
responsibilities and still collect data. of time, but observation intervals are separated by
Typically, the observer has a data sheet with con- periods without observation. Time-sample recording
secutive intervals designated for recording, and dur- permits the observer to focus on other tasks when
ing the observation period, the observer is prompted not observing the target behavior. For example, the
with auditory (through headphones so as to not dis- observation period might be divided into 15-second
rupt the ongoing behavior of the observee) or tactile intervals, but observation occurs only at the end of
(vibration) cues delivered via a timing device to the interval. Likewise, an observation period might
move from interval to interval while observing and be divided into 30-minute intervals, but observa-
recording the target behavior. As time passes, the tion and recording occur only in the last 5 minutes
observer records the occurrence of the target behav- of every 30 minutes. These intervals can be equally
ior in the appropriate interval; a blank interval indi- divided, as when an observation occurs every 15
cates the nonoccurrence of the behavior in that minutes, or variably divided to provide some flexi-
interval. In some cases, a computer is used for data bility for the observer (such as a teacher who cannot
collection, and as the observer records the behavior, observe exactly on the quarter hour). The data are
the software puts the data into the proper interval. displayed as a percentage (the number of intervals
At the end of the observation period, the number of with target behavior divided by the number of inter-
intervals in which the behavior is observed is vals of observation).
137
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
For instance, if evaluating the extent of social and interval recording with small intervals tends to
interactions between adolescents on a wing of an produce fairly accurate estimates of frequency.
inpatient psychiatric facility were desirable, con- Although Rapp et al. provided several suggestions to
ducting observations every 15 minutes might be guide decision making regarding the use of interval
possible. In this example, the observer would be and time-sample procedures, they concluded by
engaged in a job-related activity and, when suggesting that small interval sizes in interval and
prompted by a timer, look up from his or her work, time-sample procedures are likely to produce the
note whether the target behavior was occurring, and best results.
record the result. Data of this sort could identify
which of the adolescents tend to engage in social Product Recording
interactions and the typical times at which social In some cases, the outcome or the product of the
interactions are likely to occur. From this sampling behavior may be of interest, either as a primary
approach, it is possible to refine the data collection dependent variable or as a complement to direct
process toward a more precise measure of behavior. observation of the behavior itself. When the behav-
Interval and time-sample recording have benefits ior changes the physical environment, this product
and limitations. The benefit of interval recording is can be recorded as an index of the occurrence of the
that with consecutive observation intervals, no behavior. In some instances, collecting data on
instance of the target behavior is missed during the products is valuable because measuring the behavior
observation period. The limitation, especially with directly may not be possible. For example, weight is
shorter intervals, is that it requires the continu- measured in weight-loss programs because measur-
ous attention of, and frequent recording by, the ing the behavior that produces weight loss (i.e., diet
observer, making it difficult for the observer to and exercise) is usually not feasible. Examples of
engage in other activities during the observation product recording may include number of academic
period. A limitation of time-sample recording is that problems completed or number of units assembled
because observation intervals are separated by peri- in a factory. In each case, the occurrence of the
ods without observation, some instances of the tar- behavior is not observed directly; rather, the prod-
get behavior may be missed during the observation ucts of the behavior are recorded as an indication of
period. However, a benefit is that the observer can its occurrence. In such cases, a focus on the prod-
engage in other activities during the periods ucts of behavior is easier and more efficient than
between observation intervals, making the proce- recording the behavioral events as they occur. An
dure more user friendly for participant observers important note in recording permanent products is
such as teachers or parents. that although the focus is on results, if the results
Although interval and time-sample recording fall short of expected quantity or quality, the focus
procedures are used widely in behavior-analytic can then turn to evaluation of the behavior involved
research, some authors have cautioned that the in producing the products being measured (Daniels
results of these sampling procedures might not & Daniels, 2004).
always correspond highly with data collected Beyond measuring the production of tangible
through continuous recording procedures in which items, product recording can be used to measure the
every behavioral event is recorded (e.g., Rapp et al., physical damage caused by a problem behavior. For
2007; Rapp, Colby-Dirksen, Michalski, Carroll, & example, self-injurious behavior can produce tissue
Lindenberg, 2008). In summarizing the numerous damage such as bruises, lacerations, or other bodily
studies that have compared interval and time- injuries, and product recording could be used to
sample recording with continuous recording proce- assess the severity of these injuries. Iwata, Pace,
dures, Rapp et al. (2008) concluded that interval Kissel, Nau, and Farber (1990) developed the Self-
recording tends to overestimate the duration of the Injury Trauma Scale to quantify surface injury
behavior, time-sample procedures with small inter- resulting from self-injurious behavior. Other exam-
vals tend to produce accurate estimates of duration, ples of this type of product recording include the
138
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
assessment of the size of a bald area related to of the behavior as they occur, the observer’s behav-
chronic hair pulling (Rapp, Miltenberger, & Long, ior must result in a product that can be used later
1998; Rapp, Miltenberger, Long, Elliott, & Lumley, for purposes of analysis. A recording device allows
1998) or the length of fingernails as a measure of the observer to produce a permanent product from
nail biting (Flessner et al., 2005; Long, Milten- the observation session. The most commonly used
berger, Ellingson, & Ott, 1999). Still other examples recording device is a data sheet structured for the
of product recording include a measure of weight or type of recording procedure being conducted.
body mass index as an indication of changes in eat- Figures 6.1, 6.2, and 6.3 show sample data sheets
ing (Donaldson & Normand, 2009; see also Young structured for frequency recording, duration record-
et al., 2006), measuring chemicals in urine samples ing, and interval recording, respectively.
as a measure of drug ingestion (Silverman et al., Although data sheets are used most often for data
2007), or weighing food before and after a feeding collection, other recording devices, both low tech
session to assess the amount of food consumed and high tech, can be used to record instances of
(Kerwin, Ahearn, Eicher, & Swearingin, 1998; the behavior. Several types of low-tech recording
Maglieri, DeLeon, Rodriguez-Catter, & Sevin, 2000; devices have been used, such as wrist counters
Patel, Piazza, Layer, Colemen, & Swartzwelder, 2005). for frequency recording (Lindsley, 1968) or stop
An advantage of product recording is that the watches for duration recording. Still other possibili-
observer does not have to be present to record the ties include activities as simple as moving beads
occurrence of the behavior (Miltenberger, 2012) from one side of a string to the other, placing a coin
because the product can be recorded at a more con- from one pocket to another, making small tears in a
venient time after the behavior has occurred (e.g., at piece of paper, or making a hash mark on a piece
the end of the class period or after the shift in a fac- of masking tape affixed to the recorder’s sleeve to
tory). A drawback of product recording, especially record frequency (Miltenberger, 2012). In fact, it is
when used with a group of individuals, is that it feasible to record frequency with whatever may be
might not be possible to determine which person available in the environment as long as the observer
engaged in the behavior that resulted in the product. can produce a product that represents the occur-
Perhaps another student completed the academic rence of the behavior.
problems or another worker helped produce the Although recording on a data sheet is the most
units in the factory (Jessup & Stahelski, 1999). frequently used data collection process, with rapidly
Although product recording is valuable when the changing technologies there is a move to identify
interest is in the tangible outcome of the behavior, high-tech methods to streamline and automate data
there must be some way to determine which indi- collection (Connell & Witt, 2004; Jackson & Dixon,
vidual was responsible for the products being mea- 2007; Kahng & Iwata, 1998). In applied behavior
sured (e.g., did the urine sample come from the analysis research, electronic devices such as a per-
client or someone else?). Another potential problem sonal digital assistant (Fogel, Miltenberger, Graves,
with some uses of product recording is that it may & Koehler, 2010) or hand-held or laptop computers
not identify the behavior that resulted in the prod- (Gravlee, Zenk, Woods, Rowe, & Schulz, 2006;
uct. For example, correct answers to math problems Kahng & Iwata, 1998; Repp, Karsh, Felce, &
may have been produced by cheating, and weight Ludewig, 1989) are frequently used for data collec-
loss may have been produced through self-induced tion. In addition, the use of bar codes and scanners
vomiting rather than an increase in exercise or a (Saunders, Saunders, & Saunders, 1993) for data
reduction in calorie consumption. collection has been reported. With bar code scan-
ners, an observer holds a small battery-powered
Recording Devices scanning device and a sheet of paper with the bar
Once the appropriate recording procedure has been codes ordered according to behavioral topography.
chosen, the next step is to choose a recording When the target behavior is observed, the data col-
device. Because the observer must record instances lector scans the relevant bar code to record the
139
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Frequency Data Form

Child: James M. Start Date: 9/15/2010 Setting: Mrs. Johnson’s Class
Observer: R.M. Primary/Reliability
Instructions: First, indicate date of observation in the far left column. Second, place a tick
mark for each occurrence of behavior during the specified academic activity for that day.
Definition of behavior:_________________________________________________________
____________________________________________________________________________
Circle Individual Social Daily

Date Mathematics Science Writing
Time Reading Studies Total
Figure 6.1. Example of a daily frequency data sheet that involves a break-
down of the frequency of the behavior by curricular areas in a general education
classroom setting.
Duration Data Form

Child: James M. Start Date: 9/15/2010 Setting: Mathematics
Instructions: First, indicate date of observation in the far left column. Second, identify the
start time (onset) and the stop time (offset) for each occurrence of the behavior. Use more than
one line if necessary.
____________________________________________________________________________
Daily
Date Onset Offset Onset Offset Onset Offset Onset Offset
Duration
Figure 6.2. An example of a duration data sheet that provides information

on the onset and offset of each occurrence of the behavior and the frequency of
the behavior each day.
occurrence of the behavior and the time within the vault (Scott, Scott, & Goldwater, 1997). Another
observation period. The use of bar codes is, how- high-tech method of data collection involves soft-
ever, only one of several ways to conduct electronic ware for cell phones. These software applications,
recording of behavior. colloquially referred to as apps, allow behavior ana-
In one investigation evaluating a shaping proce- lysts to use the computing power of their phones
dure to increase the reach of a pole vaulter, a photo- for data collection. The advantages of this technol-
electric beam was used to determine the height of ogy are numerous; however, the most obvious bene-
the vaulter’s reach just after planting the pole for the fits are the use of a small, portable device that can
140
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Interval Data Form

Child: James M. Date: 9/15/2010 Setting: Mathematics
Instructions: Place a check mark in the appropriate column to reflect the events that occurred
in each 10–s interval.
____________________________________________________________________________
10-s Demand
Aggression Attention Ignore Escape
Intervals Placed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Figure 6.3. An example of a 10-second interval data sheet (partial or

whole) that provides information on the occurrence of the target behavior
and probable antecedents and consequences. In this example, the hypothesis
is that the aggressive behavior occurs after the delivery of a demand by the
teacher. In addition, potential responses by the teacher to the problem behav-
ior are included. When completed, this data sheet will provide information
on the temporal relationship between teacher behavior and student problem
behavior.
facilitate any form of data collection mentioned thus desired direction (e.g., Brackett, Reid, & Green,
far and the ability to graph the data. Finally, these 2007; Mowery et al., 2010). Several researchers have
graphs can be sent via text message to parents, evaluated the effects of staff reactivity to observa-
teachers, or colleagues (Maher, 2009). Undoubtedly, tions (Boyce & Geller, 2001; Brackett et al., 2007;
as technology advances, even more high-tech data Codding, Livanis, Pace, & Vaca, 2008; Mowery
collection methods will emerge. et al., 2010). Mowery et al. (2010) evaluated staff
adherence to a protocol designed to increase the fre-
Reactivity of Observation quency of staff’s positive social initiations during lei-
A long-standing concern for behavioral researchers sure activities with adults with developmental
is how observation affects performance (e.g., Par- disabilities. They evaluated the effects on staff
sons, 1974). Reactivity is the term used to describe behavior of having a supervisor absent or a supervi-
changes in behavior resulting from the act of observ- sor present in the environment to determine
ing and recording the behavior. Typically, when whether reactivity to the supervisor’s presence
reactivity occurs, the behavior changes in the would occur. Positive social interactions only
141
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
occurred at acceptable levels when the supervisor representative of typical levels (the level expected in
was present, suggesting that reactivity is an impor- the absence of observation). Confederates can be
tant issue to consider in the valid assessment of staff any variety of individuals such as a coworker, class-
performance. mate, spouse, or person external to the setting as
Considering reactivity to observation in research seen in Mowery et al. (2010), in which observers
or clinical settings is important because the target were introduced as student social workers who were
behavior may be influenced not only by the inter- in the setting to observe individuals with intellectual
vention but also by the act of observing. When disabilities.
responding comes under the stimulus control of an In recent research on abduction prevention, chil-
observer, the level of behavior in the presence of the dren were observed without their knowledge to
observer is likely to be different than in the absence assess their safety skills as a confederate approached
of the observer. Considering that most staff behavior and attempted to lure them in a store setting (Beck
must be performed in the absence of supervision, & Miltenberger, 2009). Research on child safety
conducting observations without reactivity to obtain skills training has demonstrated that children are
an accurate characterization of the behavior is more likely to respond correctly when they are
important. There are a variety of ways in which to aware of observation than when they are not aware
minimize reactivity. For instance, making observers of observation (Gatheridge et al., 2004; Himle,
a part of the regular environment for several ses- Miltenberger, Gatheridge, & Flessner, 2004). An
sions or weeks before actual data collection occurs important caveat is that the use of confederates may
may result in habituation to the presence of the raise ethical concerns and should be approached
observers. It is important to keep in mind that habit- cautiously. The use of video or other inconspicuous
uation to the observer is only likely to occur as long monitoring systems may present the same ethical
as the observer does not interact with the person concerns and thus should also be approached with
being observed and no consequences are delivered caution. Prior approval is needed when using decep-
by the observer or others who may be associated tive covert observation. Additionally, for research
with the observer. purposes such covert observation must be approved
If the setting permits, reactivity may be avoided by an institutional review board with appropriate
in other ways. Video monitoring devices mounted debriefing afterward.
unobtrusively in the setting may be used. In
instances in which the cameras can be seen by the
Interobserver Agreement
person being observed, habituation to the presence
of the camera is likely to occur in the absence of Within research and practice in applied behavior
feedback or consequences contingent on perfor- analysis, accurate data collection is important (see
mance (e.g., Rapp, Miltenberger, Long, et al., 1998). Chapter 7, this volume). Accuracy refers to the
In addition, using an observation room equipped extent to which the recorded level of the behavior
with a one-way observation window provides the matches the true level of the behavior (Cooper et al.,
observer an opportunity to conduct unannounced 2007; Johnston & Pennypacker, 1993; Kazdin,
observations. If the use of an observation window is 1977). To evaluate accuracy, a researcher must be
not possible, the use of confederates may be consid- able to obtain a measure of the true level of the
ered. Although confederates are present in the set- behavior to compare with the measurement of the
ting to collect data (unobtrusively), a benign behavior produced by the observer. The difficulty
purpose for their presence other than data collection arises in obtaining the true level of the behavior, as
is provided to those being observed. That is, decep- most recording is done by humans who must dis-
tion is used to conceal the true purpose of the criminate the occurrence of the behavior from non-
observers’ presence. As mentioned in the Mowery occurrences (a stimulus–control problem). A “truer”
et al. (2010) study, confederates may be used to level of the behavior may be obtained through
increase the chances that the data collected are mechanical means, but equipment also may fail
142
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
on occasion and produce errors. Alternatively, perspective on the acceptable level of IOA and sug-
automated recording devices may fail to register gested that the
responses that vary slightly in topography or loca-
level of agreement that is acceptable is
tion. Thus, knowing the true level of the behavior
one that indicates to the researcher that
with certainty is impossible, and therefore accuracy
the observers are sufficiently consistent
is not measured in behavioral research.
in their recording of the behavior, that
Instead, behavior analysts train observers so that
the behaviors are adequately defined,
the data they collect are in agreement with those
and that the measures will be sensitive to
collected by another observer who has received
change in the client’s performance over
training in recording the target behavior. Although
time. (p. 118)
measuring agreement between observers provides
no information about the accuracy of either set of Kazdin suggested that the number and complexity
observations, it does improve the believability of the of behaviors being recorded, possible sources of
data. That is, when two independent observers agree bias, expected level of change in the behavior, and
on every occurrence and nonoccurrence of behavior, method of computing IOA are all considerations in
one has more confidence that they are using the deciding on an acceptable level of IOA. For example,
same definition of the target behavior, observing and if small changes in behavior are likely with the inter-
recording the same responses, and marking the form vention, then higher IOA would be demanded.
correctly (Miltenberger, 2012). If a valid definition However, if larger changes are expected, then lower
of the behavior of interest is being used, then high levels of IOA might be tolerated. The bottom line is
agreement scores increase the belief that the behav- that behavior analysts should strive for levels of IOA
ior has been recorded accurately; although, again, as high as possible (e.g., 90% or more) but consider
accuracy has not been measured. the factors that might contribute to lower levels and
A frequently used measure of agreement between make adjustments as warranted by these factors.
observers is simply the percentage of observations IOA can be calculated in a variety of ways. How it
that agree, a measure commonly referred to as IOA. is computed depends on the dimension of the behav-
IOA is calculated by dividing the number of agree- ior being evaluated and how it is measured. Next we
ments (both observers recorded the occurrence or describe common methods for calculating IOA.
nonoccurrence of the behavior) by the number of
agreements plus disagreements (one observer Frequency Recording
recorded the occurrence of the behavior and the To calculate IOA on frequency recording, the
other recorded the nonoccurrence of the behavior) smaller frequency is divided by the larger frequency.
and multiplying the quotient by 100. For an ade- For example, if one observer records 40 occurrences
quate assessment of IOA, two independent data col- of a target behavior and a second independent
lectors are recommended to be present during at observer records 35, the percentage of IOA during
least one third of all observation sessions across all that observation session is 35/40, or 87.5%. The lim-
participants and phases of a clinical intervention or itation of IOA in frequency recording is that there is
research study (Cooper et al., 2007). This level of no evidence that the two observers recorded the
IOA assessment (one third of sessions) is an arbi- same behavioral event even when IOA is high. For
trary number, and efforts to maximize the number example, if one observer recorded nine instances of
of assessments that produce strong percentages of the behavior and the other observer recorded 10
agreement should result in greater confidence in instances, the two observers ostensibly agreed on
the data. nine of the 10 instances for an IOA of 90%. It is pos-
Cooper et al. (2007) suggested that research sible, however, that the observers were actually
studies maintain 90% or higher IOA but agreed recording different instances of the behavior. One
that 80% or higher may be acceptable under some way to increase confidence that the two observers
circumstances. Kazdin (2010) offered a different were agreeing on specific responses in frequency
143
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
IOA is to collect frequency data in intervals and there is no evidence that the two observers were
then compare the frequency in each interval (see recording the same instances of the behavior. Real-
Frequency Within Interval section later in this time recording is a way to overcome this problem.
chapter). Dividing the observation period into
shorter, equal intervals permits a closer look at the Interval and Time-Sample Recording
recording of frequencies in shorter time blocks. In Computing IOA with interval data requires an interval-
this way, there can be more confidence that the by-interval check for agreement on the occurrence
observers recorded the same instances of the behav- and nonoccurrence of the behavior throughout the
ior when agreement is high. To further enhance observation period. The number of intervals of agree-
confidence that observers are recording the same ment is then divided by the number of intervals in the
behavioral events, it is possible to collect data on observation period to produce a percentage of agree-
behavior as it occurs in real time. With real-time ment. An agreement is defined as an interval in which
recording, it is possible to determine whether there both observers had a marked interval (indicating that
is exact agreement on each instance of the behavior. the behavior occurred) or an unmarked interval (indi-
cating that the behavior did not occur). Using only
Real-Time Recording one target behavior for this example, consider a
When using real-time recording, the onset and offset 10-minute observation session with data recorded at
of the behavior are recorded on a second-by-second 10-second intervals (60 intervals total). If the number
basis. Therefore, IOA can be calculated by dividing of intervals of observation with agreements is 56 of
the number of seconds in which the two observers 60, the percentage of IOA is 56/60, or 93.3%.
agreed that the behavior was or was not occurring Two variations of IOA calculations for interval
by the number of seconds in the observation ses- recording, which correct for chance agreement with
sion. Typically, an agreement on the onset and offset low-rate and high-rate behavior, are occurrence-
of the behavior can be defined as occurring when only and nonoccurrence-only calculations. The
both observers recorded the onset or offset at exactly occurrence-only calculation is used with low-rate
the same second. This form of IOA is the most strin- behavior (from which chance agreement on nonoc-
gent because agreement is calculated for every sec- currence is high) and involves calculating IOA using
ond of the observation period (e.g., Rapp, only agreements on occurrence and removing agree-
Miltenberger, & Long, 1998; Rapp, Miltenberger, ments on nonoccurrence from consideration (agree-
Long, et al., 1998). Alternatively, IOA could be con- ments on occurrence divided by agreements plus
ducted on the frequency of the behavior, but an disagreements on occurrence). The nonoccurrence-
agreement would only be scored when both observ- only calculation is used with high-rate behavior
ers recorded the onset of the behavior at the same (from which chance agreement on occurrence is
instant or within a small window of time (e.g., high) and involves calculation of IOA using only
within 1 or 2 seconds of each other). agreements on nonoccurrence and removing agree-
ments on occurrence from consideration (agree-
Duration Recording ments on nonoccurrence divided by agreements
IOA for duration recording is calculated by dividing plus disagreements on nonoccurrence).
the smaller duration by the larger duration. For
example, if one observer records 90 minutes of Cohen’s Kappa
break time taken in an 8-hour shift and the reliabil- Kappa is another method of calculating observer
ity observer records 85 minutes, the agreement agreement, but it corrects for the probability that
between observers is 85/90, or 94.4%. The same lim- two observers will agree as a result of chance alone.
itation described earlier for IOA on frequency Kappa is computed using the following formula:
recording pertains to IOA on duration recording.
Po − Pc
Although the duration recorded by the two observ- = ,
ers may be similar, unless the data are time stamped, 1 − Pc
144
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
where Po is the proportion of agreement between Kappa can be affected by other factors, including
observers (sum of agreements on occurrences and the distribution of the ratings of the observers (Sim
nonoccurrences divided by the total number of & Wright, 2005). Because these latter factors have
intervals) and Pc is the proportion of agreement little to do with the degree to which two observers
expected by chance. The latter may be obtained agree, there is little agreement on what constitutes
using the following formula: an acceptable kappa value. Within the social sci-
ences, Kazdin (2010) suggested that a kappa value
 ( O1o )( O2o )  +  ( O1n )( O2 n )  of 0.7 or higher reflects an acceptable level of agree-
Pc =  , ment. Lower criteria for acceptable kappa values
I2
may be found (e.g., Fleiss, 1981; Landis & Koch,
where O1o is the number of occurrences recorded by 1977), but these are as arbitrary as the cutoff sug-
Observer 1, O2o is the number of occurrences gested by Kazdin (von Eye & von Eye, 2008). Per-
recorded by Observer 2; O1n and O2n are nonoccur- haps for this reason, kappa is less often used by
rence counts, and I is the number of observations applied behavior analysts than is IOA.
made by each observer.
For example if Observer 1 scored nine intervals Frequency Within Interval
out of 10 and Observer 2 scored eight intervals out of Calculating IOA for frequency within interval mini-
10 (see Exhibit 6.1), kappa would be calculated as mizes the limitation identified for IOA on frequency
follows: Po = .90; PC = (72 + 2)/102 = .74, κ = .62. recording (high agreement even though the two
Kappa values can range from −1 to 1, with 0 observers might be recording different instances of
reflecting a chance level of agreement. No single rule behavior). For example, if agreement is calculated
for interpreting an obtained kappa value may be within each of a series of 20-second intervals, then
given because the number of different categories there is no chance that a response recorded by
into which behavior may be classified will affect Observer A in Interval 1 will be counted as an agree-
kappa. If only two categories are used (e.g., occur- ment with a different response recorded by Observer
rence vs. nonoccurrence), then the probability of a B in Interval 12. To calculate frequency-within-in-
chance agreement is higher than if more categories terval agreement, calculate a percentage of agree-
had been used. Higher probabilities of chance agreement between observers for each interval (smaller
ment are reflected in lower kappa values. Thus, if number divided by larger number), sum the per-
the preceding example had used three categories centages for all the intervals, and divide by the num-
(e.g., slow-, medium-, or high-rate responding) and ber of intervals in the observation period. Exhibit 6.2
IOA had been the same (90%), then kappa would illustrates an IOA calculation for frequency-within-
have been more than .62. interval data for 10 intervals for two observers. Each
X corresponds to an occurrence of the behavior in
an interval.
Exhibit 6.1
Recordings for Observer 1 and Observer 2 in 10 Ethical Considerations
Observation Intervals Several ethical issues must be considered when con-
ducting observations and measurement as part of
Interval research or clinical practice in applied behavior
analysis (e.g., Bailey & Burch, 2005). First, a behav-
1 2 3 4 5 6 7 8 9 10 ior analyst should observe and record the person’s
Observer 1
behavior only after receiving written consent from
X X X X X X X X X the individual or the individual’s parent or guardian.
Observer 2 As part of the consent process, the individual must
X X X X X X X X be apprised of and agree to the ways in which the
data will be used (research presentation or publication,
145
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Exhibit 6.2
Frequency-Within-Interval Recordings for Two Observers
and Interobserver Agreement Calculation
Interval
1 2 3 4 5 6 7 8 9 10
Observer 1
XXX X XX XX X XXX XX
67% 100% 100% 100% 100% 100% 50% 100% 50% 100%
Observer 2
XX X XX XX XX XXX X
Note. Interobserver agreement = (67 + 100 + 100 + 100 + 100 + 100 +

50 + 100 + 50 + 100)/10 = 86.7%.
clinical decision making). If the behavior analyst adequate measurement of behavior requires clear
identifies new uses for the data after the original definitions of the target behavior, precise specifica-
consent has been obtained, new consent must be tions of recording logistics and procedures, appro-
obtained from the individual for the new ways in priate choice of recording devices, and consideration
which the data will be used. Second, the individual of reactivity and IOA. The validity of conclusions
must know when and where observation will take that can be drawn from experimental manipulations
place unless the individual provides written consent of controlling variables or evaluations of treatment
for surreptitious or unannounced observation. effectiveness depends on the adequacy of the obser-
Third, observation and recording must take place in vation and measurement of the behaviors targeted in
such a way that confidentiality is maintained for the these endeavors.
individual receiving services or participating in
research. To maintain confidentiality, the observer References
must not draw attention to the person being
Aikman, G., & Garbutt, V. (2003). Brief probes: A method
observed and must not inform any other people for analyzing the function of disruptive behaviour in
about the observations unless the individual being the natural environment. Behavioural and Cognitive
observed has provided written permission to do so. Psychotherapy, 31, 215–220. doi:10.1017/S1352465
803002108
In addition, behavior analysts must use pseudonyms
and disguise other identifying information in pre- Allen, K. D. (1998). The use of an enhanced simpli-
fied habit reversal procedure to reduce disruptive
sentations and publications. Fourth, observers must outbursts during athletic performance. Journal of
treat the individual being observed and others in the Applied Behavior Analysis, 31, 489–492. doi:10.1901/
setting with dignity and respect at all times during jaba.1998.31-489
the course of their participation in research or as Athens, E. S., Vollmer, T. R., & St. Peter Pipkin, C. C.
they are receiving clinical services. (2007). Shaping academic task engagement with
percentile schedules. Journal of Applied Behavior
Summary Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some
Observation and measurement are at the heart of Journal of Applied Behavior Analysis, 1, 91–97.
applied behavior analysis because behavior (and its doi:10.1901/jaba.1968.1-91
controlling variables) is the subject matter of both Bailey, J. S., & Burch, M. R. (2002). Research methods in
research and practice. As discussed in this chapter, applied behavior analysis. Thousand Oaks, CA: Sage.
146
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bailey, J. S., & Burch, M. R. (2005). Ethics for behavior negative reinforcement: Effects on teacher and stu-
analysts. Mahwah, NJ: Erlbaum. dent behavior. School Psychology Review, 34, 220–231.
Beck, K. V., & Miltenberger, R. G. (2009). Evaluation of DiGennaro-Reed, F. D., Codding, R., Catania, C. N.,
a commercially available program and in situ train- & Maguire, H. (2010). Effects of video modeling
ing by parents to teach abduction-prevention skills on treatment integrity of behavioral interventions.
to children. Journal of Applied Behavior Analysis, 42, Journal of Applied Behavior Analysis, 43, 291–295.
761–772. doi:10.1901/jaba.2009.42-761 doi:10.1901/jaba.2010.43-291
Borrero, C. S. W., & Borrero, J. C. (2008). Descriptive Donaldson, J. M., & Normand, M. P. (2009). Using goals
and experimental analyses of potential precursors setting, self-monitoring, and feedback to increase
to problem behavior. Journal of Applied Behavior calorie expenditure in obese adults. Behavioral
Analysis, 41, 83–96. doi:10.1901/jaba.2008.41-83 Interventions, 24, 73–83. doi:10.1002/bin.277
Bosch, A., Miltenberger, R. G., Gross, A., Knudson, P., Fleiss, J. L. (1981). Statistical methods for rates and pro-
& Brower-Breitweiser, C. (2008). Evaluation of portions (2nd ed.). New York, NY: Wiley.
extinction as a functional treatment for binge eating.
Behavior Modification, 32, 556–576. doi:10.1177/ Flessner, C. A., Miltenberger, R. G., Egemo, K., Jostad, C.,
0145445507313271 Gatheridge, B. J., Neighbors, C., . . . Kelso, P. (2005).
An evaluation of the social support component of
Boyce, T. E., & Geller, E. S. (2001). A technology to mea- simplified habit reversal. Behavior Therapy, 36, 35–42.
sure multiple driving behaviors without self-report doi:10.1016/S0005-7894(05)80052-8
or participant reactivity. Journal of Applied Behavior
Analysis, 34, 39–55. doi:10.1901/jaba.2001.34-39 Fogel, V. A., Miltenberger, R. G., Graves, R., & Koehler,
S. (2010). Evaluating the effects of exergaming on
Brackett, L., Reid, D. H., & Green, C. W. (2007). Effects physical activity among inactive children in a physi-
of reactivity to observations on staff performance. cal education classroom. Journal of Applied Behavior
Journal of Applied Behavior Analysis, 40, 191–195. Analysis, 43, 591–600. doi:10.1901/jaba.2010.43-591
doi:10.1901/jaba.2007.112-05
Gatheridge, B. J., Miltenberger, R., Huneke, D. F.,
Brown, R. A., Palm, K. M., Strong, D., Lejuez, C.,
Satterlund, M. J., Mattern, A. R., Johnson, B. M., &
Kahler, C., Zvolensky, M., . . . Gifford, E. (2008).
Flessner, C. A. (2004). A comparison of two pro-
Distress tolerance treatment for early-lapse smok-
grams to teach firearm injury prevention skills to 6-
ers: Rationale, program description, and prelimi-
and 7-year-old children. Pediatrics, 114, e294–e299.
nary findings. Behavior Modification, 32, 302–332.
doi:10.1542/peds.2003-0635-L
doi:10.1177/0145445507309024
Codding, R. S., Livanis, A., Pace, G. M., & Vaca, L. Gravlee, C. C., Zenk, S. N., Woods, S., Rowe, Z., &
(2008). Using performance feedback to improve Schulz, A. J. (2006). Handheld computers for direct
treatment integrity of classwide behavior plans: observation of the social and physical environment.
An investigation of observer reactivity. Journal of Field Methods, 18, 382–397. doi:10.1177/1525822X
Applied Behavior Analysis, 41, 417–422. doi:10.1901/ 06293067
jaba.2008.41-417 Gresham, F. M., Gansle, K. A., & Noell, G. H. (1993).
Connell, J. E., & Witt, J. C. (2004). Applications of Treatment integrity in applied behavior analysis with
computer-based instruction: Using specialized soft- children. Journal of Applied Behavior Analysis, 26,
ware to aid letter-name and letter-sound recogni- 257–263. doi:10.1901/jaba.1993.26-257
tion. Journal of Applied Behavior Analysis, 37, 67–71. Harris, F. C., & Ciminero, A. R. (1978). The effects of
doi:10.1901/jaba.2004.37-67 witnessing consequences on the behavioral recording
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). of experimental observers. Journal of Applied Behavior
Applied behavior analysis (2nd ed.). Upper Saddle Analysis, 11, 513–521. doi:10.1901/jaba.1978.11-513
River, NJ: Pearson Education. Hayes, S. C., Barlow, D. H., & Nelson-Gray, R. O. (1999).
Daniels, A. C., & Daniels, J. E. (2004). Performance man- The scientist practitioner: Research and accountabil-
agement: Changing behavior that drives organizational ity in the age of managed care. Boston, MA: Allyn &
effectiveness. Atlanta, GA: Performance Management. Bacon.
DiGennaro, F. D., Martens, B. K., & Kleinmann, A. E. Hayes, S. C., Wilson, K. G., Gifford, E., Bissett, R., Piasecki,
(2007). A comparison of performance feedback M., Batten, S., . . . Gregg, J. (2004). A preliminary trial
procedures on teachers’ treatment implementation of twelve-step facilitation and acceptance and commit-
integrity and students’ inappropriate behavior in spe- ment therapy with polysubstance-abusing methadone-
cial education classrooms. Journal of Applied Behavior maintained opiate addicts. Behavior Therapy, 35,
Analysis, 40, 447–461. doi:10.1901/jaba.2007.40-447 667–688. doi:10.1016/S0005-7894(04)80014-5
DiGennaro, F. D., Martens, B. K., & McIntyre, L. L. Himle, M. B., Miltenberger, R. G., Flessner, C., & Gatheridge, B.
(2005). Increasing treatment integrity through (2004). Teaching safety skills to children to prevent
147
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
gun play. Journal of Applied Behavior Analysis, 37, in the treatment of oral-digit habits exhibited by
1–9. doi:10.1901/jaba.2004.37-1 individuals with mental retardation. Journal of
Himle, M. B., Miltenberger, R. G., Gatheridge, B., & Applied Behavior Analysis, 32, 353–365. doi:10.1901/
Flessner, C. (2004). An evaluation of two procedures jaba.1999.32-353
for training skills to prevent gun play in children. MacDonald, R., Sacramone, S., Mansfield, R., Wiltz, K.,
Pediatrics, 113, 70–77. doi:10.1542/peds.113.1.70 & Ahern, W. (2009). Using video modeling to teach
Iwata, B. A., Pace, G. M., Kissel, R. C., Nau, P. A., & reciprocal pretend play to children with autism.
Farber, J. M. (1990). The Self-Injury Trauma (SIT) Journal of Applied Behavior Analysis, 42, 43–55.
scale: A method for quantifying surface tissue dam- doi:10.1901/jaba.2009.42-43
age caused by self-injurious behavior. Journal of Mace, F. C., Prager, K. L., Thomas, K., Kochy, J., Dyer,
Applied Behavior Analysis, 23, 99–110. doi:10.1901/ T. J., Perry, L., & Pritchard, D. (2009). Effects of
jaba.1990.23-99 stimulant medication under varied motivational
Jackson, J., & Dixon, M. R. (2007). A mobile computing operations. Journal of Applied Behavior Analysis, 42,
solution for collecting functional analysis data on a 177–183. doi:10.1901/jaba.2009.42-177
pocket PC. Journal of Applied Behavior Analysis, 40, Maglieri, K. A., DeLeon, I. G., Rodriguez-Catter, V. R., &
359–384. doi:10.1901/jaba.2007.46-06 Sevin, B. M. (2000). Treatment of covert food steal-
Jessup, P. A., & Stahelski, A. J. (1999). The effects of ing in an individual with Prader-Willi syndrome.
a combined goal setting, feedback and incentive Journal of Applied Behavior Analysis, 33, 615–618.
intervention on job performance in a manufactur- doi:10.1901/jaba.2000.33-615
ing environment. Journal of Organizational Behavior Maher, E. (2009). Behavior Tracker Pro. Retrieved from
Management, 19, 5–26. doi:10.1300/J075v19n03_02 http://www.behaviortrackerpro.com/btp/Welcome.
Johnston, J. M., & Pennypacker, H. S. (1993). Readings html
for strategies and tactics of behavioral research (2nd Malott, R., & Trojan-Suarez, E. A. (2004). Elementary
ed.). Hillsdale, NJ: Erlbaum. principles of behavior (5th ed.). Upper Saddle River,
Kahng, S. W., & Iwata, B. A. (1998). Computerized NJ: Prentice Hall.
systems for collecting real-time observational data. Marckel, J. M., Neef, N. A., & Ferreri, S. J. (2006). A pre-
Journal of Applied Behavior Analysis, 31, 253–261. liminary analysis of teaching improvisation with the
doi:10.1901/jaba.1998.31-253 picture exchange communication system to children
Kazdin, A. E. (1977). Artifact, bias, and complexity with autism. Journal of Applied Behavior Analysis, 39,
of assessment: The ABCs of reliability. Journal of 109–115. doi:10.1901/jaba.2006.131-04
Applied Behavior Analysis, 10, 141–150. doi:10.1901/ Mash, E. J., & McElwee, J. (1974). Situational effects on
jaba.1977.10-141 observer accuracy: Behavioral predictability, prior
Kazdin, A. E. (2010). Single case research designs: Methods experience, and complexity of coding categories.
for clinical and applied settings (2nd ed.). New York, Child Development, 45, 367–377. doi:10.2307/1127957
NY: Oxford University Press. Mayfield, K. H., & Vollmer, T. R. (2007). Teaching math
Kerwin, M. L., Ahearn, W. H., Eicher, P. S., & Swearingin, skills to at-risk students using home-based peer
W. (1998). The relationship between food refusal tutoring. Journal of Applied Behavior Analysis, 40,
and self-injurious behavior: A case study. Journal of 223–237. doi:10.1901/jaba.2007.108-05
Behavior Therapy and Experimental Psychiatry, 29, Meidinger, A. L., Miltenberger, R. G., Himle, M., Omvig,
67–77. doi:10.1016/S0005-7916(97)00040-2 M., Trainor, C., & Crosby, R. (2005). An investiga-
Landis, J. R., & Koch, G. G. (1977). The measurement of tion of tic suppression and the rebound effect in
observer agreement for categorical data. Biometrics, Tourette’s disorder. Behavior Modification, 29,
33, 159–174. doi:10.2307/2529310 716–745. doi:10.1177/0145445505279262
Lerman, D. C., Iwata, B. A., Smith, R. G., & Vollmer, T. R. Miltenberger, R. G. (2012). Behavior modification:
(1994). Restraint fading and the development of alter- Principles and procedures (5th ed.). Belmont, CA:
native behaviour in the treatment of self-restraint and Wadsworth.
self-injury. Journal of Intellectual Disability Research, Miltenberger, R., Rapp, J., & Long, E. (1999). A low tech
38, 135–148. doi:10.1111/j.1365-2788.1994.tb00369.x method for conducting real time recording. Journal of
Lindsley, O. R. (1968). Technical note: A reliable wrist Applied Behavior Analysis, 32, 119–120. doi:10.1901/
counter for recording behavior rates. Journal of jaba.1999.32-119
Applied Behavior Analysis, 1, 77–78. doi:10.1901/ Miltenberger, R. G., Woods, D. W., & Himle, M. (2007).
jaba.1968.1-77 Tic disorders and trichotillomania. In P. Sturmey
Long, E. S., Miltenberger, R. G., Ellingson, S. A., & Ott, (Ed.), Handbook of functional analysis and clinical
S. M. (1999). Augmenting simplified habit reversal psychology (pp. 151–170). Burlington, MA: Elsevier.
148
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Miltenberger, R. G., Wright, K. M., & Fuqua, R. W. Rapp, J. T., Miltenberger, R. G., Long, E. S., Elliott, A. J.,
(1986). Graduated in vivo exposure with a severe & Lumley, V. A. (1998). Simplified habit reversal
spider phobic. Scandinavian Journal of Behaviour for chronic hair pulling in three adolescents: A clini-
Therapy, 15, 71–76. doi:10.1080/16506078609455763 cal replication with direct observation. Journal of
Mowery, J., Miltenberger, R., & Weil, T. (2010). Applied Behavior Analysis, 31, 299–302. doi:10.1901/
Evaluating the effects of reactivity to supervisor jaba.1998.31-299
presence on staff response to tactile prompts and Repp, A. C., & Karsh, K. G. (1994). Hypothesis-based
self-monitoring in a group home setting. Behavioral interventions for tantrum behaviors of persons with
Interventions, 25, 21–35. developmental disabilities in school settings. Journal
Mozingo, D. B., Smith, T., Riordan, M. R., Reiss, M. L., of Applied Behavior Analysis, 27, 21–31. doi:10.1901/
& Bailey, J. S. (2006). Enhancing frequency record- jaba.1994.27-21
ing by developmental disabilities treatment staff. Repp, A. C., Karsh, K. G., Felce, D., & Ludewig, D.
Journal of Applied Behavior Analysis, 39, 253–256. (1989). Further comments on using hand-held com-
doi:10.1901/jaba.2006.55-05 puters for data collection. Journal of Applied Behavior
Parsons, H. M. (1974). What happened at Hawthorne?
Science, 183, 922–932. doi:10.1126/science.183. Riordan, M. M., Iwata, B. A., Finney, J. W., Wohl, M. K.,
4128.922 & Stanley, A. E. (1984). Behavioral assessment and
treatment of chronic food refusal in handicapped
Patel, M. R., Piazza, C. C., Layer, S. A., Coleman, R., &
children. Journal of Applied Behavior Analysis, 17,
Swartzwelder, D. M. (2005). A systematic evaluation
327–341. doi:10.1901/jaba.1984.17-327
of food textures to decrease packing and increase
oral intake in children with pediatric feeding dis- Saunders, M. D., Saunders, J. L., & Saunders, R. R. (1993).
orders. Journal of Applied Behavior Analysis, 38, A program evaluation of classroom data collection
89–100. doi:10.1901/jaba.2005.161-02 with bar codes. Research in Developmental Disabilities,
14, 1–18. doi:10.1016/0891-4222(93)90002-2
Pedhazur, E., & Schmelkin, L. (1991). Measurement,
design, and analysis: An integrated approach. Hillsdale, Schrandt, J. A., Townsend, D. B., & Poulson, C. L.
NJ: Erlbaum. (2009). Teaching empathy skills to children with
autism. Journal of Applied Behavior Analysis, 42,
Peterson, L., Homer, A. L., & Wonderlich, S. A. (1982). 17–32. doi:10.1901/jaba.2009.42-17
The integrity of independent variables in behavior
analysis. Journal of Applied Behavior Analysis, 15, Scott, D., Scott, L. M., & Goldwater, B. (1997). A perfor-
477–492. doi:10.1901/jaba.1982.15-477 mance improvement program for an international-
level track and field athlete. Journal of Applied Behavior
Plavnick, J. B., Ferreri, S. J., & Maupin, A. N. (2010). Analysis, 30, 573–575. doi:10.1901/jaba.1997.30-573
The effects of self-monitoring on the procedural
integrity of behavioral intervention for young chil- Silverman, K., Wong, C. J., Needham, M., Diemer, K.
dren with developmental disabilities. Journal of N., Knealing, T., Crone-Todd, D., . . . Kolodner, K.
Applied Behavior Analysis, 43, 315–320. doi:10.1901/ (2007). A randomized trial of employment-based
jaba.2010.43-315 reinforcement of cocaine abstinence in injection
drug users. Journal of Applied Behavior Analysis, 40,
Raiff, B. R., Faix, C., Turturici, M., & Dallery, J. (2010). 387–410. doi:10.1901/jaba.2007.40-387
Breath carbon monoxide output is affected by speed
of emptying the lungs: Implications for laboratory Sim, J., & Wright, C. C. (2005). The kappa statistic in
and smoking cessation research. Nicotine and Tobacco reliability studies: Use, interpretation, and sample
Research, 12, 834–838. doi:10.1093/ntr/ntq090 size requirements. Physical Therapy, 85, 257–268.
Rapp, J. T., Colby, A. M., Vollmer, T. R., Roane, H. S., Skinner, B. F. (1956). A case history in scientific method.
Lomas, J., & Britton, L. N. (2007). Interval record- American Psychologist, 11, 221–233. doi:10.1037/
ing for duration events: A re-evaluation. Behavioral h0047662
Interventions, 22, 319–345. doi:10.1002/bin.239 Stickney, M. I., & Miltenberger, R. G. (1999). Evaluation of
Rapp, J. T., Colby-Dirksen, A. M., Michalski, D. N., direct and indirect measures for the functional assess-
Carroll, R. A., & Lindenberg, A. M. (2008). Detecting ment of binge eating. International Journal of Eating
changes in simulated events using partial-interval Disorders, 26, 195–204. doi:10.1002/(SICI)1098-
recording and momentary time sampling. Behavioral 108X(199909)26:2<195::AID-EAT9>3.0.CO;2-2
Interventions, 23, 237–269. doi:10.1002/bin.269 Sundberg, M. L., & Michael, J. (2001). The benefits of
Rapp, J. T., Miltenberger, R. G., & Long, E. S. (1998). Skinner’s analysis of verbal behavior for children
Augmenting simplified habit reversal with an aware- with autism. Behavior Modification, 25, 698–724.
ness enhancement device: Preliminary findings. doi:10.1177/0145445501255003
Journal of Applied Behavior Analysis, 31, 665–668. Therrien, K., Wilder, D. A., Rodriguez, M., & Wine, B.
doi:10.1901/jaba.1998.31-665 (2005). Preintervention analysis and improvement
149
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
of customer greeting in a restaurant. Journal of in children. Journal of Applied Behavior Analysis, 26,
Applied Behavior Analysis, 38, 411–415. doi:10.1901/ 53–61. doi:10.1901/jaba.1993.26-53
jaba.2005.89-04
Wallace, M. D., Iwata, B. A., & Hanley, G. P. (2006).
Touchette, P. E., Macdonald, R. F., & Langer, S. N. Establishment of mands following tact train-
(1985). A scatterplot for identifying stimulus con- ing as a function of reinforcer strength. Journal of
trol of problem behavior. Journal of Applied Behavior Applied Behavior Analysis, 39, 17–24. doi:10.1901/
Analysis, 18, 343–351. doi:10.1901/jaba.1985.18-343 jaba.2006.119-04
Twohig, M. P., Masuda, A., Varra, A. A., & Hayes, S. C. Wilder, D. A., Zonneveld, K., Harris, C., Marcus, A., &
(2005). Acceptance and commitment therapy as Reagan, R. (2007). Further analysis of antecedent
a treatment for anxiety disorders. In S. M. Orsillo interventions on preschoolers’ compliance. Journal of
& L. Roemer (Eds.), Acceptance and mindfulness- Applied Behavior Analysis, 40, 535–539. doi:10.1901/
based approaches to anxiety: Conceptualization and jaba.2007.40-535
treatment (pp. 101–129). New York, NY: Kluwer.
doi:10.1007/0-387-25989-9_4 Wolf, M. M. (1978). Social validity: The case for subjec-
tive measurement or how applied behavior analysis is
Twohig, M. P., Shoenberger, D., & Hayes, S. C. (2007). finding its heart. Journal of Applied Behavior Analysis,
A preliminary investigation of acceptance and com- 11, 203–214. doi:10.1901/jaba.1978.11-203
mitment therapy as a treatment for marijuana depen-
dence in adults. Journal of Applied Behavior Analysis, Wong, C. J., Sheppard, J.-M., Dallery, J., Bedient, G.,
40, 619–632. doi:10.1901/jaba.2007.619-632 Robles, E., Svikis, D., & Silverman, K. (2003). Effects
of reinforcer magnitude on data-entry productivity in
VanWormer, J. J. (2004). Pedometers and brief e-counseling: chronically unemployed drug abusers participating
Increasing physical activity for overweight adults. in a therapeutic workplace. Experimental and Clinical
Journal of Applied Behavior Analysis, 37, 421–425. Psychopharmacology, 11, 46–55. doi:10.1037/1064-
doi:10.1901/jaba.2004.37-421 1297.11.1.46
Vollmer, T. R., Marcus, B. A., Ringdahl, J. E., & Roane, Wright, K. M., & Miltenberger, R. G. (1987). Awareness
H. S. (1995). Progressing from brief assessments to training in the treatment of head and facial tics.
extended experimental analyses in the evaluation Journal of Behavior Therapy and Experimental
of aberrant behavior. Journal of Applied Behavior Psychiatry, 18, 269–274. doi:10.1016/0005-7916(87)
Analysis, 28, 561–576. 90010-3
von Eye, A., & von Eye, M. (2008). On the marginal Young, J., Zarcone, J., Holsen, L., Anderson, M. C., Hall,
dependency of Cohen’s κ. European Psychologist, 13, S., Richman, D., . . . Thompson, T. (2006). A mea-
305–315. doi:10.1027/1016-9040.13.4.305 sure of food seeking in individuals with Prader-Willi
Wagaman, J. R., Miltenberger, R. G., & Arndorfer, R. E. syndrome. Journal of Intellectual Disability Research,
(1993). Analysis of a simplified treatment for stuttering 50, 18–24. doi:10.1111/j.1365-2788.2005.00724.x
150
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 7
Generality and Generalization

of Research Findings
Marc N. Branch and Henry S. Pennypacker
For generalization, psychologists must two related, but separable, subject matters, behav-
finally rely, as has been done in all the ioral science and actuarial science. We also focus
older sciences, on replication. (Cohen, on the issue of generality across individuals and
1994, p. 997) how traditional methods can give the illusion of
such generality. In the second major section, we
Confirmation comes from repetition. . . .
discuss dimensions of generality in addition to
Repetition is the basis for judging . . . sig-
generality across individuals. Here we define scien-
nificance and confidence. (Tukey, 1969,
tific generality and several other forms of generality
pp. 84–85)
as well. In so doing, we introduce the roles of rep-
As the general psychology research community lication, both direct and systematic, in assessing
becomes increasingly aware (e.g., Cohen, 1994; Lof- generality of research results. We argue that repli-
tus, 1991, 1996; Wilkinson & Task Force on Statisti- cation, instead of statistical inference, is an alter-
cal Inference, 1999) of the limitations of traditional native primary method for determining not only
group designs and statistical inference methods with the reliability of results but also for assessing and
regard to assessing reliability and generality of characterizing the generality of scientific findings.
research findings, we present an alternative approach In the third major section, we discuss generaliza-
that has been substantially developed in the branch of tion of treatment effects, the fundamentals of tech-
psychology now known as behavior analysis. In this nology transfer, and the practices that characterize
chapter, we outline how individual subject methods, translational research. There, we write of program-
that is, so-called single-case designs, provide straight- ming for and assessment of generalizability of sci-
forward and, in principle, simple methods to assess entific findings to applied settings. We expand our
the reliability and generality of research findings. view then to the engineering issues of technology
development (or technology transfer and transla-
tional research) as a capstone demonstration of
Overview
generalization based on an understanding of gen-
The chapter consists of three major sections. In erality of research findings.
the first, we summarize the limitations of tradi-
tional methods, especially as they relate to assess-
Limitations of Traditional
ing reliability and generality of research findings
Methods
concerning behavior. We make the case that tradi-
tional methods have obscured an important dis- The traditional group-mean, statistical-inference
tinction that has led to psychology’s consisting of approach to analyzing research results has faced
Preparation of this chapter was supported by National Institute on Drug Abuse Grant DA004074.
DOI: 10.1037/13937-007
151
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Branch and Pennypacker
consistent criticism for more than 4 decades (e.g., A p value, which results from a significance test,
Bakan, 1966; Carver, 1978; Cohen, 1994; Gigeren- is a conditional probability. Specifically, it is the
zer, Krauss, & Vitouch, 2004; Loftus, 1991, 1996; probability, if the null hypothesis is true, of obtain-
Meehl, 1967, 1978; Nickerson, 2000; Rozeboom, ing data of a particular sort. That is, in algebraic
1960). Most of that criticism has focused on what symbols, it is p = P(Data|H0). The important point
those methods have to say about the reliability of is that p ≠ P(H0|Data), which is what a researcher
research findings, which is appropriate because if would presumably really like to know. In other
findings are not reliable, there is no need to assess words, a p value does not provide quantitative infor-
their generality. These methods, however, have also mation about whether the null hypothesis is true,
been criticized with respect to theory testing and which is apparently widely misunderstood. Because
development, issues that directly relate to generality. it does not provide the oft-assumed information
We treat these two categories of criticism separately. about the likelihood of the null hypothesis being
true, a p value of .01 does not mean that the proba-
Significance Testing and Reliability bility of the null hypothesis being true is 1 in 100.
After all of the carefully reasoned criticism of signifi- In fact, it conveys nothing quantitative about the
cance testing that has been published, one would truth of the null hypothesis. To see why, note that
hope that a clear understanding of its limits would changing the order of conditionality in a condi-
exist among professional psychologists. That, how- tional probability is crucially important. Consider
ever, appears not to be true, as noted by Cohen such examples as P(Dead|Electrocuted) versus
(1994), who lamented that P(Electrocuted|Dead) or P(Cloudy|Raining) versus
P(Raining|Cloudy). The first probability in each pair
after 4 decades of severe criticism, the
tells nothing about the second, just as P(Data|H0)
r itual of null hypothesis significance
reveals nothing about P(H0|Data). A p value, there-
testing . . . still persists. [As does] near
fore, has quantitative meaning only if the null
universal misinterpretation of p as the
hypothesis is true, but when performing statistical
probability that H-sub-0 is false, [and]
tests not only does one not know whether the null
the misinterpretation that its comple-
hypothesis is true, one probably assumes it is not.
ment is the probability of successful rep-
The important fact is that a finding of statistical sig-
lication. (p. 997)
nificance, via a small p value, does not imply that
Cohen’s assertion is supported by survey evidence the null hypothesis is unlikely to be true. The incor-
revealing that a substantial majority of academic rect logic underlying the mistaken conclusion (cf.
research psychologists incorrectly interpret p values Falk & Greenbaum, 1995) apparently goes as fol-
and statistical significance (Haller & Krauss, 2002; lows: If the null hypothesis is true, data of a certain
Kalinowski, Fidler, & Cumming, 2008; Oakes, sort are unlikely. I obtained data of that sort, so
1986). That a significant proportion of professional therefore the null hypothesis is unlikely to be true.
psychologists do not appreciate what statistical That so-called logic is precisely the same as the fol-
significance and, especially, p values represent is lowing: If the next person I meet is an American, he
apparent testimony to a weakness in the training of or she is unlikely to be the President. I just met the
research psychologists, a failing that lies at the feet President. Therefore, he or she is unlikely to be an
of those of us who are engaged in teaching them. In American.
fact, Haller and Krauss (2002) included a sample The fundamental misunderstanding of what a p
of statistical methodology instructors in their study value is leads directly to the more serious problem
and found that 80% of them were mistaken in their of assuming that it indicates something quantitative
understanding of p values, so it comes as less of a about the reliability, that is, the likelihood of repli-
surprise that the misconceptions are widespread. The cation, of the finding. A common misunderstanding
following discussion, therefore, is another attempt to (see Haller & Krauss, 2002, and Oakes, 1986, for
make clear what a p value is and what it means. evidence) is that a p value, for example of .01, is the
152
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Generality and Generalization of Research Findings
complement of the probability of replication should of generality. It is here that we show that psychol-
the experiment be repeated. That is, the mistaken ogy, presumably because of the widespread use of
assumption is that if one conducted the experiment significance testing, has developed two distinguish-
100 times, one should replicate the result on 99 of able subject matters.
those occasions (at least on average). If one knew
that the null hypotheses were true, then that would Significance Testing and Generality
be a correct interpretation of the p value. Of course, Traditional significance testing approaches in psy-
though, one does not know whether H0 is true chology are generally based on data averaged across
(again, one usually hopes it is not). In fact, one con- individuals. As is well known, the mean from a
ducts the statistical test so that one can make what group of individuals (a sample) provides an estimate
one (mistakenly) hopes is an educated guess about of the mean of the entire population from which the
whether it is true. Thus, to say on the basis of a sample is drawn, and that estimate can be bounded
small p value that a result is statistically reliable is by confidence intervals that provide information
to strain the meaning of reliable beyond reasonable (not the probability, however, that the population
limits. mean falls within the interval; see Smithson, 2003)
This limitation of statistical significance is not about how confident one can be that the population
based on technical details of the null hypothesis. mean lies within such intervals. Thus, the sample
That is, the problem does not lie with whether the mean provides information about a parameter that
underlying distribution is formally normal or near applies to the entire population. That fact appears to
normal or whether the statistical test involved is imply substantial generality; it applies to the entire
demonstrably robust with respect to violations of population (however delimited), so generality
assumptions about the underlying distribution. The appears maximized. This raises two important issues.
limitation is based in the logic of the approach. All First is the question of representativeness of the
the assumptions about the distributional character- means, both sample and population. That is, identi-
istic null hypothesis might in fact be true, but that is cal or similar means can result from substantially
not relevant when one is speaking of what a p value different distributions of scores. Two examples that
indicates. illustrate this fact are given in Figures 7.1 and 7.2.
A major limitation of statistical significance, In Figure 7.1, four distributions of 20 scores are
therefore, is that it does not provide direct informa- arrayed horizontally in the upper panel. In the top
tion about the reliability of research findings. With- row, the values are arithmetically separated, whereas
out knowledge about reliability, no examination of in the other three, they are clustered in various
generality can occur because repeatability is the ways. Note that none of the four is particularly “nor-
most basic test of generality. Notwithstanding that mal” in appearance, that is, clustered in the middle.
limitation, however, significance testing based on The four plots in the lower panel show—with the
group means may be seen, incorrectly, to have top plot corresponding to the top distribution in the
implications for generality of findings across sub- upper panel, and so on—the means (solid points)
jects. Adherence to this view unfortunately gains and standard deviations (bars) of the four distribu-
strength as sample size increases. In fact, however, tions. They are, as planned, identical. These data
regardless of sample size, no information about show that identical means and standard deviations,
intersubject generality can be extracted from a the stock in trade of inferential statistics, can be
significance statement because no knowledge is obtained from very different distributions of values.
afforded concerning the number of subjects for That is, in these cases the means and standard devia-
whom the effect actually occurred. We examine tions do not provide a particularly informative or
the implications of this fact in more detail below. representative indication of what the individual val-
Aside from the limits surrounding reliability just ues are, which implies that when dealing with aver-
described, other characteristics of group-mean data ages of measures, or averages across individuals,
warrant examination as we move into a discussion attention must be paid to the representativeness of
153
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 7.1. Upper panel: Four distributions of values,

with each symbol representing one value on the x-axis. Lower
panel: The corresponding means and standard deviations of
the four corresponding distributions from the upper panel.
From The Elements of Graphing Data (rev. ed., p. 215), by
W. S. Cleveland, 1994, Summit, NJ: Hobart Press. Copyright
1994 by AT&T Bell Laboratories. Reprinted with permission.
14 14
12 12
10 10
8 8
Y
6 6
4 4
2 2
0 0
0 5 10 15 20 0 5 10 15 20
X X
14 14
12 12
10 10
8 8
6 6
Y
4 4
2 2
0 0
0 5 10 15 20 0 5 10 15 20
X X
Figure 7.2. Anscombe’s quartet. Each of the four graphs shows 11 x–y pairs and the best-
fitting (least-squares estimate) straight line through the points. The slopes and intercepts of the
lines are identical. From “Graphs in Statistical Analysis,” by F. J. Anscombe, 1973, American
Statistician, 27, pp. 19–20. Copyright 1973 by the American Statistical Association. Adapted
with permission. All rights reserved.
154
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
the mean, not just its value, or even its standard like stem-and-leaf plots, box plots, and quantile–
deviation. Figure 7.2, which contains what is known quantile plots, are available (Cleveland, 1994;
as Anscombe’s quartet (Anscombe, 1973), provides Tukey, 1977). They provide a more complete
an even more dramatic illustration of how focusing description of measures from individuals, or a useful
only on the average of a set of numbers can lead one subset (as can be the case with quantile–quantile
to miss important features of that set. The four plots), than do simple means and standard errors or
graphs in Figure 7.2 plot 11 values in x–y coordi- means and confidence intervals. We presume that
nates and show the best-fitting (via the least-squares as null-hypothesis significance-testing approaches
method) straight line to the data. Obviously, the dis- become less prevalent, more effort will be directed
tributions of points are quite different in the four toward developing new and better techniques for
sets. Nevertheless, the means for the x values are all comparing distributions, methods that will include
the same, as are their standard deviations. The same and make evident the measures from individuals.
is true for the y values (yielding eight instances of
the sort shown in Figure 7.1). In addition, the slopes Two Separable Subject Matters
and intercepts of the straight lines are identical for for Psychology?
all four sets, as are the sums of squared errors and In some instances, the difference between a popula-
sums of squared residuals. Thus, all four would tion parameter, such as the population average, and
yield the same correlation coefficient describing the the activity of an individual is obvious. For example,
relation between x and y. consider the average rate of pregnancy in women
The point of these illustrations is to indicate between 20 and 30 years old. Suppose that rate is
that a sample mean, even though a predictor of a 7%. That, of course, is a useful statistic and can be
population mean, is not necessarily a good descrip- used to predict how many women in that age cate-
tion of individual values, so it is not necessarily a gory will be pregnant. More important for the pres-
good indicator of the generality across individual ent purposes, however, is that the value, 7%, applies
measures. When the measures come from individ- to no individual woman. That is, no woman is 7%
ual people (or other nonhuman animals), it follows pregnant. A woman is either pregnant or she is not.
that the average of the group may not reveal, and What of situations, however, in which an average
may well conceal, much about individuals. It is is representative of the behavior of individuals? For
important to remember, therefore, that sample example, suppose that a particular teaching tech-
means from a group of individuals permit infer- nique is discovered to result in a 10% increase in
ences about the population average, but these performance on some examination and that the
means do not permit inferences to individuals improvement is at or near 10% for every individual.
unless it is demonstrated that the mean is, in fact, Is that not a case in which a group average would
representative of individuals. Surprisingly, it is rare permit estimation of a population mean that is, in
in psychology to see the issue of representativeness fact, a good descriptor of the effect of the training
of an average even mentioned, although recently, for individuals and, because it applies to the popula-
in the domain of randomized clinical trials, the tion, has wide generality? The answer is yes and no.
limitations attendant to group averages have been The point to be made here is somewhat subtle,
gaining increased mention (e.g., Penston, 2005; and so we elaborate on it with an example. Consider
Williams, 2010). a situation in which a scientist is trying to determine
Many experimental designs, nevertheless, involve the relation between amount of practice at solving
comparison across groups with large numbers of five-letter anagrams and subsequent speed at solving
subjects, which raises the question of the practical- six-letter anagrams. Suppose, specifically, that no
ity of presenting the data of every individual. The practice and 10, 50, 100, and 200 anagrams of prac-
concern is legitimate, but the problem is not solved tice are to be compared. After the practice, subjects
by resorting to the study of group averages only. who have never previously solved anagrams, except
Excellent techniques for comparing distributions, for those seen in the practice phase, are given 50 new
155
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
anagrams to solve, and the time to complete is The point is not to diminish the value of actuar-
recorded. Because total practice might be a determi- ial data, nor to suggest that psychologists abandon
nant of speed, the scientist opts to use a between- the collection and analysis of such data. If means are
groups design, with each group being exposed to highly representative, such data can offer predic-
one of the practice regimens. That is, the hope is to tions at the individual subject level. Even if the
extract the seemingly pure relation between practice means are not highly representative, organizations
and later speed, uncontaminated by prior relevant such as insurance companies and governments can
practice. The scientist then averages the data from and do make important use of such information in
each group and uses those means to describe the determining appropriate shared risk or regulatory
function relating amount of practice to speed of solv- policy, respectively. The point is, using insurance
ing the new, more difficult anagrams. In an actual rates as an example, that just because you are in
case, variability would likely be found among indi- a particular group, for example, that of drivers
viduals within each group, so one issue would be between the ages of 16 and 25, for which the mean
how representative the average is of each member rate of accidents is higher than for another group,
of each group. For our example, however, assume does not indicate that you personally are more likely
that the average is representative, even perfectly so to have an automobile accident. It does mean, how-
(i.e., every subject in a group gives exactly the same ever, that for the insurance company to remain prof-
value). The scientist has generated a function, proba- itable, insurance rates need to be higher for all
bly one that describes an increase in speed of solving members of the group. Similarly, with respect to
anagrams as a function of amount of prior practice. health policy, even though most people who smoke
In our example, that function allows us to predict cigarettes do not get lung cancer, the incidence of
exactly what an individual would do if exposed to a lung cancer, on a relative basis, is substantially
certain amount of practice. Even though the means greater, on average, in that group. Because the group
for each group are representative and therefore per- is large, even a low incidence rate yields a substan-
mit prediction about individual behavior, the impor- tial number of actual lung cancer cases, so it is in
tant point is that the function has no meaning for the government’s, and the population’s, interest to
an individual. That is, the function does not describe reduce the number of people who smoke cigarettes.
something that would occur for an individual The crux of the matter is that actuarial and
because no individual can be exposed to different behavioral data, although related in that the former
amounts of practice for the first time. The function depend on the latter, are distinguishable and, there-
is an actuarial account, not a description of a fore, should be distinguished. Psychology, to the
behavioral process. It is, of course, to the extent extent that it relies solely on the methods of inferen-
that the means are representative, a useful finding. tial statistics that use averages across individuals,
It is just not descriptive of a behavioral process in becomes an actuarial science, not a science of behav-
an individual. To examine the same issue at the ioral processes. The methods described in this
level of an individual would require investigation chapter are aimed at including in psychology its
of sequences of amounts of practice, and that oft-stated goal of being a science of behavior (or of
examination would have to include experiments the mind). Behavioral and inferred mental processes
that factor in the role of repeated practice. Obvi- really make sense only at the level of the individual.
ously, such an endeavor is considerably more com- (The same is true of physiology, which has become a
plicated than the study that generated the actuarial rather exact science in part because of the influence
curve, but it is the only way to develop a science of of Claude Bernard, 1865/1957.) A person’s behavior,
individual behavior. The ontogenetic roots of including thinking, imagining, and so forth, is par-
behavior cumulate over lifetimes. In later portions ticular to that person. That is, people do not share
of this chapter, we discuss how the complications their minds or their behavior with others, just as
may be confronted. they do not share their physiology. A counterargument
156
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
is that behavior and mental activity are too variable Let us compare how a more traditional between-
from individual to individual to permit a scientific groups approach might fare in dealing with the
analysis. We based this chapter on the more opti- issue. We apply music to one group and not to
mistic view that such activity is amenable to study at another. What will result will depend on the distri-
the level of the individual. Because a good deal of bution of baseline accuracy across individuals.
application of psychological knowledge involves Figure 7.3 shows three possible population distribu-
dealing with individuals, for example, as in psycho- tions. In B, most people have low accuracy, in C
therapy, understanding at the level of the individual most have high accuracy, and in A people fall into
remains a worthy goal. Support for the viewpoint two groups with respect to baseline accuracy. If one
that a science of individual behavior is possible, performed the experiment on groups and took the
however, requires an elaboration of how an individ- group mean to be the indicator of the effect of the
ual subject–based analysis can yield information that independent variable, the conclusion would depend
is more generally applicable to more than one or a on the underlying distribution. In A, the conclusion
few individuals.
Why Single-Case Designs Do Not

Mean That N = 1
Traditional approaches, with the attendant limita-
tions described thus far, likely arose, at least in part,
because of a legitimate concern about focusing
research on individual subjects who are studied
repeatedly through time (more on this later). Such
research is usually performed with relatively few
subjects, leaving open the possibility that effects
seen might be limited with respect to generality
across other individuals. An example, modeled after
one offered by Sidman (1960), provides a response
to such misgivings. Suppose we were interested in
whether listening to classical music while solving
arithmetic problems improves accuracy. Using a
single-case approach, the study is started with a single
subject. We might first establish a baseline of accu-
racy (more on this later) by measuring it over several
successive exposures. Next, we would test the sub-
ject with the music present and then with it absent.
Suppose we find that accuracy is increased when
music is present and reverts to normal when it is
not. Suppose also that unbeknownst to us, the
effect music will have depends on the baseline level
of accuracy; if accuracy is initially low, it is
enhanced by the presence of music, whereas if it is Figure 7.3. Three hypothetical frequency distribu-
initially high, it is reduced when the music is on. tions characterizing the number of people display-
We might mistakenly conclude, on the basis of the ing different baseline rates. From Tactics of Scientific
Research: Evaluating Experimental Data in Psychology
results from the one subject, that music increases
(p. 149), by M. Sidman, 1960, New York, NY: Basic
accuracy of solving the kinds of arithmetic prob- Books. Copyright 1988 by Murray Sidman. Reprinted
lems used. with permission.
157
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
might well be that music has no effect, with the low- discuss later how to decide how many subjects to
ered accuracy in people with high baseline accuracy test.) What the approach involves is studying each
canceling out the increases that result among those subject essentially as an independent experiment.
with low baseline accuracy. If the population is dis- Generality across subjects is therefore examined
tributed as in B, the conclusion would be that music directly by seeing how often the experiment’s effects
increases accuracy because the mean would move in are replicated. A second major point is that the
the direction of improved accuracy. The important apparent virtues of studying many subjects, a stan-
point is that simply considering the group average dard aspect of traditional research designs in psy-
makes it less likely that the baseline dependency that chology, are realized only if the data from each
underlies the effect will be seen. subject are individually analyzed.
Let us now compare what might transpire with
the single-case approach, an approach based on rep- Null-Hypothesis Significance Testing
lication. Having seen the effect in the first subject, and Theory Development
we recruit a second and do the experiment again. A major goal in any science is the development of
Suppose that the population distribution is as theory, and there is a sense in which theory has clear
depicted in Figure 7.3B. The most likely scenario is relevance to generality. Effective theories are those
that the second subject will also have low baseline that account for a wide array of research results. That
accuracy because someone sampled from the popu- is, they apply generally. The way in which signifi-
lation is most likely to manifest modal characteris- cance testing is most commonly used in psychology,
tics. We get the same result and could, mistakenly, however, mitigates against the orderly development
conclude that music enhances arithmetic accuracy. and testing of theories and against the analysis of
That is, we make the same mistake as with the competing theories. The problem was first identified
group-average approach. The difference between the as a paradox by Meehl (1967; see also Meehl, 1978).
two approaches, however, is that the group mean The problem is a logical one based largely on the
approach makes it more difficult to discover the choice of the null hypothesis as “no effect.” The logic
underlying, real effect. The single-case approach, of the common approach is as follows. An investiga-
however, if enough replications are done, will even- tor has a hypothesis that imposition of a variable, X,
tually and inevitably reveal the problem because will change another measure, Y. This hypothesis is
sooner or later someone with high baseline accuracy sometimes called the alternative hypothesis. The
will be examined and show a decrease. A key phrase null hypothesis is then chosen to be that X will not
in the previous sentence is “if enough replications change Y, that is, that it will be without effect. Next,
are done.” Whether that happens is likely to depend the X condition is imposed, and Y is measured. A
on the perceived importance of the effect. If it is comparison is then made of Y without X and Y with
deemed important, it is likely to be subjected to X. A statistic is then calculated that is generally a
additional research, which will, in turn, lead to addi- ratio of changes in Y as a result of X over changes in
tional replications. Thus, the single-case approach is Y as a result of anything else. In more technical
not some sort of panacea with respect to identifying terms, the statistic is effect variance over error vari-
such relations, but it offers a direct path to correc- ance. The larger the statistic, the smaller the p value,
tive action. Of course, it is possible to ferret out the and the more likely it is that statistical significance is
baseline dependency using a group-mean approach, achieved and the null hypothesis rejected. Standard
but that will happen only if attention is paid to the teaching demands that even though one can decide
data of individual subjects in a group. In the single- to reject the null hypothesis, logic prevents one from
case approach, those data are automatically scruti- accepting the alternative hypothesis. Instead, one
nized. A major point is that single case does not would say that if the null hypothesis is rejected, the
necessarily imply that only one or even only a few alternative hypothesis gains support.
subjects be examined. Some research questions The paradox noted by Meehl (1967) arises
might involve examination of many subjects. (We from the nature of the statistic itself. The size of the
158
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
statistic is controlled by two values, the effect size The use of a particular value as the null hypothesis
and the error variance, so it can be increased in two therefore requires that one’s theory be quantitative
ways. The way of interest for this discussion is via enough to generate a specific value. This approach is
a decrease in error variance, the denominator. A what characterizes tests of goodness of fit (those that
major way of decreasing error variance is through use significance tests) of quantitatively specified
increased experimental rigor (one avenue of which functions.
is to increase the number of subjects). To the degree This approach of setting the null hypothesis at
that extraneous variables (the “anything else” men- a value predicted by theory is nevertheless not
tioned earlier) can be eliminated or held constant, immune to the previously described weaknesses of
error variance should decrease, making it more significance testing in general. If, however, signifi-
likely that that statistic will be large enough to war- cance testing is used to make decisions, at least this
rant a decision as to statistical significance. The latter approach does not suffer from the weakness of
paradox, therefore, is that as experimental rigor is making it easier to support a researcher’s theory,
increased—that is, as experimental techniques are regardless of what it is, as methods improve.
refined and improved—statistical significance In this section of the chapter, we have made the
becomes more likely, with the consequence that the case, we hope, that commonly used psychology
alternative hypothesis gains support, no matter what research methods have limitations in assessing reli-
the alternative hypothesis is. That does not seem ability and generality of research findings. In addi-
like a recipe for cumulative progress in science. Sim- tion, the methods have resulted in many areas of
ple null-hypothesis significance testing with the null psychology being largely actuarial, group-average–
hypothesis set at no effect cannot, by itself, help to focused science rather than aimed at the behavior of
develop theory. individuals. In the next section, we describe the
Meehl (1967) described one approach that can basics of an alternative approach that is based on
obviate this paradox, which is to use significance replication rather than significance testing and
testing with a null hypothesis that is not “no effect.” group averages. It is useful to remember that impor-
Instead, the null hypothesis is what the theory (or tant science was conducted before the invention of
alternative hypothesis) predicts. Consider how the significance testing, and what follows is a descrip-
logic operates when this tactic is used. As experi- tion of the application of methods used to establish
mental rigor increases, error variance is decreased, most of modern physics and chemistry (and physiol-
making it more likely that the resulting statistic will ogy) to the study of behavior. The approach focuses
reach a critical value. When that value is achieved, on understanding behavioral processes, rather than
the null hypothesis is rejected, but in this case it is actuarial ones, and has already yielded a good deal
the investigator’s theory that is rejected. Rather than of success, as other chapters in Volume 2 of this
increased experimental rigor resulting in its being handbook illustrate. We should note, nevertheless,
easier for one’s theory to gain support, it results in that even if the goal is actuarial prediction and influ-
its being easier to reject one’s theory. Increasing ence, the methods of statistical inference are limited
experimental control puts the theory to a more rig- in what they can achieve with respect to reliability
orous test, not an easier one as is the case when of research findings. As we argue, the only sure way
using the no-effect, or no-difference, null hypothe- to examine reliability of results is to repeat them, so
sis. The harder one works to reject a theory and fails replication is the key strategy for both subject mat-
to succeed, the more confidence one has in the ters of psychology.
theory.
Training in statistical inference, at least for psy-
Assessing Reliability and
chologists, does not usually emphasize that the null
Generality via Replication
hypothesis need not be no effect. It can, neverthe-
less, as just noted, be some particular effect. Note The two distinguishable categories of replication are
that it has to be some specific value other than zero. direct replication and systematic replication,
159
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
although, as we show, the distinction is not a sharp subject will eat of that particular food after 12 hours
one. Most researchers are familiar with the concept of not eating.
of direct replication, which refers to repeating an One thing that direct replication can provide, via
experiment as exactly as possible. If the results are a sequence of direct, intrasubject replications such
the same or similar enough, the initial effect is said as that just described, is a baseline. The left segment
to be replicated. Direct replication, therefore, is of Figure 7.4 shows that there appears to be a steady
mainly used to assess the reliability of a research baseline amount of intake in our example. A ques-
finding, but as we show, there is a sense in which it tion that might arise is how many observations are
also provides information about generality. System- needed to establish a baseline, that is, to come up
atic replication is the designation for a repetition with a convincing assessment? The answer is that it
of the experiment with something altered to see depends. There is no rule or convention about how
whether the effect can be observed in changed cir- many replications are needed to render an outcome
cumstances. If the results are replicated, then the considered reliable in the eyes of the scientific com-
generality of the finding is extended to the new cir- munity. One factor of importance is how much is
cumstances. Many varieties of systematic replication already known. In some of the more advanced
exist, and it is the strategy most relevant to examin- physical sciences, a single replication (usually by a
ing the generality of research findings. different research team) might be adequate. In our
example, the researcher might have conducted simi-
Direct Replication: Within-Subject lar research previously, discovered that the baseline
Reliability and Baselines value does not change after 10 observations, and
In the first part of this section, we describe the thus deemed 10 replications enough. The researcher
methods and roles of direct replication with the who chooses replication as a strategy to determine
same experimental subject (i.e., a truly single-case reliability of findings, therefore, does not have the
experiment). We open with this simplest case, and comfort of a set of conventions (akin to those avail-
with an example, not only to illustrate how the strat- able to investigators who use conventional levels
egy can be used, but also to deal more clearly with of statistical significance) to decide whether to
reservations about and limitations of the approach conclude if an effect is reliable enough to warrant
as well as how decisions about characteristics of the reporting to the scientific community. Instead, the
replicative process may be made. investigator’s judgment plays a role, and his or her
For our example, suppose that we want to mea- scientific reputation is dependent to some degree on
sure the amount of a certain kind of food eaten after
some period without food. We let our subject eat 300
after 12 hours of fasting; suppose that she eats
250
250 grams. Direct replication of this observation
would require that we do the same test again. One 200
Grams Eaten
possible, but unlikely, result would be that she

150
would eat 250 grams again, providing an exact repli- Baseline - Food 1 Food 2 Food 1
cation. The amount eaten would more likely be 100
slightly different, say 245 grams. We might then
conduct another replication to see whether the trend 50
toward eating less was replicable. Suppose on that
0
occasion our subject eats 257 grams, making it less 0 2 4 6 8 10 12 14 16 18 20 22 24 26
likely that there is a trend toward less ingestion with Successive Tests
successive tests. We could repeat the process again
Figure 7.4. Hypothetical data from a series of obser-
and again. By repeatedly observing the amount eaten
vations of eating. The first 10 points and last six points
after a 12-hour fast, we gain more confidence with are amounts eaten of Food 1. The middle six points are
each successive measurement about how much our amounts eaten of Food 2.
160
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
that judgment. One of the comforts of a set of con- times if, in the investigator’s judgment, such confir-
ventions is that if a researcher abides by them and mation was necessary. Each successful replication
results are later discovered, via failed attempts at increases confidence that the independent variable,
replication, not to be reliable, that researcher’s repu- change of food type, is responsible for the change
tation suffers little. In contrast, one can argue that in eating.
there are both advantages and disadvantages to rely-
ing on replication. Important advantages are having Direct Replication: Between-Subjects
the benefit of informed judgment, especially of a Reliability and Generality
seasoned investigator, and the fact that social pres- After all this work, an immediate limitation is that
sure rides more directly on the researcher’s reputa- the findings, so far as we know, may well apply
tion. The disadvantage comes from the lack of an only to the one person studied. Our first result is
agreed-on set of conventions. Principled arguments based on intrasubject replication. If the goal of the
about which is better for science can be made for research was to see whether the change in food can
both positions, but we favor the view that science, influence eating, then it may be the case that no fur-
as a social–behavioral activity, will fare better, or at ther replication is needed. It is likely, however, that
least no worse, if researchers are held more account- our interest extends beyond what is possible to what
able for their conclusions about reliability and is common. In that case, additional replication is in
generality than for their adherence to a set of arbi- order, which brings us to the next type of direct rep-
trary, often misunderstood conventions. lication, replication with different subjects, or inter-
Returning to the role of a baseline construed as a subject replication. Intersubject replication is used to
set of intrasubject replications, such baselines can examine generality, in this case across subjects, and
serve as bases of comparison for effects of experi- in this single-case design N is extended to more
mental changes. For example, after establishing a than 1. Intersubject replication makes clear the fuzz-
baseline of eating of the first food, we could change iness of the distinction between direct and system-
what the food is, perhaps less tasty or more or less atic replication. The latter is generally defined as a
calorie laden. The second set of points in Figure 7.4, replication with something changed (see below),
which in essence depict measures from a second and a new subject is certainly a change. We also sug-
set of replications, have been chosen to indicate a gest that systematic replication is a main strategy for
decrease. The reliability of the effect is illustrated by assessing generality, and by studying a second sub-
the successive similarity of values, and judgments ject, generality across individuals is on trial. It is
about how many replications are needed would even possible to suggest that most replications, even
be based on the same sorts of considerations as intrasubject replications, are, in fact, systematic. For
involved in the original baseline. A usual check example, in the intrasubject replication described
would involve return to the original food, and the above, time is different for successive observations,
third set of points indicates a likely result, once and the subject brings a different history to each
again with a series of replications. The overall exper- observation period. It nevertheless has become stan-
iment, therefore, is an example of the ubiquitous dard to characterize replications in which the proce-
A-B-A design (see Chapter 1, this volume). dures are essentially the same as direct replications.
Replication, of course, need not refer only to a As we outline shortly, systematic replications are
series of successive measurements under identical characterized by changes in procedure or conditions
conditions to produce a baseline. If the type of find- that can be quite substantial.
ing summarized in Figure 7.4 were especially coun- As noted in the section Significance Testing and
terintuitive or at considerable odds with existing Generality earlier in this chapter, an emphasis on
knowledge, one might well repeat the entire project, replication with individual subjects approaches the
Food 1 to Food 2 to Food 1, and that, too, would issue of subject generality by increasing the number
constitute a direct intrasubject replication. In fact, of subjects studied. Suppose, for the sake of our
the entire project could be carried out multiple example, we study a second subject, performing the
161
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
entire experiment, baseline, new food, baseline, and eaten rather than a decrease. Here, there is no ques-
the whole sequence, over again. There are two major tion that an important aspect of the first result has
classes of outcomes. One, we get the same effect. not been replicated. What is to be done then? The
Two, we do not. Let us deal initially with the former answer lies in the assumption of determinism that is
possibility. The first issue is what we would accept at the core of behavioral science. If there is a differ-
as “same.” The second person’s baseline level would ence observed between Subject 1 and Subject 2, that
likely not be exactly the same, and in fact, it might difference is the result of some other influence. That
be notably different, averaging, say, 200 grams. is, people do not differ for no reason. In fact, the
Should we count that as a failure to replicate? The failure to replicate the exact intake levels at baseline
answer is (again), it depends. If our major concern must also be a result of some factor. Failure to repli-
was the exact amount eaten and the factors contrib- cate, therefore, is an occasion on which to initiate a
uting to that, then the result might well be consid- search for the variable or variables responsible for
ered a failure to replicate. We will hold off for a bit, the differences in outcomes. Suppose, for example,
however, on what to do in the face of such failures, that Subject 1 was female, and Subject 2 was male.
and move forward on the assumption that we are Tests with other men and women (note the expan-
not so much concerned with the exact amount eaten sion of N) could reveal whether this factor was
as with whether the change in food results in a important in determining the outcome. Similarly,
change in amount eaten. In that case, we might rep- we have already assumed different baseline levels, so
licate, with the second subject, the whole sequence it might be the case that baseline level is related to
of conditions, Food 1, Food 2, and back to Food 1. the direction of change in intake, a hypothesis that
Two possibilities exist: The results are the same as can be examined by studying additional subjects. It
for the first subject or they are not, and again, conse- is interesting that examination of this second possi-
quently, an important issue is what is meant by bility could be aided if the issue of different base-
same. The results are unlikely, especially in behav- lines between Subject 1 and Subject 2 had been
ioral science, to be identical quantitatively, and, assumed to be a failure to replicate. In that case, we
in fact, if the baseline is different, the change in would have focused on reasons for the difference
intake cannot be identical in both absolute and rela- and may have identified factors that determine base-
tive terms, so we are left to decide whether to focus line level. If that were so, it might be possible to
on what is different or on what is similar. In this control the baseline levels and to change them sys-
stage of the discussion, let us assume that intake tematically, thus providing a direct method for
decreased, as it had for the first subject. In that case, studying the relation between baseline level and the
we might feel confident that an important feature of effect of changing the food.
the data has been replicated. A next question, then, Another possible reason that disparate effects are
would be whether additional replication with other observed between subjects is differing sensitivity to
subjects is needed. In this particular example, the the particular value of the independent variable
answer would most likely be yes, but as is generally used. In the example just described, the indepen-
the case, the real answer is that it depends on what dent variable was characterized qualitatively as a
the goals of the experiment are. change in food type, making assessment of sensitiv-
Behavioral scientists, by and large, tend to focus ity to it difficult to assess. If, however, the indepen-
on similarities rather than differences, so if features dent variable can be characterized quantitatively, for
of data reveal similarity across individuals, those instance by carbohydrate content in our example,
similarities are likely to be pursued. Consider, there- the technique of systematic replication, elaborated
fore, a situation in which the data for the second below, can be used to examine the possibility.
subject are dissimilar, not only in quantitative terms An important issue in considering direct replica-
but in qualitative ones as well. For example, sup- tion arises when intersubject replication succeeds
pose that for the second subject the change from but intrasubject replication does not. Taking our
Food 1 to Food 2 results in an increase in amount example, suppose that when the conditions were
162
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
changed back to Food 1 with our first subject (cf.

Figure 7.4), eating remained at the lower level,
which would prevent replication of the effect in
Subject 1. Such a result indicates either that some
variable other than the change of food was responsi-
ble for the decrease in eating or that the exposure to
Food 2 has produced a long-lasting change in eat-
ing. Support for the second view can come from
attempts at intersubject replication. If experiments
with subsequent subjects reveal that a shift from
Food 1 to Food 2 results in a relatively permanent
decrease in eating, the effect is verified.
When initial effects are not recaptured after
intervening experience that produces a change, the
change is said to be irreversible. Using replication to Figure 7.5. Number of days of continuous absti-
nence from smoking cigarettes in two groups of sub-
examine irreversible effects requires intersubject jects. Circles are data from individuals. Open bars and
replication, so we have here another instance in brackets show the group means and standard errors
which N = 1 does not mean that only one subject of those means. Subjects represented by the left bar
need be studied. Many effects in psychology are irre- received vouchers contingent on abstinence, whereas
those represented by the right bar received vouchers
versible, for example, those that we call learning, so independent of their behavior. The top bracket and
the individual subject approach requires that inter- asterisk indicate that the mean difference was statisti-
subject replication be used to assess the reliability of cally significant at the .01 level. From “Voucher-Based
Contingent Reinforcement of Smoking Abstinence
such effects, and in so doing the generality of the
Among Methadone-Maintained Patients: A Pilot Study,”
effect across subjects is automatically examined. by K. E. Dunn, S. C. Sigmon, C. S. Thomas, S. H. Heil,
A focus on each subject individually, of course, and S. T. Higgins, 2008, Journal of Applied Behavior
does not prevent the use of traditional data analysis Analysis, 41, p. 533. Copyright 2008 by the Society for
the Experimental Analysis of Behavior, Inc. Reprinted
approaches, should an investigator be so inclined with permission.
(for inferential statistical analyses appropriate to
single-case research designs, see Chapters 11 and averages. Such a display of data provides consider-
12, this volume). Some, for example, might want to ably more useful information than do presentations
present group averages so that actuarial predictions that include only means or results of tests of statisti-
can be made. Standard techniques can be used sim- cal significance.
ply by engaging in the usual sorts of data manipula-
tion. An emphasis on the data from individuals, Systematic Replication: Parametric
nevertheless, can be used to enhance the presenta- Experiments
tion. For example, consider a study by Dunn, To this point, our emphasis has been on the
Sigmon, Thomas, Heil, and Higgins (2008), who intra- and intersubject generality and reliability of
compared two conditions aimed at reducing ciga- effects, and we have argued that individual subject
rette smoking. In one, vouchers were given contin- approaches can be effectively used to assess it. Gen-
gent on breath samples that indicated that no erality of effects, however, is not limited to general-
smoking had occurred, whereas in the other the ity across individuals, and it is to other forms of
vouchers were given independently of whether the generality, culminating with scientific generality, to
subject had smoked. Figure 7.5 shows some of the which we now turn.
results. The bars show group means, and the dots As noted earlier, systematic replication refers to
show data from each individual, illustrating the replication with something changed, and, as also
degree to which effects were replicable across noted, a case can be made that replication with a
patients and the representativeness of the group new subject is a form of systematic replication in
163
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
that it is an experiment with something changed,

namely the experimental subject. From such replica-
tions come assessments of the across-subject gener-
ality of effects. In this section, we discuss other sorts
of changes between experiments that constitute sys-
tematic replication. To do so, let us begin again with
our example of effects of food type on eating. Sup-
pose that after obtaining the data in Figure 7.4, we
perform a systematic replication of the study rather
than a direct repetition. For example, we might
notice that Food 2’s carbohydrate content is higher
than that of Food 1. We decide, therefore, to alter
the carbohydrate content of Food 2 (and let us
assume, likely impossible, without changing the Figure 7.6. Hypothetical data for three subjects
showing the relationship between carbohydrate content
taste) so that it matches that of Food 1, and repeat
and amount eaten.
the experiment. Such an experiment would examine
the generality of Food 2’s effect on eating to a new
carbohydrate level. If adjusting Food 2’s carbohy- independent variable been assessed, we might have
drate amount to equal that of Food 1 resulted in the been less than impressed with the degree of inter-
switch in foods having no effect on eating, two subject replicability of the data. The results of para-
things can be concluded. One, the original result metric examination, however, reveal a good deal of
was not replicated. In such cases, it is often wise similarity across the three subjects: All show the
to replicate the original experiment to determine same basic relation. At low percentages, the amount
whether unknown variables might have been eaten is roughly constant within each individual.
responsible. Two, carbohydrate amount is identified As the percentage increases, the amount eaten
as a likely important variable. Thus, systematic rep- decreases until the percentage reaches a value above
lication is not only a method for discovering gener- which further increases are associated with no
ality of effects, it is also an approach that can lead to changes in amount eaten. Second, and this is a key
finding controlling variables. characteristic of parametric evaluation, the data sug-
Continuing our description of types of systematic gest that only a range of levels of the independent
replication, let us assume we decide to examine variable result in a change in behavior. That is, para-
more fully the role of carbohydrates in eating. Our metric experiments permit the identification of
original experiment may be conducted several times boundary conditions, or limiting conditions, outside
but with a different carbohydrate mix for Food 2 on of which a variable is relatively ineffective. As we
each occasion. Each repetition of the experiment, show later when dealing with the issue of scientific
then, constitutes a systematic replication because a generality, information about boundary conditions
new value of carbohydrate is used for each instance. can be extremely important.
Experiments that systematically vary the value of a Figure 7.6 also illustrates how parametric experi-
variable are called parametric experiments, and they ments can help deal with the problem of lack of
play an especially important role in assessing gener- intersubject replicability when a single value of an
ality. Consider the data in Figure 7.6, which are independent variable is examined. Recalling our
constructed to emulate what might result if several original example of comparison of food types, con-
intersubject replications of a parametric experiment sider what could have happened if our first two sub-
were conducted. jects were Subjects 1 and 3 of Figure 7.6 and Food 1
Parametric examination provides a number of had contained 20% carbohydrate and Food 2 had
advantages when assessing the reliability and gener- contained 25%. Changing the food type would have
ality of results. First, had only a single value of the produced a change for Subject 1 but not for Subject 3,
164
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
leading to a conclusion that we had failed to repli- other type involves using earlier findings as a starting
cate the food change effect across subjects. The point, or baseline, for examination of other variables.
parametric examination, however, shows that both As an example, consider the phenomenon of false
subjects are similar in how food intake was influ- memory in the laboratory, produced by a procedure
enced by carbohydrate content, except that behavior originally developed by Deese (1959) and later elabo-
of the two subjects was sensitive in a slightly differ- rated by Roediger and McDermott (1995). In these
ent range. One of the most satisfying outcomes of studies, subjects said they recalled or recognized
parametric experiments is when they reveal similari- words that were not presented. A great deal of
ties that cannot be judged when only a single value research followed the original demonstrations, and
of an independent variable is tested. these experiments varied procedural details, measure-
It is worth noting, too, that parametric experiment techniques, subject characteristics, and so forth.
ments can reveal that apparent intersubject replica- In each instance, therefore, in which the false memory
bility can be misleading regarding how a variable effect was reproduced, the reliability of the phenome-
influences behavior. It is possible that tests with a non was demonstrated and its generality extended.
single value of an independent variable might lead Using the reproduction of previous findings as a start-
to very similar quantitative results for several sub- ing point for subsequent research, therefore, is a use-
jects, whereas a parametric analysis reveals that very ful and productive technique for examining reliability
different functions describing the relation between and generality of research outcomes.
the independent variable happen to cross or come Sidman (1960), in his characterization of tech-
close together at the particular value of the indepen- niques of systematic replication, described a type he
dent variable evaluated. called “systematic replication by affirming the conse-
Parametric experiments illustrate one of the quent” (p. 127). Essentially, this approach is very
strengths of being able to characterize independent similar to the idea of hypothesis testing because the
variables quantitatively. Experiments that determine systematic replication is not based on simply chang-
how much of this yields how much of that provide ing some aspect of the experiment to see whether
more information about generality than do experi- effects can still be reproduced but rather on what the
ments that simply test whether a particular value investigator sees to be the implications of previous
of an independent variable has an effect. They can results. That is, the replication may be based on the
identify similarity where none is evident with a sin- investigator’s interpretation of what the data mean.
gle value of an independent variable, and they can For example, consider our fictitious study of the
also determine whether apparent similarity is effects of carbohydrate content on eating. That
unrepresentative. result, and perhaps those of other experiments,
We should note that parametric experiments are might suggest that the phenomenon is not specific
not limited in application to only primary indepen- to eating. Carbohydrate ingestion possibly leads to
dent variables, such as that shown in our fictitious general lethargy or low motivation for voluntary
example. Any variable associated with an experiment behavior. If we suspect that, we might devise other
can be systematically varied. As an example, the experiments that could be viewed as systematic rep-
experiment just described could be conducted under lications based on the possible implications of the
a range of temperatures, a range of degrees of hydra- previous findings. If the results were consistent with
tion of the subjects, a range of times without food the lethargy interpretation, the view would gain in
before the test, and any of several other variables. credence; if they were not, the view might well be
Those experiments, too, would provide information abandoned. As Sidman (1960) noted, definite con-
about the range of conditions under which the inde- clusions may not be drawn from successful replica-
pendent variable of carbohydrate content exerts its tions by affirming the consequent, but, as he also
effects in the circumstances of the experiment. noted, the approach is essential to science. The
Parametric experiments, although very important, degree to which one’s confidence in an interpreta-
are not the only kind of systematic replications. One tion of data grows with successful replications
165
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
depends on many things, not the least of which is promote further research into the origins of the dif-
how counterintuitive the predicted outcome is. ferences if the differences are judged to be impor-
tant. The scientist and the scientific community
Types of Generality Assessed and make judgments about the need for further investi-
Established by Systematic Replication gation of the differences that are always present in
Johnston and Pennypacker (2009) offered a useful replications.
characterization of the dimensions along which gen- Generality of also plays an essential role in sci-
erality can be examined. They initially suggested a ence. Johnston and Pennypacker (2009) described
dichotomy between “generality of” and “generality several categories of generality of, but here we focus
across.” Generality across is simple to understand. on one in hopes of making the concept clear: gener-
As we have already noted, replication can be used to ality of process. Our example is a behavioral process
determine generality across subjects or situations, a familiar to most psychologists, specifically the pro-
type of generality usually of considerable interest. cess of reinforcement of operant (purposive) behav-
Systematic replication comes to the fore in the ior. Reinforcement refers to the increase in likelihood
assessment of generality across species and across of behavior as a result of earlier instances being fol-
settings. By definition, systematic replication is an lowed by certain consequences, which is the pro-
attempt at replication with something different, so if cess. Systematic replications across an immense
the species is changed, or if something (or a lot) range of both behavioral activities and a very large
about the setting is altered, the replication attempt is range of consequences have been shown to provide
a systematic one. In both cases, the issue of what instances of the process. For example, in addition to
constitutes a successful replication may arise. Con- the traditional lever press and key peck, activities
sider, for example, if we decided to attempt a cross- ranging from the electrical activity of an impercepti-
species replication of our experiments with food ble movement of the thumb (de Hefferline, Keenan,
types, and our new species was a mouse. Obviously, & Harford, 1959), to vocal responses of chicks
mice would eat considerably less, and therefore a (Lane, 1960), to generalized imitation in children
precise, quantitative replication would not be possi- with developmental delays (Baer, Peterson, & Sher-
ble. We might (actually, probably would), however, man, 1967), to the extensive range of activities
argue that the replication was successful if the rela- described in the use of reinforcement in the treat-
tion between carbohydrate content and eating was ment of behavior disorders (e.g., Martin & Pear,
replicated, that is, if at low concentrations there was 2007; Ullman & Krasner, 1966) have all been shown
little effect on eating, but as carbohydrate content as instances of the process. Similarly, the range of
increased, the amount eaten decreased until some events used as effective consequences to produce
level is reached above which further decreases were reinforcement is also broad. Consequences such as
not seen (cf. Figure 7.6). praise, food, intravenous drug administration, open-
What if the content values at which the decreases ing a window, reducing a loud noise, access to exer-
begin and end differ between the species? For exam- cise, and many, many others have been effectively
ple, mice may begin to show a decline when the used to produce reinforcement. All the reports
food reaches 15% carbohydrate, whereas with the may be viewed as describing systematic replications
humans, decreases are not evident until the food of the earliest experiments on the process (e.g.,
contains 25% carbohydrate. Is that a failure to repli- Skinner, 1932; Thorndike, 1898).
cate? Again, the answer is yes and no. The business This generality of process is what stands as the
of science is to find regularities in nature, so empha- rationale for speaking of reinforcement theory. The
sis is properly placed on similarities. Differences argument is similar to that offered for the motion of
virtually always exist, so they are easy to find. Nev- objects. Whatever those objects are, and whether
ertheless, they cannot be ignored entirely, but their they are falling, floating, being ballistically pro-
main role is not to indicate that the similarities jected, or orbiting in outer space, they can be sub-
evident are somehow unimportant, but rather to sumed under the notion of gravitational attraction,
166
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Newton’s theory of gravity. An even more dramatic c omposition, has no effect at all on eating. That, of
example is provided by living things. All manner of course, would be clear indication of a limit to the
plants and animals populate the earth, and their dif- across-subjects generality of the effect on eating. It
ferences are obvious and virtually countless. What is would, however, represent an increase in scientific
less obvious but explains the variety is that all life generality because it specifies more clearly the con-
can be considered to have developed from the opera- ditions required to produce the phenomenon of
tion of three processes: variation, selection, and reduced food intake. As stated by Johnston and
retention (Darwin, 1859). The sameness of cellular Pennypacker (2009), “A procedure can be quite
architecture, including nuclear material (e.g., DNA valuable even though it is effective under a narrow
and RNA), also attests to the similarity. Likewise, all range of conditions, as long as we know what those
the myriad instances of reinforcement suggest that conditions are” (pp. 343–344). The vital role that
considering them instances of a single process is rea- systematic replication, and even failures of system-
sonable. As noted earlier, an important goal of sci- atic replication, can play in establishing scientific
ence is to discover uniformities. In fact, as Duhem generality therefore becomes evident. Scientific gen-
(1954) noted, one of the key features of explanation erality represents an understanding of the variables
is identification of the like in the unlike. Objects responsible for a phenomenon.
look different, are made of different substances, and
may or may not be moving in variety of ways, but
Generalization, Technology
they are similar in how they are affected by gravity.
Transfer, and Translational
Behavioral activities take on many forms, and as just
Research
noted, so can the consequences of those activities.
Nevertheless, they can (on many occasions) exhibit The function of any science is the acquisition of
the phenomenon known theoretically as reinforce- basic knowledge. A secondary benefit is often the
ment, an instance of generality of process. possibility of applying that knowledge in ways that
impart benefit to some element of the culture at
Scientific Generality large. For example, Galileo’s basic astronomic obser-
Another extremely important concept is scientific vations eventually led to improved navigation proce-
generality, a type of generality that has some coun- dures with attendant benefits to the colonial powers
terintuitive characteristics. Scientific generality is of 17th-century Europe. Pasteur’s discovery in 1863
important for at least two reasons. One, scientific of the microorganisms that sour wine and turn it
generality speaks to scientists’ ability to reproduce into vinegar, and the observation that heat would
their own findings and those of other scientists, as kill them, led eventually to the germ theory of dis-
well. Two, scientific generality speaks directly to ease and the development of vaccines.
the possibility of effective application and translation In the case of behavior analysis, a relatively
of laboratory findings to the world at large, as dis- young science, sufficient basic knowledge has been
cussed more fully later in the last section of this acquired to permit vigorous attempts at application.
chapter. Scientific generality is defined by knowl- A discipline known as applied behavior analysis,
edgeable reproducibility. That is, it is not character- discussed extensively elsewhere in Volume 2 of this
ized in terms of breadth of applicability, but instead handbook, is the primary result of these efforts,
in terms of identification of factors that are required although application of the findings of behavior
for a phenomenon to occur. To illustrate the differ- analysis are to be found in a variety of other disci-
ence between scientific generality and, for example, plines including medicine, education, and manage-
generality across people, consider again the fictitious ment, to name but a few.
experiment on food types. Suppose that the original In this section, we describe issues surrounding
experiments were all performed with male subjects. attempts to apply laboratory research findings in the
On an attempt at replication with female subjects, wider world at large. Specifically, we discuss topics
it is discovered that food type, or carbohydrate related to applying research findings from controlled
167
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
laboratory or therapeutic settings to new situations designing the intervention intended to effect the
or less controlled environments. First, we describe change in behavior. That is, it is not always suffi-
the issue of generalization of behavioral treatment cient to simply arrange circumstances that produce
effects from treatment settings to real-world circum- a desired change in behavior in the circumscribed
stances. Then we outline basic general strategies environment in which the treatment is undertaken.
for effective transfer of technologies, taking into Instead, procedures should be used that increase the
account the known scientific generality of behav- probability that the change will be enduring and
ioral processes. Finally, we offer comments on the manifested in those parts of a client’s environment
notion of translational research, a matter of much in which the changes are useful. That insight has
contemporary interest. been followed by the development of general strate-
gies to enhance the likelihood that behavior changes
Generalization of Applications occur not only in the treatment environment but
One of the earliest subjects of discussion that arose also in other appropriate ones.
with the development of behavior therapy and For example, Miltenberger (2008) described
behavior modification techniques was the issue several general strategies that can be used to pro-
referred to as generalization (e.g., Yates, 1970). Spe- mote generalization of treatment effects. The most
cifically, there was concern about whether improve- direct strategy is to arrange for rewards to occur
ments produced in a therapy setting would also immediately after instances of generalization occur.
appear in other, nontherapy (e.g., everyday life) sit- Such an approach essentially entails taking treat-
uations. The term generalization was borrowed from ment to the environments in which it is hoped the
a core behavioral process discovered by experimen- new behavioral patterns will occur. That is, the
tal psychologists, that after learning to respond in a training environment is not explicitly delimited.
particular way in the presence of a particular stimu- Such an approach is now widespread in applied
lus, say frequency of a tone, the same behavior may behavior analysis partly as a consequence of an
also occur in the presence of other more or less sim- emphasis on analyzing reinforcement functions
ilar stimuli, say, other frequencies of the tone. It is before implementing treatment (see Iwata, Dorsey,
an apparently simple logical step to suggest that Slifer, Bauman, & Richman, 1982). This approach
behavior learned in a therapy environment might to problem behavior entails discovering whether
also appear in nontherapy, real-world environments, the behavior is maintained by reinforcement, and if
and when it does so, the result can be called general- it is, identifying what the reinforcers are in the
ization (but see Johnston, 1979, for problems with environments in which the problem behavior
such a simple extrapolation). Because applied occurs. Once the reinforcers responsible for the
behavior analysis generally involves establishing maintenance of the problem behavior are identi-
conditions that alter behavior, the issue of whether fied, then procedures based on that knowledge are
those changes are restricted to the learning situa- implemented in the situations in which the behav-
tions arranged or whether they become evident in ior occurs.
other situations is usually important. For example, A related second strategy identified by Milten-
if a child who engages in aggressive behavior is berger (2008) is consideration of the conditions
exposed to a treatment program to reduce aggres- operating in the environments in which the changed
sion, a goal would be to decrease aggression not behavior would be occurring. The idea here is that
only in the treatment setting but in all settings. behavior that is changed in the therapeutic setting,
In a seminal article, Stokes and Baer (1977) dis- for example learning effective social skills, will lead,
cussed the issue of generalization of treatment if performed, to more satisfying social interactions
effects. A key contribution of their article was to in the nontherapy environment, and those successes
indicate that in general, if effects of a treatment are will help to solidify the gains made in the therapy
to be manifested in a variety of circumstances, sessions. In designing the therapeutic goals, there-
achieving that outcome must be considered in fore, consideration is given to what sorts of behavior
168
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
are most likely to be successful in the nontherapy In this section, we focused on particular strate-
environment. gies for ensuring that desired changes in behavior
A less obvious strategy applies when the nonther- established through therapeutic methods occur and
apy environment appears to offer little or no support persist in nontraining or nontherapy environments,
for the changed behavior. An example is when ther- that is, in the everyday world. Employment of tactics
apy is aimed at training an adolescent to walk away emerging from the strategies described has yielded
from aggressive provocation in a schoolyard. Behav- many successes, and the methods are part of the
ing in such an aggression-thwarting manner is not armamentarium of applied behavior analysts. These
likely to result in positive outcomes with peers, who techniques to promote generalization of behavior
are instead likely to provide taunts and jeers after changes have emerged from a consideration of
such actions. In such a case, it may be prudent to try fundamental behavioral processes that have been
to change the normal consequences in the school- identified and analyzed in basic research and then
yard by having teachers or other monitors provide subsequently validated as effective through applied
positive consequences (perhaps in the form of privi- research. They represent, consequently, what can be
leges, praise, etc.) for such actions when they occur. called successful transfer from basic science to effec-
That is, the strategy here involves altering the con- tive technology, namely, an instance of what has
tingencies operating in the nontherapy environment. come to be called technology transfer. In the next
A fourth general strategy is to try to make the section, we discuss some general principles of effec-
therapy setting more like the nontherapy environ- tive technology transfer.
ment in which the changed behavior is to occur. A
study by Poche, Brouwer, and Swearingen (1981) Technology Transfer
illustrated this approach. They taught abduction People often use the term technology to refer to the
prevention skills to preschool children, but in so body of applied knowledge and practices that ema-
doing incorporated a relatively large number of nate from basic science. The term technology transfer
abduction lures in the training. The intent was that refers to the process by which the discoveries or
by including a wide variety of possible lures that inventions of basic science actually make their way
might be used by would-be kidnappers, the training into the body of technology and become available
would be more effective in real-world situations for use outside the laboratory by any individual
than if it had not involved those variations. The gen- willing to undergo the expense of acquiring the
eral strategy in this case was to train with as many technology. Technology transfer can occur with its
likely relevant situations as possible. Another way to basis in any science, so general principles exist that
view this strategy is that it involves incorporating apply across scientific disciplines. The process is
stimuli that are present in the nontherapy environ- somewhat complex, and pausing to review some
ment into the training. of the basic details that apply when a technology is
A fifth approach is somewhat less intuitive, but brought to the commercial level will be helpful. The
research has suggested that it may be effective. The criteria, both legal and practical, that must be met
core idea is that if a variety of different forms of for successful technology transfer set an upper limit
effective behavior are established by the therapy or against which applications of science can be
training, the chance of effective behavior occurring evaluated.
in the nontherapy environment is better, and as a A discovery or invention is an item of intellectual
result the successful behavior will be supported and property. It belongs to the inventor or discoverer or
continue to occur. As a simple illustration, Milten- whoever sponsored the research that led to its exis-
berger (2008) offered the example of teaching a shy tence. The transfer process usually involves a second
person a variety of specific ways to ask for a date, party taking ownership of the property and thus the
which provides the person with several actions to right to exploit it for commercial gain. It therefore
try, some of which are likely to be successful outside must be protected, usually by means of a patent or
of therapy. copyright. Once ownership is secured, it can be
169
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
transferred in exchange for a lump sum payment application by pharmaceutical companies of the
or, more often, a license agreement whereby the reproducible effects of reinforcement schedules in
licensee acquires the exclusive right to produce and evaluating drugs. A standard approach was to estab-
distribute the technology and the licensor is entitled lish a known baseline of performance by an animal
to a royalty based on sales. Thus, for example, the using a particular schedule of reinforcement, then
Quaker Oats Company executed a licensing agree- evaluate the perturbation, if any, of an experimental
ment with the University of Florida that allowed the compound on that performance. If the perturbation
company to produce and distribute Gatorade exclu- could be reliably reproduced (repeated), the relation
sively in exchange for a royalty in the form of a per- between the compound and its effect on behavior
centage of revenue. was affirmed (cf. McKearney, 1975; Sudilovsky,
Gershon, & Beer, 1975).
Requirements for Technology Transfer Verification is most akin to the concept of gener-
For a candidate technology to meet the eligibility ality. In establishing the generality of a behavioral
requirements for transfer, it must meet three crite- process or phenomenon, researchers seek to specify
ria: quantification, repetition, and verification. the range of conditions under which the phenome-
These terms arise from the engineering literature non occurs while eliminating those that are irrele-
(e.g., Hench, 1990); their counterparts in the behav- vant or extraneous. Similarly, when transferring a
ioral literature will become obvious in a moment. technology, the recipient must be afforded a com-
Let us consider these characteristics in more detail plete specification of the necessary and sufficient
and see how they conform to the products of behavior- conditions for reproduction of the effects of the
analytic research. technology. Extensive research to nail down the
First, we discuss quantification. Behavior analy- parameters of generality is the only way to achieve
sis has long used the measurement strategies of the this objective. It cannot be obtained by appeal to the
natural sciences (Johnston & Pennypacker, 1980) results of significance tests for all of the reasons
with the result that, as Osborne (1995) stated, detailed earlier.
A simple yet elegant example of a well-established
Physical standards of measurement bind
behavioral technology that has easily transferred is
behavior analysis to the physical and
the Invisible Fence for animal control, which is a
natural sciences. Interpretation of depen-
direct application of the principles of signaled avoid-
dent variables need not change from
ance. A wire is buried underground around the
experiment to experiment. It is a feature
perimeter of the enclosed area in which the animals
of . . . idemnotic measures that response
are to remain. The animal wears a collar that receives
frequencies on a particular parameter of a
auditory signals (beeps) and delivers electric shocks
fixed-ratio schedule of reinforcement can
through two electrodes that contact the animal’s
be compared validly within sessions and
neck. As the animal comes within a few feet of the
across sessions, within laboratories and
wire, a beep sounds. If the animal proceeds, it
across laboratories, within species and
receives a shock. The owner teaches the animal to
across species. (p. 249)
withdraw on hearing the beep by carrying (or lead-
We are therefore able to state precisely and unam- ing) the animal into the proximity of the beep, then
biguously the quantitative characteristics of behav- shouting “No!” and carrying or leading the animal
ior resulting from application of a particular away from the beep. After several such trials, the ani-
procedure. mal is released and will eventually receive the shock.
Repetition is the practical use of replication as Its escape response has been well learned, and it will
discussed earlier. The phenomenon must be able to very likely never contact the shock again. Rather, it
be reproduced at will for it to serve as a component will avoid it when it hears the beep.
of a transferrable technology. An early (late 1950s A more elaborate example of a behavioral tech-
and early 1960s) example of this feature is the nology that has been successfully transferred is the
170
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
MammaCare Method of manual breast examination analysis were not ready to enjoy the stimulative
as a means of early detection of breast cancer effects of the Bayh–Dole Act at the time it became
(Pennypacker, 1986). This example is unusual law. Analogous federal attention to the clinical and
because the basic research was conducted by the behavioral sciences has emerged in the form of the
same individuals who eventually took the technol- National Institutes of Health’s National Center for
ogy to market. A capsule history of the development Research Resources, which makes Clinical and
of MammaCare and its subsequent transfer is avail- Translational Science Awards. Mace and Critchfield
able online (Pennypacker, 2008). (2010) cited a statement by the acting director of the
In brief, a high-fidelity simulation of human National Institutes of Health’s Office of Behavioral
breast tissue was created with the help of materials and Social Sciences to the effect that “its [the insti-
science engineers. Patent protection for this device tute’s] mission is science in pursuit of fundamental
was obtained. Basic psychophysical research using knowledge about the nature and behavior of living
this simulation was conducted to determine the lim- systems and the application of that knowledge to
its of detectability by human fingers of lifelike simu- extend healthy life and reduce the burdens on ill-
lations of breast tumors and other abnormalities. ness and disability” (p. 307).
Once these limits were known, behavioral studies The aim is to accelerate the application of basic
isolated the components of a technique that allowed biomedical research to clinical problems. We now
an examiner to approach the established psychophys- turn our attention to this type of research.
ical limits. Early translational research established
that practicing the resulting techniques on the simu- Translational Research
lation enabled examiners to detect real lumps in live From our perspective, translational research is a
breast tissue at about twice their baseline accuracy. node along the continuum from basic bench science
Extensive research was then undertaken to establish to the sort of application that results from technol-
procedures for teaching the new examination tech- ogy transfer. The continuum is abstractly defined
nique, which became known as MammaCare. by generality. The ultimate goal of translational
Technology transfer became possible when stan- research may be broadly seen as establishing the
dards of performance were established and training limits of generality of whatever variable, process, or
methods were devised that could be readily repeated procedure is of interest.
and whose results could be verified. As a result, Translational research is therefore a somewhat
individuals wanting to offer such training either to less stringent endeavor than full technology transfer
the public or to other medical professionals (people to bridge the gap from bench to bedside. A distin-
who routinely perform clinical breast examinations, guishing feature of this approach is that the basic
e.g.) may now become certified and can operate scientist and clinician often collaborate in a syner-
independently. gistic manner. This practice will likely accelerate the
Technology transfer was greatly accelerated in development of candidate technologies because the
the United States by the passage in 1980 of the applied aspect is undergoing constant examination
Bayh–Dole Act, which made it possible for institu- and refinement.
tions conducting research with federal funds to Lerman (2003) has provided an excellent over-
retain the products of that research. Offices of view of translational research in behavior analysis.
licensing and technology soon appeared in all of the She correctly observed that the bulk of the literature
major research universities, which in turn began on applied behavior analysis consists of reports of
licensing to private organizations the products of translational research. She went on to describe a
their sponsored research. The resulting revenue in series of maturational stages of this endeavor, begin-
the form of fees and royalties has been of significant ning with the early demonstrations of the generality
benefit to these institutions. Most of this activity, of the process of reinforcement across species,
however, has taken place in the hard sciences, individuals, and settings. From these emerged con-
engineering and medicine. Fields such as behavior cerns with other basic processes such as extinction,
171
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
stimulus generalization, discrimination, and the research in behavior analysis. They stressed the
effects of basic contingencies of reinforcement. importance of conducting research that can be of
As these types of demonstrations increasingly more or less immediate benefit to society if substan-
proved beneficial to clinical populations, a new dimen- tial societal support for such research is to occur. In
sion of this research emerged that focused on issues of our view, the likelihood of such research actually
training, maintenance of benefits, and even the implica- attaining that criterion would be augmented to the
tions of such practices for public policy. Concurrently, extent that researchers keep as their ultimate goal
focus shifted from individual cases to larger entities development of a technology that can be transferred
such as schools, corporations, and even military units. as we have discussed.
At the same time, a small but growing body of In fact, very few examples of commercially trans-
translational research will explicitly hasten the devel- ferrable technology have yet to emerge from behav-
opment of mature technologies that can be transferred ior analysis (Pennypacker & Hench, 1997). There is,
with the usual attendant financial and cultural bene- however, sufficient promise in the replicability of the
fits. One such effort is a study by St. Peter Pipkin, discipline’s basic findings to encourage development
Vollmer, and Sloman (2010), who examined the of transferrable technologies, and the availability of
effects of decreasing adherence to a schedule of differ- substantial research support is critical. More transla-
ential reinforcement of alternative behavior, first in a tional research aimed at identifying and isolating the
laboratory setting and then, using the exact same conditions under which specified procedures can be
reinforcement schedule parameters, in an educational assured to consistently generate measurable and
setting with two individuals with educational handi- desirable effects on the behavior of individuals will
caps. They explored the generality of a procedure hasten the emergence of such technologies.
across settings and populations and further docu-
mented the effects of deliberate failure to impose the
Summing Up
differential reinforcement of alternative behavior
schedule as specified, either by not delivering rein- The subdiscipline known as behavior analysis offers
forcement when required or by “accidentally” allow- an alternative approach to assessing reliability and
ing undesirable behavior to contact reinforcement at generality of research findings, that is, an approach
various times. These manipulations constitute an that is different from that used by most psychologi-
attempt to demonstrate the consequences of break- cal researchers today. The methods that provide ave-
downs in treatment integrity, which in some cases can nues to assessing reliability and generality may be of
be highly destructive and in others may be negligible. interest to psychologists who approach the field of
The type of translational research just described behavior (or mind) from perspectives other than
constitutes an important step toward the develop- those shared by behavior analysts. At this juncture
ment of a technology that can be transferred in in the history of behavioral science, the methods
the sense discussed earlier. Treatment integrity is might be especially attractive to researchers who are
directly analogous to what the engineers call verifi- coming into contact with the substantial limitations
cation. St. Peter Pipkin et al. (2010) provided guid- of traditional methods that rely on group averages
ance for establishing a range of allowable treatment and null-hypothesis significance testing. In the early
integrity failure within which effectiveness may be sections of this chapter, we reiterated those weak-
maintained, which is akin to specifying tolerances in nesses because it appears that many behavioral
a manufacturing process or allowable failure rates of researchers are not aware of them.
components in complex equipment. Our main thrust in this chapter has been to
describe and characterize types of replication and
General Considerations for the roles that they play in determining the reliability
Translational Research and generality of research outcomes. We have espe-
Mace and Critchfield (2010) offered an informative cially emphasized the role replication can play in
perspective on the current role of translational assessing the generality of research findings, both
172
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
across subjects and conditions and of theoretical Carver, R. P. (1978). The case against statistical sig-
assertions. We have argued, in fact, that replication, nificance testing. Harvard Educational Review, 48,
378–399.
both direct and systematic, currently represents the
Cleveland, W. (1994). The elements of graphing data.
only set of methods that can determine whether
Summit, NJ: Hobart Press.
results are reliable and how general they are. Our
Cohen, J. (1994). The earth is round (p < .05). American
claim is a strong one, and it is not that replication is Psychologist, 49, 997–1003. doi:10.1037/0003-066X.
an alternative set of methods but rather that it is the 49.12.997
only way to determine reliability and generality Darwin, C. (1859). On the origin of species by means of
given current knowledge. Replication has served the natural selection, or the preservation of favoured races
more developed sciences very effectively, and it is in the struggle for life. London, England: John Murray.
our contention that it can serve behavioral science, Deese, J. (1959). On the prediction of occurrence of par-
too. At the very least, we hope we have convinced ticular verbal intrusions in immediate recall. Journal
of Experimental Psychology, 58, 17–22. doi:10.1037/
the reader that paying greater attention to replica- h0046671
tion will advance behavioral science more surely and de Hefferline, R. F., Keenan, B., & Harford, R. A. (1959).
more rapidly than the methods currently in fashion. Escape and avoidance conditioning in human sub-
In the final section of the chapter, we focused on jects without their observation of the response.
issues of applying science to problems the world Science, 130, 1338–1339.
faces. Once reliability and generality of research Duhem, P. (1954). The aim and structure of physical theory
(P. P. Wiener, Trans.). New York, NY: Princeton
findings have been established to an appropriate University Press.
degree, it is sometimes possible to take advantage of
Dunn, K. E., Sigmon, S. C., Thomas, C. S., Heil, S. H.,
that knowledge for the betterment of people and & Higgins, S. C. (2008). Voucher-based contin-
society. There are guideposts about how best to do gent reinforcement of smoking abstinence among
that, and we have discussed some of them. methadone-maintained patients: A pilot study.
The other chapters in this handbook present a doi:10.1901/jaba.2008.41-527
wide-ranging description of the research and appli-
Falk, R., & Greenbaum, C. W. (1995). Significance tests
cation domains that constitute behavior analysis. die hard: The amazing persistence of a probabilistic
We see those chapters as testament to the coherent misconception. Theory and Psychology, 5, 75–98.
science and technology that can be developed when doi:10.1177/0959354395051004
the markers of reliability and generality have been Gigerenzer, G., Krauss, S., & Vitouch, O. (2004).
established through research founded on direct and The null ritual: What you always wanted to know
about significance testing but were afraid to ask. In
systematic replication. D. Kaplan (Ed.), The Sage handbook of quantitative
methodology for the social sciences (pp. 391–408).
References Thousand Oaks, CA: Sage.
Anscombe, F. J. (1973). Graphs in statistical analysis. Haller, H., & Krauss, S. (2002). Misinterpretations of
American Statistician, 27, 17–21. doi:10.2307/2682899 significance: A problem students share with their
teachers? Methods of Psychological Research—Online,
Baer, D. M., Peterson, R. F., & Sherman, J. A. (1967). The 7(1), 1–20.
development of imitation by reinforcing behavioral
Hench, L. L. (1990, August). From concept to commerce:
similarity to a model. Journal of the Experimental
The challenge of technology transfer in materials.
MRS Bulletin, pp. 49–53.
jeab.1967.10-405
Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E.,
Bakan, D. (1966). The test of significance in psychologi- & Richman, G. S. (1982). Toward a functional
cal research. Psychological Bulletin, 66, 423–437. analysis of self-injury. Analysis and Intervention in
doi:10.1037/h0020412 Developmental Disabilities, 2 3–20.
Bayh-Dole Act, Pub. L. 96-517, § 6(a), 94 Stat. 3018. Johnston, J. M. (1979). On the relation between general-
(1980). ization and generality. Behavior Analyst, 2, 1–6.
Bernard, C. (1957). An introduction to the study of experi- Johnston, J. M., & Pennypacker, H. S. (1980). Strategies
mental medicine. New York, NY: Dover. (Original and tactics of human behavioral research. Hillsdale,
work published 1865) NJ: Erlbaum.
173
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Johnston, J. M., & Pennypacker, H. S. (2009). Strategies account of a review of Johnston and Pennypacker’s
and tactics of behavioral research (3rd ed.). New York, Strategies and Tactics of Behavioral Research (2nd ed.)
NY: Routledge. and others. Journal of the Experimental Analysis of
Kalinowski, P., Fidler, F., & Cumming, G. (2008).
Overcoming the inverse probability fallacy: A com- Pennypacker, H. S. (1986). The challenge of technol-
parison of two teaching interventions. Methodology: ogy transfer: Buying in without selling out. Behavior
European Journal of Research Methods for the Analyst, 9, 147–156.
Behavioral and Social Sciences, 4, 152–158. Pennypacker, H. S. (2008). A funny thing happened on
Lane, H. (1960). Control of vocal responding in chickens. the way to the fortune, or lessons learned during 25
Science, 132, 37–38. doi:10.1126/science.132.3418.37 years of trying to transfer a behavioral technology.
Behavioral Technology Today, 5, 1–31. Retrieved from
Lerman, D. C. (2003). From the laboratory to community http://www.behavior.org/resource.php?id=188
application: Translational research in behavior analy-
sis. Journal of Applied Behavior Analysis, 36, 415–419. Pennypacker, H. S., & Hench, L. L. (1997). Making
doi:10.1901/jaba.2003.36-415 behavioral technology transferrable. Behavior
Analyst, 20, 97–108.
Loftus, G. R. (1991). On the tyranny of hypothesis test-
ing in the social sciences [Review of The empire of Penston, J. (2005). Large-scale randomized trials—A mis-
chance: How probability changed science and everyday guided approach to clinical research. Medical Hypotheses,
life]. Contemporary Psychology, 36, 102–105. 64, 651–657. doi:10.1016/j.mehy.2004.09.006
Poche, C., Brouwer, R., & Swearingen, M. (1981).
Loftus, G. R. (1996). Psychology will be a much better
Teaching self-protection to young children. Journal of
science when we change the way we analyze data.
Current Directions in Psychological Science, 5,
jaba.1981.14-169
161–171. doi:10.1111/1467-8721.ep11512376
Roediger, H., & McDermott, K. (1995). Creating false
Mace, F. C., & Critchfield, T. S. (2010). Translational memories: Remembering words not presented in
research in behavior analysis: Historical tradi- lists. Journal of Experimental Psychology: Learning,
tions and imperative for the future. Journal of the Memory, and Cognition, 21, 803–814. doi:10.1037/
Experimental Analysis of Behavior, 93, 293–312. 0278-7393.21.4.803
doi:10.1901/jeab.2010.93-293
Rozeboom, W. W. (1960). The fallacy of the null hypoth-
Martin, G., & Pear, J. (2007). Behavior modification: What esis significance test. Psychological Bulletin, 57,
it is and how to do it (8th ed.). Upper Saddle River, 416–428. doi:10.1037/h0042040
NJ: Pearson.
Sidman, M. (1960). Tactics of scientific research. New
McKearney, J. W. (1975). Drug effects and the environ- York, NY: Basic Books.
mental control of behavior. Pharmacological Reviews,
27, 429–436. Skinner, B. F. (1932). On the rate of formation of a con-
ditioned reflex. Journal of General Psychology, 7,
Meehl, P. E. (1967). Theory-testing in psychology and 274–286. doi:10.1080/00221309.1932.9918467
physics: A methodological paradox. Philosophy of
Smithson, M. (2003). Confidence intervals. London,
Science, 34, 103–115.
England: Sage.
Meehl, P. E. (1978). Theoretical risks and tabular aster- Stokes, T. F., & Baer, D. M. (1977). An implicit technol-
isks: Sir Karl, Sir Ronald, and the slow progress of ogy of generalization. Journal of Applied Behavior
soft psychology. Journal of Consulting and Clinical Analysis, 10, 349–367. doi:10.1901/jaba.1977.10-349
Psychology, 46, 806–834. doi:10.1037/0022-006X.
46.4.806 St. Peter Pipkin, C., Vollmer, T. R., & Sloman, K. N.
(2010). Effects of treatment integrity failures during
Miltenberger, R. G. (2008). Behavior modification: differential reinforcement of alternative behavior:
Principles and procedures (4th ed.). Belmont, CA: A translational model. Journal of Applied Behavior
Thompson. Analysis, 43, 47–70. doi:10.1901/jaba.2010.43-47
Nickerson, R. S. (2000). Null hypothesis significance test- Sudilovsky, A., Gershon, S., & Beer, B. (Eds.). (1975).
ing: A review of an old and continuing controversy. Predictability in psychopharmacology: Preclinical and
Psychological Methods, 5, 241–301. doi:10.1037/ clinical correlations. New York, NY: Raven Press.
1082-989X.5.2.241
Thorndike, E. L. (1898) Animal intelligence: An experi-
Oakes, M. (1986). Statistical inference: A commentary mental study of the associative processes in animals.
for the social and behavioral sciences. Chichester, Psychological Review, 11(4, Whole No. 8).
England: Wiley.
Tukey, J. W. (1969). Analyzing data: Sanctification or
Osborne, J. G. (1995). Reading and writing about detective work? American Psychologist, 24, 83–91.
research methods in behavior analysis: A personal doi:10.1037/h0027108
174
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Tukey, J. W. (1977). Exploratory data analysis. Reading, and explanations. American Psychologist, 54, 594–604.
MA: Addison-Wesley. doi:10.1037/0003-066X.54.8.594
Ullman, L. P., & Krasner, L. (Eds.). (1966). Case stud- Williams, B. A. (2010). Perils of evidence-based medicine.
ies in behavior modification. New York, NY: Holt, Perspectives in Biology and Medicine, 53, 106–120.
Rinehart & Winston. doi:10.1353/pbm.0.0132
Wilkinson, L., & Task Force on Statistical Inference. (1999). Yates, A. J. (1970). Behavior therapy. New York, NY:
Statistical methods in psychology journals: Guidelines Wiley.
175
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 8
Single-Case Research Designs

and the Scientist-Practitioner
Ideal in Applied Psychology
Neville M. Blampied
Darwinism implies . . . an intense aware- this model and argue that the adoption of single-
ness that all categorical or essentialist case research strategies provides an effective solu-
claims about living things are overdrawn— tion to many of these problems.
anyone who says that all cases of this
thing or that thing are naturally one way
Psychology Defines Science—The
or another are saying something that isn’t
Inference Revolution
so. . . . Repetition is the habit of nature,
but variation is the rule of life. . . . Belief John Arbuthnot (Gigerenzer, 1991; Kendall &
in the primacy of the single case is not Plackett, 1977), a physician and mathematician
an illusion nurtured by fancy but a hope in the household of Queen Anne, is said to have
quietly underscored . . . by science. The proved the existence of God (a “wise creator”) in
general case is the tentative abstract 1710 by way of a kind of significance test, but it was
hypothesis; the case right there is the real not until the middle years of the 20th century that
thing. (Gopnik, 2009, pp. 197–198) psychology made significance testing into a kind of
god. Before these developments, psychologists had
How to cope with variation within repetition; how used a range of tabular and especially graphical
to balance the abstract and the particular—these techniques for analyzing what were often complex
issues, so deftly stated by Gopnik (2009), challenge data with large sample sizes (Smith, Best, Cylke, &
all science, including behavioral science. In this Stubbs, 2000). Smith et al. (2000) noted that
chapter, I consider how psychology responded to “history clearly shows that worthy contributions to
these challenges by adopting the now-dominant psychology do not inherently depend on the bene-
paradigm for research design and analysis within diction of p values. Well into the 20th century psy-
the discipline and how this strongly influenced the chologists continued to produce lasting, even
adoption of an ideal model—the scientist-practitioner canonical, achievements without using inferential
model—for applying science. I then consider prob- statistics” (p. 262).
lems and difficulties that have arisen with both The changes in both research practice and data
research and its application through adherence to analysis that occurred in (Western) psychology in
I acknowledge my profound debt to Murray Sidman and thank him and Rita Sidman for their many kindnesses. I also acknowledge my enduring
gratitude to the academic staff of the Department of Psychology, University of Auckland, 1964–1969, for their rigorous introduction to the science of
behavior, and especially to John Irwin, who introduced me to the writings of B. F. Skinner. I am also grateful for the opportunity to discuss research
methodology with my friend and colleague Brian Haig and for his comments on this chapter. I also acknowledge the assistance provided by an unpub-
lished thesis by K. J. Newman (2000) in providing very useful background on the scientist-practitioner model of clinical psychology; helpful com-
ments from Martin Dorhay; and many excellent suggestions from the editors, Kennon A. Lattal and Gregory J. Madden. Assistance in the preparation
of this chapter was provided by Emma Marshall, who was supported by a Summer Scholarship jointly funded by the University of Canterbury and the
Tertiary Education Commission of New Zealand.
DOI: 10.1037/13937-008
177
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
the period from 1935 to 1955 have been called an 1940. These early examples were largely of applied
inference revolution (Gigerenzer, 1991) and have research, and it was from applied psychology that
been claimed by some to have had many of the inferential statistics spread to experimental research
properties of a Kuhnian paradigm shift (Rucci & (Gigerenzer et al., 1989). Fisher’s statistical tests
Tweney, 1980). The rapidity and thoroughness of were not the only practices adopted. His advocacy of
this revolution was remarkable, and it had profound factorial designs, that is, experiments investigating
and lasting consequences, not least because it com- more than one level of an independent variable, was
prehensively redefined, in an operational way, what also influential, reflecting his view that experimental
“science” was and is and ought to be in the context design and statistical analysis are “different aspects
of psychological research and, therefore, what the of the same whole” (Fisher, 1935, p. 3).
science part of a scientist-practitioner should be. There was a hiatus in the dissemination and
The inference revolution was foreshadowed adoption of Fisher’s statistical methods during
toward the end of the 19th century by Galton, the World War II, but there was then an acceleration
inventor of the correlation coefficient and the con- in the postwar years, so by the mid-1950s, Fisher’s
trol group (Dehue, 2000; Gigerenzer et al., 1989), statistical tests were widely reported in published
who wrote approvingly about “scientific men” who articles across many psychology journals; many
would “devise [statistical] tests by which the value textbooks had been published to assist with the
of beliefs may be ascertained” and who would “dis- teaching of these methods; and leading universities
card contemptuously whatever may be found to be were requiring graduate students to take courses in
untrue” (as quoted in Schlinger, 1996, p. 72). The these methods (Hubbard, Parsa, & Luthy, 1997;
foundations of the revolution were, however, not Rucci & Tweney, 1980). Since that time, more than
laid until the 1920s when R. A. (Sir Ronald) Fisher, 80% of published articles in psychology journals
statistician, geneticist, and evolutionary biologist, have typically reported significance tests (Hubbard
published his profoundly influential works (Wright, et al., 1997).
2009). Statistical Methods for Research Workers Thus, by the time the inference revolution was
(Fisher, 1925) and The Design of Experiments complete—in the mid-1950s—psychology, or at
(Fisher, 1935) gave the world factorial designs, least the academic, English-speaking domain of the
randomization of research elements to treatment discipline, had developed a consensus about what
conditions, the analysis of variance, the analysis it meant to be a science and to do science.1 The key
of covariance, the null hypothesis (i.e., the hypothe- attributes of this model of science are outlined in
sis to be nullified), and null-hypothesis significance Exhibit 8.1. In a mutually strengthening cycle, they
tests (NHST; Yates & Mather, 1963). Although were taught in the methods courses, written about
some fellow statisticians were suspicious (Yates & in the methods textbooks, practiced in the labora-
Mather, 1963), Fisher’s ideas spread rapidly, espe- tory, required by editors, published in the journals,
cially in biology (see Appendix 8.1 for further infor- and imitated by other researchers. Until recently,
mation on, and definition of terms for, NHST). this standard model of psychological science
Given that Fisher was working as an agricultural (henceforth termed the standard group statistical
scientist and geneticist at the time he wrote these model, or standard model for short) has gone largely
books, the spread of his ideas to biology was expli- unchallenged in the mainstream of the discipline.
cable. Rather more surprising was the rapid adop- As it also happened, this consensus about scientific
tion of Fisher’s methods by psychologists. Rucci and methods was achieved at about the same time as
Tweney (1980) identified 1935 as the first year psy- major developments were also occurring in the
chological research using analysis of variance was understanding of clinical psychology, the topic to
published, with 16 further examples published by which I now turn.
This consensus adopted most of Fisher’s approach to statistics and research design but also incorporated, in an ad hoc and anonymous way, aspects of
1
Jerzy Neyman and Egon Pearson’s perspective (Gigerenzer, 1991; Gigerenzer et al., 1989; Hubbard, 2004; see Appendix 8.1).
178
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Single-Case Research Designs and the Scientist-Practitioner Ideal in Applied Psychology
Bootzin, 2007; Korchin, 1983; O’Donnell, 1985;

Exhibit 8.1 Reisman, 1966, 1991; Routh, 1998), although its
Key Characteristics of the Standard Model origins can be traced back to antiquity (Porter,
of Research in Psychology 1997; Routh, 1998). By the beginning of the 20th
century, clinics in both Europe and the United
Recruit as manya participantsb as possible. States were devoted to the study of mental disease,
(Quasi)randomly allocate these participants to experimental mental retardation, and children’s educational and
conditions. adjustment problems, in which a nascent clinical
Acquire one or a few measures of the dependent variable
psychology was evident (Bootzin, 2007). The most
from each participant.
Use measures of central tendency (mean, median, etc) and unequivocally psychological of these was the clinic
variance to aggregate data for each dependent variable. established by Lightner Witmer at the University
Compute inferential statistics one dependent variable at of Pennsylvania in 1896 (McReynolds, 1997).
a time, comparing the aggregated measures for each
Witmer’s clinic focused on children (McReynolds,
group.
Operate under a null hypothesis (H0) for which the population 1997; Reisman, 1966), and it was a place where they
mean differences are zero. could be assessed by psychologists (and social work-
Use p values relative to a criterion to make an accept– ers), drawing on the psychologists’ knowledge of
reject decision concerning the null hypothesis and any
alternative hypothesis.
sensory, perceptual, and cognitive systems and,
Regard results of the study as being of scientific value only where possible, remedial actions could be pre-
if H0 has been rejected at the criterion level. scribed. Witmer presciently
aPower analysis permits the computation of a minimum emphasized two points: one, as a teacher,
sample size for a given level of power. Investigators are the clinician was to conduct demon-
still encouraged to regard larger samples as preferable
to smaller samples (e.g., Streiner, 2006). strations . . . so that [students] would
bIn principle, these participants are regarded as being be instructed in the art and science of
drawn from some specified population, but in psycho- psychology; two, as a scientist, the clini-
logical research the population of interest from which cian was to regard each case as in part a
the sample is considered to have been drawn is rarely research experiment in which the effects
specified with any precision (Wilkinson & Task Force
on Statistical Inference, 1999), an omission that strictly of his procedures and recommendations
negates the drawing of inferences from the sample. were to be discovered. (Reisman, 1966,
p. 86).
Bootzin (2007) noted that these small experi-
Clinical Psychology and the Rise ments anticipated the development of single-case
of the Scientist-Practitioner Ideal research methods in psychology, with Witmer him-
self emphasizing his “belief that we shall more profit-
O’Donnell (1985) argued that in the period from ably investigate these causes [of development] by the
about 1870 until World War I, the emergent disci- study of the individual” and that he studied “not the
pline of psychology had to surmount three major abnormal child but . . . the individual child” (Witmer,
challenges: to differentiate itself from philosophy, 1908, as quoted in McReynolds, 1997, p. 142).
to avoid becoming a branch of physiology, and to Witmer established courses for training graduate
become seen as sufficiently useful to society that students in practical psychology at the University of
both the discipline and its practitioners garnered Pennsylvania in 1904–1905 and founded a journal—
institutional, social, and economic support. From The Psychological Clinic—in March 1907. In the ini-
this matrix of historical processes, especially the tial issue of this journal, Witmer (1907/1996, p. 251)
last, emerged clinical psychology. Consistent with first used the term clinical psychology—“The methods
this argument, there is agreement among historians of clinical psychology are . . . invoked wherever the
of psychology that clinical psychology had its imme- status of an individual mind is determined by obser-
diate origins in the late 1900s (e.g., Benjamin, 2005; vation and experiment, and pedagogical treatment
179
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
applied to affect a change”—and thus signaled the in the United States doubled, and this increase con-
founding of a new field of psychology. Other U.S. tinued apace in subsequent years (Reisman, 1991).
universities soon established clinics in the Witmer This growth stimulated further institutional
model; a few hardy professional psychologists activity, and in August 1949 a famous conference,
worked outside academia, all of them in institutions the Boulder Conference on Graduate Education in
serving people with intellectual disabilities; and at Clinical Psychology, was held at the University of
least one institution offered an internship (Reisman, Colorado in Boulder (Raimy, 1950). This confer-
1966; Routh, 2000). So was clinical psychology born. ence endorsed the recommendations of the Shakow
The mature form with which psychologists and committee and gave an official imprimatur to what
the public are familiar did not develop, however, is now termed the Boulder model for the training
until toward the end of World War II. As a profes- of applied and clinical psychologists as scientist-
sion, psychology had grown in numbers and influ- practitioners (Bootzin, 2007). This scientist-
ence during the war. The end of the war produced a practitioner ideal, although contested and disputed,
rapidly growing gap between professional resources, has ever since remained the dominant ideal of
especially the number of clinical psychologists, and the clinical psychologist throughout the English-
the needs of postwar society. To meet these needs, speaking world (e.g., Martin, 1989; Shapiro, 2002).
more graduates in clinical and applied psychology What this ideal entails has been specified in different
in general were required. A committee of the Ameri- ways, but Barlow, Hayes, and Nelson (1984) sug-
can Psychological Association, chaired by David gested that it specified three aspects or roles for
Shakow, worked to develop appropriate graduate practitioners as (a) consumers of new research,
curricula, accreditation processes, and internship (b) empirical evaluators of their own practice, and
opportunities and funding (Committee on Training (c) producers of scientifically credible research from
in Clinical Psychology, 1947; Reisman, 1991). The their own practice setting.2
committee defined clinical psychology as having Although developed specifically in regard to clin-
“systematic knowledge of human personality and . . . ical psychology, the model in principle embraced a
principles and methods by which it may use this wide understanding of applied psychology, so that
knowledge to increase the mental well being of the many forms of practice could be included (B. B.
individual” (Committee on Training in Clinical Psy- Baker & Benjamin, 2000; Raimy, 1950). It has been
chology, 1947, p. 540). adopted by other applied areas, such as counseling
The report further defined clinical psychology psychology (e.g., Corrie & Callahan, 2000; Vespia &
as “both a science and an art” and stated that the Sauer, 2006), health psychology (e.g., Sheridan et al.,
graduate must have “applied and theoretical knowl- 1989), neuropsychology (e.g., Rourke, 1995), orga-
edge in three major areas: diagnosis, therapy, and nizational and personnel psychology (e.g., Jex &
research” (Committee on Training in Clinical Psy- Britt, 2008), and school (educational) psychology
chology, 1947, p. 540). Clinical training programs (e.g., Edwards, 1987).
should include psychological research methods Given the close coincidence in time of the com-
and statistics as well as clinical courses and incorpo- pletion of the inference revolution that operationally
rate a research dissertation leading to a doctorate defined the scientific method in psychology, the
(Committee on Training in Clinical Psychology, affirmation of the scientist-practitioner ideal at the
1947; Thorne, 1945). Between 1947 and 1949, the Boulder conference, and the rapid growth of gradu-
number of American Psychological Association– ate training programs accredited according to the
accredited doctoral programs in clinical psychology Boulder model, it is hardly surprising that the
Consistent with the ethos of the time (the 1950s), those who formulated the scientist-practitioner model appear to have endorsed an essentially lin-
2
ear, or one-way, model of the link between basic science and applied science (Reich, 2008). In psychology, this model appears not to have been much
debated or challenged, but in the history and philosophy of science, the debate has been extensive (Balaram, 2008). Stokes (1997) developed a mul-
tidimensional typology for characterizing the relationship between science and its application, and it has been suggested that psychology belongs in
Stokes’s “Pasteurian quadrant” (Reich, 2008; see also Price & Behrens, 2003). This domain is characterized by high concurrent interest in both basic
science and its applications (Reich, 2008) and clearly has links to the idea of translational research as well (Lerman, 2003).
180
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
science part of the scientist-practitioner ideal came ensuing 50-plus years? In answering this question, I
to be identified with the standard group statistical first reflect briefly on some of the consequences that
model (Aiken, West, & Millsap, 2008; Aiken, West, followed from this development. Second, I consider
Sechrest, & Reno, 1990; Rossen & Oakland, 2008). whether there was any alternative to history as it
Frank (1984) has suggested that the conspicuous happened, because if there was no alternative, then
fact that most of clinical practice had not been no matter the consequences, psychologists have had
derived from science was what led the field to seize the best of all possible worlds.
on method, as endorsed in academe by research- The first and most obvious consequence to note
ers, as the common element linking psychologists is that clinical science, the foundation for the scientist-
together. By the late 1960s, influential expositions of practitioner, necessarily enjoyed both the benefits
research designs in clinical research (e.g., Kiesler, and the problems (if any) of the underpinning Fish-
1971) endorsed group statistical research as the pri- erian model, because there can be no doubt that
mary scientific method. clinical research has overwhelmingly been con-
Although the scientist-practitioner ideal has been ducted within this tradition. Problems, however,
criticized, disputed, modified, and lamented (e.g., there were. Dar, Serlin, and Omer (1994) reported
Albee, 2000; Lilienfeld, Lynn, & Lohr, 2003; Peterson, an extensive methodological review of the use of sta-
2003; Stricker, 1975; Tavaris, 2003), it continues to be tistical tests in research published in the Journal of
vigorously affirmed as an enduring ideal in applied Consulting and Clinical Psychology in the three
psychology (e.g., B. B. Baker & Benjamin, 2000; T. B. decades spanning from 1967 to 1988 and showed
Baker, McFall, & Shoham, 2008; Belar & Perry, 1992; that this research almost without exception was
Kihlstrom & Kihlstrom, 1998; McFall, 2007; Soldz & conducted within the standard NHST paradigm.
McCullogh, 2000) as both informing and inspiring They also documented the occurrence of a large
graduate training and professional practice. number and variety of misinterpretations of statisti-
Thus, by the midpoint of the 20th century a con- cal tests in this research, including the abuse of
vergence of two powerful movements within psy- p values as having meaning they do not have (see
chology had occurred. One defined what science Chapter 7, this volume), the absence of confidence
was and how it should be done; the other specified intervals, and the gross inflation of Type I error.
that having each graduate personally become both Where trends existed over the decades, they were
a scientist and a practitioner through training and often toward the frequency of misinterpretation get-
supervised practice in both research and application ting worse rather than better. Dar et al. (1994) con-
was the way in which professional applied psychol- cluded that there was “a growing misuse of null
ogy should be conducted. As it happened and still hypothesis tests” (p. 79) in psychotherapy research.
happens, training of graduate students in main- In this, clinical research echoed the situation in psy-
stream settings, particularly the major universities, chological research as a whole, where widespread
whether of general experimentalists or of aspiring misinterpretation of statistical tests has been notori-
practitioners, emphasized the standard group statis- ous for decades (Cohen, 1990, 1994; see Balluerka,
tical model as the primary and necessary pathway to Gomez, & Hidalgo, 2005, and Nickerson, 2000, for
pure and applied knowledge (Aiken et al., 1990, comprehensive reviews). Given the centrality of
2008; Rossen & Oakland, 2008). conclusions based on the outcome of NHST for
determining the scientific status of research and its
likelihood of publication, this state of affairs is seri-
Some Consequences of this History
ous for all research, including clinical research.
The preceding section, in brief, concerned how his- Has the situation improved more recently? Fidler
tory unfolded for psychology in the 20th century, et al. (2005) reported on a similar survey of empiri-
specifically in the combination of a prescription for cal articles published in the same eminent journal
doing science with an ideal form of applied science for five time periods spanning from 1993 to 2001.
practitioner. How have things turned out in the NHST-based methods continued to dominate
181
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
research, but misinterpretations and misapplications extraordinarily difficult to imagine doing research
of statistical methods also continued to be conspicu- that did not entail the computation of group averages
ous. Some improvement over the Dar et al. (1994) as the initial step in any analysis (e.g., Cairns, 1986).
survey was noted, but Fidler et al. (2005) commented There was nothing particularly remarkable about
tartly, “In a major journal dedicated to the research this use of averaging in the case of Fisher’s own
of psychotherapy . . . clinical significance [rather than research, dealing as he did with agricultural produce
statistical significance] should be relevant to more for which interindividual differences in the proper-
than 40% of articles” (p. 139). Similar surveys of ties of individual grains of rice or kernels of wheat
other major research journals have suggested a and so forth are not of particular interest, for which
common picture of continuing misinterpretation of such items are so numerous as to preclude examination
Fisherian statistics and resistance to change (Schatz, of each one anyway, and for which the commercial
Jay, McComb, & McLaughlin, 2005; Vacha-Haase, and economic system dealt with agricultural com-
Nilsson, Reetz, Lance, & Thompson, 2000) despite modities in bulk. Given this, the development and
much effort to improve statistical methods (e.g., Erceg- use of sampling theory that permitted estimation of
Hurn & Mirosevich, 2008; Rodgers, 2010; Wilkinson & population parameters and the making of inferences
Task Force on Statistical Inference, 1999). from samples to populations made sense and contin-
Given this lamentable state of affairs from the ues to be appropriate when actuarial or population-
mid-1960s to the present, little reason exists to level issues are at stake.
suppose that clinical science has been spared the Critically, however, for psychology, these prac-
deleterious outcomes some critics have suggested tices in agricultural research meshed neatly with the
all psychological research has suffered as a result of pursuit by Quetelet, a century before Fisher, of
using the dominant statistical paradigm (Michael, “ideal” aspects of humankind according to the nor-
1974; Rosenthal, 1995). These outcomes include mal distribution (the normal law of error; Gigeren-
serious lack of power to detect experimental effects zer et al., 1989). Quetelet’s search for the ideal
(e.g., Cohen, 1962; Sedlmeier & Gigerenzer, human led directly to Fisher’s inferential statistics
1989), the lack of genuinely cumulative knowledge (Johnston & Pennypacker, 1993). The question of
in psychology (e.g., F. L. Schmidt, 1996), the con- whether this combination of teleological assump-
spicuous divergence of the methods of psychology tions about perfect human types and a focus on the
from those used in other natural and social sci- estimation of aggregate properties of commodities
ences (Blaich & Barreto, 2001; Meehl, 1978), and was a good model for clinical and other applied psy-
the failure of psychology to develop unambigu- chologies, in which the cases being served are often
ously falsifiable theories (e.g., Meehl, 1978). diverse and individual differences are potentially of
Indeed, Meehl (1978) said of the adoption of NHST great importance, seems not to have been asked dur-
that it was “a terrible mistake, is basically unsound, ing the inference revolution, during the Boulder
poor scientific strategy, and one of the worst things conference, or since.
that ever happened in the history of psychology Nevertheless, a range of scholars have criticized
[emphasis added]” (p. 817). psychology’s addiction to averaging across cases.
A second, and perhaps less documented conse- The evolutionary biologist Steven Jay Gould (1985),
quence of the joining of clinical science with the in “The Median Isn’t the Message,” a touching auto-
standard research model has been the ubiquitous use biographical memoir of his first cancer diagnosis,
of averaging across participants. One of the direct, recounted how his realization that the distribution
remarkable, but little-commented-on features of the of survival time postdiagnosis was skewed beyond
inference revolution was the almost universal and the median 8 months gave him hope. The point of
unquestioned adoption of experimental groups his writing about this experience, however, was to
necessitating group averaging as almost the first step make a more general point: “I believe that the fallacy
in any data analysis procedure. Most psychologists of reified variation—or failure to consider the ‘full
today and for the past 50-plus years would find it house’ of all cases—plunges us into serious error
182
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
again and again” (Gould, 1997, p. 46). The error by, the underlying statistical model because hypoth-
Gould was referring to is to focus exclusively on the eses are about populations, not samples or individu-
mean (or median) and ignore variability, forgetting als (Hays, 1963, p. 248), and yield useful knowledge
that variability is as central a phenomenon in bio- when it is the population that is of concern. As
logical systems as is any central tendency. The same Valsiner (1986) noted, however, major difficulties
point had been made by the physiologist Claude arise with the (largely unacknowledged) deductive
Bernard (1865/1957) more than a century before steps by which psychologists get back from the pop-
(Thompson, 1984). ulation to the individual, as they must do in applied
Developmental researchers and theorists are work. The problem, another version of the unifor-
notable among psychologists who have criticized mity myth identified by Kiesler (1966), is that the
averaging across cases (e.g., Bornstein & Lamb, induction from the sample to the population is
1992; Cairns, 1986), perhaps because their subject assumed rather than demonstrated to have general-
matter involves dynamic changes over time that ity to any individual case, but the assumption is
have different trajectories in different individuals. often false unless the population mean has very
Among these scholars, Valsiner (1986) has noted small variance (Rorer, 1991), which impairs the
both the double standard prevailing in psychology— validity of the subsequent deductions. Without
purported deep interest in individuals combined this deductive step, however, individual application
with the constant practice of averaging across of group research–based scientific knowledge is
individuals—and has cogently explained the dan- not possible.
gers of this. He wrote,
In psychological discourse (both scien- Was There Any Alternative to
tific and applied), the individual subject the Standard Model?
is constantly being given high relevance.
I turn now to the question of whether there were,
In contrast, the individual case is usually
and are, alternatives to the adoption of the standard
forgotten in the practice of psychologi-
model for research and its incorporation in the
cal research because it is being replaced
scientist-practitioner model. As noted earlier, Wit-
by samples of subjects that are assumed
mer’s conceptualization of both clinical science and
to represent some general population.
clinical practice was focused on the individual, and
Overwhelmingly, psychologists study
an emphasis on the individual was evident in the
samples of subjects and often proceed
early definitions of clinical psychology (Reisman,
to generalise their findings to the ideal
1966, 1991; see the section Clinical Psychology and
(generic, i.e., average or prototypical)
the Rise of the Scientist-Practitioner Ideal). More-
abstract individual. Furthermore, charac-
over, during the time of both the early adoption of
teristics of that abstracted individual may
statistical methods and the development of clinical
easily become attributed to particular,
psychology, Gordon Allport emphasized the need
concrete individuals with whom [they]
for psychology to be both nomothetic (concerned
work and interact. The inductive infer-
with general laws) and ideographic (the study of
ence from samples of concrete subjects to
unique individuals), noting, “The application of
the abstract individual and from it (now
knowledge is always to the single case” (Allport,
already deductively) back to the multi-
1942, p. 58; also see Molenaar, 2004). For the most
tude of concrete human beings is guided
part, however, these influences did not withstand
by a number of implicit assumptions . . .
the power of the inference revolution (Barlow &
that obscure insight into the science and
Nock, 2009).
hamper its applications. (p. 2)
There was an exception. Since the 1930s, B. F.
The inductive inference from the sample to the Skinner and the science of behavior analysis he
population is consistent with, and indeed mandated founded have sustained trenchant criticism of
183
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
averaging across cases and rejection of NHST while f amily, class, work group, organization, or commu-
maintaining a commitment to experimentation and nity (Blampied, 1999; Valsiner, 1986).
quantification (Gigerenzer et al., 1989; Hersen &
Barlow, 1976). Skinner maintained an unwavering
Development of Applied
conviction that psychology should be a science of
Single-Case Research Designs
individual behavior—“Individual prediction is of
tremendous importance, so long as the organism is The experimental research designs that Skinner and
to be treated scientifically as a lawful system” his colleagues and students developed for their
(Skinner, 1938, p. 444)—and eschewed averaging experimental laboratory investigations and that
across cases. In his magisterial exposition of were systematized in Sidman (1960) are termed
behavior-analytic research design principles, intrasubject replication designs. They were strongly
Sidman (1960) noted, influenced by the concept of the steady state in phys-
ical chemistry and experimental physiology (Ber-
Reproducible group data describe some
nard, 1865/1957; Skinner, 1956; Thompson, 1984).
kind of order in the universe, and as such
To understand the influence of environmental vari-
may well form the basis of a science. It
ables on behavior, the performance of an individual
cannot, however, be a science of individ-
subject was measured repeatedly in a standardized
ual behavior except of the crudest sort.
apparatus until performance was judged to be stable.
And it is not a science of group behavior
If variability persisted, the sources of this variability
in the sense that the term “group” is
in the environment were investigated so that it could
employed by the social psychologist. It
be minimized. Sources of variability were thus to be
is a science of averaged behavior of indi-
understood and experimentally reduced rather than
viduals who are linked together only by
statistically controlled (Sidman, 1960), thus generat-
the averaging process itself. Where this
ing an experimental phase called the baseline. Once
science fits in the scheme of natural
baseline stability was achieved, an independent vari-
phenomena is a matter for conjecture.
able could be introduced and behavior observed until
My own feeling is that it belongs to the
stability returned. Direct visual comparisons, by way
actuarial statistician, and not to the
of graphs, of the performance in the baseline and
investigator of behavioral processes.
experimental conditions permitted the detection and
(pp. 274–275).
quantification of the experimental effect. Replica-
Skinner (1938) and Sidman (1960) also main- tions (S. Schmidt, 2009) by way of return to baseline
tained that any natural behavioral process must be and subsequent reinstatement of the experimental
demonstrable in the behavior of an individual (Sid- variable demonstrated the reliability of the experi-
man, 1960) and that group averaging risked creating mental control achieved and were the basis of infer-
synthetic rather than real phenomena. ences about the independent variable being the agent
The principles and research practices developed of the change. Further replications were performed
by Skinner and other behavior analysts have clearly with the same or a small number of other subjects in
demonstrated the possibility of both a basic and an the same experiment to strengthen conclusions (see
applied science of individual behavior (e.g., Cooper, Chapter 7, this volume). Systematic replications of
Heron, & Heward, 2007; Mazur, 2009) without the procedure using parametric variations of the
the use of group-average data or statistical infer- independent variable permitted a functional analysis
ence procedures in the single-subject–single-case of the relation between the independent variable and
research tradition (many other terms have been the target behavior (Sidman, 1960), further strength-
used, including N = 1 and time-series designs; see ening causal inferences. Using these procedures,
Hayes, 1981). Use of the word case signals that the
thing studied need not be a single individual but important processes in behavior are
might be a group entity of some kind, such as a revealed in a continuous, orderly, and
184
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
reproducible fashion. Concepts and laws comparing baseline and intervention phases and to
derived from such data are immediately make causal inferences. Hersen and Barlow (1976)
applicable to the behavior of the individ- published the first book-length systematic exposi-
ual, and they should permit us to move tion of these applied single-case designs, and only a
on to the interpretation of behavior in little innovation, although much refinement, has
the world at large with the greatest pos- occurred since (McDougall, 2005).
sible speed. (Skinner, 1953, p. 78) Several things are notable about this sequence of
events. First, the development of experimental
Beginning in the early 1950s, Skinner, assisted single-case research occurred at almost the same
by Lindsley, extended research in operant behavior time as the widespread adoption of Fisherian statis-
from investigations of animals such as rats and tics in psychology in the immediate postwar years
pigeons to that of adult humans, particularly those and was explicitly regarded by its protagonists as an
hospitalized with a diagnosis of schizophrenia alternative to the Fisherian tradition (Gigerenzer
(Lindsley & Skinner, 1954; Skinner, 1954). Other et al., 1989; Skinner, 1956). Although the first text-
researchers, notably Bijou (Kazdin, 1978; Morris, book for teaching Fisherian statistics to psycholo-
2008), began to deploy similar procedures to gists appeared in 1940 (Lindquist, 1940), most
research and assess the behavior of children. As universities offering graduate degrees did not begin
Kazdin (1978) noted, this research had an increas- systematically teaching statistics to students until
ingly applied focus that moved it from basic research after World War II (Gigerenzer et al., 1989; Rucci &
to evaluation, and then inexorably toward interven- Tweney, 1980), by which time Skinner’s (1938) first
tion, using basic techniques such as shaping, rein- book had appeared, single-case research was being
forcement, and extinction to change symptomatic published (albeit not as prolifically as it was after the
and problem behavior (Kazdin, 1978). This focus founding of the Journal of the Experimental Analysis
generated the field now called applied behavior of Behavior in 1958), and Sidman’s (1960) Tactics of
analysis (Baer, Wolf, & Risley, 1968). O’Donnell’s Scientific Research was only a decade away.
(1985) observation that “nearly every facet of The opportunity then was for the scientist part of
applied work originally derived from . . . interest in the scientist-practitioner ideal to be based on single-
the child” (p. 237) was as true of the emergence of case research rather than, or as well as (Jones, 1978),
applied behavior analysis as it had been of Witmer’s Fisherian inferential statistics. This alternative
clinical psychology. research design tradition has, however, had negligi-
In moving from basic operant research to clinical ble influence on psychological research as a whole or
applications, behavior analysis adapted the intrasu- on applied professional psychology in particular,
bject research designs used in the laboratory. Early and it has never been part of the graduate curricu-
studies often reported data in cumulative graph lum (Aiken et al., 1990, 2008; Rossen & Oakland,
form (see Ullmann & Krasner, 1966, and Ulrich, 2008). Why single-case research designs had such
Stachnik, & Mabry, 1966, for examples), but by the little influence on mainstream psychology and how
early 1960s reversal designs were being presented in science and practice might be different had they had
what is now the conventional form (e.g., Ayllon & more impact are interesting questions for historians
Haughton, 1964). Problems with reversal designs and sociologists of science. Gigerenzer et al. (1989)
led to the development of other applied designs such suggested that several factors led to the dominance
as the multiple baseline (Baer et al., 1968; Kazdin, of Fisher’s ideas in psychology in the critical period.
1978). All these designs involve time-series data Many leading North American experimentalists were
from one or a few participants, and all involve taught by Fisher himself, and Fisher’s influence was
experimental phases designated baseline and inter- enhanced by his holding a visiting professorship in
vention. They differ in the way in which replication the United States in the 1930s. The relatively small
is used within and across participants to establish size of the community of experimental psychologists
the reliability of the behavior changes detected by assisted the rapid dissemination of the new statistics,
185
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
a process facilitated by the publication of informa- experimental designs and statistical procedures
tion about them in textbooks and influential jour- which are held inappropriate to the subject matter
nals (Rucci & Tweney, 1980). This process was under study” (Bergin & Strupp, 1970, p. 25). The
enhanced when editors of important journals, such evidence presented by Bergin and Strupp (1970,
as the Journal of Experimental Psychology, made the 1972) provided an opportunity for psychology in
use of inferential statistics a requirement for accep- general, and clinical psychology in particular, to
tance of manuscripts, which happened from about reevaluate its commitment to the standard model of
1950 onward (Gigerenzer et al., 1989). Fisher’s writ- research, but this did not happen other than within
ings were in English, and much of the critical schol- the single-case research tradition. Hersen and Barlow
arly work about statistics and research design was agreed that the scientist-practitioner ideal had largely
published in French and German (Gigerenzer et al., failed in practice because of a growing split between
1989) and was, therefore, less accessible to English- science and practice and that the split was substan-
speaking workers, although Bernard’s work was tively, but not exclusively, the result of the mismatch
available in English translation by 1949. It is also between the realities of clinical practice and what
true that Skinner’s ideas about experimentation and scientific research methods based on the standard
inference, although broadly contemporaneous, were model could inherently accomplish. Instead of the
not as well developed or as effectively disseminated standard Fisherian model of research, Hersen and
at the critical juncture, so by an initially narrow Barlow suggested that single-case designs were much
lead, Fisherian ideas had the opportunity to become better suited to the needs of clinical research, espe-
dominant in psychology. cially the evaluation of interventions.
Second, in contrast to the inference revolution in The argument developed by Hersen and Barlow
psychology, in which the key ideas about research (1976) has been further extended by Barlow et al.
design and analysis were imported from other fields (1984); Hayes, Barlow, and Nelson-Gray (1999);
and then applied to the subject matter of the disci- and Barlow, Nock, and Hersen (2009) and endorsed
pline, applied behavior analysis and applied single- by others (e.g., Blampied, 1999, 2000, 2001; Hayes,
case research designs grew organically and directly 1981; Kazdin, 2008; Morgan & Morgan, 2001,
from the underpinning experimental research in the 2009). If one poses the question “What benefits does
field, both with respect to the subject matter studied single-case research offer the scientist-practitioner?”
(the effects of the environment on the behavior of one can find answers by considering how single-case
individuals) and the methods used, which is surely a research obviates some of the major problems inher-
significant virtue when one considers what science a ent in reliance on the standard model, and second,
scientist-practitioner may need by way of both the from considering further positive contributions that
content and the methods of science and how single- single-case research can make to applied science.
case research designs may help resolve problems in
the field. Single-Case Research Is an Alternative to
NHST-Based Research
The fact that most psychologists past and present
What Single-Case Research Can
have ignored the enduring, cogent, and powerful
Offer the Scientist-Practitioner
criticisms of the standard NHST model of psycho-
In their initial exposition of applied single-case logical research is not evidence that these criticisms
designs, Hersen and Barlow (1976) stated that single- do not matter (Harlow, Mulaik, & Steiger, 1997;
case research was highly relevant to the scientist- Nickerson, 2000; F. L. Schmidt & Hunter, 1997).
practitioner ideal and that it provided an alternative Even the defenders of NHST and related practices
to the prevailing standard model of research. They have acknowledged that the criticisms have substan-
drew on the work of Bergin and Strupp (1970, 1972), tive merit (e.g., Cortina & Dunlap, 1997; Frick,
who claimed that among psychotherapy researchers, 1996; Hagen, 1997; Harlow et al., 1997; Wainer,
“there is a growing disaffection from traditional 1999), and proponents of reform such as the Task
186
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Force on Statistical Inference (Wilkinson & Task for large numbers of participants (Streiner, 2006).
Force on Statistical Inference, 1999) have sought As the number of participants in a research project
changes in research practice, data analysis, and multiplies, so too do the cost, complexity, and time
reporting that acknowledge much of the criticism. required to complete the research (a point made by
Unfortunately, as noted earlier, the practice of Bergin & Strupp, 1972). Multisite, multiinvestiga-
research has been glacially slow in adopting these tor, multimillion-dollar randomized controlled trial
reforms. This slowness may, in part, be because research designs are now the gold standard for psy-
one effect of the proposed reforms is to continue to chotherapy research (e.g., Beutler & Crago, 1991;
embed group statistical procedures and NHST, with Wessley, 2001). Participation in such research
all of their inchoate aspects (Gigerenzer, 1991; Hub- is largely out of the question for most scientist-
bard, 2004), into the heart of the research enterprise practitioners (indeed, it is out of reach of most
while requiring that extensive additional complex researchers), so it is hardly surprising that they
procedures be added to the research toolkit. This is do not actively produce research in their practice
demanding enough for researchers, and it is even (Barlow et al., 1984; Hayes et al., 1999). Single-case
more so for practitioners. research, in contrast, done by individual practitio-
Given that single-case research principles and ners with individual cases, can make useful contri-
practice resonate with many of the criticisms of the butions to science.
standard model, for instance in the emphasis on
graphical techniques, replication, and the rejection Single-Case Research Avoids Confusing
of NHST-based inference (Sidman, 1960), it is odd Clinical With Statistical Significance
that few if any of the critics or reformers have con- As many critics have noted, the overwhelming focus
sidered the potential of single-case research as on NHST p values has seriously distorted psycholog-
an alternative approach (Blampied, 1999, 2000). ical research in general and clinical research in par-
Indeed, the task force did not consider single-case ticular (e.g., Cohen, 1990, 1994; Dar et al., 1994;
research until it replied to comments on its report Fidler et al., 2005; Hubbard & Lindsay, 2008;
(American Psychological Association Board of Sci- Meehl, 1978). The obsession with statistical signifi-
entific Affairs, 2000), and even then the response cance has distorted judgment about research find-
was brief and unenthusiastic. Notwithstanding, ings and led to persistent confusion between
single-case research designs remain a complete, statistical and clinical significance, fundamentally
quantitative, coherent, internally consistent, and because p is a function of sample size and, with
proven alternative to the standard model of a sufficiently large sample, infinitesimally small
experimentation. group-mean differences may be statistically signifi-
cant (Hubbard & Lindsay, 2008).
Single-Case Research Is an Effective Clinical significance, in contrast, is determined at
Alternative to Group-Based Research the individual level, and rather than typically being
As suggested earlier, psychology’s commitment achieved by small degrees of change, requires sub-
to seeking ideal types by way of averaging over indi- stantive alleviation of distress, restoration of function,
viduals is as deep rooted as its commitment to and attainment of quality-of-life goals (Blampied,
NHST-based inference. Yet this commitment poses 1999; Jacobson & Truax, 1991). Clinically, psycholo-
profound difficulties, both conceptual and practical, gists need to know that interventions have produced
for applied science, as Valsiner (1986) and others effective change, but null-hypothesis–based statistics
have so cogently noted. The conceptual difficulties do not reliably indicate this. Although techniques
(discussed earlier) relate to how abstract principles such as the use of effect size estimates and meta-
and hypotheses stated at the level of the population analysis of group research (e.g., Bornstein, 1998;
are to be properly applied to the highly variable Kline, 2004) are an improvement over p values, large
individual cases with which applied scientists must effect sizes may be associated with little clinical
deal. The practical difficulties arise from the demand change (Hayes et al., 1999). Techniques to compute
187
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
the magnitude of clinically significant change (e.g., being assessed is the necessity of assigning some-
Jacobson & Truax, 1991) require analysis at the level times large numbers of individuals to control condi-
of the individual rather than the group, suggesting tions, which are, by definition, not expected to
that determining clinical significance is an intractable produce much in the way of therapeutic effect. At the
problem for group research. Furthermore, contribut- same time, those in the treatment group are exposed
ing to the group mean may be individuals whose to whatever risks are entailed in the novel treatment
scores indicate large-magnitude change, those whose (Hersen & Barlow, 1976). Single-case research does
scores have changed only slightly, and those whose not eliminate these risks, because it imposes base-
scores have deteriorated (Hersen & Barlow, 1976). line and treatment phases that are the equivalent
For clinical work to progress, it is essential to know of control and experimental conditions. It does so,
about changes that are more than negligible in a however, with one or a few participants, and so the
positive way but that are still less than clinically effects of the treatment can be assessed while neither
significant and about deterioration; otherwise, it is withholding treatment from a large number of indi-
impossible to engage in systematic improvement of viduals nor exposing a substantial number of indi-
therapies or to move toward the oft-stated goal of viduals to whatever risks are inherent in the
matching therapies to clients (Hayes et al., 1999; research. For this reason, at the very least, ethical
Kazdin, 2008). Concealing a range of therapy out- principles should insist that a novel therapy be
comes within a small but statistically significant mean shown to be effective in a series of well-conducted
effect does not assist with this objective. single-case studies before any large-scale random-
Because single-case research does not amalgam- ized controlled trials are embarked on. In contrast,
ate case data but keeps the data separate and identi- the general practice is typically to go from uncon-
fiable throughout the research and because it has no trolled case studies to randomized controlled trials
equivalent to the ubiquitous p value, it cannot mis- in one step. Note also that wait-list control designs
take statistical for clinical significance (although it can be re-formed as multiple-baseline-across-cases
can either over- or understate the import of its find- designs, with a consequent reduction in waiting
ings). It can relate the magnitude of change or lack time for therapy for most wait-listed cases and
of change to each individual participant relative to increases in internal validity (see Chapter 5, this
that participant’s own baseline. It heeds the admoni- volume, for a more comprehensive consideration of
tion by Cohen (1994) that there is “no objective, the multiple-baseline design). Moreover, Kratoch-
mechanical ritual” (p. 1001) for determining the will and Levin (2010) have made recommendations
importance of research findings and substitutes the about how single-case designs might be modified by
scientist-practitioner’s trained, professional judg- the inclusion of randomization so as to make them
ment as to the importance of the outcomes achieved more credible within the wider community of
in the research (Hammond, 1996). That the key applied researchers.
data are presented graphically so that the changes Institutional, policy, and political factors have all
achieved have to be obvious to visual inspection long been recognized to strongly influence the con-
provides some assurance that only substantive duct of research (e.g., Bergin & Strupp, 1970, 1972),
changes will be reported and permits all other view- often to the detriment of innovation and conceptual
ers to apply their judgment, thus subjecting conclu- advances, especially through decisions about what
sions to ongoing peer review and continuous research to fund, publish, and incorporate in policy
revision as understanding of clinical significance and application (Rozin, 2009). Changes in research
changes with time and context. practice that would lead to the wider use and dis-
semination of single-case research within applied
Single-Case Research Enhances Ethics and and clinical science will require attention to these
Accountability institutional and political factors. The powerful con-
One of the persisting ethical problems with standard tingencies residing in research funding, particularly
group research protocols in which interventions are involving the assessment of the scientific merit of
188
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
proposals incorporating single-case designs, will treatments fail in any case, that failure is detected,
have to change. These changes clearly cannot be and new approaches can be tried.
accomplished by individuals acting alone. They may In addition to being ideally adapted to the initial
require concerted action by larger groups represent- phases of research and development, single-case
ing both scientists and practitioners in advocating research is ideally suited for what is increasingly
changes to institutions, funding agencies, and gov- being called translational research—the systematic,
ernment. Perhaps it is time to consider another task collaborative pursuit of applications of science
force, sponsored jointly by the American Psycholog- linked to basic research (Lerman, 2003; Thompson
ical Association, the Association for Psychological & Hackenberg, 2009). Ironically, Olatunji, Feldner,
Science, and the Association for Behavior Analysis Witte, and Sorrell (2004), in discussing the link
International to address these inherently political between the scientist-practitioner model and trans-
issues and to advocate for reform both within and lational research, recommended that even more rig-
beyond the discipline (Orlitzky, 2011). orous training in the standard model of research is
A further strength of single-case research is that necessary before the scientist-practitioner model can
it enhances accountability in research and, even embrace translational research. To the contrary, I
more important, in practice (Acierno, Hersen, & would assert that training in single-case research is
Van Hasselt, 1996). The data for each individual needed before translational research can be widely
participant can be examined by whoever wants to undertaken, because much translational research, as
judge the outcomes claimed to have been achieved. with other applied research, needs to be applied at
Nobody, including any participants, critics, ethical the individual rather than the population level.
review bodies, insurers, and family members, has to There is no point in having translational research
rely on understanding the possibly abstruse com- recapitulate the errors of the past through slavish
plexities of statistical analyses to understand what adherence to the standard model.
the treatment achieved (Blampied, 1999). This Single-case research designs also have an under-
strength does not remove the possibility of conflict appreciated contribution to make to the establish-
and disagreement, but it does change the character ment of generality of treatment. Consider the
of any such debate, with transparency, openness, situation that prevails after a successful randomized
and comprehensibility likely to be of long-term ben- trial of some new therapy. As has been widely noted
efit to all who are concerned about the ethics of (e.g., Kazdin, 2008; Seligman, 1995), the very rigor
research and practice, clients and practitioners of this research protocol, requiring as it does highly
above all (F. L. Newman & Tejeda, 1996). Wolf’s selected participants with a single, carefully estab-
(1978) seminal work on the social validity of behav- lished diagnosis, highly trained and monitored thera-
ior analysis and the assessment of the acceptability pists normally working from a manual, and atypical
of interventions to multiple constituencies, includ- therapy settings (e.g., university research clinics),
ing clients, families, and other interested parties, has limits the generality of any findings. Indeed, such
further extended the ways in which accountability research is now termed efficacy research to distin-
can be achieved in both research and practice. guish it from effectiveness research—the research
that establishes the utility of the therapy in typical
Single-Case Research Enhances Innovation therapy settings with typical therapists and typical
and the Exploration of Generality clients, now also referred to as research into clinical
Implicit in the preceding argument is the utility of utility (Howard, Moras, Brill, Martinovich, &
single-case research in the early phases of therapy Lutz, 1996). Although Chambless and Hollon (1998)
innovation. Because of its small scale, it can be used acknowledged the role that single-case research
to evaluate novel ideas and new procedures with might play in establishing the efficacy of psychother-
low cost and little risk. This style of research is apy, the focus of efficacy and effectiveness research
adaptable, and protocols can be changed as develop- has been almost exclusively on randomized trials
ments in the research warrant (Hayes, 1981). If (e.g., Kazdin, 2008; Kratochwill & Levin, 2010).
189
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
Using group-based randomized trials and sys- Single-Case Research Resolves the Double
tematic exploration of every dimension along which Standard Around Psychology’s Focus on
the participants, the therapy, the therapists, and the the Individual
therapy context might be varied in the search for If one believes, as the founders of clinical and
generality would entail a lifetime of research and a applied psychology clearly did, that “the individual
fortune just to establish the domain of general effec- is of paramount importance in the clinical science of
tiveness of a single therapy. Staines (2008) argued human behavior change” (Hersen & Barlow, 1976,
that deficiencies in the way in which most standard p. 1), then single-case research is essential because
psychological research is conducted severely limit it delivers a scientific understanding of phenomena
the generality of its findings because of what he at the individual level and removes the need for
terms the generalization paradox, and he recom- the inductive–deductive contortions exposed by
mended the use of multiple studies and multiple Valsiner (1986; see the section Some Consequences
methods as ways out of the paradox. He did not of This History). It can, in practice, remove the dis-
explicitly recommend the use of single-case research tinction between the scientist and the practitioner;
but might well have done so. Although still a poten- science and practice become one coherent
tially large undertaking, single-case research can be enterprise.
used to map the generality of any established ther- This grounding of applied clinical science in the
apy (see Drabman, Hammer, & Rosenbaum, 1979, individual—both the individual client and the indi-
for the initial idea of the generalization map). It vidual practitioner—is probably the most important
does this through replication (see Chapter 7, this contribution of single-case research to the scientist-
volume). practitioner model, because it gives the ideal
Initially, direct replication, in which the attri- model real substance (Barlow et al., 1984, 2009;
butes of the cases, therapy, and context are kept Blampied, 1999, 2001; Hayes et al., 1999; Hersen &
constant, is used to establish the reliability of the Barlow, 1976). “Repetition is the habit of nature, but
research findings. Generality can then be explored variation is the rule of life” (Gopnik, 2009, p. 197),
by systematic replication (Hayes et al., 1999; Sid- and whether the cases dealt with by a scientist-
man, 1960), in which various attributes of the cases practitioner are individuals, families, or social
(e.g., age, gender, symptom severity), of the treat- groups or are larger and more complex entities such
ment (e.g., number of sessions), and of the context as classes, work groups, and organizations or even
of intervention (e.g., homes, classrooms) are sys- health systems (Hayes et al., 1999; Morgan & Mor-
tematically varied. As these dimensions are gan, 2009), each case will be inherently unique. Yet,
explored, the initial treatment outcome may be no matter how exceptional and singular a particular
maintained, enhanced, or diminish or even disap- case may be, single-case research designs permit a
pear. Additional clinical replication, combining mul- scientific analysis to be done—a claim that cannot
ticomponent treatment programs and clinically be made about the standard model. This is why
diverse clients and settings, further extends the there is such goodness of fit between the scientist-
evidence for generality (Barlow et al., 1984, 2009). practitioner model and single-case research and why
As the generalization space is mapped in this way, the failure to recognize the synergies between them
the therapy can be adjusted for maximally effective has been a seriously consequential missed opportu-
combinations of participant, procedure, and con- nity for the discipline.
text. Replicated failures are also important because
they mark possible boundary conditions and stimu-
Conclusion
late new innovations and a new cycle of research.
Note that for this reason, the reporting of such fail- I can confidently conclude that the goodness of fit
ures is to be encouraged, in contrast with the prac- between single-case research principles and practice
tice in the standard research tradition of not and the scientist-practitioner model of applied prac-
reporting failures to reject H0. tice is multifaceted, substantial, and potentially
190
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
highly beneficial, although to achieve these benefits very good idea, and it should be adopted now, for
will require many changes to the way psychologists “scientist-practitioners [will] have no difficulty
teach students, fund, analyze and publish research, finding interesting work in [the] future. They are
and practice psychology (Blampied, 2001; Rozin, trained to solve behavioral problems, and the world
2009). Historically, this alliance could have hap- promises to provide no shortage of those” (Benjamin,
pened, but it did not, which can be seen as a very 2005, p. 27).
considerable misfortune for the discipline and its
aspirations to enhance human welfare. Instead, the
Appendix 8.1: Terminology and
scientist-practitioner ideal has been distorted, dis-
Key Aspects of Null-Hypothesis
tressed, and thwarted by attempting to pursue sci-
Statistical Testing
ence with a scientific methodology inappropriate to
the purpose. For any set of numerical data, one can compute
History is the story of what has happened, not measures of central tendency (medians, means, etc.)
what might have happened or what one wishes or and of variation (standard deviations, etc.) as well as
thinks should have happened. The generation and other measures such as correlations. These measures
adoption of the scientist-practitioner model for are called descriptive statistics. If one takes the view
clinical and applied psychology is deservedly seen that the individuals who provided these data are a
as an inspired and noble choice. It continues as an sample from some population of interest (and one
inspiring ideal to the present day. That the concep- does not necessarily have to take this view), then
tualization of what the science part of the scientist- one can regard these descriptive statistics as estima-
practitioner duality was thought to be should have tors of the corresponding “true” population scores
been captured by the view of science prevailing in and can compute further statistics, such as the stan-
the wider discipline was probably inevitable. What dard error of the mean and a confidence interval,
is much more regrettable is how little consideration that tell one how good an approximation these esti-
the discipline’s forebears gave to the aptness of their mations are. Going a step further, if one has two or
choice of scientific method, especially in the light of more data sets and wants to know whether they rep-
Bergin and Strupp’s (1970, 1972) research. Equally resent samples from the same population or from
as regrettable is how they, and we, have persistently different populations, one uses inferential statistics.
ignored criticism of the chosen method and how As noted in the chapter’s introduction, thanks to the
fervent, enduring, and exclusive adherence to the work of Fisher and others, the core of such inferen-
method has been (Ziliak & McCloskey, 2008). Also tial statistics are null-hypothesis significance tests
regrettable has been the continuing blindness of (NHST), of which Student’s t is the prototype. The
those entrusted with the scientist-practitioner null hypothesis (termed H0) is normally (in psychol-
ideal to the mismatch between the methods of the ogy) a nil hypothesis, that is, the samples are from
adopted science and the needs of practice, despite the same population and therefore the true mean
the warning of prophets such as Bergin and Strupp difference is zero (nil), and is so to an infinite num-
and Meehl and despite the existence of an alterna- ber of decimal places. NHST assumes H0 to be true,
tive in the work of Skinner and those who developed and computes a probability (p), which is the theo-
and practiced applied single-case research. But it is retical probability that had samples of the same size
not too late to change. as those observed been drawn from the same popu-
I’ve said this before (quoting Stricker, 2000; see lation, the test statistic (e.g., the value of t) would be
Blampied, 2001), but I will say it again: Gandhi, as large (or larger) as it is observed to be.
once asked what he thought about Western civiliza- For Fisher, the p value was one important piece
tion, replied that it was a good idea and that some- of evidence, to be considered along with other evi-
body should try it sometime. Even more so, an dence, in the judgment made by the experimenter as
alliance between the scientist-practitioner model to the implausibility of the observations assuming
of clinical practice and single-case research is a H0. If the p value suggests that the data are rare
191
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
(under H0), this constitutes inductive evidence References

against H0 and H0 can be rejected, and the smaller Acierno, R., Hersen, M., & van Hasselt, V. B. (1996).
the value of p (other things being equal), the stron- Accountability in psychological treatment. In V. B.
ger this inductive evidence (cf. Wagenmakers, van Hasselt & M. Hersen (Eds.), Sourcebook of psy-
chological treatment manuals for adult disorders
2007). Fisher came to accept that p < .05 was a con- (pp. 3–20). New York, NY: Plenum Press.
venient (but not sanctified) value for experimenters
Aiken, L. S., West, S. G., & Millsap, R. E. (2008). Doctoral
to use in making such judgments in single experiments, training in statistics, measurement, and methodology
but he also agreed that facts were best established by in psychology: Replication and extension of Aiken,
multiple experiments that permitted consistent West, Sechrest, and Reno’s (1990) survey of PhD pro-
rejection of H0 (Wright, 2009). grams in North America. American Psychologist, 63,
32–50. doi:10.1037/0003-066X.63.1.32
The alternative Neyman–Pearson paradigm is
termed hypothesis testing because it involved the Aiken, L. S., West, S. G., Sechrest, L., & Reno, R. R. (1990).
Graduate training in statistics methodology and mea-
identification of two hypotheses, H0 and an alterna- surement in psychology. American Psychologist, 45,
tive hypothesis, HA. Fisher bitterly contested this 721–734. doi:10.1037/0003-066X.45.6.721
viewpoint and never accepted the alternative para- Albee, G. W. (2000). The Boulder model’s fatal flaw.
digm, but it is the version that has become dominant American Psychologist, 55, 247–248. doi:10.1037/
in psychology, generally without reference to the 0003-066X.55.2.247
initial controversy. This paradigm postulates both Allport, G. W. (1942). The use of personal documents in
H0 and HA. HA is generally an assertion that some psychological science. New York, NY: Social Science
Research Council.
known factor or treatment was responsible for the
observed mean difference. In this paradigm, if the American Psychological Association Board of Scientific
Affairs, Task Force on Statistical Inference. (2000).
obtained value of p is smaller than some long-run, Narrow and shallow. American Psychologist, 55,
prespecified error rate criterion (e.g., p < .05), 965–966. doi:10.1037/0003-066X.55.8.965
called alpha, then H0 may be rejected, a result that Ayllon, T., & Haughton, E. (1964). Modification of symp-
is said to be statistically significant at the alpha level, tomatic verbal behavior of mental patients. Behaviour
and one may accept HA. Research and Therapy, 2, 87–97. doi:10.1016/0005-
7967(64)90001-4
With two hypotheses available, two errors may
be made. One may reject H0 when it is true (Type I Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some
error) or one may fail to reject H0 and accept HA Journal of Applied Behavior Analysis, 1, 91–97.
when H0 is false (Type II error), the probability of doi:10.1901/jaba.1968.1-91
which error is beta. The power of a test to accept HA Baker, B. B., & Benjamin, L. T. (2000). The affirma-
given that it is true or, better stated, the power to tion of the scientist-practitioner: A look back at
detect an experimental effect, is 1 − β. Neyman Boulder. American Psychologist, 55, 241–247.
(1950) argued that control of Type I errors was doi:10.1037/0003-066X.55.2.241
more important than control of Type II errors, Baker, T. B., McFall, R. M., & Shoham, V. (2008).
Current status and future prospects of clinical psy-
hence the emphasis in psychology on alpha level
chology: Toward a scientifically principled approach
and continuing indifference to the power of studies to mental and behavioral health care. Psychological
(Cohen, 1994; Sedlmeier & Gigerenzer, 1989). The Science in the Public Interest, 9, 67–103.
Neyman–Pearson paradigm does not permit induc- Balaram, P. (2008). Science, invention, and Pasteur’s
tive inference about hypotheses to be made. Rather, quadrant. Current Science, 94, 961–962.
it permits inductive behavior, that is, the making of Balluerka, N., Gomez, J., & Hidalgo, D. (2005). The
decisions on the basis of evidence from statistical controversy over null hypothesis significance testing
tests, these decisions being akin to those made in revisited. Methodology: European Journal of Research
Methods for the Behavioural and Social Sciences, 1,
industrial quality-control contexts to control the 55–70. doi:10.1027/1614-1881.1.2.55
long-run rate of production of defective products
Barlow, D. H., Hayes, S. C., & Nelson, R. O. (1984). The
(Hubbard, 2004; Hubbard & Lindsay, 2008; see also scientist practitioner: Research and accountability in
Nickerson, 2000, and Wagenmakers, 2007). educational settings. New York, NY: Pergamon Press.
192
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Barlow, D. H., & Nock, M. K. (2009). Why can’t we be Bornstein, M. H., & Lamb, M. E. (1992). Development
more idiographic in our research? Perspectives on in infancy: An introduction (3rd ed.). New York, NY:
Psychological Science, 4, 19–21. doi:10.1111/j.1745- McGraw-Hill.
6924.2009.01088.x
Cairns, R. B. (1986). Phenomena lost: Issues in the study
Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single- of development. In J. Valsiner (Ed.), The individual
case experimental designs: Strategies for studying subject and scientific psychology (pp. 97–111). New
behavior change (3rd ed.). Boston, MA: Pearson. York, NY: Plenum Press.
Belar, C. D., & Perry, N. W. (1992). The National Chambless, D. L., & Hollon, S. D. (1998). Defining
Conference on Scientist-Practitioner Education and empirically supported therapies. Journal of Consulting
Training for the Professional Practice of Psychology. and Clinical Psychology, 66, 7–18. doi:10.1037/0022-
American Psychologist, 47, 71–75. doi:10.1037/0003- 006X.66.1.7
066X.47.1.71 Cohen, J. (1962). The statistical power of abnormal-social
Benjamin, L. T. (2005). A history of clinical psychol- psychological research: A review. Journal of Abnormal
ogy as a profession in America (and a glimpse of and Social Psychology, 65, 145–153. doi:10.1037/
its future). Annual Review of Clinical Psychology, 1, h0045186
1–30. doi:10.1146/annurev.clinpsy.1.102803.143758 Cohen, J. (1990). Things I have learned (so far). American
Bergin, A. E., & Strupp, H. H. (1970). New directions Psychologist, 45, 1304–1312. doi:10.1037/0003-066X.
in psychotherapy research. Journal of Abnormal 45.12.1304
Psychology, 76, 13–26. doi:10.1037/h0029634 Cohen, J. (1994). The earth is round (p < .05). American
Bergin, A. E., & Strupp, H. H. (1972). Changing fron- Psychologist, 49, 997–1003. doi:10.1037/0003-066X.
tiers in the science of psychotherapy. New York, NY: 49.12.997
Aldine. Committee on Training in Clinical Psychology. (1947).
Bernard, C. (1957). An introduction to the study of Recommended graduate training program in clini-
experimental medicine (H. C. Green, Trans.). New cal psychology. American Psychologist, 2, 539–558.
York, NY: Dover. (Original work published 1865) doi:10.1037/h0058236
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007).
Beutler, L. E., & Crago, M. (Eds.). (1991). Psychotherapy
Applied behavior analysis (2nd ed.). Upper Saddle
research: An international review of programmatic
River, NJ: Pearson.
studies. Washington, DC: American Psychological
Association. doi:10.1037/10092-000 Corrie, S., & Callahan, M. M. (2000). A review of the
scientist-practitioner model: Reflections on its
Blaich, C. F., & Barreto, H. (2001). Typological thinking,
potential contribution to counselling psychology
statistical significance, and the methodological diver-
within the context of current health care trends.
gence of experimental psychology and economics.
British Journal of Medical Psychology, 73, 413–427.
Behavioral and Brain Sciences, 24, 405.
doi:10.1348/000711200160507
Blampied, N. M. (1999). A legacy neglected: Restating the Cortina, J. M., & Dunlap, W. P. (1997). On the logic
case for single-case research in cognitive-behaviour and purpose of significance testing. Psychological
therapy. Behaviour Change, 16, 89–104. doi:10.1375/ Methods, 2, 161–172. doi:10.1037/1082-989X.
bech.16.2.89 2.2.161
Blampied, N. M. (2000). Comment: Single-case research Dar, R., Serlin, R. C., & Omer, H. (1994). Misuse of statis-
designs: A neglected alternative. American Psychologist, tical tests in three decades of psychotherapy research.
55, 960. doi:10.1037/0003-066X.55.8.960 Journal of Consulting and Clinical Psychology, 62,
Blampied, N. M. (2001). The third way: Single-case 75–82. doi:10.1037/0022-006X.62.1.75
research, training, and practice in clinical psy- Dehue, T. (2000). From deception trials to control
chology. Australian Psychologist, 36, 157–163. reagents: The introduction of the control group
doi:10.1080/00050060108259648 about a century ago. American Psychologist, 55,
Bootzin, R. R. (2007). Psychological clinical science: Why 264–268. doi:10.1037/0003-066X.55.2.264
and how we got to where we are. In T. R. Treat, R. R. Drabman, R. S., Hammer, D., & Rosenbaum, M. S.
Bootzin, & T. B. Baker (Eds.), Psychological clinical (1979). Assessing generalization in behavior modi-
science (pp. 1–28). New York, NY: Taylor & Francis. fication with children: The generalization map.
Bornstein, M. (1998). The shift from significance test- Behavioral Assessment, 1, 203–219.
ing to effect size estimation. In N. R. Schooler (Ed.), Edwards, R. (1987). Implementing the scientist-
Comprehensive clinical psychology: Vol. 3. Research and practitioner model: The school psychologist as data-
methods (pp. 313–349). Amsterdam, the Netherlands: based problem solver. Professional School Psychology,
Elsevier. 2, 155–161. doi:10.1037/h0090541
193
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
Erceg-Hurn, D. M., & Mirosevich, V. M. (2008). Modern in the age of managed care (2nd ed.). Boston, MA:
robust statistical methods: An easy way to maximize Allyn & Bacon.
the accuracy and power of your results. American Hays, W. L. (1963). Statistics for psychologists. New York,
Psychologist, 63, 591–601. doi:10.1037/0003-066X. NY: Holt, Rinehart & Winston.
63.7.591
Hersen, M., & Barlow, D. H. (1976). Single-case experi-
Fidler, F., Cumming, G., Thomason, N., Pannuzzo, mental designs: Strategies for studying behavior change.
D., Smith, J., Fyffe, P., . . . Schmitt, R. (2005). Oxford, England: Pergamon Press.
Toward improved statistical reporting in the Journal
of Consulting and Clinical Psychology. Journal of Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z.,
Consulting and Clinical Psychology, 73, 136–143. & Lutz, W. (1996). Evaluation of psychotherapy:
doi:10.1037/0022-006X.73.1.136 Efficacy, effectiveness, and patient progress.
American Psychologist, 51, 1059–1064. doi:10.1037/
Fisher, R. A. (1925). Statistical methods for research work- 0003-066X.51.10.1059
ers. London, England: Oliver & Boyd.
Hubbard, R. (2004). Alphabet soup: Blurring the dis-
Fisher, R. A. (1935). The design of experiments. London, tinctions between p’s and α’s in psychological
England: Oliver & Boyd. research. Theory and Psychology, 14, 295–327.
Frank, G. (1984). The Boulder model: History, rationale, doi:10.1177/0959354304043638
and critique. Professional Psychology: Research and Hubbard, R., & Lindsay, R. M. (2008). Why p values are
Practice, 15, 417–435. doi:10.1037/0735-7028. not a useful measure of evidence in statistical sig-
15.3.417 nificance testing. Theory and Psychology, 18, 69–88.
Frick, R. W. (1996). The appropriate use of null hypoth- doi:10.1177/0959354307086923
esis testing. Psychological Methods, 1, 379–390. Hubbard, R., Parsa, R. A., & Luthy, M. R. (1997). The
doi:10.1037/1082-989X.1.4.379 spread of statistical testing in psychology. Theory
Gigerenzer, G. (1991). From tools to theories: A heuristic and Psychology, 7, 545–554. doi:10.1177/0959354
of discovery in cognitive psychology. Psychological 397074006
Review, 98, 254–267. doi:10.1037/0033-295X. Jacobson, N. S., & Truax, P. (1991). Clinical significance:
98.2.254 A statistical approach to defining meaningful change
Gigerenzer, G., Swijtink, Z., Porter, T., Datson, L., in psychotherapy research. Journal of Consulting and
Beatty, J., & Kruger, L. (1989). The empire of chance: Clinical Psychology, 59, 12–19. doi:10.1037/0022-
How probability changed science and everyday life. 006X.59.1.12
Cambridge, England: Cambridge University Press. Jex, S. M., & Britt, T. W. (2008). Organizational psy-
Gopnik, A. (2009). Angels and ages: A short book about chology: A scientist-practitioner approach (2nd ed.).
Darwin, Lincoln, and modern life. London, England: Hoboken, NJ: Wiley.
Quercus. Johnston, J. M., & Pennypacker, H. S. (1993). Readings
Gould, S. J. (1985). The median isn’t the message. for strategies and tactics of behavioral research (2nd
Discover, 6, 40–42. ed.). Hillsdale, NJ: Erlbaum.
Gould, S. J. (1997). Life’s grandeur. London, England: Jones, R. R. (1978). A review of: Single-case experimen-
Vintage. tal designs: Strategies for studying behavior change
by Michel Hersen and David H. Barlow. Journal of
Hagen, R. L. (1997). In praise of the null hypothesis Applied Behavior Analysis, 11, 309–313. doi:10.1901/
statistical test. American Psychologist, 52, 15–24. jaba.1978.11-309
doi:10.1037/0003-066X.52.1.15
Kazdin, A. E. (1978). History of behavior modification:
Hammond, G. (1996). The objections to null hypoth- Experimental foundations of contemporary research.
esis testing as a means of analysing psychological Baltimore, MD: University Park Press.
data. Australian Journal of Psychology, 48, 104–106.
Kazdin, A. E. (2008). Evidence-based treatment and prac-
doi:10.1080/00049539608259513
tice: New opportunities to bridge clinical research
Harlow, L. L., Mulaik, S. A., & Steiger, J. H. (Eds.). and practice, enhance the knowledge base, and
(1997). What if there were no significance tests? improve patient care. American Psychologist, 63,
Mahwah, NJ: Erlbaum. 146–159. doi:10.1037/0003-066X.63.3.146
Hayes, S. C. (1981). Single-case research designs and Kendall, M. G., & Plackett, R. L. (Eds.). (1977). Studies
empirical clinical practice. Journal of Consulting and in the history of statistics and probability (Volume 2).
Clinical Psychology, 49, 193–211. doi:10.1037/0022- High Wycombe, England: Griffin.
006X.49.2.193
Kiesler, D. J. (1966). Some myths of psychotherapy
Hayes, S. C., Barlow, D. H., & Nelson-Gray, R. O. (1999). research and the search for a paradigm. Psychological
The scientist practitioner: Research and accountability Bulletin, 65, 110–136. doi:10.1037/h0022911
194
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kiesler, D. J. (1971). Experimental designs in psychother- Meehl, P. E. (1978). Theoretical risks and tabular aster-
apy research. In A. E. Bergin & S. L. Garfield (Eds.), isks: Sir Karl, Sir Ronald, and the slow progress of
Handbook of psychotherapy and behavior change soft psychology. Journal of Consulting and Clinical
(pp. 36–74). London, England: Wiley. Psychology, 46, 806–834. doi:10.1037/0022-006X.
46.4.806
Kihlstrom, J. F., & Kihlstrom, L. C. (1998). Integrating
science and practice in an environment of managed Michael, J. (1974). Statistical inference for individual
care. In D. K. Routh & R. J. DeRubes (Eds.), The sci- organism research: Mixed blessing or curse.
ence of clinical psychology: Accomplishments and future Journal of Applied Behavior Analysis, 7, 647–653.
directions (pp. 281–293). Washington, DC: American doi:10.1901/jaba.1974.7-647
Psychological Association. doi:10.1037/10280-012 Molenaar, P. C. M. (2004). A manifesto on psychology as
Kline, R. B. (2004). Beyond significance testing: Reforming idiographic science: Bringing the person back into
data analysis methods in behavioral research. scientific psychology, this time forever. Measurement:
Washington, DC: American Psychological Association. Interdisciplinary Research and Perspective, 2, 201–218.
doi:10.1037/10693-000 Morgan, D. L., & Morgan, R. K. (2001). Single-
Korchin, S. J. (1983). The history of clinical psychology: participant research design: Bringing science to
A personal view. In M. Hersen, A. E. Kazdin, & A. S. managed care. American Psychologist, 56, 119–127.
Bellack (Eds.), The clinical psychology handbook doi:10.1037/0003-066X.56.2.119
(pp. 5–19). New York, NY: Pergamon Press. Morgan, D. L., & Morgan, R. K. (2009). Single-case
Kratochwill, T. R., & Levin, J. R. (2010). Enhancing research methods for the behavioral and health sciences.
the credibility of single-case intervention research: Los Angeles, CA: Sage.
Randomization to the rescue. Psychological Methods, Morris, E. K. (2008). Sidney W. Bijou: The Illinois years,
15, 124–144. doi:10.1037/a0017736 1965–1975. Behavior Analyst, 31, 179–203.
Lerman, D. C. (2003). From the laboratory to community Newman, F. L., & Tejeda, M. J. (1996). The need for
application: Translational research in behavior analy- research that is designed to support decisions in
sis. Journal of Applied Behavior Analysis, 36, 415–419. the delivery of mental health services. American
doi:10.1901/jaba.2003.36-415 Psychologist, 51, 1040–1049. doi:10.1037/0003-
066X.51.10.1040
Lilienfeld, S. O., Lynn, S. J., & Lohr, J. M. (2003). Science
and pseudoscience in clinical psychology. In S. O. Newman, K. J. (2000). The current implementation status
Lilienfeld, S. J. Lynn, & J. M. Mohr (Eds.), Science of the Boulder model. Unpublished master’s the-
and pseudoscience in clinical psychology (pp. 1–14). sis, University of Canterbury, Christchurch, New
New York, NY: Guilford Press. Zealand.
Lindquist, F. F. (1940). Statistical analysis in educational Neyman, J. (1950). First course in probability and statis-
research. Boston, MA: Houghton Mifflin. tics. New York, NY: Holt.
Lindsley, O. R., & Skinner, B. F. (1954). A method for Nickerson, R. S. (2000). Null hypothesis significance
the experimental analysis of the behavior of psy- testing: A review of an old and continuing con-
chotic patients. American Psychologist, 9, 419–420. troversy. Psychological Methods, 5, 241–301.
doi:10.1037/1082-989X.5.2.241
Martin, P. R. (1989). The scientist-practitioner model and
O’Donnell, J. M. (1985). The origins of behaviorism:
clinical psychology: Time for a change? Australian
American psychology, 1870–1920. New York, NY:
Psychologist, 24, 71–92. doi:10.1080/00050068
New York University Press.
908259551
Olatunji, B. O., Feldner, M. T., Witte, T. H., & Sorrell,
Mazur, J. (2009). Learning and behavior. Upper Saddle J. T. (2004). Graduate training of the scientist-
River, NJ: Prentice Hall. practitioner: Issues in translational research and sta-
McDougall, D. (2005). The range-bound changing crite- tistical analysis. Behavior Therapist, 27, 45–50.
rion design. Behavioral Interventions, 20, 129–137. Orlitzky, M. (2011). How can significance tests be dein-
doi:10.1002/bin.189 stitutionalized? Organizational Research Methods.
McFall, R. M. (2007). On psychological clinical science. Advance online publication. doi:10.1177/1094428
In T. A. Treat, R. R. Bootzin, & T. B. Baker (Eds.), 111428356
Psychological clinical science: Papers in honor of Peterson, D. R. (2003). Unintended consequences:
Richard M. McFall (pp. 363–396). New York, NY: Ventures and misadventures in the education of pro-
Psychology Press. fessional psychologists. American Psychologist, 58,
McReynolds, P. (1997). Lightner Witmer: His life and 791–800. doi:10.1037/0003-066X.58.10.791
times. Washington, DC: American Psychological Porter, R. (1997). The greatest benefit to mankind. London,
Association. doi:10.1037/10253-000 England: HarperCollins.
195
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neville M. Blampied
Price, R. H., & Behrens, T. (2003). Working Pasteur’s Schatz, P., Jay, K. A., McComb, J., & McLaughlin, J. R.
quadrant: Harnessing science and action for com- (2005). Misuse of statistical tests in Archives of
munity change. American Journal of Community Clinical Neuropsychology publications. Archives
Psychology, 31, 219–223. doi:10.1023/A:10239 of Clinical Neuropsychology, 20, 1053–1059.
50402338 doi:10.1016/j.acn.2005.06.006
Raimy, V. C. (Ed.). (1950). Training in clinical psychology Schlinger, H. D. (1996). How the human got its spots.
(Boulder conference). New York, NY: Prentice Hall. Skeptic, 4, 68–76.
Reich, J. W. (2008). Integrating science and practice: Schmidt, F. L. (1996). Statistical significance testing and
Adopting the Pasteurian model. Review of General cumulative knowledge in psychology: Implications
Psychology, 12, 365–377. doi:10.1037/1089-2680. for training of researchers. Psychological Methods, 1,
12.4.365 115–129. doi:10.1037/1082-989X.1.2.115
Reisman, J. M. (1966). The development of clinical psychol- Schmidt, F. L., & Hunter, J. E. (1997). Eight common
ogy. New York, NY: Appleton-Century-Crofts. but false objections to the discontinuation of signifi-
cance testing in the analysis of research data. In L. L.
Reisman, J. M. (1991). A history of clinical psychology Harlow, S. A. Mulaik, & J. H. Steiger (Eds.), What if
(2nd ed.). New York, NY: Taylor & Francis. there were no significance tests? (pp. 38–64). Mahwah,
Rodgers, J. L. (2010). The epistemology of mathemati- NJ: Erlbaum.
cal and statistical modeling: A quiet methodologi- Schmidt, S. (2009). Shall we really do it again? The pow-
cal revolution. American Psychologist, 65, 1–12. erful concept of replication is neglected in the social
doi:10.1037/a0018326 sciences. Review of General Psychology, 13, 90–100.
Rorer, L. G. (1991). Some myths of science in psychol- doi:10.1037/a0015108
ogy. In D. Cicchetti & W. M. Grove (Eds.), Thinking Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of sta-
clearly about psychology: Vol. 1. Matters of public tistical power have an effect on the power of studies?
interest (pp. 61–87). Minneapolis: University of Psychological Bulletin, 105, 309–316. doi:10.1037/
Minnesota Press. 0033-2909.105.2.309
Rosenthal, R. (1995). Progress in clinical psychology: Is Seligman, M. E. P. (1995). The effectiveness of psycho-
there any? Clinical Psychology: Science and Practice, 2, therapy: The Consumer Reports study. American
133–150. doi:10.1111/j.1468-2850.1995.tb00035.x Psychologist, 50, 965–974. doi:10.1037/0003-066X.
Rossen, E., & Oakland, T. (2008). Graduate preparation 50.12.965
in research methods: The current status of APA- Shapiro, D. (2002). Renewing the scientist-practitioner
accredited professional programs in psychology. model. Psychologist, 15, 232–234.
Training and Education in Professional Psychology, 2,
42–49. doi:10.1037/1931-3918.2.1.42 Sheridan, E. P., Perry, N. W., Johnson, S. B., Clayman,
D., Ulmer, R., Prohaska, T., . . . Beckman, L. (1989).
Rourke, B. P. (1995). The science of practice and the Research and practice in health psychology. Health
practice of science: The scientist-practitioner model Psychology, 8, 777–779. doi:10.1037/h0090321
in clinical neuropsychology. Canadian Psychology/
Psychology Canadienne, 36, 259–277. Sidman, M. (1960). Tactics of scientific research. New
Routh, D. K. (1998). Hippocrates meets Democritus: A
history of psychiatry and clinical psychology. In A. S. Skinner, B. F. (1938). The behavior of organisms. New
Bellack & M. Hersen (Eds.), Comprehensive clinical York, NY: Appleton-Century-Crofts.
psychology: Vol. 1. Foundations (pp. 2–48). Oxford, Skinner, B. F. (1953). Some contributions of an experi-
England: Elsevier. mental analysis of behavior to psychology as a
Routh, D. K. (2000). Clinical psychology training: A his- whole. American Psychologist, 8, 69–78. doi:10.1037/
tory of ideas and practices prior to 1946. American h0054118
Psychologist, 55, 236–241. doi:10.1037/0003-066X. Skinner, B. F. (1954). A new method for the experimental
55.2.236 analysis of the behavior of psychotic patients. Journal
Rozin, R. (2009). What kind of empirical research of Nervous and Mental Disease, 120, 403–406.
should we publish, fund, and reward. Perspectives Skinner, B. F. (1956). A case history in scientific method.
on Psychological Science, 4, 435–439. doi:10.1111/ American Psychologist, 11, 221–233. doi:10.1037/
j.1745-6924.2009.01151.x h0047662
Rucci, A. J., & Tweney, R. D. (1980). Analysis of variance Smith, L. D., Best, L. A., Cylke, V. A., & Stubbs, A. D.
and the “second discipline” of scientific psychol- (2000). Psychology without p values: Data analysis at
ogy: A historical account. Psychological Bulletin, 87, the turn of the 19th century. American Psychologist,
166–184. doi:10.1037/0033-2909.87.1.166 55, 260–263. doi:10.1037/0003-066X.55.2.260
196
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Soldz, S., & McCullogh, L. (Eds.). (2000). Reconciling editorial policies regarding statistical significance
empirical knowledge and clinical experience: The and effect size. Theory and Psychology, 10, 413–425.
art and science of psychotherapy. Washington, DC: doi:10.1177/0959354300103006
American Psychological Association. doi:10.1037/
10567-000 Valsiner, J. (1986). Where is the individual subject
in scientific psychology? In J. Valsiner (Ed.), The
Staines, G. L. (2008). The causal generalization paradox: individual subject and scientific psychology (pp. 1–14).
The case of treatment outcome research. Review of New York, NY: Plenum Press.
General Psychology, 12, 236–252. doi:10.1037/1089-
2680.12.3.236 Vespia, K. M., & Sauer, E. M. (2006). Defining character-
istic or unrealistic ideal: Historical and contemporary
Stokes, D. E. (1997). Pasteur’s quadrant: Basic science and perspectives on scientist-practitioner training in coun-
technological innovation. Washington, DC: Brookings selling psychology. Counselling Psychology Quarterly,
Institution Press. 19, 229–251. doi:10.1080/09515070600960449
Streiner, D. L. (2006). Sample size in clinical research: When Wagenmakers, E. (2007). A practical solution to the per-
is enough enough? Journal of Personality Assessment, 87, vasive problems of p values. Psychonomic Bulletin and
259–260. doi:10.1207/s15327752jpa8703_06 Review, 14, 779–804.
Stricker, G. (1975). On professional schools and profes-
Wainer, H. (1999). One cheer for null hypothesis sig-
sional degrees. American Psychologist, 30, 1062–1066.
nificance testing. Psychological Methods, 4, 212–213.
doi:10.1037/0003-066X.30.11.1062
doi:10.1037/1082-989X.4.2.212
Stricker, G. (2000). The scientist-practitioner model:
Gandhi was right again. American Psychologist, 55, Wessley, S. (2001). Randomised controlled-trials: The
253–254. doi:10.1037/0003-066X.55.2.253 gold standard. In C. Mace, S. Moorey, & B. Roberts
(Eds.), Evidence in the psychological therapies (pp.
Tavaris, C. (2003). The widening scientist-practitioner 46–60). Hove, England: Brunner-Routledge.
gap. In S. O. Lilienfeld, S. J. Lynn, & J. M. Mohr
(Eds.), Science and pseudoscience in clinical psychol- Wilkinson, L., & Task Force on Statistical Inference.
ogy (pp. ix–xviii). New York, NY: Guilford Press. (1999). Statistical methods in psychology journals:
Guidelines and explanations. American Psychologist,
Thompson, T. (1984). The examining magistrate for 54, 594–604. doi:10.1037/0003-066X.54.8.594
nature: A retrospective review of Claude Bernard’s
An introduction to the study of experimental medicine. Witmer, L. (1996). Clinical psychology. American
Journal of the Experimental Analysis of Behavior, 41, Psychologist, 51, 248–251. (Original work published
211–216. doi:10.1901/jeab.1984.41-211 1907) doi:10.1037/0003-066X.51.3.248
Thompson, T., & Hackenberg, T. D. (2009). Wolf, M. M. (1978). Social validity: The case for subjec-
Introduction: Translational science lectures. Behavior tive measurement or how applied behavior analysis is
Analyst, 32, 269–271. finding its heart. Journal of Applied Behavior Analysis,
11, 203–214. doi:10.1901/jaba.1978.11-203
Thorne, F. C. (1945). The field of clinical psychology,
past, present, future [Editorial]. Journal of Clinical Wright, D. B. (2009). Ten statisticians and their impacts
Psychology, 1, 1–20. for psychologists. Perspectives on Psychological Science,
Ullmann, L. P., & Krasner, L. (Eds.). (1966). Case stud- 4, 587–597. doi:10.1111/j.1745-6924.2009.01167.x
ies in behavior modification. New York, NY: Holt, Yates, F., & Mather, K. (1963). Ronald Aylmer Fisher.
Rinehart & Winston. Biographical Memoirs of Fellows of the Royal Society of
Ulrich, R., Stachnik, T., & Mabry, J. (1966). Control of London, 9, 91–120.
human behavior. Glenview, IL: Scott, Foresman. Ziliak, S. T., & McCloskey, D. N. (2008). The cult of
Vacha-Haase, T., Nilsson, J. E., Reetz, D. R., & statistical significance. Ann Arbor: University of
Thompson, B. (2000). Reporting practices and APA Michigan Press.
197
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 9
Visual Analysis in Single-Case

Research
Jason C. Bourret and Cynthia J. Pietras
The visual analysis, or inspection, of graphs show- behavior patterns (see Ferster & Skinner,
ing the relation between environmental (indepen- 1957/1997). An example of a cumulative record is
dent) variables and behavior is the principal method shown in Figure 9.1. On a cumulative record, equal
of analyzing data in behavior analysis. This chapter horizontal distances represent equal lengths of time;
is an introduction to such visual analysis. We begin equal vertical distances represent equal numbers of
by describing the components, and construction, responses. The slope of the curve in a cumulative
of some common types of graphs used in behavior record indicates the rate of responding. Researchers
analysis. We then describe some techniques for ana- sometimes include an inset scale on cumulative
lyzing graphic data, including graphs from common records to indicate the rate of responding repre-
single-subject experimental designs. sented by different slopes, although usually more
precise calculations of rate are also provided. Small
vertical lines oblique to the prevailing slope of the
Types of Graphs and Their
line, traditionally called pips, typically indicate rein-
Construction
forcer deliveries. When the response pen reaches the
Of the many ways to graph data (see Harris, 1996), top of the page, it resets to the bottom, producing a
the graph types most frequently used by behavior straight vertical line. Researchers may also program
analysts are cumulative frequency graphs, bar the response pen to reset at designated times (e.g.,
graphs, line graphs, and scatterplots. Each of these is when a schedule change occurs), to visually separate
described in more detail in this section. data collected under different conditions. Cumula-
tive records were traditionally generated by now-
Cumulative Frequency Graphs obsolete, specially designed machines (cumulative
Cumulative frequency graphs show the cumulative recorders). More contemporarily, computer software
number of responses across observation periods. programs that record and plot each response as it
The earliest, and most common, such graph used by occurs have been used to generate these records.
behavior analysts is the cumulative record. In the Cumulative records can also be constructed after
other graph types discussed in this section, mea- data collection is complete, if the time of occurrence
sures of behavior during an observation period are of each response and all other relevant events during
collapsed into a single quantity (e.g., mean response a session are recorded.
rate during a session) that is represented on a graph Although the cumulative record was one of the
by a single data point. By contrast, cumulative most commonly used graphs in the early years of the
records show each response and when it occurred experimental analysis of behavior, it has since fallen
during an observation period. Thus, cumulative out of favor as researchers increasingly present data
records provide a detailed picture of within-session averaged across a single or multiple sessions. It is
DOI: 10.1037/13937-009
199
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
HIGHER RATE
“SMOOTH” “GRAINY”
RESPONDING
RESPONSES
RESPONDING
LOWER RATE
NO RESPONDING
PIPS
EVENT PIN
TIME
Figure 9.1. Example of patterns that may be observed on a cumulative record.

Cumulative responses are shown on the vertical axis, and time is shown on the
horizontal axis. Each response moves the response pen a constant distance in
the vertical direction, and the paper moves vertically at a constant speed. Shown
(right to left) are smooth curves indicating constant rates of responding, grainy
curves indicating irregular patterns of responding, shallow curves indicating low
rates of responding, and steep curves indicating high rates of responding. Flat
portions indicate no responding. Pips (downward deflections of the response
pen) usually indicate reinforcer deliveries. Movements of the event pen are used
to signal changes in experimental contingencies or stimulus conditions. Data are
hypothetical.
useful in the experimental analysis of behavior not across conditions. Typically, bars are separated
as a primary means of data analysis, but as a means from each other, but related bars may be grouped
of monitoring within-session performance. Another together. One variation of the standard (vertical) bar
notable exception to the decline of the cumulative graph is a horizontal bar graph, in which the cate-
record is research published by Gallistel and his col- gorical variable is plotted on the y-axis. On these
leagues (e.g., Gallistel et al., 2007). In this research graphs, the length of the bar along the x-axis shows
line, cumulative records (some of them quite cre- the value of the dependent variable. Bars graphs may
atively constructed with more than simple responses also be drawn so that bars can deviate in either
on the vertical axis) are used extensively to better direction from a center line. Such graphs may be
understand the process by which organisms allocate used, for example, to show increases or decreases
their behavior between concurrently available from baseline values that are represented by a center
sources of food. horizontal line at zero. Bar graphs are similar to his-
tograms, but histograms (in which the bars touch
Bar Graphs each other) have interval x-axis values and typically
Bar graphs show discrete categories (i.e., a nominal show frequency distributions (see Figure 9.3). In
scale; Stevens, 1946) along the horizontal (x) axis bar graphs, the y-axis scale usually begins at the
and values of the dependent variable on the vertical lowest possible value (e.g., zero) but may begin at a
(y) axis (Shah, Freedman, & Vekiri, 2005). In higher value if the low range would be devoid of
behavior analysis, bar graphs are often used to show data. Constraining the lower range of y-axis values
percentages (e.g., of correct responses) or average will make differences across conditions appear
performance across stable sessions or conditions. As bigger than they would have been had the range
shown in Figure 9.2, bar graphs facilitate compari- extended to zero, a factor to consider when evaluat-
sons of performances (i.e., the height of each bar) ing the clinical (or other) importance of the visually
200
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Visual Analysis in Single-Case Research
100 When group statistical designs are used, bar

graphs frequently summarize the differences in
80 group mean performances. Reliance on statistical
analyses of grouped data may lead to the omission of
Percent Correct
60 error bars from such graphs, a practice that obscures

the size of individual differences within groups. Fig-
40 ure 9.4 shows a variation on the between-groups bar
graph that displays the performance of individual
20
participants (individual data points) while maintain-
ing the ease of comparing measures of central ten-
0
dency (height of the bars). Graphs constructed in
No 5-s 20-s this way make it possible to evaluate to what extent
Timeout Timeout Timeout
the height of the bar describes the behavior of indi-
100 vidual participants. A second advantage of this type
of bar graph is that it allows readers to evaluate
80
whether the within-group variance is normally dis-
tributed, an important factor when evaluating the
Percent Correct
60
appropriateness of the statistical analyses used.
Finally, the graphing conventions of Figure 9.4
40
encourage the researchers to consider those individ-
uals in the treatment group for whom the treatment
20
produced no positive effect. As often noted by
0
No 5-s 20-s
Timeout Timeout Timeout
Figure 9.2. Examples of bar graphs. Data are hypo-

thetical.
0.32
0.28
Proportion of Responses
0.24
0.20
0.16
0.12
0.08
0.04
Figure 9.4. A bar graph that shows mean perfor-
0.00
0 1 2 3 4 5 6 7 8 9 >10
mance and the performance of individuals making up
the mean. The height of the bar shows the mean percent-
Interresponse Time Bins (s) age of urine samples negative for cocaine and opiates,
and the closed circles show the percentage of negative
Figure 9.3. Example of a histogram. Data are hypo- samples for each individual undergoing employment-
thetical. based abstinence reinforcement treatment for cocaine
dependence. From “Employment-Based Abstinence
apparent difference. When bar graphs show mea- Reinforcement as a Maintenance Intervention for the
sures of central tendency (e.g., means), error bars Treatment of Cocaine Dependence: A Randomized
Controlled Trial,” by A. DeFulio, W. D. Donlin, C. J.
(the vertical lines in Figure 9.2) should be included Wong, and K. Silverman, 2009, Addiction, 104, p. 1535.
to depict the variance in the data (e.g., between- Copyright 2009 by John Wiley & Sons, Ltd. Used with
session differences in percentage correct). permission.
201
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
Sidman (1960), the data from these individuals axis. Tick marks can point inward or outward, but
serve to illustrate that the behavior is not fully they should point outward if inward-pointing ticks
understood and, by investigating the factors affect- would interfere with the data. Tick marks are
ing these individuals’ behavior further, more effec- labeled to indicate the interval value. Tick-mark
tive interventions will follow. intervals should be frequent enough that a reader
can determine the value of a data point, but not so
Line Graphs (Time Series and Relational) frequent that the axis becomes cluttered. Intervals
Line graphs are used to depict behavior across time between major ticks may be marked with unlabeled
(time-series line graphs) or relations between two minor ticks. In Figure 9.5, for example, every fifth
variables (relational line graphs; see Tufte, 1983). interval is labeled, and minor tick marks indicate
With time-series line graphs (e.g., see Figure 9.5), intervals between them. If multiple graphs appear in
the horizontal axis (referred to as the abscissa or the a figure with the same axis scale, then only the tick
x-axis) illustrates the time point at which each data marks on the bottom and left graphs, respectively,
point was collected, and behavior is plotted at each are labeled. The x- and y-axis scales should be large
of these time points on the vertical y-axis (ordinate). enough to encompass the full range of the data;
On relational line graphs, the x-axis shows values of however, they should not exceed the data range, or
the independent variable, and the y-axis shows a the graph will contain empty space and compress
central tendency measure of the dependent variable. the data unnecessarily. Starting the axis scales at val-
Data points in both types of line graphs are con- ues below the minimum data range may make data
nected with straight lines. points at the minimum value (e.g., zero) more visi-
Figure 9.5 shows the parts of a time-series line ble by preventing them from falling directly on the
graph. Axis labels indicate the variables plotted. If x- or y-axis. As with bar graphs, line graphs nor-
multiple graphs appear in a figure with the same mally start at zero, but if the data range is great,
axes, then, to reduce the amount of text in the fig- there may be a break in the axis with the larger
ure, only the x- and y-axes on the bottom and left- numbers indicated after the break.
most graphs, respectively, are labeled (for an When plotting data for individual participants
example, see Figure 9.6). The scales of both the in separate graphs within a single figure, it is
x- and y-axes are divided into intervals. Successive sometimes not realistic to represent data for each
intervals on an axis are equally sized and are marked
with short lines, called tick marks, that intersect the
Figure 9.6. Examples of linear and logarithmic (log)

scales. The upper two graphs show data plotted using
linear y-axis scales. The lower two graphs show the
Figure 9.5. Diagram of parts of a time-series line same data, but plotted using log (base 10) y-axis scales.
graph. Data are hypothetical. Data are hypothetical.
202
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
articipant within a single range on the y-axis (e.g.,

p the space allocated to each phase. Figure legends,
the range for one participant may be between one and phase labels, are usually placed within the rect-
and 10 responses and for another, between 400 and angular space created by the x- and y-axes. Figure
450 responses). It is always better to use the same captions are placed below graphs and describe what
ranges on the y-axis, but when this is not possible, is plotted, the axes, any abbreviations or symbols
different ranges for different participants may be that appear in the graph, and any axis breaks.
used, and this deviation must be noted. Linear interval scales are the most common
The aspect ratio, or the height-to-width ratio of scales used in time-series line graphs, but logarith-
the y- and x-axes, should not distort the data. Too mic (log) interval scales and semi-log interval scales
great an aspect ratio (i.e., a very tall graph) may (in which the x- or y-axis is log scaled and the other
magnify variability, or small effects, and too small an is linearly scaled) are also used. Log scales are help-
aspect ratio (i.e., a very long graph) may obscure ful in more normally distributing data sets that are
important changes in behavior or variability in a skewed toward large values (Cleveland, 1994),
data set (Parsonson & Baer, 1986). A 2:3 y:x aspect transforming curvilinear data into linear data (Shull,
ratio (Parsonson & Baer, 1978) or a 1.0:1.618 aspect 1991) and showing proportional changes in behav-
ratio (Tufte, 1983) has been recommended. Breaks ior (Cooper, Heron, & Heward, 1987). Because the
on the y-axis may be used if there are outlier data logarithm of zero is undefined, log scales have no
points. Outliers are idiosyncratic data points that far zero. Log base 10, base 2, and base e (natural logs)
exceed the range of other data (e.g., data points are the most common log scales (see Cleveland,
more than 3 standard deviations from the mean). 1994, for some recommendations for the use of vari-
Breaks on the x-axis may be used if there are breaks ous log bases).
in data collection. Illustrations of data plotted on both a linear scale
Data points are marked with symbols, and a data and a semi-log (base 10) scale are shown in Figure
path is created by connecting the data points with 9.6. In the upper left graph, response rates in Phase
straight lines. When multiple data paths are shown B appear to increase more quickly between sessions
on a graph, each data type is represented by a dis- than in Phase A. This difference, however, may be
tinct symbol, and a central figure legend provides a attributed to the greater absolute value of the
concise description of each path. A common graph- response rate. Plotted on a log scale (lower left
ing convention in applied behavior analysis is to graph), it is visually apparent that the rate of change
describe each data path with text and an arrow is similar in both phases. In the upper right graph,
pointing from the text to the corresponding data there appears to be a large shift in performance from
path. Using a central legend, as in Figure 9.5, facili- Phase A to Phase B. The arithmetic scale of the
tates the transmission of information because scan- y-axis, however, compresses the low rates in Phase A.
ning the graph is not required to find figure legend When data are plotted on a log scale, the low rates
information. A second advantage of the central leg- are more visible and the increase in responding in
end is that it avoids the possibility that arrows Phase B can be seen to be part of an increasing trend
pointing to a particularly high or low data point may that began in Phase A.
influence the visual analysis of the data.
Phase changes in time-series line graphs are Scatterplots
denoted by vertical lines extending the length of the Scatterplots present a dependent variable in relation
y-axes. Phase-change lines are placed between the to either an independent variable (in which case the
last data point of a condition and the first data point graph may be described as a relational graph; see
of the new condition. Data paths are broken across Tufte, 1983) or another dependent variable. When
phase-change lines to avoid the appearance that both measures are dependent variables, either measure
behavior change produced by an independent vari- can be plotted on the horizontal axis, although if
able occurred before the change in condition. one measure is conceptualized as a predictor vari-
Descriptive phase labels are centered at the top of able, it is plotted on the x-axis, and the other, the
203
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
be used to show other types of interactions, such as

changes in interresponse time distributions on a
reinforcement schedule across sessions (Gentry,
Weiss, & Laties, 1983) or effects of different
drug doses on response run length distributions
(Galbicka, Fowler, & Ritch, 1991).
Other graphing techniques have been used to
depict specific kinds of relations. Staddon and Sim-
melhag (1971), for example, used detailed flow
charts to graphically show the conditional probabili-
ties of different responses given an initial response.
Davison and Baum (2006) depicted the number of
Figure 9.7. Example of a scatterplot. Data responses to different alternatives in a choice experi-
are hypothetical. ment as different-sized circles (bubbles). This tech-
nique could also be useful in showing, for example,
criterion variable, is plotted on the y-axis. An exam- time allocated to playing with multiple toys by a
ple of a scatterplot is shown in Figure 9.7. In this child across successive time periods.
figure, which shows data from a hypothetical experi- These examples are but a few of specialized
ment investigating choice between two concurrently graphs that may be useful in enhancing the visual
available reinforcement schedules, the log of the depiction of specific data sets or aspects of data sets.
ratio of response rates on the right and left alterna- For a more complete description of graph types, see
tives is plotted on the y-axis and the log of the ratio Harris (1996). Cleveland and McGill (1984, 1985)
of reinforcement rates is plotted on the x-axis. In offered some useful advice on how to choose graph
scatterplots, data points are not connected with types to show data with maximum clarity. In under-
lines, usually because measures are independent of taking a graphical analysis of data, there are no
each other (e.g., they are data points from different immutable rules concerning which graphs to use
conditions or participants) or because they are not for depicting what. Use is based on precedence, but
sequentially related. Sometimes, however, lines or investigators also need to think outside the axes
curves are fit to data on scatterplots to indicate the (so to speak) in using graphs to tell the story of
form of the relation between the two variables (see their data.
Interpreting Relational Graphs section). The line in
Figure 9.7 shows the best-fitting linear regression General Recommendations for
line. That data points fall near this line indicates a Graph Construction
linear relation (matching) between response rates Many features of a graph influence a reader’s reac-
and reinforcement rates. tion to the data. Even small details such as tick
marks, axis scaling, data symbols, aspect (y:x) ratio,
Other Types of Graphs and so forth can affect a graph’s impact, and poor
The types of graphs we have discussed do not repre- graphing methods can lead to misinterpretations of
sent an exhaustive list of the types of graphs used by results (Cleveland, 1994). Creating graphs that are
behavior analysts and subjected to visual analysis. accurate, meaningful, rich in information, yet read-
For example, sometimes examining data across time ily interpretable, therefore, requires planning, exper-
and within individual sessions is useful, in which imenting, reviewing, and close attention to detail
case a three-dimensional graph would be appropri- (Cleveland, 1994; Parsonson & Baer, 1992). For
ate, with the dependent variable on the y-axis, some additional recommendations on producing
within-session time on the x-axis, and successive useful graphs, see Baron and Perone (1998), Cleve-
sessions on the third (z) axis (e.g., Cançado & land (1994), Johnston and Pennypacker (1993),
Lattal, 2011). Three-dimensional graphs may also Parsonson and Baer (1978, 1986), and Tufte (1983).
204
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
When preparing graphs for publication, the Publica- Figure 9.2). When making this comparison, atten-
tion Manual of the American Psychological Association tion should be given to the y-axis scale to determine
(American Psychological Association, 2010) also whether the range has been truncated. In bar graphs
offers valuable advice. depicting average performance within a phase, a
critical element to evaluate is the length of the error
bars. Very long error bars suggest that the perfor-
Interpreting Graphical Data
mance may not have been stable, and so it will be
In the sections that follow, we describe some strate- important to evaluate the stability criterion used. If
gies for visually analyzing graphical data presented the range of the data is used, long error bars may
in cumulative records, bar graphs, time-series line also occur if an outlying data point was included in
graphs, and scatterplots. We also discuss the visual the data set depicted in the graph. If so, then the
analysis of graphical data generated by some com- average value depicted by the height of the bar may
monly used single-subject experimental designs. not represent most of the data; in such cases, the
If the graph is published, the first step in visual median would be a better measure of central ten-
analysis is to determine what is plotted by reading dency. Error bars also indicate the overlap of data
all of the text describing the graph, including the points across conditions. For example, Figure 9.2
axis labels, condition labels, figure legend, and fig- shows results from a hypothetical experiment that
ure caption. The next step is the analysis of patterns evaluated the effects of three time-out durations
in the graphical data. after incorrect responses on match-to-sample perfor-
mance. The height of the bar is the mean, and the
Interpreting Cumulative Records error bar shows the standard deviation. In the top
In cumulative records, changes in rate of responding graph, the error bars are long, and the mean of the
and variability in responding are analyzed by exam- 20-second condition overlaps with the variance in
ining changes in the slope of the records (Johnston the 5-second condition. Thus, differences between
& Pennypacker, 1993). Several patterns that may be the 5- and 20-second time-out duration conditions
distinguished in cumulative records are shown in are less convincing than the difference between the
Figure 9.1, a hypothetical cumulative record. The no time-out and the 20-second conditions. Error
first smooth curve shows responding occurring at a bars provide no information about trends in the
steady, constant rate, whereas the second shows data, however, and a reader must look to the text of
grainy responding, or responding occurring in the article or to other graphs for evidence that the
unsystematic bouts of high and low rates separated performances plotted in a bar graph represent stable
by varying periods of not responding. The flat por- responding.
tion of the third curve indicates no responding. The Care should also be taken to consider which
greater slope of the fourth curve compared with the measure of variability is represented by the error
third curve indicates a higher rate of responding. bars. The standard deviations plotted in Figure 9.2
Cumulative records also allow an analysis of quantify the average deviation of each data point
responding at a more local level. For example, in from the condition mean and, therefore, are an
Figure 9.1, the second curve from the left, between appropriate measure of variability when mean val-
the third and fourth pip, shows that responding ues are reported (interquartile ranges usually
occurred first at a low rate, then rapidly increased, accompany medians). Some errors bars will depict
then gradually decreased again before the reinforcer the standard error of the mean, and readers should
delivery. Such a fine-grained analysis is not possible interpret these with caution. The standard error
with other graph types. of the mean is used to estimate the standard devia-
tion among many different means sampled from a
Interpreting Bar Graphs normally distributed population of values. As such,
When visually analyzing bar graphs, the relative it tells one less about the variability in the data than
heights of bars are compared across conditions (see does the standard deviation. Moreover, the standard
205
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
error of the mean is calculated by dividing the sam- inhibit one’s ability to detect small but clinically
ple standard deviation by the square root of n (i.e., important effects of an experimental manipulation
the number of values used to calculate the mean); (Sidman, 1960). Thus, whenever possible, condi-
thus, error bars depicting the standard error of the tions should remain unchanged until stability is
mean will be increasingly more narrow than the achieved.
standard deviation as the number of data points Stability of time-series data may be assessed by
included in the data set increases. If the standard visual inspection or quantitative criteria (see Perone,
error of the mean had been plotted in Figure 9.2 1991; Sidman, 1960; Chapter 5, this volume). Both
instead of the standard deviation, the visually appar- will evaluate bounce and trend, with the latter
ent difference between all three conditions would catching patterns that may be missed by the quanti-
seem greater even with no change in the data set tative criterion. Bounce refers to unsystematic
plotted. changes in behavior across successive data points,
The general strategies we have outlined (i.e., whereas trend refers to a systematic (directional)
consider the difference in the measure of central change (Perone, 1991).
tendency in light of the variability in the data to Figure 9.8 shows baseline data for two partici-
evaluate how convincing the difference is) are for- pants. The baseline depicted in the top panel has
malized by common inferential statistical tests. considerably more between-session variability than
Behavior analysts wanting to reach a broader audi-
ence of scientists (including extramural grant
reviews), professionals, and public policymakers
may wish to use these tests in addition to conduct-
ing a careful visual analysis of the data.
Analyzing Time-Series Data in

Line Graphs
The analysis of time-series data is the most prevalent
visual inspection practice in behavior analysis. Basic
and applied behavior analysts use visual analysis
techniques to determine when behavior has stabi-
lized within a phase, to judge whether behavior has
changed between phases, and to evaluate the evi-
dence that the experimental variable has affected
individual behavior.
Evaluating stability. Once an experiment is under-

way, one of the first decisions a researcher must
make is, When should a phase change be made? In
most cases, this question is answered by evaluating
the stability (i.e., consistency) of responding over
time (see Chapter 5, this volume). If behavior is
not stable before the condition change (e.g., there
is a trend in the direction of the anticipated treat-
ment effect), then attributing subsequent shifts in Figure 9.8. Hypothetical baseline data
responding to the experimental manipulation will with added mean lines (solid horizontal
be unconvincing (Johnston & Pennypacker, 1993). lines), range lines (long dashed horizon-
tal lines), trimmed range lines (short
Moreover, an unstable baseline (i.e., one contain- dashed horizontal lines), and regression
ing a great deal of between-session variability) will lines (solid trend lines).
206
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
that depicted in the lower panel. The extent to minimum and maximum y-axis data points are
which the researcher will be concerned with this located, and those values are marked at the corre-
bounce in the data will depend on how large the sponding x-axis midpoint. Finally, two lines on the
treatment effect is likely to be. If a very large effect is graph are drawn connecting the two minimum data
expected, then the intervention data should fall well points from each half and the two maximum data
outside of the baseline range, and therefore the rela- points from each half. Converging lines suggest
tively weak experimental control established in the decreasing variability (bounce) across the phase,
baseline phase would be acceptable. If, however, a diverging lines suggest increasing variability, and
smaller effect is anticipated, then the intervention parallel lines suggest that the variability is constant.
data are unlikely to completely separate from the The next characteristic of the baseline data to be
range of data in the baseline, making detection of an considered, when deciding when to change phases,
intervention effect impossible. Under these condi- is the extent to which there is a trend in the data. A
tions, the researcher would be well served to further first step can be to plot a line of best fit (i.e., linear
identify the source of variability in the baseline. regression) through the baseline data. Any graphing
Indeed, if the researcher succeeds in this endeavor, a software package will suffice. Researchers should be
potent behavior-change variable may be identified. aware, however, that a line of best fit can be unduly
Visually analyzing bounce may be facilitated by affected by outliers (Parsonson & Baer, 1992). One
the use of the horizontal lines shown in Figure 9.8. alternative to linear regression that was recom-
The solid horizontal line shows the mean of the mended by Cleveland (1994) is the curve-smoothing
entire phase (i.e., the mean level of the data path) loess technique. The loess technique is less sensitive
and allows one to see graphically how much each to outliers and does not assume that data will con-
data point deviates from an ostensibly appropriate form to any particular function (e.g., a straight
measure of central tendency.1 The dashed lines fur- line). This technique smoothes data and makes pat-
thest from the mean line illustrate the range of the terns more visible by plotting, for each x-axis value,
data (i.e., they are drawn through the single data an estimate of the center of the distribution of y val-
point furthest from the mean), whereas the dashed ues falling near that x value (akin to a moving aver-
lines within these dashed lines show a trimmed age; for descriptions, see Cleveland, 1994; Cleveland
range in which the furthest data point from the & McGill, 1985). Linear regression, however, has
mean is ignored (see Morley & Adams, 1991). the advantage of being a more widely used tech-
Drawing range and trimmed range lines may be use- nique, and it quantifies the linear relation between
ful when considering how large the intervention the two variables (i.e., estimates the slope and
effect will have to be to discriminate a difference y-intercept).
between the baseline and intervention data. Clearly, In the upper panel of Figure 9.8, the line of best
to produce a visually apparent difference, the inter- fit indicates an upward trend in the baseline data,
vention implemented in the top panel will have to suggesting that if no intervention is implemented,
produce a much larger effect than that implemented the rate of response will continue to increase over
in the bottom panel. Neither range lines nor time. This is problematic if one expects the interven-
trimmed range lines will depict changes in variabil- tion to increase response rates. In the lower panel of
ity within a condition, however. To visualize Figure 9.8, the trend line is horizontal and overlaps
changes in variability within conditions, Morley and with the mean line. Thus, in the absence of an
Adams (1991) suggested plotting trended range experimental manipulation, the best prediction
lines. To construct these, the data in a condition are about future behavior is that it will remain stable
divided in half along the x-axis, and the middle with little between-session variability. Baseline data
x-axis value of each half is located. For each half, the need not be completely free of trends before a phase
Plotting a mean line is appropriate only if the data in the phase are free of extreme values that will pull the mean line away from the center of the dis-
1
tribution. Under such cases, a median line would be a better visual analysis tool.
207
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
is ended and the intervention is begun. If the base- different experimenters conducting sessions,
line data are trending down (up), and the interven- changes in levels of food deprivation, or perhaps
tion is anticipated to increase (decrease) responding, practice effects if two sessions are conducted each
then the baseline trend is of little concern. A modest day. Cycles may be difficult to detect if there is
trend in the direction of the anticipated intervention a good deal of between-session variability, but
effect is also acceptable as long as the intervention plotting data in various formats may help reveal
proves to produce a large change in trend and mean cyclical patterns. For example, plotting each data
level. Finally, continuing a baseline until it is free of point as a deviation from the mean using a vertical
trends and the bounce is minimal is sometimes bar graph can make patterns in the variability more
impractical. In applied settings, it may be unethical apparent (see Morley & Adams, 1991).
to continue a baseline until stability has been The same strategies for evaluating the stability of
achieved because to do so delays the onset of treat- baseline data are used to evaluate the stability of
ment. These practical and ethical concerns, how- data in an intervention phase. Figure 9.10 repeats
ever, must be balanced with the goal of constraining the data from Figure 9.8, but adds data from an
the effects of extraneous variables so that orderly intervention phase. In the upper panel, the line
effects of independent variable manipulations can be of best fit reveals an upward trend in the interven-
observed in subsequent conditions. It may be of lit- tion phase, although the final four data points sug-
tle use (and may also be considered unethical) to gest that the behavior may have asymptoted. The
conduct an intervention if the data are so variable researcher collecting these data should continue the
that it is impossible to interpret the effects of treat- intervention phase to determine whether the perfor-
ment. Thus, researchers should be especially con- mance has reached an asymptote or will increase fur-
cerned with achieving stability when treatment effects ther given continued exposure to the intervention.
are unknown or are expected to be small (i.e., when
one is conducting research rather than practice).
Visual inspection of trends within a data set
sometimes reveals nonlinear repetitive patterns, or
cycles. Some cycles result from feedback loops cre-
ated by self-regulating behavior–environment inter-
actions (Baum, 1973; Sidman, 1960), whereas others
result from extraneous variables. Identifying the
source of cyclical patterns is sometimes necessary to
produce behavior change. In Figure 9.9, for exam-
ple, every other data point is higher than the preced-
ing one. Such a pattern could be the result of
Figure 9.10. Hypothetical baseline and intervention

data with added mean lines (solid horizontal lines),
range lines (long dashed horizontal lines), trimmed
Figure 9.9. Example of a figure range lines (short dashed horizontal lines), and regres-
showing a cyclical pattern. Data sion lines (solid trend lines). Baseline data are the same
are hypothetical. as in Figure 9.8.
208
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
In the lower panel, a similar upward trend may be change in trend occurs (at least three data points are
observed in the intervention phase, but over the required to detect a trend). A short latency to
final 10 sessions of the phase, the performance has change indicates that the experimental manipulation
stabilized because there is little bounce around the produced an immediate effect on behavior, whereas
mean line and no visually apparent trend. a long latency to change indicates either that an
extended exposure to the change in the independent
Evaluating differences across phases. The second variable is required before behavior changes (such
use of visual analysis of time-series data involves as during extinction) or that the change is caused by
comparing the baseline and intervention data to an extraneous variable. Again, we consider the ques-
determine whether the difference makes a compel- tion of the causal relation between the behavior
ling case that behavior has changed between phases. change and the intervention later in the chapter.
Determining whether behavior change was an effect In the top panel of Figure 9.10, approximately
of the intervention (assuming a compelling differ- six sessions were required before the trend and
ence is observed) is a different matter, and one that mean level in the intervention phase appear distin-
we consider in more detail next. guishable from baseline. In the lower graph, changes
Five characteristics of the data should control the in trend and mean level were observed in the first
evaluation of behavior change across phases. The three sessions after the phase change, showing more
first is the change in level. Level refers to the imme- clearly that the data in the two phases are distinct.
diate change in responding from the end of one Although short latencies to change suggest that
phase to the beginning of the next (Kazdin, 1982). behavior has changed across phases, this change
Level is assessed by comparing the last data point may be temporary and, therefore, additional obser-
from a condition to the first data point of the subse- vations should be made until one is convinced that
quent condition. In the top graph of Figure 9.10, the change is enduring. How many additional obser-
the change in level was a decrease from about 46 vations are necessary will be affected by factors such
responses per minute to about 26 per minute. In the as baseline variability (as in the top panel of Figure
lower panel, the level increased from about 30 to 48 9.10) and how novel the finding is (skeptical scien-
responses per minute. Level may be used to evaluate tists prefer to have many observations when the
the magnitude of treatment effect. Large changes in intervention is novel). Under most conditions, an
level suggest a potent independent variable, but only intervention that produces a large but temporary
when the data collected in the remainder of the behavior change is of limited utility.
intervention phase continue at the new level, as in The third characteristic of time-series data that is
the lower panel of Figure 9.10. The level change in used when visually evaluating differences across
the upper panel of Figure 9.10 is inconsistent with phases is the mean shift (Parsonson & Baer, 1992).
most of the remaining intervention data and, there- Mean shift refers to the amount by which the means
fore, appears to be another instance of uncontrolled differ across phases. In both panels of Figure 9.10,
between-session variability. As this example illus- there is an upward mean shift from baseline to
trates, a level change is neither necessary nor suffi- intervention. The bottom graph, however, illus-
cient to conclude that behavior changed in the trates a shift that is visually more compelling. The
intervention phase. reason for this takes us to the fourth characteristic
The second, related characteristic that will affect controlling visual analysis activities: between-phase
judgments of treatment effects is latency to change. overlap. In the upper panel of Figure 9.10, as the
Latency to change is the time required for change in range lines illustrate, five of eight data points in the
responding to be detected after the onset of a new intervention condition fall within the range of the
experimental condition (Kazdin, 1982). To evaluate preceding baseline data, and, therefore, the differ-
latency to change, a researcher must examine multi- ence is not convincing. Perhaps, in the top graph, if
ple data points after the condition change to deter- additional data were collected during the interven-
mine whether a consistent change in level or a tion phase, and assuming responding remained at
209
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
the upper plateau characterizing the final interven-

tion sessions, the difference might be compelling. In
the lower graph of Figure 9.10, the level change,
mean shift, and limited between-phase overlap in
range make the difference visually apparent.
The fifth characteristic of the data that will affect
visual evaluation of between-phase differences is
trend. As noted earlier, if the baseline data are trend-
ing up (or down) and responding increased (or
decreased) during the intervention phase (upper
graph of Figure 9.10), then the mean shift will not
be convincing unless the trend line is much steeper
in the intervention phase than at baseline. In the
upper graph of Figure 9.10, the baseline data show a
slight upward trend. Data in the subsequent inter-
vention phase show a steeper trend. The greater the
difference in trend is, the clearer it is that the mean
shift in the intervention is not simply a continuation
of the baseline trend.
When evaluating mean shifts, floor or ceiling Figure 9.11. Example of the conservative dual-
effects must be considered. These effects occur criterion technique applied to hypothetical intervention
when performance has reached a minimum or maxi- data. In the baseline phase, solid horizontal lines are
mum, respectively, beyond which it cannot change mean lines, and solid trend lines are regression lines.
To analyze intervention effects, these lines are super-
further. For example, if baseline response rates are imposed onto the intervention phase and raised by 0.25
low and an intervention is expected to decrease standard deviation from the baseline means. See text for
responding, mean shifts may be small because details.
response rates have little room to further decrease.
Readers skeptical of visual analysis practices may exceeds a criterion based on the binomial equation
be unsettled by the use of terms and phrases such as (i.e., exceeds the number that would be expected by
judgment, visually apparent, and much steeper. How chance). Fisher et al. found that the use of CDC pro-
much steeper is “much steeper?” Although any cedures improved agreement on visual inspection
interpretation of data requires the researcher make a (data were hypothetical, and intervention effects
variety of judgment calls (e.g., which statistic to use, were computer generated).
how to handle missing data), Fisher, Kelley, and Figure 9.11 shows the CDC applied to the data
Lomas (2003) sought to reduce the number of judg- shown in Figure 9.10. In the top panel, three data
ments by developing the conservative dual-criterion points in the intervention phase fall above the two
(CDC) technique to aid the visual analysis of single- criterion lines. Following the table presented in
case data. This method, illustrated in Figure 9.11, Fisher et al. (2003) for treatment conditions with
involves extending the baseline mean and trend eight data points, seven data points should be above
lines into the intervention phase and raising both of both lines to conclude that a compelling difference
these lines by 0.25 standard deviation (or lowering exists between phases. In the lower panel, the CDC
the lines by 0.25 standard deviation, if the intervention requires that 12 of the 15 data points in the inter-
is anticipated to decrease responding). A difference vention condition appear above both lines, a crite-
across conditions is judged as meaningful when the rion easily met, so the researcher may conclude that
number of intervention-phase data points falling behavior changed across phases.
above both lines (or below both lines in the case of Although the CDC method appears to improve
an intervention designed to decrease a behavior) the accuracy of judgments of behavior change, only
210
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
a few studies have yet investigated this technique systematic behavior changes with each manipula-
(Fisher et al., 2003; Stewart, Carr, Brandt, & tion provide evidence for a causal relation. Figure
McHenry, 2007). The Fisher et al. (2003) procedure 9.12 shows the previously considered data set now
is, of course, but one technique for making visual extended to include a second baseline and a second
assessment of data more objective. It is ultimately intervention condition. The bottom graph is eas-
incumbent on the investigator or therapist to pro- ily interpreted. The visually apparent difference
vide convincing evidence of an effect, whether between the first baseline phase and the first inter-
through some formal set of rules as illustrated by vention phase is reversed in the return to baseline.
Fisher et al. or by amplifying the effect to the point In the second baseline phase, responding was well
at which reasonable people agree on it, through outside the range in which data should have fallen
increased control over both independent and extra- had no experimental manipulation been imple-
neous variables. mented. Further evidence for an intervention effect
Assuming that appropriate decisions were made is that responding returned to the level observed
about stability and a visually apparent behavior in the original baseline. The reintroduction of the
change was observed from baseline to the interven- intervention (fourth phase) reverses the downward
tion phase, the next task is to evaluate the role of the trend in the second baseline, yielding a striking level
intervention in that behavior change. Evaluating the shift, mean shift, and minimal variability. There is
causal role of the intervention requires that an appro- very little overlap in the data across conditions, the
priate experimental design be used, a topic falling latency to change is short, and there are no trends
under the scope of Chapter 5, this volume. Here, we that make interpretation difficult. The mean level is
largely confine our discussion to the visual analysis close to the mean level obtained in the first exposure
techniques appropriate to the most commonly used to the intervention, thus replicating the effect. These
single-case research designs. In these sections of the data thus make a strong case for the intervention as
chapter, the visual analysis focuses on answering the an effective means of influencing behavior.
question, “Did the intervention change behavior?”
Comparison designs. The data shown in the lower
panel of Figure 9.11 come from a comparison design
(or A-B design). There is evidence of a convinc-
ing change in behavior across conditions, level and
mean level differ, variability and overlap of the data
points across conditions are not interfering, and the
latency to change is short. Despite stable data, one
cannot conclude that the intervention produced the
visually apparent behavior change. Although the
rapid level change suggests an intervention effect,
one cannot rule out extraneous variables that may
have changed at the same time that the intervention
was introduced (e.g., in addition to the interven-
tion, Participant 2 may have been informed that if
his productivity did not improve, his job would be
in jeopardy). When visually analyzing data, a differ-
ence in behavior between two phases is insufficient
evidence that the intervention, and not extraneous
variables, produced the change.
Figure 9.12. Example of a reversal design. Data from
Reversal designs. In a reversal design, the experi- the first baseline and intervention phases are the same
mental variable is introduced and removed, and as shown in Figures 9.8 and 9.10. Data are hypothetical.
211
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
The upper panel of Figure 9.12 tells a different 40

story. When the baseline conditions are reestab-
30
lished in the third phase, there is a precipitous Intervention 1
downward trend in behavior. Although this behav- Intervention 2
20 No Intervention
ior change is consistent with the removal of an effec-
tive intervention, the between-session variability in 10
the preceding condition renders an unconvincing A
the argument for a between-phase behavior change. 0
Clearly, the hypothetical researcher who collected
40
these data failed to continue the first intervention
phase long enough for a stable pattern of behavior to
Responses per Min

30
develop. If responding had stabilized in the upper
plateau reached at the end of the first intervention 20
phase, the sharp reduction in responding in the sec-
ond baseline may have been more compelling. 10
When the intervention is again introduced, the B
0
downward trend levels off, but the data points over-
lap considerably with the data points for the preced- 40
ing baseline condition. Furthermore, the mean level
in the second intervention phase did not closely rep- 30
licate the mean level of the first intervention phase.
20
Multielement designs. Figure 9.13 shows data
10
from three hypothetical multielement experimental
designs (Barlow & Hayes, 1979). In this design, C
0
conditions alternate (often after every session), and 2 4 6 8 10 12 14 16 18 20
consistent level changes are suggestive of a func- Sessions
tional (causal) relation between variables arranged
in the condition and the behavior of interest. Visual Figure 9.13. Examples of data from multielement
analysis of multielement design data requires evalu- designs. Data are hypothetical.
ation of sequence effects in addition to variability,
mean shift, trend, overlap, and latency to change. conditions is obvious. The effects of each experi-
Detection of sequence effects requires close atten- mental manipulation on response rate are reliable
tion to the ordering of conditions and patterning in (each repetition of a condition allows a test of reli-
the data (i.e., if responding in one condition is con- ability), and it would be extremely unlikely that
sistently elevated when preceded by one of the other some extraneous variable would happen to produce
conditions). increases in response rate in each Intervention 1
The data in Figure 9.13 represent response rate session and none in any other session. Finally, the
in three different conditions, two interventions, and effect of Intervention 1 does not appear to be depen-
a no-intervention control condition. The top graph dent on the prior condition, and in at least one case,
is easily interpreted. The mean level in Intervention the effect lasts when two consecutive Intervention
1 is higher than in the other two conditions, and 1 sessions are completed. These data provide com-
there is no difference in mean level between Inter- pelling evidence that Intervention 1 is responsible
vention 2 and the no-intervention condition. The for producing higher response rates than either
data are relatively stable (i.e., there is little variabil- Intervention 2 or the no-intervention condition.
ity), and there are no trends to complicate interpre- The middle graph contains more within-
tation. Thus, the difference in behavior between the condition variability. The mean level is higher in the
212
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Intervention 1 condition; however, there is consid- 40 Baseline Intervention

erable overlap in the range of response rates
30
observed in each condition. Because it is not clear
that behavior is distinct across conditions, the ques-
20
tion of causation is moot. In the third graph, the
mean level of the data in Intervention 1 is high, the 10
mean level in the no-intervention condition is low, A
but the data in Intervention 2 are more difficult to 0
interpret. During some sessions, the response rate is
40
low; during others, it is high. This graph shows a
hypothetical sequence effect. Each time an Interven-
Responses per Min

30
tion 2 session follows an Intervention 1 session, the
response rate is high; otherwise, the response rate is 20
low. The rate during the Intervention 2 sessions is
affected by the preceding condition, which compli- 10
cates interpretation of the effects of Intervention 2. B
0
If the high-rate Intervention 2 sessions were merely
a carry-over effect of Intervention 1, then the no- 40
intervention sessions that follow Intervention 1
sessions should show comparable high rates. A 30
researcher who obtains findings of this sort will con-
duct further experimentation to clarify the processes 20
responsible for the sequence effect.

10
C
Multiple-baseline designs. Multiple-baseline 0
designs are frequently used in applied settings, 2 4 6 8 10 12 14 16 18 20
either when it would be unethical to remove the
Sessions
treatment or because the treatment is expected to
produce an irreversible effect. The design involves a Figure 9.14. Example of data from a multiple-
series of comparison designs in which the researcher baseline design. Data are hypothetical.
implements the treatment variable at different
times across participants, behaviors, or contexts. A Designs section, however, these data alone are
researcher visually analyzing data from a multiple- insufficient to support causal statements. The
baseline design must evaluate whether there are behavior change could also have been caused by an
convincing changes in mean level from baseline to extraneous variable that changed at the same time
treatment conditions, whether the effects are rep- as the intervention (e.g., a change in classroom
licated across baselines, and whether changes in contingencies). If the latter were true, then one
behavior occur only when the treatment is imple- might expect this variable to affect behavior in the
mented for each baseline. other baselines. To evaluate this, one examines the
Figure 9.14 illustrates data from a multiple- other baselines for behavior change that corre-
baseline design. In the top panel, a brief baseline sponds with the introduction of the intervention in
precedes the intervention. The intervention pro- the first graph (i.e., at Session 5). Figure 9.14
duces level changes and mean shifts easily discrim- shows evidence of this, which strengthens the case
inated as behavior change. There is no latency to that the intervention produced the behavior change
change, and there are no trends or overlap between observed in the top panel.
data points across conditions to complicate data Further evidence that the intervention is related to
interpretation. As noted in the Comparison the behavior change must be gathered in the remaining
213
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
panels of Figure 9.14 because the intervention is

implemented at different points in time across the
baselines. In the second graph, the data in baseline
show an upward trend. Because the effect of the inter-
vention is a decrease in the mean level, it does not
negatively affect identifying the change in behavior in
the second phase. In examining the third graph, these
baseline data were unaffected by the phase change
depicted in the second graph, which provides further
evidence that the behavior change observed in the
first two graphs is a function of the intervention and
not some uncontrolled variable. In the third graph,
the baseline is relatively stable, and there is a large, Figure 9.15. Example of data from a changing-
immediate reduction in response rate after implemen- criterion design. The figure shows the number of ciga-
tation of the intervention, which replicates the effects rettes smoked per day. Solid horizontal lines depict
observed in the first two graphs, providing strong evi- the criterion for each phase. From “The Changing
Criterion Design,” by D. P. Hartmann and R. V. Hall,
dence of the effects of the intervention. 1976, Journal of Applied Behavior Analysis, 9, p. 529.
Copyright 1976 by the Society for the Experimental
Changing-criterion designs. In changing-criterion Analysis of Behavior, Inc. Used with permission.
designs, a functional relation between the interven-
tion and behavior change is established by (a) peri- they did, because if a trend line was drawn through
odically changing the contingency specifying which these baseline data, the subsequent decreases in
responses (e.g., those with a force between 20 and smoking would be predicted to occur in the absence
30 g) will lead to experimenter-arranged conse- of an intervention. Had the researchers collected
quences and (b) observing that behavior approxi- more baseline data, they would likely have found
mates the specified criterion (Hall & Fox, 1977; that Day 7 was uncharacteristic of this individual’s
Hartmann & Hall, 1976). Typically, the criterion in rate of smoking and could have more clearly estab-
graphs of changing-criterion designs is indicated lished the stable rate of baseline smoking.
by horizontal lines at each phase. Visual analysis In subsequent phases (B–G), the criterion number
of changing-criterion designs, as with that of other of cigarettes was systematically decreased, as indi-
designs, requires an assessment of variability, level, cated by the horizontal line in each phase. Changes
mean shift, trend, overlap, and latency to change but in the criterion tended to produce level changes, with
also requires an assessment of the relation between many subsequent data points falling exactly on the
behavior and the criterion. criterion specified in that phase. Each phase estab-
Figure 9.15 shows data from Hartmann and Hall lishes a new mean level approximating the criterion.
(1976), who used a changing-criterion design to There are no long latencies to change, and the vari-
assess the effectiveness of a smoking cessation pro- ability in the data and overlap in data points across
gram. During the intervention, the participant was conditions are not sufficient to cause concern.
fined a small amount of money for smoking above a Finally, with the exception of Phase F, there is no
criterion number of cigarettes and earned a small downward trend in any phase, suggesting that if the
amount of money for smoking fewer cigarettes. In criterion remained unchanged, smoking would
the top graph, the number of cigarettes smoked per remain at the current depicted level. Thus, the visual
day is shown across successive days of the interven- analysis of these data raises concerns about the
tion. In baseline, the number of cigarettes smoked downward trend in the baseline, but these concerns
per day was stable over the first 6 days but fell are largely addressed by the repeated demonstrations
precipitously on Day 7. Ideally, the researchers of control over smoking rate in each condition. If the
would not have begun the intervention on Day 8, as study were ongoing and concerns remained, the
214
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
researchers could set the next criterion (Phase H)

above the last one. If smoking rate increased to the
new criterion, then additional evidence for interven-
tion control would be established while nullifying
concerns about the downward trend in baseline.
Interpreting Relational Graphs

Researchers who conduct time-series research may
report their outcomes using relational graphs. In these
cases, each data point represents the mean (or another
appropriate measure of central tendency) of steady-
state responding from a condition. When evaluating
these data, measures of variability, such as error bars,
Figure 9.16. Example of curve fitting.
are also assessed to help determine whether respond- From “Delay or Probability Discounting
ing was stable (see Interpreting Bar Graphs section). in a Model of Impulsive Behavior: Effect
Data on relational graphs are evaluated by ana- of Alcohol,” by J. B. Richards, L. Zhang,
S. H. Mitchell, and H. de Wit, 1999,
lyzing the clustering and trend of the data points. Journal of the Experimental Analysis of
Data that appear horizontal across all values of the Behavior, 71, p. 132. Copyright 1976 by
x-axis indicate that the independent or predictor the Society for the Experimental Analysis
variable has no effect on behavior or that there is no of Behavior, Inc. Used with permission.
correlation between the two dependent variables.
Sometimes behavior changes in a linear fashion when quantitative methods are used to describe data,
across the range of x-axis values of the independent however, visual analysis is used as a supplement. For
variable. When both axes of the graph are scaled lin- example, visual analysis can help researchers choose
early, a linear relation indicates that changing the which type of curve to fit to the data, evaluate whether
independent variable produces a constant increase data trends are linear and thus appropriate for calcu-
or decrease in behavior. Nonlinear relations indicate lating Pearson correlation coefficients, or determine
that the behavioral effect of the independent variable whether the data have systematic deviations from the
changes across x-axis values. Figure 9.16 provides fit of a quantitative model. For example, Figure 9.16
an example. Here, the subjective value of a $10 shows the best fits of both exponential and hyperbolic
reward is plotted as a function of the delay to its models to the subjective value of delayed money. The
delivery. Both axes are linear, and the relation figure shows that the exponential model systemati-
between the variables is nonlinear. cally predicts a lower y-axis value than that obtained
Relational graphs, when properly constructed, at the highest x-axis value.
allow the researcher to quickly evaluate the relation Reference lines may be added to relational
between variables. It is common, however, to evaluate graphs (or other graph types) to provide a point of
relational data more precisely with quantitative meth- comparison to the data. For instance, in a graph
ods, including curve-fitting techniques (e.g., linear showing discrete-trial performances, such as
and nonlinear regression), Pearson’s correlation matching-to-sample, reference lines may be plotted
coefficient, and quantitative models (see Chapters 10 at values expected by chance. In a graph depicting
and 12, this volume). Curve-fitting techniques clarify choice data, reference lines might be plotted at val-
the form of the relation between the independent ues indicative of indifference.
and dependent variables, and Pearson’s correlation
coefficient quantifies the relation between two depen-
Conclusion
dent variables. Quantitative models may describe
more complex behavior–environment relations and Graphs provide clear and detailed summaries of
are used to make predictions about behavior. Even research findings that can guide scientific decisions
215
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bourret and Pietras
and efficiently communicate research results. Visual Cleveland, W. S. (1994). The elements of graphing data
analysis, as with any form of data analysis, requires (rev. ed.). Summit, NJ: Hobart Press.
training and practice. The use of visual analysis as a Cleveland, W. S., & McGill, R. (1984). Graphical per-
method of data interpretation requires graph readers ception: Theory, experimentation, and application
to the development of graphical methods. Journal
to make sophisticated decisions, taking into account of the American Statistical Association, 79, 531–554.
numerous aspects of the data. This complexity can doi:10.2307/2288400
make the task seem daunting or subjective; how- Cleveland, W. S., & McGill, R. (1985). Graphical per-
ever, visual analysis in conjunction with rigorous ception and graphical methods for analyzing scien-
experimental procedures is a proven, powerful, and tific data. Science, 229, 828–833. doi:10.1126/
science.229.4716.828
flexible method for generating scientific knowledge.
The development of effective behavioral technol- Cooper, J. O., Heron, T. E., & Heward, W. L. (1987).
Applied behavior analysis. Columbus, OH: Merrill.
ogies provides evidence of the ultimate utility of the
Davison, M., & Baum, W. M. (2006). Do conditional
visual analysis techniques used in behavior-analytic
reinforcers count? Journal of the Experimental
research. Data analyzed by means of visual inspec- Analysis of Behavior, 86, 269–283. doi:10.1901/
tion have contributed to a technology that produces jeab.2006.56-05
meaningful behavior change in individuals across a DeFulio, A., Donlin, W. D., Wong, C. J., & Silverman,
wide range of skill domains and populations, including K. (2009). Employment-based abstinence reinforce-
individuals with no diagnoses and those with diag- ment as a maintenance intervention for the treatment
of cocaine dependence: A randomized controlled
noses including attention deficit/hyperactivity disor- trial. Addiction, 104, 1530–1538. doi:10.1111/j.1360-
der, autism, an array of developmental disabilities, 0443.2009.02657.x
pediatric feeding disorders, and schizophrenia, to Didden, R., Duker, P. C., & Korzilius, H. (1997). Meta-
name a few (Didden, Duker, & Korzilius, 1997; analytic study on treatment effectiveness for problem
Lundervold & Bourland, 1988; Weisz, Weiss, Han, behaviors with individuals who have mental retarda-
tion. American Journal on Mental Retardation, 101,
Granger, & Morton, 1995). Because of its history of 387–399.
effective application and advantages for the study of
Ferster, C. B., & Skinner, B. F. (1997). Schedules of rein-
the behavior of individuals, behavior analysts forcement. Acton, MA: Copley. (Original work pub-
remain committed to visual inspection as a primary lished 1957) doi:10.1037/10627-000
method of data analysis. Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003).
Visual aids and structured criteria for improving
inspection and interpretation of single-case designs.
References Journal of Applied Behavior Analysis, 36, 387–406.
American Psychological Association. (2010). Publication doi:10.1901/jaba.2003.36-387
manual of the American Psychological Association (6th Galbicka, G., Fowler, K. P., & Ritch, Z. J. (1991). Control
ed.). Washington, DC: Author. over response number by a targeted percentile
Barlow, D. H., & Hayes, S. C. (1979). Alternating treat- schedule: Reinforcement loss and the acute effects of
ments design: One strategy for comparing the effects d-amphetamine. Journal of the Experimental Analysis of
of two treatments in a single subject. Journal of Behavior, 56, 205–215. doi:10.1901/jeab.1991.56-205
Applied Behavior Analysis, 12, 199–210. doi:10.1901/ Gallistel, C. R., King, A. P., Gottlieb, D., Balci, F.,
jaba.1979.12-199 Papachristos, E. B., Szalecki, M., & Carnone, K. S.
Baron, A., & Perone, M. (1998). Experimental design (2007). Is matching innate? Journal of the Experimental
and analysis in the laboratory study of human oper- Analysis of Behavior, 87, 161–199. doi:10.1901/jeab.
ant behavior. In K. A. Lattal & M. Perone (Eds.), 2007.92-05
Handbook of research methods in human operant Gentry, G. D., Weiss, B., & Laties, V. G. (1983). The
behavior (pp. 45–91). New York, NY: Plenum Press. microanalysis of fixed-interval responding. Journal of
Baum, W. M. (1973). The correlation-based law of effect. the Experimental Analysis of Behavior, 39, 327–343.
Journal of the Experimental Analysis of Behavior, 20, doi:10.1901/jeab.1983.39-327
137–153. doi:10.1901/jeab.1973.20-137
Hall, R. V., & Fox, R. G. (1977). Changing-criterion
Cançado, C. R. X., & Lattal, K. A. (2011). Resurgence designs: An alternative applied behavior analysis
of temporal patterns of responding. Journal of the procedure. In B. C. Etzel, J. M. LeBlanc, & D. M.
Experimental Analysis of Behavior, 95, 271–287. Baer (Eds.), New developments in behavioral research:
216
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Theory, method, and application (pp. 151–166). (Eds.), Experimental analysis of behavior, Parts 1 and
Hillsdale, NJ: Erlbaum. 2. Techniques in the behavioral and neural sciences
Harris, R. L. (1996). Information graphics: A comprehen- (Vol. 6, pp. 135–171). New York, NY: Elsevier.
sive illustrated reference. Atlanta, GA: Management Richards, J. B., Zhang, L., Mitchell, S. H., & de Wit, H.
Graphics. (1999). Delay or probability discounting in a model
Hartmann, D. P., & Hall, R. V. (1976). The changing cri- of impulsive behavior: Effect of alcohol. Journal of
terion design. Journal of Applied Behavior Analysis, 9, the Experimental Analysis of Behavior, 71, 121–143.
527–532. doi:10.1901/jaba.1976.9-527 doi:10.1901/jeab.1999.71-121
Johnston, J. M., & Pennypacker, H. S. (1993). Strategies Shah, P., Freedman, E. C., & Vekiri, I. (2005). The compre-
and tactics of behavioral research (2nd ed.). Hillsdale, hension of quantitative information in graphical
NJ: Erlbaum. displays. In P. Shah & A. Miyake (Eds.), The
Cambridge handbook of visuospatial thinking (pp. 426–
Kazdin, A. E. (1982). Single-case research designs: Methods 476). New York, NY: Cambridge University Press.
for clinical and applied settings. New York, NY:
Oxford University Press. Shull, R. L. (1991). Mathematical description of operant
behavior: An introduction. In I. H. Iversen & K. A.
Lundervold, D., & Bourland, G. (1988). Quantitative Lattal (Eds.), Experimental analysis of behavior, Parts
analysis of treatment of aggression, self-injury, and 1 and 2. Techniques in the behavioral and neural sci-
property destruction. Behavior Modification, 12, ences (Vol. 6, pp. 243–282). New York, NY: Elsevier.
590–617. doi:10.1177/01454455880124006
Sidman, M. (1960). Tactics of scientific research. Oxford,
Morley, S., & Adams, N. I. (1991). Graphical analysis of England: Basic Books.
single-case time series data. British Journal of Clinical
Psychology, 30, 97–115. doi:10.1111/j.2044-8260. Staddon, J. E. R., & Simmelhag, V. L. (1971). The
1991.tb00926.x “superstition” experiment: A reexamination of its
implications for the principles of adaptive behavior.
Parsonson, B. S., & Baer, D. M. (1978). The analysis and Psychological Review, 78, 3–43. doi:10.1037/h0030305
presentation of graphic data. In T. R. Kratochwill
(Ed.), Single-subject research: Strategies for evaluating Stevens, S. S. (1946). On the theory of scales of measure-
change (pp. 101–165). New York, NY: Academic Press. ment. Science, 103, 677–680. doi:10.1126/science.103.
2684.677
Parsonson, B. S., & Baer, D. M. (1986). The graphic
analysis of data. In A. Poling & R. W. Fuqua (Eds.), Stewart, K. K., Carr, J. E., Brandt, C. W., & McHenry, M. M.
Research methods in applied behavior analysis: Issues (2007). An evaluation of the conservative dual-
and advances (pp. 157–186). New York, NY: Plenum criterion method for teaching university students to
Press. visually inspect AB-design graphs. Journal of Applied
Behavior Analysis, 40, 713–718.
Parsonson, B. S., & Baer, D. M. (1992). The visual analy-
sis of data, and current research into the stimuli Tufte, E. R. (1983). The visual display of quantitative infor-
controlling it. In T. R. Kratochwill & J. R. Levin mation. Cheshire, CT: Graphics Press.
(Eds.), Single-case research design and analysis: New Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., &
directions for psychology and education (pp. 15–40). Morton, T. (1995). Effects of psychotherapy with
Hillsdale, NJ: Erlbaum. children and adolescents revisited: A meta-analysis
Perone, M. (1991). Experimental design in the analysis of of treatment outcome studies. Psychological Bulletin,
free-operant behavior. In I. H. Iversen & K. A. Lattal 117, 450–468. doi:10.1037/0033-2909.117.3.450
217
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 10
Quantitative Description
of Environment–Behavior
Relations
Jesse Dallery and Paul L. Soto
In 1687, Sir Isaac Newton invented a profound new a case study in quantitative analysis. Although we
way of thinking about how objects move across have selected matching theory as our case study
space and time. Newton tackled familiar empirical because of its widespread application in behavior
facts such as the observation that objects fall when analysis, the techniques and issues we discuss
dropped, but the description of falling objects was could be generalized to any quantitative model of
elevated to a new realm of precise, quantitative anal- environment–behavior relations, and where appro-
ysis. Among other feats, his analysis produced the priate we highlight these extensions.
universal law of gravitation, which yielded consider-
able predictive and practical advantages over previ-
On the Utility of Quantitative
ous accounts (Kline, 1959). The stunning successes
Models
of classical mechanics, of landing rockets on the
moon, would not have been possible before New- Models specify relations among dependent variables
ton’s framework. In addition to its predictive and and one or more independent variables. These rela-
practical advantages, the universal law of gravitation tions can be specified using words (a verbal model)
unified seemingly disparate phenomena: It described or mathematics (a mathematical or quantitative
moving bodies both here on Earth and in the heav- model). Verbal models can be just as useful as quan-
ens. Newton’s account also provided a foundation titative models in describing the causes of behavior.
for attempts to explain gravity. As sciences have For instance, an applied behavior analyst does not
matured, from astronomy to zoology, so too has need quantitative models to assess determinants of
their use of quantitative analysis and their ability to self-injurious behavior or to generate verbal behav-
describe, unify, and explain. The field of behavior ior for a child with autism. However, there are also
analysis has witnessed similar transformations in examples in which quantitative models are useful in
how people describe an organism’s behavior across the applied realm (Critchfield & Reed, 2009; Fisher
space and time. & Mazur 1997; Mazur, 2006; McDowell, 1982).
Fortunately, one does not need Newton’s calcu- Quantitative models of choice can help the analyst
lus to understand and appreciate these exciting tease apart controlling variables in a naturalistic set-
advances in behavioral science. In this chapter, we ting (McDowell, 1982; see Fuqua, 1984, for some
explain key techniques involved in quantitative caveats to this assertion), or they may be useful in
analysis. We describe how quantitative models are terms of evaluating preferences for treatment
evaluated and compared. To provide some theoreti- options (Fisher & Mazur, 1997). Even if the benefits
cal backbone to our explication of techniques, we are not immediate, knowledge of quantitative
use a model of choice known as matching theory as accounts could lead to alternative treatments
We thank Rachel Cassidy and Bethany Raiff for comments on an earlier version of this chapter.
DOI: 10.1037/13937-010
219
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
(McDowell, 1982), or it could inspire new reinforcement interacts with other, background
insights into the determinants of problem behavior sources of reinforcement. Assumptions about and
(Critchfield & Kollins, 2001; Fisher & Mazur, 1997; descriptions of interactions between variables can be
Nevin & Grace, 2000; see Volume 2, Chapters 5 particularly troublesome and obscure if formulated
and 7, this handbook). in verbal terms. Quantitative models force specificity
Behavioral scientists should appreciate and about the nature of these interactions. For example,
understand a wide variety of analytic methods to Staddon (1984) analyzed a model of social dynamics
understand the determinants of behavior. Quantita- by expressing the model quantitatively. Staddon’s
tive analysis entails a rich and powerful set of tools. quantitative analysis revealed several inconsistencies
From 1970 to 2000, basic behavior-analytic science (and even contradictions) in the corresponding ver-
saw an increase from 10% to almost 30% in articles bal formulation. Staddon concluded that “unaided
published in the Journal of the Experimental Analysis verbal reasoning is almost useless in the analysis of
of Behavior that used equations to describe behavior the dynamics of interactions, human or mechanical”
(Mazur, 2006). In addition to understanding the (p. 507).
advances described in these articles, other intellec- Second, the predictions of a quantitative model
tual and practical payoffs are derived from knowl- are more specific than the predictions of a verbal
edge of quantitative analysis. These benefits may be description, which allows one to observe minor but
delayed, but they can be profound. The English phi- systematic deviations from the predictions. For
losopher Roger Bacon (c. 1214–1294, as quoted in instance, one could verbally describe where a pro-
Kline, 1959) noted, jectile might land after being launched into the air
(e.g., “over there”), or one could use an equation to
Mathematics is the gate and the key of the
make a precise prediction (e.g., 157 feet, 2 inches,
sciences. . . . Neglect of mathematics works
due north). Any deviation from the latter location
injury to all knowledge, since he who is
may be precisely measured, and records may be kept
ignorant of it cannot know the other sci-
to determine whether the equation makes systematic
ences or the things of this world. And what
errors (e.g., it consistently overestimates the dis-
is worse, men who are thus ignorant are
tance traveled). Similarly, one could hypothesize
unable to perceive their own ignorance
verbally about how much reinforcement would be
and so do not seek a remedy. (p. 1)
necessary to establish some level of appropriate
Quantitative models have at least four benefits. behavior (e.g., one might say one needs to identify a
First, quantitative models force a greater degree of “powerful” reinforcer or deliver the reinforcer “fre-
precision than their verbal counterparts. Assume quently”). As with the estimate of the projectile, a
several reinforcer rates have been experimentally hypothesis might be qualitatively correct, but it will
arranged, from low to high, and the response rates be more precise if one uses established equations of
obtained at each reinforcer rate have been measured. reinforced responding. Admittedly, this degree of
A verbal description might assert that at low rein- precision may not be necessary in an applied con-
forcer rates, increases in reinforcer rate produce text. Nevertheless, a hallmark of behavioral science
rapid increases in response rate, but that response is the predictive power of its explanations, and, as
rate eventually reaches a maximum at higher rein- these examples imply, quantitative models surpass
forcer rates. The same description of the relation verbal models in making predictions.
between response rate and reinforcer rate is embod- Third, to the extent that the predictions and con-
ied by Herrnstein’s hyperbolic equation (described ditions under which one should observe them are
in more detail later). Herrnstein’s equation states more precise, the more falsifiable the theory
succinctly and precisely how behavior will change becomes. As the philosopher of science Karl Popper
with increases in reinforcement. Moreover, Herrn- (1962) noted, falsifiability is a virtue of scientific
stein’s equation precisely asserts how reinforcement theory. Consider Sir Arthur Eddington’s test of
from the experimentally arranged schedule of Albert Einstein’s theory of general relativity. One of
220
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Quantitative Description of Environment–Behavior Relations
general relativity theory’s critical predictions is that representation1 of the data reduced to a
gravity bends light. To test this prediction, Edding- minimal number of terms. A theoretical
ton measured the amount of shift in light from stars construction may yield greater generality
close to the sun (Silver, 1998). The sun’s powerful than any assemblage of facts. It will not
gravitational field, according to general relativity stand in the way of our search for func-
theory, should produce measurable shifts in the tional relations because it will arise only
light emanating from the nearby stars. Eddington after relevant variables have been found
waited for an eclipse (the eclipse of 1919), when the and studied. Though it may be difficult
stars adjacent to the sun were observable, and he to understand, it will not be easily
measured the amount of shift in light emanating misunderstood. (p. 100).
from those stars. He found not only that the light
Skinner’s (1950/1972) assessment of theory,
from these stars did indeed bend around the sun,
however, was tempered by a recommendation that
but also that the exact amount of shift was as pre-
psychologists must first establish an experimental
dicted by Einstein’s theory. An observation (or more
analysis of how relevant variables affect behavior.
realistically, a series of observations) to the contrary
Ultimately, however, quantitative theory increases
would have falsified the theory. Similarly, Herrn-
the precision and generality of explanations and
stein (1974) noted that certain predictions made by
improves the ability to predict and influence
his quantitative theory of reinforced responding
behavior.
were especially vulnerable to “empirical confirma-
tion or refutation” (p. 164). All good scientific theo-
ries, whether verbal or quantitative, are falsifiable. Structure and Function of
One virtue of most quantitative theories is that the Quantitative Models
conditions under which one can refute them are
Although quantitative models can generate new pre-
relatively clear (McDowell, 1986).
dictions and explanations of behavior, they are often
Fourth, quantitative modeling encourages
developed inductively on the basis of descriptions of
researchers to unify diverse phenomena; to see simi-
empirical facts, and this is where we start. After rig-
larities in the determinants of behavior across set-
orous, parametric, experimental analysis, research-
tings, cultures, and species (Critchfield & Reed,
ers plot a dependent variable (e.g., response rate,
2009; Lattal, 2001; Mazur, 2006; see Chapter 7, this
interresponse times, ratio of time spent engaged in
volume). Another way to put this is that a goal of a
one activity over another activity) as a function of an
science of behavior is to discover invariance, or reg-
independent variable (e.g., reinforcer rate, time in
ularity in nature. For example, people see the diver-
session, reinforcers delivered for one activity relative
sity of structure in the animal kingdom as a result of
to another activity) in graphical space. They exam-
evolutionary processes and the diversity of geologi-
ine the shape of the relation. Is it described by a
cal events as a result of tectonic processes. The
straight line or a curve? In behavioral science, rela-
mathematician Bell (1945, p. 420) defined invari-
tions characterized by straight lines or monotonic
ance as “changelessness in the midst of change, per-
curves (curves that change in one direction) are
manence in a world of flux, the persistence of
common. Researchers may also model behavioral
configurations that remain the same despite the
processes that show a bitonic relation, which is a
swirl and stress of countless transformations” (also
curve that changes in two directions (e.g., a curve
quoted in Nevin, 1984, p. 422). Skinner
that increases and then decreases or vice versa).
(1950/1972) also saw the discovery of invariance as
Although these shapes are not complicated, the
a worthy goal:
equations that describe them may appear to be. If
Beyond the collection of uniform rela- the equations are intimidating at first, start with
tionships lies the need for a formal the shapes and the specific environment–behavior
A formal representation means a mathematical representation.

1
221
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
relations described by the shapes. Therefore, use dissect the anatomy of the equation that describes
careful visual inspection (see Chapter 9, this vol- the relation. In behavioral and psychology journals,
ume) of the data. equations may appear in the introduction of an arti-
The importance of visual analysis is illustrated cle. Thus, dissecting the equation may require some
nicely by Anscombe’s quartet (Anscombe, 1973), detective work and skipping ahead in the article to
which is a series of four datasets that are shown in find a graph that shows the shape described by the
Figure 10.1. Across the four panels of the figure, the equation. Comparing the equation with the graph is
mean and variance of the data points in each panel useful. Here is an example of a common equation in
are equivalent, as is the linear equation describing behavioral science:
the relation between the independent (x-axis) and
dependent (y-axis) variables: y = 0.5x + 3. Even a  r 
R = k (1)
casual visual analysis, however, reveals that the .
 r + re 
shape of each dataset is remarkably distinct and that
only the data in the upper left panel are appropri- This equation, known as Herrnstein’s hyperbolic
ately described by a linear equation. A purely quan- equation, describes a hyperbolic relation between
titative analysis alone is not sufficient. Careful use of reinforcer rate, r, and response rate, R. Figure 10.2
quantitative analysis involves careful visual inspec- shows two examples of this hyperbolic shape
tion of the data (Parsonson & Baer, 1992; see Chap- described by the equation. The first step to under-
ter 9, this volume), not to mention rigorous standing a new equation is to identify the variables
experimental analysis (Perone, 1999; Sidman, 1960; in the equation, or the environmental and behav-
see Chapter 5, this volume). ioral events that are measured directly in the experi-
After a visual analysis of the relation between ment. In the case of Herrnstein’s hyperbolic
independent and dependent variables, one must equation, the experimenter measures response rate,
Figure 10.1. Anscombe’s (1973) quartet. The four graphs show different
data sets that are described by the same linear equation, which is indicated
by the straight line in each panel. The datasets also have the same mean
and variance. From “Graphs in Statistical Analysis,” by F. J. Anscombe,
1973, American Statistician, 27, pp. 19–20. Reprinted with permission
from The American Statistician. Copyright 1973 by the American Statistical
Association. All rights reserved.
222
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
response rates increase rapidly with increases in

reinforcer rates. As re increases, however, as in the
bottom curve, the effects of increases in reinforcer
rates are essentially dampened. Because r and re are
summed in the denominator, smaller changes occur
in R with increases in reinforcer rates.
Parameters usually reflect the operation of envi-
ronmental, behavioral, or physiological processes.
For example, the parameter re in Herrnstein’s hyper-
bolic equation is thought to reflect the rate of back-
ground, extraneous reinforcers. In our example,
these extraneous reinforcers represent all of the
Figure 10.2. Two examples of Herrnstein’s hyper- reinforcers in the child’s environment (video games,
bolic equation, which predicts how response rate
food, etc.) except for the reinforcers delivered as a
changes with increases in reinforcer rate. The only dif-
ference between the two curves is the value of re. consequence of disruptive behavior. The degree to
which the interpretations of parameters are con-
R, as a function of reinforcer rate, r. For example, firmed empirically is an interesting and exciting fea-
assume one measured rates of engaging in disruptive ture of quantitative modeling. As we discuss in more
behavior as a function of rates of social attention for detail in the Evaluating Nonlinear Models: Nonlin-
this behavior. The dependent variables (e.g., ear Regression section, even if an equation describes
response rates) will always appear on the left side of a dataset extremely well (e.g., the curve comes very
the equation, and the independent variables (e.g., close to the obtained data), it does not necessarily
reinforcer rates) will always appear on the right side mean that the interpretations of the parameters are
of the equation. supported. Eventually, such inferences should be
Next, identify the parameters. The parameters supported by direct experimental analysis of the
are the quantities in the equation that are not controlling variables, whether biological or environ-
directly measured; rather, they are estimated statisti- mental (Dallery & Soto, 2004; Shull, 1991). For
cally (see Evaluating Linear Models: Linear Regres- instance, if re is thought to measure extraneous rein-
sion and Evaluating Linear Models: Nonlinear forcers, its value should increase or decrease as rein-
Regression sections). The parameters affect the forcers are added or subtracted, respectively, from
shape and position of the curve (or line), and a com- the environment in which the target behavior is
puter program that conducts regression analyses measured.
will find values for the parameters so that the curve One way to understand an equation is to plot the
(or line) comes as close to the data as possible equation using multiple parameter values as in Fig-
(called best-fit parameter values). To illustrate how ure 10.2. Creating multiple plots of an equation can
the value of a parameter affects the shape of the easily be done using a spreadsheet program such as
curve in Figure 10.2, we used two values for the Microsoft Excel (at the time of this writing, Excel is
parameter re and the same value for the parameter k. perhaps the most common spreadsheet program in
If one considers just the top curve, for example, the use). As described earlier, Figure 10.2 illustrates
curve predicts response rates given one value of k, that the smaller the value of re, the steeper the curve
one value of re, and all reinforcer rates between zero rises toward its asymptote, k. Although not shown,
and 200 reinforcers per hour. The curve can also plotting Equation 1 with different values of k illus-
predict response rates for higher reinforcer rates, but trates how increases in k increase the maximum
for practical purposes, we do not show these in the value reached by the curve (100 in Figure 10.2).
graph. The value of re is small in the top curve. In Understanding how the shape of the function
terms of the equation, the value of re does not pro- changes as the parameter values change is essential
duce a large impact in the denominator, and so because changes in the shape reflect changes in
223
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
behavior. In an applied context, increasing re as pecks a lighted key for an immediately delivered
depicted in Figure 10.2 would mean that rates of small amount of grain instead of waiting for a larger
disruptive behavior would decrease, even if rein- amount. These examples of choice, of choosing one
forcer rates for disruptive behavior remain the same activity over another, are amenable to quantitative
(compare the predicted response rates between analysis. One common way in which psychologists
curves when the reinforcer rate is 100 reinforcers/ study choice in the operant laboratory is by using
hour; McDowell, 1986). concurrent schedules of reinforcement. In the
There is obviously more to dissecting and digest- pigeon example, two concurrent, mutually exclusive
ing a quantitative model than examining the shape alternatives were available (pecking for small pay-
defined by the equation. Researchers need to ask offs or waiting for larger payoffs). This situation is
whether they have enough data to evaluate the also common in naturalistic, clinical settings: A
model (e.g., a general rule of thumb is to have twice child may choose to engage in disruptive behavior
as many data points as the number of parameters), rather than appropriate behavior (Borrero &
whether there are known properties of behavior or Vollmer, 2002); a smoker may choose to light up or
alternative models that they need to consider in forgo a cigarette (Dallery & Raiff, 2007).
evaluating the model, and whether they need to The relative rates at which the behavior
consider statistical or theoretical requirements for occurs—in the laboratory or in the world outside
estimated parameter values (we discuss these con- the laboratory—can be powerfully governed by rela-
siderations in more detail in the Evaluating Linear tive rates of reinforcement earned for engaging in
Models: Linear Regression and Evaluating Nonlinear them. To quantitatively model the relation between
Models: Nonlinear Regression sections). relative responding and reinforcement over time, the
This brief introduction provides a starting point response and reinforcer rates need to be denoted in
from which to approach equations in behavioral sci- mathematical terms. In the case of the pigeon
ence. All equations in behavioral science share the responding on a two-alternative concurrent sched-
same general structural and functional characteris- ule of reinforcement, the rate of responding on one
tics. The tactics presented in this section represent a alternative can be represented by R1, and the rate of
broad, yet effective strategy to dissect equations. responding on the other by R2. The rate at which
They may be summarized as follows: Visually reinforcers are delivered on the two alternatives can
inspect the shape of the dataset when plotted in be represented by r1 and r2. Some authors use differ-
graphical space, decompose the variables and ent letters, but they signify the same quantities.
parameters in the equation, explore how the plot of In 1961, Richard Herrnstein examined the rela-
the equation changes as the parameter values tion between rates of reinforcement and responding
change, and consider what these changes mean in in a seminal study. His subjects were pigeons, and
terms of behavior and its causes. they could peck two lighted keys that were mounted
on the front panel (one on the left and one on the
right) of an experimental operant chamber. Pecking
Introduction to a Quantitative
the keys sometimes resulted in brief access to food.
Description of Choice
Specifically, the rates of food access varied according
One purpose of quantitative analysis is to under- to variable-interval (VI) schedules of reinforcement,
stand the complex interaction between the determi- which delivered a reinforcer for the first response
nants of behavior (e.g., biological, environmental, after some interval had elapsed. For example, the left
pharmacological) and some measure of behavior key may have been scheduled to provide access to
(e.g., response rate, latency). For instance, research- food every 30 seconds on average, and the right key
ers may be interested in when and why a bee travels may have only provided access to food once every
to a new field of flowers for nectar, why a shopper 60 seconds on average. After varying these rates of
on a diet picks up some candy in the checkout line, reinforcement across a wide range of values, Herrnstein
or why a pigeon in an operant chamber impulsively plotted the proportion of responses allocated to one
224
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 10.3. Examples of matching and deviations from matching. The left
panel shows perfect matching. The right panel shows two common deviations
from perfect matching. The dashed curve is an example of bias, and the dotted
curve represents an example of insensitivity.
key (in our example, responding on the left key is As the predictions of Equation 2 were compared
designated as R1) as a function of the proportion of with behavior, however, researchers soon discov-
reinforcers delivered on that key. Herrnstein found ered that perfect matching was actually a special
that the shape of the data path was a straight line. See case of a more generalized equation (Baum, 1974;
the diagonal line in the left panel of Figure 10.3. Data Staddon, 1968). Obtained data tended to deviate in
points indicate hypothetical obtained response and two ways from the straight line shown in the left
reinforcer proportions. The straight line predicts that panel of Figure 10.3. First, animals sometimes
the proportion of responses should equal, or match, showed a bias for one alternative over another. Per-
the proportion of reinforcers obtained at that alterna- haps one response key was easier to peck or, in a
tive. So, for example, if the proportion of reinforcers naturalistic setting, the attention derived from dis-
on the left key equaled .25, then the proportion of ruptive behavior was “better” than the minor atten-
behavior allocated to that side would also equal .25. tion for appropriate behavior. In the right panel of
This shape can be described by an equation: Figure 10.3, the impact of such asymmetries on
R1 r behavior is shown by the dashed curve. Because the
= 1 . (2) curve is bent upward, one can deduce in the context
R1 + R 2 r1 + r2
of our example that left responses are occurring
Equation 2 describes a straight line passing more frequently than predicted by Equation 2 (the
through the origin (x = 0, y = 0) with a slope of solid line). Note that this bias toward the left alter-
1—a relation termed perfect matching. Herrnstein’s native is not caused by the rates of reinforcement
(1961) study, and Equation 2, has inspired hun- associated with the left and right alternatives. Sec-
dreds of studies on choice. The applications have ond, animals also showed some degree of insensitiv-
ranged from the laboratory to clinical settings (Bor- ity to changes in the rates of reinforcement. The
rero & Vollmer, 2002; Murray & Kollins, 2000; St. dotted curve shows an example of such insensitivity;
Peter et al., 2005; Taylor, Lincoln, & Foster, 2010; note how the curve bends toward indifference, or a
see Volume 2, Chapter 7, this handbook), social proportion of .5. The curve is above the straight line
dynamics (Borrero et al., 2007; Conger & Killeen, of equality at x-axis values less than .5 and below
1974), and education (Martens & Houk, 1989). The the straight line at x-axis values above .5.
matching law has also served as a basis for studies in A single equation describing sensitivity to rela-
behavioral ecology (Houston, 1986) and behavioral tive reinforcer rate across the two alternatives and
neuroscience (Lau & Glimcher, 2005). bias toward one alternative over the other (not
225
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
resulting from differences in reinforcer rate) would where a is the slope, log b is the intercept, and the
be useful. Unfortunately, bias and sensitivity are logged response ratio and logged reinforcer ratios
hard to detect visually and to quantify when propor- are the dependent and independent variables,
tions are used. They are hard to detect because a respectively. This straight line is different from the
plot of an animal’s behavior may show biased line described by Equation 2 in that its slope and
responding (e.g., the plot tends to bend upward) intercept can vary; perfect matching specifies a slope
and insensitivity (the plot also bends toward indif- of 1 and an intercept of 0.
ference) at the same time. Also, data from live Because the logarithmic form of the generalized
organisms usually demonstrate more variability than matching equation is a straight line, one can evaluate
our graphical example, which further complicates the equation’s descriptive accuracy by fitting a line
visual detection of bias and insensitivity. Staddon to the data. Bias is indicated by the intercept of the
(1968) found that plotting ratios instead of propor- straight line, log b, and sensitivity is indicated by the
tions aided in visually detecting deviations from strict slope of the line, a. One example of a line described
matching. The ratio form of matching is algebraically by Equation 5 is shown in Figure 10.4. First, note
equivalent to the proportional form (to obtain the that the slope is less than 1.0. This means that for
ratio form from the proportional form, invert both every unit change in the reinforcer ratio, there is less
sides of Equation 2, separate the terms involved, and than a one-unit change in the response ratio. Con-
subtract one from both sides), and it is written sidering what the value of the slope means in terms
of unit changes in the dependent and independent
R1 r1 variables is often useful. Second, note that the line is
= . (3)
R 2 r2 elevated above the origin, which shows some bias for
the response alternative in the numerator, which is
Baum (1974) modified Equation 3 to account for R1, or the left key, in our example. Of course, the
deviations from matching by adding two parameters, line could be steeper than 1.0, and it could intersect
a and b, to Equation 3: the ordinate below the origin. Consider what this
r 
a
R1
= b 1  , (4)
R2  r2 
where a quantifies sensitivity to the ratio of rein-

forcer rates on alternatives 1 and 2, and b quantifies
bias. Equation 4 is known as the generalized match-
ing equation, or power–function matching, because
the independent variable is raised to a power, a. Tra-
ditionally, Equation 4 is evaluated by plotting the
logarithm of the response ratio as a function of the
logarithm of the reinforcer ratio. Logarithmic trans-
formation2 of Equation 4 results in a straight line
when the equation’s predictions are plotted in
graphical space. The logarithmic form of the gener- Figure 10.4. An example of the generalized
alized matching equation is matching equation. The line represents a best-
fitting line obtained via least-squares regression
to the obtained response and reinforcer rates. See
R  r  the text for the equation. The data points in col-
log  1  = a log  1  + log b, (5)
 R2   r2  umns E and F from Table 10.2 are plotted.

The logarithm of the product of two numbers is the sum of the logarithms of the individual numbers, and the logarithm of an exponentiated number
2
is the product of the exponent and the logarithm of the number.
226
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
would mean in terms of behavior. It would mean ratios can be described by a straight line—but is it? If
that the change in response ratios is more than one visual inspection suggests that the data are orderly,
unit for every unit change in reinforcer ratios (called sensible, and linear, one can evaluate this theory of
overmatching; Baum, 1979) and that there is a bias choice statistically by performing linear regression.
toward the response alternative in the denominator, More generally, when researchers want to evaluate a
the right key. Now one has a single, elegant equation relation between an independent and a dependent
to describe how two variables (i.e., reinforcer rates variable, and the relation between them can be
on two alternatives), other asymmetries in the prop- graphed as a straight line, they use linear regression
erties of the responses or reinforcers (i.e., changes in (if the variables are not linearly related, they use
the intercept or bias), and the degree of insensitivity nonlinear regression, which is covered in the Evalu-
to changes in reinforcer rates (i.e., change in the ating Nonlinear Models: Nonlinear Regression sec-
slope or sensitivity) affect choice. tion). When the two variables are not causally
One log transforms the ratios to evaluate the gen- related, researchers use a correlation coefficient. For
eralized matching equation for several reasons. The example, one would not want to perform regression
main reason, as implied earlier, is that it is easier to to describe the relation between response rate and
detect departures from strict matching. The log response latency. As a rule of thumb when deciding
transform also turns the curves produced by bias between regression and correlation, ask whether the
and insensitivity into straight lines (McDowell, independent variable plotted along the abscissa is
1989), and thus bias and insensitivity are reflected causing the changes in the dependent variable. If it
by changes in intercept and slope, respectively. is, use regression. If the variable plotted along the
Indeed, quantitative analysis has several useful data abscissa is a dependent variable (response latency),
transformations, which are used for a variety of rea- and so is the variable plotted along the ordinate (e.g.,
sons, for example, to increase interpretability of a response rate), then a correlation is more appropri-
graph or to meet assumptions for statistical analysis ate. Also, before conducting a so-called ordinary
(e.g., normality). Some balk at the idea of transform- regression analysis, whether linear or nonlinear, it is
ing behavioral data; after all, one does not observe important to ensure that the assumptions of ordinary
logarithms of behavior. This argument misses the regression are not violated. For convenience, these
point of transforms, which is to magnify the ability assumptions are listed in Table 10.1.
to describe and explain environment–behavior rela- Consider the data shown in Table 10.2. These
tions. For instance, a logarithmic transform might data show how a human might interact during a dis-
help one to see the spread of data across the range of cussion with two confederates. The dependent vari-
values of the independent variable, especially when able is how much the individual interacts with each
that range is very large. That is, when plotting loga- confederate or, more specifically, the rate of words
rithms of the data, the spacing of data between num- per minute directed toward each confederate (col-
bers 1 and 10 is the same as the spacing between the umns C and D). The independent variable is the rate
numbers 10 and 100. Indeed, even a simple measure of social reinforcers per hour (e.g., agreement state-
of response rate is a transformation. Transforma- ments, nods; columns A and B) delivered by each of
tions can have advantages, much as the metric sys- the confederates. The social reinforcers are delivered
tem has advantages over the U.S. system of according to VI schedules. The rates of reinforce-
measurements (see Shull, 1991, and Tukey, 1977, ment are varied across 10 conditions. Thus, when
for further discussion of transformations). the reinforcer rate is 5.97 reinforcers per hour for
Confederate 1 and 4.63 reinforcers per hour for
Confederate 2, the rates of words per minute
Evaluating Linear Models:
directed toward each confederate are 6.84 and 4.66,
Linear Regression
respectively. It is important to remember the units
The generalized matching equation states that the in which the variables are expressed; they are impor-
relation between log reinforcer and log response tant in understanding any quantitative model.
227
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
Table 10.1
Assumptions of Ordinary Least-Squares Regression
■■ The values of the independent variable are known precisely. All of the variability in the data is in the values of the dependent variable.
■■ The variability in the dependent variable, the error, follows a normal bell-shaped distribution.
■■ The degree of error, or scatter, in the dependent variable is the same at all points along the line (or curve). Another way to say this is
that the standard deviation of the scatter must be the same at all points along the line (or curve), which is known as homoscedasticity.
■■ The observations are independent.
Note. From Fitting Models to Biological Data Using Linear and Non-Linear Regression: A Practical Guide to Curve Fitting
(p. 57), by H. Motulsky and A. Christopoulos, 2004, Oxford, England: Oxford University Press. Copyright 2004 by
Oxford University Press. Adapted by permission of Oxford University Press, Inc.
Table 10.2
Regression Calculations for a Linear Model
A: rnf rate B: rnf rate C: rsp rate D: rsp rate E: log rnf F: Log rsp G: predicted log
Condition by Conf. 1 by Conf. 2 to Conf. 1 to Conf. 2 ratio ratio (Y ) rsp ratio (Y′) H: Y − Y′ I: (Y −Y′)2
1 5.97 4.63 6.84 4.66 .11 .17 .19 −0.02 0.0004
2 2.35 5.46 4.40 3.77 −.37 .07 −.19 0.26 0.0676
3 7.09 2.31 6.29 1.55 .49 .61 .49 0.12 0.0144
4 1.55 11.15 2.50 11.38 −.86 −.66 −.58 −0.08 0.0064
5 11.08 2.20 8.60 2.01 .70 .63 .66 −0.03 0.0009
6 2.42 3.55 1.20 1.96 −.17 −.21 −.03 −0.18 0.0324
7 4.96 1.17 2.06 0.55 .63 .57 .60 −0.03 0.0009
8 2.46 4.04 1.80 2.07 −.22 −.06 −.07 0.01 0.0001
9 4.57 2.55 2.48 1.42 .25 .24 .30 −0.06 0.0036
10 1.13 5.29 0.91 2.48 −.67 −.44 −.43 −0.01 0.0001
Note. Columns A–D show response and obtained reinforcer rates for a subject engaging in speech directed at Conf. 1 or
2. Columns E and F present the logged rsp. and rnf. rates, which are the data used to perform linear regression. Column
G shows the predicted log rsp. ratios calculated by Equation 5 with a = .80 and log b = .10. Column H shows the resid-
uals, and column I shows the squared residuals. The sum of the squared residuals is SSREG = 0.13, the sum of squared
deviations from the mean is SSTOT =1 .78. The r2 is calculated as 1− (SSREG/SSTOT) = .93. Conf = confederate; rnf =
reinforcement; rsp = response.
Columns E and F show the logged reinforcer ratios Linear regression may be calculated with a statis-
(r1/r2) and response ratios (R1/R2), respectively. The tics program or by an algorithm in a spreadsheet
data from columns E and F are the data plotted program. These programs start with initial values of
graphically in Figure 10.4. The independent vari- the slope and intercept and then adjust these values
able, the logged reinforcer ratio, is always plotted over a series of iterations so that the line is as close
along the abscissa; it is important that the indepen- to the obtained data as possible.3 The initial values
dent variable be plotted along the abscissa and the used by programs may be unknown to the user. If
dependent variable be plotted on the ordinate when one uses a spreadsheet with regression capabilities,
performing linear regression. for example, one needs to enter the initial values
The parameters of a line can also be calculated directly from formulas derived from linear regression theory rather than minimizing the residual sum-
3
of-squares. We chose to illustrate the curve-fitting approach to calculating the parameter values because the logic should be easier to understand in
the context of a more familiar function form (i.e., a line) relative to less familiar forms (i.e., curves) and because this method can be applied to both
linear and nonlinear regression.
228
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
into the spreadsheet. Currently, for pedagogical pur- matter; one can square the residuals and then sum
poses, assume the starting values must be entered. them. (There is another important reason to square
The starting values will depend on the particular the residuals, which we come back to this later in the
dataset being evaluated. One shortcut in determin- Rationale for Squaring the Sum of Squared Devia-
ing starting values is to inspect the plotted data. If tions in Regression Analysis section). We want this
the slope of the line is positive, one can use 1 as its sum of squared residuals to be as small as possible,
starting value. If it is negative, one can use −1. One so the algorithm seeks to minimize this sum. The
can also examine where the line will intersect the procedure is as follows: The program increases or
ordinate at zero on the abscissa, and then enter that decreases the parameter values and asks, is the sum
value as the starting value for the intercept. Once of squared residuals smaller than the last iteration?
these values are entered, the equation makes predic- The program continues to change the values to
tions about the response ratios. For example, let us make the sum smaller. At some point, when changes
use an intercept of 0 and a slope of 1 and predict the in the parameter values produce very small changes
logged response ratio when the logged reinforcer in the sum of squared residuals, the algorithm stops
ratio is 0.11. Insert these values into the right side of and reports the best-fitting parameter values. The
Equation 5: description of Table 10.2 lists the best-fitting param-
eter values obtained via least-squares regression
R  r 
log  1  = a log  1  + log b, (which is called least squares because of the process
 R2   r2  of minimizing the sum-of-squared deviations
R  between the line and the data). In column G, we
log  1  = 1 ( 0.11) + 0, and show the predicted data points, labeled Y′. Equation 5,
 R2 
the generalized matching equation, predicts these
R  data points—they can be calculated by hand by
log  1  = 0.11.
 R2  inserting the slope and intercept values given in the
table description and any one of the log reinforcer
The predicted log response ratio is 0.11. The ratios in Table 10.2. The deviation between the
prediction is not too far off from what we observed obtained data, Y, and the predicted data, Y′, is
(0.17; first row in column F). Because the regres- shown in column H. If each of these values is
sion analysis has already been conducted, how- squared, we obtain the squared deviations, shown in
ever, we know the prediction could be better. Note column I. The sum of these deviations in column I is
that one could, of course, select values of a and b what the algorithm minimizes to obtain the best-
such that the prediction at one x value exactly fitting parameter values.
equaled the obtained y value, but the key is to To show the accuracy of the regression model
bring the fitted line as close as possible to all the visually, vertical lines can be drawn between each
data points. data point and the predicted line (these lines repre-
The vertical distances or deviations between the sent the residuals), which is shown in the left panel
calculated predictions and the obtained data points of Figure 10.5. The distances are very small. In lin-
are called residuals. We do not show all of the calcu- ear (and nonlinear) regression, these distances are
lations here, but there will be residuals for all of the compared with the distances given by another
data points; for our single data point the residual is model, the null hypothesis. The null model asserts
0.17 − 0.11 = 0.06. At first, given the initial param- that the mean of the obtained data can account for
eter values, the residuals will be large. The algo- (predict) the results better than the more compli-
rithm has not adjusted the parameter values to make cated two-parameter model. The null model is more
these distances smaller, which is the same thing as parsimonious. There are no parameters, just the cal-
making the line fall closer to the data. How does the culated mean of the data. The accuracy of this model
algorithm make these distances smaller? Whether is shown in the right panel of Figure 10.5. To quan-
the distances are positive or negative does not titatively characterize this visual comparison
229
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
Figure 10.5. A visual representation of how linear regression is a matter of

model comparison. The plot on the left shows the regression model, the alterna-
tive hypothesis. The plot on the right shows the null model. In linear regres-
sion, one compares the vertical distances between the data points and the line
on the left to the vertical distances between the data points and the horizontal
line on the right.
between the two models, sum the squared residuals and the distances between the data and a horizontal
in the regression model (denoted SSREG), and sum line at the mean. In other words, r2 is a ratio of how
the squared deviations in the null model (denoted well the model predicts the data relative to the sim-
SSTOT; TOT stands for total deviation from the pler null model.
mean). The ratio of these quantities is subtracted The r2 is just the first step in evaluating the mod-
from 1, which provides the first index of whether el’s accuracy. Examine the left panel in Figure 10.5,
the model is accurate: particularly the patterns of the data points around
the line. One can see that about the same number of
 SS  data points are above or below the line, that the data
r 2 = 1 −  REG . (6)
 SSTOT  points are approximately the same distance from the
line, and that the deviations, positive or negative,
The r2 resulting from the linear regression analy- are not systematically related to the x-axis values, a
sis is shown in Table 10.2. If the value of SSREG is condition known as homoscedasticity. Violations of
close to SSTOT, then the value of the ratio will be homoscedasticity, a condition known as heterosce-
close to 1, which will result in an r2 close to 0. If dasticity, means that there is a serious problem with
SSREG is very small compared with SSTOT, the ratio the data or the model, even if one has a high r2. To
will be small and the r2 value will be close to 1. understand why, examine Figure 10.6. The left and
Because r2 is a ratio, it can be negative when the null middle panels show different examples of heterosce-
model predicts the data better than the regression dasticity. In the left panel, the model (the line) obvi-
model. In other words, a negative value is obtained ously and systematically misses the data. When the
when SSREG is larger than SSTOT. Some authors desig- log reinforcer ratio is negative, the line overpredicts
nate the r2 in different ways, such as by reporting and then underpredicts the data. When the log rein-
the variance accounted for in terms of a percentage forcer ratio is positive, the line tends to overpredict
(r2 multiplied by 100), abbreviated VAF or VAC. the data. The systematic pattern of deviations vio-
Whether r2, variance accounted for, or even variance lates the requirement of homoscedasticity. In the
explained, all of these represent a relative compari- middle panel, it is harder to see how the data depart
son of distances between the data and the predicted from homoscedasticity, which is often the case in
line (or curve; see the section Evaluating Nonlinear quantitative modeling. Although the r2 may be high,
Models: Nonlinear Regression later in this chapter) it does not tell the whole story. Careful visual
230
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 10.6. Two examples of fits of a linear equation to data showing het-
eroscedasticity (left and middle panels) or homoscedasticity (right panel). The
left panel shows that the model underpredicts the data when reinforcer ratios
are smaller. The middle panel shows a more subtle example, and it may be
difficult to see heteroscedasticity. The right panel shows what appears to be a
homoscedastic pattern, in which the data are evenly spread along the range of
the fit.
inspection reveals that more data points are above shows more clearly than that of Figure 10.6 that the
the line and that within a certain range of the x-axis, data are heteroscedastic. The right panel shows a
approximately −0.8 to −0.2, all the data points fall desirable pattern of residuals, one that shows
above the line. Finally, in the right panel, one can homoscedasticity. To perform a residual analysis
see that the number of points above and below the using the data from Table 10.2, plot the data in col-
line are approximately equal, that the deviations umn H, the raw residuals, as a function of the inde-
above and below are roughly equal, and that there is pendent variable shown in column E, the logged
no systematic relation between positive and negative reinforcer ratios.
deviations and x value, in keeping with the require- Residual analysis can be performed in other ways
ment of homoscedasticity. To be sure whether the (Cook & Weisberg, 1982). The standardized resid-
model produces systematic departures from ual could be plotted as a function of the indepen-
homoscedasticity, one needs to perform a finer dent variable. The standardized residual is
grained analysis, which is called residual analysis. calculated by dividing each raw residual by the stan-
Most programs routinely report residual analyses, dard deviation of residuals. Standardized residuals
and we recommend careful inspection and reporting allow one to more easily detect the presence of outli-
of these results. ers (e.g., a residual ±2 standard deviations from the
Residual analysis can be performed in several mean of the residuals). To detect patterns in the
ways. All of them use the same general procedure: residuals, one could use inferential statistics to sup-
Evaluate the pattern of residuals as a function of the plement visual analysis. For example, in some cases
independent variable or as a function of the pre- a correlation between the residual and the indepen-
dicted data (e.g., each Y′ in column G of Table 10.2) dent variable could be calculated (e.g., Dallery, Soto,
to determine whether the residuals are evenly and & McDowell, 2005; McDowell, 2005). A significant
randomly distributed. For example, one of the most correlation would indicate systematic departures
basic ways to perform residual analysis is to plot the from homoscedasticity, and a nonsignificant correla-
raw residuals (i.e., the difference between the pre- tion (which is desirable) indicates homoscedasticity.
dicted and obtained values) as a function of the There are more advanced techniques to detect pat-
independent variable. The residuals from Figure terns in the residuals, which involve fitting curves to
10.6 are plotted in Figure 10.7 as a function of their the residuals. Some authors use cubic, quadratic,
corresponding reinforcer ratios. The order of the and other polynomial equations to detect these devi-
panels corresponds to the panels in Figure 10.6. ations from homoscedasticity (e.g., see Sutton,
Thus, the left panel shows obvious departures from Grace, McLean, & Baum, 2008). Additionally, there
homoscedasticity. The middle panel in Figure 10.7 are techniques that involve examining the number
231
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
Figure 10.7. Residual plots of data shown in Figure 10.6. The order of pan-
els reflects the order in Figure 10.6. See text for further details.
of residuals that have a positive or negative sign and than overall measures of model accuracy such as r2,
the number of consecutive residuals of the same they allow researchers to more easily evaluate model
sign, called runs of residuals (McDowell, 2004; accuracy (see also Dallery et al., 2005). Sutton et al.
Motulsky & Christopoulos, 2004). Given the logic concluded that although the r2s of the two models
of residual analysis, the latter runs test should show were equivalent, analysis of the residuals yielded sub-
that an equal number of residuals are above and tle, yet important distinctions between the models.
below the line and that there are no consecutive Thus far, we have considered three essential steps
runs of one sign or the other. in performing linear regression: careful visual
There are several potential consequences if non- inspection of the data, calculating r2, and performing
random residuals are obtained. First, it may mean residual analysis. Researchers must be sure that a
that the assumption of random scatter is violated line describes the data. If it does not, they may need
and that using an ordinary least-squares regression to evaluate a different model, but only if the alterna-
technique may not be appropriate. We explain why tive model makes theoretical sense when applied to
this is the case in the Rationale for Squaring the the data. This issue requires critical analysis. There
Deviations in Regression Analysis section. Second, are almost always theoretical reasons for choosing a
the equation might be incorrect. Perhaps the equa- particular model, and thus regression analysis in
tion form is incorrect, a parameter is missing from particular and quantitative modeling in general are
the equation, or the parameters have not been not simply a matter of obtaining a good fit.
adjusted properly (see Motulsky & Christopoulos,
2004, on troubleshooting bad fits). Something may
Additional Questions When
also possibly have gone wrong with the experiment.
Evaluating Linear Regression
Perhaps some uncontrolled variable was responsible
for the systematic departures, or an error was made Do the Parameter Values Make Sense?
in transcription or data analysis. Before dismissing a Estimated parameter values must be plausible, both
model because of nonrandom residuals, one should in terms of precedent and in relation to the dataset
be sure that an error in the experiment or analysis is to which the equation is fitted. For example, in the
not the source of the nonrandom residuals. case of linear regression, the y-intercept should be
Although residual analysis is critical in evaluat- plausible given the obtained data in the experiment.
ing the accuracy and validity of a particular model, What should one expect to see if the value of the
for example, the generalized matching equation, independent variable is 0? In the case of generalized
residual analysis can also be used to compare mod- matching, the independent variable is 0 when the
els. Sutton et al. (2008) used several datasets to dis- reinforcer rates are equal. Using the example from
tinguish two models of choice and performed what the preceding section, assume each confederate
might be called a meta-analysis of residuals. Because delivered 60 reinforcers per hour; thus, 60/60 is 1,
residuals can show deviations at a finer grained level and the log of 1 is 0. A safe initial assumption,
232
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
therefore, is that one would observe equal response Therefore, the amount of variability is not enough to
rates on the two alternatives (i.e., a log response invalidate the use of ordinary linear regression tech-
ratio close to 0), and some deviation might be niques. If the variability in the independent variable
expected if the response alternatives differ in some is high, alternative regression approaches are avail-
way (e.g., one confederate is more genuine or the able (Brace, 1977; Riggs, Guarnieri, & Addelman,
quality of feedback is different). That is, one might 1978; although the same considerations apply for
expect some bias. If the obtained parameter value is nonlinear regression, these approaches apply only to
widely discrepant from the existing literature, then linear regression). To our knowledge, there are no
something is most likely wrong. Solving the discrep- specified rules about how much variability is too
ancy may require some detective work. Perhaps the much to warrant an alternative approach.
problem is lack of experimental control or human or
measurement error, or some variable other than Is Another Model More Appropriate?
reinforcer rate is exerting an effect. Even when the model accounts for the variability in
the data points (i.e., a high r2), the residuals are
Is There Error in the Values of the homoscedastic, and the parameter values are sensi-
Independent Variable? ble, there may be other questions regarding the
When researchers perform regression analysis, they model that cannot be answered by linear regression
are predicting the values of the dependent variable analysis. For example, one may ask whether another
at specific values of the independent variable, which model of concurrent choice can account for more of
means that they need to obtain precise numbers for the variance in the data. In the case of concurrent
their obtained data. In our earlier example, this schedules of reinforcement, in addition to the gener-
would mean precise reinforcer rates. For example, if alized matching law, alternative models can account
one uses an averaged value of 6.0 reinforcers per for choice (e.g., Dallery, McDowell, & Soto, 2004;
hour as one of the independent variable values in McDowell & Kessel, 1979). One may also ask
linear regression, one must be sure that the rein- whether a more parsimonious model can account
forcer rates going into this average are extremely for the same amount of variance. In general, a more
close to 6.0 reinforcers per hour. If the measurement parsimonious model would be one that contained
of a particular value of the independent variable has fewer parameters. To our knowledge, no current
a lot of error or variability, one cannot make precise model can account for the data and is more parsimo-
predictions. Regression analysis assumes that there nious than generalized matching. In other areas of
is no, or very little, error variance in the indepen- research, however, questions about parsimony and
dent variable. In the case of a generalized matching accuracy are still quite relevant (e.g., Mazur, 2001;
analysis, there will be some variability in the inde- McKerchar et al., 2009). Questions about accuracy
pendent variable, the obtained log reinforcer ratio, and parsimony can be addressed using model com-
because the obtained log reinforcer ratio is plotted parison approaches, which we discuss later (see also
and depends in part on the rate of responding at Mazur, 2001, for an example of model comparison).
each alternative. (One could plot the experimentally The underlying assumptions of the selected
programmed reinforcer ratio, which has no error, model should also be examined. Indeed, a highly
but the programmed ratio may not be what the ani- accurate model may be based on faulty assumptions.
mal experiences.) Because the schedules for each There are no tidy rules for discovering these
alternative are VI schedules, the variability in rein- assumptions; it takes critical thinking and a thor-
forcer rates will be minimal compared with the vari- ough understanding of the mathematical model.
ability in response rates, and thus the resulting One assumption of generalized matching, for exam-
reinforcer ratio will have minimal variability. For ple, is that sensitivity to reinforcer rates (i.e., a in
example, under a VI 60-second schedule, a pigeon Equation 5) should be constant for an individual
could press once a minute or 100 times a minute subject across changes in reinforcer magnitude or
and receive about the same number of reinforcers. quality. By constant, we mean that a should not
233
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
change if, for instance, a is estimated using small key—the scatter of response rates is normally dis-
food pellets or large food pellets. If sensitivity to tributed, or Gaussian. Most of the numbers will be
reinforcer rates does change when different rein- close to the average, but some scatter will also be
forcer magnitudes are used, then the assumptions observed further away from the average. In other
underlying the generalized matching equation have words, scatter close to the mean is more likely than
been violated, and the application of the equation is scatter far away from the mean.
invalid. At least one study has shown that sensitivity Assume the average is known because the pigeon
to reinforcer rates does change depending on rein- has been observed for 100 consecutive days. The
forcer magnitude (Elliffe, Davison, & Landon, average is 50 responses per minute. On Days 101
2008). Elliffe et al.’s (2008) findings question a and 102, one student takes measurements, and on
matching-based understanding of how multiple Days 103 and 104, another student takes measure-
independent variables—reinforcer rate and ments. The first student obtains 45 and 55, and the
magnitude—interact and govern choice. Whether second obtains 49 and 59. What if the same students
these findings will be confirmed by future research made two more measurements each? Which pair of
is an important issue, and such work may inspire observations would be more likely the second time
new models that will yield new insights into the around? This is a question about predicting behav-
determinants of choice. ior and how to maximize accuracy. To quantify
accuracy, calculate deviations from the mean.
Smaller deviations mean better accuracy. What if the
Rationale for Squaring the
researcher just calculates the deviations from the
Deviations in Regression Analysis
mean for both pairs? Ten units would be obtained
The main feature of the regression analysis is mini- for both pairs of observations (5 + 5 = 10 and 1 +
mizing the sum-of-squared deviations. The story 9 = 10). The researcher would conclude that both
behind the reason researchers seek to minimize the pairs are equally as likely because they have equal
squared deviations, or perform so-called least-squares total deviations from the mean. However, these cal-
regression, is an interesting one. In 1801, at the age of culations do not take into account a property of the
23, Carl Friedrich Gauss used the method of least deviations around the mean: the normal, Gaussian
squares to predict, with unprecedented precision, the distribution. To take this property into account,
path of a dwarf planet named Ceres. He made about the deviations are squared, that is, 52 + 52 = 50 and
22 observations over a 40-day period before Ceres 12 + 92 = 82. On the basis of these calculations, the
went behind the sun. To predict where it would reap- first pair of observations is more likely (see also
pear almost half a year later, he developed the method Motulsky & Christopoulos, 2004).
of least squares. The particular equation he used to The reason that squaring the deviations produces
predict the path of the planet need not concern us. the most accurate predictions can be seen in Figure
What made the method work was Gauss’s assump- 10.8. The abscissa shows the values of the observa-
tion that any error in the observations followed a nor- tions, and the ordinate shows the frequency of their
mal distribution. To illustrate why, we provide a occurrence. Consistent with the example, behavior
more familiar example than planetary motion. has been measured for 100 days. On the basis of the
Consider one of the more highly controlled distribution, one can see how the observations made
experimental preparations in behavioral science: a by Student A are more likely; the likelihood of Stu-
pigeon pecking a lighted key for brief access to dent A’s observations of 45 and 55 is greater than
grain. After every 20 pecks, access is granted for a that of the observations made by Student B.
brief period. As response rate is measured from If the observational error in a system under study
session to session, the numbers will vary. One day shows a Gaussian distribution, whether planetary
the response rate is higher, the next it is lower. motion or a pigeon’s pecking, then least-squares
However, all of the numbers revolve around some regression will be most accurate in generating the
average. The distribution of the numbers is the best parameter values, the values that make the best
234
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 10.8. An example of a Gaussian distribution.

The graph shows the frequency of observations, in this case
response rates, over the course of 100 sessions. The distribu-
tion has a mean of 50. Two pairs of observations are taken,
one by Student A (indicated by the As in the graph) and two
by Student B (indicated by the Bs in the graph).
predictions. The error could be intrinsic to the sys- We do not present these techniques because of space
tem itself, or perhaps the variation is the result of limitations (see Motulsky & Christopoulos, 2004),
some extrinsic factor such as measurement error or but we do want to emphasize why least-squares
how much the experimenter feeds the pigeon regression would not be appropriate for these distri-
between sessions. Regardless of the source, the butions. In a nutshell, it is because some residuals
observations over time, such as response rates over will have undue influence on the fit (similar to the
the course of 100 sessions, should show a Gaussian effect of an extremely large or small value on the
distribution. It is important to understand how and average of a set of values). The logic also explains
why the sum-of-squared deviations incorporate the why outliers require some special consideration in
Gaussian distribution in making predictions. regression, even if one is confident that the scatter of
Other methods to perform regression do not observations follows a Gaussian distribution.
assume a Gaussian distribution. Usually, least- Because the deviation to the outlier will be large, it
squares regression is a justified practice to assess a will have a large impact when one attempts to fit the
model, but there are times when the distribution of curve (or line). Consequently, the outliers will have
observations do not show Gaussian scatter; some- a disproportionate impact on the values of the
times the scatter of observations increases as the parameters, which would be unfortunate if the out-
value of the independent variable increases, perhaps lier was a fluke observation (e.g., the food dispenser
the scatter is wider than a Gaussian distribution, or stopped working one day and so did the animal).
perhaps the distribution is skewed and therefore There are methods to reduce the influence of outli-
non-Gaussian. For example, distributions of interre- ers, and these methods are called robust least-
sponse times are often skewed, and therefore using squares regression. Computer programs will differ
linear regression on mean interresponse times in which robust methods are offered, so we advise a
would not be appropriate in these cases. Techniques search for these robust methods if outliers are pres-
are available to perform regression on the basis of ent (and the reason for the outlier has been critically
these distributions that involve different methods of analyzed and including the outlier in the regression
weighting the residuals when calculating the best fit. analysis is justified).
235
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
An Extension of Matching Theory’s pigeon may engage in other behavior such as walking
Quantitative Description of Choice around, wing flapping, and so on, and presumably,
each behavior produces some natural reinforcer. Sim-
Just as the development of linear models often
ilarly, a human engaged in political speech could talk
begins with data, so too does the development of a
about a different topic, fidget, or stare into space.
nonlinear model. For example, imagine we have
Therefore, a single alternative arrangement is really
arranged a laboratory situation in which a human
a concurrent schedule, in which one choice is the
has one person to talk to, not two, and the rates of
target alternative and the other is the aggregate of
reinforcer delivery for talking about a certain topic,
all extraneous behaviors. Responding in a single-
say politics, are varied across conditions. In this
alternative situation can be expressed as
case, reinforcers consist of brief periods of eye con-
tact (Beardsley & McDowell, 1992). Eye contact is R r
= , (7)
provided according to several VI schedules of rein- R + R e r + re
forcement (manipulated across conditions), but
only when politics are discussed. We find that the where Re and re represent the rates of extraneous
rates of political speech increase with reinforcer responding and reinforcement, respectively (see
rates. The relation is not a straight line, however; it Herrnstein, 1974, for a further discussion). In other
is a curve. (Two examples of such curves are words, Re and re represent any responding and rein-
depicted in Figure 10.2.) At first, responding forcement extraneous to the target alternative. The
increases rapidly when reinforcer rates increase, and subscripts have been dropped from the target alter-
eventually responding levels off. The equation, of native because only one schedule of reinforcement is
course, will reflect this hyperbolic shape in the explicitly arranged.
observed data, and in that sense it does not take us Second, Herrnstein (1970) also assumed that R
beyond our observations. Our model, however, (target responding) and Re (extraneous responding)
should extend our observations in several ways. Can are exhaustive of the total amount of behavior possi-
it unify our observations of human behavior with ble in a given environment. In other words, R + Re is
the behavior of other organisms? Can it make novel all an animal can do in a given situation. Equation 7
and testable assertions about why the data take the can be solved for R by letting k = R + Re and then
shape they do? rearranging, which produces the familiar Equation 1:
Before answering these questions, let us sketch the
development of the hyperbolic equation. In 1968,  r 
R = k .
Catania and Reynolds published an enormous and  r + re 
elegant dataset on how pigeons’ rate of pecking
changed with changes in rate of VI reinforcement. The parameter k is now interpreted as total
The relation, as in our example of social dynamics, behavior possible in a situation. By assuming that
appeared hyperbolic. Thus, researchers knew that any all behavior is choice and that the total amount of
quantitative account of reinforced responding on a behavior in a situation is constant, Herrnstein
single VI alternative must be consistent with a hyper- (1970) derived Equation 1 from Equation 2.
bolic form. In 1970, Herrnstein demonstrated that Herrnstein’s (1970) theory consists of a quantita-
one can obtain a hyperbolic model from the original tive statement about how reinforcer rate affects
matching equation, Equation 2. In deriving a hyper- response rate (i.e., hyperbolically). More important,
bolic equation from Equation 2, Herrnstein made sev- the theory also explains why reinforcement affects
eral assumptions. First, he assumed that even when behavior in the manner specified by Equation 1:
only one alternative is specifically arranged, other Reinforcement alters the distribution of a constant
extraneous, unarranged behavioral alternatives exist. amount of behavior (Herrnstein, 1970, 1974). Now,
Consider the pigeon’s situation in Catania and Reyn- at least conceptually, one can see how Herrnstein’s
olds. In addition to pecking the lighted key, the model links responding in situations in which only
236
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
one choice is experimentally arranged and situations and reinforcer rates, and the curve represents the
in which two choices are experimentally arranged. least-squares fit of the hyperbolic equation.
Both represent choice situations. Indeed, Herrnstein Least-squares regression for nonlinear models
went so far as to say that all behavior is choice. A works in the same way as it does for linear models.
pigeon’s key pecking and a person’s talking can be As before, start with some initial parameter values
described by the same equation. The theory also for k and re. Here, it may be more difficult to guess
explains why one feature of the hyperbolic shape is where to start for the initial values; it is easier if one
seen: asymptotic responding. Once enough reinforc- examines a graph of the data. Inspect the data points
ers are delivered for some activity, the limits of the in Figure 10.9. The value of the asymptote k is
distribution of behavior are presumably reached measured in the units of the dependent variable—
(i.e., exclusive responding on the arranged alterna- usually responses per minute. Because k reflects the
tive), and thus asymptotic levels of responding are
observed.
Table 10.3
Regression Calculations for a Nonlinear Model

Evaluating Nonlinear Models:
Nonlinear Regression
B: rsp/min C: predicted
The preceding conceptual analysis highlights some A: rnf/hr (Y ) rsp/min (Y′) D: Y − Y ′ E: (Y − Y′)2
of the surprising and interesting consequences of 180 32.0 31.46 0.54 0.2916
quantitative models. Researchers can unify phenom- 138.6 30.1 30.45 −0.35 0.1225
ena and explain why they see similar functional rela- 98 28.0 28.78 −0.78 0.6084
tions across situations and species. These 22.5 18.8 17.68 1.12 1.2544
15 16.3 14.14 2.16 4.6656
consequences are moot unless the model is accurate,
9 6.7 10.09 −3.39 11.4921
however, and as such we must return to analysis.
Table 10.3 shows empirical observations in the Note. Columns A and B show response and reinforcer
single-alternative situation involving social interac- rates, and column C shows predicted response rates cal-
tion and political speech, and it follows the same culated by Equation 1 with k = 35.41 and re = 22.57.
Other conventions are identical to those in Table 10.1.
general format as Table 10.2. Figure 10.9 shows a Regression calculations: SSTOT = 478.43, SSREG = 18.43,
plot of the data in columns A and B, the response R2 = 0.96. rnf = reinforcer; rsp = response.
Figure 10.9. The curve shows Herrnstein’s (1970) hyper-

bolic equation fitted via least-squares regression to the data in
Table 10.3.
237
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
asymptote, one could start with the maximum

response rate observed, or 32 responses per minute.
Mathematically, the parameter re represents the rein-
forcer rate necessary to obtain half the asymptotic
response rate, k. The reinforcer rate for the maxi-
mum response rate was 180 reinforcers per hour, so
half of this reinforcer rate could be used as the ini-
tial value of re. We should note that, by definition, re
represents the aggregate rate of reinforcement from
sources extraneous to the target alternative. These
extraneous reinforcers are typically not measured
directly; rather, they are estimated in terms of the
reinforcer delivered on the instrumental alternative.
For example, if political speech were reinforced with
eye contact, then an re of 10 would be read as 10 eye
contacts per hour (the time unit for re is usually Figure 10.10. Plot of the residuals of the fit
reinforcers per hour). of Herrnstein’s (1970) hyperbolic equation. The
residual plot does not suggest systematic devia-
In performing nonlinear regression, having a
tions for homoscedasticity.
sense of what the parameters mean allows the
researcher to use the data to determine initial
parameter values. Then, the computer program (or residual is plotted as a function of the independent
algorithm) inserts the initial estimates (32 responses variable, which is shown in Figure 10.10. One can
per minute and 90 reinforcers per hour) into Herrn- see that the scatter of residual values is relatively
stein’s (1970) equation to obtain a predicted even across the range of reinforcer rates. More data
response rate for each of the measured reinforcer may be needed to assess the assumptions of least-
rates. The computer program adjusts the parameter squares regression, but the analysis suggests that
values to minimize the sum-of-squared residuals. there are no obvious departures from the
Table 10.3 shows the best-fitting parameter values of assumptions.
k and re obtained via least-squares regression. The
predicted data based on these values are show in
Additional Questions When
column C.
Evaluating Nonlinear Regression
The first index of the model’s accuracy is the R2
(by convention, results of nonlinear regression are Do the Parameters Make Sense?
reported using R, and results of linear regression are To answer this question, as in linear regression, one
reported using r). Similar to what was depicted in has to know what the parameters mean and in what
Figure 10.5, the R2 compares the vertical distances of units they are measured. If researchers obtain an
the data to the curve with the vertical distances of the estimate of k of 1,000, and they have measured
data to a horizontal line at the mean. More specifi- response rates in units of words per minute, this
cally, a ratio of the sum-of-squared residuals to the estimate of asymptotic response rates would not be
total sum-of-squares is calculated. In our example, feasible. Moreover, even if the maximum observed
SSREG is 18.43, and SSTOT is 478.43. Subtracting the response rate were 100 or so, one should be suspi-
ratio of SSREG from SSTOT from 1 yields an R2 of .96. cious of such a parameter value because it exceeds
A residual analysis is also performed to assess the observations by orders of magnitude. That is, if the
whether there are systematic departures from parameter value exceeds what is reasonable on the
homoscedasticity. The R2 tells only part of the story. basis of observed values, the researchers should
Again, one could perform several different kinds of think carefully about the obtained parameter value.
residual analysis (see earlier), but for simplicity the It is indeed possible to obtain very large ks, but this
238
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
is usually because the researcher has not collected typically accounts for more than 90% of the variance
enough data under high reinforcer rates (i.e., in in response rates under VI schedules in laboratory,
which response rate is close to asymptote). In applied, and naturalistic settings (see de Villiers &
choosing the VI schedules, the researchers have not Herrnstein, 1976; Fisher & Mazur, 1997; McDowell,
sampled the parameter space effectively. For exam- 1988; and Williams, 1988, for reviews), there are
ple, one would say the parameter space has not been some reasons to think it may not be wholly correct.
sampled adequately if only the three lowest rein- As noted by Shull (1991), “The fact that an equation
forcer rates shown in Figure 10.9 (three leftmost of particular form describes a set of data well does
data points) were experimentally arranged. Even not mean that the assumptions that gave rise to the
visually, using these three data points, the research- equation are supported” (p. 247). The assumptions
ers would not be able to make a reasonable guess underlying Herrnstein’s (1970) hyperbola have been
about the asymptote of the function. Their guess examined in terms of how the parameters change
might be large (and uncertain). Similarly, the esti- with manipulations of environmental variables (see
mate produced by the regression program might be Dallery & Soto, 2004, for a review). According to
large (and uncertain, as measured by the standard the equation, re should vary directly with changes in
error of the parameter estimate, for example; Motul- the extraneous reinforcer rate, and this prediction
sky & Christopoulos, 2004). Given that the parame- has some empirical support (Belke & Heyman,
ter space has been adequately sampled, as shown in 1994; Bradshaw, Szabadi, & Bevan, 1976). Other
Figure 10.9, the obtained k of 35.41 responses per studies, however, have questioned this prediction
minute makes sense given the observations. The (Bradshaw, 1977; Soto, McDowell, & Dallery, 2005;
obtained re of 22.57 reinforcers per hour also makes White, McLean, & Aldiss, 1986). Similarly, there are
sense. Although it may seem odd that extraneous conflicting findings for the prediction of a constant k
reinforcers are measured in units of the delivered as reinforcer properties are manipulated. Recall that
reinforcer, eye contacts, the value of eye contacts is this is required by Herrnstein’s (1974) theory: Rein-
plausible given the observed data. forcement alters the distribution of a constant
amount of behavior among the available alternatives
Is There Error in the Values of the (de Villiers & Herrnstein, 1976, p. 1151; Herrnstein,
Independent Variable? 1974). Thus, the evidence has suggested some limita-
As noted for linear regression, when one performs tions to the assumptions underlying the hyperbola,
nonlinear regression analysis, one is predicting the and these limitations are independent of the criteria
values of the dependent variable at specific values of by which one evaluates the appropriateness of the
the independent variable. We do not go into this model in describing a particular data set. Moreover,
issue in detail because the arguments are the same as other models account for the same hyperbolic rela-
outlined in the Evaluating Linear Models: Linear tions as Herrnstein’s model and predict the condi-
Regression section. Briefly, recall that if there is a lot tions under which k should remain constant or vary
of variability in the independent variable, one can- (Dallery, McDowell, & Lancaster; 2000; Dallery et al.,
not answer with precision the question of how the 2004; McDowell, 1980; McDowell & Dallery, 1999).
dependent variable changes with changes in the There is another reason to question the validity
independent variable. of Herrnstein’s equation. Recall that the original,
strict matching equation, on which Herrnstein’s
Is Another Model More Appropriate? equation is based, was modified to account for bias
As with linear models, when evaluating a nonlinear and insensitivity. If bias and insensitivity are ubiqui-
model, one should consider whether there is evi- tous in concurrent schedules, they should be incor-
dence that questions the assumptions underlying porated into Herrnstein’s (1974) account of
the model or evidence for other candidate models single-alternative responding. We do not develop
that seek to describe the same environment– the equation that incorporates these parameters into
behavior relation. For example, although Equation 7 Herrnstein’s original equation (see McDowell, 2005,
239
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
for the derivation). As one might imagine, incorpo- thus any improvement in goodness of fit relative to
rating these parameters makes the equation appear another model must be considered in the context of
more complicated. However, the equation describes this increase in the number of parameters.
the same hyperbolic shape, the same environment– Before comparing models using either the sum-of-
behavior relation. Evaluating the equation involves squares F test or AIC, researchers must determine
the same steps outlined earlier. Indeed, in compar- whether both models provide a reasonable and accu-
ing the family of equations that make up the so-called rate account of the data: (a) Do they describe the data
modern theory of matching with the classic theory well (i.e., high r2 or R2), (b) are the residuals ran-
of matching, McDowell and colleagues (Dallery domly distributed, and (c) are the parameter values
et al., 2005; McDowell, 2005) made extensive use sensible? If the answer to each of these three ques-
of residual analysis to detect subtle, yet important tions is yes, then researchers can proceed to model
deviations from the original, classic theory of match- comparison with the F test or AIC. If the answer to
ing. Although Dallery et al. (2005) and McDowell one of the questions is no, then researchers must
(2005) concluded that the modern theory repre- either resolve the issue (e.g., if nonsensible parame-
sented a superior theory, alternative quantitative ters are the result of experimental error) or choose
accounts also provide useful, general, and informa- between the models on the basis of the residuals or
tive descriptions of the same environment–behavior goodness of fit (i.e., if one model has nonrandom
relations (e.g., Killeen & Sitomer, 2003; Rachlin, residuals or does a poor job of describing the data,
Battalio, Kagel, & Green, 1981). then choose the other model). In addition, research-
ers must ensure that all datasets are expressed in the
same units. They need to compare apples to apples. If
Model Comparison
the dependent variable in one data set is expressed in
Evaluating the accuracy and generality of a single responses per second and in another it is expressed in
quantitative theory is an important first step in responses per minute, the sum-of-squared deviations
describing environment–behavior relations. Com- will be much larger in the former than in the latter. A
paring the model with other accounts is also a criti- comparison with different units is not appropriate.
cal feature of the scientific process. Indeed, making The researcher must either reexpress the data so that
distinctions between models can help further isolate the units are identical (e.g., multiply or divide by 60
controlling variables, develop critical empirical tests in the previous example) or find a way to standardize,
to distinguish the models (Mazur, 2001), and poten- or normalize, the data before making the comparison.
tially lead to new insights about the processes The appropriate model comparison test depends
responsible for behavior change in applied settings on whether the models are nested. Models are
(Nevin & Grace, 2000). In the next section, we nested when one model is a simpler case of the other
present two techniques to compare models. One model. For instance, the original (Equation 2) and
technique, the extra sum-of-squares F test (Motul- generalized matching (Equation 5) equations are
sky & Christopoulos, 2004), is based on conven- nested models because Equation 4 can be reduced to
tional inferential statistics, and the other technique, Equation 2 when a and b are equal to 1. If the mod-
Akaike’s information criterion (AIC; Akaike, 1974; els are nested, they can be compared with an extra
Motulsky & Christopoulos, 2004), is based primar- sum-of-squares F test or with the AIC. If they are
ily on information theory (Burnham & Anderson, not nested, they can only be compared using AIC
2002). The objective of each technique is to provide (or an alternative statistical approach using paramet-
a quantitative basis on which to judge whether one ric or nonparametric methods).
model provides a better account of the data than
does another. Both methods evaluate goodness of fit Extra Sum-of-Squares F Test to
relative to model complexity (i.e., the number of Compare Nested Models
parameters). This is because the more parameters a Let us use strict matching and generalized matching
model has, the better it will describe the data, and as an example of model comparison using the data in
240
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Table 10.2. Perfect matching is described by Equa- Table 10.4

tion 5 with a and b set to 1. Generalized matching is
also described by Equation 5, but the parameters are Obtained Logged Reinforcer and Response
free to vary. Assume that both models describe the Ratios and Predicted Log Response Ratios and
data well, have randomly distributed residuals, and Corresponding Regression Calculations for Strict
have sensible parameter values. The logic of the Matching and Generalized Matching
extra sum-of-squares F test is relatively simple.
Goodness-of-fit is being compared along with the D: predicted
complexity of each model. The goodness-of-fit is C: predicted log log rsp ratio
measured by the sum-of-squared residuals. The com- A: log B: log rsp rsp ratio from from general-
plexity is measured by the number of parameters in rnf ratio ratio strict matching ized matching
each model. More parameters that are free to vary 0.11 0.17 0.11 0.19
mean more complexity. The rule of parsimony −0.37 0.07 −0.37 −0.19
asserts that simpler models should be preferred, so 0.49 0.61 0.49 0.49
any increase in complexity should be more than off- −0.86 −0.66 −0.86 −0.58
0.70 0.63 0.70 0.66
set by an increase in accuracy. Before going into the
−0.17 −0.21 −0.17 −0.03
specific calculations, the F ratio compares accuracy 0.63 0.57 0.63 0.60
(sum-of-squares) and complexity (parameters). If −0.22 −0.06 −0.22 −0.07
the calculated F ratio is high enough (and the corre- 0.25 0.24 0.25 0.30
sponding p value is low enough), then the more −0.67 −0.44 −0.67 −0.43
complicated model can be accepted (generalized
Regression calculations
matching, in which the parameters are free to vary).
SSTOT 1.78
If not, the simpler model is accepted (strict match- SSREG 0.34 0.13
ing, in which the parameters are not free to vary). r2 0.81 0.93
To calculate the extra sum-of-squares F test, one
Note. Columns A and B show the logged response and
begins by calculating the sum-of-squared residuals reinforcer ratios, and columns C and D show the predicted
for each model. The sum-of-squared residuals for log response ratios from strict matching (Equation 5 with
strict matching in this case is 0.34; for generalized a and b equal to 1) and generalized matching (Equation 5)
matching, it is 0.13 (see Table 10.4). A smaller sum- with a = 0.80 and log b = 0.10 and their respective regres-
sion calculations. rnf = reinforcer; rsp = response.
of-squares is expected for generalized matching, and
in general for models with more parameters that are
free to vary. Next, the degrees of freedom for each sum-of-squares. Similarly, the relative difference in
model are calculated as the number of data points degrees of freedom is calculated by subtracting the
minus the number of parameters. For strict match- alternative hypothesis degrees of freedom from the
ing, the degrees of freedom are 10 (10 data points simpler hypothesis degrees of freedom, and then
minus 0 free parameters); for generalized matching, dividing that difference by the alternative hypothesis
the degrees of freedom are 8 (10 data points minus 2 degrees of freedom. These numbers are summarized
parameters). Now, the relative difference in sums- in Table 10.5. The relative difference in the sums-of-
of-squares and the relative difference in degrees of squares is 1.62 ([0.34 − 0.13]/0.13) and the relative
freedom are calculated. The relative difference in difference in the degrees of freedom is 0.25 ([10 −
sums-of-squares is calculated by subtracting the 8]/8). The F ratio is the ratio of the relative differ-
alternative hypothesis sum-of-squares from the ence in the sum-of-squares to the relative difference
simpler hypothesis4 sum-of-squares, and then divid- in the degrees of freedom, and it is a measure of how
ing that difference by the alternative hypothesis much better the alternative model describes the data
In model comparison, we prefer the term simpler hypothesis to null hypothesis. We agree with Rodgers (2010) that the term null hypothesis has too
4
much baggage and often refers to no effect or to chance processes. The modeling approach does not posit a null (or nil) effect; rather, it entails a spe-
cific, but simpler, hypothesis.
241
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
Table 10.5 or nonnested models. We use our previous example

to illustrate the use of AIC for model comparison.
Calculations Involved in the Extra Sum-of-Squares The logic is the same as in the extra sum-of-squares
F Test for Model Comparison of the Descriptions F test: Accuracy and complexity are compared.
of Strict Matching and Generalized Matching Increases in complexity must be offset by increases
Accounts of the Data in Table 10.4 in accuracy. The first step in using AIC for model
comparison is to calculate the AIC for each model
Model SS df using the equation
Simpler hypothesis (strict matching) 0.34 10.00
Alternative hypothesis (generalized 0.13 8.00
 SS 
matching) AIC = N ln  REG  + 2K, (8)
Difference 0.21 2.00  N 
Relative difference 1.62 0.25
Ratio (F ) 6.48 where N is the number of data points, K is the num-
p .02 ber of parameters in the model plus 1, and SSREG is
the model’s residual sum-of-squares. Consider Equa-
Note. Strict matching = Equation 5 with a and b equal to
1; generalized matching = Equation 5. tion 8. The AIC value depends both on how well the
model describes the data (how small the residual
relative to how many more degrees of freedom the sum-of-squares is; SSREG) and on how many parame-
alternative model uses. In this case, the value is 6.48 ters the model has (K). When comparing two mod-
(1.62/0.25) and the associated p value is .02. The p els, the model more likely to be correct is the one
value can be found in an F ratio statistics table or by with the lower AIC value because a lower AIC value
using an online calculator. Remember to use the represents a better balance between goodness of fit
correct degrees of freedom when looking up or cal- and number of parameters. When the number of
culating a p value. In this case, the degree of free- data points is small relative to the number of param-
dom for the numerator is 2 (the difference in eters, which may often be the case in behavioral
degrees of freedom of our simpler and alternative experiments, AIC values will be small and should be
hypotheses, 10 and 8, respectively), and the degree corrected (Motulsky & Christopoulos, 2004). The
of freedom for the denominator is 8 (the alternative corrected AIC formula is
hypothesis degree of freedom). Typically, the crite-
rion alpha value is set at .05, so in this case, one 2K ( K + 1)
AICC = AIC + . (9)
would reject the simpler hypothesis that the simpler N − K −1
model, strict matching, provides a better account
of the data. In the previous example, the residual sum-of-
squares was 0.34 for strict matching and 0.13 for
Akaike’s Information Criterion to generalized matching. The number of data points is
Compare Nested Models the same for each model (10), and K is 1 for strict
Another technique used for comparing models is matching and 3 for generalized matching. Thus, one
AIC. As noted earlier, AIC is based on information AIC value for each model can be calculated. The
theory. The basis of the AIC calculation is beyond the corrected AIC for strict matching is −31.31, and the
scope of this chapter (for a brief discussion of the AIC for generalized matching is −33.43. Because
basis of AIC, see Motulsky & Christopoulos, 2004; the AIC value for generalized matching (designated
for a more detailed discussion, see Burnham & as AICB) is lower than the AIC value for strict
Anderson, 2002). The AIC calculation allows one to matching (designated as AICA), one can say that
determine which model is more likely to be correct generalized matching is more likely to be correct.
(the one with the lower AIC value) and how much Some authors just present the individual AICs to
more likely it is to be correct. Unlike the extra sum- make comparisons between models; the model with
of-squares F test, AIC can be used to compare nested a smaller AIC is preferred. Another approach is to
242
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
calculate the probability that one model is more before subjecting them to the AIC. The sum-of-
likely by using the following equation: squares F test cannot be used with nonnested mod-
els. We do not provide an example of using the AIC
−0.5( AICB − AIC A ) with nonnested models because the calculations
e
Probability = −0.5( AICB − AIC A )
. (10) described in the previous section are exactly the
1+ e
same whether AIC is used for comparing nested or
If the corrected AIC for generalized matching, nonnested models. When evaluating the probability
AICB (−33.43), and the corrected AIC for strict that the more likely model is correct relative to
matching, AICA (−31.31), are substituted into Equa- another model (Equation 10), subtract the AIC for
tion 10, the probability that generalized matching is the less likely model (i.e., the model with the larger
correct is found to be approximately .74. Alterna- AIC value) from the AIC for the more likely model
tively, it can be said that the chance that generalized (i.e., the model with the smaller AIC value).
matching is correct is 74%. Note, the difference One area in which AIC could be applied to model
could be calculated the other way and would result comparison is in research on temporal discounting.
in a probability of approximately .26 that strict Temporal discounting equations describe the rate at
matching is correct relative to generalized matching. which a reinforcer loses value as the delay to its
Finally, an evidence ratio can be calculated that receipt increases (Critchfield & Kollins, 2001;
quantifies how much more likely one model is to be Mazur, 1987). Several nonnested quantitative mod-
correct relative to another model by dividing the els (e.g., exponential decay, hyperbolic decay) have
probability that one model is correct by the proba- been proposed that account for high proportions of
bility that the other model is correct. For the current the variance in responding (for a recent comparison
example, the probability that generalized matching of four discounting models, see McKerchar et al.,
is correct (.74) could be divided by the probability 2009). Assuming that residual analysis does not
that strict matching is correct (.26), resulting in an reveal departures from randomness, nonnested AIC
evidence ratio of 2.85, which means that generalized analysis may be useful in making comparisons
matching is 2.85 times more likely to be correct between these models.
than strict matching. These calculations are summa-
rized in Table 10.6. Model Comparison to Evaluate Whether
an Intervention Changed Behavior
Akaike’s Information Criterion to The model comparison approach can also be used to
Compare Nonnested Models assess whether some intervention (e.g., an environ-
When comparing models that are not nested, when mental or pharmacological intervention) produced a
neither model is a simpler version of the other significant change in behavior. For example, the
model, researchers must use AIC. They must scruti- model comparison could be used to evaluate
nize each model (i.e., r2, residuals, parameters) whether a drug or some other intervention produced
Table 10.6
Calculations Involved in Akaike’s Information Criterion for Comparison of Strict (Equation 5 With A and
B Equal to 1) and Generalized Matching (Equation 5) Accounts of the Data in Table 4
Akaike’s
information
Model SSREG N Parameters criterion Probability Evidence ratio
Simpler hypothesis (strict matching) 0.34 10 0 −31.31 .26
Alternative hypothesis (generalized matching) 0.13 10 2 −33.43 .74 2.85
243
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
a change in the dose–response curve in a behavioral The central question in model comparison is, did
pharmacology experiment. Similarly, one could ask an intervention produce a significant change in the
whether a curve, say a demand curve (which relates function describing the relation between the inde-
consumption to response requirement), changed as pendent and dependent variables? Consider a case in
a function of a manipulation such as income (num- which the researcher has collected two datasets, one
ber of reinforcers available per day) or degree of obtained before and one obtained after an interven-
food deprivation. Here, the comparison is not a tion. The model comparison approach asks whether
change in the value of a single data point, as in an separate fits of an equation to the data, pre- and pos-
analysis of variance comparing means; the compari- tintervention, for example, are better than a single fit
son is between two models (i.e., equations) of the of the equation to both datasets. If the separate fits
relation across many data points (e.g., comparing a are not better, it suggests that the intervention had
fit of the generalized matching law under control no effect on behavior. Note that the basic logic of all
and drug conditions). model comparison is entailed by this procedure: A
One could also assess whether a particular param- more complicated model (one equation fitted sepa-
eter value changed as a result of some intervention. rately to each dataset) is being compared with a
For example, one could evaluate whether the sensi- more parsimonious model (one equation fitted to
tivity parameter in the generalized matching equa- both datasets). An example of this comparison is
tion varies as a function of some manipulation such shown graphically in Figure 10.11. The left panel
as drug administration. Similarly, researchers in shows the alternative hypothesis, and the right panel
behavioral pharmacology may be interested in shows the simpler hypothesis (more details about
whether a drug alters a parameter value in a quantita- the figure are discussed later). Either the F test or
tive model (e.g., Pitts & Febbo, 2004). Such changes the AIC can be used to determine whether a particu-
may indicate that certain behavioral mechanisms lar intervention changed behavior, but we present
(e.g., changes in sensitivity to reinforcer amount or only the F test. (For the AIC, one would need to cal-
delay) are responsible for a particular drug effect culate the sum-of-squared residuals, N and K, for
(e.g., an increase in impulsive choice). In the interest each model. The calculations for the AIC were given
of brevity, we provide a detailed example of model earlier.) Motulsky and Christopoulos (2004) pro-
comparison and only the general logic and procedure vided a succinct discussion of some factors that may
for parameter comparison. lead a researcher to favor one test over the other.
Figure 10.11. A model comparison approach for nested models. The left
panel shows the more complicated alternative hypothesis. The alternative
hypothesis is that separate fits of the equation provide a better account of the
data. The more parsimonious, simpler hypothesis, shown in the right panel,
is that one equation can account for all of the data. The data points represent
the logged response and reinforcer ratios presented in Table 10.6.
244
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
In this example, assume we have conducted an each data set). Thus, we also fit the generalized
experiment with pigeons in which we have manipu- matching equation to each data set separately. The
lated the presence of discriminative stimuli. The dis- alternative hypothesis is shown graphically by the
criminative stimuli, such as different colored lights, dashed and dotted lines in the left panel of Figure
signal the availability of reinforcement at each of the 10.11. The parameter values for our dataset without
two alternatives. If behavior is more sensitive to discriminative stimuli are a = 0.80 and log b =
changes in reinforcer rates when discriminative 0.10; the parameter values for our dataset with dis-
stimuli are used, we can make a prediction: A fit of criminative stimuli are a = 1.28 and log b = 0.11.
the generalized matching equation with discrimina- As always, before moving on to model compari-
tive stimuli will differ from the fit of the generalized son using an F test or the AIC, one must evaluate
matching equation without discriminative stimuli. each model separately in terms of visual inspection,
The first dataset, obtained without discriminative R2, residuals, and the other criteria listed earlier. It
stimuli, and the second dataset, obtained with dis- would not be appropriate to perform model compari-
criminative stimuli, are presented in Table 10.7 and son if one of the models produces nonrandom resid-
plotted in the left and right panels of Figure 10.11. uals or unrealistic parameter values (e.g., Dallery et al.,
The simpler hypothesis is that the discriminative 2005). Because no problems occurred with either the
stimuli have no effect on behavior and that a single simpler or the alternative models (not shown), we
fit describes all the data. Thus, we fit the generalized can move on to statistical comparison. We calculate
matching equation to all the data pooled together, the residual sum-of-squares for the two-parameter
which is also called global fitting. The global fit is (one fit with two parameters) global model and the
indicated by the solid line in the right panel of Fig- more complicated four-parameter (two fits each with
ure 10.11. The values of the parameters for global fit two parameters) alternative model. The total residual
are a = 1.09 and log b = 0.09. The alternative sum-of-squares for the simpler hypothesis, that a
hypothesis is that the discriminative stimuli did single global fit provides a better account of the data,
indeed change behavior and that separate fits of the is 0.71. The alternative hypothesis, that two separate
generalized matching equation are better (one fit for fits of the equation provide a better account of the
data, has a sum of squares of 0.34. The value of 0.34
Table 10.7 was calculated by summing the individual fit sums-
of-squares of 0.13 and 0.21. Next, we calculate the
Log Response and Reinforcer Ratios Obtained degrees of freedom for each model. The degrees of
Under Conditions With and Without freedom for the global fit is 18 (20 data points minus
Discriminative Stimuli two parameters); for the alternative hypothesis, it is
16 (20 data points minus four parameters—i.e., two
Without discriminative With discriminative parameters for each fit). As before, we can now cal-
stimuli stimuli culate the relative difference in the sums-of-squares
Log rnf ratio Log rsp ratio Log rnf ratio Log rsp ratio ([0.71 − 0.34]/0.34 = 1.09) and degrees of freedom
0.11 0.17 −0.3 −0.22 ([18 − 16]/16 = 0.125) to allow the calculation of
−0.37 0.07 −0.25 −0.12 an F ratio. The F ratio is 8.72 (1.09/0.125), with a
0.49 0.61 1 1.6 numerator degree of freedom of 2 (the simpler
−0.86 −0.66 −0.95 −0.99 hypothesis degree of freedom minus the alternative
0.7 0.63 0.43 0.62 hypothesis degree of freedom) and a denominator
−0.17 −0.21 −0.84 −1.2
degree of freedom of 16 (the alternative hypothesis
0.63 0.57 −0.77 −0.83
0.43 0.43
degree of freedom), which has a p value of approxi-
−0.22 −0.06
0.25 0.24 mately .003. These calculations are summarized in
−0.85 −0.85
−0.67 −0.44 0.06 0.08 Table 10.8. Assuming a threshold value of .05, we
reject the simpler hypothesis and conclude that sepa-
Note. rnf = reinforcer; rsp = response. rate fits of the equation provide a better account of
245
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
Table 10.8 each dataset. Because a parameter value must be

shared in the simpler model, this technique requires
Extra Sum-of-Squares F Test Comparison of a a program that can calculate least-squares regression
Single Global Fit of the Generalized Matching with shared parameters, such as GraphPad Prism
Equation (Equation 5) With Both Data Sets in Table (GraphPad Software, Inc., La Jolla, CA) or Microsoft
10.7 Versus Two Separate Fits of the Generalized Excel. Finally, as with the previous model compari-
Matching Equation, One for Each Data Set son example, compare the sum-of-squared residuals
and the complexity (degrees of freedom) obtained
Model SS df for both models, calculate the F ratio, and evaluate
Simpler hypothesis (one global fit) 0.71 18.000 significance using an F table. Following the same
Alternative hypothesis (separate fit 0.34 16.000 procedure outlined earlier, the AIC could also be
for each data set) used to evaluate whether an intervention changed
Difference 0.37 2.000
Relative difference 1.09 0.125
the value of a parameter.
Ratio (F ) 8.72
p .003
Conclusion
In this chapter, we introduced and developed the
the data than a single global fit. We can also see that mechanics of how quantitative models are
the slope increased when discriminative stimuli were assessed and compared. This knowledge should
present and, as suggested by the difference in the be coupled with an appreciation of historical,
intercepts, perhaps there were some small biases for conceptual, and philosophical issues concerning
certain colors over others. quantitative modeling in science in general and
A model comparison approach could also be behavioral science in particular (e.g., Critchfield
used to assess whether a particular parameter value & Reed, 2009; Kline, 1959; Marr, 1989; Moore,
changed as a result of some intervention (for a more 2008, 2010; Nevin, 1984; Rodgers, 2010; Smith,
in-depth discussion of this technique, see Chapter G 1990). Space limitations preclude us from dis-
of Motulsky & Christopoulos, 2004). In the exam- cussing these philosophical issues beyond
ple just described using discriminative stimuli, we acknowledging them, but we hope the reader is
could have asked whether just the sensitivity param- now better equipped to consider quantitative
eter changed as opposed to the entire function. The modeling from a conceptual and philosophical
general procedure is similar to what we have perspective and thereby bring quantitative model-
described in this section. Construct two models, a ing into the broader context of how science is
simpler and an alternative hypothesis, and ask conducted. Indeed, just as models of phenomena
whether the residuals are random, whether the in other sciences have evolved on the basis of new
parameters are sensible, and so on. In constructing empirical evidence, models in behavioral science
the simpler model, again use a global fitting proce- have and will evolve with the accumulation of
dure, but with one key difference: Fit the equation empirical evidence. We have emphasized that the
to each dataset separately with the constraint that precise description and critical evaluation of
the value of the parameter of interest is shared behavior’s controlling variables are at the heart of
between the two fits (e.g., the sensitivity parameter, quantitative analysis in behavioral science. Evalu-
a), and the other parameters are allowed to vary ating quantitative models requires clear and criti-
across the two fits. Least-squares regression in this cal thinking and some specialized tools such as
case involves finding one value of the shared param- residual analysis and model comparison
eter and separate values of the other parameters, one approaches. The benefits of quantitative models
for each data set, that minimize the total residual are considerable; they yield new insights into the
sum-of-squares. As before, the alternative model causes of behavior and allow for a high degree of
involves completely separate fits of the equation to precision, falsifiability, and generality.
246
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
References Conger, R., & Killeen, P. (1974). Use of concurrent

operants in small group research: A demonstration.
Akaike, H. (1974). A new look at the statistical model Pacific Sociological Review, 17, 399–416.
identification. IEEE Transactions on Automatic Control,
19, 716–723. doi:10.1109/TAC.1974.1100705 Cook, D. R., & Weisberg, S. (1982). Residuals and influ-
ence in regression. New York, NY: Chapman & Hall.
Anscombe, F. J. (1973). Graphs in statistical analysis.
American Statistician, 27, 17–21. doi:10.2307/2682899 Critchfield, T. S., & Kollins, S. H. (2001). Temporal dis-
Baum, W. M. (1974). On two types of deviation from the counting: Basic research and the analysis of socially
matching law: Bias and undermatching. Journal of important behavior. Journal of Applied Behavior
the Experimental Analysis of Behavior, 22, 231–242. Analysis, 34, 101–122. doi:10.1901/jaba.2001.34-101
doi:10.1901/jeab.1974.22-231 Critchfield, T. S., & Reed, D. D. (2009). What are we
Baum, W. M. (1979). Matching, undermatching, and doing when we translate from quantitative models?
overmatching in studies of choice. Journal of the Behavior Analyst, 32, 339–362.
Experimental Analysis of Behavior, 32, 269–281. Dallery, J., McDowell, J. J., & Lancaster, J. S. (2000).
doi:10.1901/jeab.1979.32-269 Falsification of matching theory’s account of single-
Beardsley, S. D., & McDowell, J. J. (1992). Application alternative responding: Herrnstein’s k varies with
of Herrnstein’s hyperbola to time allocation of natu- sucrose concentration. Journal of the Experimental
ralistic human behavior maintained by naturalistic Analysis of Behavior, 73, 23–43. doi:10.1901/jeab.
social reinforcement. Journal of the Experimental 2000.73-23
Analysis of Behavior, 57, 177–185. doi:10.1901/ Dallery, J., McDowell, J. J., & Soto, P. L. (2004). The
jeab.1992.57-177 measurement and functional properties of reinforcer
Belke, T. W., & Heyman, G. M. (1994). Increasing and value in single-alternative responding: A test of linear
signaling background reinforcement: Effect on the system theory. Psychological Record, 54, 45–65.
foreground response-reinforcer relation. Journal of
Dallery, J., & Raiff, B. R. (2007). Delay discounting
predicts cigarette smoking in a laboratory model of
doi:10.1901/jeab.1994.61-65
abstinence reinforcement. Psychopharmacology, 190,
Bell, E. T. (1945). The development of mathematics. New 485–496. doi:10.1007/s00213-006-0627-5
York, NY: McGraw-Hill.
Dallery, J., & Soto, P. (2004). Herrnstein’s hyperbola
Borrero, J. C., Crisolo, S. S., Tu, Q., Rieland, W. A., Ross, and behavioral pharmacology: Review and cri-
N. A., Francisco, M. T., & Yamamoto, K. Y. (2007). tique. Behavioural Pharmacology, 15, 443–459.
An application of the matching law to social dynam- doi:10.1097/00008877-200411000-00001
ics. Journal of Applied Behavior Analysis, 40, 589–601.
Dallery, J., Soto, P. L., & McDowell, J. J. (2005). A test of
Borrero, J. C., & Vollmer, T. R. (2002). An application of the formal and modern theories of matching. Journal
the matching law to severe problem behavior. Journal of the Experimental Analysis of Behavior, 84, 129–145.
of Applied Behavior Analysis, 35, 13–27. doi:10.1901/ doi:10.1901/jeab.2005.108-04
jaba.2002.35-13
de Villiers, P. A., & Herrnstein, R. J. (1976). Toward a
Brace, R. A. (1977). Fitting straight lines to experimental
law of response strength. Psychological Bulletin, 83,
data. American Journal of Physiology: Physiology and
1131–1153. doi:10.1037/0033-2909.83.6.1131
Comparative Physiology, 233, 94–99.
Bradshaw, C. M. (1977). Suppression of response rates in Elliffe, D., Davison, M., & Landon, J. (2008). Relative
variable-interval schedules by a concurrent schedule reinforcer rates and magnitudes do not control
of reinforcement. British Journal of Psychology, 68, concurrent choice independently. Journal of the
473–480. doi:10.1111/j.2044-8295.1977.tb01617.x Experimental Analysis of Behavior, 90, 169–185.
doi:10.1901/jeab.2008.90-169
Bradshaw, C. M., Szabadi, E., & Bevan, P. (1976). Human
variable-interval performance. Psychological Reports, Fisher, W. W., & Mazur, J. E. (1997). Basic and applied
38, 881–882. doi:10.2466/pr0.1976.38.3.881 research on choice responding. Journal of Applied
Burnham, K. P., & Anderson, D. R. (2002). Model selec- 1997.30-387
tion and multimodal inference: A practical information-
theoretic approach (2nd ed.). New York, NY: Fuqua, R. W. (1984). Comments on the applied relevance
Springer-Verlag. of the matching law. Journal of Applied Behavior
Catania, A. C., & Reynolds, G. S. (1968). A quantita-
tive analysis of the responding maintained by Herrnstein, R. J. (1961). Relative and absolute strength of
interval schedules of reinforcement. Journal of the response as a function of frequency of reinforcement.
Experimental Analysis of Behavior, 11, 327–383. Journal of the Experimental Analysis of Behavior, 4,
doi:10.1901/jeab.1968.11-s327 267–272. doi:10.1901/jeab.1961.4-267
247
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dallery and Soto
Herrnstein, R. J. (1970). On the law of effect. Journal of McDowell, J. J. (1989). Two modern developments in
the Experimental Analysis of Behavior, 13, 243–266. matching theory. Behavior Analyst, 12, 153–166.
doi:10.1901/jeab.1970.13-243 McDowell, J. J. (2004). A computational model of selec-
Herrnstein, R. J. (1974). Formal properties of the match- tion by consequences. Journal of the Experimental
ing law. Journal of the Experimental Analysis of Analysis of Behavior, 81, 297–317. doi:10.1901/jeab.
Behavior, 21, 159–164. doi:10.1901/jeab.1974.21-159 2004.81-297
Houston, A. (1986). The matching law applies to wagtails’ McDowell, J. J. (2005). On the classic and modern theories
foraging in the wild. Journal of the Experimental Analysis of matching. Journal of the Experimental Analysis of
of Behavior, 45, 15–18. doi:10.1901/jeab.1986.45-15 Behavior, 84, 111–127. doi:10.1901/jeab.2005.59-04
Killeen, P. R., & Sitomer, M. T. (2003). MPR. Behavioural McDowell, J. J., & Dallery, J. (1999). Falsification
Processes, 62, 49–64. doi:10.1016/S0376-6357(03) of matching theory: Changes in the asymptote
00017-2 of Herrnstein’s hyperbola as a function of water
deprivation. Journal of the Experimental Analysis of
Kline, M. (1959). Mathematics and the physical world. Behavior, 72, 251–268. doi:10.1901/jeab.1999.72-251
New York, NY: Dover.
McDowell, J. J., & Kessel, R. (1979). A multivariate rate
Lattal, K. A. (2001). The human side of animal behavior. equation for variable-interval performance. Journal
Behavior Analyst, 24, 147–161. of the Experimental Analysis of Behavior, 31, 267–283.
Lau, B., & Glimcher, P. W. (2005). Dynamic response- doi:10.1901/jeab.1981.36-9
by-response models of matching behavior in rhesus McKerchar, T. L., Green, L., Myerson, J., Pickford, T. S.,
monkeys. Journal of the Experimental Analysis of Hill, J. C., & Stout, S. C. (2009). A comparison
Behavior, 84, 555–579. doi:10.1901/jeab.2005.110-04 of four models of delay discounting in humans.
Marr, M. J. (1989). Some remarks on the quantitative Behavioural Processes, 81, 256–259. doi:10.1016/
analysis of behavior. Behavior Analyst, 12, 143–151. j.beproc.2008.12.017
Martens, B. K., & Houk, J. L. (1989). The application Moore, J. (2008). A critical appraisal of contemporary
of Herrnstein’s law of effect to disruptive and on- approaches in the quantitative analysis of behavior.
task behavior of a retarded adolescent girl. Journal Psychological Record, 58, 641–664.
of the Experimental Analysis of Behavior, 51, 17–27. Moore, J. (2010). Philosophy of science, with special
doi:10.1901/jeab.1989.51-17 consideration given to behaviorism as the philosophy
of the science of behavior. Psychological Record, 60,
Mazur, J. E. (1987). An adjusting amount procedure for
137–150.
studying delayed reinforcement. In M. L. Commons,
J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Motulsky, H., & Christopoulos, A. (2004). Fitting mod-
Quantitative analyses of behavior: The effects of delay els to biological data using linear and non-linear
and of intervening events on reinforcement value regression: A practical guide to curve fitting. Oxford,
(pp. 55–73). Hillsdale, NJ: Erlbaum. England: Oxford University Press.
Mazur, J. E. (2001). Hyperbolic value addition and gen- Murray, L. K., & Kollins, S. H. (2000). Effects of methyl-
eral models of animal choice. Psychological Review, phenidate on sensitivity to reinforcement in children
108, 96–112. doi:10.1037/0033-295X.108.1.96 diagnosed with attention deficit hyperactivity disor-
der: An application of the matching law. Journal of
Mazur, J. E. (2006). Mathematical models and the Applied Behavior Analysis, 33, 573–591. doi:10.1901/
experimental analysis of behavior. Journal of the jaba.2000.33-573
doi:10.1901/jeab.2006.65-05 Nevin, J. A. (1984). Quantitative analysis. Journal of
McDowell, J. J. (1980). An analytic comparison of doi:10.1901/jeab.1984.42-421
Herrnstein’s equations and a multivariate rate equa-
tion. Journal of the Experimental Analysis of Behavior, Nevin, J. A., & Grace, R. C. (2000). Behavioral momentum
33, 397–408. doi:10.1901/jeab.1980.33-397 and the law of effect. Behavioral and Brain Sciences, 23,
73–90. doi:10.1017/S0140525X00002405
McDowell, J. J. (1982). The importance of Herrnstein’s
Parsonson, B. S., & Baer, D. M. (1992). The visual analy-
mathematical statement of the law of effect for
sis of data, and current research into the stimuli
behavior therapy. American Psychologist, 37, 771–779.
controlling it. In T. R. Kratochwill & J. R. Levin
doi:10.1037/0003-066X.37.7.771
(Eds.), Single-case research design and analysis: New
McDowell, J. J. (1986). On the falsifiability of match- directions for psychology and education (pp. 15–40).
ing theory. Journal of the Experimental Analysis of Mahwah, NJ: Erlbaum.
Behavior, 45, 63–74. doi:10.1901/jeab.1986.45-63 Perone, M. (1999). Statistical inference in behavior analy-
McDowell, J. J. (1988). Matching theory in natural sis: Experimental control is better. Behavior Analyst,
human environments. Behavior Analyst, 11, 95–109. 22, 109–116.
248
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Pitts, R. C., & Febbo, S. M. (2004). Quantitative analy- Journal of the Experimental Analysis of Behavior, 84,
ses of methamphetamine’s effects on self-control 185–225. doi:10.1901/jeab.2005.09-05
choices: Implications for elucidating behavioral Staddon, J. E. R. (1968). Spaced responding and choice:
mechanisms of drug action. Behavioural Processes, A preliminary analysis. Journal of the Experimental
66, 213–233. doi:10.1016/j.beproc.2004.03.006 Analysis of Behavior, 11, 669–682. doi:10.1901/
Popper, K. R. (1962). Conjectures and refutations. London: jeab.1968.11-669
Routledge. Staddon, J. E. R. (1984). Social learning theory and the
Rachlin, H., Battalio, R., Kagel, J., & Green, L. (1981). dynamics of interaction. Psychological Review, 91,
Maximization theory in behavioral psychology. 502–507.
Behavioral and Brain Sciences, 4, 371–417. doi:10.1017/ St. Peter, C. C., Vollmer, T. R., Bourret, J. C., Borrero,
S0140525X00009407 C. S., Sloman, K. N., & Rapp, J. T. (2005). On the
Riggs, D. S., Guarnieri, J. A., & Addelman, S. (1978). role of attention in naturally occurring matching
Fitting straight lines when both variables are subject relations. Journal of Applied Behavior Analysis, 38,
to error. Life Sciences, 22, 1305–1360. doi:10.1016/ 429–443. doi:10.1901/jaba.2005.172-04
0024-3205(78)90098-X
Sutton, N. P., Grace, R. C., McLean, A. P., & Baum, W.
Rodgers, J. L. (2010). The epistemology of mathemati- M. (2008). Comparing the generalized matching law
cal and statistical modeling: A quiet methodologi- and contingency discriminability model as accounts
cal revolution. American Psychologist, 65, 1–12. of concurrent schedule performance using residual
doi:10.1037/a0018326 meta-analysis. Behavioural Processes, 78, 224–230.
Shull, R. L. (1991). Mathematical description of operant doi:10.1016/j.beproc.2008.02.012
behavior: An introduction. In I. H. Iversen & K. A. Taylor, D., Lincoln, A. J., & Foster, S. L. (2010).
Lattal (Eds.), Experimental analysis of behavior Impaired behavior regulation under conditions of
(Vol. 2, pp. 243–282). New York, NY: Elsevier. concurrent variable schedules of reinforcement in
Sidman, M. (1960). Tactics of scientific research. New children with ADHD. Journal of Attention Disorders,
York, NY: Basic Books. 13, 358–368.
Silver, B. L. (1998). The ascent of science. New York, NY: Tukey, J. W. (1977). Exploratory data analysis. Reading,
Oxford University Press. MA: Addison-Wesley.
Skinner, B. F. (1950). Are theories of learning necessary? White, K. G., McLean, A. P., & Aldiss, M. F. (1986).
Psychological Review, 57, 193–216. doi:10.1037/ The context for reinforcement: Modulation of the
response-reinforcer relation by concurrently avail-
h0054367
able extraneous reinforcement. Animal Learning and
Smith, L. D. (1990). Models, mechanisms, and explana- Behavior, 14, 398–404. doi:10.3758/BF03200085
tion in behavior theory: The case of Hull versus
Williams, B. A. (1988). Reinforcement, choice,
Spence. Behavior and Philosophy, 18, 1–18.
and response strength. In R. C. Atkinson, R. J.
Soto, P. L., McDowell, J. J., & Dallery, J. (2005). Effects Herrnstein, G. Lindzey, & R. D. Luce (Eds.), Stevens’
of adding a second reinforcement alternative: handbook of experimental psychology (2nd ed., Vol. 2,
Implications for Herrnstein’s interpretation of re. pp. 167–244). New York, NY: Wiley.
249
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 11
Time-Series Statistical Analysis

of Single-Case Data
Jeffrey J. Borckardt, Michael R. Nash, Wendy Balliet, Sarah Galloway, and Alok Madan
For years, time-series designs were almost entirely to practical settings (Jacobson & Christensen, 1996;
limited to the behavior analysis movement begun by Westen & Bradley, 2005; Westen, Novotny, &
B. F. Skinner (Jones, Vaught, & Weinrott, 1977; Thompson-Brenner, 2004). The American Psycho-
Michael, 1974; Morgan & Morgan, 2001; Sidman, logical Association’s Division 12 Task Force for
1960). Contrary to the current dominant approach in Promotion and Dissemination of Psychological
psychology, luminaries of scientific psychology based Procedures (1995) explicitly recognized time-series
their work and theory on systematic observation of designs as important methodological approaches
one organism at a time (Ebbinghaus, 1913; Kohler, that can test treatment efficacy (Chambless &
1925; Pavlov, 1927; Skinner, 1938, 1956; Watson, Hollon, 1998; Chambless & Ollendick, 2001).
1925; for a review, see Morgan & Morgan, 2001). In Westen and Bradley (2005) suggested that psycho-
the view of these pioneering researchers, behavioral therapy researchers “would do well to use clinical
processes occur at the level of the individual, and the practice as a natural laboratory for identifying prom-
large-N paradigm dominant in modern psychology ising treatment approaches” (p. 267). Hayes, Barlow,
obscures a proper analysis of these processes. More and Nelson-Gray (1999) provided a book-length
explicitly, a top-down approach in which data are argument suggesting that single-case research
aggregated across large sample sizes and generalized designs are better suited to the daily task of evaluat-
to an individual might fail to describe the behavioral ing the efficacy of interventions used by clinical psy-
process of individuals in the sample and may there- chologists than are the large-N research designs in
fore have limited applicability to individuals (see which these clinicians were trained during graduate
Chapter 7, this volume). However, from a bottom-up school. In accord with these arguments, the APA
approach, when one measures the ebb and flow of an Presidential Task Force on Evidence-Based Practice
individual’s behavior before, during, and after an (2005) endorsed systematic single-case studies as
intervention, one can test whether, when, and some- contributing to effective psychological practice.
times even why an intervention is effective. For sev- The clinical field seems to be recognizing that
eral decades, researchers such as Barlow and Hersen assaying aggregate effect is not the only empirical
(1984), Bergin and Strupp (1970), and Kazdin (1982, window from which to view the nature of behavioral
1992) have persuasively argued that practitioner- change; in addition, the systematic observation of
generated case-based time-series designs are true one or a few individuals can be scientifically sound
experiments and, as such, ought to stand alongside and instructive. Unfortunately, no upsurge of empir-
the more commonly accepted group designs (e.g., ically grounded case studies has occurred in clinical
randomized controlled trial). psychology. In fact, many practitioners still despair
Many researchers have been interested in whether over the relevance of psychotherapy research to
and how laboratory-validated interventions translate practice, perhaps because no therapy has been
DOI: 10.1037/13937-011
251
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Borckardt et al.
designated as efficacious on the weight of time- we focus primarily on a simple preintervention–

series data as prescribed by American Psychological postintervention design because it is the most fun-
Association task forces. Moreover, arguments about damental unit of inferential analysis across
efficacy are framed almost exclusively in terms of time-series designs, and hence the most relevant.
group designs (Howard, Krause, Caburnay, Noel, &
Saunders, 2001; Jacobson & Christensen, 1996; Fundamentals of Single-Case Research
Kotkin, Daviet, & Gurin, 1996; Morrison, Bradley, Proper single-case research design and good data-
& Westen, 2003; Nathan, Stuart & Dolan, 2000; sampling strategies can go a long way toward
Nielsen et al., 2004; Seligman, 1995; VandenBos, enhancing the validity of the inferences that are ulti-
1996; Westen & Morrison, 2001). Hence, despite mately available to the researcher. Observing change
calls for its resurrection, the time-series design in across time is the most fundamental feature of single-
psychotherapy outcome research remains underused. case time-series outcome designs. At the outset, the
researcher must make critical decisions about the
data that are to be collected. In a typical time-series
Statistical Analysis of Clinical
study, the behavior of a patient or research subject is
Single-Case Data
measured repeatedly on several variables across base-
Unsubstantiated case reports justifiably exert little line, treatment, and follow-up phases. The measures
influence among skeptical scientists. Social scientists chosen are, of course, determined by the nature of
who receive little formal training in single-case the research questions, the opportunity for measure-
research designs may lump the results of studies ment, and the soundness of the measures themselves.
using these designs into a class that includes unsub- Individual data points are the basic units of a
stantiated case reports. As such, these researchers time-series data set, and they typically represent
(and they are probably mostly psychologists) will observations, ratings, or counts of some predefined
dismiss the outcomes of single-case studies. behavioral unit over time. The person making the
Statistical analysis of single-case data has the observations can be the patient, the research subject,
potential to provide evidence of treatment effects the researcher, significant others, or a clinician. The
that are more widely accepted among social scien- content of the measures could be symptoms, specific
tists than are visual inspections of time-series data. behavior, physical status, or medication require-
Single-case data may be subject to inferential statisti- ments, to name a few. The single-case study is
cal analyses. Behavioral outcomes may be objec- enriched when observations are collected across
tively quantified along a number of dimensions and more than one source and more than one variable
summarized using familiar descriptive statistics. (Strupp, 1996). For example, when investigating a
Error variability can be calculated, permitting infer- clinical case involving treatment for pain, the patient
ence about whether change after treatment onset is a might report his or her daily level of pain, engage-
chance outcome. When researchers can assign a ment in predefined pain behavior, involvement in
very low probability that the observed behavior healthy activities, and amount of daily as-needed
change is a chance outcome, they can be more com- medication used. In addition, the spouse might
fortable entertaining alternative explanations. Of report the patient’s level of activity and frequency of
course, inferential statistics alone tell one nothing pain-related complaints. When considering the data
about the nature of the relation between an inter- points in a sequence over time, one can think of this
vention and observed changes in behavior, and thus set as a data stream. This stream can be divided into
researchers look to the quality of the research design phases on the basis of the hypotheses and actions of
to permit more meaningful inferences. Because the the researcher. For instance, a baseline phase can
family of time-series designs is large, a survey of encompass the stream of observations of a particular
these designs is beyond the scope of this chapter (see behavior during the period of time before the intro-
Barlow & Hersen, 1984; Kazdin, 1982; and Chapter 5, duction of an intervention. Subsequently, an inter-
this volume, for comprehensive descriptions). Here vention is implemented, and the researcher can
252
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Time-Series Statistical Analysis of Single-Case Data
continue to observe the data stream that would now Another data collection strategy involves counting
belong to the postintervention phase. This example the frequency of a response or response class during
illustrates the classic A-B design, wherein data are a predefined set of time periods and examining
collected during Phase A—the baseline or preinter- changes in rates. However, potentially valuable
vention phase—and collection continues after the information about the variability of a behavior over
onset of an intervention (i.e., Phase B). This simple time is lost with this approach if the periods during
design is the basic building block of most available which the counts take place are too long and thus
time-series research designs. may not be appropriate to answer questions about
In time-series data, behavior change requires a the change process over time.
metric anchored in time. Therefore, observations Frequency of observation is also pertinent to
should be repeated evenly across time (e.g., daily, time-series designs. Although frequent observations
weekly) so that the interval between measurements (e.g., minute to minute or hourly) can yield large
is the same throughout the study (i.e., consistent data sets, analytic problems related to serial depen-
temporal resolution); otherwise, statistical artifacts dence can be amplified by such high-frequency data
can occur. Whether the focus is heart rate, blood collection practices (discussed in the Statistical Ana-
pressure, hair pulling, itching, medication usage, or lytic Techniques for Single-Case Data section).
self-mutilation, repeated observations sampled con- However, sampling infrequently (e.g., weekly,
sistently over time and phase establish the topogra- monthly) may yield data streams that more closely
phy of change. The number of observations for each resemble data sets made up of independent observa-
phase can be different, but the interval between tions, and these streams tend to be too short to per-
observations (e.g., hours, days, weeks) should ide- mit rigorous statistical analysis. Moreover, when an
ally be the same for all phases of the study. For investigator does not make observations frequently
example, mood ratings might be collected once per enough (insufficient temporal resolution) and then
day from a patient during the baseline phase of a attempts to infer trends regarding the underlying
study, and thus these ratings should also be col- distribution from which the data points are drawn,
lected daily after treatment onset (not reduced aliasing is more likely and the chance for inferential
to weekly or increased to three times per day). error increases (see Figure 11.1). Aliasing occurs
8
Actual Behavioral Pattern
7
Insufficient/Inconsistent
Sampling
6
Behavioral Ratings
0
1 3 5 7 9 11 13 15 17 19 21 23 25
Time
Figure 11.1. Example of a series of hypothetical behavioral ratings over

time. When sampled inconsistently and infrequently, a different pattern
can emerge in the sample that is not reflective of the actual underlying
behavioral pattern. This pattern is known as aliasing.
253
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Borckardt et al.
when the sampling rate is insufficient to accurately problem, complete any further psychological testing
represent the underlying pattern of the behavior of if indicated, define the treatment plan, and schedule
interest. Because the researcher is only able to access the first treatment session. Hence, when the patient
his or her sample of observations, if the sampling returns for the first treatment session, he or she has
frequency is insufficient to fully capture the under- completed seven daily ratings in addition to the two
lying pattern, trends may emerge in the sample that to six previously completed. In this manner, nine to
do not accurately reflect the behavioral patterns 13 baseline observations are realized before treat-
in question. ment begins. In laboratory settings, the researcher
Selecting the right sampling frequency to ade- typically has more flexibility and can design studies
quately address the research question of interest is to permit collection of many more baseline data
no trivial task, but in clinical settings we have found points than is feasible in clinical practice.
daily measurement to be well tolerated by patients Although extended baselines are not strictly nec-
and well suited to the statistical requirements of sev- essary to construct a statistical model of the data set,
eral time-series analytic techniques. The clinical stable baselines with many observations over a long
researcher is often interested in knowing how the period of time are conceptually preferable. When
data map against intervention phases (e.g., baseline, researchers have the luxury of long baselines, they
treatment, follow-up); therefore, the total number of have more confidence about attributing any
observations in the entire data set as well as the observed change in behavior to the intervention
number of observations in each phase is important. itself rather than to random error or chance.
Statistically, the usual time-series study encountered
in naturalistic clinical practice has about 10 to 20 Supplementing With a Standardized
total data points (Center, Skiba, & Casey, 1985; Outcome Measure in Clinical Settings
Jones et al., 1977; Sharpley, 1987). In addition to rating sheets, researchers may use
standardized measures to obtain data from subjects.
Baseline Observations In clinical settings, standardized outcome measures
Obtaining a meaningful number of baseline observa- (e.g., the Outcomes Questionnaire 45, the Beck
tions is a critical aspect of quality time-series design. Depression Inventory, the Symptom Checklist–90)
Understandably, clinical patients object to delays in can be administered once at intake and periodically
treatment; however, this delay is likely to be less of throughout treatment. Results obtained from these
an issue in laboratory studies. However, it is possi- measures across larger time intervals (once a month,
ble at an intake session for clinicians to record a once a week) can enrich the more fine-grained anal-
number of potential dependent variables related to ysis of the continuously monitored measures (for a
the patient’s presenting problem and proposed treat- review of outcome assessment measures, see
ment plan. At the end of an intake interview, the Maruish, 2004). Although statistical modeling of
patient can be told that within a few days he or she these data at the individual patient level is not viable
will receive a packet of customized daily rating given the small number of data points, if the trend of
sheets to track symptom status (e.g., number of standardized scores (e.g., Beck Depression Inventory
panic symptoms) along with a telephone call from total scores) across time tracks reasonably well
the therapist to schedule a second appointment. against the improvement indexed by daily observa-
Within 2 days after the intake appointment, the tions, the validity of the daily measures is enhanced.
therapist can mail customized response sheets con- Figure 11.2 shows a hypothetical example of daily
sisting of three to four dependent variables to the depression ratings using a simple 10-point numeric
patient. The patient can hand in the rating sheet rating scale (on which 1 = not sad at all and 10 =
(typically covering 2–6 days) on arriving for the extremely sad) is graphed alongside periodic Beck
next appointment. During this meeting, the thera- Depression Inventory scores. The two measures are
pist and patient review the results of previous test- highly correlated over time and follow the same
ing, elaborate on the nature and scope of the clinical trend, which supports the validity of the numeric
254
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
25
Daily Depression Ratings (1 = not

20 sad at all, 10 = extremely sad)
Beck Depression Inventory Scores
Patient-Reported Scores 15
10
0
1 3 5 7 9 11 13 15 17 19
Time
Figure 11.2. Hypothetical daily depression ratings using a simple 10-point

numeric rating scale on which 1 = not sad at all and 10 = extremely sad graphed
alongside periodic Beck Depression Inventory scores
ratings scale used to capture mood on a daily basis the mean symptom status of the control group were
in the hypothetical patient. Supplemental assess- actually drawn from the same population of means,
ment on a standardized measure also provides a the question is, how likely is it that the treatment
common metric, allowing the clinical researcher to group’s aggregate symptom status would surpass
compare the extent and relevance of the patient’s that of the control group by the amount actually
therapeutic gains with those of patients in other observed (or more)? In other words, how viable is
studies (e.g., changes in Beck Depression Inventory the notion that mere random sampling fluctuation
scores over time can be compared against data from (e.g., error variance) accounts for whatever benefit
other studies using this standardized measure). is observed in the treatment group relative to that of
the control group? Scientific psychology has at its
disposal a formidable array of parametric and non-
Single-Case Data: Unique Properties
parametric statistics designed to detect nonrandom
and Considerations
shifts in population parameters.
Single-subject data is fundamentally different from
group-level data, and several aspects of such data Data Fluctuation in Time-Series Studies
deserve special consideration. In the sections that In a case-based time-series study, dispatching the
follow, we provide a theoretical backdrop for under- sampling fluctuation explanation is more complex
standing why single-subject data needs special treat- because from the onset there are not one but two
ment and discuss some of the unique characteristics types of problem fluctuation in a time-series data
and statistical challenges associated with its analysis. stream: one random and one not. There is of course
the intrasubject counterpart to the sampling error
Data Fluctuation in Group Designs encountered with group designs. After all, few
The generic outcome question for a randomized patients score an 8 on all of their baseline mood rat-
controlled trial hinges on a conditional probability: ings and a 3 on every treatment rating. There is,
If the mean symptom status of the treated group and however, another source of variability peculiar to
255
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Borckardt et al.
time-series designs: fluctuation resulting from mono- coefficient, and so on. The autocorrelation coeffi-
tonic trends, periodicity, or behavioral drifts in the cient (AR) for a stream of data can be calculated as
data occurring across time (Suen, 1987). This source
1 n−1
of predictable fluctuation in the data is termed auto- ( )(
∑ x − x x i +k − x
n − k i=1 i
)
correlation or serial dependence and is encountered in AR = ,
most other areas of natural science (e.g., soil erosion, 1 n
(
∑ x −x
n i=1 i
2
)
menses, economic recovery) in which time is a key
element of the research question. The presence of where n = the number of data points, k = lag, and
autocorrelation violates the fundamental assumption x = the data point value at time i.
of most conventional parametric and nonparametric In laboratory and applied work, the incidence of
statistics: independence of observations. autocorrelation in behavioral time-series data is gen-
erally viewed as sufficient to cause serious inferen-
Autocorrelation tial bias if conventional statistics or even visual
For observations to be independent, each and every analytic techniques are used (Busk & Maracuilo,
datum must be its own unique source of informa- 1988; Jones, Weinrott, & Vaught, 1978; Matyas &
tion. That is, the datum must be unrelated to preced- Greenwood, 1990; Suen, 1987; but see Huitema,
ing or subsequent observations (as in a series of coin 1985; Huitema & McKean, 1998; Huitema, McKean,
flips). In group research, this assumption is often rel- & McKnight, 1999). Moreover, it does not matter if
atively secure. In time-series studies in which, for the autocorrelation coefficient itself is statistically
example, a researcher is investigating a patient’s significant. What matters is “the degree of distortion
mood, the patient’s mood on Day 1 has the potential visited upon the t and F statistics when the autocor-
to partially determine the patient’s mood on Day 2, related data are analyzed via those procedures”
and subsequently the patient’s mood on Day 2 simi- (Sharpley & Alavosius, 1988, p. 246). For instance,
larly becomes the point of departure for mood on whether it is significant or not, an autocorrelation
Day 3, and so on: The data points are not indepen- coefficient of .10 can inflate t and F values 110% and
dent of each other. Because the data points are drawn 200%, respectively, when the autocorrelation coeffi-
from the same person over time, the variability from cient is .6. For this reason, time-series designs
data point to data point is attenuated to some extent require special statistical treatment.
and constrained within the confines of the single Although it is a statistical nuisance, by its nature
individual generating the data. For this reason, when serial dependence reflects the momentum and grad-
conventional inferential group statistics (e.g., t, F, ualism of physiological, behavioral, and emotional
chi-square, and sign tests) are mistakenly applied to repair. Because it is an index of serial dependence,
autocorrelated data sets, variability is underesti- autocorrelation can reveal something about the ebb
mated, and subsequently the effect or variability ratio and flow of behavioral change over time. For this
is inflated. Spuriously high ts, Fs, and rs are gener- reason, autocorrelation is the natural subject matter
ated, and as a result researchers may conclude that of a behavioral science. Whatever inferential statistic
an effect exists when such a conclusion is not justi- is applied to single-case time-series data, we believe
fied (Hibbs, 1974; Sharpley & Alavosius, 1988). it should approach autocorrelation not as noise that
Autocorrelation can be quantified for time-series obscures change, but as music that accompanies it.
data sets and is typically expressed across different Put differently, the preferred statistic gauges the
lags of the data. For example, the correlation of pairs occurrence of change while preserving its structure.
of data points immediately adjacent to each other in
time from a single time-series data set represents lag
Statistical Analytic Techniques
1 (i.e., data points at time i, correlated with data
for Single-Case Data
points at time i + 1). Similarly, data points at time i
can be correlated with data points at time i + 2, In this section, we discuss several approaches to
which would represent the lag-2 autocorrelation analyzing single-case time-series data. Although
256
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
numerous approaches have been developed and same subject over time should be related to each
used for examining time-series data, each model other and not independent. The power to detect sig-
possesses limitations and optimally operates under nificant autocorrelation coefficients is influenced by
certain assumptions. When selecting a technique, it n size, and the number of data points typically col-
is important to consider that elements of the lected, at least in clinical time-series research, is
research design and characteristics of the data often small. Thus, with short data streams, even if
stream as inherent properties of single-case time- the autocorrelation coefficient is not statistically sig-
series designs (i.e., autocorrelation and possibly nificantly different from zero, one should avoid
data-stream length) can pose challenges to accurate application of OLS regression techniques to single-
parameter estimation. case data. Moreover, GLS (which may be capable of
minimizing inferential error associated with auto-
Ordinary and Generalized Least-Squares correlated residuals) requires a large number of data
Regression points to accurately estimate the parameters neces-
Ordinary least squares (OLS) is a method for esti- sary to yield valid and reliable results (see Ferron,
mating the unknown parameters in a linear regres- 2002). As with almost all traditional regression
sion model. This method minimizes the sum of approaches (including GLS), one should have at
squared distances between the observed responses least 30 data points per phase (preferably 50) to
in the data set and the responses predicted by lin- ensure that the output is robust and valid (Robey,
ear approximation. An independent measure (e.g., Schultz, Crawford, & Sinner, 1999), which is often
phase) can be numerically coded and used to predict untenable in clinical case research.
a series of observations collected over time. How-
ever, adequate performance of OLS methods assumes Randomization and Permutation Tests
independence and normality of the residuals of the To overcome some of the limitations associated with
fitted model, an assumption that is violated in time- the application of OLS regression for time-series
series data. When autocorrelation is present, one analysis, several nonparametric tests have been pro-
may use generalized least-squares (GLS) estimation posed that involve randomization and permutation
rather than OLS. With GLS, the estimated OLS resid- of time-series data sets (Ferron & Onghena, 1996;
uals are used to estimate the error covariance matrix Ferron & Sentovich, 2002; Manly, 1997; Mielke &
for the model. GLS estimation then minimizes the Berry, 1994). Randomization tests are statistical tests
sum of squares of the residuals weighted by the that involve generating numerous iterations of ran-
inverse of the sample covariance matrix, which may dom assignment of data points to treatments. Per-
permit more robust performance in the face of auto- mutation of the order of the data in a time-series
correlated residuals. These approaches are widely data set can permit determination of whether the
used, and the OLS and GLS regression output can be same results would have been obtained if the data
interpreted by using commercial statistical software were assigned to rearranged placements. That is,
packages. data points representing a behavioral outcome mea-
As noted earlier, to ensure the accuracy and sure of interest spanning two phases of intervention
validity of OLS regression, assumptions of a normal can be randomly ordered several times, and the
distribution and independence of the residuals from impact of the phase intervention can be determined
the fitted OLS model must be met. Unfortunately, by evaluating the importance of data-point ordering
these assumptions are often violated when working in achieving the observed differences between
with single-case data. It is difficult to make a com- phases. With this approach, a single-case researcher
pelling case that data collected using a single-case can determine how likely the pattern of a dependent
design are independent even if the autocorrelation measure is to occur if the data are randomly ordered
correlation coefficient for a data stream is not signif- across intervention phases. The extent to which the
icantly different from zero, because one would patterns of the dependent measure change when
expect a priori that data points collected from the the data points are randomly assigned to different
257
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Borckardt et al.
intervention phases determines the probability Autoregression and Autoregressive

that the data patterns are uniquely tied to the Integrated Moving Average
intervention. Autoregressive integrated moving average (ARIMA)
Although once seen as computationally exces- and autoregressive models can be fitted to time-
sive, modern computers can conduct randomization series data to better understand the data, to predict
tests quickly and efficiently. The most important future points in the series, and to determine the
advantage of randomization tests is that they are effects of an independent variable on the dependent
nonparametric and consequently are not based measure over and above the influence of auto
on distributional assumptions (Arndt et al., 1996; correlation and moving-average components.
Hooton, 1991; Ludbrook, 1994; Recchia & Roc- The ARIMA model is generally referred to as an
chetti, 1982; Wilson, 2007). Randomization tests are ARIMA(p, d, q) model, where p, d, and q are integers
also fairly straightforward, are easy to apply, and greater than or equal to zero and refer to the order
may be used with very short data streams (Edging- of the autoregressive, differencing, and moving
ton & Onghena, 2007). More important, the experi- average components of the model, respectively.
menter must designate a certain time point at which Autoregressive analyses first partial out the effects
the treatment can be administered and then ran- of autocorrelation on a data stream and then esti-
domly assign each time point to a treatment. This mate the impact of the intervention over and above
random assignment not only enhances the study’s the effects of autoregressive parameters. These tests
internal validity, it also justifies the application of are probably the most appropriate and widely used
randomization tests (Todman & Dugard, 1999). tests for analysis of autocorrelated data streams, and
Ferron, Foster-Johnson, and Kromrey (2003) inves- they can reliably indicate whether an intervention is
tigated randomization tests with and without ran- associated with change in a dependent variable over
dom assignment and concluded that the absence of and above the effects of serial dependence. The user
random assignment questions the legitimacy of can indicate how many lags of autocorrelation to
using a randomization test (p. 285) and that ran- partial out and, with ARIMA, to indicate whether to
domization tests need to be based on permutations difference the data to improve stationarity (i.e., sub-
that mirror the random assignment of subjects to tracting pairs of serial data points to remove trend),
conditions used in the experiment. This is only a whether to model moving average components (i.e.,
minor constraint for researchers, and attention to it smoothing out of short-term fluctuations by averag-
can actually enhance the internal validity of single- ing groups of data points over time) of the data
case research. Although some have suggested that stream, or both. These tests can provide appropriately
randomization tests may not be significantly influ- conservative estimates of intervention effects after
enced by the effects of autocorrelation in the data controlling for unique properties of time-series data.
stream (Ferron et al., 2003), parametric and non- The user can avoid ambiguity around model specifi-
parametric statistics alike assume that data points cation by using a technique called overfitting, in
are drawn from independent sources. This is, of which one specifies to control for numerous lags of
course, untrue in single-case research. Good (1994) autocorrelation rather than determining the exact
stated that “all hypothesis-testing methods rely on number of lags that may pose inferential problems
the independence and/or exchangeability of the (e.g., fitting an ARIMA(15, 0, 0) model to all data).
observations” (p. 149). True significance probability These approaches typically require between 30 and
values are underestimated for positively autocorre- 50 data points per phase to yield acceptable sensitiv-
lated residuals (Franklin, Allison, & Gorman, 1996 ity and selectivity. As previously discussed, data sets
p. 172). Thus, more work is needed on randomiza- of this length are rarely encountered in clinical single-
tion approaches to ensure validity and accuracy case research, especially when data are collected on
when applied to autocorrelated single-case data real patients seeking outpatient psychotherapy. If one
before these approaches can be recommended with- happens to have a sizable time-series data stream
out qualification. (e.g., from a lab-based study or from a clinical study
258
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
using a high-frequency sampling scheme), these Interrupted Time-Series Experiments

methods should probably be the first-choice statisti- The interrupted time-series experiment (ITSE)
cal analytic technique. approach uses autocorrelation information along
with phase slope and mean data to model the effects
Hierarchical Linear Modeling of interventions on a single case’s time-series data
Hierarchical linear modeling (HLM), also known stream. This approach provides information regard-
as multilevel modeling, is a more advanced form of ing the individual phase means and slopes and
simple linear regression and multiple linear regres- whether they differ from each other and from zero
sion. Multilevel analysis allows variance in outcome after controlling for autoregressive parameters.
variables to be analyzed at multiple hierarchical Although the number of data points required to per-
levels (e.g., within an individual, over time, and mit valid estimation of autoregressive parameters
between groups of individuals), whereas in simple has been suggested as limiting its use with short
linear and multiple linear regression, all effects time-series data streams often encountered in clini-
are modeled to occur at a single level. For repeated- cal practice, Crosbie (1993, 1994) proposed the use
measures data (such as time-series data encoun- of a modified ITSE approach wherein an autocorre-
tered in clinical case research), time can be lation correction for short data streams is applied
considered as a level that occurs within patients. (ITSACORR). ITSACORR permits analysis of short
Moreover, the covariance matrix structure (repre- autocorrelated data streams with phase lengths as
senting the interrelationships between all variables low as 7 data points each. This approach appears to
in a model) can be specified for each model as control for Type II error with short autocorrelated
unstructured or even autoregressive, which can data sets because false positive rates associated with
effectively relax assumptions of independence of this test are roughly equal to nominal alpha (.05).
observations. Although this approach is typically Evidence has even suggested good false-positive
used to analyze data sets in which observations are control with phase lengths as short as 5 data points
nested within subjects who may be nested in any each (Crosbie, 1993,1994). ITSACORR offers famil-
variety of settings (e.g., treatment groups, commu- iar F and t statistics regarding the significance of
nities, schools), it can be used to analyze single-case process change with respect to both slope and level
data. It becomes even more appropriate when a few across treatment phases.
or even several participants in the study are provid- Although ITSACORR appears to offer good selec-
ing time-series data with different subject-level tivity with short autocorrelated data streams, its
characteristics of clinical interest (e.g., type of ther- power to detect effects may be a limiting factor to its
apy used, sex, ethnicity, age) and time-based vari- widespread use. Crosbie (1993, 1994) showed ade-
ables of interest (e.g., baseline, treatment onset, quate power of ITSACORR to detect large effects in
follow-up phase). HLM is a powerful and flexible short data streams. The definition of large warrants
statistical tool for managing complex data sets with elaboration. Crosbie and others have suggested that
time-series elements as well as between-subject a large effect in time-series is represented by a
factors. HLM handles missing data points with min- 10–standard deviation change from baseline to the
imal problems and thus tolerates inconsistent tem- intervention phase, and a medium effect is repre-
poral resolution while allowing for sophisticated sented by a 5–standard deviation change. Although
modeling of numerous potential explanatory fac- some evidence has supported the presence of such
tors. As with autoregression and ARIMA, HLM also large effects in the single-case research literature,
permits evaluation of slope and intercept changes time-series statistical tests should probably reliably
over time and between groups, with minimal nega- detect much smaller effects than 10–standard devia-
tive bias from autoregressive parameters. However, tion changes between phases. The software to run
HLM also typically requires between 30 and 50 data ITSACORR has historically been offered for free;
points per phase to yield acceptable sensitivity and however, this software has become hard to find over
selectivity. the past several years, and it appears Crosbie has
259
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Borckardt et al.
pulled it out of circulation. This is unfortunate many popular statistical software packages have yet
because, despite its limitations with respect to to implement robust methods (Stromberg, 2004).
power, it may still have a place in analysis of short, Although it is still unclear which robust methods
autocorrelated clinical time-series data, especially would be best suited for handling short autocorre-
when one wants to ensure highly conservative infer- lated data streams of the like encountered in clinical
ences. Although the stand-alone software for ITSA- research, there is some hope that derivations of
CORR appears to be unavailable at the present time, existing robust analytic techniques may become use-
the mathematical models for it are provided by ful to clinical researchers in the near future if these
Crosbie (1993, 1994). tests become more accessible.
Robust Regression Control-Chart Techniques and Celeration

Robust statistics tend to emulate traditional statistical Trend-Line Analysis
methods but are designed to be robust to outliers or Visual analytic techniques offer a straightforward
other small departures from traditional model assump- approach to evaluating the effects of various inter-
tions. Thus, robust regression is a form of regression ventions on single-case outcome measures (see
analysis designed to circumvent some limitations of Chapter 9, this volume); however, some of these
traditional parametric and nonparametric methods. techniques have been criticized for being too subjec-
Traditional least-squares estimates are pulled toward tive (e.g., Matyas & Greenwood, 1990). To enhance
outliers (abnormally high or low values of a few data the objectivity of visual analysis procedures, a sug-
points in a data set), thereby inflating error variance gestion has been that they can be combined with
estimates, whereas robust statistical estimates are less some traditional statistical analytic approaches and
influenced by violation of assumptions. probability theory. Numerous control-chart tech-
Huber (1972) introduced maximum likelihood niques exist that involve graphically representing a
estimation for regression. One criticism of the dependent variable over time (Pfadt & Wheeler,
method is that it appears to be robust to outliers in 1995; Stoumbos, Reynolds, Ryan, & Woodall,
the dependent measure but may not be resistant to 2000). Users can then also visually represent some
outliers in predictor variables. In fact, when there variability estimates and apply decision rules to
are outliers in predictor variables, this method may determine the “unusualness” of patterns seen in the
offer little advantage over OLS. Fortunately, in case- dependent measure. Figure 11.3 shows a hypotheti-
based time-series research, the time-based phase cal time-series data set representing behavioral rat-
variable rarely has outliers. ings during a baseline phase and an intervention
Robust statistics appear to be a viable alternative phase. The mean from the baseline phase is repre-
to traditional statistical analytic approaches, espe- sented with a solid black line and is carried over into
cially when assumption violations are expected to the intervention phase. Dotted lines are included
be common. Moreover, they appear to be slowly representing 1, 2, and 3 standard deviations (sigma)
growing in popularity in recent years. Nonetheless, around the mean of the baseline phase data and also
despite their superior performance over least- carried over into the intervention phase. As can be
squares estimation in many situations in which seen in Figure 11.3, 9 of the 11 data points in the
assumption violations are present, robust methods intervention phase fall outside 3 standard deviations
for regression are still not widely used. Several rea- of the baseline mean. If there is reason to expect that
sons may explain their lack of popularity (Hampel the intervention will produce a gradual effect (e.g.,
et al., 1986). Several competing (traditional) meth- the intervention involves the acquisition of complex
ods are better known among researchers. Although skills), then one would be tempted to conclude that
computation of robust estimates is much more this 3–standard deviation difference is significant.
computer intensive than traditional least-squares Notice in Figure 11.3 that the baseline data have
estimation, this problem has become less relevant as an upward slope, and most traditional control-chart
computing power has increased greatly. Unfortunately, techniques do little to account for such trends.
260
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
10 BASELINE PHASE INTERVENTION PHASE
Behavioral Ratings
3-Si gma
6
2-Si gma
5
1-Si gma
4
Ba s el i ne Mea n
3
0
1 3 5 7 9 11 13 15 17 19 21 23
Time
Figure 11.3. Hypothetical example of behavioral ratings over time span-

ning a baseline phase and an intervention phase. As is common with many
control-chart techniques, the mean and standard deviation estimates of the
baseline phase are graphed and extended into the intervention phase. The
data points in the intervention phase can then be evaluated relative to these
extended mean and variability lines.
10 BASELINE PHASE INTERVENTION PHASE
8
Behavioral Ratings
0
1 3 5 7 9 11 13 15 17 19 21 23
Time
Figure 11.4. Celeration trend-line analysis using the same hypothetical data
as Figure 11.3. The data points are graphed along with a ordinary least-squares
trend line representing the baseline phase, and this line is extended into the
intervention phase. Data points in the intervention phase falling on either side of
the extended trend line are counted, and a binomial test can be applied.
However, in celeration trend-line analysis, the user number of data points in the treatment phase falling
draws a trend line based on the baseline-phase data on either side of the extended trend line. Simple
and extends it into the intervention phase (see binomial tests can be run on the number of data
Figure 11.4). From here, the user can examine the points falling on either side of the trend line, and the
261
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Borckardt et al.
user can even calculate a p value representing the then one may regard the difference as significant.
pattern (Franklin et al., 1996). If one assumes that Monte Carlo simulations indicated that the CDC
each data point in the intervention phase has an method had acceptable rates of Type I error with
equal probability of falling above or below the trend data sets as small as five observations in baseline
line (i.e., no phase effect), and only 1 of the 11 inter- and treatment. The statistical power of the CDC
vention phase data points fall below the extended was more acceptable than that of the ITSACORR,
trend line (Figure 11.4), then a simple binomial test requiring a difference of 2 standard deviations (10
would indicate that the probability of this occurring baseline and 10 treatment data points) to detect a
by chance alone is quite low (p = .005). Thus, the difference with 80% probability. An added benefit of
null hypothesis is rejected, and one may regard the the CDC method is that Fisher et al. provided data
change in behavior across phases as significant. suggesting that a service delivery staff could be
Control-chart techniques are intuitive efficiently trained to use the CDC method with
approaches that combine user-friendly and logical minimal error.
visual techniques with familiar parametric (or even
nonparametric, in some cases) statistics. This sta- Tryon’s C
tistical calculation portion of these approaches can Tryon’s (1982) C is a time-series–specific test based
be implemented with virtually any statistical soft- on the mean-squares successive difference test. It is
ware package. Unfortunately, these approaches primarily designed to indicate the presence of trends
(along with most visual and statistical approaches in a data stream. If there is a consistent trend in the
to short time-series data analysis) are negatively data, the value of C increases. One can use C to test
influenced by serial dependence (Matyas & Green- for trend in the baseline and treatment phases. If no
wood, 1990). When autocorrelation is present, trend is found in the baseline, but one is detected in
Type I error rates become inflated and researchers the treatment phase, it might indicate a shift in
are more likely to infer the presence of an effect trend, presumably associated with the onset of the
when none is there. intervention. Although not offered in many com-
To address this concern, Fisher, Kelly, and mercial statistical packages, the formula for calculat-
Lomas (2003) conducted Monte Carlo simulations ing C is straightforward and not too labor intensive
to evaluate the Type I and Type II error rates of and thus can be hand calculated easily (Franklin
additional control-chart techniques. Their results et al., 1996):
supported the use of a conservative dual-criterion n−1
(CDC) method. The first criterion for determining ∑ (Y − Y i i+1
)2
whether a set of time-series intervention data points C =1− i=1
,
n
deviates significantly from baseline is evaluated by
2∑(Yi − Y ) 2
first fitting a least-squares regression line through i=1
the baseline data and extending the line through the
intervention phase (as in Figure 11.4). If a binomial where Yi is the dependent variable at time i within
–
test indicates that behavior has changed significantly the time series (e.g., the baseline) and Y the mean of
from what would be predicted from the baseline the time series.
trend, then the first criterion has been satisfied. The This statistic seems well suited to handle the
second criterion is evaluated by drawing a horizon- common types of questions that a single-case
tal line (slope = 0) through the baseline mean and researcher might ask. However, Tryon’s C does not
extending it through the treatment data, and then appear to be a unique indicator of trends and shifts
conducting a binomial test as before. The conserva- in data streams that are associated with process
tive modifier of the CDC involves adjusting both change alone. In other words, the presence of
lines by 0.25 standard deviation in the direction of serial dependence in the data stream might contrib-
the expected treatment effect. If both binomial ute substantially to the appearance of a trend in a
tests reveal a significant deviation from baseline, data stream, and C would not permit the user to
262
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
ifferentiate the artifacts caused by autocorrelation

d power [>.80] with 5 data points per phase with a
from genuine trends. In fact, C has been suggested 5–standard deviation effect size). The user does not
to be a better index of serial dependence than of the have to implement simple phase vectors (zeros and
presence of trends in the data (Franklin et al., 1996; ones) against which to correlate the dependent mea-
Robey et al., 1999). Additionally, C becomes much sure; rather, the user can establish independent
more powerful with longer data streams, and its measures of any type or pattern to correspond with
significance test is highly influenced by series the research design used (e.g., slopes, A-B-A-B
length. designs [see Chapter 5, this volume], custom pat-
terns). Software to implement SMA is freely avail-
Simulation Modeling Analysis able for Windows and Macintosh operating
Based loosely on bootstrapping methods (Wilcox, systems (http://clinicalresearcher.org/software.htm).
2001), simulation modeling analysis (SMA) is spe- However, SMA does not provide any information
cially designed for analyzing short, autocorrelated regarding the fidelity of the simulation data to the
data streams typically found in clinical practice actual properties of the original data, and problems
(Borckardt et al., 2008). SMA first estimates the with the reliability of parameter estimates from
autocorrelation coefficient for the data stream short time-series data streams may have an impact
in question and corrects it for small-n bias using on test accuracy. SMA does perform well with lon-
Crosbie’s (1993, 1994) method. Separately, SMA ger time-series data streams (>30 points per phase).
quantifies the relation between the dependent mea- Its performance has not yet been evaluated with
sure and a dummy-coded intervention (e.g., zeros skewed distributions or extreme outliers.
for baseline and ones for treatment phase) using
either a parametric or a nonparametric correlation
Recommendations
coefficient. Next, SMA uses the phase n-sizes and
autocorrelation estimate to generate thousands of To date, there is no clear, best statistical approach to
random-normal data streams programmed with the analyze short streams of single-case time-series data.
same autocorrelation and n-size information as the Although many approaches have been proposed,
clinical data stream in question. For each of these and some appear better than others, they all have
short, autocorrelated, randomly generated data some limitations. When data sets are large enough
streams, a correlation coefficient is calculated (approximately 30 data points per phase), the
between it and the intervention phase vector. Each researcher has a reasonably rich palette of statistical
of these correlation coefficients that exceed the tools from which to choose, all of which are flexible
value of the correlation between the original data and perform quite well (e.g., ARIMA, autoregres-
and the phase vector is flagged and counted. The sion, GLS regression, HLM). However, when the
empirical p value provided by SMA is the proportion data set is sparse, the analytic options dwindle. At
of flagged correlations to the total number of ran- this point, it is probably too early to endorse any
dom simulation data streams generated (determined specific statistical approaches for short, autocorre-
by the user or set to 5,000 by default). Thus, SMA lated time-series data streams. Although robust
answers the question, how likely is it for a com- regression techniques, CDC, ITSACORR, and SMA
pletely random data stream of the same length as show promise, more work is needed to verify that
yours, with the same amount of autocorrelation as these methods are capable of reliable Type I and
yours, to evidence a correlation with your phase Type II error performance with short, serially
vector as large as your data stream did? dependent streams of data.
This approach appears to offer good Type I error
control with very short (approximately 5 data points
Conclusions
per phase) autocorrelated data streams. It also
appears to offer adequate power to detect much The behavioral data yielded by single subjects in
smaller effect sizes than ITSACORR (adequate naturalistic and controlled settings likely contain
263
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Borckardt et al.
information valuable to scientists and practitioners References

alike. Although some of the properties unique to APA Presidential Task Force on Evidence-Based Practice.
this data complicate statistical analysis, progress has (2005). Report of the 2005 Presidential Task Force on
been made in developing specialized techniques for Evidence-Based Practice. Washington, DC: American
Psychological Association.
rigorous data evaluation. Certainly, we did not
exhaustively cover all available time-series analysis Arndt, S., Cizadlo, T., Andreasen, N. C., Heckel, D.,
Gold, S., & O’Leary, D. S. (1996). Tests for compar-
techniques; rather, we highlighted a few time-series ing images based on randomization and permuta-
analytic genres. No perfect tests are currently avail- tion methods. Journal of Cerebral Blood Flow and
able to analyze short autocorrelated data streams, Metabolism, 16, 1271–1279. doi:10.1097/00004647-
but there are some promising approaches that war- 199611000-00001
rant further development. Barlow, D. H., & Hersen, M. (1984). Single case experi-
mental designs. New York, NY: Pergamon Press.
From a clinical perspective, case-based time-series
designs will likely not dissolve the formidable episte- Bergin, A. E., & Strupp, H. H. (1970). The directions
in psychotherapy research. Journal of Abnormal
mological gap between practice and research, but Psychology, 76, 13–26.
their use might help bring the two disciplines within
Borckardt, J. J., Nash, M. R., Murphy, M. D., Moore, M.,
shouting distance of each other on a more regular Shaw, D., & O’Neil, P. (2008). Clinical practice as nat-
basis. Enhanced communication pivots on compro- ural laboratory for psychotherapy research: A guide to
mise by both parties. For their part, practitioners case-based time-series analysis. American Psychologist,
63, 77–95. doi:10.1037/0003-066X.63.2.77
must concede that replicable systematic observation
is a necessary requirement of evidence; in turn, Busk, P. L., & Maracuilo, R. C. (1988). Autocorrelation
in single-subject research: A counter-argument to the
researchers must concede (indeed, rediscover) that myth of no autocorrelation. Behavioral Assessment,
carefully conducted ideographic studies can yield 10, 229–242.
empirically sound findings about change processes. Center, B. A., Skiba, R. J., & Casey, A. (1985). A method-
For some, these concessions will not be forthcoming. ology for the quantitative synthesis of intra-subject
Still, a robust clinical science requires an ongoing design research. Journal of Special Education, 19,
387–400. doi:10.1177/002246698501900404
productive discourse between a critical mass of
Chambless, D. L., & Hollon, D. D. (1998). Defining
researchers and practitioners. Herein is the twofold
empirically supported therapies. Journal of Consulting
promise of case-based time-series designs. First, their and Clinical Psychology, 66, 7–18. doi:10.1037/0022-
careful use enables clinical researchers and practitio- 006X.66.1.7
ners to make contributions that are fully congruent Chambless, D. L., & Ollendick, T. H. (2001). Empirically
with the evidence-driven ethos of scientific discourse. supported psychological interventions: Controversies
By rising above mere clinical anecdote, practitioners and evidence. Annual Review of Psychology, 52,
685–716. doi:10.1146/annurev.psych.52.1.685
earn a more prominent and respected voice on mat-
Crosbie, J. (1993). Interrupted time-series analysis with
ters of theory, research, policy, and training. The brief single-subject data. Journal of Consulting and
clinical setting can indeed become the natural labora- Clinical Psychology, 61, 966–974. doi:10.1037/0022-
tory envisioned by Westen and Bradley (2005) and 006X.61.6.966
Peterson (2004). Second, time-series designs yield Crosbie, J. (1994). Interrupted time-series analysis with
findings especially pertinent to how therapeutic short series: Why it is problematic; how it can be
improved. In J. M. Gottman (Ed.), The analysis of
change unfolds, not in aggregate, but individually. change (pp. 361–395). Mahwah, NJ: Erlbaum.
Although almost entirely neglected by contemporary
Ebbinghaus, H. (1913). Memory (H. A. R. C. E. Busenius,
investigators, single-organism research of this kind Trans.). New York, NY: Teachers College.
has a luminous and storied lineage. By harnessing Edgington, E. S., & Onghena, P. (2007). Randomization
time-series designs and rigorous analytic techniques tests (4th ed.). Boca Raton, FL: Chapman & Hall/CRC.
alongside group experimental methodologies, we Ferron, J. (2002). Reconsidering the use of the general
might well accelerate the progress we are making in linear model with single case data. Behavior Research
understanding the anatomy and mechanisms of Methods, Instruments and Computers, 34, 324–331.
doi:10.3758/BF03195459
behavioral change.
264
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Ferron, J., Foster-Johnson, L., & Kromrey, J. D. (2003). Psychological Methods, 3, 104–116. doi:10.1037/
The functioning of single-case randomization tests 1082-989X.3.1.104
with and without random assignment. Journal of
Huitema, B. E., McKean, J. W., & McKnight, S. (1999).
Experimental Education, 71, 267–288. doi:10.1080/
Autocorrelation effects on least-squares interven-
00220970309602066
tion analysis of short time series. Educational and
Ferron, J., & Onghena, P. (1996). The power of random- Psychological Measurement, 59, 767–786. doi:10.1177/
ization tests for single-case phase designs. Journal of 00131649921970134
Experimental Education, 64, 231–239. doi:10.1080/00
220973.1996.9943805 Jacobson, N. S., & Christensen, A. (1996). Studying the
effectiveness of psychotherapy: How well can clinical
Ferron, J., & Sentovich, C. (2002). Statistical power of trials do the job? American Psychologist, 51, 1031–1039.
randomization tests used with multiple-baseline doi:10.1037/0003-066X.51.10.1031
designs. Journal of Experimental Education, 70, 165,
178. Jones, R. R., Vaught, R. S., & Weinrott, M. R. (1977).
Time-series analysis in operant research. Journal of
Fisher, W. W., Kelley, M. E., & Lomas, J. E. (2003). Applied Behavior Analysis, 10, 151–166. doi:10.1901/
Visual aids and structured criteria for improving jaba.1977.10-151
visual inspection and interpretation of single-case
designs. Journal of Applied Behavior Analysis; 36, Jones, R. R., Weinrott, M. R., & Vaught, R. S. (1978).
387–406. doi:10.1901/jaba.2003.36-387 Effects of serial dependency on the agreement
between visual and statistical inference. Journal of
Franklin, R. D., Allison, D. B., & Gorman, B. S. (1996). Applied Behavior Analysis, 11, 277–283. doi:10.1901/
Design and analysis of single-case research. Mahwah, jaba.1978.11-277
NJ: Erlbaum.
Kazdin, A. E. (1982). Single-case research designs: Methods
Good, P. (1994). Permutation tests: A practical guide to for clinical and applied settings. New York, NY:
resampling methods for testing hypotheses. New York, Oxford University Press.
NY: Springer-Verlag.
Kazdin, A. E. (1992). Research design in clinical psychol-
Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & ogy (2nd ed.). Boston, MA: Allyn & Bacon.
Stahel, W. A. (1986). Robust statistics: The approach
based on influence functions. New York, NY: Wiley. Kohler, W. (1925). The mentality of apes. New York, NY:
Harcourt.
Hayes, S. C., Barlow, D. H., & Nelson-Gray, R. O. (1999).
The scientist practitioner: Research and accountabil- Kotkin, M., Daviet, C., & Gurin, J. (1996). The Consumer
ity in the age of managed care (2nd ed.). Needham Reports mental health survey. American Psychologist,
Heights, MA: Allyn & Bacon. 51, 1080–1082. doi:10.1037/0003-066X.51.10.1080
Hibbs, D. A. (1974). Problems of statistical estimation Ludbrook, J. (1994). Advantages of permutation (ran-
and causal inference in time-series regression mod- domization) tests in clinical and experimental phar-
els. In H. L. Costner (Ed.), Sociological methodology, macology and physiology. Clinical and Experimental
1973–1974 (Vol. 5, pp. 252–308). San Francisco, CA: Pharmacology and Physiology, 21, 673–686.
Jossey-Bass. doi:10.1111/j.1440-1681.1994.tb02570.x
Hooton, J. W. (1991). Randomization tests: Statistics for Manly, B. F. J. (1997). Randomization, bootstrap, and
experimenters. Computer Methods and Programs in Monte Carlo methods in biology. Boca Raton, FL:
Biomedicine, 35, 43–51. doi:10.1016/0169-2607(91) Chapman & Hall/CRC.
90103-Z Maruish, M. E. (2004). The use of psychological testing for
Howard, K. I., Krause, M. S., Caburnay, C. A., Noel, S. B., treatment planning and outcomes assessment. Mahwah,
& Saunders, S. M. (2001). Syzygy, science, and psy- NJ: Erlbaum.
chotherapy: The Consumer Reports study. Journal Matyas, T. A., & Greenwood, K. M. (1990). Visual analy-
of Clinical Psychology, 57, 865–874. doi:10.1002/ sis of single-case time series: Effects of variability,
jclp.1055 serial dependence, and magnitude of intervention
Huber, P. J. (1972). Robust statistics. Annals of Mathe of effects. Journal of Applied Behavior Analysis, 23,
matical Statistics, 43, 1041–1067. doi:10.1214/ 341–351. doi:10.1901/jaba.1990.23-341
aoms/1177692459 Michael, J. (1974). Statistical inference for individual
Huitema, B. E. (1985). Autocorrelation in applied organism research: Mixed blessing or curse?
behavior analysis: A myth. Behavioral Assessment, 7, Journal of Applied Behavior Analysis, 7, 647–653.
107–118. doi:10.1901/jaba.1974.7-647
Huitema, B. E., & McKean, J. W. (1998). Irrelevant auto- Mielke, P. W., & Berry, K. J. (1994). Permutation tests
correlation in least-squares intervention models. for common locations among samples with unequal
265
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Borckardt et al.
variances. Journal of Educational and Behavioral Skinner, B. F. (1956). A case history in scientific method.
Statistics, 19, 217–236. American Psychologist, 11, 221–233. doi:10.1037/
h0047662
Morgan, D. L., & Morgan, R. K. (2001). Single-participant
research design: Bringing science to managed care. Stoumbos, Z. G., Reynolds, M. R., Ryan, T. P., &
American Psychologist, 56, 119–127. doi:10.1037/ Woodall, W. H. (2000). The state of statistical pro-
0003-066X.56.2.119 cess control as we proceed into the 21st century.
Journal of the American Statistical Association, 95,
Morrison, K. H., Bradley, R., & Westen, D. (2003). The
992–998. doi:10.2307/2669484
external validity of controlled clinical trials of psy-
chotherapy for depression and anxiety: A natural- Stromberg, A. J. (2004). Why write statistical software?
istic study. Psychology and Psychotherapy: Theory, The case of robust statistical methods. Journal of
Research, and Practice, 76, 109–132. doi:10.1348/ Statistical Software, 10(5). Retrieved from http://
147608303765951168 www.jstatsoft.org/v10/i05
Nathan, P. E., Stuart, S. P., & Dolan, S. L. (2000). Strupp, H. H. (1996). The tripartite model of the
Research on psychotherapy efficacy and effective- Consumer Reports Study. American Psychologist, 51,
ness: Between Scylla and Charybdis? Psychological 1017–1024. doi:10.1037/0003-066X.51.10.1017
Bulletin, 126, 964–981. doi:10.1037/0033-2909. Suen, H. K. (1987). On the epistemology of autocor-
126.6.964 relation in applied behavior analysis. Behavioral
Nielsen, S. L., Smart, D. W., Isakson, R. L., Worthen, V. E., Assessment, 9, 113–124.
Gregersen, A. T., & Lambert, M. J. (2004). The Task Force for Promotion and Dissemination of
Consumer Reports effectiveness score: What did con- Psychological Procedures. (1995). Training in and
sumers report? Journal of Counseling Psychology, 51, dissemination of empirically-validated psychological
25–37. doi:10.1037/0022-0167.51.1.25 treatments: Report and recommendations. Clinical
Pavlov, I. P. (1927). Conditioned reflexes. New York, NY: Psychologist, 48, 3–23.
University Press. Tryon, W. W. (1982). A simplified time-series analysis
Peterson, D. R. (2004). Science, scientism, and profes- for evaluating treatment interventions. Journal of
sional responsibility. Clinical Psychology: Science and Applied Behavior Analysis, 15, 423–429. doi:10.1901/
Practice, 11, 196–210. doi:10.1093/clipsy.bph072 jaba.1982.15-423
Pfadt, A., & Wheeler, D. J. (1995). Using statistical pro- VandenBos, G. R. (1996). Outcome assessment of psy-
cess control to make data-based clinical decisions. chotherapy. American Psychologist, 51, 1005–1006.
Journal of Applied Behavior Analysis, 28, 349–370. doi:10.1037/0003-066X.51.10.1005
doi:10.1901/jaba.1995.28-349 Watson, J. B. (1925). Behaviorism. New York, NY: Norton.
Recchia, M., & Rocchetti, M. (1982). The simulated ran- Westen, D., & Bradley, R. (2005). Empirically supported
domization test. Computer Programs in Biomedicine, complexity. Current Directions in Psychological Science,
15, 111–116. doi:10.1016/0010-468X(82)90062-9 14, 266–271. doi:10.1111/j.0963-7214.2005.00378.x
Robey, R. R., Schultz, M. C., Crawford, A. B., & Sinner, C. A. Westen, D., & Morrison, K. (2001). A multidimensional
(1999). Single-subject clinical-outcome research: meta-analysis of treatments for depression, panic,
Designs, data, effect sizes, and analyses. Aphasiology, and generalized anxiety disorder: An empirical exam-
13, 445–473. doi:10.1080/026870399402028 ination of the status of empirically supported thera-
Seligman, M. E. P. (1995). The effectiveness of psycho- pies. Journal of Consulting and Clinical Psychology, 69,
therapy: The Consumer Reports study. American 875–899. doi:10.1037/0022-006X.69.6.875
Psychologist, 50, 965–974. doi:10.1037/0003-066X. Westen, D., Novotny, C. M., & Thompson-Brenner, H.
50.12.965 K. (2004). The empirical status of empirically sup-
Sharpley, C. F. (1987). Time-series analysis of behav- ported psychotherapies: Assumptions, findings, and
ioural data: An update. Behaviour Change, 4, 40–45. reporting in controlled clinical trials. Psychological
Bulletin, 130, 631–663. doi:10.1037/0033-2909.
Sharpley, C. F., & Alavosius, M. P. (1988). 130.4.631
Autocorrelation in behavioral data: An alternative
perspective. Behavioral Assessment, 10, 243–251. Wilcox, R. R. (2001). Fundamentals of modern statistical
methods: Substantially improving power and accuracy.
Sidman, M. (1960). Tactics of scientific research. New New York, NY: Springer-Verlag.
Wilson, J. B. (2007). Priorities in statistics, the sensitive
Skinner, B. F. (1938). The behavior of organisms. New feet of elephants, and don’t transform data. Folia
York, NY: Appleton-Century-Crofts. Geobotanica, 42, 161–167. doi:10.1007/BF02893882
266
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 12
New Methods for Sequential

Behavior Analysis
Peter C. M. Molenaar and Tamara Goode
Sequential behavior analysis is firmly rooted in sequential behavior analysis, such as those we
learning theory (Sharpe & Koperwas, 2003). The describe here, no longer require the quantification
identification and analysis of antecedents and conse- of these dependencies in this manner. The inclusion
quences of target behavior are basic to behavior the- of the state in the model, which we discuss in the
ory; sequential behavior analysis arranges and next section, both simplifies these models and
assesses explicit temporal dependencies between allows for the estimation of more complex systems
present and past behavior, and present and future and processes.
behavior, using time-series data. Behavior theory is In the following, our focus is new statistical mod-
currently expanding in exciting new directions such els describing the sequential dependencies of behav-
as artificial neural network simulation (Schmajuk, ior processes. The statistical models are derived as
1997) and neuroscience (Timberlake, Schaal, & special instances of a general dynamic systems
Steinmetz, 2005). This expanded scope of behavior model—the so-called state space model (SSM). The
theory presents new challenges to sequential behav- SSM allows for the estimation of a complex system
ior analysis, as do recent developments in psycho- that includes inputs, outputs, and states. A simple
metrics and measurement. In this chapter, we but useful example of such a system is one that
review new methods for sequential behavior analysis many students experience in an introductory psy-
that can be instrumental in meeting these chology lab: that of training a rat to press a lever. An
challenges. input could be a tone signaling that reinforcement is
The earliest methods for analyzing time-series available, the state could be hunger, and the output
data were simple curve-fitting procedures, in which could be the pressing of the lever. The general SSM
slopes and intercepts were compared pre- and not only provides for a transparent classification of a
postintervention. These methods are greatly limited, wide range of behavioral process models but also
owing to ubiquitous violations of the assumption enables the specification of a general estimation
that data collected in two different conditions are scheme to fit these models to the data. Moreover,
independent. A generating function procedure has and more important, the general SSM constitutes an
also been used, in which signal (process) is sepa- excellent paradigm to introduce optimal control of
rated from noise (error). Curve-fitting procedures behavior processes—a powerful extension of
and generating function procedures both quantify sequential behavior analysis that, to the best of our
the dependence of present observations on those knowledge, is considered here for the first time. In
preceding (also known as lead–lag relations; Gott- behavior analysis, optimal control is understood to
man, McFall, & Barnett, 1969). Newer methods for mean the least variability possible in behavior as a
We gratefully acknowledge funding provided by National Science Foundation Grant 0852147, which made Molenaar’s work possible, and the
University Graduate Fellowship provided by the Pennsylvania State University, which made Goode’s work possible.
DOI: 10.1037/13937-012
267
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Molenaar and Goode
function of maximally controlling the environmen- obtained in individual and group analyses have been
tal variables. Here, we are using optimal control as noted (Aoki, 2002). This lack of lawful relations
engineers use the term, which is somewhat similar. between individual and aggregated data pertains not
As in behavior analysis, variability in the output, or only to mean trends but to all aspects of probability
behavior that is being modeled, is minimized. How- distributions (variances, correlations, etc.). From a
ever, researchers are also able to specifically model theoretical point of view, a main emphasis of devel-
the desired level of the behavior and estimate the opmental systems theory (Ford & Lerner, 1992) is
level of those variables that influence the behavior. that developmental processes should be investigated
Additionally, researchers can set parameters to mini- at the individual level, not the group level. In this
mize cost of any of those input variables, that is, chapter, we deal with analysis at the individual level,
determine the minimum level of input (e.g., medica- on the basis of these and other findings that illus-
tion, reinforcement) that is necessary to obtain the trate the problems of relations between aggregated
desired level of behavior. Thus, in this chapter we and individual findings (for a mathematical discus-
discuss the SSM model, which has been applied, sion of this problem and the theory of ergodicity, see
albeit sparingly, to sequential behavior analysis. We Molenaar, 2004, 2008).
conclude by introducing optimal control models,
which have not to our knowledge been applied to
Dynamic Systems Models and
sequential behavior analysis but have the potential
General State Space Models: A
to more precisely model variables controlling behav-
Scheme for Sequential Analysis
ior as noted earlier. This model can then be applied
to whatever process is under examination, and In what follows, we refer to the unit of analysis
behavior can be controlled much more precisely. (individual subject, dyad, triad, etc.) as a system.
The presentation in this chapter is heuristic, The behavior of a system unfolds in time, constitut-
emphasizing the interpretation of process models ing a behavior process. Hence, the appropriate class
while providing published references for further of models for behavior processes is the class of
details. Some formal notation is indispensible, and dynamic systems models, that is, models that can
we define it in the course of the development of the represent change over time. That change may be lin-
different methods. We first discuss, however, indi- ear over the course of the observation, changing
vidual versus group analyses, which has important consistently from one unit of time to the next; how-
implications for any sequential analysis of behav- ever, nonlinear change is more common. For exam-
ioral processes. ple, most representations of a complete learning
process are represented by an initial asymptote at 0,
during which the target behavior is not observed at
Individual Versus Group Analysis
all. During the learning process, the target behavior
The question of the relation between individual and is observed more and more frequently; finally, the
aggregated models of behavior—the application of target behavior asymptotes again when fluency is
group findings to an individual—has a long history. achieved. This process is most often represented by
Sidman (1960) reviewed the older literature, rang- the nonlinear logistic function (Howell, 2007).
ing from Merrell (1931) to Estes (1956), and built a The class of dynamic systems models is large,
case for single-case designs while criticizing group- encompassing linear and nonlinear time-series
data experiments (see also Chapters 5 and 7, this models, artificial neural network models, and many
volume). Hannan’s (1991) monograph is devoted to more. A considerable variety of dynamic systems
the issue of aggregation and disaggregation in the models, however, can be conceived of as special
social sciences, that is, the relation (if any) between cases of the SSM. In particular, most dynamic
individual and group analyses. This is also a major systems models that are of interest for sequential
theme in macroeconomics, in which similar negative analysis of behavioral processes are special cases of
conclusions about the relation between results the SSM. In this section, the SSM serves as the
268
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
New Methods for Sequential Behavior Analysis
organizing principle in presenting a range of different from zero, and specifically, for every one-
dynamic models for the sequential analysis of unit change in the latent state at time t − 1, one
behavioral processes. would expect to see a change in the latent state at
time t of 0.6 units.
Orientation and Notational Conventions
In the material that follows, we denote matrices (i.e., Linear Gaussian SSM
two-dimensional data structures [arrays] composed We start with the linear Gaussian SSM. The term
of rows and columns) by capital boldface letters and Gaussian refers to a normally distributed data set; a
vectors (one-dimensional arrays; i.e., a single col- common example of a Gaussian distributed data set
umn of rows) by lowercase boldface letters. The is the heights of adult men. An example of a simple
superscript T denotes transposition (interchanging linear Gaussian model is the relation of adult men’s
rows and columns). Because vectors are column vec- height and weight; generally speaking, as height
tors, transposing a vector makes it a row vector. increases, weight increases proportionally at a
Random processes are not denoted in a special way; constant rate.
which processes are random is defined in the text. The linear Gaussian SSM consists of two sub-
Manifest (observed) processes are denoted by models: the measurement submodel and the
Roman letters; latent processes are denoted by dynamic submodel. As we describe in subsequent
Greek letters. sections of this chapter, this decomposition into
A manifest process is that which is measured (e.g., measurement and dynamic submodels applies to
the number of problem behaviors a child with autism each SSM, not just the Gaussian SSM. The measure-
exhibits in a period of time). A latent process is a pro- ment submodel of the Gaussian SSM is made up of a
cess, state, or class of behavior that one is measuring Gaussian (normally distributed) behavioral process
with manifest variables. For example, for these mod- that has been observed at equidistant measurement
els one may label aggression a latent state and mea- occasions (e.g., 20-minute observation sessions con-
sure it by observing several behaviors, such as hair ducted at the same time each day), a latent Gaussian
pulling, hitting, biting, and kicking. A more technical state process (to be explained shortly) η, a manifest
definition is that the latent state contains all available (observed) input series u, and a Gaussian error pro-
information about the history of the relevant behav- cess . Thus, the measurement submodel specifies
ior or class of behavior. For example, Kuhn, Hard- that the behavior process y at time t is a function of
esty, and Luczynski (2009) conducted a functional the individual mean (μ), the latent process, the
analysis of antecedent social events and their effect input, and error:
on motivation to access preferred edible items by an
y(t) = μ + Λη(t) + Πu(t) + (t), (1a)
individual with severe problem behavior. If one was
to analyze these data using an SSM, the manipulation with Λ and Π containing regression coefficients
of antecedent social events by having the therapist (i.e., values allowed to vary so as to provide a better
consume food items would be the inputs in the SSM fit to the data; these values may be interpreted in
and the change in the participant’s motivation, as ways discussed later). Equation 1a can be inter-
reflected by decreased socially appropriate respond- preted as a traditional factor model of behavior, one
ing, would be the latent state in the SSM. in which the latent state process η(t) contains all
We also discuss parameter estimation, which is available information about the history of the behav-
the quantification of the relation between two vari- ior process y(t).
ables and the assessment of whether that relation is The dynamic submodel of the linear Gaussian
significant. In the example for the linear Gaussian SSM is expressed as
SSM presented in the next section, the relation
η(t) = κ + βη(t − 1) + Γu(t) + ζ(t). (1b)
between the latent state at time t − 1 and the latent
state at time t is 0.6 and is significant, which means The dynamic model links the previous time point
that the relation between the two time points is t − 1 to the current time point t, which is to say that
269
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Molenaar and Goode
the previous state affects the present state. We spec- manipulation) and 1 for the introduction of the
ify the latent Gaussian process η at time t (i.e., the intervention condition. These specifications yield
left side of Equation 1b) as a function of the mean the following special case of the linear Gaussian
κ, η at the previous time point (t − 1), input u at SSM, in which Λ = 1 and (t) = 0; the state process
time t, and error ζ. The matrix β contains regression is being treated as observed (i.e., no free parameter
coefficients linking the past state process η(t − 1) is used in scaling the Gaussian state process and no
to the current state process η(t). The matrix Γ also error); thus, we can drop them from the measure-
contains regression coefficients, values that quantify ment submodel:
the effect of the input u(t) on the state process η(t).
The degree to which one cannot predict η(t) given Measurement submodel: y(t) = η(t) + Πu(t).
η(t − 1) is represented by ζ(t). The vector κ con- Dynamic submodel: η(t) = βη(t − 1) + ζ(t). (2)
tains mean levels (intercepts).
Similarly, κ and Γu(t) are dropped from the
Examples. For the first example, we refer to dynamic submodel because the model is centered so
Glass, Willson, and Gottman (1975) and their argu- that the mean is 0; thus, κ = 0 and u(t) = 0 for the
ment for interrupted time series as an alternative baseline condition and 1 for the experimental
to experimental designs. Here we have a simulated condition.
single-case univariate time series y(t) obtained dur- Figure 12.1 illustrates these simulated data. The
ing a baseline condition lasting from t = 1 until continuous line depicts y(t) (our behavioral out-
t = T1 (the end of the baseline condition) and an come); the broken line, u(t) (our input, which, in
experimental condition lasting from t = T1 + 1 until our example, would correspond to pre- and postint-
T2. We set T1 = 50 and T2 = 100. For example, this ervention). The obtained estimates of the parameters
simulation could represent the number of appropri- (designated with a caret above each parameter) in
ate responses emitted before and after the introduc- Equation 2 are ̂ = 0.9 and ̂ = 0.6, and the esti-
tion of some behavioral intervention with a child mated variance ̂ 2 of error ζ(t) is ̂ 2 = 0.9 (̂ is the
diagnosed with autism. The mean μ is 1 for the estimate of the effect on the outcome of the experi-
baseline condition; our input u(t) = 0 for the base- mental manipulation). All parameter estimates are
line condition (i.e., the absence of the intervention significant at nominal α = .01 (details not presented),
3
Magnitude
-1 y(t)
u(t)
-2
-3
0 10 20 30 40 50 60 70 80 90 100
Time
Figure 12.1. Single-subject time series with experimental manipulation at

time = 51.
270
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
which implies that ̂ = 0.9 is a significant effect of that the HMM would be used when the outcome,
the experimental intervention condition and that y(t), is known but the state is unknown, or hidden
̂ = 0.6 is a significant effect of η at the previous (i.e., latent, as in the Gaussian linear SSM example);
time point t − 1; in other words, there is a signifi- a Markov model is one in which the state can be
cant relation between the quantified value of the determined simply by the value of y(t). Recalling
latent state η at time t − 1 on time t. that in a behavioral analysis, y(t) is the behavioral
For example, if the state being targeted was outcome at a given point in time (e.g., one of the
aggressive behavior, as measured by instances of hit- data points in Figure 12.1), the experimenter is
ting, kicking, and biting, the significant result of η interested in the organism’s state so as to predict the
at the previous time point t − 1 would indicate that next behavioral event y(t + 1), or the next state.
the level of aggressive behavior at time point t − 1 The HMM is used to estimate the probability of
has a significant relation to the level of aggressive transition from one state to another, given the
behavior at time point t. observed data. An HMM could be used to analyze
An example of data from applied behavior analy- the transitions between different categories of
sis in which this model might be applied is in Kuhn behavior; for example, observational data could be
et al. (2009). They conducted a functional analysis gathered over time of problem behavior (e.g.,
of antecedent social events and their effect on moti- aggression, self-injurious behavior, and disruption).
vation to access preferred edible items by an individ- These data could then be analyzed using an HMM,
ual with severe problem behavior. Manipulating which would result in the probabilities of transition
antecedent social events by having the therapist con- from one state (category of problem behavior) to
sume food items (inputs, u[t]) changed the partici- another. The resulting probabilities could be used to
pant’s motivation (as measured by an increase or more precisely tailor a behavioral intervention. For
decrease in socially inappropriate responding), a example, if the target behavior was self-injurious
latent state, η. Although Kuhn et al.’s analysis did behavior, and an HMM revealed that the probability
not explore a sequential relation between motivation of transition from aggression to self-injurious behav-
at time t − 1 and time t, it is possible that analysis of ior was much higher than the probability of transi-
these data using the linear Gaussian SSM would tion from aggression to disruption, intervention
identify any such dependencies, assuming sufficient could be targeted more precisely before the occur-
occasions of data (a total of 100 occasions or more) rence of self-injurious behavior.
were collected to perform the analysis. The defining equations for the HMM are similar
This first example involves what is perhaps the to Equations 1a and 1b defining the linear Gaussian
simplest possible instance of a linear Gaussian SSM. SSM. For the HMM, y(t) is a categorical process
The linear Gaussian SSM can be straightforwardly with p categories—the number of states. The latent
extended to continuous time (Simon, 2006). state process η(t) is also a categorical process. The
standard HMM can be defined as follows:
Implementation. These analyses can be implemented
in the Fortran computer program MKFM6, which uses Measurement submodel: y(t) = Λη(t) + (t). (3a)
an expectation-maximization (EM) algorithm to fit lin- Dynamic submodel: η(t) = βη(t − 1) + ζ(t). (3b)
ear Gaussian SSMs. The EM algorithm is more efficient
Λ is the probability that y(t) is a certain category,
than the more commonly used maximum likelihood
given a value of η(t). β is the conditional probability
algorithm. This computer program can also estimate
that η(t) is a certain category, given a value of η(t − 1).
multivariate Gaussian time series obtained with mul-
The measurement error process (t) and the process
tiple subjects in a replicated time-series design.
noise ζ(t) are categorical processes with a priori
Hidden Markov Model fixed properties (Elliott, Aggoun, & Moore, 1995).
A hidden Markov model (HMM) is similar in many Examples. Until now, applications of HMMs for the
ways to the linear Gaussian SSM. One similarity is sequential analysis of behavior processes have been
271
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Molenaar and Goode
rare. Notable exceptions are Courville and Touretzky We can similarly understand the elements of the
(2002) and Visser, Raijmakers, and Molenaar (2007). Λ matrix as shown in Table 12.2. When the present
We discuss Visser, Raijmakers, and Molenaar’s state is the guessing state (η[t] = 1), the probability
(2002) application of the HMM to implicit learning, that the outcome is 1 (continued guessing) is .7;
or learning without awareness, of a rule-governed when the present state is the learned state (η[t] = 2),
sequence, using simulated data to illustrate. The the probability that the outcome is 1 is 0; when the
data were simulated according to a state process η(t) present state is the guessing state (η[t] = 1), the prob-
with two categories (q = 2) to represent two states, ability that the outcome is 2 is 0; when the present
an unlearned state and a learned state. The observed state is the unsure state (η[t] = 3), the probability
process y(t) was simulated with three categories that the outcome is 1 is .3.
(p = 3) to represent three possible responses; that is, Figure 12.2 depicts the observed process y(t)
three different buttons that participants could press and latent state process η(t). The category values
in sequence during the task to represent the rule of the latent state process have been increased by
governing the observed sequence. In this instance, four units for clarity. The estimates of the condi-
the probabilities for η(t) given the values of η(t − 1) tional probabilities specified in Tables 12.1
contained in the matrix β are shown in Table 12.1. and 12.2 are very close to their true values (see
Thus, when the previous state is the guessing state Visser et al., 2002, for further details). As can be
(η[t − 1] = 1), the probability that the current state observed by the plot of the data, when the state
is also the unlearned state is .9; when the previous is in the learned category (a value of 6 on the
state is the learned state (η[t − 1] = 2), the probabil- y-axis), the response emitted is more likely to be
ity that the current state is the guessing state is .3. in Categories 2 or 3.
Table 12.1 Table 12.2

Probabilities of η(t) Given η(t − 1) = 1 and Probabilities of y(t) Given η(t) = 1 and η(t) = 2
η(t − 1) = 2
y(t) = (t) = 1 (t) = 2
(t) = (t − 1) = 1 (t − 1) = 2
1 .7 0
1 .9 .3 2 0 .4
2 .1 .7 3 .3 .6
5
Category Number
1 Y
η
0
0 10 20 30 40 50 60 70 80 90
Time
Figure 12.2. Hidden Markov model process y and latent process η.
272
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
An HMM could be used to analyze the probability regression coefficients. The process noise ζ(t) again
of transition among different categories of problem represents the lack of predictability of η(t) given η(t − 1).
behavior in an individual with mental retardation, as The vector κ contains mean levels (intercepts).
in Kuhn et al. (2009). Observational data of stereo-
Example. An example of data from the behavior
typic movements, self-injurious behavior, and dis-
analysis literature that could be analyzed using this
ruptive behavior could be collected over time. If the
approach is found in Cunningham (1979). The pre-
target behavior is disruptive behavior, an HMM may
diction of extinction or conditioned suppression of
reveal, for example, that the probability of transition-
a conditioned emotional response y(t) as a function
ing from stereotypic movement to disruptive behav-
of sobriety η(t) could be assessed. Cunningham
ior is lower than the probability of transitioning from
trained a conditioned emotional response in sober
self-injurious behavior to disruptive behavior. An
rats. Extinction was conducted under conditions
intervention could then be designed accordingly and
of high doses of alcohol. Conditioned suppression
targeted more precisely.
returned when those rats were in a sober condition.
The GLDM could be used to determine at what level
Parameter estimation. Parameter estimation in
of sobriety (e.g., different doses of alcohol) condi-
HMMs and hidden hierarchical Markov models can
tioned suppression returns.
be accomplished by means of the EM algorithm
mentioned in the Linear Gaussian SSM section. This Parameter estimation. Parameter estimation in
algorithm, when used for HMMs and hidden hierar- the GLDM is accomplished by means of the EM
chical Markov models, is known as the Baum–Welch algorithm described in the Linear Gaussian SSM sec-
(forward–backward) algorithm. It has been imple- tion. The nonlinear response function in Equation
mented in the R software package DEPMIXS4. Visser 4a necessitates a special approach to carry out the
and Speekenbrink (2010) provided illustrative estimation of η(t) (Fahrmeir & Tutz, 2001). A beta
examples and instruction. Fraser (2008) presented a version of the computer program concerned can be
transparent derivation of this algorithm. obtained from Peter C. M. Molenaar.
Generalized Linear Dynamic Model Linear Gaussian SSM With

Another instance of the SSM is the generalized linear Time-Varying Parameters
dynamic model (GLDM). In the GLDM, the mani- The SSMs considered in the previous sections all
fest process y(t) is categorical (as in the standard have parameters that are constant—consistent in
HMM) and the latent state process η(t) is linear (as both level and pattern—over time. Yet learning and
in the linear Gaussian SSM). This model can be con- developmental processes often have changing statis-
ceived of as a dynamic generalization of log-linear tical characteristics that can only be captured accu-
analysis and serves as an alternative to the HMM. rately by allowing model parameters to be time
The GLDM is defined as varying. For example, erratic environments with
inconsistent contingencies have been associated
y(t) = λ[η(t)] + ε(t) (4a)
with the development of attachment problems in
and children (Belsky, Garduque, & Hrncir, 1984).
Several model types exist for the sequential anal-
η(t) = κ + βη(t − 1) + Γu(t) + ζ(t). (4b)
ysis of processes with changing statistical character-
These formulas are similar to the formulas in the istics (so-called nonstationary processes), several of
HMM and Linear Gaussian SSM sections. In the which are described by Priestley (1988) and Tong
GLDM, λ[η(t)] is a nonlinear function of η(t), the so- (1990). Here, we focus on a direct generalization of
called response function (Fahrmeir & Tutz, 2001). the linear Gaussian SSM, allowing its parameters to
The matrix β again contains regression coefficients be time varying in arbitrary ways. This approach is
linking the prior state process η(t − 1) to the current based on related work by Young, McKenna, and
state process η(t). The matrix Γagain also contains Bruun (2001) and Priestley (1988).
273
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Molenaar and Goode
For this generalization, we introduce a new vec- unknown parameters. The process ξ(t) represents
tor to our formulas, θt, which contains all unknown lack of predictability of θt. If an element of ξ(t) has
parameters. Notice that θt is time varying, as indi- zero (or small) variance, then the corresponding
cated by the subscript t. The individual parameters parameter in θt is (almost) constant in time, whereas
in θt may, or may not, be time varying; one of the this parameter will be time varying if the variance of
tasks of the estimation algorithm is to determine this element of ξ(t) is significantly different from zero.
which individual parameters in θt are time varying
Examples. One example of the application of this
and, if so, how they vary in time. With this specifica-
model is based on data gathered from a father and
tion, the linear Gaussian SSM with time-varying
his biological son. This application could allow an
parameters is obtained as the following generaliza-
intervention to be planned and implemented to
tion of Equations 1a and 1b:
reduce escalating negative interactions that might
y(t) = μ[θt] + Λ[θt]η(t) + Π[θt]u(t) + ε(t), (5a) improve the long-term relationship between father
η(t) = κ[θt] + β[θt]η(t − 1) and son. These data were presented by Molenaar
+ Γ[θt]u(t) + ζ(t), (5b) and Campbell (2009) in what appears to be the first
application of this model to psychological process
and data. The data concern a three-variate time series of
θt = θt − 1 + ξ(t). (5c) repeated measures of emotional experiences of sons
with their biological fathers during 80 consecutive
The interpretation of the parameter matrices in
interaction episodes over a period of about
Equations 5a and 5b is the same as for Equations 1a
2 months. The self-report measures collected at the
and 1b, but now each parameter matrix or vector
conclusion of each episode were involvement, anger,
depends on elements of the vector θt in which all
and anxiety. Here we focus on a single biological
unknown parameters are collected. For instance,
son, whose data are depicted in Figure 12.3.
Λ[θt] specifies that the unknown parameters in the
The following instance of Equations 5a through
matrix Λ are elements of the vector θt of time-
5c was used to analyze the data in Figure 12.3:
varying parameters. Equation 5c is a new addition,
tracking the possibly time-varying behavior of the y(t) = η(t), (6a)
Figure 12.3. Observed series for biological son. From “Analyzing

Developmental Processes on an Individual Level Using Nonstationary Time
Series Modeling,” by P. C. M. Molenaar, K. O. Sinclair, M. J. Rovine, N. Ram,
and S. E. Corneal, 2009, Developmental Psychology, 45, p. 263. Copyright 2009
by the American Psychological Association.
274
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
η(t) = β[θt − 1]η(t − 1) + ζ(t), (6b) episodes, after which it stabilizes during the final
half of the interaction episodes. β3(t) in Equation 7,
and
which quantifies the effect of anxiety at the previous
θt = θt −1 + ξ(t). (6c) interaction sequence on involvement at the next
The matrix β[θt] contains the possibly time- interaction sequence, increases across the initial half
varying regression coefficients linking η(t) to of the interaction episodes, after which it stabilizes
η(t − 1). Here we only consider the part of the during the final half of the interaction episodes. It is
model explaining the involvement process. This part noteworthy that β3(t) is negative during the initial
can be represented as interaction episodes, but positive during the later
interaction episodes. When β3(t) is negative, there is
Inv(t) = β1(t − 1) * Inv(t − 11) a negative relation between anxiety at time t − 1 and
+ β2(t − 11) * Ang(t − 11) involvement at time t, that is, increased anxiety pre-
+ β3(t − 11) * Anx(t − 11) + ζ(t), (7) dicts decreased involvement at each subsequent
where Inv = involvement, Ang = anger, and interaction or decreased anxiety predicts increased
Anx = anxiety. Thus, involvement at time t is a func- involvement at each subsequent interaction. When
tion of involvement at the previous time point (t − 1), β3(t) is positive, there is a positive relation between
anger at the previous time point (t − 1), and anxiety anxiety at time t − 1 and involvement at time t, that
at the previous time point. The beta coefficients at is, increased anxiety predicts increased involvement
time t in Equation 7 indicate that involvement, anger, at each subsequent interaction or decreased anxiety
and anxiety vary with time; the coefficient quantifies predicts decreased involvement at each subsequent
the relation between time t and time t − 1. interaction. When β3(t) is zero, anxiety at time t − 1
Figure 12.4 shows the estimates of β1(t), β2(t), has no effect on involvement during the subsequent
and β3(t) across the 80 interaction episodes and interaction. Additional procedural details and data
illustrates the intertwined nature of involvement, can be found in Molenaar, Sinclair, Rovine, Ram,
anger, and anxiety. β1(t), which quantifies the effect and Corneal (2009).
of involvement at time t − 1 on involvement at time t An example of data from the behavior analysis
decreases across the initial half of the interaction literature that could be analyzed using this approach
Figure 12.4. Involvement at t + 1. From “Analyzing Developmental

Processes on an Individual Level Using Nonstationary Time Series Modeling,”
by P. C. M. Molenaar, K. O. Sinclair, M. J. Rovine, N. Ram, and S. E. Corneal,
2009, Developmental Psychology, 45, p. 267. Copyright 2009 by the American
Psychological Association.
275
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Molenaar and Goode
can be found in McSweeney, Murphy, and Kowal differential equations to analyze bifurcations (sud-
(2004). They examined habituation as a function of den qualitative reorganizations) in human motion
reinforcer duration. Rate of responding y(t) as a dynamics. Haccou and Meelis (1992) gave a compre-
function of habituation η(t) as habituation varies hensive treatment of continuous time Markov chain
over time β[θt − 1]η(t −1) could be assessed more models for the analysis of ethological processes.
precisely using a linear Gaussian SSM with time-
varying parameters.
Remarks on Optimal-Control-of-
Behavior Processes
Parameter estimation. Parameter estimation in
Equations 5a through 5c is accomplished by the EM The concepts of homeostasis and control have long
algorithm. The estimation step requires considerable played a role in behavior theory (Carver, 2004;
reformulation of the original model, resulting in a Johnson, Chang, & Lord, 2006; McFarland, 1971).
nonlinear analogue. A beta version of the Fortran This role, however, has been confined to the theo-
program with which the linear Gaussian SSM with retical level. If one of the SSMs considered earlier
time-varying parameters is fitted to the data can be has been fitted to a behavioral process, then one
obtained from Peter C. M. Molenaar. can use this model to optimally control the process
as it unfolds by means of powerful mathematical
SSMs in Continuous Time techniques. Here, we present optimal control as a
The class of SSMs in continuous time is large, computational technique to steer a behavioral pro-
including (non)linear stochastic differential equa- cess to desired levels. The fitting of this model to a
tions, Fokker-Planck equations, and so forth, and complex behavioral process, such as the develop-
therefore cannot be reviewed within the confines of ment of aggressive behavior, allows the precise esti-
this chapter. One application of SSMs in continuous mation of parameters that can then be applied to
time, the point process, could be applied to data ameliorate the problem behavior and correct those
reported by Nevin and Baum (1980). Point pro- problems in an efficient manner.
cesses can be used to determine the timing of a To introduce the concept of computational feed-
repeated discrete response. Nevin and Baum dis- back, consider the following simple behavior pro-
cussed probabilities of the termination and initiation cess model in discrete time:
of a responding burst in free-operant behavior. An
y(t + 1) = βy(t) + γu(t) + ε(t + 1). (8)
assumption of one model discussed is that, during a
burst, interresponse times are approximately con- Here y(t) denotes a manifest univariate behavior
stant. A point process could test this assumption as process, u(t) is a univariate external input, and
well as whether the transition from responding to (t + 1) is process noise. From an intervention per-
termination is a function of that timing. Rate of spective, the desired level of the behavioral process
responding and rate of reinforcement data could be is y*, and achieving this level is obtained by experi-
analyzed using the point process, and the results mentally manipulating u(t) (e.g., medication, rein-
would indicate whether interresponse times are con- forcement contingency).
stant or varied during a burst and whether the tran- Given Equation 8 and y*, the goal of the compu-
sition from responding to termination is a function tational feedback problem is to set the value of u(t)
of constant interresponse time or varied interre- (i.e., the level of the value of the independent vari-
sponse time. able) in a way that minimizes the difference between
For excellent reviews of SSMs in continuous the expected value of y(t) and y*. To do this
time, see, for instance, Gardiner (2004) and Risken requires an estimate of the expected value of the
(1984). The linear Gaussian SSMs considered here behavior of interest given the extant record of that
are special in that the analogues in continuous time behavior up to time t − 1: E[y(t|t − 1)]. The devia-
share most of their statistical properties. Molenaar tion between expected and desired behavior is
and Newell (2003) applied nonlinear stochastic referred to as a cost function. A simple cost function
276
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
is C(t) = E[y(t|t − 1) − y*]2. Again, the external deviation of the optimally controlled behavioral
input u(t) will be chosen in a way that minimizes process from y* = 0 is much smaller than the
C(t) for all times t. Because E[y(t|t – 1)] = βy(t) + analogous deviation of the behavioral process without
γu(t), the optimal level of the independent variable, control Y.
u*(t), is given by This example of optimal feedback control is sim-
ple in several respects. The behavioral process to be
u*(t) = [−βy(t) + y*] / γ. (9)
controlled is univariate, or a single dependent vari-
Equation 9 is a feedback function in which the able; in actuality, both are often multivariate. For
optimal input u*(t), which will codetermine the example, escalating aggression may be measured by
value y(t + 1), depends on y(t). Thus, the optimal frequency and co-occurrence of several other
input for time t + 1 to steer the behavior y to an opti- responses, such as hair pulling, slapping, hitting,
mal value y* is dependent on the value of y at time t. kicking, and scratching. Also, Equation 8 assumes a
known linear model. In reality, the model can be
Examples. Figure 12.5 depicts a manifest behav- any one of the SSMs considered previously, with
ioral process y(t), t = 1, . . ., 50 (labeled Y). This parameters that are unknown and have to be esti-
y(t) is uncontrolled (i.e., not subject to any experi- mated. In our simple example, the cost function is a
mental manipulation) and was simulated according to quadratic function that does not penalize the costs
Equation 8) with β = 0.7 and γ = 0.9. The goal of exercising control actions. In general, both the
of our intervention will be to achieve a desired level deviations of the behavioral process from the desired
of behavior, which we have set at zero, y* = 0. level and the external input (e.g., medication, rein-
Application of Equation 9 yields the u*(t) values forcement contingency) are penalized according to
shown in the optimal external input function in separate cost functions that can have much more
Figure 12.5. Application of these optimal values of intricate forms. This example, however, serves to
the independent variable, u*(t), in Equation 8 yields illustrate that control of behavior can be quantified
the optimally controlled behavioral process that is and estimated, thus allowing for much more precise
labeled optimally controlled behavioral process in and efficient manipulations of experimental vari-
Figure 12.5. It is evident from Figure 12.5 that the ables designed to modify behavior.
15
10
Levels of Control and Response
-5
-10
Y
-15 UO
YO
-20
0 5 10 15 20 25 30 35 40 45
Time
Figure 12.5. Simple optimal feedback control. UO = optimal external input;

Y = behavioral process without control; YO = optimally controlled behavioral
process.
277
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Molenaar and Goode
An example from the behavior-analytic literature states. Several instances of this general SSM were
in which this analysis might be applied is an extension presented; however, the SSM covers a much broader
of the Kuhn et al. (2009) study mentioned earlier in range of dynamic models. For instance, a close cor-
this chapter. Once the evaluation of antecedent respondence exists between artificial neural network
events to problem behavior had been completed, models and state space modeling (Haykin, 2001).
and those events that led to reduction of target This correspondence opens up the possibility to
behavior identified, implementation of the treatment reformulate artificial neural network models as spe-
plan could be assessed using an optimal control cific instances of nonlinear SSMs. For instance, the
feedback model. neural network models considered by Schmajuk
In general, well-developed mathematical theories (1997, 2002) are variants of adaptive resonance the-
exist for optimal feedback control in each of the ory networks that, in their most general form, con-
types of SSMs considered here. Kwon and Han stitute coupled systems of nonlinear differential
(2005) presented an in-depth description of optimal equations (Raijmakers & Molenaar, 1996) obeying
control in linear SSMs (see also Molenaar, 2010); the general nonlinear SSM format. Similar remarks
Elliott et al. (1995) is the classic source for optimal apply to reinforcement networks (Wyatt, 2003).
control in HMMs; Costa, Fragoso, and Marques Our hope is that this presentation of the class of
(2005) discussed optimal control in SSMs with models known as SSMs has illustrated how they can
regime shifting (i.e., systems models for processes be useful in behavior analysis—specifically in mod-
undergoing sudden changes in their dynamics). eling behavior and behavior change as a process and
to include all aspects of the behavior change pro-
cess: the state of the organism, the inputs and out-
Conclusion
puts occurring during the process, and the feedback
Because the class of dynamic systems models is too that occurs as the process continues. Use of these
large to comprehensively cover within the confines of models can enhance the further development and
this chapter, we have reviewed several simple exam- growth of the field.
ples of this class of models: the linear Gaussian SSM,
the HHM, the hierarchical hidden Markov model, the References
GLDM, the linear Gaussian SSM with time-varying Aoki, M. (2002). Modeling aggregate behavior and fluc-
parameters, and SSMs in continuous time. Notable tuations in economics: Stochastic views of interacting
omissions from our review are regime shifts (Costa agents. Cambridge, England: Cambridge University
Press.
et al., 2005), modeling processes having long-range
sequential dependencies (Palma, 2007), and chaotic Belsky, J., Garduque, L., & Hrncir, E. (1984). Assessing
performance, competence, and executive capacity
processes having dynamics evolving at multiple time in infant play: Relations to home environment and
scales (Gao, Cao, Tung, & Hu, 2007). These types of security of attachment. Developmental Psychology, 20,
process models are of interest for sequential behavior 406–417. doi:10.1037/0012-1649.20.3.406
analysis (Torre & Wagenmakers, 2009) and again Carver, S. C. (2004). Self-regulation of action and affect.
can be formulated as special instances of SSMs; how- In R. F. Baumeister & K. D. Vohs (Eds.), Handbook of
self-regulation (pp. 13–39). New York, NY: Guilford
ever, their discussion requires consideration of much Press.
more technical concepts, and for that reason, we have Costa, O. L. V., Fragoso, M. D., & Marques, R. P. (2005).
omitted them from this review. We therefore refer to Discrete-time Markov jump linear systems. London,
the preceding excellent references given for further England: Springer-Verlag.
details about these model types. Courville, A. C., & Touretzky, D. S. (2002). Modeling
These new methods for the sequential analysis of temporal structure in classical conditioning. In T.
J. Dietterich, S. Becker, & Z. Ghahramani (Eds.),
behavior processes reviewed here can be conceived Advances in neural information processing systems
of as special cases of a general SSM, a model that (Vol. 14, pp. 3–10). Cambridge, MA: MIT Press.
allows for the quantification of complex systems Cunningham, C. L. (1979). Alcohol as a cue for extinc-
that are made up of inputs, feedback, outputs, and tion: State dependency produced by conditioned
278
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
inhibition. Animal Learning and Behavior, 7, 45–52. McSweeney, F. K., Murphy, E. S., & Kowal, B. P. (2004).
doi:10.3758/BF03209656 Varying reinforcer duration produces behavioral
Elliott, R. J., Aggoun, L., & Moore, J. B. (1995). Hidden interactions during multiple schedules. Behavioural
Markov models: Estimation and control. New York, Processes, 66, 83–100. doi:10.1016/j.beproc.2004.
NY: Springer-Verlag. 01.004
Estes, W. K. (1956). The problem of inference from Merrell, M. (1931). The relationship of individual growth
curves based on group data. Psychological Bulletin, to average growth. Human Biology, 3, 37–70.
53, 134–140. doi:10.1037/h0045156 Molenaar, P. C. M. (2004). A manifesto on psychology as
Fahrmeir, L., & Tutz, G. (2001). Multivariate statistical idiographic science: Bringing the person back into
modeling based on generalized linear models (2nd ed.). scientific psychology, this time forever. Measurement:
Berlin, Germany: Springer-Verlag. Interdisciplinary Research and Perspective, 2, 201–218.
doi:10.1207/s15366359mea0204_1
Ford, D. H., & Lerner, R. M. (1992). Developmental sys-
tems theory: An integrative approach. Newbury Park, Molenaar, P. C. M. (2008). On the implications of the
CA: Sage. classical ergodic theorems: Analysis of develop-
Fraser, A. M. (2008). Hidden Markov models and dynami- mental processes has to focus on intra-individual
cal systems. Philadelphia, PA: Society for Industrial variation. Developmental Psychobiology, 50, 60–69.
and Applied Mathematics. doi:10.1137/1.978089 doi:10.1002/dev.20262
8717747 Molenaar, P. C. M. (2010). Note on optimization of
Gao, J., Cao, Y., Tung, W. W., & Hu, J. (2007). Multiscale individual psychotherapeutic processes. Journal of
analysis of complex time series: Integration of chaos Mathematical Psychology, 54, 208–213. doi:10.1016/j.
and random fractal theory, and beyond. Hoboken, NJ: jmp.2009.04.003
Wiley. Molenaar, P. C. M., & Campbell, C. G. (2009). The new
Gardiner, C. W. (2004). Handbook of stochastic methods: person-specific paradigm in psychology. Current
For physics, chemistry and the natural sciences (3rd Directions in Psychological Science, 18, 112–117.
ed.). Berlin, Germany: Springer-Verlag. doi:10.1111/j.1467-8721.2009.01619.x
Glass, G., Willson, V., & Gottman, J. M. (1975). Design Molenaar, P. C. M., & Newell, K. M. (2003). Direct
and analysis of time-series experiments. Boulder: fit of a theoretical model of phase transition
Colorado University Press. in oscillatory finger motions. British Journal of
Gottman, J. M., McFall, R. M., & Barnett, J. T. (1969). Mathematical and Statistical Psychology, 56, 199–214.
Design and analysis of research using time-series. doi:10.1348/000711003770480002
Psychological Bulletin, 72, 299–306. doi:10.1037/ Molenaar, P. C. M., Sinclair, K. O., Rovine, M. J., Ram, N.,
h0028021 & Corneal, S. E. (2009). Analyzing developmental
Haccou, P., & Meelis, E. (1992). Statistical analysis of processes on an individual level using nonstationary
behavioural data: An approach based on time-structured time series modeling. Developmental Psychology, 45,
models. Oxford, England: Oxford University Press. 260–271. doi:10.1037/a0014170
Hannan, M. T. (1991). Aggregation and disaggregation in Nevin, J. A., & Baum, W. M. (1980). Feedback func-
the social sciences. Lexington, MA: Lexington Books. tions for variable-interval reinforcement. Journal of
Haykin, S. (Ed.). (2001). Kalman filtering and neural net- the Experimental Analysis of Behavior, 34, 207–217.
works. New York, NY: Wiley. doi:10.1002/0471221546 doi:10.1901/jeab.1980.34-207
Howell, D. C. (2007). Statistical methods for psychology. Palma, W. (2007). Long-memory time series: Theory and methods.
Belmont, CA: Thomson. Hoboken, NJ: Wiley. doi:10.1002/9780470131466
Johnson, R. E., Chang, C. H., & Lord, R. G. (2006). Priestley, M. B. (1988). Non-linear and non-stationary
Moving from cognition to behavior: What the time series analysis. London, England: Academic
research says. Psychological Bulletin, 132, 381–415. Press.
doi:10.1037/0033-2909.132.3.381
Raijmakers, M. E. J., & Molenaar, P. C. M. (1997). Exact
Kuhn, D. E., Hardesty, S. L., & Luczynski, K. (2009). ART: A complete implementation of an ART net-
Further evaluation of antecedent social events dur- work, including all regulatory and logical functions,
ing a functional analysis. Journal of Applied Behavior as a system of differential equations capable of stand-
Analysis, 42, 349–353. doi:10.1901/jaba.2009.42-349 alone running in real time. Neural Networks, 10,
Kwon, W. H., & Han, S. (2005). Receding horizon control. 649–669. doi:10.1016/S0893-6080(96)00111-6
London, England: Springer-Verlag. Risken, H. (1984). The Fokker-Planck equation: Methods
McFarland, D. (1971). Feedback mechanisms in animal of solution and applications. Berlin, Germany:
behavior. London, England: Academic Press. Springer-Verlag.
279
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Molenaar and Goode
Schmajuk, N. A. (1997). Animal learning and cognition: Human Movement Science, 28, 297–318. doi:10.1016/
A neural network approach. Cambridge, England: j.humov.2009.01.001
Cambridge University Press. Visser, I., Raijmakers, M. E. J., & Molenaar, P. C. M.
Schmajuk, N. (2002). Latent inhibition and its neu- (2002). Fitting hidden Markov models to psycho-
ral substrates. Norwell, MA: Kluwer Academic. logical data. Science Progress, 10, 185–199.
doi:10.1007/978-1-4615-0841-0 Visser, I., Raijmakers, M. E. J., & Molenaar, P. C. M.
Sharpe, T., & Koperwas, J. (2003). Behavior and sequen- (2007). Characterizing sequence knowledge using
tial analyses: Principles and practice. Thousand Oaks, on-line measures and hidden Markov models.
CA: Sage. Memory and Cognition, 35, 1502–1517. doi:10.3758/
BF03193619
Sidman, M. (1960). Tactics of scientific research. Oxford,
England: Basic Books. Visser, I., & Speekenbrink, M. (2010). depmixS4: An
R-package for hidden Markov models. Journal of
Simon, D. (2006). Optimal state estimation: Kalman,
Statistical Software, 36(7), 1–21.
H∞, and nonlinear approaches. Hoboken, NJ: Wiley.
doi:10.1002/0470045345 Wyatt, J. (2003). Reinforcement learning: A brief over-
view. In R. Kühn, R. Menzel, W. Menzel, U. Ratsch,
Timberlake, W., Schaal, D. W., & Steinmetz, J. E. (Eds.).
M. M. Richter, & I. O. Stamatescu (Eds.), Adaptivity
(2005). Special issue on the relation of behavior
and learning: An interdisciplinary debate (pp. 243–264).
and neuroscience [Special issue]. Journal of the
Berlin, Germany: Springer-Verlag.
Experimental Analysis of Behavior, 84(3).
Young, P. C., McKenna, P., & Bruun, J. (2001).
Tong, H. (1990). Non-linear time series: A dynamical sys-
Identification of non-linear stochastic systems by state-
tem approach. Oxford, England: Clarendon Press.
dependent parameter estimation. International Journal
Torre, K., & Wagenmakers, E. J. (2009). Theories and of Control, 74, 1837–1857. doi:10.1080/002071
models for 1/fβ noise in human movement science. 70110089824
280
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 13
Pavlovian Conditioning
K. Matthew Lattal
The study of Pavlovian conditioning in the labora- preparations. Table 13.1 lists some of the prepara-
tory has been ongoing for more than 100 years, hav- tions that are frequently used in modern studies of
ing documented roots in early demonstrations of Pavlovian conditioning. Common Pavlovian condi-
learning mechanisms involved in stimulus-elicited tioning mechanisms occur with aversive stimuli
reflexes (Twitmeyer, 1905). In 1927, the publication (such as fear conditioning with a shock or flavor
of Pavlov’s Conditioned Reflexes, summarizing years aversion learning induced by nausea) and appetitive
of research in his and his colleagues’ laboratories, stimuli (such as magazine-approach conditioning
laid the groundwork for research and theory on clas- with food or context–drug associations with drugs
sical conditioning, which continues to be a major of abuse). These common findings in many different
focus of behavioral and neurobiological analysis preparations point to the generality of the condition-
today. This book was more than a summary of Pav- ing principles that I describe in this chapter.
lov’s research; it was a blueprint for how a rigorous At a theoretical level, the study of Pavlovian con-
experimental analysis of behavior should be under- ditioning is an area of behavior analysis that has
taken. Although the experimental preparations for long been dominated by approaches that focus on
studying Pavlovian conditioning have expanded internal theoretical mechanisms to explain the
over the years, Pavlov’s work with dogs describes emergence and maintenance of learning (see Pearce
many of the key empirical phenomena and theoreti- & Bouton, 2001). This theoretical approach, in con-
cal processes that modern researchers continue to trast to the Skinnerian approach that has long domi-
pursue. Today, research on Pavlovian (sometimes nated the study of operant behavior, relies heavily
referred to as classical, or respondent) conditioning on internal mechanisms and is explicit in making
is motivated by a theoretical approach that attempts assumptions about how the organism encodes the
to determine the ways in which organisms represent stimulus relations that it encounters in its environ-
their world and learn to respond accordingly (see ment (e.g., Dickinson, 1989; Rescorla, 1988b).
Rescorla, 1988b). The study of basic associative Indeed, it is impossible to characterize the nature of
learning processes underlying Pavlovian condition- Pavlovian research over the past 50 years without
ing has led to numerous insights into experimental describing the central importance of certain internal
design, how learning occurs, and how basic pro- theoretical concepts, such as associative strength
cesses may lay the foundation for many putative and stimulus representations. In many cases, differ-
higher forms of learning. ences in terms are a simple function of differences in
At an empirical level, the key findings that form perspective. Describing certain findings in terms of
the cornerstone of modern thinking about Pavlovian selective stimulus control or selective stimulus rep-
conditioning have been demonstrated in multiple resentations may, in many cases, have the same
Preparation of this chapter was supported by National Institutes of Health Grants DA025922 and MH077111.
DOI: 10.1037/13937-013
283
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
Table 13.1
Common Preparations Used in the Study of Pavlovian Conditioning
Unconditioned Conditioned Unconditioned Representative

Preparation Conditioned stimulus stimulus response response reference
Autoshaping Discrete keylight Food or liquid Keypeck Eating or drinking Brown & Jenkins
(1968)
Eyeblink Diffuse auditory– Airpuff or shock Eyeblink Eyeblink Hupka, Kwaterski, &
visual stimulus Moore (1970)
Fear conditioning Diffuse auditory– Shock Freezing Activity Westbrook,
visual stimulus Iordanova,
McNally,
Richardson, &
Harris (2002)
Flavor aversion Flavored liquid or Lithium chloride Drinking suppression Nausea Blair & Hall (2003)
food
Magazine approach Diffuse auditory– Food or liquid Magazine entry Eating or drinking Holland (1977)
visual stimulus
Place conditioning Tactile–visual Drug Place preference Euphoria Cunningham, Gremel,
contextual cues & Groblewski
(2006)
functional consequences, but what is known about around three basic questions that encompass much
Pavlovian conditioning has clearly come from a spe- of modern research on Pavlovian conditioning, as it
cific theoretical perspective that makes certain has been characterized in several influential reviews
assumptions about learning that are absent in most (Rescorla, 1988a, 1988b; Rescorla & Holland,
operant accounts. Rather than provide an epistemo- 1976): (a) What are the circumstances that produce
logical defense of any one perspective, in this chap- learning, (b) what is the content of learning, and
ter I describe the behavioral phenomena and related (c) how is learning expressed in performance?
theories as they are characterized in the literature,
which is dominated by a theoretical approach that
What are the Circumstances that
focuses on internal mechanisms.
Produce Pavlovian Conditioning?
Outside of the laboratory, clinicians have long
recognized the importance of Pavlovian cues in con- On the surface, there seems to be little to under-
tributing to maladaptive behavior. Understanding stand about how to produce Pavlovian conditioning.
the interaction between Pavlovian cues and pharma- Pavlov described many experiments in which a neu-
cological interventions for disorders involving fear, tral stimulus, such as the ticking of a metronome
anxiety, and substance abuse has been a major focus (the conditioned stimulus, or CS) was followed by a
of recent behavioral neurobiological research (e.g., salient, biologically relevant stimulus, such as meat
Davis, Myers, Chhatwal, & Ressler, 2006; Rothbaum powder or vinegar delivered to the mouth (the
& Davis, 2003; Rothbaum & Schwartz, 2002; Staf- unconditioned stimulus, or US). By itself, the US
ford & Lattal, 2011). These approaches continue to evokes its own response (salivation; the uncondi-
be shaped by the discoveries and sophisticated tioned response, or UR). After enough pairings,
experimental approaches taken to the study of responding emerges in the presence of the CS, even
behavior by researchers interested in the basic pro- when the US is absent (the conditioned response, or
cesses involved in Pavlovian conditioning. CR). Thus, if a CS and a US are presented together
My goal in this chapter is to provide an overview enough times, an association between them will
of modern behavioral research and theory on Pav- develop, resulting in the CS coming to elicit the
lovian conditioning. The chapter is structured response previously associated with the US. Indeed,
284
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
this is often the case, but there are many caveats to arranged in time, and the animal’s motivational
the idea that contiguity between stimuli will pro- state, among other factors. It is therefore very clear
mote learning about those stimuli. In some cases, that early in the history of Pavlovian conditioning,
contiguity is completely ineffective (e.g., Rescorla, Pavlov and his students recognized that the same
1968); in other cases, it may actually weaken the CS–US contiguity could produce differences in
association between stimuli (e.g., Kremer, 1978; behavior, depending on any number of
Lattal & Nakajima, 1998); and in still other cases, it considerations.
may be completely unnecessary for associative learn-
ing to occur (e.g., conditioned inhibition; Rescorla, Experimental Arrangements Between
1969). Moreover, the CR that develops may have a Conditioned Stimulus and an
very little resemblance to the UR (e.g., Fanselow, Unconditioned Stimulus
1980, 1982; Holland, 1977). Experimentally, CS–US contiguity (also known as a
Although most early theories of Pavlovian condi- conditioning trial) can be arranged and manipulated
tioning agreed that temporal contiguity between two in many ways. Four of the most commonly used pro-
events would produce an association between those cedures are shown in Figure 13.1. These procedures
events, it was clear from the beginning that any result in different levels of conditioned responding
number of manipulations could affect the conse- to the CS, but those differences do not always mean
quence of that contiguity for learning. For example, that there are differences in learning (as I review in a
Pavlov (1927) documented the importance of con- later section). In delayed conditioning, the onset of
siderations of CS and US salience, previous history the CS precedes the onset of the US by some amount
with the CS or US, consistency of other cues in the of time. Typically, the US is presented coincidentally
environment, how CS–US contiguities were either with the last part of the CS or with CS offset.
Figure 13.1. Four common procedural variations on the

temporal relation between the conditioned stimulus (CS) and
the unconditioned stimulus (US). The trial is defined as the
time between CS onset and US offset, and the intertrial interval
is defined as the time between CS offset and the next CS onset.
In delayed conditioning, CS onset and US presentation have a
forward relation. In simultaneous conditioning, the US is pres-
ent throughout CS presentation. In trace conditioning, a delay
occurs between CS offset and US onset (the trace interval).
Finally, in backward conditioning, the US precedes CS onset.
285
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
In simultaneous conditioning, the CS and US co- of milliseconds (e.g., Kehoe, Olsen, Ludvig, & Sut-
occur. In trace conditioning, a delay is inserted ton, 2009). In other preparations, such as magazine-
between the offset of the CS and the onset of the US, approach conditioning, in which a CS precedes the
resulting in a post-CS interval (the trace interval) in delivery of a food pellet to a magazine in an operant
which the CS is absent before the US presentation. In chamber, the optimal delay is on the order of sec-
backward conditioning, the US precedes the CS. onds (e.g., Holland, 2000). In still other prepara-
These procedural variations have been examined in tions, such as flavor aversion learning, the optimal
all of the preparations listed in Table 13.1. Of the delay may be on the order of minutes to hours (e.g.,
four arrangements shown, delayed conditioning Schafe, Sollars, & Bernstein, 1995). What is clear
results in the most robust conditioned responding from the many studies of CS–US intervals in Pavlov-
in the CS when it is tested alone after conditioning. ian conditioning is that there is a functional relation
Trace conditioning results in weaker responding between CS–US interval and conditioned respond-
to the CS, and simultaneous and backward condi- ing, but that the absolute values making up this
tioning frequently result in the weakest levels delay vary widely, from milliseconds to hours,
of responding. The challenge with these variations depending on the preparation. These differences in
on the CS–US contiguity is determining to what absolute durations defining an optimal CS–US inter-
extent the response in the presence of the CS reflects val between preparations suggest that what is con-
what the organism has learned about the CS–US considered effective depends greatly on the Pavlovian
tiguity, as I describe later in the chapter. preparation under consideration. In any given prep-
aration, however, the general finding is that perfor-
What Defines an Optimal Conditioning mance is greatest with a slightly asynchronous
Trial? forward CS–US interval (Rescorla, 1988a).
In delayed conditioning, the CS precedes the US, Other studies have found that in addition to the
which is the most commonly used procedural varia- CS–US interval (or the duration of the conditioning
tion of the CS–US arrangement. Although the CS trial, T), the interval between trials (the intertrial
must precede the US for conditioning to occur, the interval, or ITI) is important for generating Pavlovian
optimal interval for that delay differs among prepa- conditioning (e.g., Barela, 1999). The same trial can
rations. In a review of studies of the CS–US interval have very different effects on behavior, depending on
from several different Pavlovian preparations, how that trial is arranged in time. For example, with
Rescorla (1988a) demonstrated that within each long trials, very little conditioning may occur when
preparation, the relation between the conditioning those trials are separated by short ITIs, but when those
trial temporal parameters (CS–US interval) and con- trials are separated by long ITIs, conditioning will
ditioned responding was clear. When the delay emerge. These trial-spacing effects are robust across
between CS and US onset is zero (simultaneous pre- many different Pavlovian preparations, including
sentation of the CS and US), there is little or no preparations that require very short CS–US intervals,
responding (e.g., Fanselow, 1990). As the delay is such as autoshaping (e.g., Perkins et al., 1975), and
increased, responding emerges, but then falls again those that require longer CS–US intervals (on the
as the delay lengthens. Thus, the forward relation order of minutes or hours), such as flavor aversion
between the CS and the US that may be needed to learning (e.g., Domjan, 1980). Although the absolute
generate conditioning appears to be optimal. values that make up these durations differ widely from
Defining the optimal forward CS–US interval for task to task, conditioning is generally greater with
Pavlovian conditioning is difficult because the tem- shorter trial durations and longer intertrial durations.
poral characteristics vary from task to task. For
example, in preparations such as eyeblink condi- Interactions Between Trial and Intertrial
tioning, in which a CS precedes the delivery of Intervals in Delayed Conditioning
either a shock or an airpuff to the eye, the optimal A key caveat to trial and intertrial duration
delay between CS onset and US onset is on the order findings is that the predictive effects of a given trial
286
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
or intertrial duration depend on the overall temporal

context in which they are placed. Not only does the
effect of a trial duration depend on the ITI, it
depends on the ratio of the ITI to the trial duration
(the ITI:T ratio; Gibbon, Baldock, Locurto, Gold, &
Terrace, 1977). Figure 13.2A illustrates experimental
groups in which the duration of the ITI and the trial
were manipulated. As described earlier, a general
finding is that when the ITI is held constant (e.g.,
Groups 96:32 and 96:8 in Figure 13.2A), conditioned
responding emerges more quickly with the shorter
trial duration. When the trial duration is held con-
stant (e.g., Groups 96:8 and 24:8 in Figure 13.2A),
conditioned responding emerges more quickly with
the longer ITI.
A study by Gibbon et al. (1977) demonstrated
that although trial and ITI durations are important,
the key variable that determined the emergence of
conditioned responding was the ratio between them.
As the ITI:T ratio increases, rate of acquisition (i.e.,
number of conditioning trials required to reach a
predefined criterion) increases, largely independent
of the durations that fall into those ratios. An exam-
ple of this ITI:T ratio acquisition effect from a
magazine-approach experiment is shown in Figure
13.2B (Lattal, 1999). In this experiment, each trial
consisted of a white-noise CS followed by the deliv-
ery of a food pellet into the magazine. The elevation
in responding (food magazine entries) during the CS
compared with the ITI was used as the dependent
variable. As can be seen in Figure 13.2B, groups that Figure 13.2. A: Trial (T) and intertrial interval (ITI)
shared the same ITI:T ratio acquired the response at durations used in an experiment examining the ITI:T
approximately the same rate and to approximately ratio effect. B: Effects of ITI:T ratio on acquisition of
magazine-approach responding, defined as the ratio
the same terminal levels, regardless of the absolute between responding during the T and ITI periods over
durations of the ITI or T within that ratio. four daily sessions (elevation ratio = conditioned stim-
This ITI:T ratio effect discovered by Gibbon et al. ulus response rate/(conditioned stimulus response rate
(1977) has since been replicated in many other + ITI response rate). Adapted from “Trial and Intertrial
Durations in Pavlovian Conditioning: Issues of Learning
studies (Burns & Domjan, 2001; Holland, 2000), and Performance,” by K. M. Lattal, 1999, Journal of
and the acquisition effects such as those shown in Experimental Psychology: Animal Behavior Processes, 25,
Figure 2b for the most part tend to reflect differ- p. 437. Copyright 1999 by the American Psychological
Association.
ences in learning (Lattal, 1999). These findings have
had tremendous theoretical importance, leading to
the development of several influential theories sug- ITI:T ratio may not capture the entire story of trial
gesting that time and the calculation of rates of CS and intertrial duration effects (e.g., Holland, 2000;
and US occurrence are at the root of learning (e.g., Lattal, 1999), research around this finding has dem-
Gallistel, Fairhust, & Balsam, 2004; Gallistel & Gib- onstrated that there are no optimal intervals for pro-
bon, 2000; Gibbon & Balsam, 1981). Although the ducing conditioning. Instead, the intervals that will
287
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
produce conditioning will depend on the condition- is difficult and becomes especially problematic
ing preparation being examined and the overall tem- when analyzing the results of trace, simultaneous,
poral context in which these intervals are placed. or backward conditioning. One of the challenges in
assessing learning from responding to the CS is that
Does the Absence of Conditioned the conditions present during initial learning and
Stimulus–Induced Responding Reflect during testing will often be quite different depend-
an Absence of Learning? ing on the procedural variation in the CS–US
In the literature reviewed so far, I have not distin- arrangement. In the case of delayed and trace
guished between conditioned responding, learning, conditioning, for example, the animal has a history
and conditioning. The general assumption has of experiencing the CS before the US (and there-
been that more responding as a result of differen- fore in the absence of the US). Thus, when the CS
tial treatment reflects more learning. However, this is tested in the absence of the US, there will be at
assumption is not always valid, because in many least one test trial in which the conditions of acqui-
cases the behavior measured during an assessment sition and testing are the same. This means that the
of learning does not reflect the learning that has first test trial will allow for an assessment of learning
occurred. under identical conditions as occurred during
In studies of delayed conditioning reviewed ear- acquisition.
lier, for example, the general finding is that a for- However, assessing the learning that occurs dur-
ward temporal relation between the CS and the US ing backward US–CS pairings or simultaneous CS–
(i.e., CS precedes US) will result in conditioning, US pairings is difficult. During acquisition, the
depending on the overall temporal context in which presence of the US in each trial makes it difficult to
CS–US pairings occur. In the other CS–US variations disentangle the contributions of the CS and the US
illustrated in Figure 13.1 (simultaneous, trace, and to conditioned responding (because any potential
backward conditioning), responding is generally CR to the CS is confounded by the expression of a
much weaker than in delayed conditioning. The UR to the US). Testing the organism by presenting
challenge in interpreting this weak responding is to the CS in the absence of the US is also difficult,
determine the appropriate experimental conditions because the absence of the US on a test trial means
for assessing learning with these variations. Typi- that the conditions during acquisition and testing
cally, learning is inferred from responding to a CS are different (during acquisition, the US is always
during a common test some time after conditioning. present immediately before the CS or simultane-
During these tests, the CS is presented without the ously with the CS, but during testing the US is
US so that conditioned behavior in the presence absent, which may immediately signal different con-
of the CS is not confounded with unconditioned tingencies). Moreover, if the CS and US get encoded
behavior evoked by the US (see Rescorla & Holland, as a compound stimulus during acquisition, testing
1976). For example, on Day 1 of a fear conditioning the CS in the absence of the US will mean that only
preparation, a rat may receive pairings of a tone with part of the conditioning compound is assessed,
a shock. On Day 2, the tone is presented without which may be expected to weaken responding.
the shock, and the conditioned freezing response Thus, analyses of performance during test trials after
(absence of body movement) in the presence of the simultaneous or backward conditioning will be con-
tone is measured. Generally, the inference that founded by the difference in conditions from acqui-
one tends to make is that there is a direct relation sition to testing.
between the levels of responding and the strength of A similar challenge occurs when interpreting the
the CS–US association: Higher levels of freezing relatively weak responding that occurs after trace
reflect stronger learning of the tone–shock associa- conditioning. Typically, as the trace interval is
tion in fear conditioning. increased between CS offset and US onset, respond-
Inferring the state of the CS–US association from ing to the CS decreases. Thus, the temptation is to
observation of the strength of conditioned responding conclude that associative learning between the CS
288
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
and the US is impaired with increasing trace inter- promotes associative learning. The absolute dura-
vals. However, it is also possible that instead of tions that make for an effective forward relation
learning a simple association between the CS and between a CS and a US will be very different among
the US that is strong or weak, the organism learns preparations, and within a given preparation, the
about the specific temporal relation between the efficacy of a given CS–US interval will change
CS and US. Studies have found that when the CS depending on the overall temporal context in which
is tested in the absence of the US, conditioned that interval is arranged. Moreover, those CS–US
responding in the presence of the CS is low but relations (such as simultaneous, backward, and
increases as the time of the US presentation trace) that appear to lead to little conditioning can
approaches (e.g., McEchron, Tseng, & Disterhoft, actually result in a great deal of learning that is not
2003). Such increases in responding as the US necessarily expressed in responding to the CS dur-
approaches have been demonstrated in many condi- ing acquisition or testing. Thus, in evaluating the
tioning preparations with a variety of intervals (see circumstances that produce learning, it is critical to
Balsam, Sanchez-Castillo, Taylor, Van Volkinburg, & think about what the absence of responding reflects
Ward, 2009). and whether alternative strategies for assessing
The idea that animals may learn about the tem- learning exist.
poral relations between CSs and USs has also been
helpful in documenting the kind of learning that Challenges to Contiguity
may occur during simultaneous and backward The findings reviewed in the preceding section sug-
conditioning. In a series of studies (e.g., Arcediano, gest that CS–US contiguity can be arranged in
Escobar, & Miller, 2003; Barnet, Arnold, & Miller, many ways, and there is no straightforward answer
1991; Chang, Stout, & Miller, 2004; Miller & to the question of what is an effective contiguous
Barnet, 1993; see also Heth, 1976), Miller and col- relation. These findings and older findings dat-
leagues have demonstrated that organisms do ing back to Pavlov all speak to complexities in
learn about the CS–US relations in backward and contiguity—what may be regarded as an appropri-
simultaneous conditioning, but that learning is not ate contiguous relation for promoting learning
expressed in behavior when it is examined in typical depends on many factors, including the condition-
test situations. After a series of backward conditioning preparation, the time between conditioning
ing trials, Miller and colleagues demonstrated that trials, and how learning is assessed in behavior.
replacing the US with a second CS results in condi- Although all of these findings clearly demonstrate
tioning of that CS. They have suggested that this that many factors contribute to Pavlovian condi-
occurs because the new CS is presented at the time tioning, none reviewed so far challenges the idea
the original US was expected, resulting in an associ- that contiguity is critical for learning. A series of
ation being established. Thus, even though behavior experiments in the 1960s (Table 13.2) changed this
to the target CS is absent, there is evidence that a view, resulting in a paradigm shift in the way that
backward temporal association is formed between Pavlovian conditioning was approached experi-
the US and the CS. mentally and theoretically.
The clear implication of these studies is that a
wide range of conditions lead to learning. A single Predictability and Expectancy in
measure of behavior in the presence of a CS may Pavlovian Conditioning
lead one to conclude that nothing has been learned, The idea of contiguity between a CS and a US being
but if experimental designs can include additional critical for learning was challenged by three findings
assessments, then evidence of learning may emerge published around the same time in the late 1960s
in behavior. Together, these studies of delayed, (Kamin, 1968; Rescorla, 1968; Wagner, Logan,
simultaneous, backward, and trace conditioning Haberlandt, & Price, 1968). These findings all
reveal levels of complexity in the seemingly simple demonstrated that CS–US contiguity that promotes
contiguous condition between a CS and a US that learning in one situation may result in no learning in
289
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
Table 13.2
Experimental Designs and Results of Experiments by Kamin (1968, 1969); Rescorla (1966, 1967, 1968);
and Wagner, Logan, Haberlandt, and Price (1968)
Group Phase 1 Phase 2 Test Result Explanation

Kamin blocking experiment
Informative (I) — AX+ X– R I In Group I, A and X are
equal predictors of
shock.
Redundant (R) A+ AX+ X– In Group R, A is the
better predictor of
shock.
Rescorla correlational experiment

Informative (I) p(US with X) X– Ran I In Group I, X is best
p(US without X) signal of shock.
Random (Ran) p(US with X) = X– In Group Ran, X and
p(US without X) background context
are equal predictors of
shock.
Wagner et al. relative validity experiment

Informative (I) AX+, AX−, BX+, X– U I In Group I, all stimuli are
BX− equally informative.
Uninformative (U) AX+, BX− X– In Group U, A and B
are better predictors
of shock/no shock
compared with X.
Note. A, B, and X are conditioned stimuli (CS). + = unconditioned stimulus (US; shock in each experiment);
– = no US.
others, even when all other variables that may affect c ompound was conditioned before the compound
learning (such as number of CS and US presentations, phase. Kamin found that conditioning of A before the
CS durations, and ITIs) are held constant. compound phase resulted in much less conditioning
to X than occurred in the group that did not have the
Blocking. One finding that was important for the
A+ treatment before the compound phase. Thus,
development of Pavlov’s (1927) theories was over-
the previous conditioning of A blocked the condition-
shadowing: If two CSs (Stimulus A and Stimulus X)
ing of X when the two stimuli were reinforced
are presented in compound (AX) and followed by a
in compound.
US (+), the learning that occurs to either of the ele-
ments is weaker than the learning that occurs if either
element were conditioned alone. Thus, each stimulus Rescorla’s correlational experiment. As in Kamin’s
overshadowed the other, resulting in shared condi- (1968, 1969) experiments, Rescorla’s (1966, 1967,
tioning of A and X after the AX+ treatment. Kamin 1968) experiments arranged the same CS–US con-
(1968, 1969) extended this finding by conditioning tiguity in all experimental groups. These studies
Stimulus A (A+) before the compound phase, result- manipulated the probability of the US in the pres-
ing in the treatment shown in Table 13.2 (redundant ence or absence of the CS. Rescorla found that the
group). Note that in the second phase, the contiguity effects of a given CS–US probability on respond-
between X and the US is identical in the two groups; ing depended on the probability of the US in the
what differs is whether the other stimulus in the absence of the shock; as the shock became more
290
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
likely in the presence of the CS than in its absence, chamber itself. In the case of Wagner et al.’s (1968)
conditioned responding to the CS was strengthened experiment, Stimulus X did not gain associative
(informative group in Table 13.2). As the shock strength if it was presented in compound with a
became no more likely in the CS than in its absence, more predictive cue. These three findings have
responding was weakened (random group in Table since been replicated in many Pavlovian (and
13.2). Thus, the same CS–US probability produced even in operant) procedures, demonstrating the
very different results on conditioned responding utility of the notion of surprise and expectancy in
during the test, depending on the probability of the conditioning—for a US to effectively condition a CS,
US in the absence of the CS. that US must not be predictable on the basis of the
other cues available at the time of US presentation.
Relative validity effect. A final demonstra- These findings formed the basis for much of the
tion that contiguity was not sufficient for learn- current thinking about Pavlovian conditioning.
ing comes from the experiments of Wagner et al. Indeed, one of the tests of any new theory of Pav-
(1968). They adopted a strategy similar to Kamin’s lovian conditioning is to provide a framework for
(1968, 1969): Arrange a common CS–US contigu- viewing these three findings, and certainly many
ity but manipulate the identity of the other stimuli theoretical accounts have attempted to do this.
present during compound trials. The design of their Most of these theories share the common assump-
experiments is shown in Table 13.2. Here, Stimulus tion that redundant CSs are poorly conditioned
X has the same contiguity with the US in the two because they provide no new information to the
groups (reinforced on 50% of the trials), but X pro- organism (but see Stout & Miller, 2007, for a differ-
vides more predictive information about the occur- ent perspective). This notion of information and
rence of the US in the informative group than in the expectancy has been quantified in the idea of pre-
uninformative group. In the uninformative group, dictive error; learning occurs as a function of the
the strongest predictor of the US is Stimulus A difference (or error) between the US that the organ-
(always paired with the US), and the strongest pre- ism expects and the US that is actually received.
dictor of the absence of the US is Stimulus B (never Large predictive errors result in large amounts of
paired with the US). The information provided learning; small predictive errors result in little
by X is redundant with that provided by A and B, learning. As the predictive error approaches zero
which results in little to no associative strength (i.e., the US is almost perfectly predicted by the
accumulating to X. In the informative group, how- CS), no further learning occurs.
ever, Stimuli A, B, and X are each equally predic-
tive of the US, resulting in all stimuli, including X, Predictive error and the Rescorla–Wagner model.
gaining a moderate amount of associative strength. In 1972, Rescorla and Wagner collaborated on
During the test, responding was greater to Stimulus the development of a model designed to provide a
X in the informative group than in the uninforma- quantitative theoretical basis for the notion of sur-
tive group. prise and expectancy (Rescorla & Wagner, 1972;
The key implication of these findings is that Wagner & Rescorla, 1972). The key concepts
organisms learn only when the CS provides nonre- advanced by this Rescorla–Wagner model were that
dundant information about the US. In Kamin’s the organism gathers information from stimuli in an
(1968, 1969) experiments, previous conditioning of environment and uses that information to predict
Stimulus A meant that when Stimulus X was intro- the US. The goal of the model was to explain the
duced to the compound and followed by the same changes in associative strength (V) that occur dur-
US, no learning occurred to X because it did not sig- ing a given conditioning trial. The predictive error
nal anything new. In the case of Rescorla’s (1966, component of the equation is represented by the
1967, 1968) random groups, the presentation of the difference between the asymptote of learning that a
CS similarly did not signal anything about the US US could support (λ) and the associative strength of
that was not already signaled by the conditioning all stimuli present during a conditioning trial (Σ V).
291
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
Parameters associated with the CS (α) and US (β) models in learning (e.g., Herrnstein, 1970), there is
control the rate of learning. elegance in its simplicity. Any student can quickly
understand it and use it to make very clear predic-
ΔVCS = αCSβUS (λUS − Σ V). (1) tions about the conditions that should promote or
retard learning. These predictions are not always cor-
The amount of learning (change in associative rect, but some surprising predictions about the
strength of the CS, ΔVCS) was directly related to the nature of expectancy and operant reinforcement have
size of the discrepancy (λUS − Σ V) between the resulted from this model (e.g., Lattal & Nakajima,
expected (on the basis of the available cues) and 1998; Vaughan, 1982). Researchers continue to
obtained US; more learning occurred when the dis- investigate the nature of predictive error learning,
crepancy was large than when it was small. This with new techniques being developed to examine
model nicely captures the data from the Kamin some fundamental predictions about the negatively
(1968, 1969), Rescorla (1966, 1967, 1968), and accelerated nature of learning curves (e.g., Rescorla,
Wagner et al. (1968) experiments. For example, in 2001a; Thein, Westbrook, & Harris, 2008) and about
the case of blocking, if enough conditioning of A the underlying neurobiological circuits that mediate
occurs in Phase 1, the associative strength of A (VA) predictive error learning (e.g., Cole & McNally,
will approach asymptote. In Phase 2, when X is pre- 2007; Iordanova, McNally, & Westbrook, 2006;
sented in compound with A (AX+ trials), the total reviewed in McNally & Westbrook, 2006). This con-
associative strength of that compound (VAX) is equal cept of predictive error has been extended beyond
to the sum of the associative strengths of the ele- conditioning into the realm of substance abuse and
ments (VA and VX), which is already near asymptote reward systems (e.g., Schultz, 1998) and neural net-
(owing to the large value of VA). The newly intro- work modeling (e.g., Schmajuk, Gray, & Lam, 1996).
duced Stimulus X, which has no previous value, The findings reviewed in this section demon-
provides no predictive information above and strate that there are many circumstances in which
beyond that provided by Stimulus A; that is, the dis- Pavlovian conditioning will and will not occur. A
crepancy between the expected and obtained US unifying theoretical theme from all of this work is
does not change, so learning does not accrue to the that organisms code predictive relations, and
novel Stimulus X (VX remains near zero). Thus, responding occurs to the extent that conditioned
when tested alone, the failure to observe responding stimuli provide useful information to the organism.
to Stimulus X reflects impairments in conditioning Quantitative theories such as the Rescorla–Wagner
of Stimulus X. model propose various mechanisms for this predic-
The key insights from the Rescorla–Wagner tion encoding and make explicit predictions about
model were that (a) the predictive error term was cal- those circumstances in which Pavlovian conditioning
culated on the basis of the summation of all cues that will occur. These models make some rudimentary
were present on a conditioning trial and (b) the asso- assumptions about what organisms learn (in terms
ciative strength of a stimulus could have a negative of associative strength), but they were designed to
value (which makes it inhibitory, signaling the say very little about the nature of the associative
absence of an otherwise expected US; see Pearce & relations that form during conditioning. In the next
Bouton, 2001). The Rescorla–Wagner model has section, I review some work illustrating the content
faced many challenges by other models that also rely of what the organism might learn as a result of Pav-
on predictive error (e.g., Mackintosh, 1975; Pearce, lovian conditioning.
1987) and by models that emphasize other features of
the conditioning situation (e.g., Gallistel & Gibbon,
What is the Content of Pavlovian
2000; Stout & Miller, 2007). Nonetheless, the
Learning?
Rescorla–Wagner model continues to be the gold
standard against which all models of Pavlovian con- A major question that has motivated research on
ditioning are compared. As with other long-standing Pavlovian conditioning focuses on understanding
292
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
what it is that the animal learns when the experi- conditioning in pigeon autoshaping. Each trial con-
menter arranges a Pavlovian relation. Why does the sists of a 5-second presentation of a compound with
animal show a CR in the presence of the CS? To three visually distinct components (three triangles
answer this question, one needs to consider (a) how that together form a trapezoid) followed immedi-
the animal might learn about the properties inherent ately by response-independent food. After several
in the CS and US, (b) how associations might be pairings, keypecking begins to occur during the
formed between those properties, and (c) how the compound trial presentation. According to a purely
CR becomes connected to those properties. elemental learning theory (e.g., Rescorla & Wagner,
1972), responding occurs to the compound because
What Is the Nature of the Conditioned each stimulus element (the individual triangles) has
Stimulus Representation? its own associative strength (elemental learning in
Theories of learning have long speculated about how Figure 13.3), which sum together to produce the
the animal encodes the stimulus environment and response to the compound. According to a purely
how that stimulus representation enters into associa- configural account (e.g., Pearce, 1987), responding
tions with the US. Broadly, theories can be classified to the compound occurs because the compound
into those that view learning as occurring between ele- itself is its own configural stimulus (a trapezoid that
ments of the CS and the US (e.g., Estes, 1950; Pearce is not really a compound at all, in fact) that gains
& Hall, 1980; Rescorla & Wagner, 1972; reviewed in strength as that configuration is reinforced (config-
Harris, 2006) and those that view learning as occur- ural learning in Figure 13.3). Testing a triangle by
ring between the CS as a configuration and the US itself consistently reveals less responding than the
(e.g., Pearce, 1987, 1994; Rudy & O’Reilly, 1999). testing of the compound (overshadowing; see Good
The differences between these elemental and & Macphail, 1994). According to elemental theo-
configural theories are apparent in the case of com- ries, responding to the element is less than to the
pound conditioning, in which two or more neutral compound because each element shares the total
stimuli are presented together, followed by a US. strength of the compound. In the example in Figure
Figure 13.3 illustrates an example of compound 13.3, if all three triangles are equally salient, they
Figure 13.3. Elemental and configural learning approaches to condition-

ing. During a conditioning trial, a pigeon is presented with a visual stimulus
(conditioned stimulus) that is followed by food presentation (unconditioned
stimulus). After several trials, the pigeon pecks the key. According to an
elemental account, the pigeon pecks because it learns that each component
of the compound stimulus is associated with food. According to a configural
account, the pigeon pecks because it learns that the stimulus as a single con-
figuration is associated with food. CR = conditioned response.
293
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
will each accumulate one third of the total associa- elemental and configural theories of learning have
tive strength that the US can support. According to demonstrated the importance of examining the
configural accounts, responding to the element is nature of the CS in understanding the content of
less than to the compound because the element is a associative learning. The CS representation can be
different stimulus that is only partially similar to the relatively simple or complex, with multiple within-
originally reinforced stimulus. In the example in stimulus associations or with subtle differences in
Figure 13.3, responding to an individual element stimulus configurations driving learning and
should be about one third as strong as that to the performance.
configuration because about one third of the The importance of characterizing elemental and
strength of the configuration will generalize to that configural perspectives in an introduction to Pavlov-
element. ian conditioning is to emphasize that what one con-
Because some of the more influential elemental siders to be a CS is often derived from the
(e.g., Rescorla & Wagner, 1972) and configural perspective of the experimenter. How the animal
(e.g., Pearce, 1987, 1994) models make precise codes the stimulus environment will depend on
quantitative predictions, experiments can be devised many factors that are not at present well under-
to define the experimental conditions that distin- stood. Multiple aspects of the stimulus situation,
guish the two approaches. The answers to the exper- including stimuli present on the conditioning trial
imental questions of whether elemental or configural as well as background stimuli (such as the context
learning occurs are therefore answered on the basis in which conditioning occurs), will contribute to the
of which mathematical formula better predicts the learning that occurs.
results. Many experiments have been devised to dis-
tinguish between configural and elemental accounts What Is the Nature of the Unconditioned
of learning. It is clear that certain conditions may Stimulus Representation?
favor one account over another (reviewed in As described in the previous section, some theories
McLaren & Mackintosh, 2000; Rescorla, 2003; of learning have focused on the nature of the CS
Rescorla & Coldwell, 1995; Wagner, 2003). For representation (elemental vs. configural) and the
example, with stimuli from the same modality, as in resulting impacts on learning that a particular
the autoshaping example in Figure 13.3, a configural approach may predict. What one thinks of as a CS
account may better capture the learning that occurs. may, from the animal’s perspective, be multiple CSs
With stimuli from different modalities (such as that are combined together or may only be a small
when a flashing light and a tone are paired with food part of a broader configuration that includes addi-
in magazine-approach conditioning), there may be tional stimuli that are outside of the experimenter’s
more learning about the individual elements. control. Other theories of learning have suggested
The major implication of an elemental approach that the US may also be a complex stimulus that has
to learning is that the component stimulus elements multiple features. The influential works of Konorski
that are present on a conditioning trial may become (1948, 1967) appealed to different aspects of the US
associated with the US. Other experiments have in promoting associative learning. His ideas were
demonstrated that these elements may also become central to the development of Wagner’s (1981)
associated with each other (known as within- sometimes opponent process theory of learning; the
compound learning). For example, if a tone (T) and a idea that a US may have different components was
light (L) are paired with shock (TL+), further con- nicely incorporated into the affective extension of
ditioning of the tone (T+) will strengthen the fear that theory by Wagner and Brandon (1989).
response to the light, whereas extinction of the tone These theories have appealed to two components
(T−) will decrease the fear response to the light of the US: (a) the sensory component, which con-
(e.g., Cunningham, 1981; Rescorla & Cunningham, sists of the physical properties of the stimulus (i.e.,
1978). Together, experiments examining within- how it feels, tastes, or smells) and (b) the affective
compound learning and those trying to distinguish component, which consists of the emotional
294
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
reactions triggered by that stimulus. These compo- result of Pavlovian conditioning has been at the
nents may have different properties, such as the forefront of theoretical speculation since Pavlov,
immediate salience and time course of decay from who wrote of CS and US centers in the brain and the
short-term memory. CSs may enter into associations various connections that they may have with each
with these different US components, depending on other and with response systems. Research on this
how the CS and US representations overlap in time topic has characterized two very general accounts
(Wagner & Brandon, 1989; see also Delamater & of the learning that may result from Pavlovian con-
Holland, 2008; Cunningham, Okorn, & Howard, ditioning: stimulus–stimulus (S-S) learning and
1997; Cunningham, Smith, & McMullin, 2003). stimulus–response (S-R) learning.
The idea that a US may have multiple components The S-S approach to learning says that respond-
is nicely illustrated in ethanol place-conditioning stud- ing occurs because presentation of the CS activates a
ies by Cunningham and colleagues (e.g., Cunningham representation of the US. After several conditioning
et al., 1997, 2003). In their experiments, mice received trials, the link to that US representation is strong.
an intraperitoneal injection of ethanol (the US) at dif- On subsequent trials, presentation of the CS triggers
ferent time intervals relative to placement into the the memory of the US, which serves as an intermedi-
place-conditioning apparatus (the CS). A strong place ary between the stimulus and the response. Accord-
aversion developed when placement immediately fol- ing to the S-R approach, responding occurs because
lowed an ethanol injection, but a strong place prefer- the stimulus acquires the ability to elicit a response
ence developed when placement was delayed for a few and the US is no longer involved once that S-R asso-
minutes after the injection. These findings suggest that ciation is established. This account views the US as a
ethanol has multiple unconditioned properties—a catalyst that strengthens the S-R bond, but it is not
short-term aversive effect that is followed by a longer part of the association that forms as a result of
lasting positive effect. The direction of conditioning conditioning.
(place aversion or preference) in those experiments One way to separate these two accounts is to
depended on which aspect of the US becomes associ- change the value of the US after conditioning (e.g.,
ated with the CS. Colwill & Motzkin, 1994; Holland & Straub, 1979;
Together, these findings demonstrate that the see Colwill, 1994, and Dickinson, 1989, for applica-
general concept of associative strength supported by tions to operant behavior). If the S-S approach is
a US does not capture the nuances of the specific correct, devaluing the US before a test with the CS
associations that may develop between components should weaken responding to the CS. This weaken-
of the US and the CS. One of the key contributions ing would occur because the presentation of the CS
of theories such as sometimes opponent process is would activate a devalued representation of the US,
that, as with CSs, USs that support conditioning also thereby producing a weak response. If the S-R
have multiple properties that may become incorpo- approach is correct, altering the value of the US
rated into the learning that occurs during a trial. should have no effect on behavior because once the
CS–CR connection is established, the US is no lon-
What Is the Nature of the Pavlovian ger needed to generate a response.
Association? A nice illustration of the role of the US represen-
Clearly, the CS and the US have multiple aspects tation in mediating conditioned responding comes
that need to be included in the associative represen- from a magazine-approach experiment by Colwill
tation of stimuli. Thus far, I have considered the dif- and Motzkin (1994), the design of which is summa-
ferent aspects of the CS and the US that may be rized in Figure 13.4A. In their experiment, rats learned
important for associative learning. In this section, I that the presentation of one CS (CS1; a tone) would
consider how the CS and the US enter into associa- lead to a specific outcome in the food magazine
tions with each other and whether a CS–US associa- (US1; food pellets). On other trials, a different CS
tion is necessary for conditioned responding to (CS2; a light) was presented with a different out-
occur. This basic issue of what gets associated as a come (US2; liquid sucrose) in the same magazine.
295
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
The results of this experiment are shown in Fig-

ure 4B. Less responding occurred to the CS that had
its outcome devalued than to the CS that did not.
This difference is consistent with an S-S account of
learning: Postconditioning devaluation of the out-
come weakened conditioned responding to the CS
that was initially paired with that outcome. These
results, as well as many others, suggest that the CR
adapted to a change in the value of the US with
which it was associated, consistent with an S-S
approach. Other data are consistent with an S-R
account (e.g., Rizley & Rescorla, 1972), and Figure
4b shows that some responding continued in the
presence of the CS associated with the devalued out-
Figure 13.4. A: Design of an experiment examin-
ing the effects of outcome devaluation on Pavlovian come, even after extensive outcome devaluation that
responding. During Phase 1 (cue conditioning), one caused all subjects to reject the outcome by the end
conditioned stimulus (CS; a tone) signaled pellet deliv- of the devaluation phase. This result demonstrates
ery, and another CS (a light) signaled sucrose delivery. that, although Colwill and Motzkin (1994) found
During Phase 2 (outcome devaluation), a flavor aver-
sion was conditioned to the pellets. Finally, during the strong evidence for S-S learning, they also found
test, magazine-approach behavior was assessed dur- that some of the response was independent of the
ing nonreinforced presentations of each CS. The CS– status of the outcome. Thus, the idea that a single
unconditioned stimulus (US) pairings and the US that
S-S or S-R association controls behavior may be sim-
was devalued were counterbalanced in the experiment.
B: Responding during the test, defined as the percent- plistic, especially when one considers the multiple
age of observations in which the rat directed behavior aspects of CSs (such as elemental and configural
toward the magazine. LiCl = lithium chloride. Data are learning) and USs (such as sensory and affective
from Colwill and Motzkin (1994).
components) that may become associated (see
Delamater & Holland, 2008).
As a result of these trials, the rats enter the magazine
during the presentation of either CS as they learn Putting Associations in Context
the CS–US relation. Thus, the magazine response is When thinking about the mechanisms underlying
similar for the two CSs (which were counterbal- Pavlovian conditioning, one often thinks in terms of
anced, as were the outcomes). After conditioning, CSs and USs and how they do or do not become
the animals received access to one of the outcomes associated. These issues are critical for understand-
(US1) followed by injection of lithium chloride, ing the nature of learning, but considering that con-
which induces nausea (the devaluation phase). By ditioning trials always occur in a context is also
the end of the final session of devaluation, all sub- important. Context is more than just the back-
jects had stopped consuming the devalued outcome. ground against which these CS–US pairings occur; it
During an extinction test, animals received each CS is a key component of the development and expres-
on alternating trials. According to an S-S account, sion of learning during and after conditioning (see
there should be less response to CS1 (the tone) than Bouton, 2004; Hall & Honey, 1989; Westbrook,
to CS2 (the light) because presentation of the tone Iordanova, McNally, Richardson, & Harris, 2002).
should activate a representation of the food pellets, A nice demonstration of the importance of context
which are now associated with nausea. According to in modulating CS–US associations comes from a
an S-R account, there should be no difference in study by Peck and Bouton (1990). They used a
responding to the two CSs because responding is counterconditioning paradigm, in which a CS was
now under the control of the S-R association, which paired with an appetitive US in Phase 1 and then an
is independent of the status of the US. aversive US in Phase 2 (or vice versa, in separate
296
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Table 13.3
Experimental Design and Results of Peck and Bouton (1990)
Results (relative to
Group Phase 1 Phase 2 Test control group)
Aversive–appetitive Context B: tone–shock Context A: tone–food Context B: tone? Aversive conditioned
responding increased
Appetitive–aversive Context B: tone–food Context A: tone–shock Context B: tone? Appetitive conditioned
responding increased
experiments). In this type of procedure, the CR nor- with a US can result in different associations that
mally changes after the US that is predicted by the may be dependent on conditioning contexts for
CS changes. For example, a tone paired with food in retrieval. These findings lay the foundation for the
Phase 1 will elicit appetitive CRs (head jerking or material in the next section, in which I consider the
magazine entries), but when the tone is paired with issue of how learning is expressed in behavior. Many
shock in Phase 2, it will elicit aversive CRs (condi- kinds of associations can be formed as a result of
tioned freezing). conditioning, and the empirical challenge is to pro-
Peck and Bouton (1990) added a contextual vide evidence of these associations in behavior.
manipulation to this paradigm, resulting in the aver-
sive and appetitive associations forming in different
How is Learning Expressed in
contexts. Table 13.3 shows Peck and Bouton’s gen-
Performance?
eral design (the two groups were examined in differ-
ent experiments). Rats were exposed to a distinct Perhaps the major challenge confronting Pavlovian
context (Context B) during Phase 1, in which some conditioning research is determining how to mea-
rats (aversive–appetitive [Av-Ap] group;) received sure learning—a theoretical concept that is not
tone–shock pairings and others (appetitive–aversive directly observable—in behavior. It is clear that
[Ap-Av] group) received tone–food pairings. In overt behavior often belies what the organism has
Phase 2, which occurred in Context A, the contin- learned about a particular contingency. For exam-
gencies were switched, resulting in rats in the Av-Ap ple, the presence of an overt behavioral response
group receiving tone–food pairings and the Ap-Av may be due not to learning but to some general
group receiving tone–shock pairings. When testing internal aroused state; similarly, the absence of
of the tone occurred in Context B, the Ap-Av group behavior may occur for many reasons other than the
spent more time engaged in appetitive responding absence of learning. This distinction between learn-
(head jerking) relative to control groups, and the ing and the expression of learning in performance is
Av-Ap group spent more time engaged in aversive perhaps clearest in the study of extinction.
responding (freezing behavior) relative to control
groups. These results demonstrate that the CS–US Extinction: Preserving the Association
association is at least somewhat under the control of While Eliminating the Response
contextual cues (see also Bouton & Peck, 1992). Although the mechanisms underlying initial associa-
Many other studies have demonstrated that con- tive learning remain to be fully understood, predic-
texts can modulate conditioned responding, through tive information is clearly critical for new learning
direct associations with either the CS or the US or to occur. Research demonstrating the importance of
through a more hierarchical association in which predictive information has also focused on the con-
the context sets the occasion for certain CS–US conditions that promote response loss after condition-
tingencies (e.g., Nelson, 2002). Together, the finding. The most widely studied of these conditions is
ings reviewed in this section show that pairing a CS extinction, which occurs when a CS is presented
297
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
repeatedly in the absence of the US with which it since extinction. Figure 13.5 shows the results of
was previously paired. After some number of such two extinction experiments reported by Pavlov
extinction trials, the CR is weakened and often (1927). During conditioning, the CS was paired
entirely eliminated. Many studies have demon- with delivery of vinegar (the US) directly into the
strated that acquisition and extinction share com- dog’s mouth. After these pairings, tests of the CS
mon properties, including the importance of revealed high levels of salivation (Extinction Trial
predictive error, the shape of the learning curve, and 1 in Figure 13.5). Over the course of extinction
the generality between stimuli, among other proper- trials, the continued absence of the US caused the
ties (e.g., Rescorla, 2001a). CR to extinguish. Pavlov’s key experimental insight
Extinction clearly produces response loss within these extinction experiments was simply to
out necessarily producing a loss of the original asso- wait and test the animals again after some delay.
ciation. This point was emphasized by Pavlov When he did that (Test in Figure 13.5), he found
(1927) and formed the basis for much of his theoriz- that the CR that was partially (Figure 13.5A) or
ing, and most theories since Pavlov’s time have almost completely (Figure 13.5B) eliminated dur-
agreed that extinction generates new learning that ing extinction returned. This he termed spontane-
does not permanently alter the original association ous recovery, which subsequent research has shown
(see Delamater, 2004; Lattal, 2007; Rescorla, to be influenced by several factors, including the
2001b). Several key findings have demonstrated that strength of conditioning, amount of extinction, and
extinction suppresses behavior without significantly length of the postextinction retention interval (see
altering the content of the original learning. Rescorla, 2004).
Spontaneous recovery. Perhaps the best docu- Reinstatement. A second finding demonstrat-
mented demonstration that extinction eliminates ing the preservation of initial learning through
the response while preserving the original learn- extinction is reinstatement. After extinction, non-
ing comes from studies showing that extinguished contingent exposure to the US alone can reinstate
behavior returns after some time has passed conditioned responding to the CS that had previously
Figure 13.5. Two examples of extinction and spontaneous

recovery plotted from data reported by Pavlov (1927). During
conditioning, dogs received pairings of an auditory conditioned
stimulus (ticking of a metronome) with the delivery of vinegar
(the unconditioned stimulus) to the mouth. During extinction,
the conditioned stimulus was presented without the uncondi-
tioned stimulus. During the first extinction trial, conditioned
responding (salivation) was high but decreased over the course
of extinction sessions. A postextinction test conducted after a
delay revealed small (A) or large (B) spontaneous recovery of
the conditioned response. From “Extinction and the Erasure of
Memories,” by K. M. Lattal, Psychological Science Agenda, 21,
p. 3. Copyright 2007 by the American Psychological Association.
298
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
been extinguished. As with spontaneous recovery, ability to augment an instrumental response that
reinstatement is specific to the CS that was extin- leads to a common outcome (e.g., Delamater, 1996).
guished and to the US that was delivered during Together, these and several other findings (e.g.,
conditioning (e.g., Delamater, 1997). Reinstatement Leri & Rizos, 2005) all demonstrate that extinction
as a procedural tool has been particularly effective in is a response-weakening process that dampens but
the study of substance abuse and the relapse of drug does not eliminate the original association. From a
seeking after extinction (e.g., Malvaez, Sanchis- theoretical perspective, the challenge is to determine
Segura, Vo, Lattal, & Wood, 2010) the mechanisms that allow this to occur. Formal
theories of learning, such as the Rescorla–Wagner
Contextual renewal. As noted earlier, conditioned
model, suggest that the key processes that produce
behavior is modulated by contexts. The clearest
new learning also operate during extinction. These
examples of contextual determinants of behavior
theories suggest that during conditioning and
come from the study of extinction. If conditioning of
extinction, the discrepancy between the US that is
a CS occurs in one context (Context A) followed by
expected and the US that is obtained is what drives
extinction in a second context (Context B), testing
learning. Early in conditioning and early in extinc-
of the CS in Context A results in a return (renewal)
tion, this discrepancy is large: Before conditioning,
of conditioned responding. This contextual A-B-A
the organism expects no US but then receives a US,
renewal effect is robust, having been documented
and before extinction, the organism expects a US
in fear conditioning (e.g., Bouton & Bolles, 1979),
but then receives no US. Throughout the course of
appetitive operant conditioning (e.g., Nakajima,
conditioning or extinction, learning continues until
Tanaka, Urushihara, & Imada, 2000), flavor aver-
the expected outcome matches the obtained out-
sion (e.g., Rosas & Bouton, 1997), and spatial learn-
come (a US in the case of conditioning; no US in the
ing (e.g., Lattal, Mullen, & Abel, 2003), among
case of extinction). Thus, the amount of learning
many other preparations. Moreover, this contextual
that occurs on a given extinction trial is subject
renewal effect is not limited to a return to the con-
to the same predictive error mechanisms that oper-
ditioning context, as with other forms of renewal
ate during conditioning.
such as A-A-B (e.g., Bouton & Ricker, 1994; see also
Although error-correction models appear to cap-
Pavlov, 1927, p. 99, for an early description of this
ture the negatively accelerated learning curves that
phenomenon) and A-B-C (e.g., Denniston, Chang,
underlie extinction, such models do not have an
& Miller, 2003) renewal. These findings demonstrate
obvious mechanism that allows the original learning
that a change of context between extinction and test-
to be preserved throughout extinction. Indeed, in its
ing will result in a return of conditioned responding.
simplest form, associative strength to a stimulus
Pavlovian-to-instrumental transfer. Another find- increases and decreases depending on predictive
ing that does not receive as much attention as spon- error, but it occurs via a single path that does not
taneous recovery, reinstatement, and renewal is the allow the history of the stimulus to be retained.
Pavlovian-to-instrumental transfer effect. When a Consequently, there is not a clear mechanism that
Pavlovian CS (such as a tone) and an instrumental would allow phenomena such as spontaneous recov-
(operant) response (such as a lever press) lead to the ery after extinction to occur.
same outcome in different conditioning sessions, acti- Most theories developed to provide specific
vating the tone with the lever present will produce an accounts of extinction appeal to some inhibitory
increase in instrumental responding. This Pavlovian- process that operates during extinction (see Delama-
to-instrumental transfer procedure has been used to ter, 2004). There is good evidence that the error
assess the status of CS–US associations in many stud- correction mechanisms described in the Rescorla–
ies (e.g., Holland, 2004), and studies of extinction Wagner model operate during extinction (e.g.,
have shown that even when conditioned responding Leung & Westbrook, 2008), but the open question
has been eliminated in the presence of a CS, that is, what is being inhibited? Formal theories have
stimulus shows remarkably little disruption in its proposed inhibition within several pathways,
299
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
including in the representation of the CS or US itself Do Impairments in Pavlovian

(e.g., Rescorla & Cunningham, 1977; Robbins, Conditioning Reflect Impairments in
1990) and inhibition in the associations between the Learning or Performance?
CS and US or CS and CR (e.g., Rescorla, 1993). Extinction is probably the most widely known and
One influential theory of extinction has sug- studied example of a learning process in which the
gested that a key component in the development and behavior that is expressed in the presence of a stim-
expression of extinction is the context in which it ulus clearly does not reflect the entirety of what the
occurs (e.g., Bouton, 2004). Bouton and colleagues animal knows about the stimulus. As noted in sev-
have appealed to memory retrieval processes in eral places in this chapter, it is clear that in many
accounting for phenomena such as contextual other cases, examination of performance in the pres-
renewal and spontaneous recovery (e.g., Bouton, ence of a CS after conditioning will often not reflect
1991, 1993; Bouton, Westbrook, Corcoran, & Maren, the status of the learning. A common example of
2006). Central to Bouton’s (2004) theory is the idea this is that the CR that occurs after CS–US pairings
that initial conditioning and extinction result in the may differ depending on the physical properties of
formation of memories that differ in their depen- the CS, so limiting the behavioral analysis to a single
dence on the context for retrieval. Memories of ini- response system may lead to a very different inter-
tial conditioning are independent of the context pretation than an examination of a different
in which that learning occurred, as revealed in the response system. This is illustrated nicely in an
transfer of conditioned responding across testing experiment by Holland (1977), who found that
contexts. Memories for extinction, however, rely auditory and visual CSs evoke different responses
heavily on contextual cues for retrieval; when test- when they are paired with a food pellet in a
ing occurs outside of the extinction context, those magazine-approach procedure. Auditory stimuli
cues are absent, resulting in renewal of responding. evoked a head-jerking response, whereas visual
According to Bouton’s theory, spontaneous recovery stimuli evoked an orienting–rearing response. Thus,
can also be thought of as a form of contextual renewal, simply examining a single performance measure
because the passage of time itself creates a context (anticipatory entries into the food magazine) would
that differs from the extinction context (see Bouton, result in a mischaracterization of the learning that
Nelson, & Rosas, 1999, for further discussion). had occurred.
Applying Bouton’s ideas about memory retrieval in As noted earlier, certain procedural variations
extinction has led to many novel findings that have of the CS–US relation will result in low levels of
been applied with success to clinical treatments (e.g., responding to the CS that do not necessarily reflect
Collins & Brandon, 2002; Dibbets, Havermans, & the status of the associative learning about that CS.
Arntz, 2008; Vansteenwegen et al., 2006). In the case of trace conditioning, for example, weak
The study of extinction has been one of the great responding in the presence of the CS belies the often
success stories of basic research in terms of extrapo- precise temporal information that the organism has
lation of findings with animals to clinical settings encoded. Moreover, some theories of Pavlovian con-
(see Chapter 4, this volume). Extinction is a widely ditioning argue that many of the core phenomena
used behavioral intervention for many psychiatric that have been used to generate theories of learning
disorders, including posttraumatic stress disorder, (such as the trial-spacing effect and the Kamin
phobias, and substance abuse. The repeated demon- [1968, 1969] blocking effect) do not reflect deficits
strations from the laboratory that extinction does in learning. Instead, these deficits in conditioned
not eliminate the original learning challenges clini- behavior reflect deficits in the expression of learned
cians to design interventions that make extinction associations in behavior.
learning persistent, weakening contextual renewal One performance-based theory developed by
and spontaneous recovery with time, which is cur- Miller and colleagues (e.g., Stout & Miller, 2007)
rently the focus of a great deal of neurobiological suggests that CS–US contiguity results in learning,
work on extinction (see Davis et al., 2006). but performance rules determine whether that
300
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
learning is expressed in behavior. Conditioned approach to Pavlovian conditioning that incorpo-

responding is evident as a function of the associative rates many theoretical perspectives about the nature
strength of the other cues that were present during of stimulus representation and response expression
conditioning (or testing; see Gibbon & Balsam, will surely play an important role in the future
1981). As the strength of the target CS is increased refinement of these neurobiological techniques. Cer-
relative to those other cues (comparator stimuli), tainly, there are many unanswered questions about
conditioned responding emerges; as it is decreased, the basic nature of Pavlovian conditioning, meaning
conditioned responding is attenuated. Thus, in the that the field’s emphasis on rigorous behavioral
case of blocking, the blocked stimulus acquires an analyses will continue to reveal insights about basic
association with the US that is not expressed in per- learning processes.
formance owing to the established association
between the blocking stimulus and the US. Evidence References
for this idea comes from many studies by Miller and
Arcediano, F., Escobar, M., & Miller, R. R. (2003).
colleagues demonstrating that postconditioning Temporal integration and temporal backward asso-
manipulations of the associative strength of compar- ciations in human and nonhuman subjects. Learning
ator stimuli can strengthen or weaken responding to and Behavior, 31, 242–256. doi:10.3758/BF03195986
a target CS (e.g., Stout & Miller, 2007). Other stud- Balsam, P., Sanchez-Castillo, H., Taylor, K., Van
ies have demonstrated that the status of comparator Volkinburg, H., & Ward, R. D. (2009). Timing
and anticipation: Conceptual and methodological
stimuli has little to no impact on target CS behavior approaches. European Journal of Neuroscience, 30,
(e.g., Holland, 1999; Lattal, 1999; Robbins, 1988). 1749–1755. doi:10.1111/j.1460-9568.2009.06967.x
Thus, whether this specific comparator mechanism Barela, P. B. (1999). Theoretical mechanisms underly-
always controls performance is not clear. What is clear ing the trial-spacing effect in Pavlovian fear condi-
is that this alternative way of thinking about learning tioning. Journal of Experimental Psychology: Animal
Behavior Processes, 25, 177–193. doi:10.1037/0097-
and performance has revealed novel insights about 7403.25.2.177
learning and has offered the very important point that
Barnet, R. C., Arnold, H. M., & Miller, R. R. (1991).
the absence of behavior should not be taken as evi- Simultaneous conditioning demonstrated in
dence for the absence of learning. This is a critical second-order conditioning: Evidence for similar
point that is often unappreciated in neurobiological associative structure in forward and simultaneous
conditioning. Learning and Motivation, 22, 253–268.
analyses of learning (see Lattal & Stafford, 2008).
doi:10.1016/0023-9690(91)90008-V
Blair, C. A., & Hall, G. (2003). Perceptual learning
Future of Pavlovian Conditioning in flavor aversion: Evidence for learned changes
in stimulus effectiveness. Journal of Experimental
Research
Psychology: Animal Behavior Processes, 29, 39–48.
Remarkable progress has been made in understand- doi:10.1037/0097-7403.29.1.39
ing the mechanisms that underlie Pavlovian condi- Bouton, M. E. (1991). Context and retrieval in extinction
and in other examples of interference in simple asso-
tioning. There is now broad appreciation that
ciative learning. In L. Dachowski & C. F. Flaherty
Pavlovian conditioning reflects sophisticated learn- (Eds.), Current topics in animal learning: Brain,
ing processes that underlie many aspects of animal emotion, and cognition (pp. 25–53). Hillsdale, NJ:
and human behavior. The development of advanced Erlbaum.
neurobiological techniques has resulted in a recent Bouton, M. E. (1993). Context, time, and memory
emphasis on the cellular and molecular mechanisms retrieval in the interference paradigms of Pavlovian
learning. Psychological Bulletin, 114, 80–99.
of conditioning. The behavioral analysis of Pavlov- doi:10.1037/0033-2909.114.1.80
ian conditioning will continue to inform the direction
Bouton, M. E. (2004). Context and behavioral processes
of these neurobiological studies, because the key in extinction. Learning and Memory, 11, 485–494.
questions that have motivated Pavlovian research doi:10.1101/lm.78804
are now shifting into the neurobiological domain Bouton, M. E., & Bolles, R. C. (1979). Contextual con-
(see Chapter 15, this volume). The sophisticated trol of the extinction of conditioned fear. Learning
301
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
and Motivation, 10, 445–466. doi:10.1016/0023- Cunningham, C. L., Gremel, C. M., & Groblewski, P. A.
9690(79)90057-2 (2006). Drug-induced conditioned place preference
Bouton, M. E., Nelson, J. B., & Rosas, J. M. (1999). and aversion in mice. Nature Protocols, 1, 1662–1670.
Stimulus generalization, context change, and doi:10.1038/nprot.2006.279
forgetting. Psychological Bulletin, 125, 171–186. Cunningham, C. L., Okorn, D. M., & Howard, C. E.
doi:10.1037/0033-2909.125.2.171 (1997). Interstimulus interval determines whether
Bouton, M. E., & Peck, C. A. (1992). Spontaneous recov- ethanol produces conditioned place preference or
ery in cross-motivational transfer (countercondition- aversion in mice. Animal Learning and Behavior, 25,
ing). Animal Learning and Behavior, 20, 313–321. 31–42. doi:10.3758/BF03199022
doi:10.3758/BF03197954 Cunningham, C. L., Smith, R., & McMullin, C. (2003).
Bouton, M. E., & Ricker, S. T. (1994). Renewal of extin- Competition between ethanol-induced reward
guished responding in a second context. Animal and aversion in place conditioning. Learning and
Learning and Behavior, 22, 317–324. doi:10.3758/ Behavior, 31, 273–280. doi:10.3758/BF03195988
BF03209840 Davis, M., Myers, K. M., Chhatwal, J., & Ressler, K. J.
Bouton, M. E., Westbrook, R. F., Corcoran, K. A., & (2006). Pharmacological treatments that facilitate
Maren, S. (2006). Contextual and temporal modu- extinction of fear: Relevance to psychotherapy.
lation of extinction: Behavioral and biological Neurotherapeutics, 3, 82–96. doi:10.1016/j.nurx.
mechanisms. Biological Psychiatry, 60, 352–360. 2005.12.008
doi:10.1016/j.biopsych.2005.12.015 Delamater, A. R. (1996). Effects of several extinction
Brown, P. L., & Jenkins, H. M. (1968). Auto-shaping of treatments upon the integrity of Pavlovian stimulus-
the pigeon’s key-peck. Journal of the Experimental outcome associations. Animal Learning and Behavior,
Analysis of Behavior, 11, 1–8. doi:10.1901/jeab. 24, 437–449. doi:10.3758/BF03199015
1968.11-1
Delamater, A. R. (1997). Selective reinstatement of
Burns, M., & Domjan, M. (2001). Topography of spa- stimulus-outcome associations. Animal Learning and
tially directed conditioned responding: Effects of Behavior, 25, 400–412. doi:10.3758/BF03209847
context and trial duration. Journal of Experimental
Psychology: Animal Behavior Processes, 27, 269–278. Delamater, A. R. (2004). Experimental extinction in
doi:10.1037/0097-7403.27.3.269 Pavlovian conditioning: Behavioural and neurosci-
ence perspectives. Quarterly Journal of Experimental
Chang, R. C., Stout, S., & Miller, R. R. (2004). Comparing Psychology B: Comparative and Physiological Psychology,
excitatory backward and forward conditioning. 57, 97–132. doi:10.1080/02724990344000097
Quarterly Journal of Experimental Psychology B:
Comparative and Physiological Psychology, 57, 1–23. Delamater, A. R., & Holland, P. C. (2008). The influ-
doi:10.1080/02724990344000015 ence of CS–US interval on several different indices
of learning in appetitive conditioning. Journal of
Cole, S., & McNally, G. P. (2007). Temporal-difference Experimental Psychology: Animal Behavior Processes,
prediction errors and Pavlovian fear conditioning: 34, 202–222. doi:10.1037/0097-7403.34.2.202
Role of NMDA and opioid receptors. Behavioral
Neuroscience, 121, 1043–1052. doi:10.1037/0735- Denniston, J. C., Chang, R. C., & Miller, R. R. (2003).
7044.121.5.1043 Massive extinction treatment attenuates the
renewal effect. Learning and Motivation, 34, 68–86.
Collins, B. N., & Brandon, T. H. (2002). Effects of doi:10.1016/S0023-9690(02)00508-8
extinction context and retrieval cues on alcohol cue
reactivity among nonalcoholic drinkers. Journal of Dibbets, P., Havermans, R., & Arntz, A. (2008). All we
Consulting and Clinical Psychology, 70, 390–397. need is a cue to remember: The effect of an extinc-
doi:10.1037/0022-006X.70.2.390 tion cue on renewal. Behaviour Research and Therapy,
46, 1070–1077. doi:10.1016/j.brat.2008.05.007
Colwill, R. M. (1994). Associative representations of
instrumental contingencies. Psychology of Learning Dickinson, A. (1989). Expectancy theory in animal
and Motivation, 31, 1–72. doi:10.1016/S0079-7421 conditioning. In S. B. Klein & R. R. Mowrer (Ed.),
(08)60408-9 Contemporary learning theories: Pavlovian condition-
ing and the status of traditional learning theory
Colwill, R. M., & Motzkin, D. K. (1994). Encoding of
(pp. 279–308). Hillsdale, NJ: Erlbaum.
the unconditioned stimulus in Pavlovian condition-
ing. Animal Learning and Behavior, 22, 384–394. Domjan, M. (1980). Effects of the intertrial interval on
doi:10.3758/BF03209158 taste-aversion learning in rats. Physiology and Behavior,
Cunningham, C. L. (1981). Association between the ele- 25, 117–125. doi:10.1016/0031-9384(80)90191-2
ments of a bivalent compound stimulus. Journal of Estes, W. K. (1950). Toward a statistical theory of learn-
Experimental Psychology: Animal Behavior Processes, ing. Psychological Review, 57, 94–107. doi:10.1037/
7, 425–436. doi:10.1037/0097-7403.7.4.425 h0058559
302
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Fanselow, M. S. (1980). Conditioned and unconditional Holland, P. C. (1999). Overshadowing and blocking as
components of post-shock freezing. Pavlovian Journal acquisition deficits: No recovery after extinction of
of Biological Science, 15, 177–182. overshadowing or blocking cues. Quarterly Journal
of Experimental Psychology B: Comparative and
Fanselow, M. S. (1982). The postshock activity burst. Physiological Psychology, 52, 307–333. doi:10.1080/
Animal Learning and Behavior, 10, 448–454. doi:10. 027249999393022
3758/BF03212284
Holland, P. C. (2000). Trial and intertrial durations in
Fanselow, M. S. (1990). Factors governing one-trial con- appetitive conditioning in rats. Animal Learning and
textual conditioning. Animal Learning and Behavior, Behavior, 28, 121–135. doi:10.3758/BF03200248
18, 264–270. doi:10.3758/BF03205285
Holland, P. C. (2004). Relations between Pavlovian-
Gallistel, C. R., Fairhurst, S., & Balsam, P. (2004). The instrumental transfer and reinforcer devalua-
learning curve: Implications of a quantitative analy- tion. Journal of Experimental Psychology: Animal
sis. Proceedings of the National Academy of Sciences Behavior Processes, 30, 104–117. doi:10.1037/0097-
of the United States of America, 101, 13124–13131. 7403.30.2.104
doi:10.1073/pnas.0404965101
Holland, P. C., & Straub, J. J. (1979). Differential effects
Gallistel, C. R., & Gibbon, J. (2000). Time, rate, and of two ways of devaluing the unconditioned stimulus
conditioning. Psychological Review, 107, 289–344. after Pavlovian appetitive conditioning. Journal of
doi:10.1037/0033-295X.107.2.289 Experimental Psychology: Animal Behavior Processes,
5, 65–78. doi:10.1037/0097-7403.5.1.65
Gibbon, J., Baldock, M. D., Locurto, C., Gold, L., &
Terrace, H. S. (1977). Trial and intertrial dura- Hupka, R. B., Kwaterski, S. E., & Moore, J. W. (1970).
tions in autoshaping. Journal of Experimental Conditioned diminution of the UCR: Differences
Psychology: Animal Behavior Processes, 3, 264–284. between the human eyeblink and the rabbit nicti-
doi:10.1037/0097-7403.3.3.264 tating membrane response. Journal of Experimental
Psychology, 83, 45–51. doi:10.1037/h0028584
Gibbon, J., & Balsam, P. (1981). Spreading association in
time. In C. M. Locurto, H. S. Terrace, & J. Gibbon Iordanova, M. D., McNally, G. P., & Westbrook, R. F.
(Eds.), Autoshaping and conditioning theory (pp. 219– (2006). Opioid receptors in the nucleus accum-
253). New York, NY: Academic Press. bens regulate attentional learning in the blocking
paradigm. Journal of Neuroscience, 26, 4036–4045.
Good, M., & Macphail, E. M. (1994). Hippocampal lesions doi:10.1523/JNEUROSCI.4679-05.2006
in pigeons (Columba livia) disrupt reinforced preex-
posure but not overshadowing or blocking. Quarterly Kamin, L. J. (1968). “Attention-like” processes in clas-
Journal of Experimental Psychology B: Comparative and sical conditioning. In M. R. Jones (Ed.), Miami
Physiological Psychology, 47, 263–291. Symposium on the Prediction of Behavior, 1967:
Aversive stimulation (pp. 9–31). Coral Gables, FL:
Hall, G., & Honey, R. C. (1989). Contextual effects in University of Miami Press.
conditioning, latent inhibition, and habituation: Kamin, L. J. (1969). Predictability, surprise, attention,
Associative and retrieval functions of contextual and conditioning. In B. A. Campbell & R. M. Church
cues. Journal of Experimental Psychology: Animal (Eds.), Punishment and aversive behavior (pp. 279–
Behavior Processes, 15, 232–241. doi:10.1037/0097- 296). New York, NY: Meredith.
7403.15.3.232
Kehoe, E. J., Olsen, K. N., Ludvig, E. A., & Sutton, R. S.
Harris, J. A. (2006). Elemental representations of stimuli (2009). Scalar timing varies with response magni-
in associative learning. Psychological Review, 113, tude in classical conditioning of the nictitating mem-
584–605. doi:10.1037/0033-295X.113.3.584 brane response of the rabbit (Oryctolagus cuniculus).
Herrnstein, R. J. (1970). On the law of effect. Journal of Behavioral Neuroscience, 123, 212–217. doi:10.1037/
the Experimental Analysis of Behavior, 13, 243–266. a0014122
doi:10.1901/jeab.1970.13-243 Konorski, J. (1948). Conditioned reflexes and neuron orga-
Heth, C. D. (1976). Simultaneous and backward fear nization. Cambridge, England: Cambridge University
conditioning as a function of number of CS–UCS Press.
pairings. Journal of Experimental Psychology: Animal Konorski, J. (1967). Integrative activity of the brain: An
Behavior Processes, 2, 117–129. doi:10.1037/0097- interdisciplinary approach. Chicago, IL: University of
7403.2.2.117 Chicago Press.
Holland, P. C. (1977). Conditioned stimulus as a deter- Kremer, E. F. (1978). The Rescorla–Wagner model:
minant of the form of the Pavlovian conditioned Losses in associative strength in compound condi-
response. Journal of Experimental Psychology: Animal tioned stimuli. Journal of Experimental Psychology:
Behavior Processes, 3, 77–104. doi:10.1037/0097- Animal Behavior Processes, 4, 22–36. doi:10.1037/
7403.3.1.77 0097-7403.4.1.22
303
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
Lattal, K. M. (1999). Trial and intertrial durations in Nakajima, S., Tanaka, S., Urushihara, K., & Imada,
Pavlovian conditioning: Issues of learning and per- H. (2000). Renewal of extinguished lever-press
formance. Journal of Experimental Psychology: Animal responses upon return to the training context.
Behavior Processes, 25, 433–450. doi:10.1037/0097- Learning and Motivation, 31, 416–431. doi:10.1006/
7403.25.4.433 lmot.2000.1064
Lattal, K. M. (2007). Extinction and the erasure of memo- Nelson, J. B. (2002). Context specificity of excitation
ries. Psychological Science Agenda, 21, 3–4, 16–18. and inhibition in ambiguous stimuli. Learning and
Lattal, K. M., Mullen, M. T., & Abel, T. (2003). Motivation, 33, 284–310. doi:10.1006/lmot.2001.1112
Extinction, renewal, and spontaneous recovery of Pavlov, I. P. (1927). Conditioned reflexes: An investiga-
a spatial preference. Behavioral Neuroscience, 117, tion of the physiological activity of the cerebral cortex.
1017–1028. doi:10.1037/0735-7044.117.5.1017 London, England: Oxford University Press.
Lattal, K. M., & Nakajima, S. (1998). Overexpectation Pearce, J. M. (1987). A model for stimulus generalization
in appetitive Pavlovian and instrumental condition- in Pavlovian conditioning. Psychological Review, 94,
ing. Animal Learning and Behavior, 26, 351–360. 61–73. doi:10.1037/0033-295X.94.1.61
doi:10.3758/BF03199227
Pearce, J. M. (1994). Similarity and discrimination:
Lattal, K. M., & Stafford, J. M. (2008). What does it take
A selective review and a connectionist model.
to demonstrate memory erasure? Theoretical com-
Psychological Review, 101, 587–607. doi:10.1037/
ment on Norrholm et al. Behavioral Neuroscience,
0033-295X.101.4.587
122, 1186–1190, 2008. doi:10.1037/a0012993
Leri, F., & Rizos, Z. (2005). Reconditioning of drug- Pearce, J. M., & Bouton, M. E. (2001). Theories of
related cues: A potential contributor to relapse after associative learning in animals. Annual Review of
drug reexposure. Pharmacology, Biochemistry and Psychology, 52, 111–139. doi:10.1146/annurev.
Behavior, 80, 621–630. doi:10.1016/j.pbb.2005.01.013 psych.52.1.111
Leung, H. T., & Westbrook, R. F. (2008). Spontaneous Pearce, J. M., & Hall, G. (1980). A model for Pavlovian
recovery of extinguished fear responses deepens learning: Variations in the effectiveness of conditioned
their extinction: A role for error-correction mecha- but not of unconditioned stimuli. Psychological Review,
nisms. Journal of Experimental Psychology: Animal 87, 532–552. doi:10.1037/0033-295X.87.6.532
Behavior Processes, 34, 461–474. doi:10.1037/0097- Peck, C. A., & Bouton, M. E. (1990). Context and per-
7403.34.4.461 formance in aversive-to-appetitive and appetitive-to-
Mackintosh, N. J. (1975). A theory of attention: aversive transfer. Learning and Motivation, 21, 1–31.
Variations in the associability of stimuli with rein- doi:10.1016/0023-9690(90)90002-6
forcement. Psychological Review, 82, 276–298. Perkins, C. C., Jr., Beavers, W. O., Hancock, R. A., Jr.,
doi:10.1037/h0076778 Hemmendinger, P. C., Hemmendinger, D., & Ricci,
Malvaez, M., Sanchis-Segura, C., Vo, D., Lattal, K. M., J. A. (1975). Some variables affecting rate of key
& Wood, M. A. (2010). Modulation of chromatin pecking during response-independent procedures
modification facilitates extinction of cocaine-induced (autoshaping). Journal of the Experimental Analysis of
conditioned place preference. Biological Psychiatry, Behavior, 24, 59–72. doi:10.1901/jeab.1975.24-59
67, 36–43. doi:10.1016/j.biopsych.2009.07.032
Rescorla, R. A. (1966). Predictability and number of pair-
McEchron, M. D., Tseng, W., & Disterhoft, J. F. (2003). ings in Pavlovian fear conditioning. Psychonomic
Single neurons in CA1 hippocampus encode trace Science, 4, 383–384.
interval duration during trace heart rate (fear)
conditioning in rabbit. Journal of Neuroscience, 23, Rescorla, R. A. (1967). Pavlovian conditioning and its
1535–1547. proper control procedures. Psychological Review, 74,
71–80. doi:10.1037/h0024109
McLaren, I. P. L., & Mackintosh, N. J. (2000). An ele-
mental model of associative learning: I. Latent inhibi- Rescorla, R. A. (1968). Probability of shock in the pres-
tion and perceptual learning. Animal Learning and ence and absence of CS in fear conditioning. Journal
Behavior, 28, 211–246. doi:10.3758/BF03200258 of Comparative and Physiological Psychology, 66, 1–5.
doi:10.1037/h0025984
McNally, G. P., & Westbrook, R. F. (2006). Predicting
danger: The nature, consequences, and neural Rescorla, R. A. (1969). Conditioned inhibition of fear
mechanisms of predictive fear learning. Learning and resulting from negative CS-US contingencies. Journal
Memory, 13, 245–253. doi:10.1101/lm.196606 of Comparative and Physiological Psychology, 67,
504–509. doi:10.1037/h0027313
Miller, R. R., & Barnet, R. C. (1993). The role of time
in elementary associations. Current Directions in Rescorla, R. A. (1988a). Behavioral studies of Pavlovian
Psychological Science, 2, 106–111. doi:10.1111/1467- conditioning. Annual Review of Neuroscience, 11,
8721.ep10772577 329–352. doi:10.1146/annurev.ne.11.030188.001553
304
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Rescorla, R. A. (1988b). Pavlovian conditioning: It’s not Rosas, J. M., & Bouton, M. E. (1997). Renewal of a
what you think it is. American Psychologist, 43, conditioned taste aversion upon return to the con-
151–160. doi:10.1037/0003-066X.43.3.151 ditioning context after extinction in another one.
Rescorla, R. A. (1993). Inhibitory associations between S Learning and Motivation, 28, 216–229. doi:10.1006/
and R in extinction. Animal Learning and Behavior, lmot.1996.0960
21, 327–336. doi:10.3758/BF03197998 Rothbaum, B. O., & Davis, M. (2003). Applying learning
Rescorla, R. A. (2001a). Are associative changes in acquisi- principles to the treatment of post-trauma reactions.
tion and extinction negatively accelerated? Journal of Annals of the New York Academy of Sciences, 1008,
Experimental Psychology: Animal Behavior Processes, 112–121. doi:10.1196/annals.1301.012
27, 307–315. doi:10.1037/0097-7403.27.4.307 Rothbaum, B. O., & Schwartz, A. C. (2002). Exposure
Rescorla, R. A. (2001b). Experimental extinction. In R. therapy for posttraumatic stress disorder. American
R. Mowrer & S. Klein (Eds.), Handbook of contempo- Journal of Psychotherapy, 56, 59–75.
rary learning theories (pp. 119–154). Mahwah, NJ: Rudy, J. W., & O’Reilly, R. C. (1999). Contextual fear
Erlbaum. conditioning, conjunctive representations, pat-
Rescorla, R. A. (2003). Elemental and configural encod- tern completion, and the hippocampus. Behavioral
ing of the conditioned stimulus. Quarterly Journal Neuroscience, 113, 867–880. doi:10.1037/0735-
of Experimental Psychology B: Comparative and 7044.113.5.867
Physiological Psychology, 56, 161–176. doi:10.1080/ Schafe, G. E., Sollars, S. I., & Bernstein, I. L. (1995).
02724990244000089 The CS–US interval and taste aversion learning: A
Rescorla, R. A. (2004). Spontaneous recovery. Learning brief look. Behavioral Neuroscience, 109, 799–802.
and Memory, 11, 501–509. doi:10.1101/lm.77504 doi:10.1037/0735-7044.109.4.799
Rescorla, R. A., & Coldwell, S. E. (1995). Summation Schmajuk, N. A., Gray, J. A., & Lam, Y. W. (1996). Latent
in autoshaping. Animal Learning and Behavior, 23, inhibition: A neural network approach. Journal of
314–326. doi:10.3758/BF03198928 Experimental Psychology: Animal Behavior Processes,
22, 321–349. doi:10.1037/0097-7403.22.3.321
Rescorla, R. A., & Cunningham, C. L. (1977). The era-
sure of reinstated fear. Animal Learning and Behavior, Schultz, W. (1998). Predictive reward signal of dopamine
5, 386–394. doi:10.3758/BF03209584 neurons. Journal of Neurophysiology, 80, 1–27.
Rescorla, R. A., & Cunningham, C. L. (1978). Stafford, J. M., & Lattal, K. M. (2011). Is an epigenetic
Within-compound flavor associations. Journal of switch the key to persistent extinction? Neurobiology
Experimental Psychology: Animal Behavior Processes, of Learning and Memory, 96, 35–40. doi:10.1016/j.
4, 267–275. doi:10.1037/0097-7403.4.3.267 nlm.2011.04.012
Rescorla, R. A., & Holland, P. C. (1976). Some behav- Stout, S. C., & Miller, R. R. (2007). Sometimes-
ioral approaches to the study of learning. In M. competing retrieval (SOCR): A formalization of the
R. Rosenzweig & E. L. Bennett (Eds.), Neural comparator hypothesis. Psychological Review, 114,
mechanisms of learning and memory (pp. 165–192). 759–783. doi:10.1037/0033-295X.114.3.759
Cambridge, MA: MIT Press. Thein, T., Westbrook, R. F., & Harris, J. A. (2008). How
Rescorla, R. A., & Wagner, A. R. (1972). A theory of the associative strengths of stimuli combine in com-
Pavlovian conditioning: Variations in the effective- pound: Summation and overshadowing. Journal of
ness of reinforcement and nonreinforcement. In Experimental Psychology: Animal Behavior Processes,
A. H. Black & W. F. Prokasy (Eds.), Classical condi- 34, 155–166. doi:10.1037/0097-7403.34.1.155
tioning II: Current research and theory (pp. 64–99). Twitmeyer, E. B. (1905). Knee-jerks without stimula-
New York, NY: Appleton-Century-Crofts. tion of the patellar tendon. Psychological Bulletin, 2,
Rizley, R. C., & Rescorla, R. A. (1972). Associations in 43–44.
second-order conditioning and sensory precondi- Vansteenwegen, D., Vervliet, B., Hermans, D., Beckers, T.,
tioning. Journal of Comparative and Physiological Baeyens, F., & Eelen, P. (2006). Stronger renewal
Psychology, 81, 1–11. doi:10.1037/h0033333 in human fear conditioning when tested with an
Robbins, S. J. (1988). Role of context in performance acquisition retrieval cue than with an extinction
on a random schedule in autoshaping. Journal of retrieval cue. Behaviour Research and Therapy, 44,
Experimental Psychology: Animal Behavior Processes, 1717–1725. doi:10.1016/j.brat.2005.10.014
14, 413–424. doi:10.1037/0097-7403.14.4.413 Vaughan, W., Jr. (1982). Choice and the Rescorla-Wagner
Robbins, S. J. (1990). Mechanisms underlying spontane- model. In M. L. Commons, R. J. Herrnstein, & H.
ous recovery in autoshaping. Journal of Experimental Rachlin (Eds.), Quantitative analyses of behavior: Vol.
Psychology: Animal Behavior Processes, 16, 235–249. 2. Matching and maximizing accounts (pp. 263–279).
doi:10.1037/0097-7403.16.3.235 Cambridge, MA: Ballinger.
305
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Matthew Lattal
Wagner, A. R. (1981). SOP: A model of automatic mem- Wagner, A. R., Logan, F. A., Haberlandt, K., & Price, T.
ory processing in animal behavior. In N. E. Spear (1968). Stimulus selection in animal discrimination
& R. R. Miller (Eds.), Information processing in ani- learning. Journal of Experimental Psychology, 76
mals: Memory mechanisms (pp. 5–47). Hillsdale, NJ: (2, Pt. 1), 171–180. doi:10.1037/h0025414
Erlbaum.
Wagner, A. R., & Rescorla, R. A. (1972). Inhibition in
Wagner, A. R. (2003). Context-sensitive elemental the- Pavlovian conditioning: Application of a theory. In
ory. Quarterly Journal of Experimental Psychology B: R. A. Boakes & M. S. Halliday (Eds.), Inhibition and
Comparative and Physiological Psychology, 56, 7–29. learning (pp. 301–336). New York, NY: Academic
doi:10.1080/02724990244000133 Press.
Wagner, A. R., & Brandon, S. E. (1989). Evolution of Westbrook, R. F., Iordanova, M., McNally, G.,
a structured connectionist model of Pavlovian Richardson, R., & Harris, J. A. (2002). Reinstatement
conditioning (AESOP). In S. B. Klein & R. R. Mowrer of fear to an extinguished conditioned stimulus:
(Eds.), Contemporary learning theories: Pavlovian con- Two roles for context. Journal of Experimental
ditioning and the status of traditional learning theory Psychology: Animal Behavior Processes, 28, 97–110.
(pp. 149–189). Hillsdale, NJ: Erlbaum. doi:10.1037/0097-7403.28.1.97
306
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 14
The Allocation
of Operant Behavior
Randolph C. Grace and Andrew D. Hucks
Choice has long been a major topic of research in after an unpredictable delay had elapsed according
the experimental analysis of behavior. Although to the schedule value. Pigeons were trained with a
quantifying just how major is difficult, a search of given pair of VI schedules in daily sessions until
the PsycInfo database found that of the 3,576 arti- response rates had stabilized, after which the sched-
cles published in the Journal of the Experimental ules were changed and a new condition was begun.
Analysis of Behavior from 1958 through 2008, 452 Herrnstein found that the proportion of responses to
(12.6%) included the word choice in the abstract. each key was approximately equal to the proportion
However, this figure likely underestimates the of reinforcers obtained from that key:
extent of research on response allocation in the
operant literature. Figure 14.1 displays results of BL BL
=
the database search at 5-year intervals and shows BR + BL BL + BR
that the incidence of choice has increased from
or
less than 3% of the abstracts in the journal’s first
10 years to an average of approximately 20% of the BL R L
= . (1)
abstracts in the past 25 years. Mazur (2001) found BR R R
that the proportion of Journal of the Experimental
Analysis of Behavior articles reporting experiments In Equation 1, known as the matching law, B is
in which “subjects could choose among two or the response rate and R is the reinforcement rate,
more operant responses” (p. 96) was 47% in 1997 with the subscripts L and R representing the left and
through 1998. There is no doubt that researchers right alternative schedules, respectively. Herrnstein’s
in the operant tradition have allocated a consider- (1961) report of matching of relative response and
able proportion of their behavior to the study of reinforcement rates in concurrent schedules stimu-
choice. lated a great deal of subsequent research on choice.
The article usually cited as the starting point for For previous reviews of research on matching and
research on operant choice is Herrnstein (1961). choice behavior, see Davison and McCarthy (1988),
This classic experiment was one of the first pub- Fisher and Mazur (1997), R. R. Miller and Grace
lished reports (although not the first; see Findley, (2003), Nevin (1998), Staddon and Cerutti (2003),
1958) to use the concurrent-schedules procedure. and Williams (1988, 1994).
In Herrnstein’s version of this procedure, hungry Most research on choice after Herrnstein (1961)
pigeons could respond on two lighted keys in an can be described in terms of two major categories:
experimental chamber. Each key was associated (a) studies that have explored the origin of matching
with an independent variable-interval (VI) schedule, behavior by testing different explanatory theories
which reinforced responding with access to food and models and (b) studies that have used the
DOI: 10.1037/13937-014
307
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
0.25
0.20
Relave Frequency
0.15
0.10
0.05
Year
Figure 14.1. The relative frequency of articles published in the Journal of

the Experimental Analysis of Behavior that included choice in the keywords
or abstract.
empirical generalization of matching as a starting (Fantino, 1969), the contextual choice model
point for developing quantitative models for choice (Grace, 1994), and the hyperbolic value-added
that are applicable to a wide range of situations. model (Mazur, 2001). In the third section, we
Many different theories for matching have been pro- review research on the acquisition of choice behav-
posed, including momentary maximizing (Hinson & ior. In contrast with previous research that has typ-
Staddon, 1983a, 1983b; Shimp, 1966, 1969), melio- ically used steady-state methodologies, research on
ration (Herrnstein & Vaughan, 1980), molar maxi- acquisition has explored how response allocation
mizing (Baum, 1981; Rachlin, Green, Kagel, & adapts when the conditions of reinforcement are
Battalio, 1976; Staddon & Motheral, 1978), the changed.
kinetic model (Myerson & Miezin, 1980), the cumu-
lative effects model (Davis, Staddon, Machado, &
Matching as a Descriptive
Palmer, 1993), fix and sample (Baum, Schwendi-
Framework for Choice
man, & Bell, 1999), and the stay–switch model
(MacDonall, 2009). Research to test these and other A major line of research following Herrnstein
theories has yielded many interesting results and (1961) has been the evolution of the matching law
increased researchers’ understanding of behavior, into a descriptive framework for choice behavior.
but it is fair to say that no consensus has emerged in The first significant proposal was Baum and Rach-
favor of a generally accepted explanation for match- lin’s (1969) suggestion that the independent
ing. Because this issue has no clear resolution and variable was the relative value obtained from the
space is limited, we focus on research in the second alternative schedules. Noting that studies on con-
category. current schedules after Herrnstein had found
This chapter is organized as follows. In the first matching of response allocation to relative rein-
section, we review the evolution of the matching forcement magnitude (i.e., seconds of access to
law as a model for choice under concurrent sched- grain; Catania, 1963) and to relative immediacy of
ules, culminating in the generalized matching law reinforcement (i.e., the reciprocal of delay to rein-
(Baum, 1974a). Next, we consider research on forcement; Chung & Herrnstein, 1967), Baum and
choice between delayed reinforcers in the concurrent- Rachlin (1969) suggested that these results, as well
chains and adjusting-delay procedures, which as Herrnstein’s, could be understood as matching
has led to several models for choice based on the to the relative value of the reinforcement obtained
matching law, including delay-reduction theory from the alternatives, with value defined as a
308
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
The Allocation of Operant Behavior
c ombination of reinforcement rate, immediacy, and Generalized Matching
magnitude: 1.2
BL R L 1/ D L M L X L VL
= ⋅ ⋅ ⋅ = . (2) 0.8
BR R R 1/ DR M R X R VR
Log Response Ratio

0.4
In Equation 2, D is the delay to reinforcement, y = 0.81x + 0.04
M is the magnitude of reinforcement, X is an R² = 0.99
0
unspecified aspect of reinforcement (e.g., hedonic
-1.2 -0.8 -0.4 0 0.4 0.8 1.2
quality), and V is the reinforcement value, with
-0.4
subscripts L and R again indicating the left and right Data
Matching
alternatives. According to Equation 2, known as the
-0.8 Overmatching
concatenated matching law, value is determined
multiplicatively by the reinforcer dimensions on -1.2
which the alternatives differ. Equation 2 is a signifi-
Log Reinforcement Ratio
cant conceptual advance because it proposes that
matching is a general principle of choice that is Figure 14.2. Illustration of generalized matching
potentially applicable to any reinforcement variable. law. The dotted line indicates perfect matching, the
The next major step in the evolution of matching dashed line indicates overmatching (i.e., a > 1). The
regression line (solid line) indicates the fit of the gener-
was taken by Baum (1974a), who noted that some alized matching law to the data. The slope (0.81) is an
empirical deviations from matching could be under- estimate of sensitivity (a) and exemplifies undermatch-
stood in terms of a general version of Equation 1: ing. The intercept (0.04) is an estimate of log b. Data
a
from Grace (1993).
BL R 
= b L  , (3a)
BR  RR  parameters and are not explicitly linked with an
or, in logarithmic form, underlying process. Estimates of sensitivity and bias
are typically obtained by regressing the log response
BL R
log = log b + a log L . (3b) ratio on the log reinforcement ratio (Figure 14.2).
BR RR See Davison and Elliffe (2009) for a recent discus-
Equation 3 is known as the generalized matching sion of statistical issues related to regression and
law. It has two parameters: bias (b), which represents parameter estimation for behavioral choice data.
a constant proportionality in responding unrelated
to reinforcement rate (e.g., position preference), and Sources of Undermatching
an exponent (a), which represents sensitivity to rein- Debate about the value of the sensitivity parameter
forcement rate. Typically, the logarithmic form of in the generalized matching law has been consider-
the generalized matching law is used (Equation 3b), able. The critical question has been whether the
in which response allocation (measured as the log normative or typical value of a equals 1.0, consis-
response ratio) is a linear function of the log rein- tent with matching, or is less than 1.0, indicating
forcement ratio. Sensitivity and log bias correspond undermatching. Baum (1979) reported an analysis
to the slope and intercept, respectively, of this func- in which Equation 3b was fitted to all the archival
tion. When a = b = 1, the original matching law data available at the time (103 data sets from 23
(Equation 1) results. Values of a that are less than 1 studies). He found that the modal value of a was
are termed undermatching because response alloca- 1.0 when choice was measured as time allocation,
tion is less extreme than reinforcer allocation, but approximately 0.8 when response allocation
whereas values of a that are more than 1 are termed was the dependent variable. Baum examined varia-
overmatching (i.e., response allocation more extreme tion in values of a for response allocation across
than reinforcer allocation). It is important to note studies and found that results from the laboratories
that sensitivity and log bias are empirically derived of the two investigators who had contributed the
309
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
largest number of data sets (Davison and Baum) pigeons and rats have responded under dynamic
differed systematically. For data sets reported by conditions (see the section Acquisition in Concur-
Davison, modal values were near 0.8, whereas for rent Schedules later in this chapter). However, a
those reported by Baum they were approximately recent meta-analysis of concurrent-schedules stud-
1.0. Baum suggested that procedural variables (e.g., ies failed to find systematic deviations in the fits
type of VI schedule, change of delay duration, level of the generalized matching law that would be
of deprivation) might have contributed to the dif- expected with a fix-and-sample pattern (Sutton,
ferent results and proposed several factors that Grace, McLean, & Baum, 2008).
could yield undermatching. These factors included
asymmetrical pausing (which would affect response Generalized Matching Law as
allocation but not time allocation as typically mea- Descriptive Model
sured), heterogeneity of preference, and patterns of Baum (1979) found that Equation 3b accounted
changeover responding. Other possible causes of for 90.3% of the variance in response allocation,
undermatching include reduced discriminability of averaged across the archival data sets. Thus, in the
the schedules (Davison & Jenkins, 1985) or type of generalized form represented in Equation 3, the
VI schedule used (e.g., exponential vs. arithmetic matching law provides an excellent description of
progressions; Elliffe & Alsop, 1996). Baum con- choice in concurrent schedules. The power function
cluded that matching should be considered norma- generalization in Equation 3 may be joined with
tive, with deviations to be attributed to procedural Baum and Rachlin’s (1969) suggestion that different
or other factors. dimensions of reinforcement are combined multipli-
Baum et al. (1999) offered an alternative explana- catively to yield a concatenated generalized match-
tion for undermatching in terms of a specific pattern ing law:
of choice responding. They studied the relationship
 R  r  1/ D  d  M  m
a a a
between average number of responses per visit to an BL
= b L   L
  L (4a)
alternative and choice in concurrent VI–VI schedules BR  R R  1/ DR   M R 
with pigeons. They found that average responses per or
visit for the richer alternative increased directly with
response allocation but remained low and approxi- BL R 1/ D L
log = log b + a r log L + a d log
mately constant for the leaner alternative, regardless BR RR 1/ DR
of preference (see Baum et al., 1999, Figure 5). Baum ML
et al. described this pattern as “fix and sample” + a m log . (4b)
MR
because it implied a tendency for subjects to remain
at the richer alternative while occasionally sampling According to Equation 4b, response allocation in
the leaner alternative, and they suggested that it was concurrent schedules is determined by the additive
consistent with expectations based on forging theory combination of different dimensions of reinforce-
(Houston & McNamara, 1981). Baum et al. showed ment, each with its own sensitivity parameter (ar, ad,
that when their results were expressed as preference and am, for rate, delay, and magnitude, respectively).
for the richer over the leaner alternative, under- Equation 4 is a significant extension of the
matching could be produced by a bias toward the matching law, enabling it to apply to a broad range
leaner alternative, that is, for subjects to make more of choice situations. One of the most important
responses to the lean alternative per visit than pre- is self-control, which has been a major focus of
dicted by matching. research because of its obvious relevance for human
Whether fix and sample can provide a convinc- behavior (Logue, Rodriguez, Pena-Correal, &
ing account of undermatching remains unclear. Mauro, 1984). In a self-control situation, subjects
Baum and Davison (2004) and Aparicio and Baum face a choice between a small reinforcer available
(2006) have found evidence for the development of immediately (or after a short delay) and a larger
the fix-and-sample pattern in experiments in which reinforcer available after a longer delay. Typically,
310
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
overall reinforcement is maximized by choosing comparison is a rich and rapidly developing branch
the delayed, larger reinforcer, which is termed self- of statistical theory (for accessible reviews, see Burn-
control (Rachlin & Green, 1972; see Rachlin, 1995, ham & Anderson, 2002; Cutting, 2000; Pitt, Myung,
for review). By contrast, choosing the smaller, & Zhang, 2002). As noted earlier, the generalized
less delayed reinforcer is termed impulsivity. For matching law accounts for a high proportion of
example, consider that pigeons are given a choice the variance in concurrent-schedules data (Baum,
between a small reinforcer (e.g., 2-second access to 1979), but goodness of fit can provide only weak
grain) delayed by 1 second or a large reinforcer (e.g., evidence in favor of a model (Roberts & Pashler,
6-second access to grain) delayed by 6 seconds. If 2000). An alternative model might provide a better
for simplicity one assumes that ad = am = 1, then account of the data, or the predictions of the gener-
Equation 4 predicts a 2:1 ratio of responses for the alized matching law might systematically deviate
small reinforcer (i.e., the 6:1 delay ratio is greater from the data.
than the 2:6 magnitude ratio). However, if the
Contingency discriminability model. Davison and
delays to both the small and the large reinforcers
Jenkins (1985) proposed an alternative model for
are increased by the same amount, then Equation 4
concurrent schedules based on the assumption that
predicts a reversal of preference. For example, if the
subjects might not discriminate perfectly between
delays are both increased by 10 seconds, then pre-
the alternatives. This idea was supported by an ear-
dicted preference for the small reinforcer is only
lier study by J. T. Miller, Saunders, and Bourland
33% (the 16:11 delay ratio is no longer enough to
(1980), who arranged a procedure in which the
compensate for the 2:6 magnitude ratio). Empirical
alternative in effect was signaled by one of two line
support for such preference reversals has been
orientations projected on a key and pigeons could
obtained in both human and nonhuman choice
switch between alternatives by pecking a second,
(Green & Snyderman, 1980; Kirby & Herrnstein,
changeover key (Findley, 1958). J. T. Miller et al.
1995; see Green & Myerson, 2004, for review).
found that as the difference between the line orien-
Equation 4 assumes that for each reinforcer
tations decreased, sensitivity to relative reinforce-
variable, response allocation is a power function of
ment rate also decreased. On the basis of this result,
relative reinforcement, and the effects of different
Davison and Jenkins proposed a model for choice in
variables are additive and independent. However, it
concurrent schedules that assumed subjects might
may be viewed more broadly as a framework for the
misallocate reinforcers obtained from the alternatives:
description of behavioral choice. From this perspec-
tive, a matching relationship is assumed—response BL R − pR L + pR R
allocation in a two-alternative choice situation is =b L . (5)
BR R R − pR R + pR L
taken to be a function of the reinforcement obtained
from the alternatives—and the goal of research is to Equation 5 is known as the contingency discrim-
identify the functional laws relating them (Killeen, inability model. According to Equation 5, subjects
1972; Mazur, 2006). We consider in the following match their response allocation to the proportion of
few sections evidence regarding the functional rela- reinforcers allocated to each alternative (with bias,
tion between response allocation and reinforcement b). The parameter p is a “proportional confusion”
rate, magnitude, and delay. parameter that varies between .0 and .5 and deter-
mines the likelihood that reinforcers are misallo-
Reinforcement Rate cated. When p = .0, discrimination is perfect and
Here we consider whether the generalized matching Equation 5 predicts matching. When p = .5, the
law’s assumption of a power function relationship alternatives are not discriminated at all, and subjects
between response and reinforcer allocation is valid. are indifferent. For values of p between .0 and .5,
To determine empirically whether variables are response allocation undermatches relative reinforce-
related by a power function, or some other function, ment rate, with the degree of undermatching deter-
requires a comparison of different models. Model mined by p. Davison and Jenkins (1985) showed
311
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
that Equation 5 accounted for more than 99% of the ratios, with response allocation closer to indifference
variance in predictions of the generalized matching than predicted by the generalized matching law.
law with reinforcer ratios ranging from 1:10 to 10:1 Their results supported the predictions of the con-
and a values ranging from .1 to .9. They argued that tingency discriminability model.
the contingency discriminability model could pro- Baum et al. (1999) noted that some aspects of the
vide an equivalent empirical account of the data as procedure used by Davison and Jones (1995) were
the generalized matching law but was conceptually different from most previous studies using concur-
superior because p was linked to a behavioral pro- rent schedules, particularly the use of two different
cess (i.e., response–reinforcer discriminability), light intensities to signal the alternatives. In addition,
unlike sensitivity in the generalized matching law. Baum et al. suggested that other features of Davison
Davison and colleagues (e.g., Alsop & Davison, and Jones’s procedure, such as the stability criterion
1991; Davison & Nevin, 1999; Nevin, Davison, & or changeover delay, might also have contributed to
Shahan, 2005) have developed the contingency dis- their results. They conducted an experiment that
criminability concept into a range of models for con- examined pigeons’ preference at extreme reinforce-
ditional discrimination and choice, including signal ment ratios using a two-key procedure. Baum et al.
detection and matching-to-sample procedures. provided additional sessions of training in each con-
dition after Davison and Jones’s stability criterion
Comparing the generalized matching law and had been met and arranged conditions both with
contingency discriminability model. Because the and without a changeover delay. They found, con-
contingency discriminability model and generalized trary to Davison and Jones’s results, that the data
matching law make very similar predictions within showed no significant deviations from linearity and
the usual range of reinforcer ratios studied (e.g., were consistent with the generalized matching law,
1:10–10:1), the question of how response allocation even at extreme reinforcer ratios. Baum et al. also
behaves at extreme reinforcer ratios has assumed found that their results did not change when differ-
considerable importance. Davison and Jones (1995) ent stability criteria were used or with the presence
noted that whereas the generalized matching law or absence of the changeover delay. Thus, Davison
predicted that the relationship between log response and Jones’s and Baum et al.’s results conflict and do
and reinforcer ratios was linear over the full range not resolve the question of whether the generalized
of reinforcer ratios, the contingency discriminability matching law or the contingency discriminability
model predicted that the relationship was nonlinear, model provides a better account of concurrent-
with a flattening of preference observed at extreme schedule performance.
reinforcer ratios. To test this prediction, they con- Sutton et al. (2008) pursued an alternative strat-
ducted an experiment in which pigeons were trained egy for comparing these models, which they called
on a switching-key concurrent schedule in which residual meta-analysis. The upper left panel of Fig-
the alternatives differed in terms of luminance on ure 14.3 shows data generated by the generalized
the main key. When an interval sampled from a VI matching law for reinforcer ratios between 1:10
schedule had elapsed, the reinforcer was assigned and 10:1 and the predictions of the contingency
probabilistically to one alternative or the other. discriminability model when fitted to these data.
This arrangement is known as dependent schedul- Conversely, the lower left panel shows data gener-
ing (Stubbs & Pliskoff, 1969) and ensures that the ated by the contingency discriminability model and
obtained relative rate of reinforcement is equal to the predictions of the generalized matching law
the programmed value. The reinforcer ratio was when fitted to these data. Although the fits of both
varied between 256:1 to 1:256 across conditions. models are excellent, with more than 99.9% of the
Davison and Jones showed that the contingency variance accounted for, systematic deviations are
discriminability model predicted that preference apparent. The right-hand panels of Figure 14.3
should deviate systematically from the predictions of show the residuals of the fits for each model (i.e.,
the generalized matching law at extreme reinforcer obtained minus predicted) as a function of the
312
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
1 0.04
0.8
0.03
0.6
0.02
0.4
CDM Residual
Log B1 / B2
0.2 0.01
GML data
0 0
CDM pred
-1.5 -1 -0.5 0 0.5 1 1.5 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
-0.2
-0.01
-0.4
-0.02
-0.6
-0.8 -0.03
-1
-0.04
log R1 / R2 Predicted CDM
1 0.04
0.8
0.03
0.6
0.02
0.4
GML Residual
Log B1 / B2
0.2 0.01
GML data
0 0
CDM pred
-1.5 -1 -0.5 0 0.5 1 1.5 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
-0.2
-0.01
-0.4
-0.02
-0.6
-0.8 -0.03
-1
-0.04
log R1 / R2 Predicted GML
Figure 14.3. The upper left panel shows log response allocation predicted by the generalized matching law
(GML) for a range of reinforcer ratios from 1:10 to 10:1, assuming a = 0.80 and log b = 0 in Equation 14.3b. The
line shows the fit of the contingency discriminability model (CDM), for which a logarithmic transformation of
Equation 14.5 with pr = .067 and log b = 0 accounted for more than 99.9% of the variance. The residuals from the
fits of the CDM are shown in the upper right panel. The bottom panels show the case in which log response allo-
cation data were generated by the CDM with pr = .067 and fitted by the GML, which with a = 0.80 accounted for
more than 99.9% of the variance. The residuals from the fit of the GML are shown in the lower right panel. From
“Comparing the Generalized Matching Law and Contingency Discriminability Model as Accounts of Concurrent
Schedule Performance Using a Residual Meta-Analysis,” by N. P. Sutton, R. C. Grace, A. P. McLean, and
W. M. Baum, 2008, Behavioral Processes, 78, p. 225. Copyright 2008 by Elsevier Ltd. Adapted with permission.
predicted values. The pattern in the residuals can systematic patterns such as those shown in the
be described as a third-order polynomial, which right-hand panels of Figure 14.3 if results were
differs in the sign of the cubic coefficient. If the pooled across studies. Aggregating residuals across
generalized matching law is the true model, then studies should produce an increase in statistical
the residuals of the contingency discriminability power, similar to meta-analysis based on pooling
model should have a positive cubic coefficient, effect sizes from independent studies, which is com-
whereas if the latter is the true model, then the monly used in the social sciences (Rosenthal &
residuals of the generalized matching law should DiMatteo, 2001). Sutton et al. obtained concurrent-
have a negative cubic coefficient. schedules response allocation data from 17 different
Sutton et al. (2008) noted that although the studies (total n = 771 data points). Both the general-
residuals were very small (less than 0.1% of the vari- ized matching law and the contingency discrim-
ance in the data), it might be possible to test for inability model provided an excellent description of
313
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
the data in terms of goodness of fit, with average Choice Between Delayed
variance accounted for equal to 94.7% and 94.3%, Reinforcers
respectively. Residuals were obtained from the fits of
The first investigations of the effects of reinforce-
each model and pooled across data sets. Polynomial
ment delay on concurrent schedules were reported
regressions in which the residuals were regressed
by Chung (1965) and Chung and Herrnstein (1967).
on the predicted values showed that a significant
In the latter study, pigeons responded on concurrent
polynomial with a positive cubic coefficient was
VI 1-minute–VI 1-minute schedules with a 1-second
obtained for the residuals from fits of the contin-
changeover delay. Responses on one key produced
gency discriminability model, whereas the regression
food after a standard delay (either 8 or 16 seconds),
for the residuals from fits of the generalized match-
and responses on the other key produced food after a
ing law was not significant. These findings support
delay that varied between 1 and 30 seconds across
the generalized matching law, because the obtained
conditions. Chung and Herrnstein reported that
pattern in the residuals of the contingency discrim-
response allocation matched the relative immediacy
inability model is just what would be expected if the
(i.e., reciprocal of delay) of reinforcement, consistent
relationship between log response allocation and log
with Equation 2 (cf. Williams & Fantino, 1978).
reinforcement rate was strictly linear over the range
In Chung and Herrnstein’s (1967) study, rein-
of reinforcer ratios in the sample (approximately
forced responses produced a darkening of both keys
10:1–1:10). Thus, Sutton et al.’s results provide
and the houselight and were followed by 3-second
strong support for the generalized matching law
access to grain after the delay. Because the concur-
and its assertion that a power function provides an
rent schedules were equal, their procedure may be
accurate description of the relationship between
viewed as a choice between two outcome schedules
response allocation and relative reinforcement rate.
(e.g., 8-second delay to food vs. 16-second delay to
food). Note that these outcomes are mutually exclu-
Reinforcement Magnitude
sive in that once a left-key response initiates an
In nonhuman research, reinforcement magnitude is
8-second delay to food, that delay and food delivery
typically defined as the quantity of a biologically sig-
must be completed before the keys are illuminated
nificant stimulus per reinforcer, for example, sec-
again, and the pigeon can respond to either key.
onds of access to grain (Catania, 1963), number of
This procedure, in which subjects respond during a
food pellets (Mazur, 1988), or rate of electrical brain
choice phase to produce one of two mutually exclu-
stimulation (Leon & Gallistel, 1998). In contrast to
sive outcome schedules, is known as concurrent
reinforcement rate, comparatively few studies have
chains. Although the outcome schedules were not
varied reinforcer magnitude parametrically while
differentially signaled in Chung and Herrnstein’s
holding other variables constant. In the most exten-
(1967) study, in most cases they have been.
sive study, Landon, Davison, and Elliffe (2003; see
also Elliffe, Davison, & Landon, 2008) varied rela-
tive magnitude across five levels from 7:1 to 1:7 in Concurrent-Chains Procedure
a switching-key concurrent schedule with pigeons. Figure 14.4 shows a diagram of a typical concurrent-
Magnitude was defined as the number of brief chains procedure, with pigeons as subjects. Responses
(1.2-second) food presentations per reinforcement. during a choice phase, or initial links, in which two
Landon et al. found that the generalized matching keys are illuminated white, are reinforced by con-
law provided an excellent description of their data: current VI schedules with access to one of two
Averaged across subjects, the generalized matching mutually exclusive alternatives, known as terminal
law accounted for 99.6% of the variance, and sensi- links. The name of the procedure reflects the fact
tivity to relative magnitude was 0.76. Thus, the avail- that for each key, the initial and terminal links con-
able evidence, although limited, suggests that the stitute a chain schedule. The onset of a terminal link
relationship between response allocation and relative is signaled by a change in color on the key (e.g., left
magnitude can be described as a power function. key to red, right key to green) coupled with darkening
314
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Initial Links human behavior is maintained by stimuli that are not

VI 30 s VI 30 s
biologically significant (e.g., money). In concurrent
W W chains, production of the terminal-link stimuli acts
to reinforce initial-link responding, and so response
allocation in the initial links may be interpreted as
a measure of the relative strength or value of the
R G terminal-link stimuli as conditioned reinforcers.
Although the theoretical status of conditioned
VI 10 s Terminal Links VI 20 s reinforcement has been controversial (e.g., Baum,
1974b; Schuster, 1969; Shahan, 2010), a concurrent-
chains procedure provides a behavioral measure of
Food Food the relative attractiveness of the terminal-link out-
come schedules. In addition, a converging measure
of relative value is provided by the resistance to
change of responding during the terminal link:
Initial Links Nevin and Grace (2000; see also Grace & Nevin,
Figure 14.4. The concurrent-chains procedure. 1997) have shown that resistance to change and
During the initial links two keys are lighted white, preference in concurrent chains are correlated and
and responses are reinforced by concurrent VI 30 s argued that these measures are independent assays
schedules with access to one of two mutually exclusive
of a single construct that represents the history of
terminal-link schedules. Entry into a terminal link is
signaled by a change in color on the respective key (left reinforcement correlated with a particular stimulus.
key to red, right key to green) coupled with the other As such, results from experiments using concurrent
key being darkened. Responses during the terminal chains and related procedures are important for
links are reinforced with access to food according to a
VI 10 s (left key) or VI 20 s (right key) schedule. After understanding how the value of a stimulus is deter-
reinforcement in a terminal link, the initial links are mined by the conditions of reinforcement that it sig-
reinstated and the next cycle begins. G = green key; nals (for an alternative discussion, see Moore, 2008).
VI = variable interval; R = red key; W = white key. Early studies (Autor, 1969; Herrnstein, 1964b)
reported that initial-link response allocation matched
of the other key. After reinforcement is delivered in the relative rate of terminal-link reinforcement and
a terminal link, the initial link is reinstated. The concluded that the conditioned reinforcement
usual result is that subjects respond relatively more value of a terminal-link stimulus was determined
to the initial link leading to the terminal link, which by the rate of reinforcement in its presence, which
is associated with the shorter delay to reinforce- suggested that the matching law might apply to con-
ment. For example, in Figure 14.4, if the terminal ditioned as well as primary reinforcement, because
links associated with the left and right keys were VI concurrent chains may be viewed as a concurrent
10-second and VI 20-second schedules, respectively, schedule of conditioned reinforcement.
then the pigeons would respond relatively more to
the left initial-link key. Delay Reduction Theory
The concurrent-chains procedure has tradition- Fantino (1969) arranged a concurrent-chains proce-
ally been viewed as a way of studying conditioned dure with VI 30-second–VI 90-second terminal links
reinforcement. A conditioned reinforcer is an initially and varied the duration of the equal initial-link
neutral stimulus that has acquired the ability to act as schedules from VI 40 seconds to VI 600 seconds. He
a reinforcer through a history of pairing with a pri- found that response allocation for the VI 30-second
mary reinforcer such as food (for review, see Wil- terminal link became less extreme as the initial-link
liams, 1994). Conditioned reinforcement (sometimes duration was increased. For example, when the ini-
called secondary reinforcement) has historically been tial links were VI 40 seconds–VI 40 seconds, the
an important topic for learning theory because much average preference for the initial link leading to VI
315
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
30 seconds was 95%, but with VI 600-second–VI held constant. For example, in one condition the ter-
600-second initial links, it was 60%. minal links were fixed interval (FI) 5 seconds–FI
Fantino’s (1969) results were contrary to predictions 10 seconds, and in another condition the terminal
based on the matching law (Herrnstein, 1964b) and links were FI 40 seconds–FI 80 seconds. MacEwen
the assumption that conditioned reinforcement value found that the preference for the richer terminal link
was determined by reinforcement rate, which pre- increased as the overall duration increased, a result
dicted that preference should not have changed as known as the terminal-link effect. This result is impor-
initial-link duration increased. Fantino proposed an tant because it violates expectations based on Weber’s
alternative model for conditioned reinforcement to law (Gibbon, 1977). If choice between delayed rein-
explain his results. He suggested that the strength of a forcers is similar to temporal discrimination, one
terminal-link stimulus as a conditioned reinforcer was would expect that a constant ratio of terminal-link
determined by the reduction in delay to reinforce- delays would produce a constant preference. Yet the
ment that it signaled. Initial-link response allocation terminal-link effect is a robust phenomenon, having
is then predicted by the relative delay reduction: been reported for both FI and VI terminal links
(Grace & Bragason, 2004; MacEwen, 1972; Williams
BL T − D L
= . (6) & Fantino, 1978) and terminal links that differ in
BR T − DR
reinforcer magnitude (Navarick & Fantino, 1976)
In Equation 6, B is the initial-link response rate and and probability (Spetch & Dunn, 1987).
D is the average reinforcement delay from terminal- Although DRT has been an influential theory of
link onset, with the subscripts L and R representing conditioned reinforcement and correctly predicts
the left and right alternative schedules, respectively. T the initial- and terminal-link effects, it has several
is the overall average delay to reinforcement computed limitations. Because delay reduction depends on the
from the onset of the initial links. The decrease in pref- average terminal-link delay to reinforcement, it does
erence as initial-link duration increases is known as not predict any effect of differences in the delay
the initial-link effect. Equation 6 predicts the initial- distributions. However, one of the most well-
link effect because the conditioned reinforcement established results in the concurrent-chains litera-
value of the terminal links depends on the temporal ture is preference for variability: Nonhuman
context of reinforcement; specifically, a terminal link subjects prefer a VI (or variable delay) over a FI (or
that signals a given delay to reinforcement becomes a fixed delay) terminal link, with the average delays
more effective conditioned reinforcer when the initial- equated (Herrnstein, 1964a; Killeen, 1968; Moore,
link duration is increased and reinforcers occur less 1984). Also, if the average delays are the same, pref-
frequently overall. erence for the richer alternative is greater when the
Fantino’s (1969) model, known as delay reduc- terminal links are both FI than when they are both
tion theory (DRT), has been the most influential VI (e.g., Grace, 2002). Finally, because it lacks free
account of conditioned reinforcement over the past parameters, DRT is unable to account for variability
several decades (for reviews, see Fantino, 1977, in individual-subject data and thus could not pro-
2001; Fantino, Preston, & Dunn, 1993; Fantino & vide the kind of quantitative integration for concur-
Romanowich, 2007). DRT has been applied to a rent chains that the generalized matching law
wide range of situations, including foraging (Fan- accomplished for concurrent schedules. For these
tino & Abarca, 1985), delayed matching to sample reasons, researchers have continued to pursue alter-
(Wixted, 1989), and self-control (Navarick & native models for choice in concurrent chains.
Fantino, 1976), underscoring its generality.
DRT also predicts an effect of overall terminal-link Concurrent Chains and Generalized
duration on preference in concurrent chains that was Matching: The Contextual Choice Model
later reported by MacEwen (1972). In that study, the Davison (1987) suggested that the generalized match-
initial-link schedules were constant as the duration of ing law (Equation 4) could serve as a basis for model-
terminal-link schedules was increased, with the ratio ing concurrent-chains performance because concurrent
316
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
schedules may be viewed as a concurrent chain with Although Equation 7 provides an important step
0-second terminal links. The implication is that a in terms of extending the scope of the matching law
model that reduced to the generalized matching law in to accommodate results from concurrent chains, it is
the limit as terminal-link duration approached zero limited because by calculating terminal-link value as
could potentially apply to both procedures. In the con- a function of the average delay to reinforcement, it
catenated generalized matching law (Equation 4) as fails to account for preference for variability. The
applied to concurrent chains, R refers to rate of condi- search for a method of computing the value of a dis-
tioned reinforcement (i.e., rate of terminal-link entry) tribution of reinforcer delays that would apply to
and D and M refer to the delays and magnitudes of fixed as well as variable schedules has been an
terminal-link reinforcement. However, Davison noted important goal for research (see Grace, 1996).
that the concatenated generalized matching law was
not a satisfactory model for concurrent chains because Preference for Variability:
it did not predict the initial- or terminal-link effects. The Hyperbolic Decay Model
Grace (1994) proposed a model for concurrent As noted earlier, one of the most well-known results
chains based on the generalized matching law that in the concurrent-chains literature is preference for
addressed the shortcomings noted by Davison variability, that is, response allocation that favors
(1987). In his contextual choice model (CCM), variable over fixed delays with the overall mean
sensitivity to terminal-link reinforcer variables delay equated. Preference for variability, which has
depends on the temporal context of reinforcement: also been obtained with humans (e.g., Lagorio &
Tt / Ti Hackenberg, 2010; Locey, Pietras, & Hackenberg,
 R L  r  1/ D L  d  M L  m 
a a a
BL 2009), indicates that the arithmetic mean is inade-
= b     . (7)
BR  R R   1/ DR   M R   quate for computing conditioned value. Beginning
 
with Killeen (1968), several studies attempted to
Equation 7 is similar to the generalized matching find a fixed-variable equivalence rule, that is, a
law for concurrent chains suggested by Davison method for computing the value of a distribution of
(1987) but includes an additional exponent, Tt/Ti, reinforcer delays that would apply to both fixed and
which is the ratio of the average terminal-link to variable schedules (Davison, 1969, 1972). However,
initial-link durations. Thus, the effective sensitivi- these studies yielded inconsistent results, casting
ties to terminal-link reinforcement variables are ad * doubt on the possibility of a general rule for fixed-
(Tt/Ti) and am * (Tt/Ti), which allows CCM to predict variable equivalence (Navarick & Fantino, 1974).
the initial-link effect (Tt/Ti decreases with increases Mazur (1984) noted that studies on preference for
in initial-link duration) and the terminal-link effect variability might have provided mixed data because
(Tt/Ti increases with increases in terminal-link dura- of complexities inherent in concurrent chains, partic-
tion). Equation 7 also reduces to the generalized ularly the extended duration of the choice phase (ini-
matching law in the limit as Tt approaches zero, so tial links), and suggested that more consistent results
it may be viewed as an extension of the generalized might be obtained if subjects were able to register
matching law that can accommodate both concurrent their preference with a single response. He intro-
chains and concurrent schedules. duced a new procedure for studying preference
Grace (1994) applied CCM to archival data from between fixed and variable delays. The adjusting-
concurrent-chains studies in which the terminal links delay procedure (see Figure 14.5) uses titration to
were both VI or both FI schedules (92 individual- determine an indifference point for a schedule of
subject data sets from 19 studies). Averaged across variable delays, that is, the fixed delay that is chosen
studies, the model accounted for more than 90% of the equally as often as a variable-delay schedule. In this
variance in relative initial-link responding. He con- procedure, after an intertrial interval two response
cluded that the generalized matching law and CCM keys are illuminated, for example, one red and the
made up a single framework for modeling choice in other green. A response to the red key initiates a
concurrent chains and concurrent schedules. delay to reinforcement sampled from a variable
317
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
Equation 8 is known as the hyperbolic decay

15s ITI
model, and it assumes that the value of a delayed
reinforcer is determined as a hyperbolic function of
its delay. It calculates the value (V) of a distribution
W
of delays to reinforcement, d1, dn, as the average
Start of Trial
of the delays after they have been rescaled by a
1 Peck hyperbolic function. K is a parameter that deter-
mines how rapidly values decreases with delay (see
Choice Trial
Figure 14.6).
R G
Mazur (1984) showed that the hyperbolic decay
1 Peck 1 Peck model provided an excellent description of his data,
Standard Delay
accounting for more than 90% of the average data
Adjusting Delay
Blue Houselight Orange Houselight with K = 1. Thus, his results suggest that the
adjusting-delay procedure is an effective method for
3s Food
studying the effects of delayed reinforcement and
that the function relating the value of a reward to
its delay may be described by a simple hyperbola
Next ITI (Figure 14.6).
Over the past 25 years, Mazur and colleagues
Figure 14.5. The adjusting-delay procedure.
After an intertrial interval (ITI) during which the have reported an impressive series of studies in
keys are dark, the center key is lighted white. A which the adjusting-delay procedure has been used
single response to the center key illuminates the to address specific issues relating to the general
side keys, red and green. A response to the red
key extinguishes all keys and illuminates a blue question of how the value of a stimulus is deter-
houselight, and reinforcement (3 s access to food) is mined by the conditions of reinforcement that it
provided after a delay selected from a variable dis- signals (for reviews, see Mazur, 1993, 1997, 1998).
tribution (standard delay). If the subject responds These issues include choice between fixed- and
to the green key, an orange houselight is illuminated
and reinforcement is provided after an adjusting variable-ratio schedules (Mazur, 1986b); probability
delay that is changed gradually over trials until the of reinforcement (Mazur, 1985, 1989, 1998; Mazur
subject is about equally likely to respond to the red & Romano, 1992); signaling effects in probabilistic
and green keys. This figure illustrates a free choice
reinforcement (Mazur, 1995, 1996); choice between
trial. Forced choice trials in which only one key (red
or green) is illuminated are also included to ensure alternatives differing in delay and magnitude of rein-
that the subject continues to experience both the forcement (Mazur, 1987); choice between single and
standard and adjusting delays. G = green key; ITI = multiple reinforcers (Mazur, 1986a, 2007a); transi-
intertrial interval; R = red key; W = white key.
tivity in choice between fixed and variable delays
(Mazur & Coe, 1987); choice between alternatives
schedule (standard), and a response to the green key
differing in probability and amount of reinforcement
initiates a fixed delay. Colored houselights are illumi-
(Mazur, 1988); effects of required forceful respond-
nated during the delays. The key aspect of the feature
ing (Mazur & Kralik, 1990); effects of reinforcers
is that the fixed delay is adjusted over trials to yield
during the intertrial interval (Mazur, 1994); procras-
an indifference point—the fixed delay that is chosen
tination in preference for larger, more delayed work
about equally as often as the variable delay.
requirements (Mazur, 1996, 1998); and species dif-
Mazur (1984) determined fixed-delay indiffer-
ferences between pigeons and rats in choice between
ence points for a range of variable-delay schedules
alternatives differing in delay, amount, and proba-
and found that they were well described by the fol-
bility of reinforcement (Mazur, 2005, 2007b; Mazur
lowing equation:
& Biondi, 2009).
1 n 1 Results of these studies have provided support
V= ∑
n i=1 1 + Kd i
. (8)
for Equation 8 as a model for temporal discounting,
318
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
0.8
0.6
V
0.4
0.2
0
0 20 40 60 80 100
Delay (s)
Figure 14.6. Gradient of value (V) predicted by the hyperbolic decay model
(Equation 14.8, with K = 0.2) as a function of delay.
that is, how the value of a reward changes as a func- hyperbolic discounting (Equation 8). Because the
tion of its delay. More important, the hyperbolic terminal-link stimuli are associated with shorter
decay model has been shown to accurately describe delays to reinforcement, they have greater value
choices not only of pigeons and rats but also of than the initial links. The model predicts that initial-
humans (Lagorio & Hackenberg, 2010; Rodriguez link response allocation depends on the relative
& Logue, 1988). The hyperbolic decay model has value added by the terminal-link stimuli. Formally,
been very influential in the study of intertemporal a
choice, that is, decision making by humans when BL  R L  r  VL − a d Vi 
=b   , (9)
one or more of the outcomes is delayed (for review, BR  R R   VR − a d Vi 
see Green & Myerson, 2004). The distinguishing
feature of the hyperbola as a model for intertempo- where VL and VR are the values of the left and right
ral choice is that it predicts that the rate of temporal terminal-link stimuli, Vi is the value of the initial
discounting decreases with delay; that is, delayed links, and ad is a sensitivity parameter. Note that
rewards are discounted more steeply over delays in HVA retains the same generalized-matching kernel
the short term than delays in the long term. In this as the CCM; the difference is the right-hand expres-
respect, the hyperbola contrasts with exponential sion in brackets, which gives the relative value
discounting, the normative model according to eco- added by the terminal-link stimuli.
nomic theory, which predicts that the discounting Mazur (2001) compared the ability of the HVA
rate is constant. Studies have consistently found that and CCM to describe archival data from concurrent-
the hyperbolic model performs better than the expo- chains studies. He fitted both models to the same
nential model for describing intertemporal choice by data sets used by Grace (1994) and found that both
humans (e.g., Myerson & Green, 1995). models accounted for a similar percentage of the
variance in initial-link response allocation (CCM =
Hyperbolic Value-Added Model 90.8%; HVA = 89.6%). For the sake of comparison,
Mazur (2001) proposed the hyperbolic value-added Mazur also examined predictions of a version of
model (HVA) as an alternative matching-based DRT with added sensitivity parameters and found
model for choice. Mazur’s model is based on the that it also provided a reasonably good description
hyperbolic function as a model for discounting and of the data, although somewhat less than the other
assumes that the values of both initial- and terminal- models, with an average percentage of the variance
link stimuli in concurrent chains are determined via of 83.0%. He concluded that both HVA and CCM
319
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
described the data well and could not be distin- than that for 40-second versus 50-second delay, con-
guished on the basis of overall goodness of fit. trary to Savastano and Fantino (1996). Although the
reason for the discrepancy across studies is unclear,
Testing Models for Choice Mazur suggested that procedural differences—such
Identifying the optimal model for choice in concur- as response-dependent versus response-independent
rent chains is important because it could provide terminal links—may have contributed to the differ-
a definitive answer to the question of how condi- ence in results. Resolution of this issue awaits future
tioned value is determined by delay and other rein- research, although we should note the methodologi-
forcer variables. Thus, Mazur (2001) suggested that cal difficulty of distinguishing between a null predic-
the models be evaluated by exploring situations in tion (i.e., the constant-difference effect) and a small
which they made different predictions. Such situa- decrease in preference.
tions that have been studied to date include the
constant-difference effect, three-alternative choice, Three-alternative choice. Mazur (2000) reported
and added initial-link reinforcers. two experiments that tested predictions of DRT,
HVA, and CCM for choice involving two versus
Constant-difference effect. Savastano and Fantino three alternatives. The basic idea was to test whether
(1996) noted a counterintuitive prediction of DRT: preference between a constant pair of terminal links
If the difference between a pair of VI schedules would change depending on the addition of a third
was kept constant while their absolute duration alternative. In one comparison, the initial links were
increased, preference for the shorter VI schedule independent VI 60-second schedules, and the ter-
should not change. For example, in one condition minal links were FT 3 seconds and FT 12 seconds
(short), the terminal links were VI 5 seconds and VI in one condition; in another condition, a third alter-
25 seconds, and in a second condition (long), they native was added in which the initial link was also
were VI 100 seconds and VI 120 seconds. This pre- VI 60 seconds, and food was delivered immediately
diction is counterintuitive because the ratio between on terminal-link entry (i.e., FT 0 second). Mazur
the terminal-link schedules decreases with increases showed that CCM predicted no change in the pref-
in overall duration. However, DRT predicts that preference between FT 3 seconds and FT 12 seconds
erence should not change because when a constant when the third alternative was added because the
duration is added to each schedule, the average time ratio of times spent in the terminal and initial links
to reinforcement from the onset of the initial links did not change (for two alternatives, Tt/Ti = 7.5
(T) increases by the same amount, and so the delay seconds/30 seconds; for three alternatives, Tt/Ti = 5
reductions for the terminal links do not change. seconds/20 seconds). In contrast, HVA predicts that
Savastano and Fantino (1996) noted that CCM also the proportion of responses for FT 3 seconds should
predicted the constant-difference effect, because increase in the three-alternative condition because
the decrease in preference as the terminal-link ratio the value of the initial links should increase when
was decreased was offset by the effect of increasing the third schedule is added, thus increasing the ratio
overall terminal-link duration. They reported two in the right-hand side of Equation 9. DRT makes a
experiments that confirmed the constant-difference similar prediction because the average time to rein-
effect predicted by DRT. forcement from the initial-link onset decreases in
Mazur (2002b) acknowledged the constant- the three-alternative condition. Results of Mazur’s
difference effect was problematic for HVA and experiment showed that preference for the FT 3
reported two experiments with response-independent seconds versus the FT 12 seconds increased when
schedules (fixed time [FT] and variable time [VT]) the third alternative was added, consistent with HVA
that tested it. (FT and VT schedules deliver reinforcers and DRT but contrary to CCM. These findings are
after fixed and variable time periods, respectively, similar to results from previous studies that have
without regard to a response.) He found that preference examined the effect of adding a third alternative
for 2-second versus 12-second delay was stronger (Fantino & Dunn, 1983). The findings suggest that
320
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
the average time to reinforcement from the onset of an additional condition in which a differential
the initial links, not the ratio of terminal-link and reinforcement of other (DRO) contingency was
initial-link duration, is a critical factor in terms of added to the VT schedule during the initial links so
understanding how temporal context affects choice that free food deliveries could not occur within 2
in concurrent chains, at least for the relatively seconds of an initial-link response. Results were
understudied (unfortunately) situation of three- consistent with the adventitious reinforcement
alternative choice. hypothesis: The increase in preference for the richer
terminal link that Mazur found was replicated when
Added initial-link reinforcers. Mazur (2003) VT food was added during the initial link, but when
noted that HVA and CCM made different predic- the DRO contingency was also included, preference
tions for a situation in which reinforcers were added did not change systematically. Analyses of absolute
independently of responding during the initial links. initial-link response rates were also consistent
Because the added reinforcers do not change the with adventitious reinforcement: Response rates
duration of the initial links, CCM predicts that pref- decreased proportionally for both alternatives when
erence for a constant pair of terminal-link schedules the DRO contingency accompanied the free food
should not change. However, because the added deliveries but decreased less for the richer schedule
reinforcers increase the value of the initial-link in the VT-only condition.
stimuli (and decrease the overall time to reinforce- Scoones et al.’s (2009) results showed that when
ment, T), both HVA and DRT predict that prefer- adventitious reinforcement was minimized, provi-
ence should increase when the extra reinforcers are sion of free food during the initial links did not
provided. change preference. These results pose a challenge
Mazur (2003) tested these predictions in an for both HVA and DRT because the rates of VT rein-
experiment in which the initial link was a single VI forcement were similar in their VT-only and VT +
30-second schedule that ensured the terminal-link DRO conditions. Thus, the value of the initial-link
schedules, FT 3 seconds and FT 12 seconds, were stimuli (or average time to reinforcement from
entered equally as often. Test sessions were run in initial-link onset) should have changed to the same
some conditions in which free food deliveries were extent in both conditions. However, the lack of a
provided during the initial links according to a VT change in preference in the VT + DRO condition is
30-second schedule. He found that preference for consistent with predictions of CCM.
the FT 3-second alternative increased when free
reinforcers were added, consistent with HVA and Reinforcement context and conditioned value. As
DRT but not CCM. noted previously, the core principle of DRT is that
Scoones, Hucks, McLean, and Grace (2009) sug- stimulus value varies inversely with reinforcement
gested an alternative explanation for Mazur’s (2003) context—a terminal link that signals a given delay
results, namely that the effect of free food deliveries will have greater value in a context in which rein-
on responding to the preferred key might have been forcers are infrequent than in a context in which
mitigated by adventitious reinforcement. Free food they are frequent (Fantino, 2001). This principle
would be expected to reduce responding to both accords with intuition and is consistent with well-
initial links because it weakens the contingency known phenomena such as behavioral contrast
between responding and reinforcement (Rachlin & (Williams, 2002) and evidence that the effective-
Baum, 1972). However, because pigeons were ness of operant and Pavlovian contingencies varies
already responding more in the initial link leading inversely with reinforcement context (Herrnstein,
to the FT 3-second alternative, when free food was 1970; Rescorla, 1968).
delivered it would be more likely to be temporally However, Grace (1994) noted that the terminal-
contiguous, by accident, with a response to that link effect—that preference between a pair of
alternative. Scoones et al. tested this explanation terminal links in constant ratio increases as overall
by replicating Mazur’s experiment but including duration increases—violated Weber’s law, which is
321
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
a typical result in temporal discrimination (e.g., depend on reinforcement context, other studies
Gibbon, 1977; Grondin, 2001). He suggested that have obtained conflicting results. O’Daly, Meyer,
stimulus value—defined as what subjects learn about and Fantino (2005) trained pigeons on a multiple-
the terminal-link stimulus–reinforcer relation—was chains schedule in which the terminal link was
independent of context, but that the behavioral always an FT 30-second schedule and the initial link
expression of difference in value as preference was either VI 10 seconds or VI 100 seconds in differ-
depended on the relative duration of the terminal ent components. O’Daly et al. found that when the
and initial links, which is similar to the distinction terminal-link stimuli were later compared in probe
between learning and performance that is often tests, pigeons responded more to the terminal-link
made in theories of associative learning (e.g., Stout stimulus associated with the VI 100-second initial
& Miller, 2007). link in baseline, consistent with predictions of DRT.
Several studies have attempted to test whether Similar results were obtained in a multiple-chains
stimulus value in concurrent chains depends on procedure by O’Daly, Angulo, Gipson, and Fantino
reinforcement context (Grace & Savastano, 1997, (2006), who found that pigeons preferred an FR
2000). In general, these studies have used transfer 30-second terminal-link stimulus that was preceded
designs in which after exposure to terminal-link by a VI 100-second initial link over an FR 30-second
contingencies in baseline, subjects are given a probe stimulus that was preceded by a VI 10-second initial
test in which terminal-link stimuli that differed link. In addition, O’Daly et al. found that pigeons
in baseline reinforcement context are compared. preferred a nonreinforced stimulus that accompa-
Choice in the probes is assumed to give an indepen- nied the FR 30 second and was preceded by the VI
dent measure of stimulus value. For example, Grace 100-second initial link, thereby demonstrating value
and Savastano (2000) trained pigeons on a two- transfer (Clement, Feltus, Kaiser, & Zentall, 2000).
component procedure in which in one component However, O’Daly et al. found that results from sev-
(short), the initial link was a single VI 15-second eral experiments using the successive-encounters
schedule that ensured equal access to the terminal procedure, an operant analogue foraging task, were
links, which were VI 10 seconds and VI 20 seconds. inconsistent, and only stimuli associated with rich
The schedules in the other component (long) were handling schedules (i.e., terminal links) showed
obtained by multiplying these by two (i.e., VI effects of reinforcement context.
30-second initial link, VI 20-second and VI Overall, studies that have used transfer tests to
40-second terminal links). After baseline training, assess the effect of reinforcement context on stimu-
occasional probe tests were conducted in which lus value have yielded mixed results. Apparently,
terminal-link stimuli from the different components whether the terminal-link stimuli appear as part of a
were presented together. Grace and Savastano concurrent or single chain in baseline has some
(1997, 2000) showed that according to DRT, impact on whether training in a rich versus lean rein-
pigeons should prefer the VI 20 second from the forcement context affects probe choice. The issue is
short component over the VI 20 second from the not experience with concurrent responding per se,
long component and be indifferent to the VI 10 sec- because pigeons in O’Daly et al.’s (2005, 2006) stud-
ond and VI 40 second. However, if terminal-link ies that were experimentally naive at the start of the
value was independent of baseline context, then research were given pretraining in a choice situation
the pigeons should be indifferent to the two VI before the final baseline procedure. A resolution to
20-second stimuli and prefer VI 10 second over VI this discrepancy awaits future research.
40 second. Results showed that probe preference
was well predicted by the baseline schedule values Summary: Testing models for choice. Overall,
independent of context, consistent with CCM but results of experiments that have compared predic-
not with DRT. tions of competing models for concurrent chains
Although Grace and Savastano’s (1997, 2000) have yielded mixed results. Some have favored pre-
data suggested that terminal-link value does not dictions of HVA and DRT over CCM (e.g., Mazur,
322
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
2000), and others have supported CCM (e.g., the reinforcement probability for the more reliable
Scoones et al., 2009). Evidence from studies that alternative is reduced to 75%, pigeons show a statis-
have used transfer methodologies to assess the influ- tically reliable preference for the 50% alternative
ence of reinforcement context on stimulus value provided that the outcomes are differentially sig-
have also been inconclusive, with some suggesting naled (see also Stagner & Zentall, 2010).
that reinforcement value is independent of context The stronger preference for the 50% alternative
(Grace & Savastano, 2000) and others suggesting when outcomes are signaled has been attributed to
that value depends on context (O’Daly et al., 2005, the conditioned reinforcement value of the stimulus
2006). Although both HVA and CCM provide a correlated with reinforcement. The idea is that this
strong quantitative account of results from stimulus signals a relatively greater local reduction
concurrent-chains studies and the effects of rein- in delay to reinforcement than the 100% stimulus
forcement context predicted by DRT have sub- (Dunn & Spetch, 1990) or that its value is enhanced
stantial generality, at this point it is premature to because it is paired with a signal for extinction
conclude that any of the extant steady-state models (Gipson et al., 2009). Evidence in favor of condi-
for choice based on the matching law are definitive. tioned reinforcement was reported by McDevitt,
Spetch, and Dunn (1997). They found that interpo-
Reinforcement Probability lation of a 5-second gap between the stimulus corre-
Another variable that researchers have studied is the lated with reinforcement on the 50% terminal link
probability of reinforcement for a terminal link. In decreased preference for that alternative. However,
these studies, terminal links end in either reinforce- when a similar gap was imposed on the 100% alter-
ment or extinction (i.e., blackout), with a specified native, there was no systematic effect on preference.
probability. Kendall (1974) reported that pigeons These results are consistent with Dunn and Spetch’s
responded more in the initial link for a terminal link (1990) explanation of the signaling effect in terms
that ended in reinforcement 50% of the time than in of delay reduction. However, Mazur (2005, 2007b)
one that provided reinforcement 100% of the time found that signaling reinforcement and extinction
when different stimuli were correlated with the rein- outcomes did not influence rats’ indifference points
forcement and extinction outcomes on the 50% alter- for a probabilistic schedule in the adjusting-delay
native. However, when differential stimuli were not procedure. The implication is that the signaling
correlated with outcomes on the probabilistic terminal effect may be limited in terms of generality across
link, pigeons strongly preferred the 100% schedule. species.
Kendall’s (1974) results were provocative Most of the research on reinforcer probability
because they revealed a suboptimal preference for and choice has concentrated on the effects of signal-
the alternative providing the lower overall rate of ing, or whether preference for risky outcomes
reinforcement. Subsequent studies that attempted to depends on deprivation or energy budget (Caraco,
replicate Kendall’s findings under different condi- Martindale, & Whitam, 1980; Pietras, Locey, &
tions showed that although an effect of signaling Hackenberg, 2003), and relatively few studies have
was reliable—that is, pigeons showed a stronger manipulated probability parametrically with other
preference for the 100% alternative when differential aspects of reinforcement (e.g., immediacy, magni-
stimuli were not correlated with the outcomes on tude) held constant. Recently, Mattson, Hucks,
the probabilistic terminal link—pigeons did not Grace, and McLean (2010) studied four different
show a consistent preference for the 50% alternative probability ratios (83%–17%, 17%–83%, 67%–33%,
under well-controlled conditions (e.g., Dunn & 33%–67%) and three immediacy ratios (FT 10 seconds–
Spetch, 1990; Fantino, Dunn, & Meck, 1979; FT 20 seconds, FI 20 seconds–FT 10 seconds, FT 15
Mazur, 1996; Spetch, Belke, Barnet, Dunn, & seconds–FT 15 seconds) in a factorial design, under
Pierce, 1990; Spetch, Mondloch, Belke, & Dunn, conditions in which reinforcement and extinction
1994). However, recently Gipson, Alessandri, outcomes were either signaled or unsignaled. They
Miller, and Zentall (2009) have shown that when found that the concatenated generalized matching
323
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
law including a term for the probability ratio pro- response allocation adapts when the reinforcement
vided an excellent account of the data. Sensitivity to contingencies are changed. The promise of under-
reinforcer probability was greater in unsignaled than standing acquisition is that it could give key insights
in signaled conditions, as expected on the basis of into the processes underlying steady-state choice
previous studies. However, sensitivity to immediacy and eventually provide an explanatory theory that
was also greater in unsignaled conditions, suggest- could predict matching and other results in molar
ing that the effect of signaling reinforcement and behavior allocation. We consider first research that
extinction outcomes applies not only to reinforcer has examined acquisition in simple concurrent
probability but to the effects of terminal-link rein- schedules and then studies of acquisition in concur-
forcer variables in general. rent chains.
Summary Acquisition in Concurrent Schedules

Studies that have examined the effects of different Mazur and colleagues reported a series of experi-
reinforcer variables—such as delay, rate, magnitude, ments in the 1990s that established some basic
and probability of reinforcement—on choice have acquisition phenomena in concurrent schedules.
yielded a rich corpus of data that has supported the These studies used designs in which for each condi-
development of a variety of quantitative models that tion the reinforcement probabilities or rates for a
describe choice at the molar level with a high degree concurrent-choice situation with two alternatives
of accuracy. Although these models differ in terms of were equal for several sessions and then unequal for
specific details, such differences should not obscure several sessions. Bailey and Mazur (1990; see also
their common roots in the concatenated matching Mazur & Ratti, 1991) tested whether acquisition of
law. Overall, considerable support exists for the fun- preference between two alternatives reinforced with
damental assumptions of matching: that response a specified probability (a concurrent random ratio–
allocation in a concurrent-choice situation depends random ratio schedule) depended on the ratio or
on the relative reinforcement value of the choice difference of the reinforcement probabilities. They
alternatives, with value determined as an additive found that acquisition was faster with greater ratios,
function of different reinforcement variables (Berg for example, when reinforcement probabilities were
& Grace, 2004, but cf. Elliffe et al., 2008). This sup- .12 and .02 than when they were .40 and .30. Mazur
port testifies to the robustness and adaptability of (1992) reported that with their ratio held constant,
the concatenated matching law as a framework for acquisition was faster with overall higher probabili-
understanding the allocation of operant behavior. ties and described a linear-operator model that
could account for his results. According to this
model, the strength of a response increased with
Acquisition of Choice Behavior
reinforcement and decreased with nonreinforce-
Because the matching law describes a molar rela- ment. Despite the simplicity of this model, Mazur
tionship between response allocation and relative (1992) showed that it predicted his results more
reinforcement—that is, responding and reinforce- accurately than several previous models for acquisi-
ment measured over an extended temporal epoch tion in concurrent schedules, including the kinetic
(Baum, 2003)—most studies on behavioral choice model (Myerson & Miezin, 1980), melioration
since Herrnstein (1961) have used steady-state (Herrnstein & Vaughan, 1980), and the ratio-
designs in which schedules are maintained over invariance model (Horner & Staddon, 1987; Staddon
many sessions until responding has stabilized. The & Horner, 1989). Later studies provided evidence of
success of the generalized matching law and other spontaneous recovery in choice behavior (Mazur,
models for choice provides good evidence of the 1995, 1996) and showed that for concurrent VI–VI
utility of this approach. However, in the past two schedules, acquisition was faster when the overall
decades, researchers have paid increasing attention reinforcement rate was greater and when schedules
to the acquisition of behavioral choice—how were changed more frequently (Mazur, 1997).
324
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
A second major line of investigation has been a interval is increased (Davison & Baum, 2002).
series of studies by Davison, Baum, and colleagues Landon, Davison, and Elliffe (2003) found that
using a within-session procedure to illuminate the unequal within-session reinforcer distributions gave
dynamics of choice under concurrent schedules. In similar results as equal reinforcer distributions but
their procedure (Davison & Baum, 2000), pigeons were biased toward an average sessionwide prefer-
are exposed to seven different pairs of concurrent ence. Krageloh and Davison (2003) showed that
VI–VI schedules per session in which the left:right sensitivity is greater when the reinforcer ratios are
reinforcer ratios are 1:27, 1:9, 1:3, 1:1, 3:1, 9:1, and signaled and when a changeover delay is used. There
27:1 and the overall reinforcer rate is kept constant. is also evidence for “fix and sample” in the local pat-
After an intercomponent interval during which the tern of visits to each alternative (Baum & Davison,
chamber is dark, the side keys are lighted to signal 2004) and for peak shift with signaled component
the start of a component, which ends after a fixed reinforcer ratios in preference prior to the first rein-
number of reinforcers have been earned. Sessions forcer in a component (Krageloh, Elliffe & Davison,
consist of seven components, with the reinforcer 2006). Increasing reinforcer delay has a similar
ratio for each determined pseudorandomly such that effect on preference pulses as decreasing reinforcer
all seven ratios are used. Pigeons typically receive 80 magnitude (Davison & Baum, 2007). Some of these
to 100 sessions of training in each condition, with results are similar to those of previous studies—for
the data from all but the first 15 sessions aggregated example, Buckner, Green, and Myerson (1993)
for analysis. showed that reinforcer delivery increased the length
The Davison–Baum procedure has yielded a rich of a visit to the alternative, similar to a preference
and varied spectrum of results, including “every pulse—but the Davison–Baum procedure is unique
reinforcer counts”—that is, preference shifts toward in terms of the detailed analyses it supports of
the alternative that produced the last reinforcement response allocation at multiple time scales.
(Davison & Baum, 2000), specifically, an increase in Baum and Davison (2009) proposed a linear-
preference for the just-reinforced alternative that operator model to account for results from their
decreased toward indifference after 20 seconds to procedure. In their model, the change in the log
25 seconds (preference pulse; Davison & Baum, response ratio after reinforcement is determined by
2002). Similar preference pulses were observed in a the difference between the current log response ratio
traditional steady-state design by Landon, Davison, and an asymptote, multiplied by a parameter that
and Elliffe (2002). Within components in the indicates how much weight is given to the current
Davison–Baum procedure, sensitivity to the rein- reinforcement compared with previous reinforce-
forcer ratio increases as more reinforcers are ments. Baum and Davison (2009) fitted their model
obtained and increases more rapidly when the over- to log response ratios associated with all four-
all reinforcement rate is relatively high (Davison & reinforcer sequences from several conditions in
Baum, 2000). In addition, successive reinforcers Baum and Davison’s (2004) study and then used the
from the same alternative have a smaller effect than parameter estimates to predict log response ratios
the one before (confirmations), whereas a reinforcer for sequences of one to three and five to seven rein-
that follows a series of reinforcers from the opposite forcers. They showed that the model made accurate
alternative has a relatively large effect (disconfirma- predictions and that presence or absence of a
tions; Davison & Baum, 2000). Other studies have changeover delay was reflected by the asymptote
shown that within-session variation in reinforcer parameter and change in overall reinforcement rate
magnitude ratios yields similar effects as variation in by the weighting parameter. Baum and Davison
reinforcer rate ratios (Davison & Baum, 2003), that (2009) acknowledged that their model was unusual
sensitivity is greater when the range of reinforcer in that it predicted log response allocation directly,
ratios within the session is greater (Landon & Davi- rather than response strengths or values for individ-
son, 2001), and that carryover effects from one com- ual alternatives that were then used to predict
ponent to the next decrease as the intercomponent response allocation, as is typical for linear-operator
325
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
models for choice (Mazur, 1992). Baum and Davi- view, stimuli that predict phylogenetically important
son (2009) remarked that they had thoroughly events, that is, fitness-enhancing or fitness-reducing
explored such approaches but found that they did events such as food or shock, will “guide behavior
not perform as well as one that predicted response into activities that produce fitness-enhancing events
allocation directly. and into activities that prevent the fitness-reducing
Davison, Baum, and colleagues have used similar events” (Davison & Baum, 2006, p. 281).
procedures to demonstrate that stimuli produced by At this point, it is unclear whether Davison
responding, both primary reinforcers (e.g., food) and Baum’s (2006) proposal will lead to a more
and conditioned reinforcers (e.g., presentation of a comprehensive and satisfactory understanding of
magazine light that accompanies food presenta- reinforcement processes. As they acknowledged, the
tions), can have a discriminative function that may observation that biologically relevant stimuli such as
act independently of any response-strengthening food can have discriminative as well as reinforcing
function that the stimulus might have. For example, properties is not new (e.g., Dufort, Gutman, &
Krägeloh, Davison, and Elliffe (2005) used a proce- Kimble, 1954). Moreover, research that has con-
dure in which the arranged reinforcer ratio in each trasted predictions of matching and reinforcement
session was always 1:1 but the probability of the maximization has found that pigeons’ and rats’
next reinforcer being obtained from the same or the response allocation will deviate from matching if
other alternative was varied across conditions. As a that results in an increase in overall reinforcement
result, the length of sequences of same-key reinforc- rate, provided that the subject is able to discriminate
ers was varied, and the relative reinforcement rate the increase (e.g., Heyman & Tanz, 1995; Williams,
was constant. They found that reinforcers increased 1992). These results led Williams (1992) to remark
the likelihood of responding to the same key, but that, rather than being contradictory accounts of
to a lesser extent when the length of reinforcer choice, matching and maximizing were competitive
sequences was short. The implication is that rein- processes: Organisms maximize when they can, but
forcers also had a discriminative function—that match when they cannot.
is, they signaled the location of the upcoming It is also important to note that even if Davison
reinforcer—in addition to strengthening responding and Baum’s (2006) conception of reinforcement as
on the key that produced the reinforcer. discrimination is correct, it does not challenge the
Davison and Baum (2006) studied the effects of conclusions of the research, previously discussed,
conditioned reinforcers—presentations of the food on the matching law and conditioned value. The
magazine light (which accompanied primary rein- contribution of matching-based models for choice
forcers) by itself and a 2.5-second green keylight. has been to provide a quantitative measure of the
They manipulated the correlation between the maga- relative value of the reinforcing outcomes signaled
zine light and green keylight presentations and the by the terminal-link stimuli. Assuming that there are
location of the upcoming primary reinforcer and no sequential dependencies between the location of
found that preference pulses after the conditioned the terminal link associated with reinforcement on
reinforcers depended on the correlation with food: one cycle and the next, response allocation in the
When the correlation was 1.0, positive preference initial links should provide a measure of relative
pulses resulted, but when the correlation was −1.0, terminal-link value that is not confounded by effects
negative preference pulses were obtained. Davison of reinforcers as discriminative stimuli.
and Baum suggested that conditioned reinforcers did
not serve a strengthening function; rather, they sig- Acquisition in Concurrent Chains
naled the location of upcoming food. On the basis of Research in the past decade has also examined how
these results, Davison and Baum argued that a more initial-link response allocation in concurrent chains
general, evolutionary conception of reinforcement in adapts when the reinforcement contingencies are
terms of a discriminative rather than strengthening altered. Mazur, Blake, and McManus (2001) com-
function may be appropriate. According to their pared rate of change in pigeons’ preference in
326
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
concurrent chains when either the initial- or the reached asymptote within individual sessions. Sub-
terminal-link schedules were changed. In some consequent research by Grace et al. has attempted to
ditions, both terminal-link delays were 12.5 seconds model the process by which pigeons’ response allo-
and the initial link that produced the greater pro- cation adapts in the rapid-acquisition procedure,
portion of terminal-link entries (80% vs. 20%) was with the goal of developing a model for acquisition
altered. In other conditions, both initial links that could eventually account for steady-state per-
arranged 50% of the entries, and the terminal-link formance in concurrent chains as well.
delays were altered (5 seconds vs. 20 seconds or 2 Grace et al. (2003) reported a second experiment
seconds vs. 18 seconds). Mazur et al. found that in which the same pigeons as in Experiment 1
preference adapted more rapidly when the propor- served, with one difference: Instead of two values, a
tion of entries arranged by the initial links was different FI schedule was used for the right terminal
changed rather than when the terminal-link delays link in each session by sampling from an infinite
were changed. Mazur (2002a) studied whether population of values. These values were arranged
acquisition of initial-link preference depended on so that the location of the richer alternative still
overall terminal-link duration. In his experiment, changed across sessions according to the PRBS.
terminal-link delays were equal (1 second or 20 sec- Surprisingly, Grace et al. found that there was no
onds), and the proportion of terminal-link entries disruption in performance: Sensitivity to immediacy
arranged by the initial links was changed. Overall, did not decrease compared with levels achieved at
he found no effect of terminal-link duration on the end of Experiment 1, and it did not change over
acquisition. Analyses of the effects of individual 60 sessions of training in Experiment 2. Moreover,
reinforcers showed evidence for preference pulses generalized-matching scatterplots showed that for
similar to Davison and Baum (2002): After a some pigeons, response allocation was nonlinear
response on one key had been reinforced, initial- and fell into two clusters (see Figure 14.7). Grace
link choice for that alternative was higher for et al. suggested that a process akin to categorical
approximately the next 100 responses. discrimination—in which response allocation
An alternative approach for studying acquisi- favored whichever terminal link had the shorter
tion in concurrent chains was pursued by Grace, delay in an all-or-none fashion—determined choice.
Bragason, and McLean (2003). In their Experiment Grace and McLean (2006) compared perfor-
1, the terminal link for the left alternative was mance in a condition identical to Grace et al.’s
always FI 8 second and the right terminal link was (2003) Experiment 1 with one in which a different
either FI 4 seconds or FI 16 seconds across sessions, pair of FI schedule values was used in each session.
as determined by a 31-step pseudorandom binary They reasoned that if choice was determined by the
series (Hunter & Davison, 1985; Schofield & Davi- conditioned reinforcement value of the terminal-
son, 1997). Each session consisted of 72 initial- and link stimuli, then acquisition should be faster in the
terminal-link cycles. Grace et al. showed that after former condition (minimal variation, because only
sufficient training (approximately two pseudoran- one terminal link changed between two values) than
dom binary series sequences), pigeons’ response the latter condition (maximal variation). However,
allocation was determined by the ratio of terminal- there was no systematic difference in sensitivity to
link immediacies in the current session and with no immediacy; response allocation adapted similarly in
significant control by the immediacy ratios from prior the minimal- and maximal-variation conditions.
sessions. They also found that sensitivity to relative Results from the latter condition showed individual
immediacy increased rapidly and reached an asymp- differences in response allocation; for some pigeons,
tote about midway through individual sessions. response allocation was approximately linearly
Grace et al. (2003) termed the control over related to the log immediacy ratio, whereas for oth-
choice by the current-session immediacy ratio that ers the relationship was nonlinear. Grace and
developed under the pseudorandom binary series McLean proposed a model to account for their
design rapid acquisition, because initial-link choice results. Their model assumed that after reinforcement
327
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
Beginning of Session End of Session
1.5 1.5
Log Response Ratio
Log Response Ratio

1 1
0.5 0.5
0 0
-0.5 -0.5
-1 -1
-1.5 -1.5
-2 -2
1 10 100 1 10 100
Delay Delay
Figure 14.7. Log initial-link response ratio as a function of the fixed-

interval schedule associated with the right terminal link (the left schedule
was always fixed-interval 8 seconds), separately for the beginning (left panel)
and end (right panel) for Pigeon 221 from Grace, Bragason, and McLean’s
(2003) Experiment 2. Each data point represents performance from one
session. From “Rapid Acquisition of Preference in Concurrent Chains,” by
R. C. Grace, O. Bragason, and A. P. McLean, 2003, Journal of the
Experimental Analysis of Behavior, 80, p. 248. Copyright 2003 by the Society
for the Experimental Analysis of Behavior, Inc. Adapted with permission.
in a terminal link, subjects made a decision about durations as well as with longer durations. They
whether the preceding delay was relatively short or confirmed this prediction, which is contrary to mod-
long. If the delay was judged to be short, response els for steady-state choice such as CCM, HVA, and
strength for the corresponding initial link increased, DRT, in their experiment.
whereas if it was judged to be long, response Recently, Christensen and Grace (2010) proposed
strength decreased. Changes in response strength a model for steady-state choice based on the decision
were made according to a linear-operator rule. model. They derived an expression for relative
Grace and McLean showed that the model could initial-link response strength after sustained training
account for the individual differences in their data with a pair of terminal-link schedules. The major fac-
depending on a parameter that determined how tor determining response strength was the probabil-
accurate the decisions were. When decisions were ity that a terminal-link delay was judged to be short
relatively inaccurate, response allocation was approx- relative to a criterion. According to their model, the
imately linearly related to the log immediacy ratio, criterion is calculated as the average interval between
but when decisions were relatively accurate, response all stimuli correlated with reinforcement in the situa-
allocation was a nonlinear (sigmoidal) function of tion (including both intervals between initial-link
the log immediacy ratio (see Figure 14.8). onset and terminal-link entry and terminal-link entry
Subsequently, Christensen and Grace extended and food). The probability that a delay is judged to
the decision model to predict acquisition of prefer- be short is then calculated from the normal distribu-
ence between FI terminal links when overall initial- tion with the mean equal to the criterion and with
link (Christensen & Grace, 2008) and terminal-link the standard deviation (σ) as a parameter. The stan-
(Christensen & Grace, 2009b) duration varied and dard deviation σ plays a role similar to sensitivity to
when terminal-link schedules changed systemati- delay in the generalized matching law (Equation 4):
cally in an ascending and descending series (Chris- Small values of σ mean that decisions are relatively
tensen & Grace, 2009a). Notably, Christensen and accurate and produce stronger levels of preference;
Grace (2008) showed that the decision model pre- large values of σ result in less accurate decisions and
dicted a bitonic effect of initial-link duration—with lower levels of preference. Christensen and Grace
a decrease in preference at very short initial-link showed that relative response strength could be sub-
328
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
1 1
y = 1.60x - 0.00
0.8 0.8
R² = 0.90
Predicted Log Response Ratio
Predicted Log Response Ratio

0.6 0.6
y = 0.72x - 0.00 0.4 0.4

R² = 1.00
0.2 0.2
0 0
-0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8
-0.2 -0.2
-0.4 -0.4
-0.6 -0.6
-0.8 -0.8
-1 -1
Log Immediacy Ratio Log Immediacy Ratio
1 y = 1.31x - 0.09 y = 1.37x - 0.02

R² = 0.85 R² = 0.87
Log IL Response Ratio
0.5
Log IL Response Ratio
0
-0.5
-1
-1.5
-0.8 -0.3 0.2 0.7 -0.8 -0.3 0.2 0.7
Log Immediacy Ratio Log Immediacy Ratio
Figure 14.8. The upper panels show predictions of the decision model when the value of σ is relatively
large (σ = 0.30; left panel) or small (σ = 0.075; right panel). When σ is large, the predicted response allocation
is approximately linearly related to the log immediacy ratio, but when σ is small, the relation is nonlinear. The
lower panels show results from two pigeons in Grace and McLean’s (2006) experiment, in which pigeons chose
between two different fixed-interval terminal links each session, which illustrate both linear and nonlinear
relations. From “Rapid Acquisition of Preference in Concurrent Chains,” by R. C. Grace, O. Bragason, and
A. P. McLean, 2003, Journal of the Experimental Analysis of Behavior, 80, pp. 242 and 245. Copyright 2003 by
the Society for the Experimental Analysis of Behavior, Inc. Adapted with permission.
stituted for the terminal-link value ratio in the con- some data sets, and its sensitivity values were more
catenated generalized matching law, yielding a model consistent between studies with FI and VI schedules
for steady-state choice. than those of the other models. Christensen and
The steady-state decision model was fitted to the Grace’s (2010) results show that the decision model,
same archival data sets used by Grace (1994) and originally developed to explain individual differ-
Mazur (2001). The model performed very well in ences in responding under rapid-acquisition condi-
terms of goodness of fit, with an average percentage tions, yields an account of molar, steady-state choice
of the variance across the studies that was slightly that is at least as good as existing models. Notably,
higher than for both CCM and HVA. In addition, the the model does not rely on the construct of condi-
decision model was arguably more parsimonious tioned reinforcement; rather than the terminal-link
because it required fewer estimated parameters for stimuli acquiring value, the model assumes that
329
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
iscriminating a terminal-link delay as short (or

d matching 50 years ago. The matching law has
long) strengthens (or weakens) responding in the evolved into a framework that can accurately
associated initial link. Thus, the decision model is model how response allocation in concurrent
able to accommodate results of studies showing that choice situations depends on various aspects of
temporal control of terminal-link responding and reinforcement, constituting a quantitative law of
initial-link preference can be dissociated (Grace & effect. Basic research on behavioral choice has led
Nevin, 1999; Kyonka & Grace, 2007), which is to developments such as hyperbolic discounting
problematic for accounts based on conditioned rein- and self-control, which have enhanced researchers’
forcement. However, we should note that the deci- understanding of human behavior in complex and
sion model, as currently formulated, does not give a realistic situations outside the laboratory. Although
comprehensive explanation of concurrent-chains a definitive explanation of the origins of matching
performance. For example, the model does not has yet to be achieved, research on acquisition has
accommodate effects of reinforcer magnitude and begun to identify the processes underlying choice.
probability and terminal-link signaling conditions As models for acquisition develop, they may ulti-
(Williams & Fantino, 1978) and has not been mately be able to provide a comprehensive account
applied to three-alternative choice (Mazur, 2000). of steady-state behavior as well, and molar phe-
Finally, it is important to note that most models nomena such as matching and hyperbolic discount-
for acquisition of concurrent choice—including ing may be explainable in terms of a more
Mazur (1992), Grace (2002), Grace and McLean fundamental process. In that case, research will
(2006), and Baum and Davison (2009), among oth- have identified the variables controlling the alloca-
ers (e.g., Lau & Glimcher, 2005)—are based on a tion of operant behavior and answered some of the
linear-operator approach. Linear-operator models most important and enduring problems in psychology.
have been extremely popular in research on condi-
tioning and behavior (Bush & Mosteller, 1955; References
Rescorla & Wagner, 1972) because they are simple; Alsop, B., & Davison, M. (1991). Effects of varying stimu-
flexible; well-suited for modeling discrete events, lus disparity and the reinforcer ratio in concurrent-
schedule and signal-detection procedures. Journal
such as sequences of responses or reinforcers; and
predict smooth, negatively accelerated acquisition doi:10.1901/jeab.1991.56-67
curves. Although smooth curves are typical for data
Aparicio, C. F., & Baum, W. M. (2006). Fix and sample
that have been averaged over many subjects or trials, with rats in the dynamics of choice. Journal of
Gallistel, Fairhurst, and Balsam (2004) have argued the Experimental Analysis of Behavior, 86, 43–63.
persuasively that for individual subjects and trials, doi:10.1901/jeab.2006.57-05
acquisition is better described as an all-or-none pro- Autor, S. M. (1969). The strength of conditioned rein-
cess. Indeed, there is evidence that shifts in response forcers as a function of frequency and probability of
reinforcement. In D. Hendry (Ed.), Conditioned rein-
allocation under both concurrent schedules and forcement (pp. 127–162). Homewood, IL: Dorsey Press.
concurrent chains can be abrupt rather than grad-
Bailey, J. T., & Mazur, J. E. (1990). Choice behavior
ual, provided that subjects have sufficient experi- in transition: Development of preference for the
ence with changing reinforcement contingencies higher probability of reinforcement. Journal of the
(e.g., Gallistel, Mark, King, & Latham, 2001; Kyo- Experimental Analysis of Behavior, 53, 409–422.
nka & Grace, 2007). Whether the current genera- doi:10.1901/jeab.1990.53-409
tion of linear-operator models can be extended to Baum, W. M. (1974a). Chained concurrent schedules:
explain this result is a matter for future research. Reinforcement as situation transition. Journal of
doi:10.1901/jeab.1974.22-91
Conclusion Baum, W. M. (1974b). On two types of deviation from
the matching law: Bias and undermatching. Journal
Research on behavioral choice has made consider- of the Experimental Analysis of Behavior, 22, 231–242.
able progress since Herrnstein’s (1961) report of doi:10.1901/jeab.1974.22-231
330
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Baum, W. M. (1979). Matching, undermatching, and procedure: Effects of overall terminal-link duration.
overmatching in studies of choice. Journal of the Behavioural Processes, 81, 233–237. doi:10.1016/j.
Experimental Analysis of Behavior, 32, 269–281. beproc.2009.01.006
doi:10.1901/jeab.1979.32-269 Christensen, D. R., & Grace, R. C. (2009b). Response
Baum, W. M. (1981). Optimization and the matching allocation in concurrent chains when terminal-link
law as accounts of instrumental behavior. Journal of delays follow an ascending and descending series.
the Experimental Analysis of Behavior, 36, 387–403. Journal of the Experimental Analysis of Behavior, 91,
doi:10.1901/jeab.1981.36-387 1–20. doi:10.1901/jeab.2009.91-1
Baum, W. M. (2003). The molar view of behavior and Christensen, D. R., & Grace, R. C. (2010). A decision
its usefulness in behavior analysis. Behavior Analyst model for steady-state choice in concurrent chains.
Today, 4, 78–81. Journal of the Experimental Analysis of Behavior, 94,
Baum, W. M., & Davison, M. (2004). Choice in a vari- 227–240. doi:10.1901/jeab.2010.94-227
able environment: Visit patterns in the dynamics Chung, S.-H. (1965). Effects of delayed reinforcement in
of choice. Journal of the Experimental Analysis of a concurrent situation. Journal of the Experimental
Baum, W. M., & Davison, M. (2009). Modeling the jeab.1965.8-439
dynamic of choice. Behavioural Processes, 81, Chung, S.-H., & Herrnstein, R. J. (1967). Choice and
189–194. doi:10.1016/j.beproc.2009.01.005 delay of reinforcement. Journal of the Experimental
Baum, W. M., & Rachlin, H. C. (1969). Choice as time Analysis of Behavior, 10, 67–74. doi:10.1901/
allocation. Journal of the Experimental Analysis of jeab.1967.10-67
Behavior, 12, 861–874. doi:10.1901/jeab.1969.12-861 Clement, T. S., Feltus, J., Kaiser, D. H., & Zentall, T. R.
(2000). “Work ethic” in pigeons: Reward value
Baum, W., Schwendiman, J., & Bell, K. (1999). Choice,
is directly related to the effort or time required to
contingency discriminability, and foraging theory.
obtain the reward. Psychonomic Bulletin and Review,
7, 100–106. doi:10.3758/BF03210727
355–373. doi:10.1901/jeab.1999.71-355
Cutting, J. E. (2000). Accuracy, scope, and flexibility of
Berg, M. E., & Grace, R. C. (2004). Independence of
models. Journal of Mathematical Psychology, 44, 3–19.
terminal-link entry rate and immediacy in concur-
doi:10.1006/jmps.1999.1274
rent chains. Journal of the Experimental Analysis of
Behavior, 82, 235–251. doi:10.1901/jeab.2004. Davis, D. G. S., Staddon, J. E. R., Machado, A., & Palmer,
82-235 R. G. (1993). The process of recurrent choice.
Buckner, R. L., Green, L., & Myerson, J. (1993). Short-
0033-295X.100.2.320
term and long-term effects of reinforcers on choice.
Journal of the Experimental Analysis of Behavior, 59, Davison, M. C. (1969). Preference for mixed-interval
293–307. doi:10.1901/jeab.1993.59-293 versus fixed-interval schedules. Journal of the
Burnham, K. P., & Anderson, D. R. (2002). Model
doi:10.1901/jeab.1969.12-247
selection and multimodel inference: A practical
information-theoretic approach (2nd ed.). New York, Davison, M. C. (1972). Preference for mixed-interval
NY: Springer-Verlag. versus fixed-interval schedules: Number of compo-
nent intervals. Journal of the Experimental Analysis of
Bush, R. R., & Mosteller, F. (1955). Stochastic models for
learning. New York, NY: Wiley.
Davison, M. C. (1987). The analysis of concurrent-chain
Caraco, T., Martindale, S., & Whitham, T. S. (1980).
performance. In M. L. Commons, J. E. Mazur, J. A.
An empirical demonstration of risk-sensitive forag-
Nevin, & H. Rachlin (Eds.), Quantitative analyses of
ing preferences. Animal Behaviour, 28, 820–830.
behavior: Vol. 5. The effect of delay and of intervening
doi:10.1016/S0003-3472(80)80142-4
events on reinforcer value (pp. 225–241). Hillsdale,
Catania, A. C. (1963). Concurrent performances: A NJ: Erlbaum.
baseline for the study of reinforcement magnitude.
Davison, M., & Baum, W. M. (2000). Choice in a vari-
able environment: Every reinforcer counts. Journal
299–300. doi:10.1901/jeab.1963.6-299
Christensen, D. R., & Grace, R. C. (2008). Rapid acqui- doi:10.1901/jeab.2000.74-1
sition in concurrent chains: Effects of initial-link
Davison, M., & Baum, W. M. (2002). Choice in a vari-
duration. Behavioural Processes, 78, 217–223.
able environment: Effects of blackout duration
doi:10.1016/j.beproc.2008.01.006
and extinction between components. Journal of
Christensen, D. R., & Grace, R. C. (2009a). Response the Experimental Analysis of Behavior, 77, 65–89.
allocation in a rapid-acquisition concurrent-chains doi:10.1901/jeab.2002.77-65
331
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
Davison, M., & Baum, W. M. (2003). Every reinforcer Fantino, E., & Abarca, N. (1985). Choice, optimal forag-
counts: Reinforcer magnitude and local preference. ing, and the delay-reduction hypothesis. Behavioral
Journal of the Experimental Analysis of Behavior, 80, and Brain Sciences, 8, 315–362. doi:10.1017/
95–129. doi:10.1901/jeab.2003.80-95 S0140525X00020847
Davison, M., & Baum, W. M. (2006). Do conditional Fantino, E., & Dunn, R. (1983). The delay-reduction
reinforcers count? Journal of the Experimental hypothesis: Extension to three-alternative choice.
Analysis of Behavior, 86, 269–283. doi:10.1901/ Journal of Experimental Psychology: Animal
jeab.2006.56-05 Behavior Processes, 9, 132–146. doi:10.1037/0097-
Davison, M., & Baum, W. M. (2007). Local effects of 7403.9.2.132
delayed food. Journal of the Experimental Analysis of Fantino, E., Dunn, R., & Meck, W. (1979). Percentage
Behavior, 87, 241–260. doi:10.1901/jeab.2007.13-06 reinforcement and choice. Journal of the Experimental
Davison, M., & Elliffe, D. (2009). Variance matters: The Analysis of Behavior, 32, 335–340. doi:10.1901/
shape of a datum. Behavioural Processes, 81, 216–222. jeab.1979.32-335
doi:10.1016/j.beproc.2009.01.004 Fantino, E., Preston, R. A., & Dunn, R. (1993). Delay
Davison, M., & Jenkins, P. E. (1985). Stimulus discrim- reduction: Current status. Journal of the Experimental
inability, contingency discriminability, and schedule Analysis of Behavior, 60, 159–169. doi:10.1901/
performance. Animal Learning and Behavior, 13, jeab.1993.60-159
77–84. doi:10.3758/BF03213368
Fantino, E., & Romanowich, P. (2007). The effect of
Davison, M., & Jones, B. M. (1995). A quantitative analy- conditioned reinforcement rate on choice: A review.
sis of extreme choice. Journal of the Experimental Journal of the Experimental Analysis of Behavior, 87,
Analysis of Behavior, 64, 147–162. doi:10.1901/ 409–421. doi:10.1901/jeab.2007.44-06
jeab.1995.64-147
Findley, J. D. (1958). Preference and switching under
Davison, M., & McCarthy, D. (1988). The matching law: concurrent scheduling. Journal of the Experimental
A research review. Hillsdale, NJ: Erlbaum. Analysis of Behavior, 1, 123–144. doi:10.1901/
Davison, M., & Nevin, J. A. (1999). Stimuli, reinforc- jeab.1958.1-123
ers, and behaviour: An integration. Journal of the Fisher, W. W., & Mazur, J. E. (1997). Basic and applied
Experimental Analysis of Behavior, 71, 439–482. research on choice responding. Journal of Applied
doi:10.1901/jeab.1999.71-439 Behavior Analysis, 30, 387–410. doi:10.1901/
Dufort, R. H., Gutman, N., & Kimble, G. A. (1954). One- jaba.1997.30-387
trial discrimination reversal in the white rat. Journal
of Comparative and Physiological Psychology, 47, Gallistel, C. R., Fairhurst, S., & Balsam, P. (2004). The
248–249. doi:10.1037/h0057856 learning curve: Implications of a quantitative analy-
sis. Proceedings of the National Academy of Sciences
Dunn, R., & Spetch, M. L. (1990). Choice with uncertain of the United States of America, 101, 13124–13131.
outcomes. Journal of the Experimental Analysis of doi:10.1073/pnas.0404965101
Gallistel, C. R., Mark, T. A., King, A., & Latham, P. E.
Elliffe, D., & Alsop, B. (1996). Concurrent choice: Effects (2001). The rat approximates an ideal detector of
of overall reinforcer rate and the temporal distri- changes in rates of reward: Implications for the law
bution of reinforcers. Journal of the Experimental of effect. Journal of Experimental Psychology: Animal
Analysis of Behavior, 65, 445–463. doi:10.1901/ Behavior Processes, 27, 354–372. doi:10.1037/0097-
jeab.1996.65-445 7403.27.4.354
Elliffe, D., Davison, M., & Landon, J. (2008). Relative
Gibbon, J. (1977). Scalar expectancy theory and Weber’s
reinforcer rates and magnitudes do not control
law in animal timing. Psychological Review, 84,
concurrent choice independently. Journal of the
279–325. doi:10.1037/0033-295X.84.3.279
doi:10.1901/jeab.2008.90-169 Gipson, C. D., Alessandri, J. J. D., Miller, H. C., &
Fantino, E. (1969). Choice and rate of reinforcement. Zentall, T. R. (2009). Preference for 50% rein-
Journal of the Experimental Analysis of Behavior, 12, forcement over 75% reinforcement by pigeons.
723–730. doi:10.1901/jeab.1969.12-723 Learning and Behavior, 37, 289–298. doi:10.3758/
LB.37.4.289
Fantino, E. (1977). Conditioned reinforcement: Choice
and information. In W. Honig & J. E. R. Staddon Grace, R. C. (1993). [Concurrent schedules: Effects of
(Eds.), Handbook of operant behavior (pp. 313–339). inter-reinforcer interval distributions]. Unpublished
New York, NY: Prentice Hall. experiment.
Fantino, E. (2001). Context: A central concept. Grace, R. C. (1994). A contextual model of concurrent-
Behavioural Processes, 54, 95–110. doi:10.1016/ chains choice. Journal of the Experimental Analysis of
S0376-6357(01)00152-8 Behavior, 61, 113–129. doi:10.1901/jeab.1994.61-113
332
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace, R. C. (1996). Choice between fixed and variable Herrnstein, R. J. (1964a). Aperiodicity as a factor in
delays to reinforcement in the adjusting-delay proce- choice. Journal of the Experimental Analysis of
dure and concurrent chains. Journal of Experimental Behavior, 7, 179–182. doi:10.1901/jeab.1964.7-179
Psychology: Animal Behavior Processes, 22, 362–383. Herrnstein, R. J. (1964b). Secondary reinforcement
doi:10.1037/0097-7403.22.3.362 and rate of primary reinforcement. Journal of
Grace, R. C. (2002). Acquisition of preference in concur- the Experimental Analysis of Behavior, 7, 27–36.
rent chains: Comparing linear-operator and memory- doi:10.1901/jeab.1964.7-27
representational models. Journal of Experimental Herrnstein, R. J. (1970). On the law of effect. Journal of
Psychology: Animal Behavior Processes, 28, 257–276. the Experimental Analysis of Behavior, 13, 243–266.
doi:10.1037/0097-7403.28.3.257 doi:10.1901/jeab.1970.13-243
Grace, R. C., & Bragason, O. (2004). Does the terminal- Herrnstein, R. J., & Vaughan, W. (1980). Melioration
link effect depend on duration or reinforcement and behavioural allocation. In J. E. R. Staddon (Ed.),
rate? Behavioural Processes, 67, 67–79. doi:10.1016/j. Limits to action (pp. 143–176). New York, NY:
beproc.2004.02.006 Academic Press.
Grace, R. C., Bragason, O., & McLean, A. P. (2003). Heyman, G. M., & Tanz, L. (1995). How to teach a
Rapid acquisition of preference in concurrent chains. pigeon to maximize overall reinforcement rate.
Journal of the Experimental Analysis of Behavior, 80, Journal of the Experimental Analysis of Behavior, 64,
235–252. doi:10.1901/jeab.2003.80-235 277–297. doi:10.1901/jeab.1995.64-277
Grace, R. C., & McLean, A. P. (2006). Rapid acquisition Hinson, J. M., & Staddon, J. E. R. (1983a). Hill-climbing
in concurrent chains: Evidence for a decision model. by pigeons. Journal of the Experimental Analysis of
Journal of the Experimental Analysis of Behavior, 85, Behavior, 39, 25–47. doi:10.1901/jeab.1983.39-25
181–202. doi:10.1901/jeab.2006.72-04
Hinson, J. M., & Staddon, J. E. R. (1983b). Matching,
Grace, R., & Nevin, J. A. (1997). On the relation between maximizing, and hill-climbing. Journal of the
preference and resistance to change. Journal of Experimental Analysis of Behavior, 40, 321–331.
doi:10.1901/jeab.1997.67-43
Horner, J. M., & Staddon, J. E. R. (1987). Probabilistic
Grace, R. C., & Nevin, J. A. (1999). Timing and choice choice: A simple invariance. Behavioural Processes,
in concurrent chains. Behavioural Processes, 45, 15, 59–92. doi:10.1016/0376-6357(87)90034-9
115–127. doi:10.1016/S0376-6357(99)00013-3
Houston, A. I., & McNamara, J. (1981). How to maxi-
Grace, R., & Savastano, H. I. (1997). Transfer tests of mize reward rate on two variable-interval paradigms.
stimulus value in concurrent chains. Journal of Journal of the Experimental Analysis of Behavior, 35,
the Experimental Analysis of Behavior, 68, 93–115. 367–396. doi:10.1901/jeab.1981.35-367
doi:10.1901/jeab.1997.68-93
Hunter, I., & Davison, M. (1985). Determination of a
Grace, R. C., & Savastano, H. I. (2000). Temporal con- behavioral transfer function: White-noise analysis of
text and conditioned reinforcement value. Journal session-to-session response-ratio dynamics on con-
of Experimental Psychology: General, 129, 427–443. current VI VI schedules. Journal of the Experimental
doi:10.1037/0096-3445.129.4.427 Analysis of Behavior, 43, 43–59. doi:10.1901/jeab.
Green, L., & Myerson, J. (2004). A discounting frame- 1985.43-43
work for choice with delayed and probabilistic Kendall, S. B. (1974). Preference for intermittent rein-
rewards. Psychological Bulletin, 130, 769–792. forcement. Journal of the Experimental Analysis of
doi:10.1037/0033-2909.130.5.769 Behavior, 21, 463–473. doi:10.1901/jeab.1974.21-463
Green, L., & Snyderman, M. (1980). Choice between Killeen, P. (1968). On the measurement of reinforce-
rewards differing in amount and delay: Toward ment frequency in the study of preference. Journal of
a choice model of self control. Journal of the the Experimental Analysis of Behavior, 11, 263–269.
Experimental Analysis of Behavior, 34, 135–147. doi:10.1901/jeab.1968.11-263
doi:10.1901/jeab.1980.34-135 Killeen, P. (1972). The matching law. Journal of the
Grondin, S. (2001). From physical time to the first Experimental Analysis of Behavior, 17, 489–495.
and second moments of psychological time. doi:10.1901/jeab.1972.17-489
Psychological Bulletin, 127, 22–44. doi:10.1037/0033- Kirby, K. N., & Herrnstein, R. J. (1995). Preference rever-
2909.127.1.22 sals due to myopic discounting of delayed reward.
Herrnstein, R. J. (1961). Relative and absolute strength of Psychological Science, 6, 83–89. doi:10.1111/
response as a function of frequency of reinforcement. j.1467-9280.1995.tb00311.x
Journal of the Experimental Analysis of Behavior, 4, Krägeloh, C. U., & Davison, M. (2003). Concurrent-
267–272. doi:10.1901/jeab.1961.4-267 schedule performance in transition: Changeover
333
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
delays and signaled reinforcer ratios. Journal of MacEwen, D. (1972). The effects of terminal-link fixed-
the Experimental Analysis of Behavior, 79, 87–109. interval and variable-interval schedules on respond-
doi:10.1901/jeab.2003.79-87 ing under concurrent chained schedules. Journal of
Krägeloh, C. U., Davison, M., & Elliffe, D. M. (2005). the Experimental Analysis of Behavior, 18, 253–261.
Local preference in concurrent schedules: The effects doi:10.1901/jeab.1972.18-253
of reinforcer sequences. Journal of the Experimental Mattson, K. M., Hucks, A. D., Grace, R. C., & McLean, A.
Analysis of Behavior, 84, 37–64. doi:10.1901/ P. (2010). Signaled and unsignaled terminal links in
jeab.2005.114-04 concurrent chains: I. Effects of reinforcer probability
Krägeloh, C. U., Elliffe, D. M., & Davison, M. (2006). and immediacy. Journal of the Experimental Analysis
Contingency discriminability and peak shift in of Behavior, 94, 327–352.
concurrent schedules. Journal of the Experimental Mazur, J. E. (1984). Tests of an equivalence rule for fixed
Analysis of Behavior, 86, 11–30. doi:10.1901/jeab. and variable reinforcer delays. Journal of Experimental
2006.11-05 Psychology: Animal Behavior Processes, 10, 426–436.
Kyonka, E. G. E., & Grace, R. C. (2007). Rapid acqui- doi:10.1037/0097-7403.10.4.426
sition of choice and timing in pigeons. Journal of Mazur, J. E. (1985). Probability and delay of reinforce-
Experimental Psychology: Animal Behavior Processes, ment as factors in a discrete-trial choice. Journal of
33, 392–408. doi:10.1037/0097-7403.33.4.392 the Experimental Analysis of Behavior, 43, 341–351.
Lagorio, C. H., & Hackenberg, T. D. (2010). Risky choice doi:10.1901/jeab.1985.43-341
in pigeons and humans: A cross-species comparison. Mazur, J. E. (1986a). Choice between single and mul-
Journal of the Experimental Analysis of Behavior, 93, tiple delayed reinforcers. Journal of the Experimental
27–44. doi:10.1901/jeab.2010.93-27 Analysis of Behavior, 46, 67–77. doi:10.1901/
Landon, J., & Davison, M. (2001). Reinforcer-ratio varia- jeab.1986.46-67
tion and its effects on rate of adaptation. Journal of Mazur, J. E. (1986b). Fixed and variable ratios and
the Experimental Analysis of Behavior, 75, 207–234. delays: Further tests of an equivalence rule. Journal of
doi:10.1901/jeab.2001.75-207 Experimental Psychology: Animal Behavior Processes,
Landon, J., Davison, M., & Elliffe, D. (2002). Concurrent 12, 116–124. doi:10.1037/0097-7403.12.2.116
schedules: Short- and log-term effects of reinforcers. Mazur, J. E. (1987). An adjusting procedure for study-
Journal of the Experimental Analysis of Behavior, 77, ing delayed reinforcement. In M. L. Commons, J. E.
257–271. doi:10.1901/jeab.2002.77-257 Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative
Landon, J., Davison, M., & Elliffe, D. (2003). Concurrent analyses of behavior: Vol. 5. The effect of delay and of
schedules: Reinforcer magnitude effects. Journal of intervening events on reinforcement value (pp. 55–73).
the Experimental Analysis of Behavior, 79, 351–365. Hillsdale, NJ: Erlbaum.
doi:10.1901/jeab.2003.79-351 Mazur, J. E. (1988). Choice between small certain and
Lau, B., & Glimcher, P. W. (2005). Dynamic response- large uncertain reinforcers. Animal Learning and
by-response models of matching behavior in rhesus Behavior, 16, 199–205. doi:10.3758/BF03209066
monkeys. Journal of the Experimental Analysis of Mazur, J. E. (1989). Theories of probabilistic reinforce-
Behavior, 84, 555–579. doi:10.1901/jeab.2005.110-04 ment. Journal of the Experimental Analysis of Behavior,
51, 87–99. doi:10.1901/jeab.1989.51-87
Leon, M. I., & Gallistel, C. R. (1998). Self-stimulating
rats combine subjective reward magnitude and Mazur, J. E. (1992). Choice behavior in transition:
subjective reward rate multiplicatively. Journal of Development of preference with ratio and interval
Experimental Psychology: Animal Behavior Processes, schedules. Journal of Experimental Psychology: Animal
24, 265–277. doi:10.1037/0097-7403.24.3.265 Behavior Processes, 18, 364–378. doi:10.1037/0097-
7403.18.4.364
Locey, M. L., Pietras, C. J., & Hackenberg, T. D. (2009).
Human risky choice: Delay sensitivity depends on Mazur, J. E. (1993). Predicting the strength of a condi-
reinforcer type. Journal of Experimental Psychology: tioned reinforcer: Effects of delay and uncertainty.
Animal Behavior Processes, 35, 15–22. doi:10.1037/ Current Directions in Psychological Science, 2, 70–74.
a0012378 doi:10.1111/1467-8721.ep10770907
Logue, A. W., Rodriguez, M. L., Pena-Correal, T. E., & Mazur, J. E. (1994). Effects of intertribal reinforcers
Mauro, B. C. (1984). Choice in a self-control para- on self-control choice. Journal of the Experimental
digm: Quantification of experience-based differences. Analysis of Behavior, 61, 83–96. doi:10.1901/jeab.
Journal of the Experimental Analysis of Behavior, 41, 1994.61-83
53–67. doi:10.1901/jeab.1984.41-53 Mazur, J. E. (1995). Conditioned reinforcement and
MacDonall, J. S. (2009). The stay/switch model of con- choice with delayed and uncertain primary reinforc-
current choice. Journal of the Experimental Analysis of ers. Journal of the Experimental Analysis of Behavior,
Behavior, 91, 21–39. doi:10.1901/jeab.2009.91-21 63, 139–150. doi:10.1901/jeab.1995.63-139
334
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Mazur, J. E. (1996). Past experience, recency, and spon- schedules. Behavioural Processes, 53, 171–180.
taneous recovery in choice behavior. Learning and doi:10.1016/S0376-6357(01)00137-1
Behavior, 24, 1–10. doi:10.3758/BF03198948 Mazur, J. E., & Coe, D. (1987). Tests of transitivity in
Mazur, J. E. (1997). Choice, delay, probability, and con- choices between fixed and variable reinforcer delays.
ditioned reinforcement. Learning and Behavior, 25, Journal of the Experimental Analysis of Behavior, 47,
131–147. doi:10.3758/BF03199051 287–297. doi:10.1901/jeab.1987.47-287
Mazur, J. E. (1998). Choice with delayed and probabilis- Mazur, J. E., & Kralik, J. D. (1990). Choice between
tic reinforcers: Effects of prereinforcer and postrein- delayed reinforcers and fixed-ratio schedules requir-
forcer stimuli. Journal of the Experimental Analysis of ing forceful responding. Journal of the Experimental
jeab.1990.53-175
Mazur, J. E. (2000). Two- versus three-alternative
concurrent-chain schedules: A test of three mod- Mazur, J. E., & Ratti, T. A. (1991). Choice behavior in
els. Journal of Experimental Psychology: Animal transition: Development of preference in a free-
Behavior Processes, 26, 286–293. doi:10.1037/0097- operant procedure. Animal Learning and Behavior, 19,
7403.26.3.286 241–248. doi:10.3758/BF03197882
Mazur, J. E. (2001). Hyperbolic value addition and gen- Mazur, J. E., & Romano, A. (1992). Choice with delayed
eral models of animal choice. Psychological Review, and probabilistic reinforcers: Effects of variability,
108, 96–112. doi:10.1037/0033-295X.108.1.96 time between trials, and conditioned reinforcers.
Mazur, J. E. (2002a). Concurrent-chain performance 513–525. doi:10.1901/jeab.1992.58-513
in transition: Effects of terminal-link duration and
individual reinforcers. Learning and Behavior, 30, McDevitt, M., Spetch, M., & Dunn, R. (1997). Contiguity
249–260. doi:10.3758/BF03192834 and conditioned reinforcement in probabilis-
tic choice. Journal of the Experimental Analysis of
Mazur, J. E. (2002b). Evidence against a constant- Behavior, 68, 317–327. doi:10.1901/jeab.1997.68-317
difference effect in concurrent-chain schedules.
Journal of the Experimental Analysis of Behavior, 77, Miller, J. T., Saunders, S. S., & Bourland, G. (1980).
147–155. doi:10.1901/jeab.2002.77-147 The role of stimulus disparity in concurrently avail-
able reinforcement schedules. Animal Learning and
Mazur, J. E. (2003). Effects of free-food deliveries and Behavior, 8, 635–641. doi:10.3758/BF03197780
delays on choice under concurrent-chain schedules.
Miller, R. R., & Grace, R. C. (2003). Conditioning
Behavioural Processes, 64, 251–260. doi:10.1016/
and learning. In I. B. Weiner, A. F. Healy, D. K.
S0376-6357(03)00140-2
Freedheim, & R. W. Proctor (Eds.), Handbook of psy-
Mazur, J. E. (2005). Effects of reinforcer probability, chology: Experimental psychology (pp. 355–397). New
delay, and response requirements on the choices of York, NY: Wiley. doi:10.1002/0471264385.wei0413
rats and pigeons: Possible species differences. Journal Moore, J. (1984). Choice and transformed interreinforce-
of the Experimental Analysis of Behavior, 83, 263–279. ment intervals. Journal of the Experimental Analysis of
doi:10.1901/jeab.2005.69-04 Behavior, 42, 321–335. doi:10.1901/jeab.1984.42-321
Mazur, J. E. (2006). Mathematical models and the Moore, J. (2008). A critical appraisal of contemporary
experimental analysis of behaviour. Journal of the approaches in the quantitative analysis of behavior.
Experimental Analysis of Behavior, 85, 275–291. Psychological Record, 58, 641–664.
doi:10.1901/jeab.2006.65-05
Myerson, J., & Green, L. (1995). Discounting of delayed
Mazur, J. E. (2007a). Rats’ choices between one and two rewards: Models of individual choice. Journal of
delayed reinforcers. Learning and Behavior, 35, the Experimental Analysis of Behavior, 64, 263–276.
169–176. doi:10.3758/BF03193052 doi:10.1901/jeab.1995.64-263
Mazur, J. E. (2007b). Species differences between rats and Myerson, J., & Miezin, F. M. (1980). The kinetics of
pigeons in choices with probabilistic and delayed choice: An operant systems analysis. Psychological
reinforcers. Behavioural Processes, 75, 220–224. Review, 87, 160–174. doi:10.1037/0033-295X.87.2.160
doi:10.1016/j.beproc.2007.02.004
Navarick, D. J., & Fantino, E. (1974). Stochastic tran-
Mazur, J. E., & Biondi, D. R. (2009). Delay-amount sitivity and unidimensional behavior theories.
tradeoffs in choices by pigeons and rats: Hyperbolic Psychological Review, 81, 426–441. doi:10.1037/
versus exponential discounting. Journal of the h0036953
doi:10.1901/jeab.2009.91-197 Navarick, D. J., & Fantino, E. (1976). Self-control and
general models of choice. Journal of Experimental
Mazur, J. E., Blake, N., & McManus, C. (2001). Psychology: Animal Behavior Processes, 2, 75–87.
Transitional choice behavior in concurrent-chain doi:10.1037/0097-7403.2.1.75
335
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Grace and Hucks
Nevin, J. A. (1998). Choice and momentum. In W. Roberts, S., & Pashler, H. (2000). How persuasive is a
O’Donohue (Ed.), Learning and behavior therapy good fit? A comment on theory-testing. Psychological
(pp. 230–251). Boston, MA: Allyn & Bacon. Review, 107, 358–367. doi:10.1037/0033-
295X.107.2.358
Nevin, J. A., Davison, M., & Shahan, T. A. (2005). A the-
ory of attending and reinforcement in conditional dis- Rodriguez, M. L., & Logue, A. W. (1988). Adjusting delay
criminations. Journal of the Experimental Analysis of to reinforcement: Comparing choice in pigeons and
Behavior, 84, 281–303. doi:10.1901/jeab.2005.97-04 humans. Journal of Experimental Psychology: Animal
Nevin, J. A., & Grace, R. C. (2000). Behavioral momen- Behavior Processes, 14, 105–117. doi:10.1037/0097-
tum and the law of effect. Behavioral and Brain 7403.14.1.105
Sciences, 23, 73–90. doi:10.1017/S0140525 Rosenthal, R., & DiMatteo, M. R. (2001). Meta-analysis:
X00002405 Recent developments in quantitative methods for
O’Daly, M., Angulo, S., Gipson, C., & Fantino, E. (2006). literature reviews. Annual Review of Psychology, 52,
Influence of temporal context on value in the 59–82. doi:10.1146/annurev.psych.52.1.59
multiple-chains and successive-encounters pro- Savastano, H., & Fantino, E. (1996). Differences in delay,
cedures. Journal of the Experimental Analysis of not ratios, control choice in concurrent chains.
Behavior, 85, 309–328. doi:10.1901/jeab.2006.68-05 Journal of the Experimental Analysis of Behavior, 66,
O’Daly, M., Meyer, S., & Fantino, E. (2005). Value of 97–116. doi:10.1901/jeab.1996.66-97
conditioned reinforcers as a function of tempo- Schofield, G., & Davison, M. (1997). Nonstable concur-
ral context. Learning and Motivation, 36, 42–59. rent choice in pigeons. Journal of the Experimental
doi:10.1016/j.lmot.2004.08.001 Analysis of Behavior, 68, 219–232. doi:10.1901/
Pietras, C. J., Locey, M. L., & Hackenberg, T. D. (2003). jeab.1997.68-219
Human risky choice under temporal constraints: Schuster, R. H. (1969). A functional analysis of
Tests of an energy-budget model. Journal of the conditioned reinforcement. In D. P. Hendry
Experimental Analysis of Behavior, 80, 59–75. (Ed.), Conditioned reinforcement (pp. 192–234).
doi:10.1901/jeab.2003.80-59 Homewood, IL: Dorsey Press.
Pitt, M. A., Myung, I. J., & Zhang, S. (2002). Toward a Scoones, C., Hucks, A., McLean, A. P., & Grace, R. C.
method of selecting among computational models (2009). Effects of free food deliveries and temporal
of cognition. Psychological Review, 109, 472–491. contiguity on choice under concurrent-chain sched-
doi:10.1037/0033-295X.109.3.472 ules. Psychonomic Bulletin and Review, 16, 736–741.
Rachlin, H. (1995). Self-control: Beyond commitment. doi:10.3758/PBR.16.4.736
Behavioral and Brain Sciences, 18, 109–121. doi:10.1017/ Shahan, T. A. (2010). Conditioned reinforcement and
S0140525X00037602 response strength. Journal of the Experimental
Rachlin, H., & Baum, W. M. (1972). Effects of alternative Analysis of Behavior, 93, 269–289. doi:10.1901/
reinforcement: Does the source matter? Journal of jeab.2010.93-269
the Experimental Analysis of Behavior, 18, 231–241. Shimp, C. P. (1966). Probabilistically reinforced two-choice
doi:10.1901/jeab.1972.18-231 behaviour in pigeons. Journal of the Experimental
Rachlin, H., & Green, L. (1972). Commitment, choice, Analysis of Behavior, 9, 443–455. doi:10.1901/jeab.
and self-control. Journal of the Experimental Analysis 1966.9-443
of Behavior, 17, 15–22. doi:10.1901/jeab.1972.17-15 Shimp, C. P. (1969). The concurrent reinforcement of
Rachlin, H., Green, L., Kagel, J. H., & Battalio, R. C. two interresponse times: The relative frequency of
(1976). Economic demand theory and psychological an interresponse time equals its relative harmonic
studies of choice. In G. H. Bower (Ed.), The psychol- length. Journal of the Experimental Analysis of
ogy of learning and motivation (Vol. 10, pp. 129–154). Behavior, 12, 403–411. doi:10.1901/jeab.1969.12-403
New York, NY: Academy Press. Spetch, M. L., Belke, T. W., Barnet, R. C., Dunn, R.,
Rescorla, R. A. (1968). Probability of shock in the pres- & Pierce, W. D. (1990). Suboptimal choice in a
ence and absence of CS in fear conditioning. Journal percentage-reinforcement procedure: Effects of sig-
of Comparative and Physiological Psychology, 66, 1–5. nal condition and terminal-link length. Journal of
doi:10.1037/h0025984 the Experimental Analysis of Behavior, 53, 219–234.
doi:10.1901/jeab.1990.53-219
Rescorla, R. A., & Wagner, A. R. (1972). A theory of
Pavlovian conditioning: Variations in the effective- Spetch, M. L., & Dunn, R. (1987). Choice between
ness of reinforcement and nonreinforcement. In A. reliable and unreliable outcomes: Mixed percent-
H. Black & W. F. Prokasy (Eds.), Classical condition- age reinforcement in concurrent chains. Journal of
ing II: Current research and theory (pp. 64–99). New the Experimental Analysis of Behavior, 47, 57–72.
York, NY: Appleton-Century-Crofts. doi:10.1901/jeab.1987.47-57
336
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Spetch, M. L., Mondloch, M. V., Belke, T. W., & Dunn, and contingency discriminability model as accounts
R. (1994). Determinants of pigeons’ choice between of concurrent schedule performance using a residual
certain and probabilistic outcomes. Animal Learning meta-analysis. Behavioural Processes, 78, 224–230.
and Behavior, 22, 239–251. doi:10.3758/BF03209832 doi:10.1016/j.beproc.2008.02.012
Staddon, J. E. R., & Cerutti, D. T. (2003). Operant con- Williams, B. A. (1988). Reinforcement, choice, and
ditioning. Annual Review of Psychology, 54, 115–144. response strength. In R. C. Atkinson, R. J. Herrnstein,
doi:10.1146/annurev.psych.54.101601.145124 G. Lindzey, & R. D. Luce (Eds.), Stevens’ handbook
Staddon, J. E. R., & Horner, J. M. (1989). Stochastic of experimental psychology: Vol. 2. Learning and
choice models: A comparison between Bush- cognition (2nd ed., pp. 167–244). New York, NY:
Mosteller and a source-independent reward-following Wiley.
model. Journal of the Experimental Analysis of Williams, B. A. (1992). Dissociation of theories of
Behavior, 52, 57–64. doi:10.1901/jeab.1989.52-57 choice by temporal spacing of choice opportuni-
Staddon, J. E. R., & Motheral, S. (1978). On matching ties. Journal of Experimental Psychology: Animal
and maximizing in operant choice experiments. Behavior Processes, 18, 287–297. doi:10.1037/0097-
Psychological Review, 85, 436–444. doi:10.1037/ 7403.18.3.287
0033-295X.85.5.436 Williams, B. A. (1994). The role of probability of rein-
Stagner, J. P., & Zentall, T. R. (2010). Suboptimal choice forcement in models of choice. Psychological Review,
behavior by pigeons. Psychonomic Bulletin and 101, 704–707. doi:10.1037/0033-295X.101.4.704
Review, 17, 412–416. doi:10.3758/PBR.17.3.412 Williams, B. A. (2002). Behavioral contrast redux.
Stout, S. C., & Miller, R. R. (2007). Sometimes- Learning and Behavior, 30, 1–20. doi:10.3758/
competing retrieval (SOCR): A formalization of the BF03192905
comparator hypothesis. Psychological Review, 114, Williams, B. A., & Fantino, E. (1978). Effects on choice
759–783. doi:10.1037/0033-295X.114.3.759 of reinforcement delay and conditioned reinforce-
Stubbs, D. A., & Pliskoff, S. S. (1969). Concurrent ment. Journal of the Experimental Analysis of Behavior,
responding with fixed relative rate of reinforcement. 29, 77–86. doi:10.1901/jeab.1978.29-77
Wixted, J. T. (1989). Nonhuman short-term memory: A
887–895. doi:10.1901/jeab.1969.12-887
quantitative reanalysis of selected findings. Journal of
Sutton, N. P., Grace, R. C., McLean, A. P., & Baum, W. the Experimental Analysis of Behavior, 52, 409–426.
M. (2008). Comparing the generalized matching law doi:10.1901/jeab.1989.52-409
337
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 15
Behavioral Neuroscience
David W. Schaal
The physiologist of the future will tell handbook for behavior analysts because neurosci-
us all that can be known about what is ence is becoming an increasingly behavioral enter-
happening inside the behaving organism. prise, and behavior analysts of the near future will
His account will be an important advance find themselves naturally becoming part of it. This
over a behavioral analysis, because the assertion is supported by recent neuroscience
latter is necessarily “historical”—that is research on the process of reinforcement, which I
to say it is confined to functional rela- briefly review first. I then present a selective survey
tions showing temporal gaps. Something of how the neural mechanisms of reinforcement par-
is done today which affects the behavior ticipate in complex operant behavior, followed by
of an organism tomorrow. No matter examples of the application of behavioral neurosci-
how clearly that fact can be established, ence to human problems. I conclude with a discus-
a step is missing, and we must wait for a sion of the unique conceptual difficulties behavior
physiologist to supply it. He will be able analysts face when they try to integrate behavioral
to show how an organism is changed theory with behavioral neuroscience.
when exposed to contingencies of rein- A note before I begin: Behavioral neuroscience, as
forcement and why the changed organ- currently practiced, is a vast and extremely varied
ism then behaves in a different way, enterprise. This fact is illustrated by a survey of the
possibly at a much later date. What he most recent (as of this writing) 200 articles pub-
discovers cannot invalidate the laws of lished in the Journal of Neuroscience (the official jour-
a science of behavior, but it will make nal of the Society for Neuroscience). Of these articles,
the picture of human action more nearly 38% included behavioral data of some sort; most of
complete. (Skinner, 1974, pp. 236–237) the remainder focused on neurons and glial cells.
Humans served as subjects in 35 of these studies,
Functional relations between an animal’s environ- almost half of them using functional magnetic reso-
ment and its behavior are established, maintained, nance imaging (MRI). Seventeen studies included
altered, and elaborated by experience because it has recording from single neurons in a real-time, behavior-
a nervous system. Behavioral neuroscience is the correlated fashion in rats or monkeys. An almost
investigation of how the nervous system participates bewildering variety of behaviors were studied; details
in and accounts for functional relations between of hearing or seeing (e.g., sound localization, discrimi-
environment and behavior. It is important for a nation of complex moving stimuli), movement and
chapter on behavioral neuroscience to appear in a muscle control (swimming in tadpoles, zebrafish, and
I thank Jane E. Bailly for helpful discussions during the writing of this chapter and for critical input on early versions.
DOI: 10.1037/13937-015
339
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
lamprey; rotarod performance in mice; recovery of environment. The next several sections serve as a
grip in monkeys), behaviors related to pain (pain brief review of mechanisms most likely to play a role
threshold using cold pressor test in humans, tail-flick in the process of the reinforcement.
latencies in mice and rats, tumor-induced pain in
mice, reactions to someone else’s pain in humans), Long-Term Potentiation and Depression
cognitive behaviors (working memory tasks in rats, At the neuronal level, the plasticity that is related to
monkeys, and humans; naming and semantic decision behavior change is frequently synaptic plasticity, in
making in humans; spatial memory tasks in mice; which the ability of neurons to alter the firing of
attention in a dichotic listening task in humans), and neurons they form synapses with is enhanced (or
behavior related to reinforcement value (latencies and potentiated) or decreased (or depressed). Several
choice of saccadic eye movements in monkeys, choice mechanisms for long-term changes in neuronal
in rats, drug-reinforced behavior in rats, estimates of interaction have been proposed; in the sea slug Aply-
value in humans). A chapter for behavior analysts sia californica, for example, classical, or Pavlovian,
cannot review all of what is commonly considered conditioning of siphon withdrawal has been attrib-
behavioral neuroscience, so I focus on research related uted to facilitation of the sensory neurons that stim-
to the central concept of the science, reinforcement. ulate the motor neurons that cause the conditioned
Finally, almost two thirds of the articles report- response, a type of facilitation referred to as presyn-
ing on behavioral studies included measurements aptic (Byrne & Kandel, 1996). More recent evidence
of behavioral and brain variables so that the two with Aplysia, however, has shown the contribution
could be precisely related to each other in time of a more mammalian, postsynaptic mechanism
(e.g., functional MRI, recording from single neu- (Glanzman, 2006). In mammals, the mechanisms of
rons). Although these time-locked observations are synaptic facilitation have primarily been studied
of correlated events (i.e., the dependence of one on using models of long-term potentiation (LTP) and
the other is often not established), they provide an long-term depression (LTD).
empirical rationale for selecting brain regions and LTP was first demonstrated in hippocampal neu-
neurons to manipulate to determine their causal rons by Bliss and Lomo (1973). In their experiments,
contributions to behavior. In addition, far more a high-frequency train of impulses was delivered to
sophisticated and restricted approaches to altering pyramidal cell fibers of the hippocampus. These cell
the activity of the structures so identified (e.g., inter- fibers synapse on granule cells of the dentate gyrus
ference RNA to inhibit protein synthesis, gene ther- of the hippocampus. The result of this repeated,
apy approaches to modulate expression of neuronal high-frequency stimulation of the pyramidal cells
and glial proteins, focused electrolytic lesions was an increase in the magnitude of the electrical
guided by functional data, cooling probes and spe- response of the granule cells to pyramidal cell firing,
cific chemical ligands to temporarily lesion neurons) that is, the firing of the granule cells was potentiated.
are available. Advances such as these are generating This long-lasting change in neuron excitability
outcomes that have a high probability of being reflected an increase in synaptic strength, a change
incorporated into durable, comprehensive theories in excitability at the synapse between pyramidal cells
that integrate neural and behavioral findings. and granule cells. LTP was thought to be a potential
mechanism for learning and memory, and hundreds
of studies have tended to support the claim.
Reinforcement Mechanisms
In associative LTP, weak and intense electrical
Neural plasticity is at the heart of neural theories of stimulation of different neurons that synapse onto
reinforcement-based learning. Neural plasticity is a the same neuronal group are paired, which causes
general term that refers to adaptations in neural potentiation of the response to the weak electrical
structure and activity during development, in stimulus. This associative phenomenon enhances
response to injury, or, in the context of this chapter, the face validity of LTP as a model of Pavlovian con-
as a consequence of an animal’s experience with its ditioning, in which the weak stimulus is thought to
340
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
model the activity stimulated by a conditioned stim- has emerged that the delivery of an unpredicted
ulus (CS), the intense stimulus models the activity positive reinforcer causes firing of dopamine-
stimulated by an unconditioned stimulus, and the containing cells in the ventral tegmental area, which
potentiated response to the weak stimulus is the sur- leads to dopamine release in synapses distributed
rogate for the conditioned response. Demonstrations throughout the brain. Mirenowicz and Schultz
that LTP occurs during learning (Roman, Staubli, & (1994) showed reliable phasic firing (i.e., a short-
Lynch, 1987), that agents that block LTP also block duration pattern of firing that occurs just after an
memory formation (Morris, Anderson, Lynch, & event, as distinct from tonic firing, which is long-
Baudry, 1986), and that LTP and LTD are distrib- duration firing not in phase with an event) of ven-
uted throughout the brain suggest that these forms tral tegmental area dopamine neurons in monkeys
of plasticity may underlie many learning-related that received unpredictable, response-independent
changes in synaptic activity. juice. The same neurons responded to the delivery
LTP and LTD likely play a role in several forms of of the juice during classical conditioning training in
learning. Perhaps the most well-characterized mam- which a CS preceded it. However, after conditioning
malian form of learning, from a neural perspective, was established, the neurons responded to the CS
is delay conditioning of eyeblinks in mice and rats; but not to the juice US. This basic finding (illus-
LTD at parallel fiber–Purkinje cell synapses of the trated in Figure 15.1), that neurons are more likely
cerebellum is thought to be important for delay con- to fire in response to a reinforcer when its delivery
ditioning (Kakegawa et al., 2008), although the is unpredicted, gave rise to the concept of the rein-
extent and nature of its involvement is uncertain forcement prediction error. Consistent with the
(Christian, Poulos, Lavond, & Thompson, 2004; Rescorla–Wagner (1972) model, the prediction
Welsh et al., 2005). Several studies have implicated error hypothesis holds that neurons that detect the
LTP in the amygdala in the establishment of condi- delivery of reinforcers do so relative to the predic-
tioned emotional responses (e.g., a freezing response tive value of the prereinforcer context; if a stimulus
to an auditory CS paired repeatedly with shock; Fan- reliably predicts upcoming reinforcement, that stim-
selow & Kim, 1994). Injection directly into the lat- ulus may increase neuron firing, but delivery of the
eral amygdala of a drug that blocks a subunit of the reinforcer will not. Neural plasticity in this context
N-Methyl-D-aspartate receptor, to which the excit- is evident in the response of ventral tegmental area
atory neurotransmitter glutamate binds and which is neurons to CS delivery after conditioning and
critical to many forms of LTP and LTD, blocked the changes in responsiveness to the predictable and
acquisition of the conditioned emotional response unpredictable juice reward.
(Rodrigues, Schafe, & LeDoux, 2001). Potentiation The prediction error notion is important enough
of excitatory synapses of the ventral tegmental area to examine in more detail. A study by Fiorillo,
accompanied cocaine-reinforced and food-reinforced Tobler, and Schultz (2003) revealed the issues
behavior in rats (Chen et al., 2008); potentiation did nicely. Two monkeys were trained in a Pavlovian
not occur in rats that received response-independent conditioning procedure in which different stimuli
cocaine. Moreover, although the potentiation per- were followed by juice after a 2-second delay
sisted for 3 months in rats that received cocaine according to different probabilities (ranging from
reinforcers, it had faded by 21 days in rats receiving p = 0 to p = 1.0). When training was complete,
food or sucrose reinforcers. The point of this brief licking during the 2-second delay (the conditional
survey is that these mechanisms of neural plasticity response to the CS) was an increasing function of
are ubiquitous and are likely involved in most last- the reward probability programmed to each stimu-
ing behavior change wrought by experience. lus. Fiorillo et al. simultaneously recorded the activ-
ity of dopamine neurons in areas A8 through A10 of
Reinforcement Signals the ventral midbrain. As shown in Figure 15.2c, the
Mechanisms underlying neural plasticity may be firing of these neurons was a continuous, decreasing
engaged when reinforcing events occur. A consensus function of reinforcer probability. When the reward
341
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
Task acqusition Free liquid
10
–0.5 0 0 0.8 –0.5 0 0.8 s

Sound Liquid reward Liquid
Established task Free liquid
–0.5 0 0 0.8 –0.5 0 0.8 s

Sound Liquid reward Liquid
Figure 15.1. Responses of dopamine neurons in monkeys to liquid delivery at

different periods of task performance. Top: Note the strong responses after liquid
delivery during learning of the reaction time task (left) and after free liquid (right).
Bottom: Bursts of responses of a neuron after acquisition occur predominantly after
the conditioned stimulus that precedes liquid delivery, but not to liquid delivery
itself (left). When free liquid, unpredicted by a prior conditioned stimulus, is given
again, a burst of responding is observed (right). Both recorded neurons were in
group A9 of the ventroanterior midbrain. From “Importance of Unpredictability
for Reward Responses in Primate Dopamine Neurons,” by J. Mirenowicz and
W. Schultz, 1994, Journal of Neurophysiology, 72, p. 1025. Copyright 1994 by the
American Physiology Society. Reprinted with permission.
was delivered after the stimulus that predicted the a ctivity during the 2-second delay between CS and
reward with a probability of 1.0, neuronal firing juice were both an increasing function of reward
remained at baseline. However, when a reward was magnitude. When different magnitudes were pre-
delivered after the stimulus signaling a reward prob- sented alone (i.e., not preceded by a CS), dopamine
ability of 0.0, then intense neuronal activation activity was also an increasing function of magnitude.
ensued. As shown in Figure 15.2d, when juice was Neuron firing in response to rewards of different
predicted but omitted after the stimulus, dopamine magnitudes depended on the magnitude of the pre-
neuron firing was suppressed to a greater extent dicted reward; in the presence of a stimulus predict-
when the probability of reward was higher. Finally, ing a medium-sized (0.15-milliliter) reward, a larger
dopamine neuron firing in the 2 seconds after CS reward (0.5 milliliter) increased dopamine neuron
presentation was an increasing function of reward firing and a smaller (0.05 milliliter) reward decreased
probability. Fiorillo et al. concluded, “By always it. A strong context effect was also demonstrated; in
coding prediction error over the full range of proba- an experiment in which one CS was presented before
bilities, dopamine neurons could provide a teaching either the small- or the medium-magnitude juice
signal in accord with the principles of learning origi- reward and a different CS was presented before either
nally described by Rescorla and Wagner” (p. 1901). the medium or the large reward, the medium reward
In a set of experiments using the same basic meth- elicited high rates of firing in the first context and
odology, Tobler, Fiorillo, and Schultz (2005) showed suppressed firing in the second. Finally, the same rel-
in monkeys that licking and dopamine neuron ative difference in dopamine firing occurred when
342
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 15.2. Dependence of phasic neuron responses on reward prob-

ability. A: Raster plots and histograms of single-cell firing to conditioned
stimuli and rewards as a function of reward probabilities. C: Neuron fir-
ing rate after reward presentation as a function of reward probability. D:
Suppression of firing rate owing to reward omission. E: Neuron firing rates
during the prereward CS. From “Discrete Coding of Reward Probability and
Uncertainty by Dopamine Neurons,” by C. D. Fiorillo, P. N. Tobler, and
W. Schultz, 2003, Science, 299, p. 1899. Copyright 2003 by the American
Association for the Advancement of Science. Reprinted with permission.
343
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
absolute reinforcer magnitudes were varied over a

10-fold range, again showing that neuronal responses
were correlated with relative differences in reinforcer
magnitude in a given context.
Reward- and CS-correlated activity has been
detected in the orbitofrontal cortex that varies as a
function of the preference between reinforcers as indi-
cated in an animal’s choice (Tremblay & Schultz,
1999). Several other brain areas, such as the amygdala
and orbitofrontal cortex (Schultz, Tremblay, & Holl-
erman, 2000), include neurons that have specific
responses to reward and reward-predictive events.
Thus, in a wide array of tasks involving different stim-
uli and stimulus modalities and reinforcer qualities,
magnitudes, and probabilities, large and diffusely dis-
tributed dopamine teaching signals (the magnitude of
which are prediction-error dependent) are available to
update and augment the configurations of neuronal
groups that participate in learning and behavior.
Although the prediction error concept is a pow-
erful one, its generality has limits. For example, the
magnitude of the dopamine response in the nucleus
accumbens to drug administration may be larger
when it is expected (i.e., given in a drug-associated
context or when it is self-administered rather than
given response independently). Volkow et al. (2003)
used functional scanning with 18F-deoxyglucose
positron-emitted tomography to study the psycho-
motor stimulant drug methylphenidate’s effects on
active cocaine users. Subjects were scanned on 4
separate days. Before each scan, subjects were told
that they would receive either active drug or saline,
Figure 15.3. Enhancement of
and then they received either drug or saline. Thus, volume of activation (number of
drug or saline expectation and drug or saline receipt voxels) by methylphenidate in
were tested in a 2 × 2 design. Subjects completed functional magnetic resonance
imaging in humans, both when
visual analog scales to rate levels of high, drug methylphenidate was expected
effects, alertness, and restlessness before and at reg- (methylphenidate–methylphenidate
ular time points after injection. On days on which [MP/MP] condition) and when it
subjects expected and received drugs, self-reports of was not (saline–methylphenidate
[PL/MP] condition). From
drug effects were increased (about 50% more high) “Expectation Enhances the Regional
relative to days when drugs were received after Brain Metabolic and the Reinforcing
saline instructions. Likewise, activation was greater Effects of Stimulants in Cocaine
Abusers,” by N. D. Volkow, G. J.
in the thalamus and cerebellar vermis when drug
Wang, Y. Ma, J. S. Fowler, W. Zhu,
was expected (methyphenidate–methylphenidate L. Maynard, . . . J. M. Swanson,
condition in Figure 15.3) than when it was not 2003, Journal of Neuroscience, 23,
(saline–methylphenidate condition). Unexpected p. 11465. Copyright 2003 by the
Society for Neuroscience. Reprinted
methylphenidate produced larger metabolic effects with permission.
344
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
in the orbitofrontal cortex (bottom panel of Figure allow synaptic plasticity and event-correlated neural
15.3), however, which suggests that prediction activity (e.g., protein synthesis involved in the long-
errors for drug reinforcers stimulate activity in areas term maintenance of new learning). The behavioral
distinct from other reinforcers. consequences of such molecular events are vital to
Responsiveness to reinforcers is not a fixed prop- the investigation of these cellular processes, and so
erty of neurons. Studies have shown that neurons the interested behavior analyst is certainly in a posi-
that are activated by reinforcers are not fixed in their tion to contribute to such cellular-level research. In
sensitivity to reinforcing events (Paton, Belova, Mor- addition, an enormous amount of empirical and
rison, & Salzman, 2006; Salzman, Paton, Belova, & conceptual work is still required to arrive at a con-
Morrison, 2007); flexibility to adjust to changing ception of these interactive neural units that makes
environments and contingencies has been demon- unambiguous contact with the evolving observable
strated. Paton et al. (2006) trained monkeys to lick activities of intact animals; loosely stated, research-
in anticipation of juice presented 1.5 seconds after ers still do not know how brain events and behav-
one visual stimulus and to blink in anticipation of ioral events are connected. This fact, however, does
an airpuff presented after a second, distinct visual not preclude the investigation of neural participants
stimulus; after a third visual stimulus, nothing in larger, more complex behavioral contexts that
happened. They learned these responses easily. have been of interest to behavior analysts.
Neurons in the amygdala were recorded during
these tasks. Several patterns of neuronal responses
Neural Participants in Operant
were obtained; some neurons responded whenever
Behavior
a particular stimulus was presented, regardless
of its relation to upcoming events, and some The basic neural model of learning is one in which
responded differently depending on which event reinforcement strengthens distributed synaptic con-
was delivered (juice vs. airpuff). Paton et al. then nections provided certain criteria are met (e.g., there
reversed the relations so that the stimulus that is a prediction error). Even if that model and its cel-
had signaled upcoming juice now signaled upcom- lular mechanisms were completely elucidated,
ing airpuff and vice versa. The responses of many research would be required to discover how the
neurons “switched”—that is, after responding model helps researchers account for the ongoing
differentially to the juice-correlated stimulus, behavior of intact animals in diverse contexts. Some
they responded, after a transition, to the airpuff- of this research is translational, that is, it is stimu-
correlated stimulus. This shift in neuronal respond- lated by the desire to discover why people have psy-
ing preceded changes in behavior (licking and chological or behavioral problems and how to
blinking) caused by the new stimulus–outcome rela- overcome them. Other research addresses issues that
tions. Thus, flexibility in neuronal responding when seem to be pulled straight from the pages of the
contingencies change has been demonstrated, a fact Journal of the Experimental Analysis of Behavior. I
that is relevant to the evolution of larger neuronal briefly review examples of this research here.
systems that participate in behavior in a changing
setting (Edelman, 2003). Response Rates on Reinforcement
In summary, basic synaptic plasticity mecha- Schedules
nisms working in a context in which widely distrib- From a behavior analyst’s perspective, there are two
uted reinforcer-correlated neurotransmitter release broad approaches to accounting for differential rates
occurs allows multiple aspects of the stimulus set- of responding on different reinforcement schedules.
ting to be connected with multiple dimensions of These approaches may be characterized as molecu-
ongoing behavior to allow acquisition, maintenance, lar, focusing on the likely characteristics of the
and evolution of useful behavioral repertoires. Not instances of behavior that produce the reinforcer
reviewed in this chapter are the volumes of research (Peele, Casey, & Silberberg, 1984), and molar,
on the cellular and subcellular mechanisms that focusing on the overall correlation between rates of
345
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
behavior and rates of reinforcement (Herrnstein, response rate on a reinforcement schedule would
1970). The molecular view, it would seem, has not reflect the value of responding represented by the
yet made an appearance in behavioral neuroscience critic and rules for responding based on that value
(although see Corrado, Sugrue, Seung, & Newsome, represented by the actor.
2005, for molecular contributions to shifts in Niv (2007) proposed a distinct model of
choice behavior when relative reinforcement rates response rates on reinforcement schedules. Accord-
change frequently). Instead, attempts to account ing to this model, an animal responding on a free-
for response rates on reinforcement schedules are operant reinforcement schedule faces two decisions:
framed in terms of choice of a response rate on the which response to make and how fast to make it.
basis of its expected value, a decidedly molar Both depend on the expected probability and quality
approach. of the reinforcer, or its benefit, and the cost of mak-
Two extant models begin with learning based on ing the response quickly versus slowly, where a slow
prediction errors. One is called the actor–critic response is less effortful but more costly in terms
model. The critic component of the model, thought of delaying the next and all future reinforcers (the
to be located in the ventral striatum, uses phasic opportunity cost of time). As in the actor–critic
dopamine release caused by prediction errors to model, phasic inputs from ventral and dorsal stria-
update the reward-predictive value of environmental tum update current estimates of reinforcer probabil-
stimuli. The critic is hypothesized to be active in ity. However, Niv’s model also requires neural
both Pavlovian and operant procedures because activity that is correlated with the opportunity cost
both involve valuation. The actor is thought to be of time (which is directly correlated with overall
localized in the dorsal striatum and also uses an reinforcer rate). Niv proposed that the tonic level of
error-prediction signal to modify stimulus–response dopamine in basal ganglia and prefrontal areas,
or stimulus–response–reward associations that which reflects accumulated phasic dopamine inputs
guide behavioral choices (O’Doherty et al., 2004). from striatal areas, would serve as such a signal.
The difference between the two was demonstrated Increases in tonic dopamine level increase the
in humans by arranging a Pavlovian procedure that opportunity cost of time and thus promote faster
engages the ventral striatal critic component with- responding; the opposite effect follows reductions in
out engaging the dorsal striatal actor, alternating the tonic dopamine level. Tonic dopamine levels
with an operant task (which requires both critic and may be increased by increasing reinforcement rates,
actor; O’Doherty et al., 2004). On operant trials, because more rapid phasic firing of striatal dopa-
humans chose between stimuli correlated with low mine cells leads to accumulations of tonic dopamine
(0.3) or high (0.6) probability of juice; on other tri- in basal ganglia and prefrontal areas, or by artificial
als, they chose between two stimuli that predicted means, as when dopamine-enhancing drugs (e.g.,
different probabilities of a tasteless solution, referred amphetamine) are administered. Increasing food
to as neutral on the basis of ratings of pleasantness deprivation will drive up the net benefit, and hence
near zero. On Pavlovian trials, there was no choice: the opportunity cost of time, and thereby promote
Stimulus–juice (or stimulus–tasteless solution) pair- more rapid responding. Niv’s model reproduced sev-
ings were presented, and subjects simply indicated eral aspects of free-operant responding, including
which stimulus had been presented. Response laten- response rate differences on variable-ratio and yoked
cies on the Pavlovian trials were shorter for stimuli variable-interval schedules and matching on concur-
that predicted juice than for stimuli that predicted rently available schedules.
the tasteless solution. On the operant trials, subjects A behavior analyst might object to models of
more frequently chose the stimuli that predicted the behavior that postulate an inner actor and critic,
higher reinforcer probability. As predicted, ventral but these words are entirely superfluous to the
striatum MRI activity was engaged by both tasks, explanations of which they are a part. Discarding
but only the operant one increased activity in the these terms, the model is just a two-factor theory of
dorsal striatum. According to the actor–critic model, response rates on reinforcement schedules (or three
346
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
factors in Niv’s 2007 model); each factor corre- 2004, 2006). Control rats responded at the same
sponds to different aspects (or time frames) of the rate the day after taste aversion or omission training
animal’s experimental experience, and each is tenta- as they did the day before; this insensitivity is the
tively associated with the activities of neurons in dif- hallmark of stimulus-bound behavior under con-
ferent regions of the brain. So, as seems to be the trol of the dorsolateral striatum. Rats with chemi-
case with all such cognitive terms, once the behav- cal lesions of the dorsolateral striatum, however,
ioral relations and brain participants are known, the responded at low rates the next day, thereby demon-
cognitive terms become mere labels (Schaal, 2003). strating enhanced control by the dorsomedial
From a behavior-analytic perspective, what may be striatum. Although the model may account for dif-
of more concern than the cognitive labels is that ferential alterations in response rates under some
studies such as these tend to involve a very small conditions, this model does not make specific pre-
range of reinforcement parameters and thereby risk dictions about response rates on reinforcement
building an elaborate explanatory model of phenom- schedules.
ena that may occur under limited circumstances.
Behavioral neuroscientists have also differenti- Matching Law
ated two putatively distinct types of operant (instru- Groundbreaking work on neural mechanisms
mental) learning that lead to reinforcement underlying matching (Herrnstein, 1961) has been
schedule–maintained behavior, referred to as goal- conducted by William Newsome and his graduate
directed versus stimulus-bound learning (Balleine, students, in which rhesus monkeys match relative
Delgado, & Hikosaka, 2007). Goal-directed behav- rates of saccadic eye movements to relative rate of
ior is sensitive to manipulations of reinforcer value juice delivery. In one study (Sugrue, Corrado, &
or motivation, whereas stimulus-bound behavior is Newsome, 2004), monkeys initially fixated their
relatively insensitive to these and is often referred to gaze on a cross in the center of the screen (see Fig-
as habitual. Goal-directed behavior is controlled in ure 15.4A). Color stimuli were then presented to
the rodent by the dorsomedial striatum and fol- the left and right of the cross (color positions were
lows prediction–error–correction learning rules; randomized across trials) while the monkey con-
stimulus-bound behavior is controlled by the dorso- tinued to fixate on the cross. When the cross
lateral striatum and results from training well dimmed, a saccadic response occurred and was
beyond the point at which responding is stable. reinforced at a maximum overall rate of 0.15 rein-
Stimulus-bound behavior is exemplified by forcers per second. Relative reinforcement rates
schedule-maintained behavior that is insensitive to changed unpredictably during each session (ratios
changes in reinforcer value or contingency manipu- from 8:1 to 1:1 were tested). Relative response rate
lations. If activity of the dorsolateral striatum con- tracked the changed reinforcer ratios very rapidly
trols the stimulus-bound behavior, reducing its (see Figure 15.4), suggesting to Sugrue et al.
influence (e.g., by a reversible chemical lesion) (2004) a local matching rule in which a leaky inte-
should render the behavior more susceptible to grator allows temporally distant reinforcer distri-
changes in reinforcer value or contingency manipu- butions to drop out and current behavior to reflect
lations, that is, make the behavior less stimulus the most recent reinforcer distribution. They
bound and more goal directed. recorded from neurons in the lateral intraparietal
These ideas are illustrated in a series of studies in area that were active during movements to one or
which groups of rats work for sucrose according to the other side of the screen. Large lateral intrapari-
variable-interval schedules, then experience a etal area responses before choice were correlated
manipulation intended to reduce the value of the with a high probability of reinforcement for that
reinforcer (e.g., taste aversion) or degrade the choice, and small lateral intraparietal area
response–reinforcer contingency (e.g., omission responses were correlated with a low probability
training, in which rats must withhold responding of reinforcement for that choice. Although Sugrue
to receive a reinforcer; Yin, Knowlton, & Balleine, et al. noted that lateral intraparietal area neuron
347
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
Figure 15.4. A: The sequence of events of an oculomotor matching task.

Reward is delivered at the time of the go response if it is scheduled. Overall
maximum reinforcer rate was 0.15 per second, and relative reinforcer rate
varied unpredictably from the set {8:1, 6:1, 3:1, 1:1}. B: Dynamic, within-
session matching behavior. The gray curve shows cumulative choices of
the red and green targets; black lines show the average ratio of reinforcers
(red:green) within each block. Approximate matching was obtained. From
“Matching Behavior and the Representation of Value in the Parietal Cortex,”
by L. P. Sugrue, G. S. Corrado, and W. T. Newsome, 2004, Science, 304,
p. 1783. Copyright 2004 by the American Association for the Advancement
of Science. Adapted with permission.
activity is more likely responsive to reinforcer rate– shown that choice is amenable to neurobiological
correlated activity initiated elsewhere (as opposed investigation.
to being the neural signal of relative reinforcer An interesting benefit of these studies is that
value), these findings clearly show brain activity in when researchers from outside experimental psy-
real time correlated with choice on concurrent chology confront issues as familiar to behavior
reinforcement schedules. analysts as matching in concurrent schedules, meth-
Using similar methodology, Lau and Glimcher odological and theoretical issues seem to combine
(2005, 2008) recorded from phasically active neu- to generate interesting and novel analyses and inter-
rons in the oculomotor caudate of monkeys. They pretations of behavior. For example, the linear–
found neurons whose firing was correlated with nonlinear–probabilistic model of Corrado et al.
upcoming outcomes (action values) and those (2005) fits moment-to-moment choice behavior
whose firing was correlated with the outcome just well and yields model parameters that may be more
received (chosen values). In fact, 62% of task- clearly related to both the behavioral and the neural
related phasically active neurons covaried signifi- mechanisms of matching than the molar response
cantly with action-value or chosen-value outcomes. and reinforcer rates used to frame the original
Phasically active neurons’ firing also correlated with matching law (Herrnstein, 1961).
more local, trial-by-trial estimates of value. Other Similarly, the blocking of dopamine in rats
researchers (Kalenscher, Diekamp, & Gunturkun, responding for concurrently available reinforcers
2003; Padoa-Schioppa & Assad, 2006) have similarly that differ both in value and in the effort required to
348
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
earn them have repeatedly shown that effects are functional MRI to investigate whether different
more clearly related to response effort than to rein- brain areas are active in a choice procedure using
forcer value (Salamone, Correa, Farrar, Nunes, & juice or water rewards that varied in magnitude (the
Pardo, 2009). For example, Salamone et al. (1991) availability of which were signaled during choice
studied rats whose lever pressing was reinforced on a periods) among 34 humans. Choices were observed
fixed-ratio-5 schedule, in which every five responses in three separate delay contexts. In one, immediate
produced preferred pellet reinforcers in a setting reinforcement was tested against 1- or 5-minute
in which nonpreferred chow was freely available. delayed reinforcement. This context was compared
Low doses of the dopamine antagonist haloperidol with two contexts in which the relatively short delay
decreased lever pressing and increased feeding on lab was much longer (10 minutes vs. 11 or 15 minutes
chow, without altering overall food intake or prefer- in one context and 20 minutes vs. 21 or 25 minutes
ence for the reinforcer pellets in a free-access situa- in another context). Subjects’ preference for the
tion. These and many complementary findings led immediate reinforcer over the 1- and 5-minute
Salamone et al. (2009) to suggest that dopamine delayed reinforcer was much greater than their pref-
interacts with the response bias aspect of the match- erence for the shorter delayed reinforcer in the
ing law rather than with reinforcer value. It remains other two contexts. Activity in the (limbic reward)
to be seen whether continued investigation of the system was greatest with all immediate reinforcers;
neuroscience of choice behavior will, as Glimcher it decayed rapidly as both rewards were delayed.
and Rustichini (2004) suggested, result in an under- The lateral prefrontal cortex and posterior parietal
standing of value “as a concrete object, a neural sig- cortex responded similarly regardless of the dura-
nal in the human and animal brain, rather than as a tion of the delays.
purely theoretical construction” (p. 452). It is excit- This disjunction between what McClure et al.
ing, though, to see choice, the matching law, and (2007) called an impatient limbic reward system
behavioral economics under active investigation by and the more deliberate, longer term prefrontal
behavioral neuroscientists. and parietal system was revealed in an earlier study
by this group using choices involving money
Delay Discounting (McClure, Laibson, Loewenstein, & Cohen, 2004).
The concept of delay discounting has contributed This and similar studies have shown that immedi-
greatly to the understanding of human behavioral ate reinforcers might be viewed as qualitatively dif-
disorders such as drug abuse (see reviews by Dalley, ferent from delayed ones, at least in the sense that
Mar, Economidou, & Robbins, 2008; de Wit, 2009; the limbic system is engaged by the former, which
Perry & Carroll, 2008) and gambling (Petry & suggests a finding that may alter behavioral models
Madden, 2010), and it has facilitated a quantitative of delay discounting. In the absence of neurobio-
understanding of the value of delayed reinforcers logical data suggesting a qualitatively distinct func-
(Mazur, 1987; Mazur & Biondi, 2009). The concept tion of immediate reinforcers, a single model in
seems to have resonated with behavioral neuroscien- which reinforcers are discounted very steeply in
tists as well. the short-delay range may appear adequate, which
For example, McClure, Ericson, Laibson, begs the question of whether and to what extent
Loewenstein, and Cohen (2007) tested a two-part behavioral theorizing will need to be updated to
discounting model that consists of an exponential accommodate neurobiological findings. McClure
discount function (associated with activity in the et al.’s (2004, 2007) two-system approach, how-
prefrontal and parietal cortex, regions implicated ever, may not account for more of the variance at
in planning and deliberation) and a function that the behavioral level than a more conventional
is hypersensitive to immediate reinforcement exponential delay-discounting function. The ques-
(associated with activity in the mesolimbic dopa- tion of whether behavioral models need to be
mine system, or limbic reward areas). They used revised in the light of neurobiological data will, no
349
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
doubt, be faced repeatedly as behavioral neurosci- status, an intrinsic motivational or rewarding

ence advances. value of its own,” suggested Bromberg-Martin and
Hikosaka (p. 122). Given the findings of Dinsmoor
Reinforcement of Observing Behavior by et al. (1972), an interesting experiment would
Informative Stimuli examine the behavioral and neural outcomes of
Observing is another interesting and long-studied contingencies in which only the negative informa-
behavioral phenomenon recently shown to have tive stimulus was presented.
clear neurobiological correlates. Bromberg-Martin
and Hikosaka (2009) trained two macaques on an
Translational Behavioral
eye movement procedure similar to that of Sugrue
Neuroscience
et al. (2004), but in the context of an observing
response procedure (Dinsmoor, Browne, & Law- So far I have examined the neural participants in
rence, 1972; Wyckoff, 1952). Monkeys chose several specific environment–behavior relations that
between two options at trial onset and after a few have in common the central role of reinforcement.
seconds received either a large or a small water The reinforcement relation is so ubiquitous in
reinforcer. Choice of one option produced stimuli behavior that this collection of behavioral and neu-
that were predictive of upcoming reinforcer mag- ral variables is likely to appear in a wide variety of
nitude, and choice of the other option produced neurobiological investigations. Nevertheless, behav-
stimuli that were uncorrelated with the upcoming ioral neuroscience is a large field with very fuzzy
reinforcer; reinforcer probability at the end of the boundaries and with ample opportunities for behav-
trials was the same whether monkeys chose the ior analysts to make contributions. In this section, I
informative or the uninformative option. Both survey just a few such opportunities. In no sense are
monkeys strongly preferred the option that led to these brief reviews intended to be exhaustive; I
informative stimuli. In a second task, monkeys describe them here only to give the reader an idea of
chose between two informative options, but one what roles behavior analysts may play, in the labora-
presented the reinforcer-magnitude signal immedi- tory and the clinic, in the future development of
ately, whereas the other presented it nearer in time what may be considered translational behavioral
to water delivery. Both monkeys preferred the neuroscience.
immediate informative stimulus. Preference also
tracked the immediate informative stimulus after a Hippocampal Neurogenesis
color reversal. Real-time recording from midbrain New neurons proliferate and differentiate (i.e.,
dopamine neurons showed phasic excitation by neurogenesis occurs) in the adult hippocampus of
the stimulus that preceded a large reward and inhi- rats and humans, a fact that distinguishes the hippo-
bition by the stimulus that preceded a small campus from most brain areas, in which neurogene-
reward. When monkeys were, in forced trials, sis typically does not occur (see review by Ehninger
required to view uninformative random stimuli, & Kempermann, 2008). Excitatory granule cells
differential neuronal activity occurred only when proliferate from the subgranular zone of the hippo-
the water reinforcers were presented. Moreover, campus and populate the dentate gyrus, form syn-
neurons were excited when a stimulus was pre- apses, and participate in new learning. A fascinating
sented indicating the information would be avail- aspect of hippocampal neurogenesis is its capacity to
able on a trial, but not when a stimulus was be regulated by experience; it is enhanced by envi-
presented indicating that random stimuli would be ronmental enrichment (Brown et al., 2003; Kemper-
available. Thus, the behavioral preference for mann, Gast, & Gage, 2002), physical exercise (van
informative stimuli was correlated with the neural Praag, 2008; van Praag, Kempermann, & Gage,
response to opportunities to receive information 1999), and certain kinds of learning (Gould, Beylin,
about upcoming reinforcers. “These observations Tanapat, Reeves, & Shors, 1999) but is suppressed
suggest that the act of prediction has a special by stress (Gould & Tanapat, 1999).
350
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
In a study by Gould et al. (1999), for example, evidence has allowed investigation of processes in
rats were trained in either delay or trace eyeblink the brain that participate in these effects (e.g., dopa-
conditioning. Trace, but not delay, conditioning is mine release in the nucleus accumbens coincident
typically disrupted by hippocampal lesions and with drug-reinforced behavior; Di Chiara, 1995).
causes timed neuron firing and structural changes in In an example of a behavior-analytic contribu-
dendritic spines in neurons of the hippocampus; tion to research on the neurobiology of drug abuse,
thus, trace conditioning is hippocampal dependent a counterintuitive finding helped elucidate the
and delay conditioning is not. Bromodeoxyuridine critical role of contingencies of reinforcement in
(which is taken up by cells undergoing division and determining a drug’s effect. Hemby, Martin, Co,
can be stained later so that newborn cells can be Dworkin, and Smith (1995) found that extracellular
counted) was administered at the onset of training. dopamine concentration was increased in the
A huge increase in the number of new neurons was nucleus accumbens of rats when heroin was admin-
observed in the granule cell layer of the hippocam- istered response independently. When rats pressed a
pus in rats that were trained on trace procedure lever for response-dependent infusions of heroin,
but not on the hippocampal-independent tasks. Of however, nucleus accumbens concentrations of
course, in this case the neurogenesis could not really dopamine did not rise, which suggests limits on
be implicated in the actual learning—it was caused nucleus accumbens involvement in sustained heroin
by it, but neurons were too new, too immature, to self-administration.
be integrated into functioning neural groups that Subsequent research has enhanced the under-
participated in this learning. It is possible, however, standing of the functional role of this region in
that experiences such as trace conditioning, exer- drug taking and relapse. For example, Alvarez-
cise, environmental enrichment, and so forth all Jaimes, Polis, and Parsons (2008) trained rats to
enhance hippocampal neurogenesis to prepare self-administer heroin on a multiple schedule; in
animals, in some sense, for future learning. As Kem- one context, lever-pressing led to heroin infusion,
permann (2002) put it, “Hippocampal information- and in a distinct context it led to saline infusion
processing capacity is optimized for future memory (extinction). After discriminative performance
storage” (p. 247). Regardless of the specific function was established, heroin-reinforced responding
of hippocampal neurogenesis, it represents a more was extinguished in the absence of the multiple-
global sense in which brain structure and function is schedule stimuli. Reinstatement (de Wit & Stewart,
altered by experience. The relevance of findings 1981; Epstein, Preston, Stewart, & Shaham, 2006)
such as these to issues faced by aging people, who was indicated by an increase in lever pressing when
show decreases in hippocampal neurogenesis (Jess- the drug context was reintroduced during sessions
berger & Gage, 2008), and people with progressive in which presses did not produce drug. Pretest infu-
diseases of memory such as Alzheimer’s disease sion of a cannabinoid receptor antagonist into the
(Steiner, Wolf, & Kempermann, 2006) are under nucleus accumbens (and the prefrontal cortex)
investigation. dose dependently suppressed reinstatement (canna-
binoid receptor antagonists had been shown to
Drug Abuse interact with multiple drugs of abuse). Current
Behavior analysis has already proved important to understanding of drug-maintained behavior and
the neuroscience of drug abuse, in which the identi- drug relapse continues to have central roles for the
fication of abused drugs as unconditioned stimuli, nucleus accumbens and prefrontal cortex as well as
reinforcers, and discriminative stimuli has set the for differential functions of dopamine and the excit-
stage for neurobiological investigation (Lejuez, atory neurotransmitter glutamate in all phases of
Schaal, & O’Donnell, 1998). For example, behavior drug reinforcement and relapse (Kalivas & Volkow,
analysts provided some of the first unambiguous 2005). Such models are ultimately examined rela-
evidence of the reinforcing effects of abused drugs, tive to the ability of drugs of abuse to participate in
such as cocaine (Pickens & Thompson, 1968). This operant or Pavlovian relations and thus support a
351
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
critical role for behavior analysis in the neuro factor levels and neurogenesis were significantly
science of drug abuse. enhanced relative to nonrestrained rats. Thus, in
this animal model an approach to poststroke therapy
Stroke was found to be effective and to enhance stroke-
Stroke is the leading cause of long-term disability induced plasticity.
and the third leading cause of death in the United CIMT was recently subjected to a randomized
States (American Heart Association, 2010). It is controlled trial (Wolf et al., 2006). Patients who
caused in most cases when an embolism lodges in had experienced stroke 3 to 9 months earlier
a blood vessel of the brain (i.e., most strokes are received either customary care or a 2-week course
ischemic strokes). Patients who survive a stroke of CIMT (which involved wearing a restraining mitt
often experience several neurological disorders the on the less-affected hand and repetitive task prac-
nature and severity of which depend on the extent tice and behavioral shaping with the hemiplegic
and location of the stroke damage (Dirnagl, Iadec- hand for as many as 6 hours per day). Structured
ola, & Moskowitz, 1999). Although volumes of motor tasks and interviews showed that CIMT sig-
research have been published on the cascade of cel- nificantly improved function, and most of these
lular events initiated by ischemia and leading to improvements were maintained at the 12-month
stroke damage, and manipulations of some of these follow-up. Thus, stroke is currently viewed as cre-
events can reduce the size of experimentally induced ating both frank neural damage and a neuroplastic
strokes in the laboratory (e.g., see H. Zhao et al., environment around the ischemic damage that can
2005), none of this work has resulted in therapies be exploited by training experiences. The role of
to limit stroke damage if the embolism is not quickly behavioral contingencies in the neurobehavioral
eliminated. Thus, both laboratory and clinical interaction arranged by CIMT is widely appreci-
research aimed at recovery from stroke are ated. As stated by Taub, Uswatte, and Morris
important. (2003),
Behavioral research on stroke in the laboratory
has primarily involved untrained reflexive or explor- First . . . CI [constraint-induced] therapy
atory behavior and behavior in simple training changes the contingencies of reinforce-
conditions. This research can result in interesting ment (provides opportunities for rein-
insights; for example, C. Zhao, Wang, Zhao, and forcement of use of the more-affected
Nie (2009) studied rats with experimental lesions arm and aversive consequences for its
caused by middle cerebral artery occlusion, which nonuse by training procedures and by
causes a unilateral cortical lesion (and deficits in constraining the less-affected UE [upper
forelimb function on the contralateral side). Some of extremity]), so that the learned non-
the rats were fitted with a plaster cast that prevented use of the more-affected UE learned in
use of the ipsilateral limb; thus, these rats received the acute and early subacute periods is
a kind of constraint-induced movement therapy counter-conditioned or lifted. Second,
(CIMT; Taub et al., 1993). After 2 weeks of training, the consequent increase in use, involving
rats were tested on a beam-walking task and a water- sustained and repeated practice of func-
maze task; they were then anesthetized and their tional arm movements induces expansion
brains were removed to assess changes in neurogen- of the contralateral cortical area control-
esis and levels of stromal-cell–derived factor, a pro- ling movement of the more-affected UE
tein involved in signaling migration of cells toward and recruitment of new ipsilateral areas.
stroke damage. Stroke caused slipping off the beam This use-dependent cortical reorganiza-
in untreated rats relative to rats without stroke; slip- tion may serve as the neural basis for the
ping was significantly reduced in limb-restrained permanent increase in use of the affected
rats. Water maze performance also improved in arm as well as other beneficial effects of
limb-restrained rats, and both stromal cell–derived CI therapy. (p. S82)
352
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Autism activity. I described these findings without question-

The contribution of behavior analysis to the under- ing, in any rigorous way, how the brain events
standing and treatment of people with autism has, by explain the observed behavior. I avoided this
any measure, been substantial (Lovaas, Koegel, & because my goal was to describe behavioral neuro-
Schreibman, 1979; McEachin, Smith, & Lovaas, science as it is practiced, so readers could get an idea
1993). The connection between autism as a neural of the kinds of questions behavioral neuroscientists
disease and autism as a behavioral disorder has been ask and how they answer them. For a behavior ana-
much more difficult to establish. Recently, Thomp- lyst, however, each finding begs the question, “How
son (2007) described the neurobiological mecha- does this brain event produce this behavior?” More
nisms by which intensive early behavior therapy may broadly, one may ask, “What does the brain do?”
overcome the behavioral deficits of children with The theoretical language of prominent neuroscien-
autism (Eldevik et al., 2009; Howlin, Magiati, & tists suggests that they believe that the brain does most
Charman, 2009; Schreibman, 2000). Some children of the things that whole animals do. Consider Francis
exposed to intensive early behavior therapy are unre- Crick’s (1995) views on the neuroscience of vision:
sponsive, and others show significant gains in func-
What you see is not what is really there;
tion (Howlin et al., 2009). Thompson suggested that
it is what your brain believes is there. . . .
the relative failure of intensive early behavior ther-
Your brain makes the best interpretation
apy in some cases may reflect different degrees or
it can according to its previous experi-
natures of neuronal pathology in these children.
ence and the limited and ambiguous
Many neuroanatomical differences between brains of
information provided by your eyes . . .
people with autism and those of typically developing
the brain combines the information pro-
people have been discovered (for a review, see Bau-
vided by the many distinct features of the
man & Kemper, 2005). Hutsler and Zhang (2010),
visual scene . . . and settles on the most
for example, recently examined dendritic spine
plausible interpretation of all these vari-
densities on cortical pyramidal cells in brains of
ous clues taken together. (p. 30)
deceased people with autism and age-matched con-
trol subjects. Elevated spine densities were found In this quote, the brain believes, combines informa-
in all subjects relative to their age-matched control tion, and makes interpretations. In cognitive neuro-
and were greatest in layer II of the frontal, parietal, science, it is not difficult to find references to brains
and temporal cortical lobes, a layer in which most believing, interpreting, constructing symbolic
connectivity is established postnatally. Because descriptions, thinking, responding to the content of
experience-dependent culling of cortical dendritic thought, acquiring and encoding information, stor-
spines occurs during normal development, increased ing and retrieving memories, and so on. Animals are
spine densities in people with autism may suggest a thought to learn because the brain learns and to
failure of this developmental mechanism. These data choose because the brain chooses. Bennett and
do not allow one to determine when in the life of the Hacker (2003) referred to this practice of “ascribing
person with autism such structural differences arose psychological attributes to the brain and its parts in
or whether and to what extent they played a role in order to explain the possession of psychological
the person’s disability. Nevertheless, they do point to attributes . . . by human beings” (p. 3) as commit-
a structural alteration in a key component of cortical ting the mereological fallacy. They argued that psy-
neurons that could be involved in social and learning chological terms, which make sense when applied to
disabilities of people with autism. whole animals, usually result in nonsense when
applied to the brain.
Bechtel (2005) argued that a science of a com-
What Does the Brain Do?
plex phenomenon is successful when scientists are
The articles reviewed in this chapter mostly con- able to name the component parts and describe the
cerned correlations between behavior and brain operations of the mechanisms of the phenomenon.
353
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
Until then, they are apt to refer to these mechanistic mechanisms and processes in the nervous system
components using terms that apply to the phenome- that allow these functional relations to be realized.
non as a whole. He used as an example the history This is a sort of mediationistic approach, so at
of the science of fermentation. Before the discovery least tentatively I will assert that brains mediate
of the operations of the mechanisms of fermenta- environment–behavior relations. Furthermore, they
tion, that is, additions or deletions of groups of do so over multiple time scales, from moment to
molecules to the carbon backbone (deamination, moment to cumulatively over an animal’s lifetime.
carboxylation, phosphorylation, etc.), scientists But mediate is a weak, nonspecific term, and to be
“invoked the vocabulary designed to explain the satisfied with it as a description of what the brain
overall behavior to describe the operation of its does is to ignore what I have called
components” (Bechtel, 2005, p. 318). For example,
the fundamental nestedness of the brain,
they said that a given sugar ferments. When the bio-
the rest of the body, and the person
chemical compounds and their reactions that make
in the world, each entity executing
up the mechanisms and operations of fermentation
processes that overlap and turn back
became understood, ferment no longer applied to the
on themselves and each other in time
operations of the mechanisms that explained fer-
and space. The firing of a neuron in the
mentation but only to the overall process.
lateral intraparietal area may be critical
Behavioral neuroscientists and behavior analysts
to the execution of a choice response
are in a similar place. Lacking sufficient knowledge
that is reflective of recent relative rein-
of the brain mechanisms and operations that
forcement rates (see, e.g., Corrado
account for behavior, they extend psychological
et al., 2005; Lau & Glimcher, 2005),
concepts to the brain level, to which they typically
but the individual neuron’s firing only
do not apply. It is important for behavior analysts to
has meaning when it is part of an inte-
recognize that this is an error, lest they make the
grated neuronal circuit . . . the activity
mistake of believing that reinforcement is a process
of which only has meaning relative to
of the nervous system. Processes in the brain make
the current environmental-behavioral
reinforcement possible (distributed release of dopa-
context . . . which itself only has mean-
mine from the ventral tegmental area, alteration of
ing relative to previously experienced
synaptic efficacy in coactive neurons, etc.), but it is
environmental-behavioral contexts . . . a
unlikely that the term reinforcement will apply to
sufficient understanding of how the brain
any of them. These brain events allow reinforce-
participates in behavior will depend
ment to occur (and thereby, in some sense, explain
on an ability to refer simultaneously to
it; Schaal, 2003), but reinforcement will remain a
events at multiple levels of integration
relation between an animal’s behavior and its envi-
and at multiple time frames, including—
ronment (see Gold, 2009, for a complementary per-
most importantly from the perspective of
spective in psychiatry, and Thompson, 2007, for an
behavior analysts—the animal’s history.
opposing one in behavior analysis).
(Schaal, 2005, pp. 690–691)
So what does the brain do? At the beginning of
the chapter, I suggested an answer to this question Behavioral neuroscientists will be closer to suc-
when I defined behavioral neuroscience as “the inves- ceeding in describing the participation of brain
tigation of the neural processes that participate in events in behavior when they reject a belief that all
and account for functional relations between envi- behavioral and psychological events emerge from
ronment and behavior.” My definition takes as the activity of the nervous system. Psychological
axiomatic the centrality of the functional relation events do not happen inside of us; they are emergent
in explanations of behavior and asserts that the in the interaction of intact organisms with their
job of behavioral neuroscience is to elucidate the environments. What Alva Noë (2009) recently said
354
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
about consciousness applies as well to all psycholog- begin introducing students to neuroscience research
ical phenomena: in their classes; a broader view of behavior analysis
will expand their students’ career opportunities as it
The brain is not the locus of conscious-
enhances the depth of their worldviews.
ness inside us because consciousness
has no locus inside us. Consciousness
References
isn’t something that happens inside us:
Alvarez-Jaimes, L., Polis, I., & Parsons, L. H. (2008).
it is something that we do, actively, in
Attenuation of cue-induced heroin-seeking behavior
our dynamic interaction with the world by cannabinoid CB1 antagonist infusions into the
around us. . . . If we want to understand nucleus accumbens core and prefrontal cortex, but
how the brain contributes to conscious- not basolateral amygdala. Neuropsychopharmacology,
33, 2483–2493. doi:10.1038/sj.npp.1301630
ness, we need to look at the brain’s job in
relation to the larger nonbrain body and American Heart Association. (2010). About stroke.
Retrieved from http://www.strokeassociation.
the environment in which we find our- org/STROKEORG/AboutStroke/About-Stroke_
selves. (p. 24) UCM_308529_SubHomePage.jsp
Balleine, B. W., Delgado, M. R., & Hikosaka, O. (2007).
The role of the dorsal striatum in reward and
Conclusions decision-making. Journal of Neuroscience, 27, 8161–
8165. doi:10.1523/JNEUROSCI.1554-07.2007
In this chapter, I have reviewed a few ways in which Bauman, M. L., & Kemper, T. L. (2005). Neuroanatomic
“an organism is changed when exposed to contin- observations of the brain in autism: A review
gencies of reinforcement and why the changed and future directions. International Journal
of Developmental Neuroscience, 23, 183–187.
organism then behaves in a different way” (Skinner, doi:10.1016/j.ijdevneu.2004.09.006
1974, p. 237), focusing primarily on experimental
Bechtel, W. (2005). The challenge of characterizing
studies of reinforcement (or reinforcer presentation) operations in the mechanisms underlying behavior.
in which behavior and neuronal activity are Journal of the Experimental Analysis of Behavior, 84,
observed and correlated in real time. The observa- 313–325. doi:10.1901/jeab.2005.103-04
tion of prediction error–bound, phasic neuronal Bennett, M. R., & Hacker, P. M. S. (2003). Philosophical
activity elicited by reward has led to compelling foundations of neuroscience. Malden, MA: Blackwell.
ideas about the role of widespread dopamine release Bliss, T. V., & Lomo, T. (1973). Long-lasting potentiation
providing teaching signals that may alter the respon- of synaptic transmission in the dentate area of the
anaesthetized rabbit following stimulation of the per-
siveness of neurons in many regions of the brain to forant path. Journal of Physiology, 232, 331–356.
the events associated with the reward (Schultz,
Bromberg-Martin, E. S., & Hikosaka, O. (2009). Midbrain
2010). Behavior analysts interested in neuroscience dopamine neurons signal preference for advance
may find the focus on the real-time behavior of indi- information about upcoming rewards. Neuron, 63,
vidual neurons in these studies to be analogous to 119–126. doi:10.1016/j.neuron.2009.06.009
the focus on the behavior of individual organisms by Brown, J., Cooper-Kuhn, C. M., Kempermann, G., Van
which our field is defined and may thus be drawn to Praag, H., Winkler, J., Gage, F. H., & Kuhn, H. G.
(2003). Enriched environment and physical activ-
this research, perhaps even to the point of engaging ity stimulate hippocampal but not olfactory bulb
in it. However, as I have also indicated in this chap- neurogenesis. European Journal of Neuroscience, 17,
ter, the field of behavioral neuroscience is broad 2042–2046. doi:10.1046/j.1460-9568.2003.02647.x
and deep; behavior analysts may participate by Byrne, J. H., & Kandel, E. R. (1996). Presynaptic facilita-
applying their unique practical and philosophical tion revisited: State and time dependence. Journal of
Neuroscience, 16, 425–435.
expertise to research on processes ranging from the
subcellular to whole-organism levels and in the Chen, B. T., Bowers, M. S., Martin, M., Hopf, F. W.,
Guillory, A. M., Carelli, R. M., . . . Bonci, A. (2008).
study of normal functioning as well as injury and Cocaine but not natural reward self-administration
disease. I encourage teachers of behavior analysis to nor passive cocaine infusion produces persistent LTP
355
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
in the VTA. Neuron, 59, 288–297. doi:10.1016/ Epstein, D. H., Preston, K. L., Stewart, J., & Shaham, Y.
j.neuron.2008.05.024 (2006). Toward a model of drug relapse: An assess-
ment of the validity of the reinstatement procedure.
Christian, K. M., Poulos, A. M., Lavond, D. G., & Psychopharmacology, 189, 1–16. doi:10.1007/s00213-
Thompson, R. F. (2004). Comment on “Cerebellar 006-0529-6
LTD and learning-dependent timing of conditioned
eyelid responses.” Science, 304, 211. doi:10.1126/ Fanselow, M. S., & Kim, J. J. (1994). Acquisition of con-
science.1093706 textual Pavlovian fear conditioning is blocked by
application of an NMDA receptor antagonist D,L-2-
Corrado, G. S., Sugrue, L. P., Seung, H. S., & Newsome, amino-5-phosphonovaleric acid to the basolateral
W. T. (2005). Linear-nonlinear-Poisson models of amygdala. Behavioral Neuroscience, 108, 210–212.
primate choice dynamics. Journal of the Experimental doi:10.1037/0735-7044.108.1.210
jeab.2005.23-05 Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003).
Discrete coding of reward probability and uncer-
Crick, F. (1995). The astonishing hypothesis. London, tainty by dopamine neurons. Science, 299, 1898–
England: Touchstone. 1902. doi:10.1126/science.1077349
Dalley, J. W., Mar, A. C., Economidou, D., & Robbins, Glanzman, D. L. (2006). The cellular mecha-
T. W. (2008). Neurobehavioral mechanisms of impul- nisms of learning in Aplysia: Of blind men and
sivity: Fronto-striatal systems and functional neuro- elephants. Biological Bulletin, 210, 271–279.
chemistry. Pharmacology, Biochemistry and Behavior, doi:10.2307/4134563
90, 250–260. doi:10.1016/j.pbb.2007.12.021
Glimcher, P. W., & Rustichini, A. (2004).
de Wit, H. (2009). Impulsivity as a determinant and Neuroeconomics: The consilience of brain and
consequence of drug use: A review of underlying decision. Science, 306, 447–452. doi:10.1126/
processes. Addiction Biology, 14, 22–31. doi:10.1111/ science.1102566
j.1369-1600.2008.00129.x
Gold, I. (2009). Reduction in psychiatry. Canadian
de Wit, H., & Stewart, J. (1981). Reinstatement Journal of Psychiatry/Revue Canadienne de Psychiatrie,
of cocaine-reinforced responding in the rat. 54, 506–512.
Psychopharmacology, 75, 134–143. doi:10.1007/
Gould, E., Beylin, A., Tanapat, P., Reeves, A., & Shors,
BF00432175
T. J. (1999). Learning enhances adult neurogenesis
Di Chiara, G. (1995). The role of dopamine in drug abuse in the hippocampal formation. Nature Neuroscience,
viewed from the perspective of its role in motiva- 2, 260–265. doi:10.1038/6365
tion. Drug and Alcohol Dependence, 38, 95–137. Gould, E., & Tanapat, P. (1999). Stress and hippocampal
doi:10.1016/0376-8716(95)01118-I neurogenesis. Biological Psychiatry, 46, 1472–1479.
Dinsmoor, J. A., Browne, M. P., & Lawrence, C. E. doi:10.1016/S0006-3223(99)00247-4
(1972). A test of the negative discriminative Hemby, S. E., Martin, T. J., Co, C., Dworkin, S. I., &
stimulus as a reinforcer of observing. Journal of Smith, J. E. (1995). The effects of intravenous heroin
the Experimental Analysis of Behavior, 18, 79–85. administration on extracellular nucleus accum-
doi:10.1901/jeab.1972.18-79 bens dopamine concentrations as determined by
Dirnagl, U., Iadecola, C., & Moskowitz, M. A. (1999). in vivo microdialysis. Journal of Pharmacology and
Pathobiology of ischaemic stroke: An integrated Experimental Therapeutics, 273, 591–598.
view. Trends in Neurosciences, 22, 391–397. Herrnstein, R. J. (1961). Relative and absolute strength of
doi:10.1016/S0166-2236(99)01401-0 response as a function of frequency of reinforcement.
Edelman, G. M. (2003). Naturalizing consciousness: A Journal of the Experimental Analysis of Behavior, 4,
theoretical framework. Proceedings of the National 267–272. doi:10.1901/jeab.1961.4-267
Academy of Sciences of the United States of America, Herrnstein, R. J. (1970). On the law of effect. Journal of
100, 5520–5524. doi:10.1073/pnas.0931349100 the Experimental Analysis of Behavior, 13, 243–266.
doi:10.1901/jeab.1970.13-243
Ehninger, D., & Kempermann, G. (2008). Neurogenesis
in the adult hippocampus. Cell and Tissue Research, Howlin, P., Magiati, I., & Charman, T. (2009). Systematic
331, 243–250. doi:10.1007/s00441-007-0478-3 review of early intensive behavioral interventions for
children with autism. American Journal on Intellectual
Eldevik, S., Hastings, R. P., Hughes, J. C., Jahr, and Developmental Disabilities, 114, 23–41.
E., Eikeseth, S., & Cross, S. (2009). Meta-
analysis of early intensive behavioral interven- Hutsler, J. J., & Zhang, H. (2010). Increased dendritic
tion for children with autism. Journal of Clinical spine densities on cortical projection neurons in
Child and Adolescent Psychology, 38, 439–450. autism spectrum disorders. Brain Research, 1309,
doi:10.1080/15374410902851739 83–94.
356
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Jessberger, S., & Gage, F. H. (2008). Stem-cell-associated Experimental Analysis of Behavior, 91, 197–211.
structural and functional plasticity in the aging doi:10.1901/jeab.2009.91-197
hippocampus. Psychology and Aging, 23, 684–691.
doi:10.1037/a0014188 McClure, S. M., Ericson, K. M., Laibson, D. I.,
Loewenstein, G., & Cohen, J. D. (2007). Time
Kakegawa, W., Miyazaki, T., Emi, K., Matsuda, K., discounting for primary rewards. Journal of
Kohda, K., Motohashi, J., . . . Yuzaki, M. (2008). Neuroscience, 27, 5796–5804. doi:10.1523/
Differential regulation of synaptic plasticity and cere- JNEUROSCI.4246-06.2007
bellar motor learning by the C-terminal PDZ-binding
McClure, S. M., Laibson, D. I., Loewenstein, G., &
motif of GluRdelta2. Journal of Neuroscience, 28,
Cohen, J. D. (2004). Separate neural systems value
1460–1468. doi:10.1523/JNEUROSCI.2553-07.2008
immediate and delayed monetary rewards. Science,
Kalenscher, T., Diekamp, B., & Gunturkun, O. (2003). 306, 503–507. doi:10.1126/science.1100907
Neural architecture of choice behaviour in a con-
McEachin, J. J., Smith, T., & Lovaas, O. I. (1993). Long-
current interval schedule. European Journal of
term outcome for children with autism who received
Neuroscience, 18, 2627–2637. doi:10.1046/j.1460-
early intensive behavioral treatment. American
9568.2003.03006.x
Journal on Mental Retardation, 97, 359–372.
Kalivas, P. W., & Volkow, N. D. (2005). The neural basis
Mirenowicz, J., & Schultz, W. (1994). Importance of
of addiction: A pathology of motivation and choice.
unpredictability for reward responses in primate
American Journal of Psychiatry, 162, 1403–1413.
dopamine neurons. Journal of Neurophysiology, 72,
doi:10.1176/appi.ajp.162.8.1403
1024–1027.
Kempermann, G. (2002). Why new neurons? Possible Morris, R. G., Anderson, E., Lynch, G. S., & Baudry,
functions for adult hippocampal neurogenesis. M. (1986). Selective impairment of learning and
Journal of Neuroscience, 22, 635–638. blockade of long-term potentiation by an N-methyl-
Kempermann, G., Gast, D., & Gage, F. H. (2002). D-aspartate receptor antagonist, AP5. Nature, 319,
Neuroplasticity in old age: Sustained fivefold induc- 774–776. doi:10.1038/319774a0
tion of hippocampal neurogenesis by long-term Niv, Y. (2007). Cost, benefit, tonic, phasic: What do
environmental enrichment. Annals of Neurology, 52, response rates tell us about dopamine and motiva-
135–143. doi:10.1002/ana.10262 tion? Annals of the New York Academy of Sciences,
Lau, B., & Glimcher, P. W. (2005). Dynamic response- 1104, 357–376. doi:10.1196/annals.1390.018
by-response models of matching behavior in rhesus Noë, A. (2009). Out of our heads: Why you are not your
monkeys. Journal of the Experimental Analysis of brain, and other lessons from the biology of conscious-
Behavior, 84, 555–579. doi:10.1901/jeab.2005.110-04 ness. New York, NY: Hill & Wang.
Lau, B., & Glimcher, P. W. (2008). Value representa- O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R.,
tions in the primate striatum during matching Friston, K., & Dolan, R. J. (2004). Dissociable roles
behavior. Neuron, 58, 451–463. doi:10.1016/ of ventral and dorsal striatum in instrumental con-
j.neuron.2008.02.021 ditioning. Science, 304, 452–454. doi:10.1126/sci-
Lejuez, C. W., Schaal, D. W., & O’Donnell, J. (1998). ence.1094285
Behavioral pharmacology and the treatment of sub- Padoa-Schioppa, C., & Assad, J. A. (2006). Neurons in
stance abuse. In J. J. Plaud & G. H. Eifert (Eds.), the orbitofrontal cortex encode economic value.
From behavior theory to behavior therapy (pp. 116–135). Nature, 441, 223–226. doi:10.1038/nature04676
Needham Heights, MA: Allyn & Bacon.
Paton, J. J., Belova, M. A., Morrison, S. E., & Salzman,
Lovaas, O. I., Koegel, R. L., & Schreibman, L. (1979). C. D. (2006). The primate amygdala represents the
Stimulus overselectivity in autism: A review of positive and negative value of visual stimuli dur-
research. Psychological Bulletin, 86, 1236–1254. ing learning. Nature, 439, 865–870. doi:10.1038/
doi:10.1037/0033-2909.86.6.1236 nature04490
Mazur, J. E. (1987). An adjusting procedure for study- Peele, D. B., Casey, J., & Silberberg, A. (1984). Primacy
ing delayed reinforcement. In M. L. Commons, J. E. of interresponse-time reinforcement in account-
Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative ing for rate differences under variable-ratio and
analyses of behavior: Vol. 5. The effect of delay and of variable-interval schedules. Journal of Experimental
intervening events on reinforcement value (pp. 55–73). Psychology: Animal Behavior Processes, 10, 149–167.
Hillsdale, NJ: Erlbaum. doi:10.1037/0097-7403.10.2.149
Mazur, J. E., & Biondi, D. R. (2009). Delay-amount Perry, J. L., & Carroll, M. E. (2008). The role of impul-
tradeoffs in choices by pigeons and rats: Hyperbolic sive behavior in drug abuse. Psychopharmacology,
versus exponential discounting. Journal of the 200, 1–26. doi:10.1007/s00213-008-1173-0
357
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
David W. Schaal
Petry, N. M., & Madden, G. J. (2010). Discounting Schultz, W. (2010). Dopamine signals for reward value
and pathological gambling. In G. J. Madden & and risk: Basic and recent data. Behavioral and Brain
W. K. Bickel (Eds.), Impulsivity: The behavioral Functions, 6, 24. doi:10.1186/1744-9081-6-24
and neurological science of discounting (pp. 273–
Schultz, W., Tremblay, L., & Hollerman, J. R. (2000).
294). Washington, DC: American Psychological
Reward processing in primate orbitofrontal cortex
Association. doi:10.1037/12069-010
and basal ganglia. Cerebral Cortex, 10, 272–283.
Pickens, R., & Thompson, T. (1968). Cocaine-reinforced doi:10.1093/cercor/10.3.272
behavior in rats: Effects of reinforcement magnitude
and fixed-ratio size. Journal of Pharmacology and Skinner, B. F. (1974). About behaviorism. New York, NY:
Experimental Therapeutics, 161, 122–129. Knopf.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of Steiner, B., Wolf, S., & Kempermann, G. (2006).
Pavlovian conditioning: Variations in the effective- Adult neurogenesis and neurodegenera-
ness of reinforcement and nonreinforcement. In A. tive disease. Regenerative Medicine, 1, 15–28.
H. Black & W. F. Prokasy (Eds.), Classical condition- doi:10.2217/17460751.1.1.15
ing: Vol. 2. Current research and theory (pp. 64–99). Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004).
New York, NY: Appleton-Century-Crofts. Matching behavior and the representation of value
Rodrigues, S. M., Schafe, G. E., & LeDoux, J. E. (2001). in the parietal cortex. Science, 304, 1782–1787.
Intra-amygdala blockade of the NR2B subunit of doi:10.1126/science.1094765
the NMDA receptor disrupts the acquisition but Taub, E., Miller, N. E., Novack, T. A., Cook, E. W., III,
not the expression of fear conditioning. Journal of Fleming, W. C., Nepomuceno, C. S., . . . Crago, J.
Neuroscience, 21, 6889–6896. E. (1993). Technique to improve chronic motor
Roman, F., Staubli, U., & Lynch, G. (1987). Evidence deficit after stroke. Archives of Physical Medicine and
for synaptic potentiation in a cortical network Rehabilitation, 74, 347–354.
during learning. Brain Research, 418, 221–226. Taub, E., Uswatte, G., & Morris, D. M. (2003). Improved
doi:10.1016/0006-8993(87)90089-8 motor recovery after stroke and massive cortical
Salamone, J. D., Correa, M., Farrar, A. M., Nunes, E. J., & reorganization following constraint-induced move-
Pardo, M. (2009). Dopamine, behavioral economics, ment therapy. Physical Medicine and Rehabilitation
and effort. Frontiers in Behavioral Neuroscience, 3, 13. Clinics of North America, 14(1, Suppl.), S77–S91.
doi:10.3389/neuro.08.013.2009 doi:10.1016/S1047-9651(02)00052-9
Salamone, J. D., Steinpreis, R. E., McCullough, L. Thompson, T. (2007). Relations among functional sys-
D., Smith, P., Grebel, D., & Mahan, K. (1991). tems in behavior analysis. Journal of the Experimental
Haloperidol and nucleus accumbens dopamine Analysis of Behavior, 87, 423–440. doi:10.1901/
depletion suppress lever pressing for food but jeab.2007.21-06
increase free food consumption in a novel food Tobler, P. N., Fiorillo, C. D., & Schultz, W. (2005).
choice procedure. Psychopharmacology, 104, Adaptive coding of reward value by dopamine
515–521. doi:10.1007/BF02245659 neurons. Science, 307, 1642–1645. doi:10.1126/
Salzman, C. D., Paton, J. J., Belova, M. A., & Morrison, science.1105370
S. E. (2007). Flexible neural representations of Tremblay, L., & Schultz, W. (1999). Relative reward pref-
value in the primate brain. Annals of the New York erence in primate orbitofrontal cortex. Nature, 398,
Academy of Sciences, 1121, 336–354. doi:10.1196/ 704–708. doi:10.1038/19525
annals.1401.034
van Praag, H. (2008). Neurogenesis and exercise: Past
Schaal, D. W. (2003). Explanatory reductionism in and future directions. NeuroMolecular Medicine, 10,
behavior analysis. In K. A. Lattal & P. N. Chase 128–140. doi:10.1007/s12017-008-8028-z
(Eds.), Behavior theory and philosophy (pp. 83–102).
New York, NY: Kluwer Academic. van Praag, H., Kempermann, G., & Gage, F. H. (1999).
Running increases cell proliferation and neuro-
Schaal, D. W. (2005). Naming our concerns about neuro
genesis in the adult mouse dentate gyrus. Nature
science: A review of Bennett and Hacker’s philo-
Neuroscience, 2, 266–270. doi:10.1038/6368
sophical foundations of neuroscience. Journal of
the Experimental Analysis of Behavior, 84, 683–692. Volkow, N. D., Wang, G. J., Ma, Y., Fowler, J. S., Zhu,
doi:10.1901/jeab.2005.83-05 W., Maynard, L., . . . Swanson, J. M. (2003).
Schreibman, L. (2000). Intensive behavioral/ Expectation enhances the regional brain metabolic
psychoeducational treatments for autism: and the reinforcing effects of stimulants in cocaine
Research needs and future directions. Journal of abusers. Journal of Neuroscience, 23, 11461–11468.
Autism and Developmental Disorders, 30, 373–378. Welsh, J. P., Yamaguchi, H., Zeng, X. H., Kojo, M.,
doi:10.1023/A:1005535120023 Nakada, Y., Takagi, A., . . . Llinás, R. R. (2005).
358
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Normal motor learning during pharmacological learning. European Journal of Neuroscience, 19, 181–
prevention of Purkinje cell long-term depression. 189. doi:10.1111/j.1460-9568.2004.03095.x
Proceedings of the National Academy of Sciences of Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2006).
the United States of America, 102, 17166–17171. Inactivation of dorsolateral striatum enhances
doi:10.1073/pnas.0508191102 sensitivity to changes in the action-outcome con-
Wolf, S. L., Winstein, C. J., Miller, J. P., Taub, E., tingency in instrumental conditioning. Behavioural
Uswatte, G., Morris, D., . . . Nichols-Larsen, D.; Brain Research, 166, 189–196. doi:10.1016/j.
EXCITE Investigators. (2006). Effect of constraint- bbr.2005.07.012
induced movement therapy on upper extremity Zhao, C., Wang, J., Zhao, S., & Nie, Y. (2009).
function 3 to 9 months after stroke: The EXCITE Constraint-induced movement therapy enhanced
randomized clinical trial. JAMA, 296, 2095–2104. neurogenesis and behavioral recovery after stroke in
doi:10.1001/jama.296.17.2095 adult rats. Tohoku Journal of Experimental Medicine,
218, 301–308. doi:10.1620/tjem.218.301
Wyckoff, L. B., Jr. (1952). The role of observing
responses in discrimination learning. Psychological Zhao, H., Shimohata, T., Wang, J. Q., Sun, G., Schaal,
Review, 59, 431–442. doi:10.1037/h0053932 D. W., Sapolsky, R. M., & Steinberg, G. K. (2005).
Akt contributes to neuroprotection by hypo-
Yin, H. H., Knowlton, B. J., & Balleine, B. W. (2004). thermia against cerebral ischemia in rats. Journal
Lesions of dorsolateral striatum preserve outcome of Neuroscience, 25, 9794–9806. doi:10.1523/
expectancy but disrupt habit formation in instrumental JNEUROSCI.3163-05.2005
359
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 16
Stimulus Control and Stimulus

Class Formation
Peter J. Urcuioli
The significance of this chapter’s topic can perhaps effects on behavior” by physically dissimilar stimuli
best be appreciated by reading various comments (cf. McIlvane, 1992; see also Hall, Mitchell, Graham,
made as recently as within the past 20 years and as far & Lavis, 2003; Jenkins, 1963; Urcuioli, 1996).
back as 70 years ago by some prominent psychologists: The distinction can be appreciated by means of
a simple example. Chairs differ from one another in
A classic problem in the analysis of
various ways—shape, color, material, presence ver-
behavior is to provide an account of how
sus absence of arm rests, and so forth—but all
physically dissimilar stimuli can have
share a sufficient number of physically similar fea-
similar and apparently interrelated effects
tures such that new chairs people have never before
on behavior. (McIlvane, 1992, pp. 76–77)
encountered can evoke previously learned reac-
Understanding how organisms come to tions (e.g., saying “chair” or sitting on one; cf.
form categories is central to understand- Mackintosh, 2000). By contrast, consider a chair, a
ing complex human behavior in general lamp, an end table, a bed, and a bureau. Despite
and language in particular. (Horne & their physical dissimilarity, these are all easily rec-
Lowe, 1997, p. 278) ognized as examples of furniture and as having
interrelated effects on behavior (e.g., where people
The problem of stimulus equivalence is a
go to buy them, where people place them). People’s
fundamental one . . . for any psychology
common reactions to them go beyond any sort of
purporting to deal in a thorough-going man-
physical resemblances, reflecting instead what they
ner with adaptive behavior. . . . How can
have learned about them (e.g., that they have the
we account for the fact that a stimulus will
same category name or that they can be found
sometimes evoke a reaction . . . with which it
inside houses). Hull (1939) was keenly aware of
has never been associated? (Hull, 1939, p. 9)
this and proposed secondary stimulus generaliza-
Clark Hull (1939), an early influential behavior- tion as a mechanism to account for how classes of
ist, proposed two answers for the question he posed. disparate objects or stimuli could develop. In
One was primary stimulus generalization. Through essence, he said that explicitly learning the same
this process, reactions explicitly learned to particu- response to a set of disparate stimuli would gener-
lar objects or stimuli also occur to some extent to ate an equivalence between them such that new
new objects or new stimuli perceptually similar to behavior learned to a subset of the stimulus class
those of training. This well-known and thoroughly would immediately generalize to the remaining
researched process (Honig & Urcuioli, 1981) is cer- class members (see, e.g., Lowe, Horne, & Hughes,
tainly one contributing factor, but it does not ade- 2005; Spradlin, Cotter, & Baxley, 1973; Urcuioli &
quately explain “similar and apparently interrelated Lionello-DeNolf, 2001).
DOI: 10.1037/13937-016
361
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
Another quotation might be helpful at this point that an elusive behavioral effect indicative of stimu-
to set the stage for what is to follow: “Class forma- lus class formation finally made its appearance once
tion . . . may be the product of any procedure that it was recognized that the functional stimulus was a
serves to partition a set of stimuli into subsets of bit more complex than researchers originally thought.
stimuli that are substitutable for one another in cer-
tain contexts” (Saunders & Green, 1992, p. 319).
Stimulus Control Primer
This remark is applicable both to stimulus classes
whose partitioning reflects shared physical features, Humans’ behavior and that of other animals do not
such as people versus flowers versus cars versus occur in an environmental vacuum. To the contrary,
chairs (Bhatt, Wasserman, Reynolds, & Knauss, certain behavior often routinely occurs in certain
1988), and to stimulus classes whose partitioning circumstances but not in others. Likewise, different
reflects common learned associations (e.g., behavior is often observed in different circum-
Vaughan, 1988; Vickery & Jiang, 2009). Further- stances. In a nutshell, these general observations
more, it underscores an important way to identify capture what learning psychologists mean by the
potential stimulus classes after training or other term stimulus control: Particular circumstances or
explicit learning experiences—namely, showing that stimuli, when present, occasion or control particular
the stimuli in question are interchangeable or sub- behavior.
stitutable for one another in certain contexts. I Consider studying, for example, ignoring all of
would prefer the adjective new to certain (cf. Goldia- the different forms this behavior can take. Studying
mond, 1962) to emphasize the fact that a stimulus is occasioned by particular circumstances—for
that, as a result of training, becomes a member of a example, those associated with an academic session
particular class can now “evoke a reaction . . . with at a college or university. Stated otherwise, studying
which it has never been associated” (Hull, 1939, occurs more frequently at college during the aca-
p. 9). In other words, that stimulus must be shown demic session than it does elsewhere. The academic
to occasion new, untrained (emergent) behavior. environment, then, can be said to control this
I organized this chapter around two desired take- behavior, a point easily appreciated by students who
home messages. The first is that disparate stimuli plan to catch up on studying while at home on
that share a common association are often treated as weekends or during semester breaks only to realize
belonging together—that is, as members of the same later on how unsuccessful their plans were. A likely
stimulus class or category. This message merely reason is that the circumstances (stimuli) associated
reinforces a long-recognized finding in the psycho- with home makes studying far less likely and other
logical literature (e.g., Hull, 1939; Jenkins & Pal- behavior far more likely.
ermo, 1964; Peters, 1935; Shipley, 1935). The To understand how stimulus control originates,
common association in question can be the occa- consider young children learning the names of
sioning of the same reinforced response, the signal- objects. Using picture books, parents typically teach
ing of the same but distinct reinforcer, or more their children the names of pictures to which they
generally association with the same outcome. I pro- point (e.g., to say “dog” to a picture of a dog, “cat”
vide examples of each of these effective associations to a picture of a cat) by providing social reinforce-
from both human and nonhuman animal research ment only when the child says the correct name.
and illustrate how the resulting stimulus classes This simple procedure eventually generates stimulus
have been demonstrated. The second message is that control over naming—that is, different pictures
any demonstration of stimulus substitutability or come to occasion different names (different behav-
interchangeability requires researchers to know pre- ior) by the child. It also illustrates how discrimina-
cisely the composition of each class member (viz., tion training effectively establishes stimulus control:
what the functional stimuli are) and how they con- Different behavior (saying “dog” vs. saying “cat,”
trol behavior. Here, my focus narrows somewhat as I respectively) is explicitly reinforced in the presence
draw on research from the animal literature showing of different stimuli (picture of a dog vs. picture of a
362
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Stimulus Control and Stimulus Class Formation
cat). Note, too, that discrimination training also This procedure has many variations, often with
involves nonreinforcement of incorrect behavior procedure-specific names. For example, arbitrary
(e.g., saying “dog” to a picture of a cat, and vice matching (also known as symbolic matching) involves
versa), thus narrowing or sharpening the control samples and comparisons that have no obvious
any particular stimulus exerts over behavior. Of physical resemblance to one another, as the preced-
course, the correct or reinforced behavior learned to ing example illustrates. Identity matching, however,
a particular stimulus (e.g., a specific dog picture) consists of samples and comparisons that are physi-
will then frequently occur or generalize to other per- cally identical, as would be the case if the child was
ceptually similar stimuli (pictures of other dogs, real required to point to a dog or a cat comparison pic-
dogs, etc.)—a process known as stimulus generaliza- ture after being shown one or the other picture as
tion (e.g., Honig & Urcuioli, 1981; Urcuioli, 2003). the sample stimulus.
Discrimination training can take a variety of Finally, successive matching refers to a set of con-
forms. For instance, successive discrimination ditional discriminations in which only one compari-
involves presenting different stimuli individually at son (rather than two or more) appears after each
different times, reinforcing the appropriate behavior sample stimulus. Successive refers to the fact that
and nonreinforcing other behavior to each. Succes- subjects must not only discriminate between succes-
sive discrimination is implied in the preceding sively (individually) presented samples but, in addi-
picture–name example. By contrast, if the parent tion, must discriminate between successively
were to show a picture of a dog and a cat together (individually) presented comparisons. Successive
and ask the child to point to one or the other (rein- matching is sometimes called go–no-go matching
forcing the correct choice), the child must differen- because subjects learn that responding (go) to each
tiate between the simultaneously presented pictures individually presented comparison is reinforced
to point appropriately to one of them, illustrating a only when that comparison is preceded by a particu-
simultaneous discrimination. In fact, this latter lar sample. When that same comparison is preceded
example in its entirety illustrates a conditional dis- by a different sample, responding is nonreinforced,
crimination: The picture to which the child must and subjects eventually learn not to respond to it
point depends (is conditional) on what the parent (no go) on those occasions.
requests. If the parent asks for the dog, pointing to Successive, simultaneous, and conditional dis-
the dog rather than to the cat is reinforced; if the criminations have been implemented in many ways
parent asks for the cat, pointing to the cat rather in research with humans and other animals. The
than to the dog is reinforced. This instructional reader will see some examples in figures included in
method, then, actually requires both a successive this chapter (e.g., Figures 16.2, 16.7, and 16.10).
and a simultaneous discrimination: To perform cor- Not surprisingly, some stimuli used with humans
rectly, the child must differentiate between the indi- (e.g., written or spoken words) cannot be used with
vidual words spoken sequentially by the parent (dog other animals. Nonetheless, these details are less
vs. cat) and must also differentiate between the con- important than the general features of the proce-
currently presented animal pictures. dures and that visual or auditory stimuli used with
In the stimulus control literature, this condi- nonhuman animals accommodate their natural sen-
tional discrimination procedure would be called sory abilities.
two-alternative matching-to-sample or matching-to-
sample for short. In the example, the samples are the
Stimulus Classes: Partitioning by
successively spoken words dog and cat. The simulta-
Common Responses
neously presented dog and cat pictures are called
comparisons or comparison alternatives. The task is to A robust finding in the psychological literature is
match the appropriate comparison picture to the that physically dissimilar stimuli that occasion the
previously heard sample, hence the name of the same reinforced response become members of the
procedure. same stimulus class. This has been demonstrated in
363
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
a variety of ways, and a good place to start is by researcher individual wooden shapes when the
describing some recent experiments by Lowe and researcher asked for a zag or a vek stimulus, all of
his associates (e.g., Horne, Lowe, & Harris, 2007; them were subsequently unable to correctly sort the
Lowe, Horne, Harris, & Randle, 2002; Lowe et al., shapes into groups when requested to do so using the
2005) involving young children between 1 and 4 “Look at this” instruction. Interestingly, most could
years old. In one study (Lowe et al., 2002, Experi- also not provide the appropriate zag or vek response
ment 1A), children were taught to say “zag” to each when asked what each shape stimulus was (i.e., they
of three arbitrary wooden shapes and to say “vek” to could not provide the common name response).
each of three other arbitrary wooden shapes with The partitioning of physically dissimilar stimuli
positive social feedback for correct responding (an into distinct classes via this sort of common
example of a successive discrimination). For the response training can also be demonstrated in a
most part, the shapes bore no obvious resemblance transfer-of-control or a transfer-of-function test
to any real-world object or figure, and moreover, (Dougher & Markham, 1994, 1996; see also Volume
their random assignment across subjects into a zag 2, Chapter 1, this handbook). The procedure works
group and a vek group minimized or eliminated any as follows. First, subjects learn a common reinforced
possibility that the shapes within a particular group response to one group of disparate stimuli and
would be perceptually more similar to one another another reinforced response to a second group of
than to shapes in the other group. After each child stimuli. After original training has been successfully
learned to consistently say “zag” and “vek” to the completed, some new behavior is explicitly trained
various stimuli, his or her ability to sort the shapes to one member of each common-response set of
into common-response groups was tested in a stimuli. This second training phase has for obvious
category-sort task. The task involved presenting all reasons been called reassignment training by Wasser-
six shapes together, pointing at one, and saying to the man, DeVolder, and Coppage (1992). Finally, sub-
child either “Look at this. Can you give me the oth- jects are tested with the remaining stimuli to see
ers?” or “What is this? Can you give me the others?” whether those stimuli, too, will now occasion the
(a conditional discrimination example). With the for- same response that has just been explicitly taught to
mer test instructions, four of nine children correctly one of their purported class members. If original
selected the zag and the vek shapes virtually without training had generated stimulus classes defined by
error, whereas the remaining five children were com- the common response conditioned to each class
pletely unable to sort the shapes in groups. However, member, then stimulus control over new behavior
when the latter five children were retested again established to one member during reassignment
using the “What is this?” instruction, all five appro- training should immediately transfer to all other
priately sorted the zag and vek shapes, averaging members (hence, transfer of control).
92.1% correct sorts (range = 83.3%–100%). These transfer effects were demonstrated in
These findings have been systematically repli- young children by Lowe et al. (2005). Original (or
cated by Miguel, Petursdottir, Carr, and Michael initial) training with arbitrary wooden shapes was
(2008), who also showed that preschool children virtually identical to that of Lowe et al. (2002). Dur-
could not sort shapes into their eventual groups ing subsequent reassignment training, each child
until they were taught to provide common labels was taught to clap to one shape in the zag group and
(north vs. south) for those shapes. Furthermore, to wave to one shape in the vek group. When the
other research has shown that success on these stim- researcher then asked, “How does this go?” to each
ulus class tests requires that children actually learn of the other shapes, every child clapped or waved in
to say “zag” or “vek” to each shape stimulus (i.e., to a class-consistent fashion.
give or be able to give a common albeit distinct Although common naming—in the verbal–spoken
response to the shapes within each group). For meaning of that term—may be a potent way for
example, Horne, Lowe, and Randle (2004) showed stimulus classes to develop (Dugdale & Lowe, 1990;
that if young children simply learned to give the Eikeseth & Smith, 1992; Mandell & Sheen, 1994;
364
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
see also Reese, 1972), it is certainly not the only the other window with their positioning random-
type of effective response, as illustrated by other ized across trials. Notice that (a) each reinforced
research using both humans and nonhuman animals comparison-choice response (e.g., R1) was occa-
(e.g., Garcia & Rehfeldt, 2008; Horne et al., 2007; sioned by more than just one sample and (b) sam-
Urcuioli, 2006a). This is underscored by Table 16.1, ples occasioning a common response (e.g., S1 and
the upper portion of which shows a generalized S3) were no more similar to one another in appear-
schematic of common-response training along with ance than samples occasioning a different response
reassignment training and a subsequent transfer-of- (e.g., S1 and S4); indeed, in this experiment, they
control test. The hypothesized stimulus classes are were most certainly less similar. To assess stimulus
shown in the lower portion of the table. class formation, Spradlin et al. then reinforced the
For example, Spradlin et al. (1973, Experiments selection of a mushroomlike comparison (R3) after
1 and 2) initially trained adolescents with intellec- the striped-circle sample (S1) and a chainlike-figure
tual disabilities to press the lower of two windows comparison (R4) after the solid-circle sample (S2)
displaying a star (Response [R] 1) after seeing either during reassignment training. Testing involved pre-
a striped circle (Stimulus [S] 1) or a horizontal bar senting, on different trials, the horizontal- and
(S3) displayed in an upper window and to press vertical-bar samples (S3 and S4, respectively) fol-
whichever lower window displayed an infinity sign lowed by the R3 and R4 choice alternatives. These
(R2) after seeing either a solid circle (S2) or a verti- infrequent test trials were inserted among all of the
cal bar (S4) displayed in an upper window. (The various training trials from original and reassign-
upper window stimuli were the samples, and the ment training and, unlike the training trials, were
lower window stimuli were the comparisons in this nonreinforced. Stimulus class formation via
matching-to-sample task.) On each training trial, common-response training should yield a preference
both comparisons were presented for response for R3 after S3, and for R4 after S4, on the test trials
choice, one in one lower window and the other in (i.e., class-consistent responding), which is precisely
what Spradlin et al. (1973, Experiments 1 and 2)
Table 16.1 found. Their average training and test performances
are depicted in the top panel of Figure 16.1.
Procedure to Assess Stimulus-Class Formation on Using 12 color slides each of people, cars, chairs,
the Basis of Partitioning by Common Reinforced and flowers, Wasserman et al. (1992) trained hun-
Responses gry pigeons to make one spatially distinct choice
response whenever a slide depicting cars or flowers
Training was shown on a center viewing screen of a four-
Stimulus-class choice panel (see left side of Figure 16.2) and a dif-
Common response Reassignment tests ferent, spatially distinct choice response whenever a
S1 → R1+ slide depicting people or chairs was shown. Thus,
S2 → R2+ S1 → R3+ S3 → R3 vs. R4 one common response was reinforced with food
S3 → R1+ S2 → R4+ S4 → R3 vs. R4 after each photographic stimulus in two disparate
S4 → R2+ groups of complex stimuli, and an alternative com-
Hypothesized stimulus classes
mon response was reinforced with food after each
S1 S2
photographic stimulus in two other disparate
S3 S4
groups of complex stimuli. To evaluate whether this
common-response training yielded two stimulus
classes—that is, [cars, flowers] and [people, chairs],
Note. S1–S4 represent different stimuli, and R1–R4 pigeons were then taught to make new spatial
represent different reinforced (+) responses. Italicized choice responses to one set of slides making up each
responses represent class-consistent responding assum-
ing the two hypothesized stimulus classes shown as the presumed class (e.g., chairs and flowers; see middle
circled elements. of Figure 16.2). Finally, the remaining sets of slides
365
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
(e.g., people and cars) were intermixed among the

reassignment training trials in a test phase (see right
side of Figure 16.2) to see whether the new spatial-
response functions directly trained to the reassigned
class members would immediately transfer to the
remaining class members. The test trials were non-
differentially reinforced: Food was delivered no mat-
ter what spatial choice response pigeons made to
each slide shown in testing. Wasserman et al. did
this to assess the pigeons’ test-trial preferences based
on prior learning and to avoid biasing those prefer-
ences in one way or another during testing.
The bottom panel of Figure 16.1 shows the data
from the second block of four test sessions in the
Wasserman et al. (1992) study. The pigeon results
correspond nicely with those reported by Spradlin
et al. (1973). Although the percentage of class-
consistent choices on the stimulus class test trials
Figure 16.1. Stimulus class (transfer-of-control)
results after common response training. Top panel: was not as high as the corresponding accuracy level
Averaged percentage of correct choice (±1 standard on the explicitly reassigned training trials, it was
error of the mean) of six adolescents with mental nevertheless significantly above the level expected
retardation on all reinforced training trials (baseline) by chance alone (indicated by the dotted line).
and average percentage of class-consistent choices
(±1 standard error of the mean) on nonreinforced Moreover, the pigeon test results are perhaps even
test trials (probes) over two test sessions. Dotted more impressive considering that they were tested
line indicates chance level of performance. Data from with a total of 24 different slides (12 showing differ-
Spradlin, Cotter, and Baxley (1973). Bottom panel:
Averaged percentage of correct choice of eight pigeons
ent people, 12 showing different cars). The adoles-
on reassigned training trials (cf. Table 16.1) and aver- cent subjects in the Spradlin et al. study were tested
age percentage of class-consistent choices on nondif- with just two sample stimuli.
ferentially reinforced test trials (probes) over four To what extent do the class-consistent choices
test sessions. Dotted line indicates chance level of
performance. Data from Wasserman, DeVolder, and exhibited in these tests truly reflect class partition-
Coppage (1992). ing by common responses in original training?
This question was answered by Urcuioli and
Common-response Training Reassignment Testing
Cars & Flowers Chairs People ?
Viewing
Screen
Food
Hopper
People & Chairs Flowers Cars ?
Figure 16.2. A schematic depiction of the apparatus, reinforced response locations for
the stimuli during common-response training and reassignment, and the class-consistent
response locations during testing in Wasserman, DeVolder, and Coppage (1992).
366
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Lionello-DeNolf (2005), who gave common- same reinforced sample–comparison relations dur-
response training to one group of pigeons but ing reassignment and experienced the same differen-
unique-response training to a second, control group tially reinforced sample–comparison relations in
before reassignment and testing (see Table 16.2). testing.
For the latter one-to-one (OTO) group, each sample For these tests, half of the common-response
stimulus occasioned a different reinforced choice (MTO) pigeons were tested on reinforced relations
response during initial training; otherwise, its reas- consistent with the hypothesized stimulus classes
signment training and testing (cf. the middle and arising from such training. The italicized responses
right portions of Table 16.2, respectively) were iden- shown in Table 16.2 indicate class-consistent rein-
tical to that for the common-response or many-to- forcement contingencies for these pigeons. For
one (MTO) group. example, if S1 and S3 are members of the same class
Because there were no common reinforced (cf. Table 16.1) and R3 was reinforced after S1 dur-
responses across samples for the OTO group during ing reassignment, then reinforcing R3 after S3 dur-
initial training, there could be no common-response ing testing is consistent with class membership and
sample classes and, thus, no a priori way to classify should yield greater-than-chance levels of reinforced
these pigeons’ choices in testing as class consistent test trial responding. The remaining pigeons were
or class inconsistent (Urcuioli & Lionello-DeNolf, tested on reinforced relations inconsistent with such
2005). Consequently, to facilitate a between-group classes (not shown in Table 16.2): For them, R4 (the
comparison, each pigeon in the OTO group was alternative response) was reinforced after S3, and
paired with a pigeon in the MTO group that had the likewise, R3 was reinforced after S4, during testing
(Urcuioli & Lionello-DeNolf, 2005). These rein-
forcement contingencies are inconsistent with class
Table 16.2
membership and should yield lower-than-chance
Design to Evaluate the Role of Common Responses levels of reinforced test trial responding (at least at
in Stimulus Class Formation (cf. Urcuioli & the outset of testing). By contrast, pigeons in the
Lionello-DeNolf, 2005, Experiment 1) OTO group were expected to respond haphazardly
(viz., at chance) at the outset of testing because
Training their samples could not have been partitioned into
Initial Reassignment Stimulus-class tests
common-response classes given the absence of
MTO
common-response training.
S1 → R1+ Figure 16.3 shows the first-session test results for
S2 → R2+ S1 → R3+ S3 → R3 vs. R4 the MTO (common-response) pigeons in both the
S3 → R1+ S2 → R4+ S4 → R3 vs. R4 consistent and the inconsistent conditions and for
S4 → R2+ all of the OTO pigeons combined. The above- versus
below-chance performances for the two MTO condi-
OTO
tions are indicative of stimulus class formation. The
S1 → R1+
chance level of accuracy for the latter group of
S2 → R2+ S1 → R3+ S3 → R3 vs. R4
S4 → R3 vs. R4 pigeons demonstrates that common-response train-
S3 → R5+ S2 → R4+
S4 → R6+
ing was indeed responsible for the different test per-
formance profiles of the MTO pigeons (Urcuioli &
Note. S1–S4 represent different stimuli, and R1–R6 rep- Lionello-DeNolf, 2005).
resent different reinforced (+) responses. Underlined
responses highlight the difference between groups in
initial training and the one-to-one sample-comparison Stimulus Classes: Partitioning by
response relation for the OTO group. Italicized responses
represent class-consistent responding for the MTO Common Outcomes
group during testing. MTO = many-to-one or common-
response group; OTO = one-to-one or unique-response In his well-known Law of Effect, Edward L.
group. Thorndike (1898) underscored the point that the
367
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
particular, albeit distinctive outcome (cf. Urcuioli,

2005). Stated otherwise and in reference to the
introductory quotation by McIlvane (1992), physi-
cally dissimilar stimuli can have similar effects on
behavior because the responses they occasion lead
to the same reinforcing outcome.
One way to demonstrate partitioning of stimuli
into classes by common outcomes is to use a transfer-
of-control design such as that shown in Table 16.3,
which provides the schematics for two phases of
training involving different stimuli and different
reinforced responses: S1 and S2 in Phase 1 versus S3
and S4 in Phase 2 and, likewise, reinforced responses
Figure 16.3. Average percentage of rein- R1 and R2 in Phase 1 versus R3 and R4 in Phase 2.
forced choices (±1 standard error of the mean) Both training phases, however, share the use of the
on the first stimulus class transfer-of-control
test for pigeons initially trained on a common-
same two reinforcing outcomes (Os), O1 and O2.
response matching task (MTO group) or a task (The reader should note that it is unnecessary to
without common comparison responses (OTO conduct these training phases separately; they can be
group). MTO test results are shown separately conducted concurrently.) The critical feature is that
for pigeons whose reinforced test trial choices
were either consistent (Cons.) or inconsistent O1 is common to two physically dissimilar stimuli
(Incons.) with stimulus class formation. (viz., S1 and S3), and O2 is common to two other
MTO = many to one, or common response; physically dissimilar stimuli (S2 and S4).
OTO = one to one, or unique response. From If these outcome associations partition the four
“The Role of Common Reinforced Comparison
Responses in Acquired Sample Equivalence,” stimuli into two classes (cf. the lower portion of
by P. J. Urcuioli and K. M. Lionello-DeNolf, Table 16.3), then S1 should be interchangeable
2005, Behavioral Processes, p. 69. Copyright with S3 (and vice versa), and likewise, S2 should be
2005 by Elsevier B.V. Adapted with permission.
Table 16.3
consequence or outcome of behavior can influence
the future likelihood of that behavior. Indeed, the Procedure to Assess Stimulus-Class Formation on
study of how behavior is affected by its conse- the Basis of Partitioning by Common Outcomes
quences defines the field of operant and instrumen-
tal conditioning. Just considering the consequence Differential outcome training
called positive reinforcement, a little reflection indi- Phase 1 Phase 2 Stimulus-class tests
cates that the reinforcing consequence or outcome S1 → R1+ (O1) S3 → R3+ (O1) S1 → R3 vs. R4
for behavior more often than not varies across S2 → R2+ (O2) S4 → R4+ (O2) S2 → R3 vs. R4
behavior. Moreover, because reinforced behavior is S3 → R1 vs. R2
typically occasioned by antecedent stimuli, those S4 → R1 vs. R2
Hypothesized stimulus classes
stimuli will also be associated with different out-
comes. Given the behavioral potency of reinforcing S1 S2
outcomes and their frequent association with ante- S3 S4
cedent stimuli, it should come as no surprise that
the typical experience of differential outcomes pro-
vides another means for generating stimulus classes. Note. S1–S4 represent different stimuli; R1–R4 represent
Indeed, a considerable body of research has con- different reinforced (+) responses, and O1 and O2 rep-
resent different outcomes. Italicized responses represent
firmed that disparate stimuli will be partitioned into class-consistent responding assuming the two hypoth-
classes on the basis of a common association with a esized stimulus classes shown as the circled elements.
368
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
interchangeable with S4 (and vice versa). Such outcomes during this group’s stimulus class test.
interchangeability would then permit each stimulus The reinforcement contingencies for pigeons in
to occasion a response it had never previously occa- each of the two control groups during testing
sioned. Research conducted with pigeons, young were matched to those of the correlated group to
children, and adults demonstrate just such an effect. ensure comparability of test performances across the
For example, Edwards, Jagielo, Zentall, and three groups.
Hogan (1982, Experiment 2) trained one group of If the sample stimuli from training were parti-
hungry pigeons on two separate identity-matching tioned into classes as a result of common outcome
tasks in which they learned in one task to respond associations in the correlated group, accuracy on the
to the same color comparison (red or green) as a very first test session should be above the level
preceding color sample (red or green) and in expected by chance alone (i.e., more than 50% cor-
another task to respond to the same black-on-white rect). By contrast, the absence of these differential
shape comparison (plus or circle) as a preceding outcome associations in the two control groups
shape sample (plus or circle). For this group, a cor- should not result in partitioning, and consequently,
rect comparison response after one color (and one their first test session accuracies should be at or
shape) sample produced peas as the food reinforcer, close to chance. The top panel of Figure 16.4 shows
whereas a correct comparison response after the this was precisely the pattern of first-session results
other color (and other shape) sample produced that Edwards et al. (1982) observed.
wheat as the food reinforcer. In other words, the A stimulus class interpretation of these results
reinforcing outcomes for this group were correlated may seem to be a stretch given that the average first-
with particular sample stimuli (viz., peas [O1] with session accuracy for the correlated group, although
S1 and S3, and wheat [O2] with S2 and S4). For two significantly above chance, was still well below the
other groups, the reinforcing outcome was either accuracy on each training task, which was in the
always the same (a mixture of peas and wheat) or neighborhood of 90% correct (data not shown).
random or uncorrelated with respect to the sample However, the fact that peas and wheat share the com-
stimuli (e.g., peas for a correct response after S1 on mon feature that both are food and both are familiar
some trials, but wheat for the same correct response to pigeons may have diminished the chance of a
after S1 on other trials, and likewise for the correct strong partitioning of the stimuli with which they are
responses following the remaining three samples). associated. Besides, if no partitioning whatsoever
In testing, the samples from the color identity- occurred, test session accuracies in the correlated
matching task were swapped with the samples from group should have been comparable to those in the
the shape identity-matching task such that pigeons two control groups, which was clearly not the case.
now responded to shape comparisons after color Maki, Overmier, Delos, and Gutmann (1995,
samples and to color comparisons after shape sam- Experiment 3) reported transfer results very similar
ples. For the correlated group, the correct (rein- to these in typical 5-year-old children. In their
forced) responses in testing were those associated experiment, each child learned two separate arbi-
with the two sample stimuli that had shared the trary matching tasks (viz., tasks in which the com-
same reinforcing outcome during training. For parison alternatives were physically dissimilar to the
example, if the red comparison response (R1) after sample stimuli). For two children, a particular col-
the red sample (S1) produced peas (O1) in one ored chip (e.g., blue) was delivered after a correct
training task and the plus comparison (R3) response following one sample in each task, and a
response after the plus sample (S3) also produced different colored chip (e.g., red) was delivered after
peas (O1) in the other training task, then a red com- a correct response following the other sample in
parison response (R1) after the plus sample (S3) in each task. Each colored chip was later exchangeable
testing was correct and reinforced with peas (O1). for small toys versus food. Thus, the samples in
Stated otherwise, the class-consistent comparison each training task were associated with different
responses were reinforced with their associated reinforcing outcomes. For two other children, every
369
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
reinforced comparison response yielded the same

colored chip (white), also exchangeable for either
toys or food (a nondifferential condition). Later, the
samples from one task were swapped for those of
the other, and vice versa, on nonreinforced test trials
to see whether those samples would preferentially
occasion particular comparison responses of the
other task. The expectation was that in the differen-
tial condition (cf. Table 16.3), the samples would
occasion class-consistent (correct) responding in
testing. For comparison, test-trial responses in the
nondifferential condition were deemed correct by
correspondence to those same responses in the dif-
ferential group.
The bottom half of Figure 16.4 shows the average
test results for each differential (D) and nondifferen-
tial (ND) child. The above-chance levels of accuracy
exhibited by the D children demonstrate transfer of
control indicative of stimulus class formation. The
chance levels of accuracy exhibited by the ND chil-
dren show that the transfer effects seen in the D
children were, indeed, the consequence of the com-
mon outcome associations in training. These effects
have been systematically replicated in a wide variety
of experiments using rats (Kruse, Overmier, Konz,
& Rokke, 1983), pigeons (Meehan, 1999; Peterson,
Figure 16.4. Stimulus class (transfer-of- 1984; Urcuioli, 1990), children (Schenk, 1994), and
control) test results after common outcome adults with mental retardation (e.g., Dube, McIl-
training. Top: Averaged percentage of rein- vane, Maguire, Mackay, & Stoddard, 1989).
forced class-consistent choices (±1 standard
error of the mean) over the first two test ses- Response-independent (Pavlovian) outcome
sions for six pigeons trained with pea and associations are also effective in partitioning stimuli
wheat reinforcers correlated with pairs of into classes (e.g., Honey & Hall, 1989; Shipley,
sample stimuli in training (Corr.) and the 1935). In one study, Hall et al. (2003, Experiment
corresponding reinforced choices (±1 stan-
dard error of the mean) for pigeons initially 1) had university students observe individual shape–
trained with a reinforcer mixture (Same) or color sequences on a computer monitor. During the
with peas and wheat uncorrelated with the 32-trial training phase, two of the shapes were
sample stimuli (Uncorr.). Data from Edwards,
Jagielo, Zentall, and Hogan (1982, Experiment
always followed by one colored rectangle, and the
2, Figure 5). Bottom: Percentage of class- other two shapes were always followed by a different
consistent (correct) choices averaged over colored rectangle (see Figure 16.5). In a subsequent
32 nonreinforced test trials for two children stimulus class test, each subject learned to make a
initially trained with different (D) outcomes
correlated with pairs of sample stimuli or with left versus right keyboard response to each shape
those outcomes occurring nondifferentially stimulus (with feedback for correct and incorrect
(ND) with respect to the samples. Data from responses). For some subjects, each correct response
Maki, Overmier, Delos, and Gutmann (1995, was mapped onto shapes that shared the same color
Experiment 3, Figure 7).
(outcome) association in training (consistent condi-
tion); for the remaining subjects, each spatial
response was mapped onto shapes that had different
370
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Training Stimulus-Class Test 1 Stimulus-Class Test 2

(Consistent)
Left R +
Left R +
Right R +
Red
Right R +
Red Red Left vs. Right R
or
Green (Inconsistent) Green Left vs. Right R
Left R +
Green
Right R +
Left R +
Right R +
Figure 16.5. A schematic depiction of the training contingencies and the

reinforced (Test 1) and nonreinforced (Test 2) transfer-of-control tests in Hall,
Mitchell, Graham, and Lavis (2003, Experiment 1). R = response.
color associations in training (inconsistent condi- response for each syllable was the same as (rather
tion). As predicted by outcome-based partitioning of than the opposite of) the reinforced response to
shapes, average accuracy was higher in the consis- the shapes associated with that syllable. The latter
tent than in the inconsistent condition: 96.4% ver- results support the idea that the syllables (outcomes)
sus 80.2% correct, respectively. In a second stimulus were either members of the stimulus classes them-
class test, subjects were given the opportunity to selves or evoked by them.
make left versus right keyboard presses (without
feedback) to the outcomes themselves. If the out-
Reversal Test Assessments
comes had directly or indirectly become members of
the partitioned classes (see also Joseph, Overmier, & Another way to assess partitioning of stimuli into
Thompson, 1997), then given the motor responses classes is to conduct reversal tests after common-
learned during the first stimulus class test, consis- response or common-outcome training. The idea
tent subjects were predicted to make the same left behind these tests is that if the response occasioned
versus right responses to the outcome colors as they by, or the outcome signaled by, one or more stimuli
had to their associated shapes—which they did, in a class is explicitly reversed vis-à-vis original
96.9% of the time. By contrast, inconsistent subjects training, that change should propagate through the
were predicted to respond haphazardly—and did. other class members. Propagation is operationally
Hall et al. (2003, Experiment 3) replicated these defined as an immediate and appropriate reversal of
results using nonsense syllables as the outcomes fol- the behavior originally occasioned by those other
lowing the shape stimuli. Specifically, subjects were class members.
more accurate in their left versus right keyboard Von Fersen and Delius (2000) provided an
responses when the same motor response was example of this by successively reversing a four-
required to shapes previously associated with the stimulus auditory discrimination in two dolphins. A
same (as opposed to a different) nonsense syllable. schematic of their procedure is shown in Table 16.4.
Furthermore, when the consistent group from this During initial training, one spatial response (left)
first stimulus class test (cf. Figure 16.5) was then was reinforced (indicated by a plus sign) after each
required to learn the left versus right responses to of two different tonal frequencies (S1 and S3), and
the nonsense syllables themselves, the discrimina- another spatial response (right) was likewise rein-
tion was more accurate when the reinforced forced after each of two other tonal frequencies
371
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
Table 16.4
Reversal Test Procedure Used by Von Fersen

and Delius (2000) to Assess Stimulus-Class
Formation on the Basis of Partitioning by Common
Reinforced Responses
Reversal test
Common response
training Leading pair Trailing pair
S1 → Left+
S2 → Right+ S1 → Right+ S3 → Left vs. Right
S3 → Left+ S2 → Left+ S4 → Left vs. Right
S4 → Right+ Figure 16.6. Reversal-test results over the
Hypothesized stimulus classes last seven reversals for two dolphins after
common-response training. Open bars show
S1 S2
average accuracy for the last eight leading trials
of the reversals. Black bars show average accu-
S3 S4 racy for the first four no-feedback trailing tri-
als of the reversals. Data from von Fersen and
Delius (2000).
Note. S1–S4 represent different stimuli, and a plus sign
indicates reinforced responses. Italicized responses (filled bars) averaged over the last seven reversals.
represent class-consistent responding assuming the
two hypothesized stimulus classes shown as the circled Leading-pair accuracies were approximately 75%
elements. correct, which may seem low but is somewhat artifi-
cial, reflecting the fact that at the outset of a reversal
(S2 and S4). (Incorrect responses produced a loud session, dolphins had no way of knowing that their
whistle.) After accuracy reached 80% correct or bet- previously reinforced (correct) responses were no
ter, these contingencies were reversed by (a) rein- longer correct (i.e., were now errors). Thus, they
forcing the opposite spatial response to just one would make one or more errors during the 10 leading-
stimulus in each common-response set for 10 trials pair trials that began a reversal session before
(e.g., S1 and S2; viz., the so-called “leading pair” of switching their responses in accordance with the
stimuli), after which (b) the remaining two stimuli reversal contingencies. More noteworthy is the
(e.g., S3 and S4; viz., the so-called “trailing pair”) much higher levels of accuracy (82% and 89% cor-
were presented for another 10 trials. More impor- rect) for the no-feedback trailing-pair trials, a result
tant, the first three trailing-pair trials ended without consistent with stimulus class formation.
any feedback (i.e., without reinforcement or the Kastak, Schusterman, and Kastak (2001, Experi-
loud whistle), so von Fersen and Delius could mea- ment 1) found similar reversal effects in two sea
sure how the explicitly taught leading-pair reversal lions trained on sets of simultaneous (S+ vs. S−)
affected the dolphin’s subsequent responses to the discriminations, each of which involved a choice
trailing pair uncontaminated by any new learning between a letter and a number, using 10 different
with the latter pair. Because the first four trailing- letters and 10 different numbers. In one experimen-
pair choices were made before any feedback was tal phase, letters were the S+ (reinforced) stimuli,
obtained on those trials, these four trials were used and numbers were the S− (nonreinforced) stimuli
to assess stimulus class formation. for an initial block of trials in a session. After achiev-
Von Fersen and Delius (2000) conducted 14 ing at least 90% accuracy during this initial block,
such reversals. Figure 16.6 shows the accuracy of the S+ and S− roles of the stimuli were reversed
each dolphin for the last eight leading-pair trials for a subsequent block of trials. Moreover, each
(open bars) and the first four trailing-pair trials reinforced letter choice produced one type of fish
372
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
(herring), whereas each reinforced number choice match each stimulus in a class to any other stimulus
produced another type (capelin), and a total of 40 in the same class (Spradlin & Saunders, 1986). The
within-session reversals were conducted to assess category sorting or category matching-to-sample
class partitioning by common outcomes. Over the task used by Lowe et al. (2002), Horne et al. (2007),
last 10 reversals, the sea lions were 82% and 72% and Miguel et al. (2008) provides precisely this sort
correct overall to the 10 S+ stimuli on their first of assessment and, as indicated earlier, has yielded
presentation after a reversal. (The idealized maxi- results that confirm this expectation in young chil-
mum percentage correct was 90% correct because dren. Sidman et al. (1989) also observed highly
the first reversal trial could not be anticipated.) In accurate matching of this type in both a typically
short, after one class member (letter or number) was capable adult and an adult with intellectual chal-
switched from an S+ to an S−, and vice versa, the lenges (see also Dube, McIlvane, Mackay, & Stod-
others followed. dard, 1987; Dube et al., 1989).
Interestingly, the effect obtained by Kastak et al. To conduct these assessments, researchers pres-
(2001) required different reinforcing outcomes for ent the sample to be matched at one spatial location
the two sets of S+ stimuli. In other words, common and the comparison alternatives from which a
association with just the presence versus absence of matching selection or selections are made at other
reinforcement per se was insufficient to induce class spatially distinct locations. Successful (class-
partitioning. By contrast, in successive discrimina- consistent) performances generally require that sub-
tion reversal experiments with pigeons (Vaughan, jects attend solely to the nominal features of the
1988), adult humans (Sidman, Wynne, Maguire, & stimuli in question and ignore their ostensibly irrel-
Barnes, 1989), and a chimpanzee (Tomonaga, evant features (such as where they appear and when
1999), a within-group association with reinforce- they appear). At first glance, this consideration
ment versus nonreinforcement is sufficient to yield would seem to be inconsequential, but it may not be
reversal effects indicative of stimulus class forma- (Dzhafarov & Colonius, 2006). When humans or
tion. For example, Vaughan (1988) randomly other animals do not behave in a manner consistent
divided 40 different photographs of trees into two with stimulus classes, it might reflect a lack of corre-
groups and reinforced pecking to any stimulus in spondence between the stimulus features that actu-
one (S+) group; pecking to any stimulus in the ally control their behavior and what the researcher
other (S−) group was not reinforced. After pigeons mistakenly believes those features to be. Stated less
learned this discrimination (viz., by pecking at high charitably, the apparent absence of stimulus class
rates to the S+ stimuli and very low rates to the S− formation in a particular situation may reflect con-
stimuli), the S+ stimuli became the S− stimuli, and ceptual shortcomings on the part of the researcher
vice versa. Once this reversal was learned, the dis- rather than conceptual limitations on the part of the
crimination was again reversed, and this was subject (McIlvane, Serna, Dube, & Stromer, 2000).
repeated more than 120 times. Vaughan found that Consequently, understanding the functional stimu-
with successive reversals, seeing just a few photo- lus (viz., what actually influences the subject’s
graphic stimuli at the beginning of a reversal session behavior) is of paramount importance, especially
was sufficient for pigeons to respond appropriately when working with nonhuman animals.
(viz., in a reversed fashion) to most of the remaining To illustrate this point, the top portion of Figure
slides in that session. 16.7 depicts an arbitrary matching training task in
which pigeons see the sample stimuli on the center
key of a three-key display. After the pigeon pecks
Other Stimulus Class Assessments
the sample, it goes off and two comparison alterna-
and the Functional Stimulus
tives appear simultaneously on the left and right
If explicit training or learning experiences generate keys (with side-key location counterbalanced across
classes of interchangeable stimuli, a reasonable trials; not shown). In this example, pecking the
expectation is that the subject should be able to vertical-lines comparison is reinforced (+) after the
373
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
Zentall, 1977; Lionello-DeNolf & Urcuioli, 2002;

Lipkens, Kop, & Matthijs, 1988; Sidman et al., 1982;
but see Schusterman & Kastak, 1993, and McIntire,
Cleary, & Thompson, 1987, for exceptions). Figure
16.8 shows data from experiments with monkeys,
baboons, and pigeons in which none exhibited
above-chance levels of accuracy on a symmetry test
(solid bars) despite highly accurate baseline perfor-
mances on the arbitrary matching training task
(open bars).
Notice that in Figure 16.7, I have labeled the
members of the possible classes nominally: red (R),
vertical lines (V), green (G), and horizontal lines
(H). However, these stimuli may not be functional
for the animal. After all, in arbitrary matching train-
ing, red and green only appeared on the center key
Figure 16.7. A schematic depiction of training and vertical and horizontal only appeared on the left
contingencies for two-alternative arbitrary match-
ing using red (R) and green (G) samples and vertical and right keys. Perhaps the stimuli for the pigeon
(V)-line and horizontal (H)-line comparisons and include location. If so, the functional samples would
symmetry test trials in which the roles of the colors be red-on-the-center-key and green-on-the-center-
and lines as samples and comparisons are reversed. key, and likewise, the functional comparisons would
The circled elements depict two possible stimulus
classes resulting from training. be vertical-on-the-left (or right)-key and horizontal-
on-the-left (or right)-key. Arbitrary matching train-
red sample and pecking the horizontal-lines comparing, then, would result in learning to match
ison is reinforced (+) after the green sample. Again, red-on-the-center-key to vertical-on-the-left (or
this is called arbitrary matching because the samples right)-key and green-on-the-center-key to horizontal-
have no obvious physical resemblance to either of on-the-left (or right)-key. However, if this is an
the comparisons. Nevertheless, the task is readily accurate portrayal of what is learned, then the osten-
learned; the question of interest is what is learned? sible symmetry test is nothing of the sort because it
One possibility is that pigeons learn that red and does not involve the necessary interchanges of stim-
vertical go together, as do green and horizontal—in uli (viz., seeing whether pigeons will match vertical-
words, that the color and line of each reinforced pair on-the-left [or right]-key to red-on-the-center-key).
are members of a class, which is indicated by the cir- Instead, the test involves novel stimuli (viz., vertical-
cled elements shown on the right side of Figure on-the-center-key, horizontal-on-the-center-key,
16.7. If so, then pigeons should now preferentially red-on-the-left [or right]-key, and green-on-the-left
peck a red comparison when presented with a verti- [or right]-key). Consequently, there is no reason to
cal sample and a green comparison when presented expect symmetry, and the data shown in Figure 16.8
with a horizontal sample, as indicated at the bottom amply confirm this expectation.
of Figure 16.7. In short, the explicitly taught red → Of course, if the functional matching stimuli
vertical lines and green → horizontal lines condi- include their location, then after learning two-
tional relations in training should be symmetrical alternative arbitrary matching (cf. Figure 16.7), sub-
(Sidman, 2000, 2008). jects should also be unable to perform accurately if
Curiously, symmetry—one of three behavioral the familiar red and green samples now appear on
indices of an equivalence relation (Sidman & Tailby, the left or the right key (with the familiar compari-
1982)—has been very difficult to demonstrate in sons on the center and remaining side key) because
nonhuman animals (e.g., D’Amato, Salmon, Loukas, this change, too, generates novel stimuli. This is pre-
& Tomie, 1985; Dugdale & Lowe, 2000; Hogan & cisely what happens (Lionello & Urcuioli, 1998):
374
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 16.9. The average percentage of cor-

rect choices in two-alternative matching by
pigeons when the sample stimuli appear in their
normal, center-key training location and in a
novel, side-key test location. Data from Lionello
and Urcuioli (1998).
only by the nominal attributes of the matching stim-

uli (i.e., location is ignored) or (b) no stimulus
changes location in the shift from training to testing.
Lionello-DeNolf and Urcuioli (2000) accomplished
Figure 16.8. Arbitrary matching train- the former by varying the locations of the samples
ing results (open bars) and symmetry test and the comparisons throughout training. The sam-
results (solid bars) for individual monkeys ple appeared on either the left or the right key on
and baboons (top) and pigeons (bottom). different trials with the comparisons then appearing
Top panel: Data from Sidman et al. (1982).
Bottom panel: From “A Test of Symmetry and on the remaining two keys. After this multiple-
Transitivity in the Conditional Discrimination location task was acquired, the samples were then
Performances of Pigeons,” by R. Lipkens, shown for the very first time on the center key (and
P. F. M. Kop, and W. Matthijs, 1988, Journal
of the Experimental Analysis of Behavior, 49,
the comparisons on the two adjacent side keys).
p. 405. Copyright 1988 by the Society for the Despite the change in location, most pigeons contin-
Experimental Analysis of Behavior. Adapted ued to perform at high levels of accuracy (i.e., they
with permission. ignored location).
However, Lionello-DeNolf and Urcuioli (2002)
Changing the location at which the familiar samples subsequently showed that despite multiple-location
and comparisons appear causes pigeons’ matching arbitrary matching training and verification that
accuracies to drop to chance levels (see Figure pigeons ignored stimulus location, they still did not
16.9). Similar results have been reported for mon- exhibit symmetry in testing. The same (null) result
keys and for rats (Iversen, 1997; Iversen, Sidman, & was obtained even when pigeons received additional
Carrigan, 1986; see also Sidman, 1992). In short, a multiple-location training to ensure that each stimu-
stimulus consistently appearing in one location may lus was seen in its eventually tested location.
not function as the same stimulus when later shown Clearly, the culprit here is not simply location, and
in a different location (Urcuioli, 2007; see also the same appears to be true for monkeys and
Urcuioli, 2008c). baboons as well (Sidman et al., 1982). That said, it
A valid symmetry test requires that either (a) would be a mistake to conclude that researchers
matching performances in training are controlled interested in stimulus class formation need not
375
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
worry about the location at which stimuli appear The upper left section of Figure 16.10 illustrates
because it may introduce an unwanted source of arbitrary successive matching training used by
stimulus control if correlated with the nominal stim- Urcuioli (2008b). The individually presented sam-
uli (cf. McIlvane et al., 2000). ples (red [R] and green [G]) are shown on the left;
Other potentially undesirable sources of stimulus to their right are the individually presented compar-
control are when each stimulus appears and whether isons (a triangle [T] and a set of horizontal stripes
it appears by itself or with other stimuli (Saunders & [H]). Some sample–comparison combinations (viz.,
Green, 1999). Regarding the latter, just because sub- R → T and G → H) ended in reinforcement, mean-
jects accurately select a particular comparison from ing that the first comparison stimulus peck after,
among other concurrently presented comparisons say, 5 seconds on these trials produced food. The
(i.e., simultaneously discriminate between them) remaining combinations (viz., R → H and G → T)
does not mean they will appropriately differentiate ended without reinforcement; the comparison sim-
between those same stimuli when each appears indi- ply went off after 5 seconds. Pigeons also received
vidually (i.e., successively discriminate between concurrent training on two other successive match-
them), and vice versa. Regarding the former, the ing tasks, hue and form identity, shown in the mid-
sample stimuli appear first in a trial with the com- dle and upper right sections of Figure 16.10.
parisons following them, so their respective ordinal Reinforcement in these tasks occurred only on trials
positions (first and second) might very well consti- in which the comparison physically matched the
tute part of the functional matching stimuli. Indeed, preceding hue (or form) sample. The original ratio-
both Pavlovian and operant conditioning provide nale for including the latter tasks (Frank & Wasser-
evidence that animals are highly sensitive to ordinal man, 2005) was to familiarize pigeons with each
cues and other forms of temporal information (e.g., stimulus in each ordinal position (i.e., as both a
Balsam & Gallistel, 2009; D’Amato & Colombo, sample and a comparison) in preparation for the
1988; Honig, 1981; Miller & Barnet, 1993; subsequent symmetry test. That test, shown in the
Terrace, 1986). bottom left section of Figure 16.10, involved pre-
These considerations are germane to recent suc- senting T and H first (viz., as samples) and R and G
cessful demonstrations of symmetry in pigeons second (viz., as comparisons).
(Frank & Wasserman, 2005; Urcuioli, 2008b) using The infrequent symmetry probe trials were inter-
successive or go/no-go matching (Wasserman, 1976; mixed among the training trials on the three already
see also Konorski, 1959). Successive matching dif- learned tasks (Urcuioli, 2008b). Each test probe
fers from the n-alternative procedure in that (a) only ended without reinforcement (to avoid new learning
one comparison is presented after a sample and on these trials) and was designated positive or nega-
(b) the single comparison appears at the same loca- tive depending on whether it reversed a reinforced
tion as the preceding sample, which avoids any or a nonreinforced arbitrary training combination,
changes in stimulus location when shifting from respectively. If training had generated stimulus
training to testing and ensures the same type of classes consisting of the elements of each reinforced
required discrimination for both samples and com- combination—[R, T] and [G, H]—these elements
parisons (viz., successive). The task involves rein- should be interchangeable with one another and
forcing responding to one comparison after one yield symmetrical responding. In other words, more
sample but not after the other, and vice versa for responding should occur to the R and G compari-
responding to the alternative comparison. Each sons when preceded by the T and H comparisons,
comparison is usually presented for an extended respectively, on symmetry probe trials.
period of time (e.g., 5 or 10 seconds) so that the rate In fact, this is precisely what was found (Urcui-
of responding to it (the number of comparison oli, 2008b, Experiment 3; see also Frank & Wasser-
responses per second) can be measured. With train- man, 2005, Experiment 1). Figure 16.11 shows
ing, subjects confine more and more of their com- probe data for three pigeons averaged over their first
parison responses to the reinforced (go) trials. two test sessions along with their arbitrary matching
376
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 16.10. Successive matching training on arbitrary matching,

hue identity matching, and form identity matching, and the four symme-
try probe trials in which the arbitrary matching sequences are reversed
(cf. Urcuioli, 2008b). Samples and comparisons are shown to the left
and right of the arrows, respectively; + and − indicate reinforced and
nonreinforced trials, respectively; and pos and neg denote the reverse of
the reinforced and nonreinforced arbitrary matching trials, respectively.
R = red; G = green; pos = positive; neg = negative.
(training) data from those same sessions. The the nonreinforced combinations as frequently as they
explicitly trained performances were, of course, did early in training when they pecked equally on all
highly discriminative: Pigeons responded much trials. By contrast, the relative frequency of nonrein-
more to the comparisons on reinforced (positive) forcement in n-alternative matching diminishes sub-
than on nonreinforced (negative) trials. More note- stantially as subjects learn the task.
worthy is the finding that they also behaved in a I have proposed (Urcuioli, 2008b) that the
highly discriminative fashion on the symmetry test equally frequent nonreinforcement of certain
trials. Specifically, comparison-response rates were sample–comparison combinations juxtaposed with
much higher on probes that were the reverse of the the reinforcement of other combinations facilitates
positive training trials than on probes that were the the formation of stimulus classes whose members
reverse of the negative training trials. This, then, is are the elements of reinforced combinations. Refer-
another example of stimulus class formation. ring back to Figure 16.10, one such class would
Given so many past failures to demonstrate sym- contain the R sample and T comparison (because
metry in nonhuman animals using the n-alternative this combination is reinforced), and another class
choice procedure, why did the successive matching would contain the G sample and the H comparison
version succeed? Presenting all stimuli singly and at (because this combination is also reinforced). Note,
the same spatial location certainly helped. More too, that this means that the R sample and H com-
important, perhaps, is that half of all successive parison, and the G sample and the T comparison,
matching training trials end in nonreinforcement no are in different classes, as they should be because
matter what the subject’s level of performance is. these combinations are never reinforced.
Even if pigeons peck the comparisons only on rein- However, this cannot be the entire story because
forced trials and never otherwise, they still experience this assumption alone predicts that arbitrary successive
377
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
positions. Interestingly, symmetry does not emerge

from such training (Frank, 2007). Clearly, some-
thing else is going on here besides ordinal
familiarity.
That something else brings us back to a consider-
ation of the functional matching stimuli. Appar-
ently, they are not just the nominal stimuli
themselves. Rather, pigeons apparently code each
successive matching stimulus in terms of “What did
I see?” and “When did I see it?”—the nominal stim-
ulus plus its ordinal position. Thus, the R sample is
just that: namely, red in the first ordinal position
(R1). Likewise, the T comparison is triangle in the
second ordinal position (T2), and so forth. Conse-
quently, the elements of the stimulus classes arising
from arbitrary matching (cf. Figure 16.10) should
not be represented as [R, T] and [G, H] but rather as
[R1, T2] and [G1, H2]. For hue identity matching,
the corresponding classes should be [R1, R2] and
[G1, G2]; for form identity matching, they should
be [T1, T2] and [H1, H2]. The reason I have itali-
cized certain elements is to draw your attention to
the fact that some classes share a common member.
(The same is true for the classes involving G and H,
but I have omitted italics for clarity.)
If common members cause their respective stim-
ulus classes to “integrate” or merge (Balsam & Gal-
Figure 16.11. Comparison-
listel, 2009; Sidman, Kirk, & Willson-Morris, 1985),
response rates (in pecks per sec-
ond) on the arbitrary matching the net result of the training depicted in Figure
training trials and for the symme- 16.10 is the two four-member classes shown in the
try probe trials (cf. Figure 16.10) top part of Figure 16.12. The solid lines indicate the
over the first two symmetry test
sessions. The data are from two explicitly trained and reinforced sample–comparison
pigeons (EXT 2 and EXT 7) run in combinations from arbitrary matching (red sample–
Urcuioli (2008b) and one pigeon triangle comparison, or R1 → T2, and green
(PRF4) run in Urcuioli (2008a). sample–horizontal comparison, or G1 → H2). The
dotted lines indicate the two probe relations that
matching training by itself should generate the stim- demonstrate the observed symmetry effect: greater
ulus classes necessary for symmetry—and it does comparison responding to the triangle sample–red
not (Richards, 1988; see also Frank, 2007). After all, comparison (T1 → R2) and the horizontal sample–
successful training (cf. Figure 16.10) appears to green comparison (H1 → G2) probe combinations
require concurrent hue and form identity training, than to the other sample–comparison test
ostensibly to avoid probe–trial combinations in combinations.
which the stimuli appear for the very first time in Are the assumptions about ordinal position and
novel ordinal positions. However, there are other class merger valid? One way to find out is to modify
ways to accomplish this—for example, by training training such that hue oddity rather than hue iden-
two other arbitrary matching tasks in which pigeons tity serves as one of the three concurrent training
see the eventually tested stimuli in their ordinal test tasks. In hue oddity, comparison responding is
378
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
merged four-member classes that should result from

using hue oddity as one of the three successive
matching training tasks. The odd reinforced training
elements can be seen in each class: the green com-
parison (G2) after the red sample (R1) in the class
on the left, and the red comparison (R2) after the
green sample (G1) in the class on the right. Note,
too, that the elements from the reinforced arbitrary
matching relations (connected by the solid lines) are
the same as those in the stimulus classes from train-
ing that involved hue matching (cf. top part of Fig-
ure 16.12). With hue oddity training, however, the
form sample and hue comparison elements repre-
senting the reverse of the reinforced training rela-
tions (e.g., T1 and R2) are not in the same class.
Figure 16.12. Top: The two four-member
stimulus classes hypothesized to result from
Rather, each class contains the elements represent-
the concurrent successive matching training ing the reverse of the nonreinforced arbitrary rela-
depicted in Figure 16.10. Bottom: The two tions (e.g., T1 and G2); hence, the prediction of
four-member stimulus classes hypothesized antisymmetry.
to result from the concurrent successive
matching training involving hue oddity Figure 16.13 shows data that confirm this coun-
rather than hue matching. Solid lines con- terintuitive prediction (Urcuioli, 2008b, Experiment
nect the elements of the reinforced 4). This figure presents probe data averaged over the
sample–comparison sequences from arbi-
first two sessions for three pigeons trained with hue
trary matching. Dotted lines connect the
elements representing the symmetrical oddity rather than hue identity as one of the three
versions of either the reinforced arbitrary successive matching training tasks. Also plotted are
matching sequences (top) or the nonre- their arbitrary matching (training) data from those
inforced arbitrary matching sequences
(bottom). R = red; G = green; T = triangle; same test sessions. Notice that each pigeon
H = horizontal stripes. responded far more frequently on probe trials that
were the reverse of the nonreinforced (negative)
reinforced only when the hue comparison mis- training trials.
matches the prior hue sample. Thus, responding to I want to emphasize two things before ending
the green comparison (G2) is reinforced after the this rather lengthy section. The first is that distinc-
red sample (R1) and responding to the red compari- tive outcomes are effective in partitioning stimuli
son (R2) is reinforced after the green sample (G1). associated with them into separate classes, a point I
This task also familiarizes subjects with each hue at made and illustrated in the section Stimulus Classes:
each ordinal position, so on that account, symmetry Partitioning by Common Outcomes earlier in this
should still be observed. By contrast, if the func- chapter. The regular reinforcement of certain
tional matching stimuli contain ordinal information, sample–comparison combinations versus equally as
the exact opposite (antisymmetry) should be regular nonreinforcement of other combinations
observed: Fewer comparison responses should occur throughout successive matching training generates
on probe trials that reverse the reinforced arbitrary classes containing the elements of a reinforced com-
matching training trials. Stated otherwise, more bination (see also Vaughan, 1988) and permitting
comparison responses should occur on probe trials the eventual emergence of new behavior. Second,
that reverse the nonreinforced arbitrary matching although it would be easy for the reader to leave
training sequences. with the message that some training procedures are
The bottom part of Figure 16.12 provides a conducive to stimulus class formation and others
visual rationale for this prediction by showing the are not when working with nonhuman animals, the
379
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
aware, their behavior becomes understandable. The

same holds true in research with nonhuman ani-
mals. To make sense of their behavior (including
why they do or do not exhibit evidence of stimulus
class formation), one must know the features of the
environment to which they attend. This is another
way of saying that researchers must identify the
functional stimuli. With nonverbal creatures,
researchers cannot (obviously) ask for that informa-
tion, so the challenge is to find other ways for them
to provide that information. This exercise can be
lengthy and sometimes daunting, but it is also a very
rewarding one when its success helps to reveal inter-
esting and theoretically important phenomena.
Concluding Comments
One of the stimulating and exciting aspects of work
in the area of stimulus control is finding that certain
types of training and learning experiences yield
novel instances of stimulus control. In particular,
otherwise familiar stimuli begin to occasion behav-
ior never explicitly reinforced in their presence
(viz., emergent behavior). Another way to say this is
some forms of stimulus control are derivative: They
are untrained, arising from other explicitly trained
stimulus control relations. This is no small matter:
It means that learning a small number of relations
can greatly expand the potential for adaptive behav-
Figure 16.13. Comparison- ior far beyond the behavior that was explicitly
response rates (in pecks per second)
for three pigeons (ODD2, ODD5, and trained.
ODD8) on the arbitrary matching Besides what I have just said, another take-home
training trials and for the symmetry message concerns the nature of the training and
probe trials over the first two sym-
learning experiences that produce this expansion.
metry test sessions after successive
matching training using hue oddity Specifically, physically dissimilar stimuli are treated
rather than hue identity as one of the as belonging together—they become interchange-
training tasks. able for one another—if subjects learn the same, dif-
ferentially reinforced response to them (e.g., giving
main message I wish to convey is why that is so. them the same name or making the same spatial or
One reason has just been mentioned (viz., the comparison response to them) or if they are associ-
impact of differential outcomes). Another, more ated with a distinctive outcome. In other words,
important one has to do with identifying the actual stimuli associated with a common reinforced
stimuli controlling behavior. Everyone has probably response or common outcome become members of
had the experience, at one time or another, of react- the same category or class. Although not especially
ing in unexpected ways and, when asked why, newsworthy to researchers in the field, this message
claiming to pay attention to certain things others bears repeating and is an important one to convey to
were not fully aware of. When people are made those unfamiliar with the supporting data.
380
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
A third take-home message is that identifying the For pigeons and other animals (e.g., Balsam &
origins of stimulus class formation presupposes that Gallistel, 2009; Miller & Barnet, 1993), this can be
researchers know exactly what the class members an important component of stimulus control. Is the
consist of. Are they simply the nominal stimuli? Or same true for humans? Do people naturally code
are they more complex versions that include fea- stimuli in terms of when as well as in terms of
tures such as a stimulus’s spatial or temporal loca- what? The notion of episodic memory, remember-
tion? If researchers do not know or are mistaken ing specific past events in people’s lives (Tulving,
about the nature of the functional stimuli, they run 1983; see also Zentall, 2006), suggests that they do.
the risk of underestimating the capabilities of their Remembering when asked to do so, however, does
subject—for example, by concluding that they do not mean that performances in tasks of the sort
not categorize when, in fact, they do or can. “Know described in this chapter are necessarily affected by
the functional stimulus” is an especially important the temporal or ordinal aspects of stimuli. To the
message because it is so easily overlooked, some- contrary, emergent effects such as symmetry often
thing I have experienced firsthand. Stimulus control displayed by humans in studies of stimulus class
has long been my primary area of research, yet I formation suggest that those aspects are essentially
have on occasion devoted unnecessarily large ignored. However, the failure of some subjects to
amounts of time to understanding certain results show these effects (e.g., Sidman et al., 1982, Experi-
(e.g., Urcuioli, 2006b, 2008c) that would have been ment 3) could mean that for them, these aspects
far less time consuming had I listened to my own are part of the functional stimuli (cf. McIlvane
message and had a less anthropocentric view of the et al., 2000).
nature of the functional stimuli. Answers to these questions must await future
The ability of my recent theory of stimulus class research. “Know the stimulus,” or at least being cog-
formation (Urcuioli, 2008b) to correctly predict nizant of what the functional stimulus might consist
emergent effects originally thought by some to be of, should provide a helpful guide. In my lab, it has
unique to humans hinges on the proper identifica- already generated a considerable number of new
tion of the functional stimulus. Although theoretical ideas for developing adequate tests in nonhuman
relevance may appear to be just an intellectual exer- animals for other emergent phenomena associated
cise (albeit an important one; cf. Skinner, 1935), with stimulus class formation—for example, reflex-
knowing the functional stimulus has important ivity (the untrained ability to match a stimulus to
practical applications, too. For instance, some meth- itself) and transitivity (the untrained ability to match
ods used to teach or to retrain basic language and A to C after learning to match A to B and B to C).
symbolic skills often consist of procedures for pro- I opened this chapter with quotations I thought
moting stimulus class formation (e.g., Cowley, were apropos to my subsequent discussion of cate-
Green, & Braunling-McMorrow, 1992; McIlvane, gorization and stimulus class formation. Perhaps
1992). Their success depends heavily on accurate it would be a fitting (and symmetrical—no pun
stimulus definition. Variations in the outcomes of intended) end to this chapter to provide a few other
these interventions may reflect unrecognized or relevant remarks:
incidental sources of stimulus control: features of
The ability to form concepts is surely
stimuli that from experimenter’s perspective appear
adaptive because concepts allow us to
irrelevant or inconsequential but from the subject’s
respond appropriately to . . . stimuli
perspective are quite the opposite (McIlvane et al.,
after having experience with only a few
2000). Consistently effective interventions will be
instances from a given category. (Laza-
achieved only by identifying these sources of stimu-
reva & Wasserman, 2008, p. 197)
lus control and, if necessary, neutralizing them.
In the preceding section of this chapter, I dis- Human language may depend on cat-
cussed the impact of ordinal information—when a egorization . . . but categorization does
particular stimulus occurs in a sequence of stimuli. not depend on language. . . . Rather than
381
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
psycholinguistics, it is the psychology of References

perception and stimu lus discrimination Balsam, P. D., & Gallistel, C. R. (2009). Temporal maps
that impinge most directly on categoriza- and informativeness in associative learning. Trends in
tion. (Herrnstein, 1984, p. 257) Neurosciences, 32, 73–78. doi:10.1016/j.tins.2008.10.004
Bhatt, R. S., Wasserman, E. A., Reynolds, W. F., Jr.,
The first quote underscores the behavioral econ- & Knauss, K. S. (1988). Conceptual behavior in
pigeons: Categorization of both familiar and novel
omy humans and other animals achieve by their
examples from four classes of natural and artificial
ability to group together disparate objects sharing stimuli. Journal of Experimental Psychology: Animal
some common association. The second reinforces Behavior Processes, 14, 219–234. doi:10.1037/0097-
this notion of adaptiveness by highlighting what 7403.14.3.219
behavioral psychologists have known for quite some Cowley, B. J., Green, G., & Braunling-McMorrow, D.
time: Stimulus class formation is not special to (1992). Using stimulus equivalence procedures
to teach name-face matching to adults with brain
humans. I hope that the reader now appreciates this injuries. Journal of Applied Behavior Analysis, 25,
point and that basic reinforcement and stimulus 461–475. doi:10.1901/jaba.1992.25-461
control processes are often sufficient to account for D’Amato, M. R., & Colombo, M. (1988). Representation
many emergent phenomena. of serial order in monkeys (Cebus apella). Journal of
Research on stimulus class formation in nonhu- Experimental Psychology: Animal Behavior Processes,
14, 131–139. doi:10.1037/0097-7403.14.2.131
man animals gives researchers another opportunity to
see firsthand the mental continuity across species sug- D’Amato, M. R., Salmon, D. P., Loukas, E., & Tomie, A.
(1985). Symmetry and transitivity in the conditional
gested by Charles Darwin (1871/1920) and to appre- relations in monkeys (Cebus apella) and pigeons
ciate its possible evolutionary origins. This type of (Columba livia). Journal of the Experimental Analysis
work is also indispensible if researchers are to eventu- of Behavior, 44, 35–47. doi:10.1901/jeab.1985.44-35
ally identify the psychological lines of fracture Darwin, C. (1920). The descent of man and selection in
between animals that are capable of language and relation to sex (2nd ed.). New York, NY: Appleton.
those that are not and between the variety of the latter (Original work published 1871)
(e.g., Thompson & Oden, 2000; Zentall, Wasserman, Dougher, M. J., & Markham, M. R. (1994). Stimulus
Lazareva, Thompson, & Ratterman, 2008). Finally, equivalence, functional equivalence, and the transfer
of function. In S. C. Hayes, L. J. Hayes, M. Sato, &
researchers who work with nonhuman animals are apt K. Ono (Eds.), Behavior analysis of language and cog-
to label much of the data presented here as evidence nition (pp. 71–90). Reno, NV: Context Press.
of animal cognition. In the introduction to Wasser- Dougher, M. J., & Markham, M. R. (1996). Stimulus
man and Zentall’s (2006a) extensive (35-chapter) classes and the untrained acquisition of stimulus
volume on comparative cognition, they stated that “a functions. In T. R. Zentall & P. M. Smeets (Eds.),
Stimulus class formation in humans and animals
cognitive process is one that does not merely result (pp. 137–152). New York, NY: Elsevier. doi:10.1016/
from the repetition of a behavior or from the repeated S0166-4115(06)80107-X
pairing of a stimulus with reinforcement. Cognitive Dube, W. V., McIlvane, W. J., Mackay, H. A., & Stoddard,
processes often involve emergent (untrained) rela- L. T. (1987). Stimulus class membership estab-
tions” (pp. 4–5). By these criteria, new behavior and lished via stimulus-reinforcer relations. Journal of
new forms of stimulus control that emerge from other the Experimental Analysis of Behavior, 47, 159–175.
doi:10.1901/jeab.1987.47-159
explicitly taught relations, the hallmark of stimulus
class formation, also exemplify animal cognition. Dube, W. V., McIlvane, W. J., Maguire, R. W., Mackay,
H. A., & Stoddard, L. T. (1989). Stimulus class for-
Although behavior analysts often eschew terms such mation and stimulus-reinforcer relations. Journal
as cognition, perhaps they should simply regard such of the Experimental Analysis of Behavior, 51, 65–76.
terminology as recognition that derivative forms of doi:10.1901/jeab.1989.51-65
behavior and stimulus control that frequently occa- Dugdale, N., & Lowe, C. F. (1990). Naming and stimu-
sion these descriptive terms have evolutionary origins lus equivalence. In D. E. Blackman & H. Lejeune
(Eds.), Behaviour analysis in theory and practice:
that transcend humans’ extraordinary language and Contributions and controversies (pp. 115–138). Hove,
inferential capabilities. England: Erlbaum.
382
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Dugdale, N., & Lowe, C. F. (2000). Testing for symmetry Information processing in animals: Memory mecha-
in the conditional discriminations of language-trained nisms (pp. 167–197). Hillsdale, NJ: Erlbaum.
chimpanzees. Journal of the Experimental Analysis of Honig, W. K., & Urcuioli, P. J. (1981). The legacy of
Behavior, 73, 5–22. doi:10.1901/jeab.2000.73-5 Guttman and Kalish (1956): 25 years of research on
Dzhafarov, E. N., & Colonius, H. (2006). Regular mini- stimulus generalization. Journal of the Experimental
mality: A fundamental law of discrimination. In H. Analysis of Behavior, 36, 405–445. doi:10.1901/
Colonius & E. N. Dzhafarov (Eds.), Measurement and jeab.1981.36-405
representation of sensations (pp. 1–46). Mahwah, NJ: Horne, P., & Lowe, C. F. (1997). Toward a theory of ver-
Erlbaum. bal behavior. Journal of the Experimental Analysis of
Edwards, C. A., Jagielo, J. A., Zentall, T. R., & Hogan, D. E. Behavior, 68, 271–296. doi:10.1901/jeab.1997.68-271
(1982). Acquired equivalence and distinctiveness in Horne, P. J., Lowe, C. F., & Harris, F. D. A. (2007).
matching to sample by pigeons: Mediation by Naming and categorization in young children: V.
reinforcer-specific expectancies. Journal of Experi Manual sign training. Journal of the Experimental
mental Psychology: Animal Behavior Processes, 8, Analysis of Behavior, 87, 367–381. doi:10.1901/
244–259. doi:10.1037/0097-7403.8.3.244 jeab.2007.52-06
Eikeseth, S., & Smith, T. (1992). The development of Horne, P. J., Lowe, C. F., & Randle, V. R. L. (2004).
functional and equivalence classes in high function- Naming and categorization in young children: II.
ing autistic children. Journal of the Experimental Listener behavior training. Journal of the Experimental
Analysis of Behavior, 58, 123–133. doi:10.1901/ Analysis of Behavior, 81, 267–288. doi:10.1901/
jeab.1992.58-123 jeab.2004.81-267
Frank, A. J. (2007). An examination of the temporal and spa- Hull, C. L. (1939). The problem of stimulus equivalence
tial stimulus control in emergent symmetry in pigeons. in behavior theory. Psychological Review, 46, 9–30.
Unpublished doctoral dissertation, University of doi:10.1037/h0054032
Iowa.
Iversen, I. (1997). Matching-to-sample performance
Frank, A. J., & Wasserman, E. A. (2005). Associative in rats: A case of mistaken identity? Journal of
symmetry in the pigeon after successive matching-to- the Experimental Analysis of Behavior, 68, 27–45.
sample training. Journal of the Experimental Analysis of doi:10.1901/jeab.1997.68-27
Iversen, I. H., Sidman, M., & Carrigan, P. (1986).
Garcia, Y. A., & Rehfeldt, R. A. (2008). The effects of Stimulus definition in conditional discriminations.
common names and common FR responses on the Journal of the Experimental Analysis of Behavior, 45,
emergence of stimulus equivalence classes. European 297–304. doi:10.1901/jeab.1986.45-297
Journal of Behavior Analysis, 9, 99–120.
Jenkins, J. J. (1963). Mediated associations: Paradigms
Goldiamond, I. (1962). Perception. In A. J. Bachrach and situations. In C. N. Cofer & B. S. Musgrave
(Ed.), Experimental foundations of clinical psychology (Eds.), Verbal behavior and learning (pp. 210–257).
(pp. 280–340). New York, NY: Basic Books. New York, NY: McGraw-Hill. doi:10.1037/11178-006
Hall, G., Mitchell, C., Graham, S., & Lavis, Y. (2003). Jenkins, J. J., & Palermo, D. S. (1964). Mediation pro-
Acquired equivalence and distinctiveness in human cesses and the acquisition of linguistic structure.
discrimination learning: Evidence for associative Monographs of the Society for Research in Child
mediation. Journal of Experimental Psychology: Development, 29, 141–191.
General, 132, 266–276. doi:10.1037/0096-
3445.132.2.266 Joseph, B., Overmier, J. B., & Thompson, T. (1997).
Food- and nonfood-related differential outcomes in
Herrnstein, R. J. (1984). Objects, categories, and dis- equivalence learning by adults with Prader-Willi syn-
criminative stimuli. In H. L. Roitblat, T. S. Bever, & drome. American Journal on Mental Retardation, 101,
H. S. Terrace (Eds.), Animal cognition (pp. 233–261). 374–386.
Hillsdale, NJ: Erlbaum.
Kastak, C. R., Schusterman, R. J., & Kastak, D. (2001).
Hogan, D. E., & Zentall, T. R. (1977). Backward associa- Equivalence classification by California sea lions
tion in the pigeon. American Journal of Psychology, using class-specific reinforcers. Journal of the
90, 3–15. doi:10.2307/1421635 Experimental Analysis of Behavior, 76, 131–158.
Honey, R. C., & Hall, G. (1989). The acquired equiva- doi:10.1901/jeab.2001.76-131
lence and distinctiveness of cues. Journal of Konorski, J. (1959). A new method of physiological
Experimental Psychology: Animal Behavior Processes, investigation of recent memory in animals. Bulletin
15, 338–346. doi:10.1037/0097-7403.15.4.338 de L’Academie Polonaise des Sciences, 7, 115–117.
Honig, W. K. (1981). Working memory and the tem- Kruse, J. M., Overmier, J. B., Konz, W. A., & Rokke, E.
poral map. In N. E. Spear & R. R. Miller (Eds.), (1983). Pavlovian conditioned stimulus effects
383
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
upon instrumental choice behavior are reinforcer with theory. In J. Leslie & D. E. Blackman (Eds.),
specific. Learning and Motivation, 14, 165–181. Experimental and applied analysis of human behavior
doi:10.1016/0023-9690(83)90004-8 (pp. 85–110). Reno, NV: Context Press.
Lazareva, O. F., & Wasserman, E. A. (2008). Categories McIntire, K. D., Cleary, J., & Thompson, T. (1987).
and concepts in animals. In H. H. Byrne (Ed.), Conditional relations by monkeys: Reflexivity, sym-
Learning and memory: A comprehensive reference metry, and transitivity. Journal of the Experimental
(Vol. 1, pp. 197–226). Oxford, England: Elsevier. Analysis of Behavior, 47, 279–285. doi:10.1901/
doi:10.1016/B978-012370509-9.00056-5 jeab.1987.47-279
Lionello, K. M., & Urcuioli, P. J. (1998). Control by sam- Meehan, E. (1999). Class-consistent differential rein-
ple location in pigeons’ matching to sample. Journal forcement and stimulus class formation in pigeons.
of the Experimental Analysis of Behavior, 70, 235–251. Journal of the Experimental Analysis of Behavior, 72,
doi:10.1901/jeab.1998.70-235 97–115. doi:10.1901/jeab.1999.72-97
Lionello-DeNolf, K. M., & Urcuioli, P. J. (2000). Transfer Miguel, C. F., Petursdottir, A. I., Carr, J. E., & Michael, J.
of pigeons’ matching to sample to novel sample loca- (2008). The role of naming in stimulus categoriza-
tions. Journal of the Experimental Analysis of Behavior, tion by preschool children. Journal of the Experimental
73, 141–161. doi:10.1901/jeab.2000.73-141 Analysis of Behavior, 89, 383–405. doi:10.1901/jeab.
2008-89-383
Lionello-DeNolf, K. M., & Urcuioli, P. J. (2002). Stimulus
control topographies and tests of symmetry in Miller, R. R., & Barnet, R. C. (1993). The role of time
pigeons. Journal of the Experimental Analysis of in elementary associations. Current Directions in
Behavior, 78, 467–495. doi:10.1901/jeab.2002.78-467 Psychological Science, 2, 106–111. doi:10.1111/1467-
8721.ep10772577
Lipkens, R., Kop, P. F. M., & Matthijs, W. (1988). A
Peters, H. N. (1935). Mediate association. Journal of
test of symmetry and transitivity in the conditional
Experimental Psychology, 18, 20–48. doi:10.1037/
discrimination performances of pigeons. Journal of
h0057482
doi:10.1901/jeab.1988.49-395 Peterson, G. B. (1984). How expectancies guide behav-
ior. In H. L. Roitblat, T. G. Bever, & H. S. Terrace
Lowe, C. F., Horne, P. J., Harris, F. D. A., & Randle, V. R. (Eds.), Animal cognition (pp. 135–147). Hillsdale, NJ:
L. (2002). Naming and categorization in young chil- Erlbaum.
dren: Vocal tact training. Journal of the Experimental
Analysis of Behavior, 78, 527–549. doi:10.1901/ Reese, H. W. (1972). Acquired distinctiveness and equiva-
jeab.2002.78-527 lence of cues in young children. Journal of Experimental
Child Psychology, 13, 171–182. doi:10.1016/0022-
Lowe, C. F., Horne, P. J., & Hughes, J. C. (2005). 0965(72)90017-3
Naming and categorization in young children: III.
Vocal tact training and transfer of function. Journal Richards, R. W. (1988). The question of bidirectional
of the Experimental Analysis of Behavior, 83, 47–65. associations in pigeons’ learning of conditional dis-
doi:10.1901/jeab.2005.31-04 crimination tasks. Bulletin of the Psychonomic Society,
26, 577–579.
Mackintosh, N. J. (2000). Abstraction and discrimination.
Saunders, R. R., & Green, G. (1992). The nonequivalence
In C. Heyes & L. Huber (Eds.), The evolution of cog-
of behavioral and mathematical equivalence. Journal
nition (pp. 123–141). Cambridge, MA: MIT Press.
Maki, P., Overmier, J. B., Delos, S., & Gutmann, A. J. doi:10.1901/jeab.1992.57-227
(1995). Expectancies as factors influencing con- Saunders, R. R., & Green, G. (1999). A discrimination
ditional discrimination performance of children. analysis of training structure effects on stimulus equiv-
Psychological Record, 45, 45–71. alence outcomes. Journal of the Experimental Analysis of
Mandell, C., & Sheen, V. (1994). Equivalence class for- Behavior, 72, 117–137. doi:10.1901/jeab.1999.72-117
mation as a function of the pronounceability of the Schenk, J. J. (1994). Emergent relations of equivalence
sample stimulus. Behavioural Processes, 32, 29–46. generated by outcome-specific consequences in
doi:10.1016/0376-6357(94)90025-6 conditional discrimination. Psychological Record, 44,
McIlvane, W. J. (1992). Stimulus control analysis 537–558.
and nonverbal instructional methods for people Schusterman, R. J., & Kastak, D. (1993). A California
with intellectual disabilities. International Review sea lion (Zalophus californianus) is capable of form-
of Research in Mental Retardation, 18, 55–109. ing equivalence relations. Psychological Record, 43,
doi:10.1016/S0074-7750(08)60116-0 823–839.
McIlvane, W. J., Serna, R. W., Dube, W. V., & Stromer, R. Shipley, W. C. (1935). Indirect conditioning. Journal of
(2000). Stimulus control topography coherence and General Psychology, 12, 337–357. doi:10.1080/00221
stimulus equivalence: Reconciling test outcomes 309.1935.9920108
384
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Sidman, M. (1992). Adventitious control by the location Tomonaga, M. (1999). Establishing functional classes
of comparison stimuli in conditional discriminations. in a chimpanzee (Pan troglodytes) with a two-
Journal of the Experimental Analysis of Behavior, 58, item sequential-responding procedure. Journal of
173–182. doi:10.1901/jeab.1992.58-173 the Experimental Analysis of Behavior, 72, 57–79.
Sidman, M. (2000). Equivalence relations and the rein- doi:10.1901/jeab.1999.72-57
forcement contingency. Journal of the Experimental Tulving, E. (1983). Elements of episodic memory. New
Analysis of Behavior, 74, 127–146. doi:10.1901/ York, NY: Oxford University Press.
jeab.2000.74-127
Urcuioli, P. J. (1990). Some relationships between out-
Sidman, M. (2008). Symmetry and equivalence relations come expectancies and sample stimuli in pigeons’
in behavior. Cognitive Studies, 15, 322–332. delayed matching. Animal Learning and Behavior, 18,
Sidman, M., Kirk, B., & Willson-Morris, M. (1985). Six- 302–314. doi:10.3758/BF03205290
member stimulus classes generated by conditional- Urcuioli, P. J. (1996). Acquired equivalences and medi-
discrimination procedures. Journal of the Experimental ated generalization in pigeon’s matching-to-sample.
Analysis of Behavior, 43, 21–42. doi:10.1901/ In T. R. Zentall & P. M. Smeets (Eds.), Stimulus
jeab.1985.43-21 class formation in humans and animals (pp. 55–70).
Sidman, M., Rauzin, R., Lazar, R., Cunningham, S., Amsterdam, the Netherlands: Elsevier. doi:10.1016/
Tailby, W., & Carrigan, P. (1982). A search for S0166-4115(06)80103-2
symmetry in the conditional discrimination of Urcuioli, P. J. (2003). Generalization. In L. Nadel (Ed.-in-
rhesus monkeys, baboons, and children. Journal of Chief), Encyclopedia of cognitive sciences (Vol. 2,
the Experimental Analysis of Behavior, 37, 23–44. pp. 275–281). London, England: Macmillan.
doi:10.1901/jeab.1982.37-23
Urcuioli, P. J. (2005). Behavioral and associative effects
Sidman, M., & Tailby, W. (1982). Conditional discrimi- of differential outcomes in discrimination learn-
nation vs. matching to sample: An expansion of the ing. Learning and Behavior, 33, 1–21. doi:10.3758/
testing paradigm. Journal of the Experimental Analysis BF03196047
Urcuioli, P. J. (2006a). Responses and acquired equiva-
Sidman, M., Wynne, C. K., Maguire, R. W., & Barnes, T. lence classes. In E. A. Wasserman & T. R. Zentall
(1989). Functional classes and equivalence relations. (Eds.), Comparative cognition: Experimental explora-
Journal of the Experimental Analysis of Behavior, 52, tions of animal intelligence (pp. 405–421). New York,
261–274. doi:10.1901/jeab.1989.52-261 NY: Oxford University Press.
Skinner, B. F. (1935). The generic nature of the con- Urcuioli, P. J. (2006b). When discrimination fails (or
cepts of stimulus and response. Journal of General at least falters). Journal of Experimental Psychology:
Psychology, 12, 40–65. doi:10.1080/00221309.1935. Animal Behavior Processes, 32, 359–370. doi:10.1037/
9920087 0097-7403.32.4.359
Spradlin, J. E., Cotter, V. W., & Baxley, N. (1973). Urcuioli, P. J. (2007). Sample and comparison location
Establishing a conditional discrimination without as factors in matching acquisition, transfer, and
direct training: A study of transfer with retarded ado- acquired equivalence. Learning and Behavior, 35,
lescents. American Journal of Mental Deficiency, 77, 252–261. doi:10.3758/BF03206431
556–566.
Urcuioli, P. J. (2008a). [Associative symmetry after
Spradlin, J. E., & Saunders, R. R. (1986). The development partially reinforced successive matching training].
of stimulus classes using match-to-sample procedures: Unpublished raw data.
Sample classification vs. comparison classification.
Analysis and Intervention in Developmental Disabilities, Urcuioli, P. J. (2008b). Associative symmetry, “anti-
6, 41–58. doi:10.1016/0270-4684(86)90005-4 symmetry,” and a theory of pigeons’ equivalence-class
formation. Journal of the Experimental Analysis of
Terrace, H. S. (1986). A nonverbal organism’s knowledge Behavior, 90, 257–282. doi:10.1901/jeab.2008.90-257
of ordinal position in a serial learning task. Journal of
Experimental Psychology: Animal Behavior Processes, Urcuioli, P. J. (2008c). The nature of the response in
12, 203–214. doi:10.1037/0097-7403.12.3.203 Simon discriminations by pigeons. Learning and
Behavior, 36, 200–209. doi:10.3758/LB.36.3.200
Thompson, R. K. R., & Oden, D. L. (2000). Categorical
perception and conceptual judgments by nonhuman Urcuioli, P. J., & Lionello-DeNolf, K. M. (2001). Some
primates: The paleological monkey and the analogi- tests of the anticipatory mediated generalization
cal ape. Cognitive Science, 24, 363–396. doi:10.1207/ model of acquired sample equivalence in pigeons’
s15516709cog2403_2 many-to-one matching. Animal Learning and
Behavior, 29, 265–280. doi:10.3758/BF03192892
Thorndike, E. L. (1898). Animal intelligence: An experi-
mental study of the associative processes in animals. Urcuioli, P. J., & Lionello-DeNolf, K. M. (2005). The
Psychological Monographs, 2(Whole No. 8). role of common reinforced comparison responses in
385
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Peter J. Urcuioli
acquired sample equivalence. Behavioural Processes, in pigeons. Psychological Science, 3, 374–379.

69, 207–222. doi:10.1016/j.beproc.2005.02.005 doi:10.1111/j.1467-9280.1992.tb00050.x
Vaughan, W., Jr. (1988). Formation of equivalence sets in Wasserman, E. A., & Zentall, T. R. (2006a). Comparative
pigeons. Journal of Experimental Psychology: Animal cognition: A natural science approach to the study
Behavior Processes, 14, 36–42. doi:10.1037/0097- of animal intelligence. In E. A. Wasserman & T. R.
7403.14.1.36 Zentall (Eds.), Comparative cognition: Experimental
Vickery, T. J., & Jiang, Y. V. (2009). Associative group- explorations of animal intelligence (pp. 3–11). New
ing: Perceptual grouping of shapes by association. York, NY: Oxford University Press.
Attention, Perception, and Psychophysics, 71, 896–909. Wasserman, E. A., & Zentall, T. R. (2006b).
doi:10.3758/APP.71.4.896 Comparative cognition: Experimental explorations
von Fersen, L., & Delius, J. D. (2000). Acquired of animal intelligence. New York, NY: Oxford
equivalences between auditory stimuli in dolphins University Press.
(Tursiops truncates). Animal Cognition, 3, 79–83. Zentall, T. R. (2006). Mental time travel in animals: A
doi:10.1007/s100710000063 challenging question. Behavioural Processes, 72,
Wasserman, E. A. (1976). Successive matching-to-sample 173–183. doi:10.1016/j.beproc.2006.01.009
in the pigeon: Variation on a theme by Konorski. Zentall, T. R., Wasserman, E. A., Lazareva, O. F.,
Behavior Research Methods and Instrumentation, 8, Thompson, R. K. R., & Rattermann, M. J. (2008).
278–282. doi:10.3758/BF03201713 Concept learning in animals. Comparative Cognition
Wasserman, E. A., DeVolder, C. L., & Coppage, D. J. and Behavior Reviews, 3, 13–45. doi:10.3819/
(1992). Nonsimilarity-based conceptualization ccbr.2008.30002
386
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 17
Attention and Conditioned

Reinforcement
Timothy A. Shahan
The ability to appropriately attend to the important reinforcers (i.e., obtaining goals), rather than such
features of a complex environment is a critical sur- stimuli becoming reinforcers themselves.
vival skill. Problems related to the allocation and
persistence of attending are associated with various Attention and Behavior
psychological disorders including attention-deficit/
Attention
hyperactivity disorder (American Psychiatric Associ-
An appropriate starting point for any treatment of
ation, 1994), autism and other developmental dis-
attention is a consideration of how to define atten-
abilities (Bryson, Wainwright-Sharp, & Smith, 1990;
tion and circumscribe what is to be discussed.
Dube & McIlvane, 1997; Lovaas, Koegel, & Schreib-
Unfortunately, clear technical definitions of atten-
man, 1979), schizophrenia (Nestor & O’Donnell,
tion are difficult to find. Consider that two book-
1998), and substance abuse (Ehrman et al., 2002;
length treatments of the topic (Pashler, 1998; Styles,
Johnsen, Laberg, Cox, Vaksdal, & Hugdahl, 1994;
1997) and a chapter on attention in Stevens’s Hand-
Lubman, Peters, Mogg, Bradley, & Deakin, 2000;
book of Experimental Psychology (Luck & Vecera,
Townshend & Duka, 2001). Although many factors
2002) provide no technical definition of the term.
surely contribute to the allocation and persistence of
Instead, all three sources note William James’s
attending, the relation between patterns of attending
(1890) famous suggestion that “everyone knows
and the resultant consequences may play an impor-
what attention is.” All three then move on to suggest
tant role. I begin this chapter with a consideration of
that, despite James’s claim, it may be more appropri-
attention and its potential relation to existing
ate to say that no one knows what attention is.
accounts of behavior maintained by its consequences
Instead, they suggest that the term attention almost
(i.e., operant, or instrumental, behavior). Next, I
certainly refers to more than one psychological phe-
review research on how differential consequences
nomenon. The phenomena typically considered to
affect the allocation and persistence of attending to
fall under the heading of attention involve limita-
important environmental stimuli. Finally, I examine
tions in the capacity of cognitive functioning and
the relation between attending to stimuli associated
selectivity of perception.
with differential consequences and the traditional
Even if attention is not a unitary psychological
concept of conditioned, or secondary, reinforcement—
phenomenon, the rest of James’s (1890) definition is
the notion that stimuli associated with reinforcers
instructive as a sort of summary of the set of phe-
themselves acquire the ability to reinforce behavior.
nomena captured under the heading. James
In considering the concept of conditioned reinforce-
suggested,
ment, I explore the utility of an alternative frame-
work based on the notion that attending to It is the taking possession by the mind, in
important stimuli is instrumental in acquiring clear and vivid form, of one out of what
DOI: 10.1037/13937-017
387
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
seem several simultaneously possible or a supervisory attentional system (Norman & Shal-
objects or trains of thought. Focalization, lice, 1986). As noted by Styles (1997), cognitive
concentration of consciousness are of its psychologists have widely recognized that providing
essence. It implies withdrawal from some a name for an entity in charge of decision making
things in order to deal effectively with does not escape the homunculus problem and that
others. (pp. 403–404) the names serve only as temporary placeholders for
a problem remaining to be solved.
Clearly, defining attention in terms of the even Experimental examinations of goal-directed con-
more poorly defined concept of consciousness is less trol of attention almost always use instructions to
than ideal. In addition, despite James’s (1890) asser- direct participants how to allocate their attention to
tion that attention involves the focusing of con- particular stimulus features, dimensions, or spatial
sciousness, it is becoming increasingly clear that locations. For example, subjects might be instructed
conscious awareness is not necessary for even appar- to attend to only the color or shape of an object or to
ently goal-related shifts in attention (see Dijkster- devote 80% of their attention to the color and 20%
huis & Aarts, 2010, for review). to the shape (e.g., Bonnel & Prinzmetal, 1998). As
Nonetheless, James’s (1890) definition does gen- another example, subjects in a cued detection task
erally capture the properties of capacity limitation might be asked to keep their gaze fixed at a central
and selectivity noted earlier. James’s assertion that location on a computer screen and then to report
people all know what attention is probably stems targets presented to the left or right of the fixation
from the common personal experience that only a point. Arrows presented at the fixation location and
small fraction of the stimuli impinging on people’s pointing to the left or the right instruct participants
senses seem to affect them at any given time. James’s where in space to allocate their attention, but on
definition also suggests that attention appears to be some percentage of the trials the arrow is inaccurate
something that organisms do. In this chapter, I (see Posner, 1980). Detection accuracy or reaction
explore how attention might be thought of in terms time data from such procedures are consistent with
similar to those used to describe other activities in what would be expected if subjects allocate their
which organisms engage. attention as instructed (e.g., greater accuracy with
A large empirical literature (see Yantis, 2000, for attended dimension, shorter reaction times with
review) has suggested that changes in what an accurate instruction).
organism is attending to can be driven automatically Instructions about what to attend to might be
by properties of stimuli themselves (e.g., salience, thought of as a way to change what Norman (1968)
novelty, abrupt onset, innate biological significance) called the pertinence of a stimulus. Norman suggested
or by goal-directed control. The phenomenon of that in addition to the physical aspects of a stimulus
goal-directed control of attention goes by many (i.e., stimulus activation), the importance of a stim-
names, including endogenous, top-down, and volun- ulus to the organism (i.e., pertinence) is involved in
tary control. A simple demonstration of such control the control of attention. The pertinence of a stimu-
is to keep your gaze fixed on a word in the center of lus reflects the current goals of the organism and its
this page and then pay attention to something in the history with a stimulus. For example, your spoken
periphery to your right and then to your left. name is a stimulus that likely commands attention,
Another example is to listen to a piece of music and not because of the physical properties of the stimu-
then shift your attention to different instruments at lus per se but because of your history with it.
different times—first the drums, then the bass, and Instructing subjects about how to allocate their
so on. In theoretical treatments of goal-directed con- attention can be seen as a way of creating language-
trol, such changes in the allocation of attention have based goals or quickly providing a relevant history
generally been hypothesized to be under the control with respect to a stimulus. Although such instruc-
of what has been called a central administrator (Kah- tional control of attending is important in demon-
neman, 1973), a central executive (Baddeley, 1986), strating that attending can be goal directed, it does
388
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Attention and Conditioned Reinforcement
not document what factors contribute to the perti- relation between two environmental stimuli (see
nence of stimuli or the goal-directed control of Chapter 13, this volume). For example, if a tone (i.e.,
attending in the natural environment. Nonetheless, conditioned stimulus [CS]) reliably predicts food
the study of learning and behavior (i.e., conditioning delivery (i.e., unconditioned stimulus [US]) that
and learning) provides a reasonably well-developed already elicits salivation (i.e., unconditioned response
framework for understanding how experience [UR]), then the tone itself will come to elicit saliva-
changes the importance of stimuli and how conse- tion (i.e., conditioned response [CR]). A large empiri-
quences affect the allocation and persistence of goal- cal literature since the time of Pavlov has provided a
directed behavior. This framework might also be wealth of information about how organisms learn the
useful for understanding the control of attention. predictive relations between stimuli and how initially
neutral stimuli acquire the ability to affect behavior
Behavior (see Mackintosh, 1974, and Rescorla, 1988, for
Although some may find it distressing that a well- reviews). Such stimulus–stimulus learning may be
articulated definition of attention was not forthcoming contrasted with operant (i.e., instrumental) condi-
in the preceding section, one could argue that provid- tioning, which involves learning the relation between
ing a definition of behavior is similarly problematic. a response and the consequences of that response.
Consider the treatment of the terms learning and behav- For example, a hungry rat may learn to press a lever
ior in Catania’s (1998) influential textbook, Learning: that produces food or a child may learn to ask nicely
for a toy. In both cases, the behavior is instrumental
This book is about learning, but from in the production of the consequence, and the future
the start we have to face the fact that we probability of the behavior is changed as a result.
won’t be able to define it. There are no Consequences that increase the probability of the
satisfactory definitions of learning. Still, behavior are called reinforcers and those that decrease
we can study it. (p. 2) the probability of the behavior are called punishers.
Behavior is no easier to define than learn- Operant behavior roughly corresponds to what is
ing. We may say glibly that behavior usually called voluntary or goal-directed behavior. A
is anything an organism does, but this vast literature on operant conditioning with humans
definition is too global. . . . Let’s not try and nonhumans has demonstrated that variations in
to resolve this problem. Our aim is to such behavior are lawfully related to the type and
examine some properties of behavior. arrangement of consequences obtained (see Mazur,
Although they sometimes share com- 2006, for review).
mon names, the phenomena of behavior The fundamental unit of operant behavior is
are varied, so we’ll probably do better by widely believed to be the discriminated operant,
considering examples than by attempting embodied by the three-term contingency between
definitions. (p. 7) (a) the discriminative stimulus context in which the
behavior occurs, (b) the behavior itself, and (c) the
I hope the parallels between my treatment of consequence of the response. Schematically, the
attention and Catania’s (1998) treatment of learning three-term contingency is often described as SD:
and behavior are obvious. As with attention, behav- B → SR, where SD is a discriminative stimulus, B is
ior may be difficult to define, but researchers have the behavior, and SR is a reinforcer for that behavior.
nevertheless made progress in understanding some An SD comes to modulate the occurrence of the
basic principles of the phenomena captured under behavior, or to set the occasion for the behavior, as a
that heading. result of the consequences experienced for the
In the study of learning and behavior, it is custom- behavior in its presence. For example, if the lever
ary to distinguish between classical (i.e., Pavlovian, pressing of a rat produces food when a light is on,
respondent) and operant (i.e., instrumental) condi- but not when it is off, lever pressing will come to
tioning. Classical conditioning involves learning the occur predominately in the presence of the light.
389
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
Although distinguishing between classical and option. Herrnstein formalized this relation with the
operant conditioning may be customary, the two matching law, which states that
types of learning clearly interact in most instances of
B1 R1
behavior. For example, not only do organisms learn = , (1)
that an SD signals the contingent relation between B1 + B2 R1 + R 2
the behavior and the reinforcer, they also learn the where B1 and B2 refer to behavior allocated to the
Pavlovian stimulus–stimulus relation between the SD two options and R1 and R2 refer to the reinforcers
and the reinforcer (see Rescorla, 1998, for a review). obtained by responding at those options (for a
Such stimulus–stimulus learning within the discrim- primer on quantitative models of behavior, see
inated operant plays an important role in the moti- Chapter 10, this volume). Thus, when organisms are
vation and persistence of operant behavior (e.g., confronted with a choice situation, they allocate
Nevin & Grace, 2000), a topic I return to later. their behavior to the available options in proportion
Another way in which classical and operant con- to the reinforcers obtained from those options.
ditioning interact is through the process of condi- Later, Herrnstein (1970) extended the matching law
tioned (i.e., secondary) reinforcement. Stimuli to behavior occurring in situations in which only
associated with the delivery of reinforcers for oper- one response is explicitly defined. Although only a
ant behavior are commonly believed to become rein- single response may be defined, the organism could
forcers themselves as a result of the Pavlovian also engage in other unmeasured behavior that may
relation between the stimulus and the existing rein- provide other unmeasured sources of reinforcement.
forcer. For example, a click associated with the This formulation has come to be known as Herrn-
delivery of a food reinforcer for a rat may also stein’s hyperbola, or the quantitative law of effect.
appear to serve as a reinforcer for the lever pressing The quantitative law of effect states that
that produces it (see Williams, 1994, for review).
kR
Although both classical and operant conditioning B= , (2)
R + Re
play a role in the treatment of attention in this chap-
ter, I focus largely on operant conditioning because where B is the rate of the measured behavior, R is the
my goal is to explore how consequences affect the rate of reinforcement for the measured behavior, the
allocation and persistence of attending. Before pro- parameter k defines asymptotic response rates when
ceeding to a discussion of how consequences affect all reinforcement is for the measured response, and
attending, I briefly review the matching law and the parameter Re represents alternative sources of
behavioral momentum theory, two influential theo- reinforcement (i.e., reinforcement for all other
ries of operant conditioning that play a critical role behavior). Thus, the frequency of an operant behav-
in the discussion. ior is described as being determined by the rate of
reinforcement for that behavior in the context of
Operant behavior as choice: The matching law. reinforcement for all other behavior. Response rates
Much of the contemporary study of operant condi- increase with increases in reinforcement rate for the
tioning focuses on how differential consequences measured behavior (i.e., R), but the degree to which
affect the allocation of behavior (i.e., choice). This response rates increase with increases in reinforce-
focus on choice is largely the result of the influ- ment rate is determined by the availability of alterna-
ence of Herrnstein’s (1961, 1970) matching law. tive reinforcers (i.e., Re). Within the framework
Herrnstein (1961) varied the rate at which two provided by the quantitative law of effect, all operant
concurrently available operant responses provided behavior is choice and is governed by the relative
food reinforcement for the key pecking of pigeons frequency of reinforcement obtained for the different
in a standard operant conditioning preparation. He activities in which an organism might engage.
found that the proportion of responses allocated to Baum (1974, 1979) extended the matching law
one of two options roughly equaled (i.e., matched) to account for common deviations from Equation 1.
the proportion of reinforcers obtained from that Although Equation 1 suggests that the distribution
390
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
of behavior to two options strictly matches the dis- Thus, in its most general form, the matching law
tribution of reinforcers, real organisms are often less suggests that the allocation of behavior to the avail-
than perfectly sensitive to the distribution of rein- able choice options is governed by the relative value
forcers. The generalized matching law suggests that of the reinforcers associated with those options.
the ratio of behavior allocated to two options is a The matching law has provided a quantitative
power function of the ratio of reinforcers obtained at framework for describing how differential conse-
the options. Thus, quences govern the allocation of behavior and has
a provided insights into how the value of reinforcers
B1 R  and stimuli associated with reinforcers might be cal-
= b 1  , (3)
B2  R2  culated. Although the matching law was developed
largely with data generated in operant conditioning
where B1 and B2 refer to behaviors 1 and 2, respec-
preparations with nonhumans, it has since been
tively, and R1 and R2 refer to reinforcers for B1 and
shown to have widespread applicability to broader
B2. The parameter b represents bias for one option or
issues in decision making, especially the apparently
the other unrelated to variations in relative rein-
irrational, suboptimal, or impulsive decision making
forcement rate. The parameter a represents sensitiv-
of humans (see Herrnstein, 1990; Herrnstein, Rach-
ity of the allocation of behavior to changes in
lin, & Laibson, 1997; and Madden & Bickel, 2010,
reinforcement ratios. Values of a less than 1 reflect
for reviews). Furthermore, the matching law and its
relative insensitivity of the allocation of behavior to
derivatives have provided a useful framework within
the ratio of reinforcers (i.e., undermatching),
which to conduct analyses of the neuroscience of
whereas values of a greater than 1 reflect hypersen-
decision making and reward valuation (e.g., Loew-
sitivity (i.e., overmatching). Equation 3 can also be
enstein & Seung, 2006; McClure, Laibson, Loewen-
extended to more than two choice options (see Sch-
stein, & Cohen, 2004; Sugrue, Corrado, &
neider & Davison, 2005), although the details of
Newsome, 2004). Perhaps the matching law may
such an extension are not of concern here.
also provide a useful framework for characterizing
Although Equation 3 suggests that choice is a func-
how differential consequences affect decisions about
tion of relative reinforcement rate, the reinforcers
the allocation of attention.
available for different options might differ with respect
to any number of parameters (e.g., amount, delay to
Persistence of operant behavior: Behavioral momen-
receipt, quality). Accordingly, Baum and Rachlin
tum theory. Behavioral momentum theory (e.g.,
(1969) proposed the concatenated matching law,
Nevin & Grace, 2000) addresses how the conditions
which suggests in its generalized matching law version
of reinforcement under which an operant behavior is
that choice is dependent on the multiplicative effects
trained affect the persistence of that behavior when
of different reinforcement parameters, such that
it is disrupted. The theory suggests that there are
B1
a1 a2 a3
 R   A   1 / d1   q1 
a4
two important aspects of operant behavior: response
= b 1   1   , (4) rates under steady-state conditions and resistance to
B2  R 2   A 2   1 / d 2   q2 
change under disruption conditions. The theory sug-
with terms as in Equation 3 and additional terms for gests that response rate is governed by the contingent
relative reinforcement amount (A1 and A2), immedi- relation between the response and the reinforcer it
acy (1/d1 and 1/d2), quality (q1 and q2), and their produces in a manner consistent with Herrnstein’s
respective sensitivity parameters a2, a3, and a4. The (1970) single-response version of the matching law
overall impact of a variety of parameters of reinforce- (i.e., Equation 2). Resistance to change is governed
ment on choice can also be collapsed into a central by the Pavlovian relation between the discriminative
intervening variable called value (i.e., V) such that stimulus context in which the behavior occurs and
a the reinforcers obtained in that context.
B1  V1  Resistance to change is typically studied by
= . (5)
B2  V2  arranging different conditions of reinforcement
391
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
(e.g., reinforcement rates or magnitudes), presented where m1 and m2 are resistance to change of respond-
one at a time and each in the presence of a distinc- ing in the presence of stimuli 1 and 2, R1 and R2 refer
tive stimulus (i.e., a multiple schedule of reinforce- to the rates of primary reinforcement obtained in the
ment). Once steady-state responding is achieved in presence of those stimuli, and the parameter b
the presence of both stimuli, a disruptor is intro- reflects sensitivity of relative resistance to change to
duced (e.g., satiation of the reinforcer, extinction). variations in relative reinforcement rates (Nevin,
The resultant decrease in behavior relative to the 1992). Equation 6 and related quantitative models
predisruption baseline level provides the measure of making up behavioral momentum theory have pro-
resistance to change. Relatively smaller decreases in vided a general framework within which to under-
responding compared with baseline reflect greater stand how reinforcement experienced in the
resistance to change. Although simple response rate presence of stimuli govern the persistence of operant
under steady-state conditions has traditionally been behavior in the presence of that stimulus. The frame-
used to measure the strengthening effects of rein- work provided by the theory for characterizing the
forcers, behavioral momentum theory suggests that persistence of operant behavior has been found to
resistance to change provides a better measure of have broad generality across reinforcer types, set-
response strength. The reason is that response rates tings, and species ranging from fish to humans (e.g.,
are well known to be influenced by operations Ahearn, Clark, Gardenier, Chung, & Dube, 2003;
(e.g., pacing contingencies such as differential rein- Cohen, 1996; Grimes & Shull, 2001; Harper, 1999;
forcement of low rate) that are generally not consid- Igaki & Sakagami, 2004; Mace et al., 1990; Quick &
ered to affect response strength (e.g., Nevin, 1974; Shahan, 2009; Shahan & Burke, 2004).
Nevin & Grace, 2000).
Higher rates of reinforcement in a stimulus con- Allocation and Persistence of Attention
text would generally be expected to produce both and Operant Behavior: Shared
higher response rates and greater resistance to Mechanisms?
change—and they do (see Nevin, 1992). However, Operant behavior and at least part of what is called
because response rates and resistance to change attention appear to be goal-related activities.
are governed separately by the operant response– Although the goal-directed aspects of attention have
reinforcer and Pavlovian stimulus–reinforcer rela- generally been manipulated with instructions,
tions, respectively, it is possible to simultaneously insights provided by quantitative theories of operant
decrease response rates and yet increase resistance behavior might be useful in understanding how
to change. In practice, this has been demonstrated goals in the natural environment help to govern
by adding reinforcers to the stimulus context that attention. For example, both behavior and attention
are not contingent on the operant response. The appear to be limited resources requiring allocation
noncontingent reinforcers decrease baseline decisions. Thus, the quantitative framework for
response rates by degrading the contingent response– choice provided by matching theory might also pro-
reinforcer relation but increase resistance to change vide a first step toward replacing placeholder con-
by improving the Pavlovian stimulus–reinforcer rela- cepts, such as a central administrator, with a
tion signaled by the discriminative stimulus context quantitative theory of how experience translates into
(e.g., Nevin, Tota, Torquato, & Shull, 1990). the differential allocation of attending. In addition,
Quantitatively, behavioral momentum theory the framework provided by behavioral momentum
suggests that relative resistance to change in the theory for understanding the persistence of operant
presence of two stimuli is a power function of the behavior might be useful for understanding the per-
relative rate of reinforcement obtained in the pres- sistence of attending under conditions of disruption.
ence of those stimuli such that Nonetheless, the suggestion that consequences
b might similarly affect what is typically called atten-
m1 R  tion and what is typically called behavior would
= b 1  , (6)
m2  R2  require empirical evidence. Behavior analysts know
392
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
how consequences affect what traditionally falls and ignoring a concurrently presented green distrac-
under the heading of operant behavior, but what tor shape in a prime trial leads to an increased reac-
about phenomena more likely to be categorized tion time when participants are required to attend to
under the heading of attention? the previously ignored green shape in the next trial
(i.e., probe). The typical interpretation of negative
priming is that it results from a switching of selec-
Attention and Differential
tive visual attention to the target and away from the
Consequences
distractor during the prime that persists until the
Human Attention and Differential probe test (see Pashler, 1998, for a review). Pashler
Consequences (1998) suggested that negative priming might occur
Until recently, there was nearly no experimental as a result of an error-correction learning mecha-
analysis of how variations in consequences affect nism because a distractor item is unhelpful at one
performance of humans in attention tasks. As noted time and therefore typically unlikely to be helpful
earlier, studies of goal-directed attention have shortly after. If Pashler’s suggestion is correct, it is
almost always relied on instructions to produce reasonable to expect that negative priming should
changes in goal-directed attention. Nonetheless, be affected by changing the consequences of attend-
research by Gopher (1992) suggested that the suc- ing to the target stimulus during the prime.
cessful allocation of attention appears to be a train- Della Libera and Chelazzi (2006) examined how
able skill that improves with practice and differential differential consequences affected negative priming
consequences. For example, participants were in a visual selective attention task. In one experi-
exposed to a demanding space-based game designed ment, prime stimuli were global numbers consisting
to simulate the complexities of divided attention of appropriately arranged local numbers (e.g., multi-
encountered by fighter pilots. The simulation was ple instances of smaller 6s at the local level arranged
complex enough that the participants often initially into a larger number 5 at the global level). Before
expressed panic and adopted suboptimal strategies. the presentation of a prime stimulus, the letter G or
Participants were instructed to focus their attention the letter L instructed the subjects to attend to the
on a subset of aspects of the situation and to periodi- global or local level, respectively. A correct response
cally shift their attention to other aspects. The par- was defined as reporting the number at the
ticipants received feedback and point-based rewards instructed level (either global or local). Thus, the
for successful performance, thus encountering dif- number at the other level became the to-be-ignored
ferential consequences with changes in their alloca- distractor. For example, if the prime was a global 5
tion of attention. As a result, the participants made up of local 6s, choosing the number 5 was cor-
avoided adopting suboptimal strategies and became rect after the instruction G and choosing the num-
proficient at the highly demanding tasks. Such ber 6 was correct after the instruction L. To
results are consistent with the suggestion that the manipulate the consequences of attending to the tar-
allocation of attention can be affected by experienc- get stimulus, correct responses to primes were fol-
ing differential consequences. lowed by either a high (€0.10) or a low (€0.01)
A more recent set of experiments examining how payoff, as reported on the computer screen. The
differential consequences affect attention used a probe stimulus was then presented 400 milliseconds
negative priming procedure (Della Libera & later. Subjects were to report the number in the
Chelazzi, 2006). Negative priming refers to a phe- probe stimulus, regardless of whether it occurred at
nomenon whereby selectively attending to a target the global level (e.g., a 5 made up of local Xs) or the
stimulus accompanied by an irrelevant distractor local level (i.e., an X made up of local 5s). In this
during a prime trial results in poorer performance procedure, negative priming would be evidenced by
(i.e., increased reaction time) when the distractor longer reaction times when the number in the probe
stimulus is used as a target shortly thereafter during stimulus occurred at the same level of the distractor
a probe test. For example, attending to a red shape stimulus in the prime (e.g., subject attended to
393
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
local-level 6s and ignored a global-level 5 in the An additional eight neutral shapes were equally as
prime, but in the probe trial, the target was at the likely to lead to the higher and lower payoffs as tar-
global level). Consistent with the hypothesis that gets or distractors. Five days after the training phase
consequences can affect the allocation of attention, with the differential payoff, subjects were exposed to
negative priming occurred only when detecting the either the same task (Experiment 1) or a different
target stimulus in the prime produced the larger visual search task (Experiment 2), both using the
payoff. In other words, the highly reinforced alloca- stimuli from the training phase but in the absence
tion of attention to the target in the prime produced of reward.
a greater shift of attention to the target and away In Della Libera and Chelazzi’s (2009) Experi-
from the distractor that was apparent in the subse- ment 1, stimuli associated with a high payoff when
quent probe test. A second experiment using the serving as the target in the training phase were more
same basic paradigm extended the result to form- difficult to ignore when they appeared as the distrac-
based stimuli and same–different choice-based tor during the subsequent test. However, stimuli
responding. Thus, shifts in attention as measured by associated with a low payoff when serving as the tar-
a common selective attention paradigm appear to be get in training were more easily ignored when they
affected by differential consequences. appeared as the distractor during the subsequent
A subsequent set of experiments by Della Libera test. Complementary results were obtained with
and Chelazzi (2009) showed that differential conse- stimuli that had served as distractors during the
quences in selective visual attention tasks could training phase. Stimuli that were associated with a
have relatively long-term effects on how attention is high payoff when they were ignored as distractors in
allocated. In both experiments, subjects were training produced less interference when they con-
exposed to a 3-day training phase in which perfor- tinued to serve as distractors in the test, but stimuli
mance on a selective visual attention task produced previously associated with a low payoff when they
different magnitude reinforcers for two sets of stim- had been ignored in training continued to interfere
uli. Specifically, trials started with a red or a green with test performance as distractors. In Experiment
cue that signaled the target in an immediately fol- 2, the testing phase was a visual search task in
lowing display. The display consisted of three differ- which a black sample shape was briefly presented
ent nonsense shapes. One of the shapes was a black and followed by two black comparison shapes. The
comparison and was presented next to two overlap- subjects’ task was to report whether the sample
ping shapes, one of which was red and the other of stimulus was present in the comparison display.
which was green. The subjects’ task was to deter- The results showed that detections in the compari-
mine whether the shape of the previously cued color son display were faster if the sample had previously
(i.e., either red or green) matched the black compar- been associated with a higher payoff as a target in
ison. Thus, the other colored shape served as a dis- the training phase. Together, the results from these
tractor stimulus to be ignored. Correct responses two experiments suggested that differential rein-
were more likely to lead to a higher payoff (€0.10) forcement for attending to or ignoring stimuli in a
for four shapes and a lower payoff (€0.01) for four selective attention task can have relatively long-
different shapes. Two of the high-payoff stimuli lasting effects on attending to or ignoring those
resulted in the higher payoff when they served as stimuli later.
the target and were correctly detected. The other An experiment by Raymond and O’Brien (2009)
two high-payoff stimuli resulted in the higher payoff examined how differences in payoff for choosing
when they served as the distractor stimulus and stimuli in a choice task affected subsequent recogni-
were correctly ignored. The same was true for the tion of those stimuli in an attentional blink proce-
low-payoff stimuli—two stimuli resulted in the dure. Subjects were first given a series of choices
lower payoff when they were correctly detected as between sets of two computer-generated faces. Half
targets and the other two resulted in the lower pay- of the 24 face stimuli were associated with either a
off when they were correctly ignored as distractors. gain (plus 5 pence) or a loss (minus 5 pence) at a
394
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
high probability (.8) or a low probability (.2). The shorter delay, but only for neutral stimuli or stimuli
other half of the stimuli were neutral and associated previously associated with a loss. The attentional
with no consequence. During exposure to this blink effect was eliminated for stimuli that were pre-
choice condition, subjects learned to choose the viously associated with a gain. Thus, the availability
stimuli associated with a higher probability of a gain of attention for stimuli in a taxing situation appears
and a lower probability of a loss. Next, the subjects to depend on the consequences encountered as a
were exposed to a standard attentional blink proce- result of previous choices involving those stimuli
dure in the absence of explicit consequences. In the such that attention was more readily available for
attentional blink procedure, subjects were shown a stimuli previously associated with a gain.
series of four brief stimuli in succession (85 milli- The studies reviewed in this section generally
seconds per stimulus). The first stimulus at Time 1 support the conclusion that consequences encoun-
was an oval made up of either circles or squares fol- tered as a result of attending to particular features of
lowed immediately by a masking stimulus. The the environment have an impact on the allocation of
masking stimulus occupied the same space as the attention in a manner similar to their impact on sim-
first stimulus and was used to stop processing of the ple operant behavior. Thus, existing theoretical
first stimulus and eliminate afterimages. The mask- accounts of operant behavior may be useful for
ing stimulus was followed by either a short describing how differential consequences affect the
(200-millisecond) or long (800-millisecond) delay, goal-directed control of attention.
during which the screen was blank. At Time 2,
immediately after the delay, the face stimuli from Conditioning, Learning, and Attention
the earlier choice task or novel face stimuli were Unlike the relatively recent focus on the impact of
presented and followed immediately by a masking consequences on the allocation of human attention,
stimulus. Finally, the subjects were asked to report theories of conditioning and learning have a long
by pressing buttons whether the stimulus at Time 1 history of assuming reinforcement-related changes
had been made up of circles or squares and then in attention. For example, theories of discrimination
whether the face at Time 2 had been familiar or learning have commonly assumed that changes in
novel. An attentional blink effect in this procedure attending to relevant stimulus dimensions are based
is evidenced by poorer performance on the face rec- on the experience of differential reinforcement in
ognition task at Time 2 after a shorter delay between the presence of stimuli (e.g., Blough, 1969; Lovejoy,
Time 1 and Time 2. The usual interpretation of the 1965; Mackintosh, 1975; Mackintosh & Little, 1969;
attentional blink effect is that it results from the fact Nevin, Davison, & Shahan, 2005; Sutherland &
that attention is a limited resource that is temporar- Mackintosh, 1971; Zeaman & House, 1963). Many
ily taxed by performance at Time 1 and is therefore such models have also been closely related to the
not available again for some small amount of time study of Pavlovian conditioning. Phenomena such
thereafter. Thus, long-delay (800-millisecond) and as blocking (Kamin, 1968), overshadowing (Pavlov,
short-delay (200-millisecond) conditions were cho- 1927), and the relative validity effect (Wagner,
sen such that attention would typically be fully Logan, Haberlandt, & Price, 1968) show that CSs
available or taxed, respectively. The results showed that are more salient or are better predictors of USs
that regardless of whether attention was fully avail- reduce conditioning to jointly available stimuli that
able or taxed, recognition of faces at Time 2 was bet- are less salient or are poorer predictors of a US. The
ter for faces previously producing a gain or a loss Rescorla–Wagner (1972) model describes these cue
than for previously neutral stimuli. Thus, stimuli competition effects as being the result of competi-
that needed to be chosen or avoided because of their tion between stimuli for a limited amount of associa-
associated consequences in the choice task pro- tive value available from a US. Other theories,
duced improved recognition performance at Time 2. however, describe cue competition effects in terms
Most important, a typical attentional blink effect of inferred changes in attending to stimuli with
was obtained when attention was taxed with the experience rather than changes in US effectiveness.
395
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
Mackintosh’s (1975) attention theory is based on

the fact that stimuli that are better predictors of
reward attract increasing attention as conditioning
proceeds. Likewise, Lubow’s (1989) conditioned
attention theory suggests that attention to reward-
predictive stimuli increases in a manner consistent
with the principles of classical conditioning. Inter-
estingly, an alternative attention-based model for-
warded by Pearce and Hall (1980) predicts many
cue competition phenomena with the opposite
assumption—that attention to a stimulus decreases
when it is a reliable predictor of reinforcement. More
recent accounts have allowed for both increases and
decreases in attention by proposing multiple atten-
tion systems for different types of learning situations Figure 17.1. Schematic of a divided-attention
(Hogarth, Dickinson, & Duka, 2010). procedure for pigeons. B = blue; R = red.
Although an evaluation of the relative merits of
these attention-based theories of conditioning is samples made up of just one of the elements (see
beyond the scope of this chapter, all the theories Figure 17.1). In Maki and Leuin, line orientation
share the assumptions that attention is critical for (i.e., vertical or horizontal) and key color (i.e., red
learning associations between stimuli and that atten- or blue) were used for the two sets of element stim-
tion itself depends on previously experienced rela- uli. For single-element sample presentations, either
tions between stimuli and reinforcers. In a general a simple color (i.e., red or blue) or line orientation
sense, such theories might be thought of as address- (i.e., white vertical or horizontal lines on a black
ing how experience changes what Norman (1968) background) was presented. For compound sample
referred to as the pertinence of stimuli. The presentations, both a color and a line orientation
approach I explore in this chapter is clearly in the were presented simultaneously (e.g., a blue key with
same tradition as these other approaches in asserting superimposed white vertical lines). After both
that at least part of what is referred to as attention is single-element and compound sample stimuli, two
modified by experience in accordance with existing comparison stimuli were presented. The two com-
accounts of conditioning and learning. Specifically, parison stimuli were always single-element stimuli
research in my laboratory has been focused on and were either the two line orientations or the two
extending existing quantitative accounts of operant colors. One of the comparison stimuli had been
conditioning to attending. To do so, my lab has present in the sample, and choice of that stimulus
relied on procedures with animals that are aimed at was followed by a food reinforcer. Choice of the
more directly measuring attention. other stimulus was not followed by a reinforcer.
To study attention more directly with animals, Because the type of comparison stimuli to be pre-
Riley and colleagues (Riley & Leith, 1976; Riley & sented is unpredictable, the pigeons must attend to
Roitblat, 1978; Zentall & Riley, 2000) suggested both elements in the compound sample to perform
using procedures more similar to those used with accurately. Using this procedure, Maki and Leuin
humans and likely to produce selective processing found that to maintain accuracy at 80% for com-
as a result of capacity limitations (cf. Mackintosh, pound samples, the sample stimuli needed to be pre-
1975). One such procedure is a modified delayed sented for longer durations than for element samples.
matching-to-sample task used by Maki and Leuin Subsequently, Maki and Leith (1973) found
(1972) to study divided attention in pigeons. The superior accuracy for element-sample trials than for
pigeons were presented with compound samples compound-sample trials when the sample duration
consisting of a combination of two elements or was fixed at the value for both types of trials. Many
396
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
related experiments have obtained a similar result that were of either two colors (hereinafter called
(cf. Gottselig, Wasserman, & Young, 2001; Langley & color trials) or two line orientations (hereinafter
Riley, 1993; Leith & Maki, 1975; Santi, Grossi, & called line trials) were presented, and a peck on the
Gibson, 1982; Zentall, Sherburne, & Zhang, 1997). comparison stimulus that had appeared in the sam-
Non–attention-based hypotheses have been pro- ple produced a food reinforcer. Across conditions,
posed to account for the poorer accuracy with com- the probability of reinforcement for correct matches
pound samples than with element samples (e.g., for the two types of comparison stimuli was varied.
generalization decrement, coding decrement), but The overall probability of reinforcement remained at
the data generally support the suggestion that this .5, but the ratio of reinforcement probabilities for
difference results from the division of attention the color and line dimensions varied across values of
required for compound sample stimuli (see Roberts, 1:9, 1:3, 1:1, 3:1, and 9:1 for the two color and two
1998, and Zentall & Riley, 2000, for reviews). line dimensions, respectively. The results showed
that variations in the ratio of reinforcement for the
The Matching Law and Allocation of two dimensions produced orderly changes in accu-
Divided Attention racy for that dimension. Specifically, accuracy
To assess the utility of the matching law for describ- increased for the dimension associated with a higher
ing the effects of variations in relative reinforcement probability of reinforcement and decreased for the
on the allocation of attending, Shahan and Podle- dimension associated with a lower probability of
snik (2006) used a divided-attention procedure with reinforcement.
pigeons similar to that of Maki and Leith (1973). Shahan and Podlesnik (2006) applied the match-
Figure 17.2 shows a schematic of the procedure. ing law to changes in accuracy with changes in rein-
Trials started with a white trial-ready stimulus, a forcement allocation using log d as the measure of
single peck on which produced a compound-sample accuracy. Log d is a common measure in behavioral
stimulus with one of two colors (i.e., green or blue) approaches to signal detection and conditional dis-
and one of two line orientations (i.e., vertical or hor- crimination performance (e.g., Davison & Nevin,
izontal). The sample stimulus was presented for 5 1999; Davison & Tustin, 1978). Log d is particularly
seconds and was terminated response indepen- useful because it is bias free and ranges from 0 at
dently. Next, single-element comparison stimuli chance performance to infinity at perfect perfor-
mance. These metric properties make it similar to
response-rate measures typically used to character-
ize simple operant responding with the matching
law. Log d was calculated separately for color and
line trials such that
 S1   S2  
log d = 0.5 log  corr   corr   , (7)
 S1incorr   S2incorr  
where S1corr and S1incorr refer to correct and incorrect
responses after presentation of one sample (e.g.,
blue) and S2corr and S2incorr refer to correct and incor-
rect responses after the other sample (e.g., green).
To characterize changes in relative accuracy with
changes in relative reinforcement, Shahan and
Podlesnik used this version of the generalized
matching law:
Figure 17.2. Schematic of the divided-attention R 
procedure for pigeons used by Shahan and log dC − log d L = a log  C  + log b, (8)
Podlesnik (2006). G = green; B = blue.  RL 
397
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
where log dC and log dL refer to log d for color and the two types of comparison stimuli affected behav-
line trials, respectively, and RC and RL refer to rein- ior only at the choice point without changing
forcers obtained on color and line trials, respec- attending to the elements of the compound samples.
tively. The parameters a and log b refer to sensitivity Even if the subjects had attended equally to the ele-
of relative accuracy to relative reinforcement and ments of the compound samples, variations in rela-
bias in accuracy for one trial type unrelated to relative reinforcement at the comparison stimuli might
tive reinforcement, respectively. Figure 17.3 shows have resulted in changes in motivation to choose the
the fits of Equation 8 to the mean data across correct stimulus at the comparison choice point. To
pigeons. Equation 8 did a good job describing the assess this alternative account, Shahan and Podle-
effects of changes in relative reinforcement on rela- snik (2007) examined whether changes in the dura-
tive accuracy for the two types of comparison trials. tion of the compound samples altered the effects of
With a (i.e., sensitivity to relative reinforcement) differential reinforcement on performance in the
equal to 0.57 and log b (i.e., bias) equal to 0.26, same divided-attention task. They compared perfor-
Equation 8 accounted for 96% of the variance in the mance with sample durations of 2.25 seconds and
averaged data. Assuming that the changes in relative 0.75 seconds and examined sensitivity of both accu-
accuracy with the two types of comparison trials racy and choice-response speeds (i.e., l/latency) at
reflect differences in attending to the elements of the the comparison choice point. If differential rein-
compound stimuli, the fact that sensitivity (i.e., the forcement affected only motivation at the choice
a parameter) was greater than zero suggests that the point, sensitivity of accuracy to changes in alloca-
allocation of attention to the elements was sensitive tion of reinforcement should not depend on the
to the relative reinforcement associated with those sample duration.
elements. The finding that log b was greater than Figure 17.4 shows mean sensitivity values for
zero reflects the fact that overall accuracy also both accuracy and choice-response speeds for the
tended to be greater for colors than for lines across short and long sample durations. Consistent with
the range of relative reinforcement rates. the findings of Shahan and Podlesnik (2006), accu-
An alternative interpretation of the changes in racy on the two types of comparison trials was sensi-
accuracy in the Shahan and Podlesnik (2006) exper- tive to variations in relative reinforcement. In
iment is that variations in relative reinforcement for addition, sensitivity of accuracy was greater for the
longer sample duration. However, choice-response
speeds were only weakly sensitive to changes in
relative reinforcement and did not differ for the
short and long sample durations. Although overall
Figure 17.3. Generalized matching Figure 17.4. Sensitivity (i.e.,

law analysis of the effects of differen- a in Equation 8) of accuracy and
tial reinforcement on pigeons’ accu- choice-response speeds to dif-
racy in Shahan and Podlesnik’s (2006) ferential reinforcement when the
divided-attention experiment. The sample was either short or long in
fitted line and parameters values were the Shahan and Podlesnik (2007)
produced by least-squares regression experiment on divided attention
of Equation 8. of pigeons.
398
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
accuracy was lower with the short samples, Shahan into differential attending. Nonetheless, it is impor-
and Podlesnik (2007) showed that the lower sensi- tant to note that the applicability of the matching
tivity of accuracy with the shorter sample was not law to attending has only been directly assessed with
likely an artifact of the lower accuracies because it pigeons. Ultimately, assessing the utility of the
appeared to result from the pigeons attending too matching law for describing goal-directed attention
much to the element associated with the lower prob- will require direct examinations with humans in
ability of reinforcement. These findings led Shahan standard preparations such as those reviewed ear-
and Podlesnik to conclude that changes in motiva- lier. Regardless, the findings reviewed in the preced-
tion at the comparison choice point were likely not ing section suggest that considering both behavior
responsible for the effects of relative reinforcement and attention as limited resources requiring similar
on relative accuracy. allocation decisions based on differential conse-
Further support for the suggestion that differen- quences may indeed be useful.
tial reinforcement affects attending to the elements
of compound sample stimuli in the experiments Behavioral Momentum and the
of Shahan and Podlesnik (2006, 2007) came from Persistence of Attending
Shahan and Quick (2009). Specifically, using an Although I am unaware of any experiments with
adjusting sample duration procedure similar to that humans or nonhumans examining how differential
of Maki and Leuin (1972), we found that sample reinforcement affects the persistence of attending
stimuli associated with a lower reinforcement rate using procedures such as those described earlier, my
require a longer sample to maintain a constant accu- laboratory has examined the issue using observing
racy. This outcome was consistent with the sugges- responses of pigeons. Observing responses produce
tion that effects of differential reinforcement in the sensory contact with stimuli to be discriminated and
divided-attention task are mediated through changes have been used as an overt analog of attention for
in attending to the sample stimuli. more than 50 years (e.g., Wyckoff, 1952). In the
These findings suggest that the allocation of typical observing-response procedure, unsignaled
attention of both humans and pigeons is affected by periods of primary reinforcement availability for
differential consequences in a manner consistent some response alternate unpredictably with periods
with simple principles of operant conditioning. The in which the response does not produce reinforce-
choice-based account provided by the matching law ment (i.e., extinction). A second response (i.e., the
appears to provide a useful framework within which observing response) produces brief access to stimuli
to understand how differential consequences affect signaling whether reinforcement is available. A stim-
the allocation of attention. For example, Shahan and ulus signaling the availability of reinforcement is
Podlesnik (2006, 2007) suggested that the sensitiv- referred to as an S+, and a stimulus signaling
ity and bias parameters of Equation 8 could be use- extinction is referred to as an S−. Figure 17.5 shows
ful for providing quantitative descriptions of an example of an observing-response procedure for
goal-driven and stimulus-driven aspects of atten- pigeons. Responses on the white key on the right
tional control, respectively. They suggested that sen- produce primary reinforcement (i.e., food) on a
sitivity (i.e., a) might be used to capture the effects variable-interval (VI) schedule of reinforcement. (A
of variations in relative reinforcement on stimulus VI schedule arranges reinforcement for the first
pertinence or goal-directed control of attention and response after a variable amount of time around a
that bias in accuracy independent of reinforcement mean value.) The periods of VI food reinforcement
variations (i.e., log b) might reflect the effects of sen- on the key alternate unpredictably with unsignaled
sory features of the stimuli (i.e., sensory activation; periods in which reinforcement is not available for
Norman, 1968). Thus, the matching law appears pecking the key (i.e., extinction is in effect). Pecks
useful as a first approximation to replace concepts on the key on the left (i.e., the observing response)
such as a central administrator with a quantitative produce 15-second periods of stimuli correlated
theory of how differential consequences translate with the conditions of reinforcement on the food
399
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
(i.e., Shahan & Podlesnik, 2006, 2007). Thus, in

terms of evaluating the applicability of quantitative
accounts of operant behavior to attending, the
observing-response procedure appears to yield con-
clusions similar to the procedures discussed earlier.
To examine how differential reinforcement
affects the persistence of attending, researchers have
used methods such as those typically used to study
the resistance to change of simple operant behavior.
For example, Shahan, Magee, and Dobberstein
(2003) arranged a multiple schedule in which
observing responses of pigeons produced different
Figure 17.5. Schematic of an observing- stimuli in the two components of the multiple
response procedure for pigeons. W = white;
FR = fixed ratio; Ext = extinction; G = green; schedule. In a rich component, observing responses
R = red; VI = variable interval. produced an S+ associated with a higher rate of pri-
mary reinforcement. In a lean component, observing
key. If the VI schedule of food reinforcement is responses produced an S+ associated with a lower
available on the food key, both keys are lighted rate of primary reinforcement. Consistent with the
green for 15 seconds (i.e., S+). If extinction is in matching law data presented earlier, Shahan et al.
effect on the food key, both keys are lighted red for found that observing occurred at a higher rate in the
15 seconds (i.e., S−). Pigeons readily learn to peck rich than in the lean component during a predisrup-
the observing key and produce the stimuli differen- tion baseline. When performance was disrupted by
tially associated with the periods of primary rein- satiating the pigeons with presession or intercompo-
forcement and extinction. In addition, they learn to nent food, observing was more resistant to change in
discriminate S+ and S− and come to respond on the rich component than in the lean component.
the food key predominately during S+. Responses Thus, using observing responses as a measure of
on the observing key are used as the measure of attention, it appears that stimuli associated with a
looking at or attending to the stimuli to be discrimi- higher rate of reinforcement command more persis-
nated (see Dinsmoor, 1985, for discussion). tent attending. Furthermore, quantitative analyses
Consistent with the research described earlier revealed that resistance to change of observing and
using the divided-attention procedure, research in simple operant behavior maintained directly by the
my laboratory has shown that the matching law is food reinforcer were similarly sensitive to the rein-
also applicable to attending as measured with an forcement ratios arranged (i.e., b in Equation 6).
observing response. For example, when the rate of Thus, extending behavioral momentum theory from
primary reinforcement associated with a stimulus simple operant behavior to attending appears to be
increases, rats’ observing of that stimulus increases relatively straightforward.
in a manner well described by the single-response Partly on the basis of the observing data, Nevin
version of the matching law (i.e., Equation 2; see et al. (2005) and Nevin, Davison, Odum, and Sha-
Shahan & Podlesnik, 2008b). Furthermore, Shahan, han (2007) suggested that the effects of reinforce-
Podlesnik, and Jimenez-Gomez (2006) found that ment on attending are governed by behavioral
when pigeons are given a choice between two momentum theory. Specifically, the theory is based
observing keys and the relative rate of S+ presenta- on a version of behavioral momentum theory that
tion is varied between the keys, pigeons allocate predicts rates of an operant response during baseline
their observing in accordance with the generalized and disruption conditions such that
matching law (i.e., Equation 3). This result is con-
−x
sistent with the results obtained in the delayed B = k exp , (9)
(r / ra )
b
matching-to-sample procedure reported earlier s
400
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
where B is response rate, rs is the rate of reinforce- suggest that behavioral momentum theory may
ment in the presence of the stimulus in which the indeed be useful for understanding how differential
behavior is occurring, and ra is the overall rate of reinforcement contributes to the allocation and per-
background reinforcement in the entire session. The sistence of attending. Nonetheless, the research
parameters k, x, and b correspond to asymptotic effort directed at examining how reinforcement
response rate, background disruption that reduces affects the persistence of attending has just begun.
responding, and sensitivity to relative reinforce- Data from a variety of typical paradigms for studying
ment, respectively. Under baseline conditions, attention will be required to more fully assess the
Equation 9 predicts functions relating response rates utility of the approach.
and reinforcement rates that are nearly indistin-
guishable from Herrnstein’s (1970) single-option
Attention, Conditioned
version of the matching law (i.e., Equation 2; see
Reinforcement, and Signposts
Nevin et al., 2005). When additional terms are
included in the numerator for specific disruptors, The research reviewed in the preceding sections
Equation 9 also captures the finding that responding suggests that differential reinforcement affects both
is more resistant to disruption in contexts associated the allocation and the persistence of attending to
with higher rates of reinforcement. On the basis of stimuli. One interpretation of how differential rein-
Shahan et al.’s (2003) finding that the persistence forcement affects attending is that stimuli associated
of observing (i.e., attending) appears to be affected with reinforcers become conditioned reinforcers as a
by reinforcement in a manner similar to simple result of Pavlovian conditioning (Mackintosh, 1974;
response rates, Nevin et al. (2005) suggested the Williams, 1994). From this perspective, the effects
following equation: of differential reinforcement on attending result
−x from the strengthening of a connection between an
p ( A s ) = exp , (10) initially neutral stimulus and an existing reinforcer.
(r / ra )
b
s As a result, initially neutral stimuli may come to
where p(As) is the probability of attending to a sam- reinforce behavior or attending that produces con-
ple stimulus and rs is now the rate of reinforcement tact with them. Thus, quantitative models of simple
associated with attending to the stimulus and ra operant behavior may describe the effects of differ-
remains the overall rate of background reinforce- ential reinforcement on the allocation and persis-
ment in the session. The other parameters are as in tence of attending because the stimuli being
Equation 9. The scalar k is not required because the attended to are themselves reinforcers. Although
asymptote of the probability of attending is at 1.0. In such an account works fairly well when considered
effect, Equation 10 assumes that the probability and only within the context of the effects of differential
persistence of attending is governed by reinforce- reinforcement on the allocation of attending, the
ment in the same way as is simple operant behavior. account encounters some difficulties within the con-
Nevin et al. (2005, 2007) have shown that infer- text of the persistence of attending. I consider these
ences about changes in attending based on Equation issues in turn in the following sections.
10 can be used in a larger model addressing the
effects of reinforcement on discrimination perfor- Conditioned Reinforcement and
mance in the steady state and during disruption. the Allocation of Attention
The use of Equation 10 in this broader model of To understand how conditioned reinforcement
stimulus control has provided an account of a wide might provide an account of the effects of differen-
variety of reinforcement-related effects in the stimu- tial reinforcement on the allocation of attending,
lus control literature, including the effects of rein- consider Shahan and Podlesnik’s (2006) divided-
forcement on the persistence of discriminative attention experiment. In that experiment, each of
performance when disrupted (e.g., Nevin, Milo, the two elements making up the compound sample
Odum, & Shahan, 2003). These initial successes stimuli in the delayed matching-to-sample procedure
401
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
was differentially associated with reinforcement. the allocation of attention. Research evaluating the
When the color dimension was associated with a different models in simple choice situations has doc-
higher probability of reinforcement than the line umented several important phenomena contributing
dimension, the color appeared to maintain more to how conditioned reinforcers govern choice (see
attending than the line. An account based on condi- Grace, 1994, and Mazur, 2001, for reviews). A
tioned reinforcement would suggest that the color potentially fruitful research endeavor might be to
served as a better conditioned reinforcer because it examine whether similar effects occur in tasks
was a better predictor of reinforcement than was the designed specifically to study the effects of differen-
line. As a result of being a more potent conditioned tial reinforcement on attention. Regardless of the
reinforcer, the color would be expected to more ultimate utility of the models for describing how
effectively strengthen attending to it. The resultant reinforcement affects the allocation of attention, an
data relating variations in relative reinforcement to interpretation of the effects of reinforcement on the
attending might be expected to conform to the persistence of attending based on conditioned rein-
matching law because a variety of quantitative forcement encounters some difficulties, as described
extensions of the generalized matching law (i.e., in the next section.
Equation 3) to choice between conditioned reinforc-
ers predict just this type of effect (see Mazur, 2001, Conditioned Reinforcement and
for review). The specific details of those models are the Persistence of Attending
not important for the present purposes, but many of As I have noted, the only research I know of exam-
the models take the general form ining the effects of differential reinforcement on the
a
r  V  persistence of attending is Shahan et al.’s (2003)
B1
= b 1   1  , (11) work with observing responses. Shahan et al.
B2  r2   V2  showed that stimuli associated with higher rates of
where B1 and B2 refer to behavior that produces con- primary reinforcement maintain more persistent
tact with two conditioned reinforcers, r1 and r2 refer attending as measured with observing responses.
to frequencies of production of the conditioned This outcome could suggest that the stimulus asso-
reinforcers, and V1 and V2 are summary terms ciated with the higher rate of primary reinforce-
describing the value or strengthening effects of the ment was a more effective conditioned reinforcer
conditioned reinforcers. The models differ in how and, hence, maintained more persistent attending.
value is calculated, but most of the approaches to In fact, observing responses are often considered
calculating value share important characteristics one of the best ways to study the conditioned rein-
with existing theoretical accounts of Pavlovian con- forcing effects of reinforcement-associated stimuli
ditioning (see Chapters 13 and 14, this volume). (e.g., Dinsmoor, 1983; Fantino, 1977; Williams,
Assuming that such models can be extended to 1994). Regardless, it is important to note that Sha-
attention, they predict relative changes in attending han et al. examined only the effects of differences
to stimuli with changes in relative value such as in primary reinforcement on the persistence of
those obtained in Shahan and Podlesnik (2006, observing. If stimuli associated with differential
2007) or even in the human attention experiments reinforcement do affect the allocation and persis-
reviewed earlier. Furthermore, the models accu- tence of attention because they have themselves
rately predict changes in the allocation of attending become conditioned reinforcers, variations in the
when the relative frequency of reinforcement- rate or value of such stimuli would themselves be
associated stimuli is varied as in the concurrent expected to contribute to the persistence of attend-
observing-response experiment of Shahan, Podle- ing in the same way as primary reinforcers. How-
snik, and Jimenez-Gomez (2006). Thus, such mod- ever, experiments from my laboratory have shown
els of conditioned reinforcement may hold promise that putative conditioned reinforcers do not con-
as a means to understand how stimuli differentially tribute to the persistence of attending in the same
associated with primary reinforcers come to govern way as primary reinforcers.
402
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Shahan and Podlesnik (2005) showed that an disruption of observing and frequency or value of
observing response that produced more frequent S+ presentations had no meaningful relationship.
presentations of a food-associated stimulus (i.e., Given that both frequency and value of S+ presenta-
S+) occurred more frequently than an observing tions would be expected to affect resistance to
response that produced less frequent S+ presenta- disruption if S+ presentations were serving as con-
tions. More important, the variation in frequency of ditioned reinforcers, Shahan and Podlesnik con-
S+ presentation occurred in the absence of differ- cluded that S+ presentations might affect the
ences in frequency of primary reinforcement. Thus, frequency of observing through some other mecha-
this result is consistent with the earlier choice nism. Thus, at present, it appears that the effects of
results and with what would be expected if S+ pre- differential reinforcement on the persistence of
sentations strengthen responding in the same fash- attending might not be mediated through the pro-
ion as primary reinforcers. Despite the effect of S+ cess of conditioned reinforcement. In short, stimuli
frequency on the rate of observing, however, differ- that might be thought to command attention
ences in S+ frequency had no impact on the persis- because they have become conditioned reinforcers
tence of observing under conditions of disruption. do not appear to have an impact on the persistence
This result is not what would be expected if S+ pre- of attending as do other reinforcers.
sentations functioned as conditioned reinforcers. Although the effects of differential reinforcement
Additional experiments by Shahan and Podlesnik on the allocation of attending discussed in the pre-
(2008a) examined the effects of variations in the ceding section might still be mediated through the
value of an S+ on observing rates and resistance to process of conditioned reinforcement, it would not
change. The value of an S+ was varied by changing be parsimonious to assert different mechanisms for
the predictive relation between S + and the delivery the allocation and persistence of attention. As I
of primary reinforcement in variety of ways. show, the debate surrounding the utility of the con-
Although the details of the manipulations are not cept of conditioned reinforcement is considerable.
important for present purposes, some of the meth- As a result, the generalized matching law–based the-
ods pitted S+ value against the rate of primary rein- ories of conditioned reinforcement might also be
forcement, and some varied S+ value while holding interpreted in other terms.
constant rates of primary reinforcement. The results
showed that as with higher frequencies of S+ pre- Signposts as an Alternative to
sentation in Shahan and Podlesnik (2005), higher Conditioned Reinforcement
valued S+ presentations maintained higher rates of A long history of skepticism surrounds the concept
observing. However, differences in the value of S+ of conditioned reinforcement. Data traditionally
presentations had no impact on the persistence of interpreted as reflecting the acquired strengthening
observing under conditions of disruption. If higher effects of stimuli associated with reinforcers may be
valued S+ presentations maintained higher observ- interpreted in a variety of other ways. Although a
ing rates because they served as more potent condi- review of all these alternatives and the supporting
tioned reinforcers, then higher valued S+ data is beyond the scope of this chapter, Shahan
presentations should also have produced greater (2010) provided a more thorough treatment of the
resistance to disruption, but they did not. subject. Many commentators have noted that the
Shahan and Podlesnik (2008b) provided a sum- effects of stimuli associated with reinforcers on
mary quantitative analysis of all these observing and response rates and choice might be more consistent
resistance-to-disruption experiments. Although with a signaling or guidance process than with an
rates of observing were clearly affected by the fre- acquired reinforcement-strengthening process (e.g.,
quency and value of S+ presentations, resistance to Bolles, 1975; Davison & Baum, 2006; Longstreth,
disruption of observing was only an orderly function 1971; Rachlin, 1976; Staddon, 1983). Although a
of the rate of primary reinforcement in the context variety of names have been used for reinforcement-
in which observing was occurring. Resistance to associated stimuli in such an account, I have
403
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
adopted signposts or means to an end as suggested by A traditional account of results such as those of
Wolfe (1936), Longstreth (1971), and Bolles (1975). Bolles (1961) and Davison and Baum (2006) would
According to a signpost account, stimuli associated suggest that stimuli associated with the delivery of a
with reinforcers guide behavior because they predict primary reinforcer might have multiple functions.
where reinforcers are located in space, time, or both. Specifically, in the Bolles experiments, one might
Thus, an observing response that produces contact interpret the effect of the click in terms of both its
with a stimulus associated with a primary reinforcer strengthening effects and its discriminative stimulus
serves to guide the organism to the reinforcer by effects. As I noted, the occurrence of operant behav-
providing feedback about how or where the rein- ior is known to be modulated by the presence of a
forcer is obtained. Behavior that produces signposts discriminative stimulus in the presence of which it
occurs because the signpost is useful for getting to has been reinforced. In the Bolles experiment, the
the primary reinforcer, not because the signpost has click associated with food delivery might be inter-
acquired the capacity to strengthen behavior in a preted to function as a discriminative stimulus for
reinforcement-like manner. In this sense, signposts pressing the appropriate lever (i.e., the same lever in
might also be thought of as a means to an end for the first experiment and the other lever in the sec-
acquiring reinforcers. ond experiment). A similar account might be
A set of two experiments by Bolles (1961) is use- applied to the related findings of Davison and Baum:
ful for contrasting a signpost-based account and a When a food-paired stimulus signals that engaging
conditioned reinforcement account. In the first in a response is likely to be reinforced, the stimulus
experiment, rats responded on two levers that were sets the occasion for repeating the response, even if
available at the same time and periodically pro- that response is not the one that produced the stim-
duced a food pellet accompanied by a click of the ulus. Thus, any conditioned reinforcement–based
pellet dispenser. Receipt of a pellet plus click on strengthening effects of the food-paired stimulus
one of the levers was predictive of future pellets for would be interpreted as being overridden by such dis-
pressing that lever for a short time. During an criminative stimulus effects of the stimulus. Although
extinction test, the two levers were present, but this account seems plausible enough on its surface, it
only one of them produced the click. Consistent is difficult to find any compelling data in the litera-
with what would be expected if the click had ture demonstrating strengthening effects of food-
acquired the capacity to function as a reinforcer, associated stimuli above and beyond such signaling
the rats preferred to press the lever that produced effects (see Bolles, 1975; Davison & Baum, 2006;
the click than the lever that did not. Nonetheless, Longstreth, 1971; Rachlin, 1976; Shahan, 2010; and
in the second experiment, receipt of a pellet plus Staddon, 1983, for reviews). In short, it does not
click on one lever predicted that subsequent pellets appear that both a signaling and a strengthening func-
were more likely on the other lever for a short tion are required to account for effects typically
period. Unlike the first experiment, during the ascribed to conditioned reinforcement; a discrimina-
extinction test the rats shifted their preference away tive or signaling effect appears to be enough.
from the lever that produced the click. If the click The suggestion that stimuli associated with rein-
was a conditioned reinforcer, one would expect it forcers might serve as signals for behavior rather
to further strengthen the response that produced it than strengthen it is also consistent with some con-
during extinction. Bolles concluded that previous temporary treatments of associative learning. For
demonstrations of apparent acquired conditioned- example, Gallistel and Gibbon (2002; see also Bal-
reinforcing effects of stimuli might result not from sam & Gallistel, 2009) suggested that conditioning
a reinforcement process, but because such stimuli occurs not because of a reinforcement-driven back-
are signposts for how subsequent food is to be propagation (i.e., strengthening) mechanism but
obtained. Davison and Baum (2006) reached a simi- because organisms learn that some events in the
lar conclusion on the basis of a related but more environment predict other events. Gallistel and
extensive set of experiments with pigeons. Gibbon suggested that organisms learn temporal,
404
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
spatial, and predictive regularities in the environ- expected to maintain more attending to it, such
ment in the absence of any reinforcement-like back- signposts would not be expected to contribute to the
propagation process. This suggestion is based on a strength of attending (as would be predicted by a
wealth of data showing that performance in both back-propagation response-strengthening condi-
Pavlovian and operant conditioning preparations is tioned reinforcement process). In terms of the allo-
often strikingly different from what would be cation of attention, discarding the notion that
expected if learning resulted from a strengthening stimuli associated with existing reinforcers them-
process. As with Gallistel and Gibbon, but based selves become reinforcers has little impact on the
largely on data separate from operant choice experi- utility of the generalized matching law–based theo-
ments, Davison and Baum (2006) have also sug- ries of conditioned reinforcement. Such theories
gested that what are traditionally referred to as provide a quantitative description of how choice
reinforcers of operant behavior have their effects by between stimuli follows changes in the predictive
predicting additional such events and, thus, guiding relations between those stimuli and reinforcers.
rather than strengthening behavior. The notion of value can be interpreted to reflect the
Although a full discussion of such a non– utility of the stimuli in terms of their efficacy in
reinforcement-based approach to conditioning and guiding organisms to reinforcers. The concept of
learning is not appropriate here (but see Shahan, value need not carry any connotation of an acquired
2010), its most important aspect is that it provides reinforcement-like strengthening effect for the
mechanisms by which initially neutral stimuli might models to be useful in describing how such stimuli
come to guide behavior as signposts without acquir- govern the allocation of behavior and attention.
ing conditioned-reinforcing effects. Because this Interpreting the models in such a way is consistent
approach also asserts that the effect of primary rein- with more general maximization-based economic or
forcers is to guide rather than strengthen behavior, regulatory approaches to instrumental action within
referring to initially neutral stimuli that come to which the models might be included (e.g., Rachlin,
guide behavior as discriminative stimuli would not Battalio, Kagel, & Green, 1981; Rachlin, Green,
be appropriate given the traditional meaning of that Kagel, & Battalio, 1976).
term. For this reason, I have adopted the term sign-
posts to describe a more general guidance and
Summary and Conclusions
means-to-an-end effect as an alternative to conditioned
reinforcement. Terminological issues aside, even if Both operant behavior and at least part of what is
one rejects the assertion that primary reinforcers do usually referred to as attention can be characterized
not strengthen behavior, an account of conditioned as goal-directed activities. Although researchers
reinforcement in terms of a signposts or means-to- have only recently started examining the effects of
an-end account is useful for understanding the differential consequences on attention, such conse-
effects of differential reinforcement on the allocation quences appear to affect attention in much the same
and persistence of attending. way as they affect simple operant behavior. At this
From the perspective of a signpost-based early stage of the investigation, quantitative theories
account, differential reinforcement might have its describing how consequences affect the allocation
impact on the allocation and persistence of attend- and persistence of operant behavior appear readily
ing because attending to stimuli that are more pre- applicable to attention. Such theories might provide
dictive of reinforcers is instrumental in guiding the a starting point for replacing placeholder concepts
organism to those reinforcers. From this perspec- such as a central administrator responsible for the
tive, interpreting the data from the Shahan and goal-directed control of attention with a quantitative
Podlesnik (2005, 2008a) experiments on the persis- framework for translating experience into differen-
tence of observing responses becomes easier. tial attending. Finally, one interpretation of the
Although more frequent production of a signpost or effects of differential reinforcement on attending
production of a more informative signpost might be to stimuli is that stimuli predictive of existing
405
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
reinforcers acquire the capacity to reinforce activity Bolles, R. C. (1961). Is the “click” a token reward?
that produces contact with them (i.e., they become Psychological Record, 11, 163–168.
conditioned reinforcers). However, an alternative Bolles, R. C. (1975). Theory of motivation (2nd ed.). New
approach suggests that such stimuli serve as sign- York, NY: Harper & Row.
posts that guide behavior to reinforcers and serve as a Bonnel, A.-M., & Prinzmetal, W. (1998). Dividing
attention between the color and the shape of
means to an end for acquiring reinforcers. Thus, dif-
objects. Perception and Psychophysics, 60, 113–124.
ferentially attending to stimuli predictive of reinforc- doi:10.3758/BF03211922
ers might be considered instrumental in acquiring Bryson, S. E., Wainwright-Sharp, J. A., & Smith, L. M.
reinforcers rather than such stimuli strengthening (1990). Autism: A developmental spatial neglect
attending through a back-propagation mechanism. syndrome? In J. Enns (Ed.), The development of atten-
Regardless of the ultimate mechanism responsible for tion: Research and theory (pp. 405–427). Amsterdam,
the Netherlands: Elsevier. doi:10.1016/S0166-
the effects of differential reinforcement on attention, 4115(08)60468-9
the fact remains that both operant behavior and atten-
Catania, A. C. (1998). Learning (4th ed.). Upper Saddle
tion are affected by consequences in a manner that is River, NJ: Prentice-Hall.
usefully described by existing quantitative models. Cohen, S. L. (1996). Behavioral momentum of typing
Because the parameters known to affect behavior in behavior in college students. Journal of Behavior
these models are easily manipulated, these theories Analysis and Therapy, 1, 36–51.
may lead to practical applications designed to affect Davison, M., & Baum, W. M. (2006). Do conditional
attention and other important behavior (see Volume reinforcers count? Journal of the Experimental
2, Chapters 5 and 7, this handbook). Analysis of Behavior, 86, 269–283. doi:10.1901/
jeab.2006.56-05
Davison, M., & Nevin, J. A. (1999). Stimuli, reinforc-
References ers, and behavior: An integration. Journal of the
Ahearn, W. H., Clark, K. M., Gardenier, N. C., Chung, B. Experimental Analysis of Behavior, 71, 439–482.
I., & Dube, W. V. (2003). Persistence of stereotyped doi:10.1901/jeab.1999.71-439
behavior: Examining the effects of external reinforc- Davison, M., & Tustin, R. D. (1978). The relation between
ers. Journal of Applied Behavior Analysis, 36, 439–448. the generalized matching law and signal-detection
doi:10.1901/jaba.2003.36-439 theory. Journal of the Experimental Analysis of
American Psychiatric Association. (1994). Diagnostic Behavior, 29, 331–336. doi:10.1901/jeab.1978.29-331
and statistical manual of mental disorders (4th ed.). Della Libera, C., & Chelazzi, L. (2006). Visual selective
Washington, DC: Author. attention and the effects of momentary rewards.
Baddeley, A. D. (1986). Working memory. Oxford, Psychological Science, 17, 222–227. doi:10.1111/
England: Oxford University Press. j.1467-9280.2006.01689.x
Balsam, P. D., & Gallistel, C. R. (2009). Temporal Della Libera, C., & Chelazzi, L. (2009). Learning to
maps and informativeness in associative learning. attend and to ignore is a matter of gains and losses.
Trends in Neurosciences, 32, 73–78. doi:10.1016/j. Psychological Science, 20, 778–784. doi:10.1111/
tins.2008.10.004 j.1467-9280.2009.02360.x
Baum, W. M. (1974). On two types of deviation from the Dijksterhuis, A., & Aarts, H. (2010). Goals, attention, and
matching law: Bias and undermatching. Journal of (un)consciousness. Annual Review of Psychology, 61,
the Experimental Analysis of Behavior, 22, 231–242. 467–490. doi:10.1146/annurev.psych.093008.100445
doi:10.1901/jeab.1974.22-231 Dinsmoor, J. A. (1983). Observing and conditioned rein-
Baum, W. M. (1979). Matching, undermatching, and forcement. Behavioral and Brain Sciences, 6, 693–728.
overmatching in studies of choice. Journal of the doi:10.1017/S0140525X00017969
Experimental Analysis of Behavior, 32, 269–281. Dinsmoor, J. A. (1985). The role of observing and atten-
doi:10.1901/jeab.1979.32-269 tion in establishing stimulus control. Journal of the
Baum, W. M., & Rachlin, H. C. (1969). Choice as time Experimental Analysis of Behavior, 43, 365–381.
allocation. Journal of the Experimental Analysis of doi:10.1901/jeab.1985.43-365
Behavior, 12, 861–874. doi:10.1901/jeab.1969.12-861 Dube, W. V., & McIlvane, W. J. (1997). Reinforcer fre-
Blough, D. S. (1969). Attention shifts in a maintained quency and restricted stimulus control. Journal of
discrimination. Science, 166, 125–126. doi:10.1126/ the Experimental Analysis of Behavior, 68, 303–316.
science.166.3901.125 doi:10.1901/jeab.1997.68-303
406
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Ehrman, R. N., Robbins, S. J., Bromwell, M. A., Lankford, Igaki, T., & Sakagami, T. (2004). Resistance to change
M. E., Monterosso, J. R., & O’Brien, C. P. (2002). in goldfish. Behavioural Processes, 66, 139–152.
Comparing attentional bias to smoking cues in cur- doi:10.1016/j.beproc.2004.01.009
rent smokers, former smokers, and non-smokers
James, W. (1890). The principles of psychology (Vol. 1).
using a dot-probe task. Drug and Alcohol Dependence,
New York, NY: Holt. doi:10.1037/11059-000
67, 185–191. doi:10.1016/S0376-8716(02)00065-0
Fantino, E. (1977). Conditioned reinforcement: Choice Johnsen, B. H., Laberg, J. C., Cox, W. M., Vaksdal, A., &
and information. In W. K. Honig & J. E. R. Staddon Hugdahl, K. (1994). Alcoholic subjects’ attentional
(Eds.), Handbook of operant behavior (pp. 313–339). bias in the processing of alcohol-related words.
Englewood Cliffs, NJ: Prentice-Hall. Psychology of Addictive Behaviors, 8, 111–115.
doi:10.1037/0893-164X.8.2.111
Gallistel, C. R., & Gibbon, J. (2002). The symbolic founda-
tions of conditioned behavior. Mahwah, NJ: Erlbaum. Kahneman, D. (1973). Attention and effort. Englewood
Cliffs, NJ: Prentice Hall.
Gopher, D. (1992). The skill of attention control:
Acquisition and execution of attention strategies. Kamin, L. J. (1968). “Attention-like” processes in clas-
In D. E. Meyer & S. Kornblum (Eds.), Attention and sical conditioning. In M. R. Jones (Ed.), Miami
performance XIV: Synergies in experimental psychol- Symposium on the Prediction of Behavior: Aversive
ogy, artificial intelligence, and cognitive neuroscience stimulation (pp. 9–31). Miami, FL: University of
(pp. 299–322). Cambridge, MA: MIT Press. Miami Press.
Gottselig, J. M., Wasserman, E. A., & Young, M. E. Langley, C. M., & Riley, D. A. (1993). Limited capacity
(2001). Attentional trade-offs in pigeons learn- information processing and pigeon matching-to-sample:
ing to discriminate newly relevant visual stimulus Testing alternative hypotheses. Animal Learning and
dimensions. Learning and Motivation, 32, 240–253. Behavior, 21, 226–232. doi:10.3758/BF03197986
doi:10.1006/lmot.2000.1081 Leith, C. R., & Maki, W. S. (1975). Attention shifts dur-
Grace, R. C. (1994). A contextual model of concurrent- ing matching-to-sample performance in pigeons.
chains choice. Journal of the Experimental Analysis of Animal Learning and Behavior, 3, 85–89. doi:10.3758/
Behavior, 61, 113–129. doi:10.1901/jeab.1994.61-113 BF03209105
Grimes, J. A., & Shull, R. L. (2001). Response- Loewenstein, Y., & Seung, H. S. (2006). Operant match-
independent milk delivery enhances persistence of ing is a generic outcome of synaptic plasticity based
pellet-reinforced lever pressing by rats. Journal of on the covariance between reward and neural activ-
the Experimental Analysis of Behavior, 76, 179–194. ity. Proceedings of the National Academy of Sciences
doi:10.1901/jeab.2001.76-179 of the United States of America, 103, 15224–15229.
doi:10.1073/pnas.0505220103
Harper, D. N. (1999). Drug-induced changes in respond-
ing are dependent on baseline stimulus-reinforcer Longstreth, L. E. (1971). A cognitive interpretation of sec-
contingencies. Psychobiology, 27, 95–104. ondary reinforcement. In J. K. Cole (Ed.), Nebraska
symposium on motivation (Vol. 19, pp. 33–81).
Herrnstein, R. J. (1961). Relative and absolute strength of Lincoln: University of Nebraska Press.
response as a function of frequency of reinforcement.
Journal of the Experimental Analysis of Behavior, 4, Lovaas, O. I., Koegel, R. L., & Schreibman, L. (1979).
267–272. doi:10.1901/jeab.1961.4-267 Stimulus overselectivity in autism: A review of
research. Psychological Bulletin, 86, 1236–1254.
Herrnstein, R. J. (1970). On the law of effect. Journal of doi:10.1037/0033-2909.86.6.1236
doi:10.1901/jeab.1970.13-243 Lovejoy, E. (1965). An attention theory of discrimina-
tion learning. Journal of Mathematical Psychology, 2,
Herrnstein, R. J. (1990). Rational choice theory: 342–362. doi:10.1016/0022-2496(65)90009-X
Necessary but not sufficient. American Psychologist,
45, 356–367. doi:10.1037/0003-066X.45.3.356 Lubman, D. I., Peters, L. A., Mogg, K., Bradley, B. P., &
Deakin, J. F. W. (2000). Attentional bias for drug
Herrnstein, R. J., Rachlin, H., & Laibson, D. I. (1997). cues in opiate dependence. Psychological Medicine,
The matching law: Papers in psychology and economics. 30, 169–175. doi:10.1017/S0033291799001269
Cambridge, MA: Harvard University Press.
Lubow, R. E. (1989). Latent inhibition and condi-
Hogarth, L., Dickinson, A., & Duka, T. (2010). Selective tioned attention theory. New York, NY: Cambridge
attention to conditioned stimuli in human discrimi- University Press. doi:10.1017/CBO9780511529849
nation learning: Untangling the effect of outcome
prediction, value, arousal and uncertainty. In C. Luck, S. J., & Vecera, S. P. (2002). Attention. In H.
J. Mitchell & M. E. Le Pelley (Eds.), Attention and Pashler & S. Yantis (Eds.), Stevens’s handbook of
associative learning (pp. 71–97). Oxford, England: experimental psychology: Vol. 1. Sensation and percep-
Oxford University Press. tion (3rd ed., pp. 235–286). New York, NY: Wiley.
407
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
Mace, F. C., Lalli, J. S., Shea, M. C., Lalli, E. P., West, B. Nevin, J. A., & Grace, R. C. (2000). Behavioral momentum
J., Roberts, M., & Nevin, J. A. (1990). The momen- and the law of effect. Behavioral and Brain Sciences, 23,
tum of human behavior in a natural setting. Journal 73–90. doi:10.1017/S0140525X00002405
of the Experimental Analysis of Behavior, 54, 163–172. Nevin, J. A., Milo, J., Odum, A. L., & Shahan, T. A.
doi:10.1901/jeab.1990.54-163 (2003). Accuracy of discrimination, rate of
Mackintosh, N. J. (1974). The psychology of animal learn- responding, and resistance to change. Journal of
ing. London, England: Academic Press. the Experimental Analysis of Behavior, 79, 307–321.
doi:10.1901/jeab.2003.79-307
Mackintosh, N. J. (1975). A theory of attention: Variations
in the associability of stimuli with reinforcement. Nevin, J. A., Tota, M. E., Torquato, R. D., & Shull, R. L.
Psychological Review, 82, 276–298. doi:10.1037/ (1990). Alternative reinforcement increases resis-
h0076778 tance to change: Pavlovian or operant contingencies?
Mackintosh, N. J., & Little, L. (1969). Intradimensional
359–379. doi:10.1901/jeab.1990.53-359
and extradimensional shift learning by pigeons.
Psychonomic Science, 14, 5–6. Norman, D. A. (1968). Toward a theory of memory
and attention. Psychological Review, 75, 522–536.
Madden, G. J., & Bickel, W. K. (Eds.). (2010). Impulsivity: doi:10.1037/h0026699
The behavioral and neurological science of discount-
ing. Washington, DC: American Psychological Norman, D. A., & Shallice, T. (1986). Attention to action:
Association. doi:10.1037/12069-000 Willed and automatic control of behavior. In R.
J. Davidson, G. E. Schwartz, & D. Shapiro (Eds.),
Maki, W. S., & Leith, C. R. (1973). Shared attention Consciousness and self-regulation (Vol. 4, pp. 1–18).
in pigeons. Journal of the Experimental Analysis of New York, NY: Plenum.
Pashler, H. E. (1998). The psychology of attention.
Maki, W. S., & Leuin, T. C. (1972). Information- Cambridge, MA: MIT Press.
processing by pigeons. Science, 176, 535–536.
doi:10.1126/science.176.4034.535 Pavlov, I. P. (1927). Conditioned reflexes. Oxford,
England: Oxford University Press.
Mazur, J. E. (2001). Hyperbolic value addition and gen-
eral models of animal choice. Psychological Review, Pearce, J. M., & Hall, G. (1980). A model for Pavlovian
108, 96–112. doi:10.1037/0033-295X.108.1.96 learning: Variations in the effectiveness of con-
ditioned but not of unconditioned stimuli.
Mazur, J. E. (2006). Learning and behavior (6th ed.). Psychological Review, 87, 532–552. doi:10.1037/
Upper Saddle River, NJ: Prentice Hall. 0033-295X.87.6.532
McClure, S. M., Laibson, D. I., Loewenstein, G., & Posner, M. I. (1980). Orienting of attention. Quarterly
Cohen, J. D. (2004). Separate neural systems value Journal of Experimental Psychology, 32, 3–25.
immediate and delayed momentary rewards. Science, doi:10.1080/00335558008248231
306, 503–507. doi:10.1126/science.1100907
Quick, S. L., & Shahan, T. A. (2009). Behavioral momen-
Nestor, P. G., & O’Donnell, B. F. (1998). The mind adrift: tum of cocaine self-administration: Effects of fre-
Attentional dysregulation in schizophrenia. In R. quency of reinforcement on resistance to change.
Parasuraman (Ed.), The attentive brain (pp. 527–546). Behavioural Pharmacology, 20, 337–345. doi:10.1097/
Cambridge, MA: MIT Press. FBP.0b013e32832f01a8
Nevin, J. A. (1974). Response strength in multiple sched- Rachlin, H. (1976). Behavior and learning. San Francisco,
ules. Journal of the Experimental Analysis of Behavior, CA: Freeman.
21, 389–408. doi:10.1901/jeab.1974.21-389
Rachlin, H., Battalio, R., Kagel, J., & Green, L. (1981).
Nevin, J. A. (1992). An integrative model for the study of Maximization theory in behavioral psychol-
behavioral momentum. Journal of the Experimental ogy. Behavioral and Brain Sciences, 4, 371–417.
Analysis of Behavior, 57, 301–316. doi:10.1901/ doi:10.1017/S0140525X00009407
jeab.1992.57-301
Rachlin, H., Green, L., Kagel, J. H., & Battalio, R. (1976).
Nevin, J. A., Davison, M., Odum, A. L., & Shahan, T. Economic demand theory and psychological studies
A. (2007). A theory of attending, remembering, of choice. In G. Bower (Ed.), The psychology of learn-
and reinforcement in delayed matching to sample. ing and motivation (Vol. 10, pp. 129–154). New York,
Journal of the Experimental Analysis of Behavior, 88, NY: Academic Press.
285–317. doi:10.1901/jeab.2007.88-285
Raymond, J. E., & O’Brien, J. L. (2009). Selective visual
Nevin, J. A., Davison, M., & Shahan, T. A. (2005). A the- attention and motivation: The consequences of value
ory of attending and reinforcement in conditional dis- learning in an attentional blink task. Psychological
criminations. Journal of the Experimental Analysis of Science, 20, 981–988. doi:10.1111/j.1467-9280.2009.
Behavior, 84, 281–303. doi:10.1901/jeab.2005.97-04 02391.x
408
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Rescorla, R. A. (1988). Pavlovian conditioning: Its not Shahan, T. A., & Podlesnik, C. A. (2007). Divided atten-
what you think it is. American Psychologist, 43, tion and the matching law: Sample duration affects
151–160. doi:10.1037/0003-066X.43.3.151 sensitivity to reinforcement allocation. Learning and
Rescorla, R. A. (1998). Instrumental learning: Nature and Behavior, 35, 141–148. doi:10.3758/BF03193049
persistence. In M. Sabourin, F. Craik, & M. Robert Shahan, T. A., & Podlesnik, C. A. (2008a). Conditioned
(Eds.), Advances in psychological science: Vol. 2. reinforcement value and resistance to change. Journal
Biological and cognitive aspects (pp. 239–257). Hove, of the Experimental Analysis of Behavior, 89, 263–298.
England: Psychology Press. doi:10.1901/jeab.2008-89-263
Rescorla, R. A., & Wagner, A. R. (1972). A theory of Shahan, T. A., & Podlesnik, C. A. (2008b). Quantitative
Pavlovian conditioning: Variations in the effective- analyses of observing and attending. Behavioural
ness of reinforcement and nonreinforcement. In A. Processes, 78, 145–157. doi:10.1016/j.beproc.
H. Black & W. F. Prokasy (Eds.), Classical condition- 2008.01.012
ing II: Current research and theory (pp. 64–99). New
York, NY: Appleton-Century-Crofts. Shahan, T. A., Podlesnik, C. A., & Jimenez-Gomez, C.
(2006). Matching and conditioned reinforcement
Riley, D. A., & Leith, C. R. (1976). Multidimensional rate. Journal of the Experimental Analysis of Behavior,
psychophysics and selective attention in animals. 85, 167–180. doi:10.1901/jeab.2006.34-05
Psychological Bulletin, 83, 138–160. doi:10.1037/
0033-2909.83.1.138 Shahan, T. A., & Quick, S. (2009, May). Reinforcement
probability affects adjusting sample duration in
Riley, D. A., & Roitblat, H. L. (1978). Selective attention
a divided-attention task. In M. Davison (Chair),
and related cognitive processes in pigeons. In S. H.
Divided attention, divided stimulus control. Symposium
Hulse, H. Fowler, & W. K. Honig (Eds.), Cognitive
conducted at the 35th annual meeting of the
processes in animal behavior (pp. 249–276). Hillsdale,
Association for Behavior Analysis International,
NJ: Erlbaum.
Phoenix, AZ.
Roberts, W. A. (1998). Principles of animal cognition.
Boston, MA: McGraw-Hill. Staddon, J. E. R. (1983). Adaptive behavior and learning.
New York, NY: Cambridge University Press.
Santi, A., Grossi, V., & Gibson, M. (1982). Differences in
matching-to-sample performance with element and Styles, E. A. (1997). The psychology of attention. East
compound sample stimuli in pigeons. Learning and Essex, England: Psychology Press. doi:10.4324/
Motivation, 13, 240–256. doi:10.1016/0023-9690 9780203690697
(82)90023-6 Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004).
Schneider, S. M., & Davison, M. (2005). Demarcated Matching behavior and the representation of value
response sequences and generalized matching. in the parietal cortex. Science, 304, 1782–1787.
Behavioural Processes, 70, 51–61. doi:10.1016/ doi:10.1126/science.1094765
j.beproc.2005.04.005 Sutherland, N. S., & Mackintosh, H. J. (1971).
Shahan, T. A. (2010). Conditioned reinforcement and Mechanisms of animal discrimination learning. New
response strength. Journal of the Experimental York, NY: Academic Press.
Townshend, J., & Duka, T. (2001). Attentional bias asso-
jeab.2010.93-269
ciated with alcohol cues: Differences between heavy
Shahan, T. A., & Burke, K. A. (2004). Ethanol- and occasional social drinkers. Psychopharmacology,
maintained responding of rats is more resistant to 157, 67–74. doi:10.1007/s002130100764
change in a context with added non-drug reinforce-
ment. Behavioural Pharmacology, 15, 279–285. Wagner, A. R., Logan, F. A., Haberlandt, K., & Price, T.
doi:10.1097/01.fbp.0000135706.93950.1a (1968). Stimulus selection in animal discrimina-
tion learning. Journal of Experimental Psychology, 76,
Shahan, T. A., Magee, A., & Dobberstein, A. (2003). 171–180. doi:10.1037/h0025414
The resistance to change of observing. Journal of
the Experimental Analysis of Behavior, 80, 273–293. Williams, B. A. (1994). Conditioned reinforcement:
doi:10.1901/jeab.2003.80-273 Neglected or outmoded explanatory construct?
Psychonomic Bulletin and Review, 1, 457–475.
Shahan, T. A., & Podlesnik, C. A. (2005). Rate of con- doi:10.3758/BF03210950
ditioned reinforcement affects observing rate but
not resistance to change. Journal of the Experimental Wolfe, J. B. (1936). Effectiveness of token-rewards for
Analysis of Behavior, 84, 1–17. doi:10.1901/jeab. chimpanzees. Comparative Psychology Monographs,
2005.83-04 12, 1–72.
Shahan, T. A., & Podlesnik, C. A. (2006). Divided atten- Wyckoff, L. B., Jr. (1952). The role of observing responses
tion performance and the matching law. Learning and in discrimination learning: Part 1. Psychological
Behavior, 34, 255–261. doi:10.3758/BF03192881 Review, 59, 431–442. doi:10.1037/h0053932
409
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Timothy A. Shahan
Yantis, S. (2000). Goal-directed and stimulus-driven Zentall, T. R., & Riley, D. A. (2000). Selective attention
determinants of attentional control. In S. Monsell & in animal discrimination learning. Journal of General
J. Driver (Eds.), Attention and performance: Vol. 18: Psychology, 127, 45–66. doi:10.1080/0022130000
Control of cognitive processes (pp. 73–103). 9598570
Cambridge, MA: MIT Press. Zentall, T. R., Sherburne, L. M., & Zhang, Z. (1997).
Zeaman, D., & House, B. J. (1963). The role of attention in Shared attention in pigeons: Retrieval failure does
retardate discrimination learning. In N. R. Ellis (Ed.), not account for the element superiority effect.
Handbook of mental deficiency: Psychological theory and Learning and Motivation, 28, 248–267. doi:10.1006/
research (pp. 159–223). New York, NY: McGraw-Hill. lmot.1996.0965
410
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 18
Remembering and Forgetting

K. Geoffrey White
Remembering and forgetting raise the question of Remembering in both humans and nonhuman
how an event at one time influences behavior that animals has been studied for more than 100 years.
occurs at another. In physics, action at a distance At about the time that Thorndike (1898) first
used to be a puzzle because it was thought that demonstrated remembering in cats, Ebbinghaus
objects could influence each other only through (1885/1964) reported the results of the first system-
direct contact. In psychology, the question of action atic experimental study of human remembering.
at a temporal distance remains a puzzle. How do Ebbinghaus recognized the difficulty caused by peo-
people remember and why do people forget? ple’s prior learning experiences for a study of
Research on human memory is wide ranging, and remembering. Accordingly, he used nonsense sylla-
naturalistic and laboratory studies of remembering bles, an innovation that has influenced the study of
in nonhuman animals are covered by the field called human memory to the present day. He also varied
comparative cognition (Roberts, 1998). Sometimes the duration of the retention interval from minutes
the different abilities reveal fascinating species- to many days. Since then, an enormous number of
specific characteristics (Wasserman, 1993), and studies have been devoted to the empirical and theo-
sometimes the species similarities owe to procedural retical study of remembering in humans. An accessi-
limits (White, Juhasz, & Wilson, 1973). This chap- ble account was provided by Baddeley (1997) in his
ter is more constrained in scope. In it, I examine the book Human Memory: Theory and Practice. Several
experimental analysis of nonhuman remembering specialist journals cover the area, and many reviews
and forgetting in laboratory procedures in which the in the Annual Review of Psychology have addressed
retention intervals are typically short and in which important issues (e.g., Jonides et al., 2008; Nairne,
parametric variation is a main focus. Can an experi- 2002; Roediger, 2008; Wixted, 2004b). The research
mental analysis contribute a solution to the puzzle has identified many variables and conditions that
of action at a temporal distance? influence remembering, such as the familiarity and
In any experiment in which remembering or for- repetition of material to be remembered, whether
getting is studied, the most important parameter is learning is spaced, and whether interfering events
the temporal distance between original learning and occur between learning and recollection. The effects
the point of remembering, because the time delay, or of some variables, however, are relative to the effects
retention interval, defines the behavior as remem- of others, making it difficult to establish lawful regu-
bering or its converse, forgetting. The process used larities (Roediger, 2008). None of this extensive
to bridge the temporal gap is loosely referred to as research with human subjects is explicitly reviewed
memory, but as I discuss later, this question is a the- here, although it tends to have influenced the direc-
oretical one that has many different interpretations. tion of research with nonhuman animals.
I thank Glenn Brown for his guidance and advice.
DOI: 10.1037/13937-018
411
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
Overview remembering in nonhuman animals, the task

requirements are made explicit through reinforce-
I begin the chapter with a preliminary description
ment contingencies, and a response language is
of the main focus of an experimental analysis of
thus developed (Goldiamond, 1966). For these
remembering. I follow this description with an
reasons, nearly all of the studies I review in this
account of the procedure most often used to study
chapter examine remembering in nonhuman
short-term remembering in nonhuman animals
animals.
(delayed matching to sample) and the advantages of
An experimental analysis of remembering
quantifying the forgetting function, the hallmark of
emphasizes observable behavior and the descrip-
a memory study. In the next section, I summarize
tion of variables of which the behavior is a func-
the empirical research on the effects of variables that
tion. Much of this chapter concerns such variables,
influence the forgetting function. These variables
but as I noted earlier, the most important variable is
are related to the to-be-remembered events: the
the retention interval—the temporal gap between
sample stimuli in delayed matching to sample; the
original learning and later remembering. Without a
retention interval; the choice response, including its
time delay, researchers would be studying perceiv-
reinforcing consequences; and the intertrial interval.
ing and would see no need to refer to memory. An
In a subsequent section, I describe the three main
additional consideration for an experimental analy-
behavioral theories of remembering, all of which
sis of behavior is that the effects of the experimen-
rely on the effects of reinforcers for remembering
tal variables are demonstrated in each individual
and their role in forgetting. In the final section, I
(Sidman, 1960). The effects of a variable are shown
consider a behavioral perspective on memory, in
by altering it across several values. Most of the
which remembering is treated as a discrimination
studies reviewed in this chapter follow these
specific to the retention interval at which remem-
principles.
bering occurs. My main purpose in this chapter is
From a behavioral perspective, remembering can
to document the effects of variables that must be
be placed in the broader context of discrimination
accounted for by successful theories, to demonstrate
and generalization. Remembering typically requires
the empirical and theoretical importance of the con-
a discrimination between concurrently available
sequences for remembering, and to suggest that
response alternatives. A choice is followed by rein-
remembering can be understood in the same terms
forcement depending on the event or stimulus to be
as discrimination.
remembered. In recognition procedures, one or
more of the stimuli to be remembered is presented
again at the time of choice. In recall procedures, the
Experimental Analysis of
to-be-remembered stimulus is not presented again
Remembering
but is associated with a choice alternative. The dis-
From a behavioral perspective, an experimental crimination thus involves both the stimulus or event
analysis of remembering has mainly been advanced to be remembered and the stimuli associated with
using nonhuman animals for two reasons. The first the choice alternatives. It is a conditional discrimi-
is the relative ease of specifying the environmental nation because the reinforced choice response is
variables that influence the individual’s behavior, conditional on the to-be-remembered stimulus.
especially when variables are manipulated across Because the to-be-remembered stimulus exerts its
different values for each individual in the study. influence on the subsequent choice after some delay,
The second is that a fundamental element deter- accurate remembering shows delayed stimulus con-
mining the individual’s behavior in any task requir- trol (Catania, 1992), which implies that the discrim-
ing learning and remembering is the reinforcement ination occurs at the time of remembering, not at
contingency. Very few studies with humans can the time at which the to-be-remembered event was
easily control the rewards, or reinforcers, for presented (Berryman, Cumming, & Nevin, 1963;
appropriate completion of the task. When studying White, 2001).
412
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Delayed Matching Task sample darkened the key and led to a delay interval
of several seconds, varying from trial to trial. At the
Fifty years ago, two innovative procedures were
end of the delay, the two patterns were presented as
reported for studying remembering over short delays
comparison stimuli on the left and right response
(generally 20 seconds or less). One, by Peterson and
keys, with their position alternating randomly across
Peterson (1959), was designed to study human
trials. Correct choices of the comparison pattern
memory. The other, by Blough (1959), was designed
that matched the sample at the beginning of the trial
to study remembering in nonhumans. Although I do
were followed by delivery of grain. A houselight illu-
not consider research on human memory in this
minated the chamber throughout the experimental
chapter, the two tasks are similar. Peterson and
session. Figure 18.1 illustrates a general version of
Peterson asked human participants to repeat three
consonants after a delay lasting several seconds.
a Fifth peck at red or green center key
During the delay, the participant counted down in starts a dark delay interval
threes from a number provided by the experimenter.
Red
The result was very rapid forgetting of the conso-
nants, interpreted by the Petersons as trace decay, a
process specific to short-term memory. As it hap-
pens, the distinction between short-term and long-
term forgetting that followed from the Petersons’
experiment has little substance (Nairne, 2002;
Suprenant & Neath, 2009). Instead, it is best under-
stood simply in terms of the relative lengths of
delays being studied (short vs. long), and the same
principles can apply to remembering over both the b After the delay interval a choice is
short and the long term. The study by Peterson and made between red and green side keys
Peterson has been very influential in stimulating a Red

Green
large body of research on human short-term mem-
ory ( Jonides et al., 2008), and the theoretical ideas
have been transferred to account for remembering
over short delays by nonhuman animals (Kendrick,
Rilling, & Denny, 1986; Roberts, 1998). Wixted
(1989) observed that in the study of nonhuman
short-term memory, “the ratio of theory to data often
seems unacceptably high, and efforts to identify
common empirical principles of memory are rela- Correct choices are reinforced
tively rare” (p. 409). I aim to improve this ratio in

this chapter. Intertrial Interval
The other innovative procedure, delayed

Figure 18.1. Illustration of the
matching-to-sample, was reported by Blough (1959) delayed matching-to-sample pro-
with pigeons as subjects. As with the task devised by cedure involving three response
Peterson and Peterson (1959), the delayed match- keys that can display different
stimuli (e.g., red and green key-
ing task was performed trial by trial, with delays lights). From “Psychophysics of
lasting for several seconds. Each trial began with the Remembering: The Discrimination
presentation of a steady or flickering light on the Hypothesis,” by K. G. White, 2002,
central response key of a three-key experimental Current Directions in Psychological
Science, 11, p. 142. Copyright 2002
chamber. The two light patterns, the samples, by the Association for Psychological
alternated randomly across trials. Responses to the Science. Adapted with permission.
413
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
the delayed matching-to-sample procedure (in f ollowed by a series of test sessions with several
which the delay and intertrial interval are usually different delays (Roberts, 1972). The sudden intro-
dark). In Blough’s study, the percentage of correct duction of nonzero delays can result in chance per-
matching responses decreased with increasing dura- formance at all nonzero delays for some individuals,
tion of the delay, just as in the experiment with or a generalization decrement from the zero delay
humans reported by Peterson and Peterson. (Rayburn-Reeves & Zentall, 2019; Sargisson &
Enhanced accuracy was interestingly correlated White, 2001). Similarly, when one delay is arranged
with whether different stereotyped behaviors, for all trials within a session, accuracy can be low or
such as bobbing up and down, occurred during at chance level (Harnett, McCarthy, & Davison,
the delay. 1984), confounded by large response bias ( Jones &
Blough’s (1959) observations of mediating behav- White, 1992), and the averaged forgetting function
iors suggested a behavioral answer to the question can appear hyperbolic in form. To maintain high
of how the temporal gap was bridged. Mediating accuracy at short delays and minimize the develop-
behaviors, however, may not always be obvious. Ber- ment of response bias at long delays, a successful
ryman et al. (1963) charted the development of strategy is to gradually lengthen the delays as train-
delayed matching to sample but were unable to ing progresses and to retain a zero or near-zero delay
identify mediating behaviors. Whereas Blough grad- in all sets of delays ( Jones & White, 1994; Sargisson
ually increased the duration of the delays, Berryman & White, 2001). The absence of bias at long delays
et al. ensured that exposure to each of the different means that the reinforcer proportion remains at its
delays was equal throughout training. Berryman arranged value, typically 0.5, and the discrimination
et al. suggested that the mediating behaviors in is not influenced by fluctuations in reinforcer pro-
Blough’s study were more likely to have developed portions. Attempts to control the reinforcer propor-
as a result of gradually lengthening the delays. Apart tion (McCarthy & Davison, 1991) can result in large
from a few studies in which responding on a lit cen- reductions in the levels of obtained reinforcers at
ter key during the delay differed after two sample long delays and large bias for other dimensions of
stimuli (e.g., Jones & White, 1994; Wasserman, the choice between comparison stimuli (Alsop &
Grosch, & Nevin, 1982) or behavioral observations Jones, 2008; Brown & White, 2009a; Jones &
during the delay were systematically recorded White, 1992; White & Wixted, 1999). Inclusion of
(Urcuioli & DeMarse, 1997), there is a dearth of a very short delay at all stages of training minimizes
studies of mediating behavior during the delay. response bias at medium and long delays (White,
Human short-term memory studies have shown 1985). The procedural advantages of varying delay
renewed interest in the process of rehearsal and its within experimental sessions yields an interesting
prevention during the delay (Berman, Jonides, & major empirical benefit—the mapping of the forget-
Lewis, 2009). The study of mediating behaviors in ting function for individual subjects.
the retention interval of nonhuman forgetting proce-
dures, and their relation to accurate remembering
Forgetting Over Delays
under a variety of conditions already shown to influ-
ence accuracy, could be a productive avenue for Blough’s (1959) study can be seen as the beginning
future research. of an experimental analysis of remembering.
Most studies of delayed matching to sample Delayed matching to sample is the most frequently
begin by training the animal to respond to the used procedure in the study of nonhuman short-
choice stimuli as a first step, with alternating term remembering, and it has been studied among a
responses producing reinforcers. The next step is wide range of species (White, Ruske, & Colombo,
to introduce the sample stimuli, which precede the 1996). Examples include humans (Adamson, Foster, &
choice stimuli without delay. After a few days of McEwan, 2000; Lane, Cherek, & Tcheremissine,
such training, a very short delay is introduced. In 2005), monkeys (D’Amato, 1973), dolphins (Her-
some early studies, training with no delay was man, 1975), mice (Goto, Kurashima, & Watanabe,
414
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
2010), and rats (Dunnett & Martel, 1990; Harper, Quantifying Forgetting
2000; Ruske & White, 1999). The trial-by-trial pro-
The beauty of the forgetting function is that it mea-
cedure allows the direct translation of the main pro-
sures performance across a wide range of levels of
cedural elements (sample, delay, choice) into the
accuracy, from high at very short retention intervals
terms of the cognitive psychology of human remem-
to low at very long intervals. Percentage of correct
bering (encoding, storage, retrieval). It also allows
choice in delayed matching to sample is the standard
the study of the effects on matching accuracy in
and most basic measure of performance at each
pigeons of variables that corresponded to main
retention interval, but increasingly a measure of dis-
effects in human short-term remembering such as
criminability is used. The problem with percentage
repetition, rehearsal, proactive interference, retroac-
correct is that it is bounded at 1.0 and can suffer
tive interference, and spaced practice (Roberts,
from ceiling effects. By transforming proportion cor-
1972; Roberts & Grant, 1976). Most important, the
rect (p) to logit p, using logit p = log10 [p/(1 − p)],
delayed matching-to-sample procedure allows
the problem of ceiling effects can be avoided. Logit p
within-subject variation of the fundamental variable
is a ratio-based measure that varies on an equal-
that defines the procedure as a memory procedure—
interval scale, as do measures of discriminability. As
the delay or retention interval. Because the func-
a result, the slopes of different forgetting functions
tion relating accuracy to delay duration typically
can be compared without encountering the problem
decreases with increasing time, the function is
that slope differences can be generated by non–
referred to as a forgetting function (White, 1985,
equal-interval measurement scales (Loftus, 1985;
2001). In the absence of delay interval variation, a
Wixted, 1990). Technically, logit p can be influenced
single data point at a given delay confounds poten-
by response bias, whereas the discriminability mea-
tial differences in the intercept of a forgetting func-
sures d′, log α, and log d estimate discriminability
tion with its slope.
separately from response bias. Macmillan and Creel-
I consider the effects on forgetting functions of
man (1991) provided a comprehensive account of
the sample stimuli, delay interval conditions, choice,
signal detection theory’s d9 and log α from choice
and intertrial interval in delayed matching to sample
theory. Log d (Davison & Tustin, 1978; Nevin, 1981)
in the sections that follow. Quantifying the forget-
is the same as log α except that it uses log to base 10.
ting function in terms of the intercept and slope of
Both logit p and log d express discriminability as
fitted functions reveals that some variables influence
the log of the ratio of correct responses to errors and
the intercept and others the slope. Additionally,
are equal when there is no response bias. The log d
quantification is a hallmark of an experimental anal-
measure is easy to calculate: log d = 0.5log10 [(cor-
ysis of behavior, and the ability to fit functions to
rect after S1 × correct after S2) / (errors after S1 ×
data provides another level of analysis in the search
errors after S2)], S1 and S2 are the two stimuli.
for order (Mazur, 2006). An excellent example is the
When there are no errors, log d cannot be deter-
comparison of auditory memory in humans and
mined. Brown and White (2005b) used a computa-
starlings in a delayed matching task in which sam-
tionally intensive analysis to show that the optimal
ples were pure tones and starling song motifs
correction in such cases is achieved by adding 0.25
(Zokoll, Naue, Herrmann, & Langemann, 2008).
to the response totals in each of the four cells of the
The forgetting functions were well described by
response matrix (correct and error responses after
exponential functions that did not differ in intercept
S1 and S2).
for the tones versus motifs. Repetition of the sam-
The form of the mathematical function that best
ples increased intercepts for the starlings but not
fits the data from delayed matching-to-sample stud-
for the humans, and rate of forgetting was greater
ies (Rubin & Wenzel, 1996) and whether the fits
for the starlings. Thus, the higher order description
depend on the measure of accuracy (Wickens, 1998)
in terms of intercepts and slopes of forgetting func-
have been extensively discussed. In practice, func-
tions facilitated an illuminating cross-species
tions that differentiate different conditions measured
comparison.
415
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
in terms of discriminability equally differentiate of varying the different components of the delayed
them when measured in terms of percentage correct, matching task: the sample stimuli, the retention
as long as the different levels of chance performance intervals, the comparison stimuli and choice
are recognized (zero for discriminability measures response, and the intertrial interval. The results of
and 50% for percentage correct; White, 2001). fitting functions to the data from the wide range of
A function that appeals from a behavioral per- studies I summarize suggest some impressive regu-
spective is the simple exponential decay function, larities: Variation in attributes of the sample stimu-
y = a · exp(−b · t), because it is “memoryless.” The lus influence the intercept of the forgetting function,
exponential function is the only mathematical func- whereas conditions during the retention interval and
tion that has a constant rate of decrement (b), with at the time of remembering influence the slope of
the property that the reduction in performance the forgetting function.
between two times depends only on that temporal
distance and not on the level of performance at ear-
Variations in Sample Stimuli
lier times (White, 2001). The exponential function
is memoryless in that performance does not depend The initial impetus for studying the effects of varia-
on changes in memory that might result from organ- tion in sample-stimulus parameters in studies of
ismic variables. A practical problem with the simple nonhuman delayed matching tasks was provided by
exponential function, however, is that it underesti- the analogy with processes of human short-term
mates accuracy at longer delays. A better fitting memory—distinctiveness, complexity, repetition,
exponential function scales time to the square root and rehearsal of the to-be-remembered events. The
(White & Harper, 1996) and retains the memoryless results, however, established many basic findings
properties of the simple exponential function, that that can now be described in terms of their effects
is, y = a · exp(b · √t) (White, 2001). on forgetting functions. The main feature that these
Power functions have also proven useful in quan- different variables have in common is that they
tifying forgetting functions. Wixted (2004a, 2004b), affect the overall difficulty of discrimination, as
Wixted and Carpenter (2007), and Wixted and shown by changes in the intercept parameter of the
Ebbesen (1991) have argued persuasively in favor of forgetting function. In other words, the different
the power function, y = a · (t + 1)b, because of its aspects of the sample stimuli influence the discrimi-
consistency with the notion of consolidation and nation independently of time.
Jost’s law (Woodworth & Schlosberg, 1954, pp. 730–
731). The power function is difficult to discriminate Sample Stimulus Disparity
from the exponential function with time scaled as Roberts (1972) showed that percentage of correct
√t in terms of their accurate description of forget- matching was overall higher for an easier color dis-
ting functions (White, 2001). Indeed, apart from crimination than for a harder color discrimination,
theoretical reasons, which particular function is used although comparison stimuli also differed owing to
for descriptive purposes does not matter greatly as the identical nature of samples and comparisons.
long as it provides a reasonable fit to the data. White (1985) described forgetting functions for two
The advantages of fitting functions to data are levels of wavelength disparity between the samples,
that the entire forgetting function can be quantified with disparity between comparison stimuli held
in terms of the parameters of the fitted function, constant. Figure 18.2 shows discriminability, log d,
typically intercept and slope, and that comparisons averaged over five pigeons in the experiment, recal-
can be made between different experimental condi- culated from data in White’s Table 1, and with the
tions in terms of their effects on either or both of the exponential function in the square root of time fitted
two parameters (White, 1985; Wixted, 1990). to the data. The fits suggest that variations in the
Where functions are fitted to data in this chapter, I disparity of the sample stimuli produce a change in
used the exponential function in the square root of the intercepts of the fitted function without affecting
time. In the following sections, I describe the results their slope.
416
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
2.5 most studies—up to 60 seconds. His data, trans-

538 vs. 576 nm: a= 1.93 b= .49 formed to logit p values, are plotted in Figure 18.3.
2.0
Discriminability Log d
501 vs. 606 nm: a= 2.51 b= .45 The intercepts of fitted exponential functions in the
1.5 square root of time increased systematically with
increasing sample duration, without an obvious
1.0
change in slope.
0.5 Foster, Temple, Mackenzie, DeMello, and Poling
(1995) varied independently both the FR require-
0.0 ment (0, 1, 3, 7, 10) and sample duration (2 sec-
0 4 8 12 16 20
Delay Interval (s) onds, 5 seconds, 10 seconds) in a delayed matching
procedure with hens as subjects. Although Foster
Figure 18.2. Discriminability as a function et al. arranged just one delay interval in the delayed
of delay for two conditions of wavelength dis-
parity between sample stimuli. Smooth curves matching task, their data clearly demonstrated that
are nonlinear least-squares fits of y = a · exp(b · increases in both FR and sample duration had inde-
√t). Data from White (1985, Table 1). pendent effects in increasing discriminability.
Fixed-Ratio Requirement Serial Compound Sample Stimuli

Repeating a to-be-remembered item can increase In the delayed paired-comparison task arranged by
accuracy. In the delayed matching task, repetition Shimp and Moffitt (1977), two stimuli were pre-
can be achieved by repeated presentations of the sented in succession and with a delay between them.
sample (Kangas, Vaidya, & Branch, 2010; Roberts & At the same time as the second stimulus was pre-
Grant, 1976; Zokoll et al., 2008), extending its dura- sented, a choice was made available—peck left if the
tion, or requiring repeated observing responses to stimuli were the same, or peck right if they differed.
the sample. Roberts (1972) varied the fixed ratio The procedure is a choice version of successive
(FR) response requirement for pigeons’ pecks on the matching to sample (Nelson & Wasserman, 1978).
sample key in a delayed matching-to-sample task in In White’s (1974) version of the task, the choice
which retention intervals were varied over 0, 1, 3, response follows the second stimulus. In all three
and 6 seconds. The FR values in different condi- procedures, lengthening the time between succes-
tions were 1, 5, and 15 (i.e., required 1, 5, and 15 sive presentation of the two stimuli decreases accu-
responses, respectively). This manipulation, along racy. The stimulus associated with a correct response
with variation in the exposure duration of the sam- is actually a compound or abstract stimulus—same
ple, was seen as affecting repetition. White (1985,
Figure 13) fitted simple exponential functions to the 2.5
14-s Sample
Discriminability Logit p
logit p transform of Roberts’ data and found that the 2.0 8-s Sample
4-s Sample
intercept of the fitted functions increased systemati- 1.5 1-s Sample
cally and the slope decreased with increasing FR 1.0
value. Both White (1985) and White and Wixted
0.5
(1999) compared the effects of FR 1 and FR 5
0.0
requirements for sample-key responding across a 0 20 40 60
range of delay intervals and reported higher inter- Delay Interval (s)
cepts for fitted exponential functions for FR 5 than
Figure 18.3. Discriminability
for FR 1 without any systematic change in slope. as a function of delay and sample
stimulus duration. Smooth curves
Sample Duration are nonlinear least-squares fits of
Grant (1976) varied the exposure duration of sam- y = a · exp(b · √t). Data from Grant
(1976). DRO = differential rein-
ple stimuli over four values ranging from 1 second forcement of other behavior; FR =
to 14 seconds and also used delays longer than in fixed ratio.
417
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
versus different. By increasing the temporal separa- with the lower intercept for elements than for serial
tion between elements of the compound, the dis- compounds in the same–different discriminations.
crimination is made more difficult. White and The forgetting function for basic categories also
McKenzie (1982) held constant the time between tended to have a greater slope than that for the
successive stimuli and varied the retention, or delay, superordinate categories, although confirmation of
interval between the second stimulus and the this trend relies on analysis of functions fitted to
choice. They also compared the forgetting functions individual data.
for the same or different compound with forgetting
functions for the element stimuli (red and green) Asymmetrical Samples
making up the compound. These functions differed Technically, delayed matching to sample is a two-
only in intercept, not in slope. In particular, the alternative forced-choice procedure. When the sam-
intercept for the compound stimuli was lower. That ples are symmetrical, there should be no reason to
is, the discrimination of same versus different was prefer one sample over the other. A variant of the
more difficult than the discrimination of the ele- standard procedure involves asymmetrical samples,
ments making up the compound, and increasing the in which the choice responses are associated with
retention interval resulted in a similar decrement in whether a sample was present or absent. Prefer-
discriminability in both cases. ence to report the absence of the sample increases
A question of interest is whether the function with increasing duration of the retention interval
relating discriminability to retention interval has the (Dougherty & Wixted, 1996; Wixted, 1993). Simi-
same slope as the function relating discriminability larly, when samples are two different durations of a
to the time between successive stimuli. Data relevant stimulus, the tendency to report the shorter of the
to this question were reported by Urcuioli and two durations increases as the retention interval
DeMarse (1997). When pigeons chose left versus lengthens—the “choose-short” effect (Spetch &
right response keys according to whether two suc- Wilkie, 1982, 1983). Gaitan and Wixted (2000)
cessively presented stimuli were the same or differ- have shown that short durations seem to function in
ent, discriminability with increasing delay between the same way as absent samples. The effect general-
successive stimuli decreased at a faster rate than did izes to number, the “choose-few” effect (Fetterman &
discriminability with increasing time between the MacEwan, 1989), and to the effects of prior training
second stimulus of a pair and the choice. This result with one of the samples (Grant, 2006). Hypotheses
suggests that the two intervals have different func- to account for the effect range from subjective short-
tions. One relates to the pairing of the elements to ening (Spetch & Wilkie, 1983) to ambiguity
form a discriminable compound (Wixted, 1989), between delay and intertrial intervals (Sherburne,
and the other relates to the delayed control by the Zentall, & Kaiser, 1998).
compound over the subsequent choice response. Ward and Odum (2007) convincingly demon-
strated that the choose-short effect can be accounted
Categorized Samples for in terms of overall control by the sample stim-
Extensive research has documented the ability of uli and not by mechanisms such as subjective short-
nonhuman animals to discriminate categories of nat- ening. In a delayed matching task with just one
ural and artificial objects at different levels (e.g., 0-second delay, pigeons chose one comparison stim-
Sands, Lincoln, & Wright, 1982; Vonk & MacDon- ulus after four generally short delays and another
ald, 2004; Wasserman, Kiedinger, & Bhatt, 1988). comparison after four longer delays. The psycho-
Lazareva and Wasserman (2009) examined the metric functions relating choice accuracy to sample
choice responses of pigeons to samples categorized duration were asymmetrical, like those of Fetterman
at basic levels (cars, chairs, flowers, people) or (1995). Various disruptors affected accuracy on
superordinate levels (natural vs. artificial) after three long-duration trials (a choose-short effect). Ward
delays (0, 1, and 4 seconds). Discriminability was and Odum used Blough’s (1996) model to analyze
lower for the basic categories at all delays, consistent their data, owing to its ability to separate stimulus
418
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
control factors from other influences. Their conclu- replotted in Figure 18.4, with percentage correct
sion is consistent with the emphasis emerging from transformed to logit p. The main difference in the
this chapter, that sample-stimulus variation affects exponential functions fitted to their transformed
overall stimulus control, as reflected in the intercept data is in the intercepts, not the slopes. That is,
of the forgetting functions. sample-specific responding enhances overall stimu-
Research on the choose-short effect tends to lus control, independent of the delay.
examine functions for the two sample stimuli sepa-
rately. Of interest is comparing full forgetting func-
Variations in Delay Interval
tions that plot discriminability as a function of
Conditions
retention interval (thus combining the effects of
the two samples) for duration samples and color Conditions during the retention or delay interval that
samples or for a range of duration samples that dif- reduce accuracy are described as instances of retroac-
fer in relative duration. The latter comparison was tive interference (Cook, 1980; Grant, 1988). Such
reported by Fetterman (1995). Reanalysis of Fetter- conditions include illuminating a houselight in a nor-
man’s data (White, 2001) showed that for samples mally dark delay, introducing other stimulus events,
that differed in duration but that were otherwise providing food, and reinforcing responses in an alter-
closely separated or more distant, forgetting func- native task. The general effect of such intruding events
tions for the easier discriminations were characteris an increase in the slope of the forgetting function.
ized by higher intercepts. Competition from reinforcers for other tasks can also
result in a reduction in the intercept of the forgetting
Sample-Specific Responding function. That is, competing reinforcers can generate
When the sample response requirement differs, not an overall reduction in discrimination accuracy.
only is there a choose-few effect but overall accuracy
is also higher than in conditions with the same ratio Retroactive Interference
requirement for sample responses (Fetterman & When the houselight is illuminated for the duration
MacEwan, 1989; Zentall & Sherburne, 1994). of a normally dark delay, matching accuracy plum-
Zentall and Sherburne (1994) trained pigeons to mets, as illustrated in Figure 18.5, for logit p trans-
respond (FR 10) or not respond (differential rein- formations of data reported by Roberts and Grant
forcement of other behavior) to color samples in a (1978). The exponential functions fitted to the data
delayed matching-to-sample task. Their results are in Figure 18.5 do not differ in intercept but have
very different slopes, or rates of forgetting. The same
1.0 FR10 vs. DRO on Samples
effect was reported for pigeons by White (1985) and
FR10 on both Samples
Dark Delay
1.0
Houselight in Delay
0.5
0.5
0.0
0 1 2 3 4
Delay Interval (s)
Figure 18.4. Discriminability 0.0

as a function of delay and different 0 4 8 12
sample stimulus response require- Delay Interval (s)
ments. Smooth curves are nonlinear
least-squares fits of y = a · exp(b · Figure 18.5. Discriminability as a
√t). FR10 = fixed-ratio 10 responses; function of delay for dark delays and illu-
DRO = differential reinforcement of minated delays. Smooth curves are nonlin-
other behavior. Data from Zentall and ear least-squares fits of y = a · exp(b · √t).
Sherburne (1994). Data from Roberts and Grant (1978).
419
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
Harper and White (1997), for monkeys by D’Amato EXT
and O’Neill (1971), and for humans with delays 2 VI-30 s
VI-15 s
filled by interfering noise (Zokoll et al., 2008). Rob-
erts and Grant (1978, Experiment 2) varied the
1
duration of houselight interpolated in a 10-second
delay interval and found that accuracy decreased
with increasing houselight duration. Other studies 0
0 4 8 12
that introduced various stimuli in the delay, includ- Delay Interval (s)
ing food presentations ( Jans & Catania, 1980) and
geometric forms (Wilkie, Summers, & Spetch, Figure 18.6. Discriminability as
1981), have reported the same general result. a function of delay and reinforce-
ment for an extraneous task in the
Harper and White (1997) argued that the delay. Smooth curves are nonlinear
increased rate of forgetting observed when the least-squares fits of y = a · exp(b ·
houselight is illuminated is caused by the increasing √t). EXT = extinction; VI = vari-
duration of houselight that is otherwise normally able interval. Data from Brown and
White (2005).
correlated with the increasing duration of the reten-
tion interval. When they included a constant interval 15, respectively), and the rate of forgetting
1.5-second illumination of the houselight at the end increased (0.53, 0.64, and 0.73, respectively). The
of each delay interval, the forgetting function (not increase in the rate of forgetting is consistent with
including the shortest delay) had the same slope as the conclusion that as the retention interval length-
the function for delays that were dark throughout. ens, the extent of interference from the competing
The effects of houselight illumination during the task increases with the delayed matching task.
delay for pigeons are not surprising because once
the light is on, the birds tend to peck at irrelevant
Conditions for the Choice Response
objects and find grain spilled from the hopper when
the reinforcers were delivered. Such behaviors dur- Typically, the comparison stimuli are assumed to be
ing the delay interval, presumably maintained by highly discriminable, although their disparity can
reinforcers extraneous to the remembering task, are influence overall matching accuracy ( Jones, 2003;
likely to compete with any possible mediating White, 1986). Additionally, the reinforcement con-
behavior during the delay. tingencies are also assumed to be unambiguous,
with no reinforcement for errors. Nonetheless, the
Competing Reinforcers reinforcement contingency for correct responses has
An explicit competing alternative in the delay inter- a powerful effect on delayed matching performance.
val was arranged by Brown and White (2005c). The absolute probability, magnitude, and delay of
Using a standard delayed matching task with red reinforcement affect matching accuracy. The sig-
and green sample stimuli and delays ranging over naled magnitude effect is the result of signaling two
four values from 0.2 second to 12 seconds, they different reinforcer magnitudes (or probabilities)
included conditions in which responding on the within sessions, with higher accuracy occurring on
center key (lit white) was reinforced at variable trials in which the larger reinforcer is signaled. The
intervals averaging 15 seconds, 30 seconds, or none relative reinforcer probability for correct choices also
(extinction). The result from their Experiment 2 is influences performance. When different reinforcer
shown in Figure 18.6, with exponential functions in probabilities, magnitudes, or other qualitative
the square root of time fitted to the log d measures. aspects (e.g., food vs. water) follow the different cor-
As reinforcers for center-key responding in the delay rect choices, the resulting enhancement in accuracy
became more frequent, the intercept of the forgetting is called the differential outcomes effect. When dis-
function decreased to a small extent (2.60, 2.54, and rupting events are introduced at different stages dur-
2.33 for extinction, variable interval 30, and variable ing the delayed matching trials, discriminability is
420
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
resistant to change in the same way as the response 1.5 4.5-s Reinforcer
rate of a single operant. Reinforcement of responses 1.5-s Reinforcer
on the previous trial influences choice on the cur- 1.0
rent trial, the local proactive interference effect pre-
viously thought to result from the influence of the 0.5
sample stimulus on the previous trial. These various
influences are all associated with the effects of the 0.0
0 4 8 12
reinforcement of correct choices in the
Delay Interval (s)
remembering task.
Figure 18.7. Discriminability
Absolute Frequency and Magnitude of as a function of delay for different
Reinforcement reinforcer magnitudes (duration of
access to food signaled within ses-
Instead of reinforcing each correct response, correct sions). Smooth curves are nonlinear
responses can be reinforced with a certain probabil- least-squares fits of y = a · exp(b · √t).
ity. When the probability of reinforcement for each From “Delayed Matching-to-
Sample Performance: Effects of
correct choice is reduced from 1.0 to 0.5 or 0.2 Relative Reinforcer Frequency and
across blocks of sessions, overall discriminability of Signaled Versus Unsignaled
decreases (Brown & White, 2009b; White & Reinforcer Magnitudes, by D.
Wixted, 1999). The same result applies when the McCarthy and P. Voss, 1995, Journal
of the Experimental Analysis of
magnitude of the reinforcers is reduced (Brown & Behavior, 63, p. 39. Copyright 1995
White, 2009b). When the magnitude of reinforcers by the Society for the Experimental
for correct choices is varied within an experimental Analysis of Behavior, Inc. Adapted
session and the different magnitudes are signaled by with permission.
different cues on each trial, accuracy or discrim-
inability is greater on trials in which the larger rein- discriminability with increasing reinforcer delay
forcer is signaled. This effect, the signaled magnitude measured from the time of choice (McCarthy &
effect, is illustrated in Figure 18.7 from a reanalysis Davison, 1986, 1991), reinforcer delays were varied
of data reported by McCarthy and Voss (1995). The only for a 0-second retention interval (see Sargisson &
effect is well documented (Brown & White, 2005a; White, 2003, Figure 1). Sargisson and White (2003)
Jones, White, & Alsop, 1995; Nevin & Grosch, varied retention interval within sessions and rein-
1990) and reflects a difference in the intercepts of forcer delays across conditions and observed a sub-
the forgetting functions but not their slopes, as is stantial and systematic reduction in the intercepts
clear in Figure 18.7. An analogous effect also occurs of the forgetting functions with increasing rein-
for signaled probabilities of reinforcement (Brown & forcer delay. Their analysis (Sargisson & White,
White, 2005a). 2003, Figure 4) demonstrated that varying the
retention interval with the delay between sample
Delay of Reinforcement and reinforcer held constant did not result in a
Interest in the effects of delaying the delivery of the constant level of discriminability. That is, the
reinforcer for correct matching responses was reduction in discriminability with increasing
sparked by the possibility that the forgetting func- retention interval duration is not caused by an
tion confounds the delay of the choice with a delay increase in the temporal distance between the
of the reinforcer for a correct choice, with both sample and reinforcer.
delays measured from the sample. That is, the for-
getting function reflects the influence of the rein- Differential Outcomes
forcer delay (Weavers, Foster, & Temple, 1998), The signaled magnitude effect and the signaled
a possibility that has been shown to be incorrect. probability effect both involve the same magnitude
In studies reporting a systematic reduction in or probability of reinforcement for correct choices
421
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
on a given trial type, but with two different trial 2

Different Outcomes
types signaled by different cues within the experi- Same Outcomes
mental session. An alternative arrangement involves
different outcomes for the two correct choices 1
within a session. For example, correct choices of red
result in one outcome, and correct choices of green
result in another. The two outcomes may differ in 0
0 2 4 6 8
quality (food vs. no food, food vs. water) or quantity
Delay Interval (s)
(magnitude or probability). Compared with separate
sessions in which outcomes of correct responses are Figure 18.8. Discriminability
the same, overall accuracy is higher when outcomes as a function of delay and trials
are different. This is known as the differential out- with different and same outcomes,
replotted from data reported by
comes effect, and it has been extensively investigated Jones and White (1994). Smooth
in a range of species, including rats (Savage & Par- curves are nonlinear least-squares
sons, 1997; Trapold, 1970), dogs (Overmier, Bull, & fits of y = a · exp(b · √t).
Trapold, 1971), horses (Miyashita, Nakajima, &
Imada, 2000), pigeons (Nevin, Ward, Jimenez- types of trials. By Session 30, a differential outcomes
Gomez, Odum, & Shahan, 2009), and humans effect emerged, and by Sessions 60 through 80, it
(Estévez, Overmier, & Fuentes, 2003; Legge & was strongly established. Figure 18.9 shows the
Spetch, 2009). result for Bird C5 in Jones and White’s study, plotted
The differential outcomes effect has influenced as percentage correct over six successive blocks of
the direction of theories of discrimination and their 10 sessions and a final block of 20 sessions. The
account of the role of the stimulus–reinforcer rela- main change in the course of the development of the
tion (Urcuioli, 2005). Jones and White (1994) differential outcomes effect was a progressive reduc-
reported a within-session differential outcomes tion in the rate of forgetting on different-outcomes
effect in pigeons, in a procedure in which trials trials (Figure 18.9, filled circles).
with different outcomes and trials with same out-
comes were differentially signaled. Using the within- Resistance to Change
sessions procedure, Jones et al. (1995) showed that The resistance to change of an operant response to
the differential outcomes effect was very different extinction or other disruptors depends directly on
from the signaled magnitude effect. Whereas the rate of reinforcement and not on response rate
signaled magnitude effect is manifest as a difference (Nevin & Grace, 2000). Odum, Shahan, and Nevin
in the intercepts of the forgetting functions but not (2005) applied resistance to change analysis to accu-
their slopes, the differential outcomes effect is best racy in delayed matching to sample by using a novel
described as a difference in slopes, whereby the rate multiple schedule (or successive discrimination)
of forgetting for trials with different outcomes is procedure (also see Nevin, Milo, Odum, & Shahan,
less than that for trials with same outcomes. Figure 2003). In the presence of two colors (red and green)
18.8 illustrates the difference in rates of forgetting presented in succession on a center key, responses
for different- and same-outcome trials, for data produced delayed matching trials at variable inter-
taken from the within-sessions procedure of Jones vals. The delayed matching trials, with retention
and White (1994). intervals that varied in duration, used blue and yel-
Jones and White (1994) also reported a study in low sample and comparison stimuli. Reinforcers for
which pigeons acquired the discrimination without correct matching responses in red and green compo-
any prior experience and with training that included nents were obtained with different probabilities.
four delays ranging from 0.01 second to 8 seconds Both the responses to red and green stimuli and
from the outset of training. For the first 10 sessions, accuracy in the delayed matching trials were resis-
performance was at chance at all delays and on both tant to various disruptors (prefeeding, food in the
422
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Days 1-10 11-20 21-30 31-40 41-50 51-60 61-80
Proportion Correct
Delay Interval (s)
Figure 18.9. The development of the differential outcomes effect over 80 sessions with one
pigeon from the study reported by Jones and White (1994). Proportion correct on different out-
comes trials is shown as filled circles and on same outcomes trials as unfilled circles. From “An
Investigation of the Differential-Outcomes Effect Within Sessions” by B. M. Jones and K. G.
White, 1994, Journal of the Experimental Analysis of Behavior, 61, p. 399. Copyright 1994 by the
Society for the Experimental Analysis of Behavior, Inc. Adapted with permission.
intercomponent interval, extinction) in the ifferent-outcome trials as a ratio of total reinforcers

d
same way. Thus, accuracy in delayed matching on same-outcome trials.
to sample depends on reinforcement rate in the
same way as does the rate of a single operant Local Proactive Interference
response. Brown and White (2009b) also reached Choice accuracy in delayed matching to sample is
this conclusion by using different measures of the lower when the sample on the current trial differs
strength of delayed matching performance when from the sample on the preceding trial than when
reinforcement probability and magnitude were var- samples are the same across consecutive trials
ied, as did Nevin, Shahan, and Odum (2008), (Grant, 1975, 2000; Hogan, Edwards, & Zentall,
who demonstrated behavioral contrast for both 1981; Roberts, 1980). This intertrial agreement
response rate and discriminability in delayed effect is a form of proactive interference because per-
matching. formance on the current trial is influenced by events
Nevin et al. (2009) used the multiple-schedule on the previous trial (A. A. Wright, Urcuioli, &
procedure to study the resistance to change of the Sands, 1986). I include it here as a condition of the
differential outcomes effect (see preceding section). choice response because reinforcers for choice
Responses in successive red and green components responses on the prior trial influence the choice on
of the multiple schedule led to delayed matching tri- the current trial. Edhouse and White (1988) termed
als with different or same outcomes, respectively. it local proactive interference to distinguish it from
The different outcomes were two probabilities of general proactive interference, in which accuracy is
reinforcers for correct yellow and blue choices. On lower with shorter intertrial intervals (see the sec-
same-outcome trials, reinforcer probabilities were tion Intertrial Interval Conditions later in this chap-
the same for correct yellow and blue choices. Across ter). Local proactive interference is manifest as a
three experiments, Nevin et al. observed a consis- steeper rate of forgetting (slope) on trials in which
tent differential outcomes effect. Resistance to dis- consecutive samples differ compared with when
ruption in the same-outcomes component was they are the same. It is nicely illustrated by the
greater than in the different-outcomes component results reported by Williams, Johnston, and Saun-
only when total reinforcement rate in the same- ders (2006), replotted in Figure 18.10. Williams
outcomes component was greater than that in the et al. studied adults with mental retardation in a
different-outcomes component. In other words, delayed matching-to-sample task with either two
resistance to change of delayed matching accuracy samples in each session or unique samples through-
was not affected by whether accuracy was enhanced out each session (cf. A. A. Wright, 2007). The expo-
by differential outcomes. Nevin et al. also showed a nential functions in the square root of time fitted to
positive relation between the magnitude of the dif- the data replotted in Figure 18.10 differ in slope but
ferential outcomes effect and total reinforcers on not intercept and account for 96% of the variance.
423
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
Same Consecutive Trials That is, local proactive interference results from
Different Consecutive Trials reinforcers for correct choices on the previous trial
Discriminability Logit p 2
influencing choices on the current trial.
Relative Reinforcer Probability

1 According to the generalized matching law (Baum,
1974), the log ratio of responses on two choice alter-
0 natives is a linear function of the log ratio of rein-
0 4 8 12 16 forcers for the choice responses. The slope of the
Delay Interval (s) function estimates the sensitivity of the response
Figure 18.10. Discriminability as ratio to changes in the reinforcer ratios. The inter-
a function of delay for consecutive cept provides a measure of bias to one or the other
trials with same or different samples. choice alternative. By varying the probability of
Smooth curves are nonlinear least- reinforcement for correct matching separately for
squares fits of y = a · exp(b · √t).
From “Intertrial Sources of Stimulus choices following each of the two sample stimuli, a
Control and Delayed Matching-to- pair of matching law functions can be plotted for
Sample Performance in Humans,” choices at each retention interval. Consistent with
by D. C. Williams, M. D. Johnston,
the intuitively plausible view that at long retention
and K. J. Saunders, 2006, Journal
of the Experimental Analysis of intervals, the samples are less effective and choice is
Behavior, 86, p. 256. Copyright 2006 predominantly governed by the reinforcers, Hartl
by the Society for the Experimental and Fantino (1996), Jones and White (1992),
Analysis of Behavior, Inc. Adapted
with permission. White and Wixted (1999), and Sargisson and White
(2007a) showed that the matching law functions
were steeper at long retention intervals than at short
Earlier theories of short-term memory in nonhu- intervals. That is, across retention interval dura-
man animals suggested that proactive interference tions, there was an inverse relation between discrim-
results from competition between conflicting traces inability and sensitivity to reinforcement, consistent
of sample stimuli established on successive trials with the more general relation in conditional dis-
(Grant, 1975; Roberts & Grant, 1976) or from fail- criminations (Nevin, Cate, & Alsop, 1993; White,
ure to discriminate the most recently seen sample 1986).
(D’Amato, 1973). When accuracy levels are high, Other studies reporting a direct relation between
however, the sample on the previous trial is con- discriminability and sensitivity to reinforcement
founded with the choice. Therefore, what may (McCarthy & Davison, 1991; McCarthy & Voss,
appear to be an influence of the sample on the prior 1995) used a specific procedure to control relative
trial is actually an effect of the prior choice, as dem- reinforcer probabilities. The controlled reinforce-
onstrated by Roberts (1980) and Edhouse and ment procedure is designed to maintain equivalence
White (1988). In these studies, accuracy on the cur- between obtained and arranged reinforcer ratios.
rent trial was higher when samples on consecutive When a reinforcer for a correct choice becomes
trials were the same than when they differed, but available, it is held until that correct choice occurs,
only when the choice on the previous trial was cor- whereas correct choices of the alternate comparison
rect (and thus reinforced). White, Parkinson, go unreinforced. This procedure, however, reduces
Brown, and Wixted (2004) arranged a reinforcer the number of reinforcers obtained at long delays for
probability of .75, thus allowing for correct choices which there is low discriminability and many errors
that went unreinforced on the previous trial. Accu- and generates a left–right bias that constrains the
racy was lower on consecutive trials with different sensitivity of the choice between comparison stimuli
samples than on trials with same samples, but only (e.g., red vs. green) to variation in the relative prob-
when the previous correct choice was reinforced. ability of reinforcers for choices ( Jones & White,
424
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
1992). When the controlled reinforcement proce- interval duration and intertrial agreement and
dure is used, low or near-zero sensitivity to rein- argued that the two effects were independent.
forcement at long retention intervals results from Whereas the intertrial agreement effect, or local pro-
a bias to choose one key (e.g., left) when (color- active interference, is manifest as a difference in the
correlated) comparison stimuli alternate, thus slope of forgetting functions, the intertrial spacing
generating indifference between comparisons inde- effect, or general proactive interference, influences
pendently of the reinforcer ratio arranged for only the intercept of forgetting functions. This con-
choices. In the extreme, when discriminability in clusion was confirmed by White’s (1985) fitting of
delayed matching is zero, choice between compari- simple exponential functions to data from the study
sons is expected to follow the usual matching law by Roberts and Kraemer (1982). For intertrial inter-
pattern, as it does when reinforcer probabilities are vals of 4, 8, 16, and 32 seconds, intercepts increased
independent for the two choices. systematically, whereas slopes did not change.
Sargisson and White (2007a) varied both relative When a normally dark intertrial interval is illu-
reinforcer probability and reinforcer delay in delayed minated, the trial-spacing effect is eliminated (Santi,
matching to sample. An increase in the delay of rein- 1984), but the intertrial agreement effect persists
forcers from the choice reduces discriminability (see (Edhouse & White, 1988, Experiment 2). Grant
the section Delay of Reinforcement earlier in this (2000) demonstrated persistence of the intertrial
chapter). Sargisson and White asked whether this agreement effect over intertrial intervals of as long as
reduction was a result of weakened contingency dis- 60 seconds. His claim that the effect was underesti-
criminability (knowing “what reinforcer goes with mated at short intertrial intervals is difficult to eval-
what response”; Davison & Nevin, 1999, p. 445) or uate, however, owing to the mixing of different
impaired conditional discrimination owing to weak- intertrial intervals within sessions, a procedure that
ened association between sample and choice (White, tends to result in the averaging of intervals (Roberts
2002). They observed increasing sensitivity to the & Kraemer, 1982). What is needed is a study in
biasing effects of reinforcement as both retention which the intertrial agreement effect is examined
interval and reinforcer delay increased, consistent over very long intertrial intervals.
with the general principle that factors that weaken
the discrimination by weakening the association
Applications: Drug Effects
between sample and choice will also increase the
biasing effect of reinforcers on choice. A main area in which the quantitative analysis of
forgetting has been applied concerns the effects of
various drugs on behavior (see also Chapter 23, this
Intertrial Interval Conditions
volume). Much of the work has been published in
In studies of human memory, accurate performance neuroscience journals. By fitting a mathematical
is facilitated by spaced learning (Baddeley, 1997). function such as the exponential function to accu-
Similarly, longer intervals between trials in non racy or discriminability measures, the forgetting
human delayed matching to sample result in higher function can be summarized in terms of its inter-
matching accuracy (Edhouse & White, 1988; Krae- cept (initial discriminability) and slope (rate of
mer & Roberts, 1984; Nelson & Wasserman, 1978; forgetting). Many drugs influence the neurotrans
Roberts, 1980; Roberts & Kraemer, 1982; White, mitter mechanisms presumed to be associated with
1985). This effect was thought to be the result of remembering. A good example is the cholinergic
decreasing interference from events on the previous antagonist, scopolamine, which reduces initial dis-
trial, either through diminishing influence of com- criminability without affecting rate of forgetting in
peting traces (Grant, 1975; Roberts & Grant, 1976) many drug studies of the cholinergic hypothesis for
or enhanced temporal discrimination of the most Alzheimer’s disease (White & Ruske, 2002). The
recently experienced sample (D’Amato, 1973). reduction in initial discriminability caused by
Edhouse and White (1988) varied both intertrial administration of scopolamine can be reversed by
425
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
administration of agonists (Harper, 2000; Ruske, by inclusion of reinforcement as a major determi-

Fisher, & White, 1997). Glucose administration nant of performance. Accordingly, they can account
can also reverse scopolamine-induced deficits as for reinforcer influences on the forgetting function,
well as the reduction in initial discriminability including the signaled magnitude effect, the differ-
that results from reducing the sample-response ential outcomes effect, and the effects of absolute
ratio requirement from five to one (Parkes & reinforcer probability. The three behavioral theories
White, 2000). I briefly summarize in this section were all pub-
A second area of interest concerns recreational lished in the Journal of the Experimental Analysis of
drugs. For example, Harper, Wisnewski, Hunt, and Behavior. All incorporate well-established principles
Schenk (2005) studied the effects of amphetamine, of reinforcer control—specifically delay reduction,
cocaine, and 3,4-methylenedioxymethamphetamine behavioral momentum, and the matching law—and
(ecstasy) on delayed matching performance in rats. all are able to predict forgetting functions, that is, a
In all cases, initial discriminability decreased with reduction in discriminability with increasing reten-
increasing dose, without affecting rate of forgetting. tion interval. The mechanisms proposed by the dif-
Lane et al. (2005) reported the first quantitative ferent models to predict the forgetting functions
analysis of the effects of marijuana on forgetting differ, however, and might ultimately provide the
functions in human delayed matching to sample. It main basis for comparisons between the models.
is noteworthy that this carefully conducted study is These mechanisms are delay reduction (Wixted,
one of the few to have shown an effect of drug 1989), diffusion (White, 2002; White & Wixted,
administration on rate of forgetting but not initial 1999), and disruption of attention to the stimulus as
discriminability, an effect that Lane et al. attributed coded during the retention interval (Nevin, Davison,
to disruption of cannabinoid receptor function in Odum, & Shahan, 2007).
the hippocampus.
A third area of interest concerns drugs used in Wixted (1989)
clinical settings. Here, too, there are instances of Fantino (1977) proposed that the discriminative
change in rate of forgetting as well as in initial dis- strength of a stimulus is given by the extent to
criminability. Examples are the effects of the anti- which its onset reduces the delay to primary rein-
psychotic chlorpromazine (Watson & Blampied, forcement. Wixted (1989) recognized that the sam-
1989) and the barbiturate phenobarbital (Watson & ple stimuli in delayed matching to sample signal the
White, 1994). Increasing doses of the dopamine presentation of the comparison stimuli that are
agonist methylphenidate, widely used to treat atten- intermittently associated with reinforcement. That
tion deficit disorder, reduces initial discriminability is, in relation to the overall delay between one rein-
without affecting rate of forgetting (F. K. Wright & forcement and the next, onset of the sample reduces
White, 2003). the delay by an amount that approximates the delay
interval t. Following the delay-reduction formula-
tion, Wixted represented the discriminative strength
Behavioral Theories of
of the sample as (dr + γ)/(t + α). The term dr is the
Remembering
delay reduction quantity and equals the total time T
Cognitive theories of short-term remembering in between reinforcements minus the average delay d
nonhumans are not considered here. They rely on from onset of the sample to the choice, and dr hap-
mechanisms such as trace decay and rehearsal and pens to equal the intertrial interval. The constants γ
temporal distinctiveness, which remain in vogue in and α allow for differential effectiveness of the inter-
current theorizing about human short-term memory trial interval and delay. A third parameter accounts
( Jonides et al., 2008; Suprenant & Neath, 2009) but for the discriminative strength of all other stimuli.
which have proven less fruitful in studies with non- Wixted calculated the strength of a sample relative
humans. Unlike cognitive theories, behavioral theo- to all other stimuli and predicted the proportion of
ries of remembering and forgetting are characterized correct choices by weighting reinforcer proportions
426
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
for correct matching responses by the discriminative (White et al., 2004) and the asymmetrical effects of
strength of the sample. The model predicts that as retention intervals in signal detection versus recog-
discriminative control by the sample decreases, con- nition procedures (White & Wixted, 2010).
trol by the reinforcement proportion increases (cf. To predict a reduction in discriminability with
Jones & White, 1992). Fits of the model to a wide increasing delay, that is, the forgetting function,
range of data accounted for high proportions of the White and Wixted (1999) assumed that the vari-
variance in the data. In most cases, the major inde- ance of the distributions increased with increasing
pendent variable was delay interval duration, and delay. White (2002) addressed the model’s short-
the model did an excellent job of predicting the for- coming in not specifying the precise relation
getting functions. One prediction of interest was the between variance and delay. He showed that the
linear relation between proportion correct and the mathematical form of the forgetting function could
ratio of the intertrial interval to the delay interval. be predicted by a specific diffusion function
This relation was reported by Roberts and Kraemer describing the increase over time of the variances
(1982) from a comprehensive manipulation of both of the stimulus effect distributions. The resulting
parameters and was predicted by Wixted’s model model retains only two parameters. The distance
with high accuracy. The relation seems to fall out between distribution means predicts the intercept
naturally from the delay reduction approach because of the forgetting function, and the rate of diffusion
the delay reduction quantity (T − d) is in most cases of the variances of the distributions predicts the
equivalent to the intertrial interval. slope of the forgetting function. Empirical evidence
for the diffusion function has not yet been
White and Wixted (1999) reported, however.
In a blend of signal detection theory and matching Another shortcoming of the White and Wixted
law, White and Wixted (1999) proposed a model (1999) model is that because choices are based on
that does not include the decision criterion of signal ratios of reinforcers, the model cannot predict the
detection theory and applies the matching law to increase in discriminability when the absolute prob-
discrete trial-by-trial choices based on reinforcer disability or magnitude of reinforcement is increased.
tributions. They assumed that on each trial, the Brown and White (2009b) addressed this problem
choice between comparison stimuli matches the by including a parameter for extraneous reinforce-
proportion of reinforcers obtained in the past by ment. The model’s overall strength is that it is based
those choices given a particular value of stimulus solely on distributions of reinforcer probabilities as
effect. The stimulus effect dimension varies from well as extraneous reinforcement. The model’s
trial to trial, the probability of which is given by a potential weakness is its inability to predict the for-
pair of normal distributions, one for each sample. getting function without making an additional
The reinforcer distributions that predict the choice assumption about a diffusion process.
responses are derived by multiplying the stimulus
effect distributions by the arranged reinforcer proba- Nevin et al. (2007)
bilities. The model can be implemented in a spread- Behavioral momentum theory (Nevin & Grace,
sheet by using normal distribution functions (also 2000) suggests that response rate relative to a base-
see Wixted & Gaitan, 2002). With only two free line is inversely related to the ratio of the reinforcer
parameters (the distance between the stimulus effect rate rs correlated with the stimulus situation in
distributions and their variance), the model predicts which responding is measured and to the overall
the inverse relation between discriminability and average reinforcer rate ra. The reinforcer ratio is
sensitivity of the ratio of choice responses to varia- raised to an exponent b, which measures resistance
tion in the reinforcer ratio reported by Jones and to change to the reinforcer ratio. Nevin et al. (2007)
White (1992) and Sargisson and White (2007a). The proposed that attending to samples, p(As), and to
model also predicts the proactive interference effects comparison stimuli, p(Ac), is given by the following
of reinforcing the choice on the previous trial equations in which x and z are general background
427
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
disruptors and qt and vt are disruptors specific to the at one delay may be independent of remembering
retention interval t: p(As) = exp[(−x · qt)/(rs /ra)b] the same event at a much longer, or shorter, delay.
and p(Ac) = exp[(−z · vt)/(rc /rs)b]. To study delay-specific discrimination, White
With variation in a parameter for sample– and Cooney (1996) trained pigeons in delayed
stimulus discriminability ds and different levels of matching tasks with 0.1-second and 4-second delays
background disruptors, x and z, the model predicts mixed randomly within sessions. In one set of con-
forgetting functions that differ in intercept and that ditions, choices of red and green comparison stimuli
are generally exponential in form. With ds held con- at the short delay were reinforced with different
stant, and variation in the retention interval disrup- probabilities, creating a strong bias to choose the
tors, q and v, the model predicts forgetting functions comparison associated with the higher reinforcer
that vary in slope. Varying the parameters to reflect probability. Choices at the long delay were nondif-
disruption of attention to different components of ferentially reinforced, and the bias at the short delay
the task allows the successful prediction of the did not generalize to choices at the long delay. In
effects of relative and absolute reinforcer probability. another set of conditions, strong reinforcer biases at
The model has difficulty in providing accurate quan- the long delay did not generalize to the nondifferen-
titative predictions for the signaled magnitude effect tially reinforced choices at the short delay. In other
(McCarthy & Voss, 1995) and the differential out- words, performance at one delay was independent of
comes effect (Nevin et al., 2009), but over a wide factors influencing remembering at another.
range of other data, the model makes impressively
accurate predictions by assuming different levels Temporal Independence
of attention to samples, coded representations of The conclusion that the discrimination made at one
the samples in the retention interval, and the time may be independent of the discrimination
comparisons. made at another, that is, temporal independence,
The Nevin et al. (2007) model has several fea- was supported by the result of another delayed
tures in common with Wixted’s (1989) model. One matching task in which reinforcers at a particular
similarity is in the effects of the sensitivity parame- delay were omitted (White, 2001). The result was a
ters γ and α in Wixted’s model and the effects of x reduction in discriminability at the delay without
and z in Nevin et al.’s model. Another is the parti- reinforcers. This result was not surprising. What was
tioning of the probability of attending versus not surprising, however, was the increase in discrim-
attending. A difference, however, is that Wixted’s inability at longer delays. Compared with functions
model is based on reinforcer proportions and does for which reinforcers were included at all delays, the
not predict the change in discriminability that result demonstrated that performance at one delay
occurs when absolute reinforcer rate is varied was independent of whether discriminability was
(Brown & White, 2009b), whereas the Nevin et al. higher or lower at a preceding delay. Temporal inde-
model is able to satisfactorily predict the reduction pendence was also reported by Nakagawa, Ether-
in discriminability with reduced overall reinforcer idge, Foster, Sumpter, and Temple (2004). In one
probability (Brown & White, 2005a). condition, they reinforced correct choices at an
intermediate delay, and choices at both shorter and
longer delays went unreinforced. The result was a
Remembering As Discrimination
nonmonotonic forgetting function, with highest dis-
White (1985, 2001, 2002) has argued that remem- criminability at the intermediate delay.
bering is a discrimination specific to the retention Discriminations made at one retention interval
interval at which it occurs. In effect, the discrimina- may be independent of discriminations made at
tion involves a compound consisting of the sample another retention interval, just as two discrimina-
and comparison stimuli and also the delay that tions about the spatial aspects of stimuli may be
forms part of the context for remembering. That is, independent. Fetterman (1996) discussed the
remembering is specific to the delay. Remembering advantages of treating remembering in the same
428
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
terms as discriminations between proximal stimuli. control of comparison-stimulus choice by both the
Temporal distance is a dimension of the stimulus sample and the delay duration.
complex that influences behavior along with other
physical aspects of the event to be remembered and Delay-Specific Remembering
the stimulus context. To illustrate, Sargisson and Evidence for delay-specific matching comes from
White (2007b) made the discrimination of delay two studies in which training in delayed matching
intervals an explicit requirement in a delayed match- tasks included two delays and two different cues
ing task in which sample stimuli were a cross and a during the sample and delay interval, one correlated
square and comparison stimuli were red and green. with the short delay and the other with the long
Following the cross, choices of red were reinforced delay (MacDonald & Grant, 1987; Wasserman
at 1-second delays and choices of green were rein- et al., 1982). When the relation between the delays
forced at 4-second delays. Following the square, and the cues was switched in probe tests, accuracy
choices of green were reinforced at 1-second delays at the long delay when it was cued by the short cue
and choices of red were reinforced at 4-second was higher than when it was cued by the long cue.
delays. After extensive training in this procedure, More interesting, in the miscue condition of the
probe tests were conducted at 10 delays between probe tests, accuracy at the long delay was actually
1 second and 4 seconds. The results from a replica- higher than at the short delay. Accuracy does not
tion in a later study (White & Sargisson, 2011) depend on how much time has passed but on the
in which probe tests were included in a maintained combination of the delay duration and stimulus
test, averaged over four pigeons, are shown in conditions at the time of remembering. As Wixted
Figure 18.11. The functions demonstrate conjoint (1989) noted,
This interesting finding suggests that
Proportion of Green Choices
Square the strength of a discriminative stimulus

1.0 Cross
may be delay specific only when one
retention interval is employed. That is, a
generalization gradient of discriminative
0.5
strength may be conditioned around a
particular delay such that it is strongest
0.0
at the baseline delay and weaker at other
1 2 3 4 delays (longer or shorter). (p. 416)
Retention Interval (s)
Wixted’s (1989) suggestion was later confirmed
Figure 18.11. Proportion of by Sargisson and White (2001). They trained inex-
choices of green given square and perienced pigeons in delayed matching to sample
cross samples as a function of delay
interval during maintained testing in with just one delay from the outset of training: 0, 2,
which, given square, choices of green 4, or 6 seconds for different groups. Once a discrimi-
were reinforced at 1-second delay nation criterion had been attained, a single session
and choices of red were reinforced was conducted with reinforced training trials and
at 4-second delay, and given cross,
choices of red were reinforced at unreinforced probe trials with different delays
1-second delay and choices of green between 0 second and 10 seconds, including the
were reinforced at 4-second delay. training delay.
Smooth curves are nonlinear least-
The results are shown in Figure 18.12. The delay-
squares fits of y = a · exp(b · √t).
From “Maintained Generalization interval functions tend to peak at the training delay,
of Delay-Specific Remembering,” reminiscent of generalization gradients along spatial
by K. G. White and R. J. Sargisson, dimensions (Honig & Urcuioli, 1981). They also flat-
2011, Behavioural Processes, 87,
p. 312. Copyright 2011 by Elsevier. ten as the training delay becomes longer, the likely
Reprinted with permission. result of the scalar property of time, where two intervals
429
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
Training delay describes the effect of temporal distance, and a gener-

T=0s alization component, which follows Shepard’s (1987)
1 exponential law of generalization. The resulting dou-
ble exponential function, similar to the mathematical
forgetting function suggested by Wicklegren (1969),
0
closely fits data that follow the exponential to √t and
perfectly fits data that follow negative exponential
T=2s functions when the training delay T = 0. The com-
1
posite function retains the characteristics of indepen-
dent variation in intercept and slope of forgetting
Discriminability
0 functions. Both parameters are influenced by tempo-

ral distance and generalization components. The rea-
T=4s
sonable fit of the equation reinforces the notion that
1
remembering is a delay-specific discrimination with
generalization along the temporal dimension.
0 Direct Remembering
Considerable interest in the question of memory has
T=6s been sparked by the theoretical question of how to
1 bridge the temporal gap between events and subse-
quent behavior and also by the practical problems
(neurological, legal, aging, everyday) that arise
0
0 2 4 6 8 10 when memory goes wrong. An important task of
Probe Delay (s)
psychology is to address the practical issues. Per-
Figure 18.12. Dis haps, however, new insights for dealing with the
criminability, log d, as a practical problems will follow from an approach that
function of delay in probe the temporal gap does not have to bridged. Watkins
trials with different delays,
after exclusive training with (1990) complained that mediationist theories of
just one delay (T). Curves memory that rely on a representation of an event
are predictions from an embodied in a memory trace to bridge the temporal
equation with temporal gap are flawed. He argued in favor of bringing out
distance and generalization
components (White, 2001). the role of the stimulus environment in determining
From “Generalization of memory. In the study of perception, two very gen-
Delayed Matching-to-Sample eral approaches have been taken. In one, perception
Performance Following
involves active construction and the processing of
Training at Different
Delays,” by R. J. Sargisson information by the brain. In the other, perception is
and K. G. White, 2001, direct, as advocated by James J. Gibson (1979). The
Journal of the Experimental notion that remembering, too, might be direct is
Analysis of Behavior, 75, p.
12. Copyright 2001 by the consistent with an emphasis on environmental
Society for the Experimental causes of remembering and forgetting (Hackenberg,
Analysis of Behavior, Inc. 1993). Briefly, in a theory of direct remembering,
Reprinted with permission. the individual system is tuned to resonate to infor-
mation available at the time of retrieval through
at long delays are less discriminable than otherwise prior learning and evolution (White, 1991). If
equally spaced intervals at short delays. The curve remembering is direct, the forgetting function
described by White (2001) fitted to the data is a com- reflects increasing temporal distance in the same
bination of a negative exponential function, which way that errors of depth perception reflect increasing
430
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
spatial distance. Similarly, errors of memory follow Alsop, B., & Jones, B. M. (2008). Reinforcer control by
the same principles as errors of perception such comparison-stimulus color and location in a delayed
matching-to-sample task. Journal of the Experimental
as geometrical illusions, for which, as Gibson Analysis of Behavior, 89, 311–331. doi:10.1901/
explained, the information creating the error jeab.2008-89-311
is actually in the environment. Gibson, a self- Baddeley, A. (1997). Human memory: Theory and practice
confessed behaviorist, made a significant contribu- (Rev. edition). Hove, England: Psychology Press.
tion to psychology of perception, and the extension Baum, W. M. (1974). On two types of deviation from the
of his views to memory has the potential to bring matching law: Bias and undermatching. Journal of
new light to many unresolved questions about the Experimental Analysis of Behavior, 22, 231–242.
doi:10.1901/jeab.1974.22-231
remembering.
Berman, M. G., Jonides, J., & Lewis, R. L. (2009). In
search of decay in verbal short-term memory. Journal
Conclusion of Experimental Psychology: Learning, Memory, and
Cognition, 35, 317–333. doi:10.1037/a0014873
The experimental analysis of remembering has suc-
Berryman, R., Cumming, W. W., & Nevin, J. A. (1963).
ceeded in its description of the effects of a range of Acquisition of delayed matching in the pigeon.
variables on the function defining the relation Journal of the Experimental Analysis of Behavior, 6,
between accuracy and temporal distance. Different 101–107. doi:10.1901/jeab.1963.6-101
parameters of the sample stimulus, such as its dura- Blough, D. S. (1959). Delayed matching in the pigeon.
tion, repetition, and complexity, influence the inter- Journal of the Experimental Analysis of Behavior, 2,
151–160. doi:10.1901/jeab.1959.2-151
cept of the forgetting function. Conditions during
the retention interval and at the time of remembering Blough, D. S. (1996). Error factors in pigeon discrimina-
tion and delayed matching. Journal of Experimental
influence the slope of the forgetting function. Rein- Psychology: Animal Behavior Processes, 22, 118–131.
forcement parameters can influence both intercept doi:10.1037/0097-7403.22.1.118
and slope. Reinforcement variables have similar Brown, G. S., & White, K. G. (2005a). On the effects
effects on both accuracy of remembering and the of signalling reinforcer probability and magnitude.
strength of a single response. Together, these findings Journal of the Experimental Analysis of Behavior, 83,
support a general view that the complex making up 119–128. doi:10.1901/jeab.2005.94-03
the sample, the delay interval, and the choice is an Brown, G. S., & White, K. G. (2005b). The optimal
integrated behavioral unit. Thus, remembering is a correction for estimating extreme discriminabil-
ity. Behavior Research Methods, 37, 436–449.
discrimination at the time of the choice response and doi:10.3758/BF03192712
follows the same principles that govern discrimina-
Brown, G. S., & White, K. G. (2005c). Remembering:
tion and generalization of other behavior. In general, The role of extraneous reinforcement. Learning and
the effect of the delay between the sample and the Behavior, 33, 309–323. doi:10.3758/BF03192860
comparison stimuli is to make the discrimination Brown, G. S., & White, K. G. (2009a). Measuring dis-
more difficult (by analogy with the effect of spatial criminability when there are multiple sources of bias.
distance). By treating remembering as an integrated Behavior Research Methods, 41, 75–84. doi:10.3758/
BRM.41.1.75
unit of behavior, seeking processes that bridge the
temporal gap becomes unnecessary. The temporal Brown, G. S., & White, K. G. (2009b). Reinforcer prob-
ability, reinforcer magnitude, and the reinforcement
gap is a component of the compound discriminative context for remembering. Journal of Experimental
stimulus and is the most relevant aspect of the indi- Psychology: Animal Behavior Processes, 35, 238–249.
vidual’s environment when it comes to remembering. doi:10.1037/a0013864
Catania, A. C. (1992). Learning (3rd ed.). Englewood
References Cliffs, NJ: Prentice-Hall.
Adamson, C., Foster, T. M., & McEwan, J. S. A. (2000). Cook, R. G. (1980). Retroactive interference in pigeon
Delayed matching to sample: The effects of short-term memory by a reduction in ambient illu-
sample-set size on human performance. Behavioural mination. Journal of Experimental Psychology: Animal
Processes, 49, 149–161. doi:10.1016/S0376- Behavior Processes, 6, 326–338. doi:10.1037/0097-
6357(00)00087-5 7403.6.4.326
431
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
D’Amato, M. R. (1973). Delayed matching and short- Foster, T. M., Temple, W., Mackenzie, C., DeMello, L. R.,
term memory in monkeys. In G. H. Bower (Ed.), The & Poling, A. (1995). Delayed matching-to-sample
psychology of learning and motivation: Advances in performance of hens: Effects of sample duration and
research and theory (Vol. 7, pp. 227–269). New York, response requirements during the sample. Journal
NY: Academic Press. of the Experimental Analysis of Behavior, 64, 19–31.
doi:10.1901/jeab.1995.64-19
D’Amato, M. R., & O’Neill, W. (1971). Effect of delay-
interval illumination level on matching behavior in Gaitan, S. C., & Wixted, J. T. (2000). The role of “noth-
the capuchin monkey. Journal of the Experimental ing” in memory for event duration in pigeons.
Analysis of Behavior, 15, 327–333. doi:10.1901/ Animal Learning and Behavior, 28, 147–161.
jeab.1971.15-327 doi:10.3758/BF03200250
Davison, M., & Nevin, J. A. (1999). Stimuli, reinforcers, Gibson, J. J. (1979). The ecological approach to visual per-
and behavior. Journal of the Experimental Analysis of ception. Boston, MA: Houghton Mifflin.
Goldiamond, I. (1966). Perception, language, and con-
Davison, M. C., & Tustin, R. D. (1978). The relation ceptualization rules. In B. Kleinmuntz (Ed.), Problem
between the generalized matching law and signal- solving: Research method and theory (pp. 183–224).
detection theory. Journal of the Experimental Analysis of New York, NY: Wiley.
Goto, K., Kurashima, R., & Watanabe, S. (2010).
Dougherty, D. H., & Wixted, J. T. (1996). Detecting a Delayed matching-to-position performance in
nonevent: Delayed presence-versus-absence dis- C57BL/6N mice. Behavioural Processes, 84, 591–597.
crimination in pigeons. Journal of the Experimental doi:10.1016/j.beproc.2010.02.022
Grant, D. S. (1975). Proactive interference in pigeon
jeab.1996.65-81
short-term memory. Journal of Experimental
Dunnett, S. B., & Martel, F. L. (1990). Proactive interfer- Psychology: Animal Behavior Processes, 1, 207–220.
ence effects on short-term memory in rats: I. Basic doi:10.1037/0097-7403.1.3.207
parameters and drug effects. Behavioral Neuroscience,
Grant, D. S. (1976). Effect of sample presentation time
104, 655–665. doi:10.1037/0735-7044.104.5.655
on long-delay matching in the pigeon. Learning
Ebbinghaus, H. (1964). Memory: A contribution to experi- and Motivation, 7, 580–590. doi:10.1016/0023-
mental psychology (H. A. Ruger & C. E. Bussenius, 9690(76)90008-4
Trans.). New York, NY: Dover. (Original work pub-
Grant, D. S. (1988). Sources of visual interference in
lished 1885)
delayed matching-to-sample with pigeons. Journal of
Edhouse, W. V., & White, K. G. (1988). Sources of pro- Experimental Psychology: Animal Behavior Processes,
active interference in animal memory. Journal of 14, 368–375. doi:10.1037/0097-7403.14.4.368
Experimental Psychology: Animal Behavior Processes,
Grant, D. S. (2000). Influence of intertrial interval dura-
14, 56–70. doi:10.1037/0097-7403.14.1.56
tion on the intertrial agreement effect in delayed
Estévez, A. F., Overmier, B., & Fuentes, L. J. (2003). matching-to-sample with pigeons. Animal Learning
Differential outcomes effect in children: and Behavior, 28, 288–297. doi:10.3758/BF03200262
Demonstration and mechanisms. Learning and
Grant, D. S. (2006). Asymmetrical sample training and
Motivation, 34, 148–167. doi:10.1016/S0023-
asymmetrical retention functions in one-to-one
9690(02)00510-6
and many-to-one matching in pigeons. Learning
Fantino, E. (1977). Conditioned reinforcement: Choice and Motivation, 37, 209–229. doi:10.1016/j.
and information. In W. K. Honig & J. E. R. Staddon lmot.2005.06.003
(Eds.), Handbook of operant behavior (pp. 313–339).
Hackenberg, T. D. (1993). Commonsense and conven-
Englewood Cliffs, NJ: Prentice-Hall.
tional wisdom. Journal of the Experimental Analysis
Fetterman, J. G. (1995). The psychophysics of remem- of Behavior, 60, 457–460. doi:10.1901/jeab.1993.
bered duration. Animal Learning and Behavior, 23, 60-457
49–62. doi:10.3758/BF03198015
Harnett, P., McCarthy, D. C., & Davison, M. C. (1984).
Fetterman, J. G. (1996). Dimensions of stimulus com- Delayed signal detection, differential reinforcement,
plexity. Journal of Experimental Psychology: Animal and short-term memory in the pigeon. Journal of
Behavior Processes, 22, 3–18. doi:10.1037/0097- the Experimental Analysis of Behavior, 42, 87–111.
7403.22.1.3 doi:10.1901/jeab.1984.42-87
Fetterman, J. G., & MacEwen, D. (1989). Short-term Harper, D. N. (2000). An assessment and compari-
memory for responses: The “choose-small” effect. son of the effects of oxotremorine, D-cycloserine,
Journal of the Experimental Analysis of Behavior, 52, and bicuculline on delayed matching-to-sample
311–324. doi:10.1901/jeab.1989.52-311 performance in rats. Experimental and Clinical
432
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Psychopharmacology, 8, 207–215. doi:10.1037/1064- and brain of short-term memory. Annual Review

1297.8.2.207 of Psychology, 59, 193–224. doi:10.1146/annurev.
psych.59.103006.093615
Harper, D. N., & White, K. G. (1997). Retroactive inter-
ference and rate of forgetting in delayed matching-to- Kangas, B. D., Vaidya, M., & Branch, M. N. (2010).
sample performance. Animal Learning and Behavior, Titrating-delay matching-to-sample in the pigeon.
25, 158–164. doi:10.3758/BF03199053 Journal of the Experimental Analysis of Behavior, 94,
Harper, D. N., Wisnewski, R., Hunt, M., & Schenk, S. 69–81. doi:10.1901/jeab.2010.94-69
(2005). (±)3,4-methylenedioxymethamphetamine, Kendrick, D. F., Rilling, M. E., & Denny, M. R. (Eds.).
d-amphetamine and cocaine impair delayed (1986). Theories of animal memory. Hillsdale, NJ:
matching-to-sample performance via an increase in Erlbaum.
susceptibility to proactive interference. Behavioral
Neuroscience, 119, 455–463. doi:10.1037/0735- Kraemer, P. J., & Roberts, W. A. (1984). Short-term
7044.119.2.455 memory for visual and auditory stimuli in the
pigeons. Animal Learning and Behavior, 12, 275–284.
Hartl, J., & Fantino, E. (1996). Choice as a function of doi:10.3758/BF03199968
reinforcement ratios in delayed matching to sample.
Journal of the Experimental Analysis of Behavior, 66, Lane, S. D., Cherek, L. M. L., & Tcheremissine, O. V.
11–27. doi:10.1901/jeab.1996.66-11 (2005). Marijuana effects on human forgetting func-
tions. Journal of the Experimental Analysis of Behavior,
Herman, L. M. (1975). Interference and auditory short- 83, 67–83. doi:10.1901/jeab.2005.22-04
term memory in the bottlenosed dolphin. Animal
Learning and Behavior, 3, 43–48. doi:10.3758/ Lazareva, O. F., & Wasserman, E. A. (2009). Effects of
BF03209097 stimulus duration and choice delay on visual cat-
egorization in pigeons. Learning and Motivation, 40,
Hogan, D. E., Edwards, C. A., & Zentall, T. R. (1981). 132–146. doi:10.1016/j.lmot.2008.10.003
Delayed matching in the pigeon: Interference pro-
duced by the prior delayed matching trial. Animal Legge, E. L. G., & Spetch, M. L. (2009). The differential
Learning and Behavior, 9, 395–400. doi:10.3758/ outcomes effect (DOE) in spatial localization: An
BF03197849 investigation with adults. Learning and Motivation,
40, 313–328. doi:10.1016/j.lmot.2009.03.002
Honig, W. K., & Urcuioli, P. J. (1981). The legacy of
Guttman and Kalish (1956): 25 years of research on Loftus, G. R. (1985). Evaluating forgetting curves. Journal
stimulus generalization. Journal of the Experimental of Experimental Psychology: Learning, Memory,
Analysis of Behavior, 36, 405–445. doi:10.1901/ and Cognition, 11, 817–820. doi:10.1037/0278-
jeab.1981.36-405 7393.11.1-4.817
Jans, J. E., & Catania, A. C. (1980). Short-term remem- MacDonald, S. E., & Grant, D. S. (1987). Effects of
bering of discriminative stimuli in pigeons. Journal signaling retention interval length on delayed
of the Experimental Analysis of Behavior, 34, 177–183. matching-to-sample in pigeons. Journal of
13, 116–125. doi:10.1037/0097-7403.13.2.116
Jones, B. M. (2003). Quantitative analysis of matching-
to-sample performance. Journal of the Experimental Macmillan, N. A., & Creelman, C. D. (1991). Detection
Analysis of Behavior, 79, 323–350. doi:10.1901/ theory: A user’s guide. New York, NY: Cambridge
jeab.2003.79-323 University Press.
Jones, B. M., & White, K. G. (1992). Stimulus discrim- Mazur, J. E. (2006). Mathematical models and the
inability and sensitivity to reinforcement in delayed experimental analysis of behavior. Journal of the
matching-to-sample. Journal of the Experimental Experimental Analysis of Behavior, 85, 275–291.
Analysis of Behavior, 58, 159–172. doi:10.1901/ doi:10.1901/jeab.2006.65-05
jeab.1992.58-159 McCarthy, D., & Davison, M. (1986). Delayed reinforce-
Jones, B. M., & White, K. G. (1994). An investigation ment and delayed choice in symbolic matching to
of the differential-outcomes effect within sessions. sample: Effects on stimulus discriminability. Journal
Journal of the Experimental Analysis of Behavior, 61, of the Experimental Analysis of Behavior, 46, 293–303.
389–406. doi:10.1901/jeab.1994.61-389 doi:10.1901/jeab.1986.46-293
Jones, B. M., White, K. G., & Alsop, B. (1995). On two McCarthy, D. C., & Davison, M. (1991). The interac-
effects of signaling the consequences for remember- tion between stimulus and reinforcer control on
ing. Animal Learning and Behavior, 23, 256–272. remembering. Journal of the Experimental Analysis of
doi:10.3758/BF03198922 Behavior, 56, 51–66. doi:10.1901/jeab.1991.56-51
Jonides, J., Lewis, R. L., Nee, D. E., Lustig, C. A., McCarthy, D., & Voss, P. (1995). Delayed matching-to-
Berman, M. G., & Moore, K. S. (2008). The mind sample performance: Effects of relative reinforcer
433
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
f requency and of signaled versus unsignaled rein- Journal of Experimental Psychology: Human
forcer magnitudes. Journal of the Experimental Experimental Psychology, 61(A), 1400–1409.
Analysis of Behavior, 63, 33–51. doi:10.1901/ doi:10.1080/17470210701557597
jeab.1995.63-33
Nevin, J. A., Ward, R. D., Jimenez-Gomez, C., Odum, A.
Miyashita, Y., Nakajima, S., & Imada, H. (2000). L., & Shahan, T. A. (2009). Differential outcomes
Differential outcome effect in the horse. Journal of enhance accuracy of delayed matching to sample
the Experimental Analysis of Behavior, 74, 245–253. but not resistance to change. Journal of Experimental
doi:10.1901/jeab.2000.74-245 Psychology: Animal Behavior Processes, 35, 74–91.
Nairne, J. S. (2002). Remembering over the short-term: doi:10.1037/a0012926
The case against the standard model. Annual Review Odum, A. L., Shahan, T. A., & Nevin, J. A. (2005).
of Psychology, 53, 53–81. doi:10.1146/annurev. Resistance to change of forgetting functions and
psych.53.100901.135131 response rates. Journal of the Experimental Analysis of
Nakagawa, S., Etheridge, R. J. M., Foster, T. M., Sumpter, Behavior, 84, 65–75. doi:10.1901/jeab.2005.112-04
C. E., & Temple, W. (2004). The effects of changes Overmier, J. B., Bull, J. A., III, & Trapold, M. A. (1971).
in consequences on hens’ performance in delayed- Discriminative cue properties of different fears and
matching-to-sample tasks. Behavioural Processes, 67, their role in response selection in dogs. Journal of
441–451. doi:10.1016/j.beproc.2004.07.005 Comparative and Physiological Psychology, 76, 478–
Nelson, K. R., & Wasserman, E. A. (1978). Temporal 482. doi:10.1037/h0031403
factors influencing the pigeon’s successive matching- Parkes, M., & White, K. G. (2000). Glucose attenuation
to-sample performance: Sample duration, inter- of memory impairments. Behavioral Neuroscience,
trial interval, and retention interval. Journal of the 114, 307–319. doi:10.1037/0735-7044.114.2.307
doi:10.1901/jeab.1978.30-153 Peterson, L. R., & Peterson, M. (1959). Short-term reten-
tion of individual items. Journal of Experimental
Nevin, J. A. (1981). Psychophysics and reinforcement Psychology, 58, 193–198. doi:10.1037/h0049234
schedules. In M. L. Commons, J. E. Mazur, J. A.
Nevin, & H. Rachlin (Eds.), Quantitative analyses of Rayburn-Reeves, R., & Zentall, T. R. (2009). Animal
behavior: Vol. 1. Discriminative properties of reinforce- memory: The contribution of generalization decre-
ment schedules (pp. 3–27). Hillsdale, NJ: Erlbaum. ment to delayed conditional discrimination reten-
tion functions. Learning and Behavior, 37, 299–304.
Nevin, J. A., Cate, H., & Alsop, B. (1993). Effects of dif- doi:10.3758/LB.37.4.299
ferences between stimuli, responses and reinforcer
rates on conditional discrimination performance. Roberts, W. A. (1972). Short-term memory in the pigeon:
Journal of the Experimental Analysis of Behavior, 59, Effects of repetition and spacing. Journal of Experimental
147–161. doi:10.1901/jeab.1993.59-147 Psychology, 94, 74–83. doi:10.1037/h0032796
Nevin, J. A., Davison, M., Odum, A. L., & Shahan, T. A. Roberts, W. A. (1980). Distribution of trials and inter-
(2007). A theory of attending, remembering, and trial retention in delayed matching to sample with
reinforcement in delayed matching to sample. Journal pigeons. Journal of Experimental Psychology: Animal
of the Experimental Analysis of Behavior, 88, 285–317. Behavior Processes, 6, 217–237. doi:10.1037/0097-
doi:10.1901/jeab.2007.88-285 7403.6.3.217
Nevin, J. A., & Grace, R. C. (2000). Behavioral Roberts, W. A. (1998). Principles of animal cognition.
momentum and the law of effect. Behavioral Boston, MA: McGraw-Hill.
and Brain Sciences, 23, 73–90. doi:10.1017/ Roberts, W. A., & Grant, D. S. (1976). Studies of short-
S0140525X00002405 term memory in the pigeon using the delayed
Nevin, J. A., & Grosch, J. (1990). Effects of signaled matching-to-sample procedure. In D. L. Medin, W.
reinforcer magnitude on delayed matching-to- A. Roberts, & R. T. Davis (Eds.), Processes of animal
sample performance. Journal of Experimental memory (pp. 79–112). Hillsdale, NJ: Erlbaum.
Psychology: Animal Behavior Processes, 16, 298–305. Roberts, W. A., & Grant, D. S. (1978). An analysis
doi:10.1037/0097-7403.16.3.298 of light-induced retroactive inhibition in pigeon
Nevin, J. A., Milo, J., Odum, A. L., & Shahan, T. short-term memory. Journal of Experimental
A. (2003). Accuracy of discrimination, rate of Psychology: Animal Behavior Processes, 4, 219–236.
responding, and resistance to change. Journal of doi:10.1037/0097-7403.4.3.219
the Experimental Analysis of Behavior, 79, 307–321. Roberts, W. A., & Kraemer, P. J. (1982). Some observa-
doi:10.1901/jeab.2003.79-307 tions of the effects of intertrial interval and delay on
Nevin, J. A., Shahan, T. A., & Odum, A. L. (2008). delayed matching to sample in pigeons. Journal of
Contrast effects in response rate and accu- Experimental Psychology: Animal Behavior Processes,
racy of delayed matching to sample. Quarterly 8, 342–353. doi:10.1037/0097-7403.8.4.342
434
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Roediger, H. L. (2008). Relativity of remembering: intervals. Psychonomic Bulletin and Review, 5, 516–
Why the laws of memory vanished. Annual Review 522. doi:10.3758/BF03208831
of Psychology, 59, 225–254. doi:10.1146/annurev.
Shimp, C. P., & Moffitt, M. (1977). Short-term memory in
psych.57.102904.190139
the pigeon: Delayed pair-comparison procedures and
Rubin, D. C., & Wenzel, A. E. (1996). One hundred some results. Journal of the Experimental Analysis of
years of forgetting: A quantitative description of Behavior, 28, 13–25. doi:10.1901/jeab.1977.28-13
retention. Psychological Review, 103, 734–760.
Sidman, M. A. (1960). Tactics of scientific research. New
doi:10.1037/0033-295X.103.4.734
Ruske, A. C., Fisher, A., & White, K. G. (1997).
Spetch, M. L., & Wilkie, D. M. (1982). A systematic bias
Attenuation of scopolamine-induced deficits in
in pigeons’ memory for food and light durations.
delayed-matching performance by a new muscarinic
Behaviour Analysis Letters, 2, 267–274.
agonist. Psychobiology, 25, 313–320.
Spetch, M. L., & Wilkie, D. M. (1983). Subjective short-
Ruske, A. C., & White, K. G. (1999). Facilitation of
ening: A model of pigeons’ memory for event dura-
memory performance by a novel muscarinic agonist
tion. Journal of Experimental Psychology: Animal
in young and old rats. Pharmacology, Biochemistry
and Behavior, 63, 663–667. doi:10.1016/S0091-
7403.9.1.14
3057(99)00037-4
Suprenant, A. M., & Neath, I. (2009). Principles of mem-
Sands, S. F., Lincoln, C. E., & Wright, A. A. (1982).
ory. New York, NY: Psychology Press.
Pictorial similarity judgments and the organization
of visual memory in the rhesus monkey. Journal of Thorndike, E. L. (1898). Animal intelligence: An experi-
Experimental Psychology: General, 111, 369–389. mental study of the associative processes in animals.
doi:10.1037/0096-3445.111.4.369 Psychological Review: Monograph Supplements, 2, i–109.
Santi, A. (1984). The trial spacing effect in delayed Trapold, M. A. (1970). Are expectancies based upon
matching-to-sample by pigeons is dependent upon different positive reinforcing events discriminably
the illumination condition during the intertrial inter- different? Learning and Motivation, 1, 129–140.
val. Canadian Journal of Psychology/Revue Canadienne doi:10.1016/0023-9690(70)90079-2
de Psychologie, 38, 154–165. doi:10.1037/h0080830 Urcuioli, P. J. (2005). Behavioral and associative effects
Sargisson, R. J., & White, K. G. (2001). Generalization of differential outcomes on discrimination learn-
of delayed matching-to-sample performance fol- ing. Learning and Behavior, 33, 1–21. doi:10.3758/
lowing training at different delays. Journal of BF03196047
the Experimental Analysis of Behavior, 75, 1–14. Urcuioli, P. J., & DeMarse, T. B. (1997). Memory pro-
doi:10.1901/jeab.2001.75-1 cesses in delayed spatial discriminations: Response
Sargisson, R. J., & White, K. G. (2003). The effect of rein- intentions or response mediation? Journal of the
forcer delays on the form of the forgetting function. Experimental Analysis of Behavior, 67, 323–336.
Journal of the Experimental Analysis of Behavior, 80, doi:10.1901/jeab.1997.67-323
77–94. doi:10.1901/jeab.2003.80-77 Vonk, J., & MacDonald, S. E. (2004). Levels of abstrac-
Sargisson, R. J., & White, K. G. (2007a). Remembering tion in orangutan (Pongo abelii) categorization.
as discrimination in delayed matching to sample: Journal of Comparative Psychology, 118, 3–13.
Discriminability and bias. Learning and Behavior, 35, doi:10.1037/0735-7036.118.1.3
177–183. doi:10.3758/BF03193053 Ward, R. D., & Odum, A. L. (2007). Disruption of tem-
Sargisson, R. J., & White, K. G. (2007b). Timing, poral discrimination and the choose-short effect.
remembering, and discrimination. Journal of the Learning and Behavior, 35, 60–70. doi:10.3758/
Experimental Analysis of Behavior, 87, 25–37. BF03196075
doi:10.1901/jeab.2007.25-05 Wasserman, E. A. (1993). Comparative cognition:
Savage, L. M., & Parsons, J. (1997). The effect of delay Beginning the second century of the study of animal
interval, intertrial interval, amnestic drugs, and dif- intelligence. Psychological Bulletin, 113, 211–228.
ferential outcomes on matching-to-position in rats. doi:10.1037/0033-2909.113.2.211
Psychobiology, 25, 303–312. Wasserman, E. A., Grosch, J., & Nevin, J. A. (1982).
Shepard, R. N. (1987). Toward a universal law of gen- Effects of signaled retention intervals on pigeon
eralization for psychological science. Science, 237, short-term memory. Animal Learning and Behavior,
1317–1323. doi:10.1126/science.3629243 10, 330–338. doi:10.3758/BF03213719
Sherburne, L. M., Zentall, T. R., & Kaiser, D. H. (1998). Wasserman, E. A., Kiedinger, R. E., & Bhatt, R. S. (1988).
Timing in pigeons: The choose-short effect may Conceptual behavior in pigeons: Categories, subcate-
result from “confusion” between delay and intertrial gories, and pseudocategories. Journal of Experimental
435
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
K. Geoffrey White
Psychology: Animal Behavior Processes, 14, 235–246. White, K. G., & McKenzie, J. (1982). Delayed stimu-
doi:10.1037/0097-7403.14.3.235 lus control: Recall for single and relational stimuli.
Watkins, M. J. (1990). Mediationism and the obfusca- Journal of the Experimental Analysis of Behavior, 38,
tion of memory. American Psychologist, 45, 328–335. 305–312. doi:10.1901/jeab.1982.38-305
doi:10.1037/0003-066X.45.3.328 White, K. G., Parkinson, A. E., Brown, G. S., & Wixted,
Watson, J. E., & Blampied, N. M. (1989). Quantification J. T. (2004). Local proactive interference in delayed
of the effects of chlorpromazine on performance matching to sample: The role of reinforcement.
under delayed matching to sample in pigeons. Journal of Experimental Psychology: Animal Behavior
Journal of the Experimental Analysis of Behavior, 51, Processes, 30, 83–95. doi:10.1037/0097-7403.30.2.83
317–328. doi:10.1901/jeab.1989.51-317 White, K. G., & Ruske, A. C. (2002). Memory deficits
Watson, J. E., & White, K. G. (1994). The effect of phe- in Alzheimer’s disease: The encoding hypothesis
nobarbital on rate of forgetting and proactive inter- and cholinergic function. Psychonomic Bulletin and
ference in delayed matching to sample. Psychobiology, Review, 9, 426–437. doi:10.3758/BF03196301
22, 31–36. White, K. G., Ruske, A. C., & Colombo, M. (1996).
Weavers, R., Foster, T. M., & Temple, W. (1998). Memory procedures, performance and processes
Reinforcer efficacy in a delayed matching-to-sample in pigeons. Cognitive Brain Research, 3, 309–317.
task. Journal of the Experimental Analysis of Behavior, doi:10.1016/0926-6410(96)00016-X
69, 77–85. doi:10.1901/jeab.1998.69-77
White, K. G., & Sargisson, R. J. (2011). Maintained
White, K. G. (1974). Temporal integration in the generalization of delay-specific remembering.
pigeon. British Journal of Psychology, 65, 437–444. Behavioural Processes, 87, 310–313. doi:10.1016/j.
doi:10.1111/j.2044-8295.1974.tb01417.x beproc.2011.06.004
White, K. G. (1985). Characteristics of forgetting func- White, K. G., & Wixted, J. T. (1999). Psychophysics of
tions in delayed matching to sample. Journal of remembering. Journal of the Experimental Analysis of
the Experimental Analysis of Behavior, 44, 15–34. Behavior, 71, 91–113. doi:10.1901/jeab.1999.71-91
doi:10.1901/jeab.1985.44-15
White, K. G., & Wixted, J. T. (2010). Psychophysics of
White, K. G. (1986). Conjoint control of performance in remembering: To bias or not to bias? Journal of the
conditional discriminations by successive and simul- Experimental Analysis of Behavior, 94, 83–94.
taneous stimuli. Journal of the Experimental Analysis of
Behavior, 45, 161–174. doi:10.1901/jeab.1986.45-161 Wickelgren, W. A. (1969). Associative strength
theory of recognition memory for pitch.
White, K. G. (1991). Psychophysics of direct remember- Journal of Mathematical Psychology, 6, 13–61.
ing. In J. A. Commons, M. C. Davison, & J. A. Nevin doi:10.1016/0022-2496(69)90028-5
(Eds.), Models of behavior: Signal detection (pp. 221–
237). New York, NY: Erlbaum. Wickens, T. D. (1998). On the form of the retention
function: Comment on Rubin and Wenzel (1996): A
White, K. G. (2001). Forgetting functions. Animal Learning
quantitative description of retention. Psychological
and Behavior, 29, 193–207. doi:10.3758/BF03192887
Review, 105, 379–386.
White, K. G. (2002). Psychophysics of remember-
ing: The discrimination hypothesis. Current Wilkie, D. M., Summers, R. J., & Spetch, M. L. (1981).
Directions in Psychological Science, 11, 141–145. Effect of delay-interval stimuli on delayed sym-
doi:10.1111/1467-8721.00187 bolic matching to sample in the pigeon. Journal of
White, K. G., & Cooney, E. B. (1996). The consequences doi:10.1901/jeab.1981.35-153
of remembering: Independence of performance at
different retention intervals. Journal of Experimental Williams, D. C., Johnston, M. D., & Saunders, K. J.
Psychology Animal Behavior Processes, 22, 51–59. (2006). Intertrial sources of stimulus control and
doi:10.1037/0097-7403.22.1.51 delayed matching-to-sample performance in humans.
White, K. G., & Harper, D. N. (1996). Quantitative 253–267. doi:10.1901/jeab.2006.67-01
reanalysis of lesion effects on rate of forgetting in
macaques. Behavioural Brain Research, 74, 223–227. Wixted, J. T. (1989). Nonhuman short-term memory: A
doi:10.1016/0166-4328(95)00172-7 quantitative reanalysis of selected findings. Journal
White, K. G., Juhasz, J. B., & Wilson, P. J. (1973). doi:10.1901/jeab.1989.52-409
Is man no more than this? Evaluative bias
in interspecies comparison. Journal of the Wixted, J. T. (1990). Analyzing the empirical course
History of the Behavioral Sciences, 9, 203–212. of forgetting. Journal of Experimental Psychology:
doi:10.1002/1520-6696(197307)9:3<203::AID- Learning, Memory, and Cognition, 16, 927–935.
JHBS2300090304>3.0.CO;2-F doi:10.1037/0278-7393.16.5.927
436
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Wixted, J. T. (1993). A signal detection analysis of Woodworth, R. S., & Schlosberg, H. (1954). Experimental
memory for nonoccurrence in pigeons. Journal of psychology (rev. ed.). New York, NY: Holt, Rinehart
Experimental Psychology: Animal Behavior Processes, & Winston.
19, 400–411. doi:10.1037/0097-7403.19.4.400
Wright, A. A. (2007). An experimental analysis of mem-
Wixted, J. T. (2004a). On common ground: Jost’s (1897) ory processing. Journal of the Experimental Analysis of
law of forgetting and Ribot’s (1891) law of retro- Behavior, 88, 405–433. doi:10.1901/jeab.2007.88-405
grade amnesia. Psychological Review, 111, 864–879.
Wright, A. A., Urcuioli, P. J., & Sands, S. F. (1986). Proactive
doi:10.1037/0033-295X.111.4.864
interference in animal memory research. In D. F.
Wixted, J. T. (2004b). The psychology and neuroscience Kendrick, M. E. Rilling, & M. R. Denny (Eds.), Theories
of forgetting. Annual Review of Psychology, 55, of animal memory (pp. 101–125). Hillsdale, NJ: Erlbaum.
235–269. doi:10.1146/annurev.psych.55.090902.
Wright, F. K., & White, K. G. (2003). Effects of methyl-
141555
phenidate on working memory in pigeons. Cognitive,
Wixted, J. T., & Carpenter, S. K. (2007). The Wickelgren Affective and Behavioral Neuroscience, 3, 300–308.
power law and the Ebbinghaus savings function. doi:10.3758/CABN.3.4.300
Psychological Science, 18, 133–134. doi:10.1111/
Zentall, T. R., & Sherburne, L. M. (1994). The role of
j.1467-9280.2007.01862.x
differential sample responding in the differential
Wixted, J. T., & Ebbesen, E. B. (1991). On the form outcomes effect involving delayed matching by
of forgetting. Psychological Science, 2, 409–415. pigeons. Journal of Experimental Psychology: Animal
doi:10.1111/j.1467-9280.1991.tb00175.x Behavior Processes, 20, 390–401. doi:10.1037/0097-
Wixted, J. T., & Gaitan, S. C. (2002). Cognitive theo- 7403.20.4.390
ries as reinforcement history surrogates: The case Zokoll, M. A., Naue, N., Herrmann, C. S., & Langemann,
of likelihood ratio models of human recognition U. (2008). Auditory memory: A comparison between
memory. Animal Learning and Behavior, 30, 289–305. humans and starlings. Brain Research, 1220, 33–46.
doi:10.3758/BF03195955 doi:10.1016/j.brainres.2008.01.049
437
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 19
The Logic and Illogic of Human

Reasoning
Edmund Fantino and Stephanie Stolarz-Fantino
The Oxford Dictionary of Psychology defines logic erception than in evolution and that astrologers
p
in a general way as “rational thinking as distinct outnumbered astronomers 20-fold. Every day’s
from irrationality” (Colman, 2009, p. 429). It newspaper reveals examples of questionable
defines rational choices or decisions as “those that beliefs held by experienced professionals as well
are in the best interests of the agent who makes as by poorly informed laypeople. These examples
them” and that “maximize expected utility” (Col- of illogic do not necessarily reflect lack of intellec-
man, 2009, p. 636). It characterizes rational beliefs tual tools for dealing with the relevant data.
as being internally consistent, rational preferences Instead, as Gilovich argued, it is more likely that
as being transitive, and rational inferences as obey- “our questionable beliefs derive primarily from
ing the rules of logic—the set of rules according to the misapplication or overutilization of generally
which valid conclusions can be drawn from sets of valid and effective strategies for knowing” (p. 2).
premises. Reason was similarly defined by neurolo- In this chapter, we present examples that support
gist Donald B. Calne (1999) in his book Within this view and that illustrate factors that help or
Reason: hinder human decision making and problem
solving.
Reason is built upon a platform of logical
induction (observations lead to conclu-
sions which allow predictions) and logi- Behavioral Approaches to Decision-
cal deduction (if a and b are two classes Making Errors
and a is contained in b, then x is in a
Behavioral approaches have proven fruitful in study-
implies that x is in b). Reason assigns
ing errors of decision making. Several examples are
priority to observation over theory
discussed in the sections that follow.
(Galileo’s knife) and simplicity over
complexity (Ockham’s [sic] razor); it also
Probability Matching
demands consistency, coherence, and
Some decision-making problems to which partici-
efficiency. (p. 286)
pants respond in a nonoptimal fashion were devel-
Given that human beings are considered the oped in the laboratory. Perhaps the most intensely
most rational of animals (e.g., Huber, 2009), their studied is probability matching, whose provenance
logical lapses can be astonishing. For example, in dates back at least to the pioneering studies of Hum-
his book How We Know What Isn’t So, a discussion phreys (1939). In probability matching (or probabil-
of the “fallibility of human reason in everyday life,” ity learning) experiments, participants are typically
Gilovich (1991) pointed out that in the United given repeated trials of a binary choice procedure
States, more people believed in extrasensory in which one outcome pays off with a probability of
DOI: 10.1037/13937-019
439
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Fantino and Stolarz-Fantino
p and the other pays off with a probability of 1 − p. Sunk-Cost Effect

For values of p different than .5, the participant People become more likely to persist in questionable
should always choose the outcome with the higher courses of action once they have made an invest-
probability—that is, when p is more than .5, the out- ment. This sunk-cost effect has interested research-
come associated with this probability should always ers because it involves the inclusion of past costs
be chosen. Assume that a green button is associated into decision making, which counters the maxim
with a payoff probability of .75, and a red button is that choices should be based on marginal costs and
associated with a probability of .25. The participant benefits—that is, on an assessment of costs and
will be correct on 75% of trials by always choosing benefits from the current point onward. Although
green. Any other systematic strategy will lead to a Arkes and Ayton (1999) showed that there are no
lesser payoff. Despite these costs, however, human clear examples of sunk-cost behavior among non
participants typically match their choices to the humans, certain lines of research with human partic-
respective probabilities, choosing the green button ipants have suggested the possibility that nonhuman
on 75% of the trials and the red button on the other animals could display this effect. For example,
25% of trials. The result of this probability matching reinforcement history has been shown to affect sub-
is a decrease in correct outcomes; instead of being optimal persistence in an investment (Goltz, 1992,
correct on 75% of the trials, the probability matcher 1999). Both the partial reinforcement extinction
is correct on (.75 × .75) + (.25 × .25) = .625, or 63% effect (Goltz, 1992) and behavioral momentum
of trials. This result has been shown in literally (Goltz, 1999) have been implicated as mechanisms
scores of studies with human participants (whereas through which reinforcement history could result in
pigeons and rats tend to respond optimally). Proba- persistence. Thus, to explore conditions of uncer-
bility matching may persist over hundreds of trials tainty and reinforcement history under which
(see Myers, 1976, for a discussion of the early pigeons might persist in a losing course of action,
work). Fantino and Esfandiari (2002) found that Navarro and Fantino (2005) designed a procedure
participants moved from matching to something that models the sunk-cost decision scenario. They
between matching and maximizing over the course defined such a scenario as one in which an invest-
of two 96-trial blocks. The most optimal responding ment has been made toward a goal, negative feed-
was by participants who received accurate task- back concerning the investment has been received,
related instructions regarding the actual probabili- and the investor can persist in the investment or
ties of reinforcement associated with each of the two abandon it in favor of a new one. In their procedure,
stimuli. A second group also showed significant pigeons began a trial by pecking on a key for food.
improvement: They were asked to advise others on The schedule on the food key arranged a course of
the best strategy to follow. Presumably, once they action with initially good prospects that turned
had articulated an effective strategy for increasing unfavorable. On a given trial, one of four fixed-ratio
reinforcement, they began to follow it more consis- (FR) schedules was in effect: short (10), medium
tently themselves. (40), long (80), or extra long (160). On half the tri-
Other decision-making problems have been als, the short ratio was in effect; on a quarter of the
developed with paper-and-pencil scenarios, for trials, the medium ratio was in effect; and on a quar-
example, those investigating the sunk-cost effect ter of the trials, either of the two long ratios was in
and base-rate neglect. To study these problems effect. With these parameters, after the pigeons
using a behavioral approach, it is desirable to devise emitted the response number required by the short
behavioral analogues. We discuss these two prob- ratio, if no reinforcement had occurred (because one
lems because in each case analogous data have been of the longer ratios happened to be in effect), then
collected from both humans and pigeons. Moreover, the initially easy endeavor became more arduous—
in one case the results for the two species appear the expected number of responses to food was now
comparable (sunk cost), whereas in the other case greater (70) than it had been at the onset of the
they are ostensibly not (base rate). trial (45).
440
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
The Logic and Illogic of Human Reasoning
Navarro and Fantino (2005) gave pigeons the experiments, presses on the computer keys were the
option of escaping the now less favorable endeavor operant and hypothetical money was the reinforcer,
by allowing them to peck an escape key that initi- and the same contingencies were used. The human
ated a new trial. If the short ratio (FR 10) was not in data mirrored those of the pigeons. These results
effect on a given trial, then once 10 responses had suggest that at least two factors that contribute to
been emitted the optimal choice was to peck the the sunk-cost effect—economic salience and pres-
escape key (and begin anew on the food key). That ence of discriminative stimuli—may affect decision
is, the expected ratio given escape was lower than making of both nonhuman animals and humans in a
the expected ratio given persistence. Notice that similar manner.
at this choice point, the pigeons encountered a The sunk-cost effect is arguably of more than
sunk-cost decision scenario. Namely, they had made academic interest. All people have likely experi-
an initial investment, they had received negative enced situations in which they have persisted at an
feedback—no reinforcement—and they could either endeavor long after it was prudent to continue, and
persist in the venture or abandon it in favor of a the sunk-cost effect has been used to help under-
better one. stand investments gone awry, such as the Concorde
This general procedure allowed examination of supersonic airplane (indeed, there is the phrase Con-
the role of uncertainty in the sunk-cost effect in corde fallacy) and the Vietnam War. However, U.S.
two ways. One way was through the presence or society highly values persistence in pursuit of one’s
absence of stimulus changes. If a stimulus change goals. Rachlin (2000) has argued that persistence is
occurred at the moment when escape became opti- the backbone of self-control and the avoidance of
mal, then the economics of the situation should impulsive decision making (see de la Piedad, Field,
have been more salient than if no stimulus change & Rachlin, 2006, for a discussion). The iconic
had occurred. Navarro and Fantino (2005) hypo American inventor Thomas Edison is said to have
thesized that pigeons responding on this procedure said, “Many of life’s failures are people who did not
with no stimulus change would persist more than realize how close they were to success when they
pigeons responding on this procedure with a stimu- gave up.” The trick, of course, is in discriminating
lus change present. The results supported their when to persist. People’s ability to discriminate well
hypothesis—when stimulus changes were absent, depends on how much relevant information they
most of the pigeons persisted to the end of every have in hand. Given adequate information (or dis-
trial. When changes were present, however, all criminative stimuli), people and pigeons appear to
pigeons escaped as soon as it became optimal (this avoid the sunk-cost effect.
trend appeared once behavior had become stable).
A second way to manipulate uncertainty is by Base-Rate Neglect
varying the difference between the expected value of This robust phenomenon results from the fact that
persisting and the expected value of escaping. The people typically underweight the importance of
closer these expected values were to each other, the base rates in decision tasks involving two or more
less salient the advantage of escaping was and the sources of information (e.g., Goodie & Fantino,
more likely the pigeons should have been to persist. 1996; Tversky & Kahneman, 1982). In base-rate
The results again supported the hypothesis: As the experiments, participants are generally provided
advantage of escaping decreased (although still with information about base rates, which concern
being optimal), persistence rose. how often each of two outcomes occurs in the gen-
Additionally, by modifying this procedure for eral population, and case-specific information, such
use with human participants, previous findings as witness testimony or the results of a diagnostic
with human participants could be extended to a medical test. Typically, the participant’s task is to
novel format. The experiments with pigeons were select the more likely of the two outcomes or to pro-
replicated with humans (Navarro & Fantino, 2005, vide a verbal or written estimate of the probability of
2007) in a computer simulation. In the human one or both outcomes. An iconic base-rate problem
441
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
described by Tversky and Kahneman (1982) is the hysicians were asked to estimate the likelihood
p
taxicab problem; a variant appears next with values that a woman with a positive mammogram had
that should make the solution transparent: breast cancer, most participants estimated the
likelihood as 75%, which was very near the test’s
A cab was involved in a hit and run acci-
sensitivity. In fact, the correct response was close to
dent at night. Two cab companies, the
8% because of breast cancer’s low base rate. Can
Green and the Blue, operate in the city.
researchers learn something valuable about the vari-
You are given the following data:
ables that control base-rate neglect by adopting a
(a) 67% of the cabs in the city are Blue
behavioral approach? For example, what if—instead
and 33% are Green.
of giving participants the relevant information, as is
(b) A witness identified the cab as
done in paper-and-pencil versions of the task—
Green. The court tested the reliability
researchers had participants experience both the
of the witness under the same circum-
base rate and the accuracy of the case-cue informa-
stances that existed on the night of the
tion in a behavioral task over many trials? Would
accident and concluded that the witness
base-rate neglect still occur? To this end, Stolarz-
correctly identified each one of the two
Fantino and Fantino (1990) suggested using a modi-
colors 50% of the time and failed 50% of
fied matching-to-sample procedure as a base-rate
the time.
analogue. In the typical matching-to-sample proce-
What is the probability that the cab
dure, the sample appears on a single lit key and is
involved in the accident was Blue rather
one of two colors, here blue and green. After the
than Green?
sample is extinguished, two comparison stimuli
For this example, if both pieces of information appear, blue and green. The human’s or pigeon’s
(base rates of the two cab types and witness accu- task is to pick the stimulus that matches the sample.
racy) were considered, the probability that the cab In contrast, in the modified procedure used in
was blue is 67% would be clear. For less transparent the base-rate analogues from our laboratory, match-
values, the information would be combined according the sample 100% of the time is not the correct
ing to Bayes’s theorem to find the precise probability response. Instead, selection of the blue or green
(Birnbaum, 1983). Participants in most studies are comparison stimulus is correct a certain percentage
not expected to calculate the exact values using of the time (resembling the binary choices of
Bayes’s theorem. However, they might be expected probability-matching experiments, discussed earlier).
to use both sources of information and come up Consider this example, which is illustrated in
with an approximation of the correct answer. Figure 19.1: After a blue sample, selection of blue is
Instead, in most studies participants overweight the correct 67% of the time, and selection of green is
case-specific information and ignore, or at least correct 33% of the time; after a green sample, selec-
underweight (neglect), the base-rate information. tion of blue is again correct 67% of the time, and
Thus, in this simple example, participants who selection of green is correct 33% of the time. In this
neglect the base rate tend to assert that the probabil- example, the sample is completely uninformative: It
ity is 50% because the witness is uninformative. should not acquire discriminative stimulus proper-
As with the sunk-cost effect, the robustness of ties in selecting either comparison stimulus. Note
base-rate neglect is not simply of academic interest. that the values used here are the same as those
Striking examples have been reported involving in the earlier taxicab problem in which the witness
assessments of school psychologists (Kennedy, Wil- testimony (sample stimulus) was uninformative and
lis, & Faust, 1997), physicians (e.g., Eddy, 1982), the base rate of blue cabs in the city was 67%. Thus,
and AIDS counselors (Gigerenzer, Hoffrage, & Tversky and Kahneman’s (1982) taxicab problem
Ebert, 1998). Misunderstanding of the importance was converted into a nonverbal delayed matching-
of base rates has real implications for people’s to-sample procedure into which different base rates
lives. For example, Eddy (1982) found that when and case-cue reliabilities could be programmed, as
442
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
in Figure 19.1? Under these circumstances, they

Start should never have picked green, even when the
50% 50% comparison stimulus was green, because blue was
correct more often. With knowledge of probability
matching, however, one might expect that green
blue green would have been chosen on 33% of trials because it
was correct 33% of the time. Or, if participants’
choices mirrored those in a single-trial paper-and-
pencil version of the taxicab problem—that is, if
blue green blue green
they showed base-rate neglect—one might expect
that green would have been chosen after a green
67% 33% 67% 33%
Percent choices rewarded sample on 50% of trials because, in this example, the
sample was uninformative (correct 50% of the time).
Figure 19.1. Matching-to-sample analogue of the These possibilities, and the actual data obtained for
base-rate problem. From “An Experientially Derived
Base-Rate Error in Humans,” by A. S. Goodie and E. this combination of base rate and sample reliability,
Fantino, 1995, Psychological Science, 6, p. 103. Copyright are shown in Figure 19.2. In fact, the data show that
1995 by John Wiley & Sons. Adapted with permission. the green sample was matched on 56% of trials,
indicating base-rate neglect. This behavioral base-
illustrated in Figure 19.1. From a behavioral per- rate neglect persisted over the 400 trials studied,
spective, this problem involves multiple sources of even when the underweighting of base rates caused
stimulus control. the participants to lose money (Goodie & Fantino,
Goodie and Fantino (1995, 1996) with humans 1995, Experiment 2). Pigeons, however, chose opti-
and Hartl and Fantino (1996) with pigeons explored mally; they learned to respond exclusively on the
this behavioral base-rate problem, with a variety of stimulus that was more likely to result in reinforce-
values in different conditions. How did human par- ment. The results from other conditions supported
ticipants do when the sample was uninformative, as the same general pattern: For humans, the sample
70
Percent Choices Matching Green Sample
60 56
50
50
40
33
30
20
10
0
0
Optimal Probability Sample Accuracy Observed
Matching
Figure 19.2. Matching to the green sample according to whether participants

select optimally, probability match, or commit base-rate neglect. The observed data
are shown on the right. From “An Experientially Derived Base-Rate Error in Humans,”
by A. S. Goodie and E. Fantino, 1995, Psychological Science, 6, p. 104. Copyright 1995
by John Wiley & Sons. Adapted with permission.
443
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
information was overweighted and the base rates responses were always reinforced and nonmatches
were neglected (although not generally ignored); for were never reinforced. After this pretraining, the
pigeons, choices were controlled by both sample pigeons displayed base-rate neglect when con-
accuracy and base rates. fronted with problems that varied base rates and
Hartl and Fantino (1996) and Stolarz-Fantino sample accuracies. As Fantino, Kanevsky, and
and Fantino (1995) proposed that differences in Charlton (2005) concluded, “After a substantial his-
learning histories between humans and pigeons may tory of matching, pigeons are likely to neglect base
have been responsible for the differences in the rates, whereas the relatively ‘uneducated’ pigeon is
results. That is, from early childhood, humans are aptly sensitive to the multiple sources of stimulus
exposed to many situations in which matching control present in the matching-to-sample task”
things that are in some way the same is reinforced. (p. 825).
Laboratory pigeons lack a comparable history, How do results from behavioral analogues of
which enables them to learn the optimal pattern of base-rate problems compare with those from more
choice in tasks such as that of Hartl and Fantino. To typical paper-and-pencil tasks used in most research
strengthen this interpretation, it would be desirable on this topic? Stolarz-Fantino, Fantino, and Van
to show (a) that humans will not neglect base rates Borst (2006, Experiment 2) gave college students
when tested on problems in which prior learning is two sessions of training—providing computer-based
not likely to interfere and (b) that pigeons would practice, with feedback about the accuracy of each
show base-rate neglect if given, for example, a his- answer—on typical base-rate questions of the type
tory of matching comparable to that of humans. used by Tversky and Kahneman (1982); the ques-
With respect to the former, Goodie and Fantino tions varied across four levels of base rate and two
(1996, 1999) demonstrated that human participants levels of witness accuracy. In a third session, partici-
would not display base-rate neglect when symbolic pants were tested on the combinations used in train-
matching-to-sample tasks were used in place of the ing and on some novel base-rate problems. Accuracy
usual identity matching-to-sample tasks used in the improved over the course of training; however, this
prior research. For example, when the sample was a improvement was not reflected in performance on
line orientation (vertical or horizontal) and the com- the novel problems. Analysis of explanations for
parison stimuli were colors (blue and green), base- participants’ answers showed that compared with an
rate neglect did not occur. Instead, participants untrained control group, the trained students were
displayed probability matching (Goodie & Fantino, more aware of the importance of considering both
1996). When the symbolic matching-to-sample task base rates and witness accuracy when estimating
involved a previously learned relationship, however, likelihood. However, this awareness was not
base-rate neglect occurred (e.g., when the sample reflected in the accuracy of their estimates. Given
was the word blue or the word green and the com- that hundreds of trials were needed for Goodie and
parison stimuli were the colors blue and green). Fantino (1999) to overcome their participants’ ten-
Similarly, when human participants had been dency to neglect base rates, perhaps this should not
exposed to base rates without samples (i.e., when be surprising.
there were no competing sources of stimulus con- In ordinary experience, base rates are learned by
trol), they were later sensitive to base rates when a integrating events that occur over a long period of
matching-to-sample procedure was introduced time, and the base rates of many important events
(Case, Fantino, & Goodie, 1999). are unknown (Fiedler, 2000). For example, a
To complete the story that base-rate neglect may woman is unlikely to be aware of the base rate for
result from prior learning, Fantino, Kanevsky, and having breast cancer that applies to a person of her
Charlton (2005) gave pigeons an extensive history age; she is more likely to be aware of the sensitivity
of pretraining (more than 100 sessions) with infor- of the mammogram (i.e., how likely it is to detect
mative case cues. During these sessions, sample cancer if she has it). People usually have more expe-
accuracy was 100%—that is, the pigeons’ matching rience with the more salient case-cue information,
444
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
much as Goodie and Fantino’s (1996) participants f ollowing order of likelihoods: Linda is active in the
had extensive experience with matching similar feminist movement Linda is a bank teller and is
stimuli. Although participants in the typical paper- active in the feminist movement Linda is a bank
and-pencil laboratory task receive both pieces of teller. It is a violation of the rules of class inclusion
information simultaneously, their past experience for Linda to be more likely to be a feminist bank
may lead them to rely more on the case cue (the teller than to be a bank teller (who may or may not
witness) than on the base rate. Experimentation be a feminist). It may also be considered a violation
with a behavioral analogue suggests that making of the monotonicity axiom of probability, with
base rates more salient—as in the study by Case implications for judgments of preference (Zizzo,
et al. (1999)—should be a productive way to induce Stolarz-Fantino, Wen, & Fantino, 2000). In this
their use, and research with the more traditional sense, the conjunction effect can be considered a fal-
paper-and-pencil task (e.g., Stolarz-Fantino et al., lacy of reasoning.
2006) supports this view. University undergraduates are unlikely to be
unfamiliar with the rules of class inclusion; how-
Conjunction Effect ever, the conjunction effect appears widespread.
The conjunction effect is another classic fallacy Tversky and Kahneman (1983) reported that sam-
addressed by Tversky and Kahneman (1983). Under ples of students (from the University of California,
certain conditions, participants judge the conjunc- Berkeley, and from Stanford University) who had
tion of two events as more probable than the less completed several statistics courses performed better
likely of the two events assessed separately. One than previous samples, but even then, 36% of
example is Tversky and Kahneman’s “Linda” prob- respondents committed the fallacy. Tversky and
lem, in which participants were asked to rank the Kahneman also reported that 60 of 62 physicians,
relative likelihood of a set of statements about Linda when tested on a medical question similar in struc-
after reading a description of her background: ture to the Linda problem, rated the conjunction of
two symptoms as more likely than the less likely
Linda is 31 years old, single, outspoken
symptom alone. According to the investigators,
and very bright. She majored in philoso-
when the logical inconsistency was pointed out,
phy. As a student, she was deeply con-
most of the physicians “appeared surprised and dis-
cerned with issues of discrimination and
mayed to have made an elementary error of reason-
social justice, and also participated in
ing” (p. 302). Logic students are also not immune
anti-nuclear demonstrations.
to the effect. Stolarz-Fantino, Fantino, and Kulik
Linda is a teacher in elementary
(1996, Experiment 3) found conjunction errors in
school.
the responses of 43% of a sample of University of
Linda works in a bookstore and takes
California, San Diego (UCSD), students who had
Yoga classes.
just completed a course in logic and who were
Linda is active in the feminist move-
tested in their logic classroom. Thus, participants’
ment.
responses must be due at least in part to lack of rec-
Linda is a psychiatric social worker.
ognition that the task calls for the application of
Linda is a member of the League of
principles of logic, class inclusion, or probability. As
Women Voters.
one of Tversky and Kahneman’s participants is
Linda is a bank teller.
reported to have said, “I thought you only asked for
Linda is an insurance salesperson.
my opinion” (p. 300).
Linda is a bank teller and is active in
The conjunction effect may occur because of the
the feminist movement.
application (or, in this case, overapplication) of
(p. 297)
some form of averaging or weighted-averaging
As reported by Tversky and Kahneman (1983), model (see Fantino, Kulik, Stolarz-Fantino, &
85% of responding undergraduates reported the Wright, 1997, for a discussion). For example,
445
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
articipants may average the likelihood scores they

p and 20 in a frequency format. Some questions asked
have assigned to two statements to reach an esti- participants to rate the conjunction of two events;
mated likelihood for their conjunction. Anderson others involved rating the conjunction of three
(1981) has found that averaging the perceived value events. An example of a likelihood question is the
of stimuli is a strategy used in making many kinds of following:
judgments; thus, it would not be surprising for it to
Krista is 20 years old and is a student at a
be misapplied in situations that appear similar to
small, liberal arts college.
those in which averaging would normally be used.
On a scale of 0 to 100, there is a likeli-
There is also evidence from behavioral (Fantino &
hood of 20 that Krista is majoring in edu-
Savastano, 1996) and decision-making (Zizzo, 2001)
cation, a likelihood of 70 that she walks
tasks that human participants respond more to com-
for exercise, and a likelihood of 30 that
pound stimuli than they do to individual stimuli,
she plays in a campus musical group.
and this tendency may contribute to the conjunction
Please indicate the likelihood of the state-
effect. In line with the possibility that stimulus con-
ment about Krista that appears below
trol may affect the strategy participants apply to rat-
by entering a number from 0 to 100 to
ing the conjunction questions, Stolarz-Fantino,
represent likelihood—for example, “0”
Fantino, Zizzo, and Wen (2003, Experiments 3 and
would be virtually impossible, and “100”
4) reported that conjunction errors were least likely
virtually certain.
when the following two conditions were met: (a) The
What is the likelihood that Krista is an
framing description was omitted (the personality
education major and also plays in a musi-
sketch of Linda in the earlier example), and (b) the
cal group?
conjunction question was preceded by a set of logic
questions (rather than opinion questions or no other An example of a frequency question is the
questions at all). Presumably, the logic questions set following:
the occasion for regarding the conjunction judg-
Eric is 23 years old and is a student at a
ment as a problem in logic; in contrast, the personal-
university in North Carolina.
ity sketch may set the occasion for treating the
Out of 100 people like Eric, 30 are
conjunction judgment as an impression formation
majoring in biology, 70 play intramural
task, one in which the averaging strategy is fre-
sports, and 20 do volunteer work with
quently applied (e.g., Anderson, 1965).
children. Out of these 100 people, to how
According to Hertwig and Gigerenzer (1999),
many would the statement below apply?
participants are far less inclined to rate conjunctions
Indicate your answer by entering a num-
as more likely than their components when the task
ber from 0 to 100.
is presented in terms of frequencies rather than in
Out of 100 people like Eric, how many
terms of probabilities. They pointed out that the
do all three of the following: major in
word probability has a wide variety of interpretations
biology, play intramural sports, and do
when used in colloquial speech. Thus, use of the
volunteer work with children?
terms likelihood and probability does not necessarily
set the occasion for participants to apply the rele- After each response, participants in the feedback
vant problem-solving practices to typical conjunc- condition saw a statement telling them whether
tion questions. Fantino, Stolarz-Fantino, and their answer was “in the range that is considered
Navarro (2003) reported on a repeated-trials study correct,” or (if the conjunction was rated higher
intended to induce students to treat the conjunction than the likelihood or frequency of one of the com-
task as one to which the principles of probability, ponent statements) “too high to be in the range that
class inclusion, or both should be applied. Partici- is considered correct.” Those in the no-feedback
pants rated 40 conjunction questions presented by condition did not receive feedback on their
computer; 20 were presented in a likelihood format responses. In addition, participants received the
446
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
questions in two groupings (Phase 1 and Phase 2) in important to participants than determining the
one of three orders: (a) likelihood questions first, answers to arcane questions posed by a visiting
followed by frequency questions; (b) frequency experimenter. Perhaps a more rational view of the
questions first, followed by likelihood questions; or human thinker and decision maker would emerge if
(c) mixed presentation of the two formats. Would the experimental questions were posed in a more
receiving feedback help participants avoid the con- personal manner with adequate incentives and more
junction effect? More important, would answering restricted access to alternative activities.
questions in the frequency format first help them The issue of adequate incentives was taken up by
avoid errors on the likelihood problems? The results Hertwig and Ortmann (2001) in a Behavioral and
were positive on both counts: Participants who Brain Sciences target article that focused on differ-
received feedback answered more questions cor- ences between the experimental techniques typically
rectly in both phases than those who received no used by economists and those used by psychologists
feedback. Participants who received frequency ques- studying, among other areas, behavioral decision
tions first scored higher than those who received making and reasoning. Two of the techniques cited
likelihood questions first; in addition, likelihood- as more common among economists—the use of
first participants improved when rating the fre- repeated trials and pay for clearly defined perfor-
quency conjunctions in Phase 2, and frequency-first mance criteria—are also typical of research in the
participants continued to do well on their Phase 2 tradition of behavior analysis. Thus, an examination
likelihood questions. Most important, students of these topics seems especially relevant.
in the mixed-presentation condition scored similarly The issue of incentives has spurred a lively debate
to those in the frequency-first condition, supporting among psychologists and economists. Many econo-
the possibility that presenting questions in the fre- mists have doubted the value of results obtained
quency format can lead participants to recognize the without adequate incentives. Indeed, Camerer and
questions as requiring the application of class- Hogarth (1999) asserted that unless a study was
inclusion principles, even when those questions financially motivated, it was “essentially impossible
appear in the less transparent likelihood format. to have it published in an economics journal” (p. 35).
Psychologists, however, have often been content to
conduct their studies of logical problems without
Can Stronger Incentives Lead to
incentives or with hypothetical incentives. The vir-
More Optimal Performance?
tues of omitting incentives are obvious. For example,
A first step toward understanding the irrationality where an economist might use real monetary incen-
observed in some domains of human reasoning and tives to encourage rational decision making, the psy-
decision making that would be obvious to a behav- chologist would save that money by omitting the
ior analyst, or nearly any psychologist for that mat- incentive or asking the participants to imagine
ter, is to ascertain whether the person is adequately receiving money. Moreover, where hypothetical
motivated. An inadequately motivated participant money is concerned, there is no limit to the amounts
may pay insufficient attention to the task at hand or that can be used. For example, Rachlin, Brown, and
might be distracted by a variety of competing activi- Cross (2000) were able to investigate temporal and
ties and alternative sources of reinforcement in probability discounting using standard amounts that
the immediate environment. This concern is espe- ranged from $10 to $1,000,000. So the issue of the
cially apt considering that many studies involving role of incentives potentially has both theoretical and
decision-making errors are carried out with paper methodological implications. In a commentary to the
and pencil in classrooms filled with dozens—or Hertwig and Ortmann (2001) target article, Fantino
even hundreds—of students. The competing activi- and Stolarz-Fantino (2001) took the position that
ties include using laptops and iPods, talking with hypothetical rewards could be as effective as real
friends, completing reading for an upcoming class, ones and that their use incurred no loss in external
and many more; any of these are arguably more validity. We now review the studies that led to this
447
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
conclusion as well as newer data that convinced us participants give their responses, given the afore-
that—for certain kinds of studies—the economists mentioned difficulties with studies carried out in
were on solid ground after all. large classrooms. In one of our studies on the con-
junction fallacy (Stolarz-Fantino et al., 2003, Exper-
When Monetary Rewards Do Not Improve iment 5), we studied participants individually for six
Performance trials (rather than the usual one trial) and varied the
In our human decision-making laboratory, we have incentive for correct answers. In one condition, the
had three opportunities to compare performance in experimenter sat next to the participant with a roll
comparable situations in which some participants of $1 bills. Participants received $3 for each answer
received hypothetical money and others received that did not constitute an example of the conjunc-
real money for optimal responding. These studies tion fallacy. Participants could earn up to $18 in just
have involved three different issues: observing a few minutes by not committing the fallacy. How-
behavior, base-rate neglect, and the conjunction ever, participants in this high-incentive condition
fallacy. We discuss each briefly in turn. did no better than those in the no-incentive control
In studies of observing, participants have the group or in any other of the groups we studied.
opportunity to produce discriminative stimuli that Moreover, there was no sign of improvement over
are correlated with the schedules of reinforcement the six trials. Thus, inattentiveness to the task owing
in effect. For example, in a typical procedure, two to a lack of appropriate incentive does not appear to
schedules alternate; the same stimulus is associated be responsible for the prevalence of the conjunction
with each (a mixed schedule). In the procedure effect (seen in as many as 80% of participants in
developed by Wyckoff (1952), performance of an some studies).
observing response produces a stimulus associated Although the rewards offered human research
with the schedule in effect. Producing the observing participants are meager compared with those offered
stimulus has no effect on the receipt of reinforce- pigeons maintained at 80% of their free-feeding
ment; it only signals which schedule is currently weights, on the basis of the results from these three
active. In behavioral terminology, it changes the sets of studies, concluding that real money was not
mixed schedule to a multiple schedule. Results from critical in generating meaningful responding in
numerous studies have shown that under most cir- studies with human participants seemed reasonable
cumstances, only the stimulus associated with the (Fantino & Stolarz-Fantino, 2001). Presumably,
richer of the two schedules maintains observing (for there are other reinforcers for human participants in
recent discussions and data, see Escobar & Bruner, these situations, including, in different studies, the
2009; Fantino & Silberberg, 2010). In one of several receipt of points, compliance with the experiment-
studies undertaken to ascertain the generality of this er’s instructions, or feedback that the response was
finding, Case and Fantino (1989) found that it did correct (Goodie, 2001). Moreover, some experimen-
not matter if points were worth money (either $0.05 tal tasks, particularly those involving reasoning, may
or $0.25). There was no difference in the rate or pat- be intrinsically reinforcing to college students. For
tern of observing among participants whose points example, some studies have used video games,
were exchangeable for money and those whose which are known to maintain high rates of participa-
points were not. Likewise, in studies of base-rate tion (e.g., Case, Ploog, & Fantino, 1990).
neglect with human participants, reinforcing correct In a review of 74 studies that varied in the
answers with a variety of incentives—ranging from amount of incentive, Camerer and Hogarth (1999)
points to varying amounts of real money—had no found that incentives improved performance of easy,
significant effect on participants’ tendency to effort-responsive tasks but did not help performance
neglect the base rates (Goodie & Fantino, 1995, of more difficult tasks. At any rate, in our studies the
Experiment 2). use of real money as a replacement or additional
As robust as the conjunction effect has proven to reward appeared to provide no additional benefit to
be, one wonders how much careful consideration performance. That the effects of rewards may not be
448
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
additive is consistent with a classic literature on and $50 for the recipient) and sometimes com-
incentives that suggests that there is an optimal level pletely self-interested ($0 for the recipient). Other
of motivation beyond which performance declines participants offer the recipient around $30, which
(Yerkes & Dodson, 1908). Indeed, recent work by approximates the mean offer (30%) in many studies
Mobbs et al. (2009) found that participants were (Camerer, 2003; Forsythe, Horowitz, Savin, &
less successful at capturing “prey” in a reward- Sefton, 1994).
pursuit task under conditions of higher reward The ultimatum game is more complex because
(about $10.00 per successful capture) than under the recipient plays an active role. The allocator
conditions of lower reward (about $1.00 per suc- (again in a one-trial game) is told to make an offer
cessful capture). to divide the money; if the offer is accepted by the
recipient, the offered distribution will be honored.
The Special Case of Economic Games However, the recipient has the option to reject the
It may be that monetary payments are not necessary offer, in which case neither the allocator nor the
for optimizing participants’ performance when other recipient receives anything. No negotiation is per-
incentives are present. However, economists who mitted. Participants tend to be somewhat more gen-
stress the importance of carrying out studies with erous in the ultimatum game (certainly there are no
real money have often focused on a different area offers of $0) than in the dictator game. Recipients
of research: economic games. Literally hundreds tend to accept offers that are at least 30% of the total
of studies have been published on the Prisoner’s (here $30) and to reject offers below 20% (Camerer,
Dilemma game and its variants, conducted by social 2003).
scientists of all stripes as well as by biologists (see In neither the ultimatum game nor the dictator
Barash, 2003, Chapter 3, and Rachlin, 1997, for dis- game do participants make their decision on the
cussions of the Prisoner’s Dilemma). Economists strict basis of maximizing their dollar amount as
have developed other games involving the distribu- would be expected by a strict adherence to eco-
tion of money. Most common among these are the nomic rationality (Forsythe et al., 1994; Güth,
dictator game and the ultimatum game, which we Schmittberger, & Schwarze, 1982). The rationale for
introduce briefly (for a fuller treatment, see Camerer, the prediction is simple. In the dictator game, the
2003; see also Chapter 20, this volume). These offer should be $0, but only a minority of offers are
games ask a participant to divide an amount of at or near zero. In the ultimatum game, the offer
money, usually between the participant and a second should be just above zero, say $1, because the recip-
person. Economists have argued that the distribution ient should accept any nonzero offer as being better
of funds when hypothetical money is the currency than nothing. In fact, though, recipients nearly
may tell researchers little about the distribution of always reject near-zero offers, and the proposers,
real money. Therefore, they would argue that studies presumably anticipating this, nearly always offer
using hypothetical money have weak external valid- considerably more (often 40%–50% of the pie).
ity (i.e., limited applicability to real-world deci- Although from a strict economic perspective these
sions). If correct, this weak external validity would results suggest nonrational decision making,
diminish the value of studies of economic distribu- whether anyone would really expect the kind of
tion games conducted with hypothetical rewards. rational behavior posited by the theory is not clear.
In the dictator game, a participant (the allocator) A wealth of other variables (including cultural and
is asked to divide a sum of money, say $100, historical ones) are presumably modulating the
between him- or herself and another person (the influence of the distribution of dollars. As Güth
recipient, typically an unseen stranger). The game is et al. (1982) pointed out, “Subjects often rely on
usually a one-trial game, and the allocator’s decision what they consider a fair or justified result. Further-
is honored (hence the dictator aspect of the game). more, the ultimatum aspect cannot be completely
Dictator game offers vary across participants; they exploited since subjects do not hesitate to punish if
are sometimes equitable (here $50 for the allocator their opponent asks for ‘too much’” (p. 384).
449
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Research on economic distribution games per- told that the other participant was a student, and the
mits researchers to ask whether the distributions other half were told that the other participant was a
differ depending on whether the money being computer. The results showed that male participants
divided is real or hypothetical. Camerer and Hog- chose optimally (i.e., maximized their earnings)
arth (1999) reported that the answer depends on the more frequently than female participants, who
game. For the Dictator Game, less generous offers tended to distribute the funds more equally. Surpris-
are made with real money. Camerer and Hogarth ingly, in each of three experiments, whether partici-
speculated that with real money, the nonfinancial pants were told that the anonymous other was
reinforcers in the situation are relatively less another student or a computer did not matter, even
effective. when real money was being divided. The fact that
Research in our laboratory has compared the participants would be equitable or competitive with
effects of real and hypothetical money in a series of a computer (rather than maximizing their real-
studies using the sharing game, a variant of the dic- money earnings) may or may not be labeled irratio-
tator game developed by Kennelly and Fantino nal, but in any case it underscores the complexity of
(2007). In this game, the participant makes a series choice. In addition, using such terminology as game
of binary choices (for self and the other player) in and player in one experiment was associated with an
which one outcome yields higher amounts for both increased frequency of competitive behavior relative
players (with the chooser receiving the smaller of to other experiments, which used more neutral lan-
the payoffs) and the other outcome yields lower guage. Relevant to our discussion of real and hypo-
amounts for both players (with the chooser receiv- thetical outcomes, participants in the monetary
ing the larger of the payoffs). For example, the conditions acted more optimally than those in the
choice might be between (a) $7 for oneself and $9 nonmonetary conditions; overall, they significantly
for the other or (b) $5 for oneself and $3 for the more often selected optimally: 68% of the time,
other. On any given trial, the participant is con- compared with 50% of the time in the hypothetical
strained to select between the two displayed money condition. In a subsequent experiment, Fan-
outcomes. tino and Kennelly (2009) found a within-subjects
Unlike the ultimatum and dictator games, effect of real versus hypothetical money in which
which do not constrain participants’ choices in this participants who were switched from a hypothetical
way, the sharing game allows one to distinguish to a real-money condition became more optimal
between those who prefer to maximize their earn- (and less likely to equalize). Overall, though, equal-
ings and those who prefer a maximized relative ity was the modal strategy used, especially in the
advantage over the other. Thus, in the preceding hypothetical money conditions (and especially for
example, choosing the $7–$9 option can be consid- female participants).
ered optimal (earnings maximization) and choosing To recapitulate, the role of monetary incentives
the $5–$3 option can be considered competitive on the quality of performance in tasks involving
(sacrifice $2 to obtain more than the other partici- decision making appears to depend on the nature of
pant). What about participants who prefer an the task. When the task is an economic distribution
exactly equal distribution? Kennelly and Fantino activity in which money is being divided, it is clear
(2007) gave participants 20 trials; each choice was that the distributions selected will be different with
always repeated once. Thus, the first and second real, as opposed to hypothetical, money. This does
trials offered the same set of outcomes, as did the not mean that it is always necessary to use real
third and fourth, and so on, which enabled partici- money, a daunting prospect when large sums are
pants to alternate choices on successive trials, being distributed. First, in situations for which qual-
enabling a more equal division of the money. Half of itative assessments—for example, determining the
the participants were distributing real money, and direction of an effect—are adequate, hypothetical
half were distributing hypothetical money. Half of money may be used profitably. Second, there are
the participants in each of these conditions were naturalistic rewards other than real money. For
450
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
example, Fantino, Gaitan, Kennelly, and Stolarz- As an example of how humans may reason, con-
Fantino (2007) used time out from a tedious task as sider the following short passage from the Nobel
a resource to be divided in the ultimatum and dicta- Laureate Jose Saramago’s (1997) masterpiece Blind-
tor games and found that it could produce distribu- ness. An individual goes suddenly blind in the first
tions comparable in optimality to those made with page of the novel. A good Samaritan helps him
real money (and more optimal than those made with arrive home safely, and this dialogue occurs (note
hypothetical money). As an explanation for human that the dialogue shifts between speakers at the capi-
participants’ often poor performance on a host of talizations): “You cannot imagine how grateful I am,
decision and logical tasks, however, it appears that Don’t thank me, today it’s you, Yes, you’re right,
lack of adequate incentive is not a major culprit. tomorrow it might be you” (Saramago, 1997, p. 4).
This is a particularly ironic example of karma
because moments later the good Samaritan steals the
Altruism: Does It Exist and Is It
blind man’s car and becomes blind himself moments
Rational?
after the theft. We suspect most readers interpret
The sharing game discussed earlier is potentially this last attack of blindness as fitting, as in the famil-
valuable as a tool to study altruistic behavior. For iar expression, “What goes around comes around.”
example, consider the following choice: Player 1 If this interpretation is to be taken seriously, how-
receives $10 and Player 2 receives $10, or Player 1 ever, one would need to know whether people really
receives $0 and Player 2 receives $100. Suppose fur- do hold such opinions. In other words, if this expla-
ther that real money is involved (the experimenter nation of instances of altruistic behavior has merit,
has a cashbox filled with $10 and $100 bills) and then at a minimum, researchers need to explore the
that the two players will never meet and would prevalence of the belief that what goes around
remain anonymous. We have in fact now done an comes around.
approximation of this study three times (although The answer to this exploration has important
once with hypothetical money only) conducted by methodological and theoretical implications. If
three different experimenters (Shawn Charlton, many people believe in the kind of reciprocity
Jamaal Clarke, and Art Kennelly). All three times expressed by “What goes around comes around,”
about 20% of our participants (UCSD students) then it would appear impossible to demonstrate
chose altruistically—that is, distributed $100 to the altruism. Some investigators (e.g., Hoffman,
anonymous second participant while taking nothing McCabe, & Smith, 1998) have pointed out that if
for themselves. Male participants were far more the potential altruists and their recipients are anony-
likely to select the altruistic option. Perhaps this mous to one another, then no possibility of reciproc-
gender difference reflects our earlier finding that ity exists. In that event, it could at least be argued
female participants are more likely to make egalitar- that any altruistic choices must represent true altru-
ian selections in the sharing game. ism (i.e., altruism not explainable in terms of benefit
We do not believe that these results necessarily to the putative altruist). If people generally believe
argue for true altruism in those 20% of participants that reciprocity is a fact of life, however, then it
who select the altruistic option. Several explanations would appear impossible to demonstrate true altru-
of this choice are consistent with traditional rein- ism whether or not the anonymity of the altruist is
forcement views. To begin with, following the preserved.
Golden Rule (“Do unto others as you would have In fact, we questioned several hundred UCSD
them do unto you”) may have been reinforced by undergraduates about their agreement with the
parents, friends, members of the clergy, and so statement “What goes around comes around” (Fan-
forth. Principles of conditioned reinforcement, stim- tino & Stolarz-Fantino, 2010). Of respondents, 86%
ulus control, and rule-governed behavior are among agreed with this statement, suggesting at least
those that could be brought to bear (see Fantino & implicit agreement with the idea that altruistic acts
Stolarz-Fantino, 2002, for a discussion). may eventually be reciprocated, even when the
451
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
recipient is totally unknown to the actor. Moreover, schedule contingencies. In a study with 3.5- to
we found at least a weak correlation between degree 5.5-year-old children, Vaughan (1985) found that
of belief in this notion and the degree of altruism children instructed about the correct sequence of
expressed in an economic distribution scenario. a chain of responses made fewer errors in acquisi-
Despite our contention that it may be impossible tion; however, their behavior did not hold up well
to demonstrate true altruism, it is clearly of interest when the instruction stimuli were discontinued. In
to identify the conditions (including reinforcement another study with children, Michael and Bernstein
history) that promote altruistic choices and to iden- (1991) studied preschool children over many ses-
tify the characteristics of individuals who make sions on a matching-to-sample task. The children
altruistic choices. Important too is a determination were assigned to one of three training conditions:
of generalized altruism—that is, are those who make instructed, contingency shaped, and imitation. They
altruistic choices in one set of circumstances (e.g., found that participants in the instructed and imita-
in the sharing game) more likely to make them in tion conditions acquired the relations in fewer ses-
other contexts (e.g., when responding to social sions than did those in the contingency-shaped
dilemma scenarios)? Is altruism rational? Perhaps it condition; however, when the rules changed, chil-
is not rational (in an economic sense) in the case in dren in the instructed and imitation conditions
which no reciprocity is possible. Where a broader were slower to adapt. Michael and Bernstein pointed
view of reciprocity exists, however, giving up some- out that children in the instructed and imitation
thing to help someone else may be regarded as a conditions behaved nearly identically and that this
good idea—and a rational act. might be an indication that they engaged in self-
instruction. This conceptualization is consistent
with findings by Lowe and his colleagues (Bentall &
Rule-Governed Behavior
Lowe, 1987; Horne & Lowe, 1993) that verbally
Accounting for such seemingly altruistic acts in proficient humans form covert rules about situations
behavioral terms requires a consideration of the they encounter (see also Rosenfarb, Newland, Bran-
impact of rules on behavior. Behavioral psycholo- non, & Howey, 1992).
gists have shown interest in the role of rules in Kudadjie-Gyamfi and Rachlin (2002) investigated
choice and problem solving. B. F. Skinner (1969) rule-governed and contingency-governed behavior
defined a rule as a contingency-specifying stimulus on a self-control task. They found that participants
and argued that behavior under the control of rules provided with a hint about the contingencies per-
may differ from behavior controlled directly by the formed better on the self-control task (and were thus
behavioral contingencies specified by the rules. closer to maximizing their earnings) but were less
Thus, when one learns to speak Italian by following sensitive to an unsignaled change in the contingen-
a set of grammatical rules in a textbook, one’s Italian cies than participants who were not given the hint.
will differ from spoken Italian learned directly by Rules do not always produce optimal behavior,
the consequences supplied by an Italian-speaking however. Hackenberg and Joker (1994) examined
community (see Hineline & Wanchisen, 1989). Spe- the effects of instructions on humans’ choices in
cifically, rule-governed (instructed) behavior may be a diminishing-returns procedure. Instructions
less sensitive to changes in environmental contin- were initially accurate in that they corresponded to
gencies than is contingency-shaped behavior. the optimal rate of reinforcement (points later
Research by Catania and his colleagues (e.g., Cata- exchangeable for money). Across blocks of sessions,
nia, Shimoff, & Matthews, 1989; Shimoff, Catania, however, the contingencies were altered systemati-
& Matthews, 1981) has shown this to be the case cally in ways that departed from the instructions.
with respect to responding to unannounced changes Participants continued to follow the instructions,
in schedules of reinforcement; Baron and Galizio despite their growing inaccuracy with respect to the
(1983) found evidence that instructions can exert contingencies, which resulted in substantial losses
strong control over responding, even overriding in potential earnings. Rules can therefore be either
452
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
beneficial or detrimental, depending on the contin- Luchins study. The changing-rules group received
gencies they specify. Indeed, many people acquire the same general instructions, but different prob-
useful rules from their parents, teachers, and other lems, each of which required a unique solution. Par-
role models. However, history is replete with exam- ticipants solved as many problems as they could
ples of rule following going dreadfully wrong. It is during the allotted time. During Phase 2, partici-
important to assess behaviorally the conditions pants in each group were assigned to one of three
under which rules are aptly and flexibly applied. test conditions; again, they solved as many problems
What is the effect of rule learning on problem- as they could within the allotted time. In one condi-
solving tasks? Research by Luchins (1942) investi- tion, the problems could be solved by a novel rule
gated this using the classic water jar problem. In the that was different from the one used by those in the
typical Luchins task, adults and children were pre- instructed- or induced-rule groups during Phase 1.
sented with the case of three water jars that held In a second condition, the Phase 1 rule would work
specified amounts of water. They were asked to to solve the problems, but they could be solved
determine how to obtain a particular amount of more efficiently by using a shortcut. This condition
water by using only the three jars. For example, par- was most similar to that of the Luchins studies. In
ticipants might be told that Jars A, B, and C held 14, the third condition, all problems had unique
163, and 25 quarts of water, respectively, and that solutions.
their task was to determine how to measure out Was there evidence of rigidity as a result of previ-
exactly 99 quarts of water. Participants were given ous problem solving? Some rigidity occurred in the
2.5 minutes in which to find the solution (B − A − second condition (most similar to that of Luchins,
C − C, or B − A − 2C); they then solved several 1942), in which the old rule still applied but for
more problems (involving different sized jars) that which a shortcut was available. However, this
could be solved using the same rule. Later in the impairment was generally short lived. Of greater
task, participants encountered a problem that could interest were the results from the first condition (all
be solved either by the same rule or by a more direct problems could be solved by a single rule different
method: A + C. Most participants failed to see the from the one used in Phase 1). Students from the
shortcut—which was used readily by participants instructed-rule group performed better in this con-
without experience with the original rule—and condition than did those from the other two groups.
tinued to make use of the more complex rule. Many Perhaps the instruction in the Phase 1 rule made
were embarrassed at having missed the obvious these participants more sensitive to the possibility of
solution when it was pointed out to them. Luchins finding an effective rule during Phase 2, at least
and Luchins (1950) conducted additional experi- when there was an effective rule to be found; they
ments that supported the view that experience with were no better at solving changing-rules problems in
a rule, even when it is self-generated, can hinder Phase 2 than were participants from the other Phase
subsequent problem solving. 1 conditions.
Fantino, Jaworski, Case, and Stolarz-Fantino Similar studies have been carried out with
(2003, Experiment 1) assessed the effects of rule use schoolchildren as participants. Kanevsky (2006)
in a task that resembled that of Luchins (1942). studied the effect of instructed and induced rules on
During Phase 1, participants (college students) were the solution of math problems by sixth-grade stu-
randomly assigned to one of three groups. The dents. For example, in one study students practiced
instructed-rule group was given an equation and solving problems that involved making round trips
told that it could be used to solve all of the prob- of some distance a certain number of times per hour,
lems. The induced-rule group received the same day, week, and so forth (X = 2 × distance × num-
problems and general instructions but were not told ber of trips). Students in the instructed-rule group
that a single rule could solve the problems and were were told how to solve such problems, whereas
not given an equation. Their instructions were more those in the induced-rule group were not. Students
analogous to those received by participants in the in the changing-rules group solved a set of problems
453
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
that all involved different rules. Students in all three competing sources of stimulus control. As with the
groups received feedback on their answers. Then sunk-cost effect and base-rate neglect (and other fal-
students in all three groups solved transfer problems lacies in decision making), the case can be made
that required a novel rule; these problems involved that gambling, as with persistence and focusing on
finding out how much time was spent per unit of case cues, is sometimes a rational course of action.
work (X = 60 T/N, where T is time and N is the For example, when resources are insufficient to sus-
number of work units). Kanevsky found that stu- tain life, gambling what is left may be more optimal
dents in the induced-rule group did best on the than not doing so. Indeed, the budget rule of behav-
novel problems during the transfer phase. Students ioral ecology predicts that organisms should shift
in the instructed-rules group and the changing-rules from being risk averse to being risk prone as budgets
group performed similarly. However, although stu- are depleted (e.g., Caraco, 1981). A review of the
dents in the instructed-rule and induced-rule groups budget rule and its relation to gambling may be
both improved over the course of the transfer phase, found in Fantino, Navarro, and O’Daly (2005).
those in the changing-rules condition did not. Here, we focus on some of the factors that appear to
Sasada (2003) used a similar design with fourth- maintain problem gambling.
grade students and verbal analogy problems and One potent factor is immediacy of reinforcement.
also found better transfer to problems solved by a Preference for variable over fixed delays with the
novel rule after induced-rule training than after same arithmetic means has been demonstrated in
instructed-rule training. Sasada found no difference countless studies with nonhumans but can be
between instructed- and induced-rule conditions in shown with humans as well (see Fantino, Navarro,
college-age participants. Overall, these results sug- & O’Daly, 2005, for a review). For example, con-
gest that rule-based problem solving (whether sider a pigeon choosing between a variable-interval
instructed or induced) need not be inflexible, but (VI) 15-second schedule and a fixed-interval (FI)
that induced rule learning may be especially effec- 15-second schedule as the outcomes in a concurrent-
tive in promoting flexible problem solving in chil- chains schedule (see Figure 19.3). The VI schedule
dren. The Kanevsky (2006) and Sasada studies consists of some interreinforcement intervals that
measured accuracy of subsequent problem solving are considerably longer or shorter than 15 seconds.
rather than efficiency (the measure used by Fantino,
Jaworski, et al., 2003). The effect of the dependent
measure and the possibility of developmental differ-
ences in the utility of instructed rules are subjects of
current investigation.
What Can Gambling Behavior Tell

Researchers About Nonoptimal
Decision Making?
In studies of the sunk-cost effect and base-rate
neglect described earlier, behavioral methodology
was applied to problems in decision making that
had not previously been viewed from a behavioral
perspective. The area of gambling is perhaps differ-
ent in the sense that the gambling context can be
seen as an obvious showcase for basic principles of Figure 19.3. Concurrent-chains schedule in
reinforcement including incentives, schedules of which the organism chooses between reinforce-
ment outcomes arranged by variable-interval
reinforcement, discriminative stimuli, conditioned (VI) and fixed-interval (FI) schedules with the
reinforcement, the partial reinforcement effect, and same mean interreinforcement interval.
454
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
If the occasional short interreinforcement intervals occur in association with substance abuse, nicotine
in the VI schedule have a disproportionate effect on dependence, and mood, anxiety, and personality
choice, then the pigeon should prefer the VI 15 sec- disorders (Petry, Stinson, & Grant, 2005).
onds even though the mean interreinforcement What kinds of discounting functions are
intervals on the two schedules are equal. Herrnstein observed in gamblers who do not meet the criteria
(1964) found just that: a strong preference for the for pathological gambling? In a study comparing
VI schedule. Researchers are finding the same effect gambling and nongambling college students, Holt,
with human participants (college students) choos- Green, and Myerson (2003) found no difference
ing between mixed- and fixed-ratio schedules of between the groups in the steepness of their tempo-
reinforcement (Meyer, Schley, & Fantino, 2011; see ral discounting functions. They did, however, find a
also Locey, Pietras, & Hackenberg, 2009). This difference between the groups in probability dis-
prospect of an immediate payoff is what may under- counting, with the gambling students discounting
lie much of the appeal of gambling (see Madden, unlikely rewards less steeply, thus showing less sen-
Ewan, & Lagorio, 2007). sitivity to risk. Similar differences in probability dis-
counting across people diagnosed with pathological
Does Lack of Self-Control Contribute to gambling and matched control participants have
Nonoptimal Decision Making? been reported by Madden, Petry, and Johnson
Many of society’s problems stem from a preoccupa- (2009). Holt et al. interpreted these results as evi-
tion with short-term gain. Large-scale problems dence against a simple view of impulsivity as a trait
are exacerbated by the tendency to place too encompassing both risk taking and the inability to
much weight on the immediate contingencies asso- delay gratification.
ciated with a societal challenge (be it mitigating Shead, Callan, and Hodgins (2008) also gave a
global warming and environmental pollution, probability-discounting task (in this case, with a real
improving infrastructure, or working to reorganize monetary incentive) to college students who had
the health care system) and give too little consider- varying degrees of gambling experience. They found
ation to the long-term contingencies, both positive no significant correlation between participants’
and negative. Likewise, people often act against degree of problem gambling severity and their proba-
their own best interests by making choices that bility discounting of either gains or losses, a result
serve immediate desires or convenience rather than consistent with their participants’ low to moderate
those that serve their future health and happiness. gambling experience. However, participants had also
Insofar as lack of self-control can be viewed as completed the Gambling Expectancy Questionnaire
underlying many nonadaptive behaviors, then dis- (Stewart & Wall, 2005) and had been categorized
counting may be seen as a mechanism by which as either relief expectancy gamblers (those who
impulsive behavior can be justified (Fantino & expected gambling to relieve negative mood), reward
Stolarz-Fantino, 2008). expectancy gamblers (those who expected gambling
One can argue that problem gambling results to enhance positive mood), or nonexpectancy gam-
from impulsiveness. If this were true, then problem blers (those who had neither expectation). Stewart
gamblers would be expected to display a tendency to and Wall (2005) found that reward expectancy gam-
discount delayed rewards more steeply; conversely, blers discounted probabilistic gains less steeply and
one would also expect them to show shallower dis- probabilistic losses more steeply than did partici-
counting for less probable rewards. Support for this pants in the other categories. Thus, there seems to be
idea comes from research showing that people with some evidence for a relationship between gambling
pathological gambling and other addictions do, in experience and probability discounting in people
fact, show steeper discounting functions than con- who do not display pathological gambling.
trol participants (e.g., Alessi & Petry, 2003; Bickel, With temporal discounting, the relationship with
Odum, & Madden, 1999; Dixon, Marley, & Jacobs, gambling may be more complex because the steeper
2003; Petry, 2001). Pathological gambling tends to temporal discounting found among some problem
455
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
gamblers may result from associated problems such

as substance abuse (Petry & Casarella, 1999). How-
ever, recent work by Weatherly, Marino, Ferraro,
and Slagle (2008) investigated the question of
whether participants’ rates of temporal discounting
are associated with their behavior in an actual gam-
bling task. Participants were given the equivalent of
$10 in tokens that they could use to play a slot
machine. Weatherly et al. found that temporal dis-
counting rates were significantly predictive of the
number of tokens inserted into the machine. In any
event, discounting continues to be an area of inter-
Figure 19.4. Concurrent-chains procedure in
est as investigators search for explanations of how which the organism chooses between reinforcement
gambling develops into a problem behavior. outcomes arranged by a fixed-ratio (FR) 10 on one
Little has been done to support the popular side and an outcome that has an equal probability
notion that individuals who are lucky enough to win of FR 10 and FR 90 on the other side. VI = variable
interval.
early gambles are more likely to develop into prob-
lem gamblers than those who are unlucky at the the simple FR 10. A second group chose between an
outset. If anything, Goltz’s (1992) research on per- FR 90 and the same mixed FR 10–FR 90. These
sistence of commitment suggested that gamblers pigeons chose the mixed schedule more often. A
should be more likely to continue in the face of third group had no pretraining. All three groups
intermittent wins. Kassinove and Schare (2001) were then exposed to the choice of FR 50 versus
found no effect of a big win (in this case, $10) on mixed FR 10–FR 90. These conditions remained in
resistance to extinction—that is, whether partici- effect until their preferences were stable. The results
pants persisted playing a slot machine simulation in are shown in Figure 19.5: The group that had cho-
the face of repeated no-win trials. Weatherly, Sauter, sen FR 10 most often in pretraining had the lowest
and King (2004) also studied the effect of a big win preference for the mixed schedule; the participants
on simulated slot machine play; the students in their chose it significantly less often than the participants
study were selected to have little or no gambling in the other two conditions. The group that had
experience. Weatherly et al. found no significant dif- chosen the mixed schedule more often in pretrain-
ference in resistance to extinction among students ing showed the highest preference for the mixed
who had a big win on the fifth trial, those who had a schedule, and preferences for control participants
small win on the 25th trial, and those who never
Phase 2: 10/90 Preference
had a winning trial. However, participants who had 0.8
a big win on the first trial persisted far less than 0.7 0.65
0.58
those who had a big win on the fifth trial, much as 0.6
0.49
0.5
would be expected on the basis of Goltz’s research.
0.4
Manipulating an individual’s early gambling his-
0.3
tory in a naturalistic setting is difficult. However, it 0.2
is very easy to do so for the pigeon. In our labora- 0.1
tory, we gave three groups of pigeons different his- 0
10 vs 10/90 90 vs 10/90 no training
torical gambling contexts (Fantino &
Phase 1: Training condition
Stolarz-Fantino, 2009). One group chose between
an FR 10 and a mixed-ratio schedule that was FR 10 Figure 19.5. Mean response proportion for the
on half the trials and FR 90 on the other half (pre- mixed (gambling) option as a function of a history
of preference for a fixed option (left bar) or a mixed
sented as the outcomes on concurrent chains; Figure option (center bar) and with no relevant history (right
19.4). Predictably, these pigeons more often chose bar).
456
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
were intermediate. Thus, it appears, at least for in economic distribution games may display greater
pigeons in this task, that historical context does optimality when real money is at stake.
make a difference in future preference for risky Fourth, rule-governed responding is acquired
outcomes. more quickly than contingency-shaped responding
but is less sensitive to changes in contingencies. Thus,
rules can help or hinder correct responding, depend-
Summary and Conclusions: Factors
ing on their applicability to the problem at hand.
Promoting Logical and Illogical
Finally, some nonoptimal behavior results from
Reasoning
generalization of responses that are effective in other
As illustrated in the examples discussed previously, contexts. Thus, for example, a history of matching
it is not the case that human reasoning is invariably led to base-rate neglect by human participants in
logical—or illogical—even on different examples of Goodie and Fantino’s (1995, 1996, 1999) studies,
the same task. However, we can make five general- and pigeons given a similar history by Fantino,
izations that may help to reconcile some seemingly Kanevsky, and Charlton (2005) behaved in the same
contradictory findings. way. Similarly, misapplication of an averaging rule
First, suboptimal responding can be lessened by can lead to apparently illogical responding in judg-
making task contingencies more transparent, as ing the likelihood of conjunctions.
Fantino and Esfandiari (2002) found when they The responses of human participants on reason-
informed some participants of the probabilities of ing tasks may seem irrational, especially in instances
reinforcement associated with each stimulus in their in which nonhumans respond more optimally. How-
investigation of probability matching. Similarly, ever, application of behavioral methods, including
Navarro and Fantino (2005, 2007) found that both the study of verbal and rule-governed behavior, has
pigeons and human participants learned to avoid the great potential to help illuminate some of the puz-
sunk-cost effect when distinctive stimuli allowed zling findings of research in human reasoning.
them to discriminate the point at which they should
escape a situation in which the prospects for rein-
References
forcement were becoming unfavorable.
Alessi, S. M., & Petry, N. M. (2003). Pathological gam-
Second, feedback as to the correctness or accu-
bling severity is associated with impulsivity in a
racy of responses appears to be helpful in improving delay discounting procedure. Behavioural Processes,
performance on problem-solving tasks. However, it 64, 345–354. doi:10.1016/S0376-6357(03)00150-5
needs to be reasonably specific to be effective. For Anderson, N. H. (1965). Averaging versus adding as a
example, Fantino, Stolarz-Fantino, and Navarro stimulus-combination rule in impression formation.
(2003) found that feedback was useful in helping Journal of Experimental Psychology, 70, 394–400.
doi:10.1037/h0022280
participants avoid the conjunction effect; this feed-
Anderson, N. H. (1981). Foundations of information inte-
back was more specific than that used by Stolarz-
gration theory. New York, NY: Academic Press.
Fantino et al. (2003) in their (unsuccessful) attempt
Arkes, H. R., & Ayton, P. (1999). The sunk cost and
to reinforce correct responding with financial Concorde effects: Are humans less rational than
rewards. For feedback to work, participants must lower animals? Psychological Bulletin, 125, 591–600.
understand the task requirements well enough to doi:10.1037/0033-2909.125.5.591
make use of it. Barash, D. P. (2003). The survival game. New York, NY:
Third, financial incentives are evidently not Times Books.
necessary or sufficient for making participants’ rea- Baron, A., & Galizio, M. (1983). Instructional control of
soning more logical, although—as discussed by human operant behavior. Psychological Record, 33,
495–520.
Camerer and Hogarth (1999)—they may help per-
formance of easy tasks that are responsive to effort. Bentall, R. P., & Lowe, C. F. (1987). The role of verbal
behavior in human learning: III. Instructional effects
However, as observed by Fantino and Kennelly in children. Journal of the Experimental Analysis of
(2009), and widely believed by economists, behavior Behavior, 47, 177–190. doi:10.1901/jeab.1987.47-177
457
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Bickel, W. K., Odum, A. L., & Madden, G. J. (1999). (pp. 249–267). Cambridge, England: Cambridge
Impulsivity and cigarette smoking: Delay dis- University Press.
counting in current, never, and ex-smokers. Escobar, R., & Bruner, C. A. (2009). Observing responses
Psychopharmacology, 146, 447–454. doi:10.1007/ and serial stimuli: Searching for the reinforcing prop-
PL00005490 erties of the S−. Journal of the Experimental Analysis
Birnbaum, M. (1983). Base rates in Bayesian infer- of Behavior, 92, 215–231. doi:10.1901/jeab.2009.92-
ence: Signal detection analysis of the cab prob- 215
lem. American Journal of Psychology, 96, 85–94. Fantino, E., & Esfandiari, A. (2002). Probability match-
doi:10.2307/1422211 ing: Encouraging optimal responding in humans.
Calne, D. B. (1999). Within reason: Rationality and human Canadian Journal of Experimental Psychology/Revue
behavior. New York, NY: Pantheon Books. Canadienne de Psychologie Expérimentale, 56, 58–63.
Camerer, C. F. (2003). Behavioral game theory: doi:10.1037/h0087385
Experiments in strategic interaction. New York, NY: Fantino, E., Gaitan, S., Kennelly, A., & Stolarz-Fantino,
Russell Sage Foundation. S. (2007). How reinforcer type affects choice in eco-
Camerer, C. F., & Hogarth, R. M. (1999). The nomic games. Behavioural Processes, 75, 107–114.
effects of financial incentives in experiments: doi:10.1016/j.beproc.2007.02.001
A review and capital-labor-production frame- Fantino, E., Jaworski, B. A., Case, D. A., & Stolarz-
work. Journal of Risk and Uncertainty, 19, 7–42. Fantino, S. (2003). Rules and problem solving:
doi:10.1023/A:1007850605129 Another look. American Journal of Psychology, 116,
Caraco, T. (1981). Energy budgets, risk and foraging 613–632. doi:10.2307/1423662
preferences in dark-eyed juncos (Junco hyemalis). Fantino, E., Kanevsky, I. G., & Charlton, S. (2005).
Behavioral Ecology and Sociobiology, 12, 213–217. Teaching pigeons to commit base-rate neglect.
doi:10.1007/BF00299833 Psychological Science, 16, 820–825. doi:10.1111/
Case, D. A., & Fantino, E. (1989). Instructions and j.1467-9280.2005.01620.x
reinforcement in the observing behavior of adults Fantino, E., & Kennelly, A. (2009). Sharing the wealth:
and children. Learning and Motivation, 20, 373–412. Factors influencing resource allocation in the shar-
doi:10.1016/0023-9690(89)90003-9 ing game. Journal of the Experimental Analysis of
Case, D. A., Fantino, E., & Goodie, A. S. (1999). Base- Behavior, 91, 337–354. doi:10.1901/jeab.2009.91-337
rate training without case cues reduces base-rate Fantino, E., Kulik, J., Stolarz-Fantino, S., & Wright, W.
neglect. Psychonomic Bulletin and Review, 6, 319–327. (1997). The conjunction fallacy: A test of averag-
doi:10.3758/BF03212337 ing hypotheses. Psychonomic Bulletin and Review, 4,
Case, D. A., Ploog, B. O., & Fantino, E. (1990). 96–101. doi:10.3758/BF03210779
Observing behavior in a computer game. Journal of Fantino, E., Navarro, A., & O’Daly, M. (2005). The sci-
the Experimental Analysis of Behavior, 54, 185–199. ence of decision-making: Behaviours related to gam-
doi:10.1901/jeab.1990.54-185 bling. International Gambling Studies, 5, 169–186.
Catania, A. C., Shimoff, E., & Matthews, B. A. (1989). An doi:10.1080/14459790500303311
experimental analysis of rule-governed behavior. In Fantino, E., & Savastano, H. (1996). Humans’ responses
S. C. Hayes (Ed.), Rule-governed behavior: Cognitions, to novel stimulus compounds and the effects of train-
contingencies, and instructional control (pp. 119–150). ing. Psychonomic Bulletin and Review, 3, 204–207.
New York, NY: Plenum Press. doi:10.3758/BF03212419
Colman, A. M. (2009). Oxford dictionary of psychology Fantino, E., & Silberberg, A. (2010). Revisiting the role of
(3rd ed.). Oxford, England: Oxford University Press. bad news in maintaining human observing behavior.
de la Piedad, X., Field, D., & Rachlin, H. (2006). The Journal of the Experimental Analysis of Behavior, 93,
influence of prior choices on current choice. Journal 157–170. doi:10.1901/jeab.2010.93-157
of the Experimental Analysis of Behavior, 85, 3–21. Fantino, E., & Stolarz-Fantino, S. (2001). Behavioral and
doi:10.1901/jeab.2006.132-04 economic approach to decision making: A common
Dixon, M. R., Marley, J., & Jacobs, E. A. (2003). Delay ground. Behavioral and Brain Sciences, 24, 407–408.
discounting by pathological gamblers. Journal of Fantino, E., & Stolarz-Fantino, S. (2002). The role of
Applied Behavior Analysis, 36, 449–458. doi:10.1901/ negative reinforcement; or: Is there an altruist in the
jaba.2003.36-449 house? Behavioral and Brain Sciences, 25, 257–258.
Eddy, D. M. (1982). Probabilistic reasoning in clini- doi:10.1017/S0140525X02290056
cal medicine: Problems and opportunities. In Fantino, E., & Stolarz-Fantino, S. (2008). Gambling:
D. Kahneman, P. Slovic, & A. Tversky (Eds.), Sometimes unseemly, not what it seems. Analysis of
Judgment under uncertainty: Heuristics and biases Gambling Behavior, 2, 61–68.
458
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Fantino, E., & Stolarz-Fantino, S. (2009, May). Principles Hackenberg, T. D., & Joker, V. R. (1994). Instructional
of choice and their applications. Paper presented at versus schedule control of humans’ choices in
the 35th Annual Convention of the Association for situations of diminishing returns. Journal of the
Behavior Analysis International, Phoenix, AZ. Experimental Analysis of Behavior, 62, 367–383.
doi:10.1901/jeab.1994.62-367
Fantino, E., & Stolarz-Fantino, S. (2010). Grandparental
altruism: Expanding the sense of cause and effect. Hartl, J., & Fantino, E. (1996). Choice as a function of
Behavioral and Brain Sciences, 33, 22–23. reinforcement ratios in delayed matching to sample.
Fantino, E., Stolarz-Fantino, S., & Navarro, A. (2003).
11–27. doi:10.1901/jeab.1996.66-11
Logical fallacies: A behavioral approach to reasoning.
Behavior Analyst Today, 4, 109–117. Herrnstein, R. J. (1964). Aperiodicity as a factor in
choice. Journal of the Experimental Analysis of
Fiedler, K. (2000). Beware of samples! A cognitive–
ecological sampling approach to judgment
biases. Psychological Review, 107, 659–676. Hertwig, R., & Gigerenzer, G. (1999). The “conjunction
doi:10.1037/0033-295X.107.4.659 fallacy” revisited: How intelligent inferences look
like reasoning errors. Journal of Behavioral Decision
Forsythe, R., Horowitz, J. L., Savin, N. E., & Sefton,
Making, 12, 275–305. doi:10.1002/(SICI)1099-
M. (1994). Fairness in simple bargaining experi-
0771(199912)12:4<275::AID-BDM323>3.0.CO;2-M
ments. Games and Economic Behavior, 6, 347–369.
doi:10.1006/game.1994.1021 Hertwig, R., & Ortmann, A. (2001). Experimental prac-
tices in economics: A methodological challenge for
Gigerenzer, G., Hoffrage, U., & Ebert, A. (1998). AIDS
psychologists? Behavioral and Brain Sciences, 24,
counseling for low-risk clients. AIDS Care, 10,
383–403.
197–211. doi:10.1080/09540129850124451
Hineline, P., & Wanchisen, B. (1989). Correlated hypoth-
Gilovich, T. (1991). How we know what isn’t so: The falli-
esizing and the distinction between contingency-
bility of human reason in everyday life. New York, NY:
shaped and rule-governed behavior. In S. C. Hayes
Free Press.
(Ed.), Rule-governed behavior: Cognition, contingen-
Goltz, S. M. (1992). A sequential learning analysis of cies, and instructional control (pp. 221–268). New
decisions in organizations to escalate investments York, NY: Plenum Press.
despite continuing costs or losses. Journal of Applied
Hoffman, E., McCabe, K. A., & Smith, V. L. (1998).
Behavior Analysis, 25, 561–574. doi:10.1901/
Behavioral foundations of reciprocity: Experimental
jaba.1992.25-561
economics and evolutionary psychology. Economic
Goltz, S. M. (1999). Can’t stop on a dime: The roles Inquiry, 36, 335–352. doi:10.1111/j.1465-7295.1998.
of matching and momentum in persistence of tb01719.x
commitment. Journal of Organizational Behavior
Holt, D. D., Green, L., & Myerson, J. (2003). Is discount-
Management, 19, 37–63. doi:10.1300/J075v19n01_05
ing impulsive? Evidence from temporal and prob-
Goodie, A. S. (2001). Are scripts or deception necessary ability discounting in gambling and non-gambling
when repeated trials are used? On the social context college students. Behavioural Processes, 64, 355–367.
of psychological experiments. Behavioral and Brain doi:10.1016/S0376-6357(03)00141-4
Sciences, 24, 412.
Horne, P. J., & Lowe, C. F. (1993). Determinants of
Goodie, A. S., & Fantino, E. (1995). An experientially human performance on concurrent schedules.
derived base-rate error in humans. Psychological Journal of the Experimental Analysis of Behavior, 59,
Science, 6, 101–106. doi:10.1111/j.1467-9280.1995. 29–60. doi:10.1901/jeab.1993.59-29
tb00314.x
Huber, L. (2009). Degrees of rationality in human and
Goodie, A. S., & Fantino, E. (1996). Learning to commit non-human animals. In S. Watanabe, A. P. Blaisdell,
or avoid the base-rate error. Nature, 380, 247–249. L. Huber, & A. Young (Eds.), Rational animals,
doi:10.1038/380247a0 irrational humans (pp. 3–21). Tokyo, Japan: Keio
University Press.
Goodie, A. S., & Fantino, E. (1999). What does
and does not alleviate base-rate neglect under Humphreys, L. G. (1939). Acquisition and extinction
direct experience. Journal of Behavioral Decision of verbal expectations in a situation analogous to
Making, 12, 307–335. doi:10.1002/(SICI)1099- conditioning. Journal of Experimental Psychology, 25,
0771(199912)12:4<307::AID-BDM324>3.0.CO;2-H 294–301. doi:10.1037/h0053555
Güth, W., Schmittberger, R., & Schwarze, B. (1982). An Kanevsky, I. G. (2006). Role of rules in transfer of math-
experimental analysis of ultimatum games. Journal ematical word problems. Dissertation Abstracts
of Economic Behavior and Organization, 3, 367–388. International: Section B. Sciences and Engineering,
doi:10.1016/0167-2681(82)90011-7 67(6), 3491.
459
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Kassinove, J. I., & Schare, M. (2001). Effects of the Myers, J. L. (1976). Probability learning and sequence
“near miss” and the “big win” on persistence at slot learning. In W. K. Estes (Ed.), Handbook of learn-
machine gambling. Psychology of Addictive Behaviors, ing and cognitive processes (Vol. 3, pp. 171–205).
15, 155–158. doi:10.1037/0893-164X.15.2.155 Hillsdale, NJ: Erlbaum.
Kennedy, M. L., Willis, W. G., & Faust, D. (1997). Navarro, A. D., & Fantino, E. (2005). The sunk
The base-rate fallacy in school psychology. Journal cost effect in pigeons and humans. Journal of
of Psychoeducational Assessment, 15, 292–307. the Experimental Analysis of Behavior, 83, 1–13.
doi:10.1177/073428299701500401 doi:10.1901/jeab.2005.21-04
Kennelly, A., & Fantino, E. (2007). The sharing game: Navarro, A. D., & Fantino, E. (2007). The role of dis-
Fairness in resource allocation as a function of criminative stimuli in the sunk cost effect. Revista
incentive, gender, and recipient types. Judgment and Mexicana de Análisis de la Conducta, 33, 19–29.
Decision Making, 2, 204–216.
Petry, N. M. (2001). Pathological gamblers, with and
Kudadjie-Gyamfi, E., & Rachlin, H. (2002). Rule- without substance use disorders, discount delayed
governed versus contingency-governed behavior in a rewards at high rates. Journal of Abnormal Psychology,
self-control task: Effects of changes in contingencies. 110, 482–487. doi:10.1037/0021-843X.110.3.482
Behavioural Processes, 57, 29–35. doi:10.1016/S0376-
6357(01)00205-4 Petry, N. M., & Casarella, T. (1999). Excessive discount-
ing of delayed rewards in substance abusers with
Locey, M. L., Pietras, C. J., & Hackenberg, T. D. (2009). gambling problems. Drug and Alcohol Dependence, 56,
Human risky choice: Delay sensitivity depends on 25–32. doi:10.1016/S0376-8716(99)00010-1
reinforcer type. Journal of Experimental Psychology:
Animal Behavior Processes, 35, 15–22. doi:10.1037/ Petry, N. M., Stinson, F. S., & Grant, B. F. (2005).
a0012378 Comorbidity of DSM-IV pathological gambling
and other psychiatric disorders: Results from the
Luchins, A. S. (1942). Mechanization in problem-solving: National Epidemiologic Survey on Alcohol and
The effect of Einstellung. Psychological Monographs, Related Conditions. Journal of Clinical Psychiatry, 66,
54(6, Whole No. 248). 564–574. doi:10.4088/JCP.v66n0504
Luchins, A. S., & Luchins, E. J. (1950). New experimen- Rachlin, H. (1997). Self and self control. Annals of the
tal attempts at preventing mechanization in problem New York Academy of Sciences, 818, 85–97.
solving. Journal of General Psychology, 42, 279–297.
doi:10.1080/00221309.1950.9920160 Rachlin, H. (2000). The science of self-control. Cambridge,
MA: Harvard University Press.
Madden, G. J., Ewan, E. E., & Lagorio, C. H. (2007).
Toward an animal model of gambling: Delay dis- Rachlin, H., Brown, J., & Cross, D. (2000).
counting and the allure of unpredictable outcomes. Discounting in judgments of delay and
Journal of Gambling Studies, 23, 63–83. doi:10.1007/ probability. Journal of Behavioral Decision
s10899-006-9041-5 Making, 13, 145–159. doi:10.1002/
(SICI)1099-0771(200004/06)13:2<145::AID-
Madden, G. J., Petry, N. M., & Johnson, P. S. (2009). BDM320>3.0.CO;2-4
Pathological gamblers discount probabilistic rewards
less steeply than matched controls. Experimental Rosenfarb, I. S., Newland, M. C., Brannon, S. E., &
and Clinical Psychopharmacology, 17, 283–290. Howey, D. S. (1992). Effects of self-generated rules
doi:10.1037/a0016806 on the development of scheduled-controlled behav-
ior. Journal of the Experimental Analysis of Behavior,
Meyer, S. F., Schley, D. R., & Fantino, E. (2011). The role 58, 107–121. doi:10.1901/jeab.1992.58-107
of context in risky choice. Behavioural Processes, 87,
100–105. Saramago, J. (1997). Blindness. New York, NY: Harcourt
Brace.
Michael, R. L., & Bernstein, D. J. (1991). Transient
effects of acquisition history on generalization in a Sasada, K. (2003). The effects of different instructions on
matching-to-sample task. Journal of the Experimental learning and transfer with verbal analogy problems.
Analysis of Behavior, 56, 155–166. doi:10.1901/ Unpublished master’s thesis, University of California,
jeab.1991.56-155 San Diego.
Mobbs, D., Hassabis, D., Seymour, B., Marchant, J. L., Shead, N. W., Callan, M. J., & Hodgins, D. C. (2008).
Weiskopf, N., Dolan, R. J., & Frith, C. D. (2009). Probability discounting among gamblers:
Choking on the money: Reward-based performance Differences across problem gambling severity and
decrements are associated with midbrain activity. affect-regulation expectancies. Personality and
Psychological Science, 20, 955–962. doi:10.1111/ Individual Differences, 45, 536–541. doi:10.1016/j.
j.1467-9280.2009.02399.x paid.2008.06.008
460
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Shimoff, E., Catania, A. C., & Matthews, B. A. (1981). Tversky (Eds.), Judgment under uncertainty: Heuristics
Uninstructed human responding: Sensitivity of low- and biases (pp. 153–160). Cambridge, England:
rate performance to schedule contingencies. Journal Cambridge University Press.
Tversky, A., & Kahneman, D. (1983). Extensional versus
doi:10.1901/jeab.1981.36-207
intuitive reasoning: The conjunction fallacy in prob-
Skinner, B. F. (1969). Contingencies of reinforcement: ability judgment. Psychological Review, 90, 293–315.
A theoretical analysis. New York, NY: Appleton- doi:10.1037/0033-295X.90.4.293
Century-Crofts.
Vaughan, M. E. (1985). Repeated acquisition in the
Stewart, S. H., & Wall, A. (2005). Ontario Problem analysis of rule-governed behavior. Journal of the
Gambling Research Centre final report: Mood priming Experimental Analysis of Behavior, 44, 175–184.
of reward and relief gambling expectancies in differ- doi:10.1901/jeab.1985.44-175
ent subtypes of gamblers. Guelph, Ontario, Canada:
Weatherly, J., Marino, J., Ferraro, F. R., & Slagle, B.
Ontario Problem Gambling Research Centre.
(2008). Temporal discounting predicts how people
Stolarz-Fantino, S., & Fantino, E. (1990). Cognition and gamble on a slot machine. Analysis of Gambling
behavior analysis: A review of Rachlin’s Judgment, Behavior, 2, 135–141.
decision, and choice. Journal of the Experimental
Weatherly, J. N., Sauter, J. M., & King, B. M. (2004).
The “big win” and resistance to extinction when
jeab.1990.54-317
gambling. Journal of Psychology, 138, 495–504.
Stolarz-Fantino, S., & Fantino, E. (1995). The experi- doi:10.3200/JRLP.138.6.495-504
mental analysis of reasoning: A review of Gilovich’s
Wyckoff, L. B., Jr. (1952). The role of observing
“How we know what isn’t so.” Journal of the
responses in discrimination learning: Part I.
doi:10.1901/jeab.1995.64-111
h0053932
Stolarz-Fantino, S., Fantino, E., & Kulik, J. (1996). The
Yerkes, R. M., & Dodson, J. D. (1908). The relation of
conjunction fallacy: Differential incidence as a func-
strength of stimulus to rapidity of habit-formation.
tion of descriptive frames and educational context.
Journal of Comparative Neurology and Psychology, 18,
Contemporary Educational Psychology, 21, 208–218.
459–482. doi:10.1002/cne.920180503
doi:10.1006/ceps.1996.0017
Zizzo, D. J. (2001). Choices between simple and com-
Stolarz-Fantino, S., Fantino, E., & Van Borst, N. (2006).
pound lotteries: Experimental evidence and neural
Use of base rates and case cue information in mak- network modelling (Department of Economics
ing likelihood estimates. Memory and Cognition, 34, Discussion Paper 57). Oxford, England: University
603–618. doi:10.3758/BF03193583 of Oxford.
Stolarz-Fantino, S., Fantino, E., Zizzo, D. J., & Wen, J. Zizzo, D. J., Stolarz-Fantino, S., Wen, J., & Fantino,
(2003). The conjunction effect: New evidence for E. (2000). A violation of the monotonicity
robustness. American Journal of Psychology, 116, axiom: Experimental evidence on the conjunc-
15–34. doi:10.2307/1423333 tion fallacy. Journal of Economic Behavior and
Tversky, A., & Kahneman, D. (1982). Evidential impact Organization, 41, 263–276. doi:10.1016/S0167-
of base rates. In D. Kahneman, P. Slovic, & A. 2681(99)00076-1
461
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 20
Self-Control and Altruism

Matthew L. Locey, Bryan A. Jones, and Howard Rachlin
Self-control and altruism are both dimensions of opposing account. Nevertheless, we show that the
choice. When a person chooses a distant or tempo- two contexts have different (if not incompatible)
rally extended reward such as good health over a implications.
relatively closer and temporally constricted reward From the economic view, both self-control and
such as smoking a cigarette, the person’s act is said altruism may be understood in terms of discount
to be self-controlled rather than impulsive. Corre- functions—reward value as a function of the delay
spondingly, when a person chooses a reward for to its receipt or as a function of social distance to the
another person or a group of other people at a cost receiver—and their interaction. We summarize
to him- or herself, as when giving anonymously to research on delay and social discount functions, use
charity, the person is said to be behaving altruisti- discount functions to describe self-control and altru-
cally rather than selfishly. Our main purpose in this ism in the laboratory, and relate those studies to
chapter is to compare these two dimensions of self-control and altruism in people’s everyday lives.
behavior—to show how they are and are not analo- Discount functions (their interaction and their inte-
gous. We make this comparison in two conceptual gration), however, are not sufficient to account for
contexts: the economic view and teleological behav- many complex cases of self-control and altruism.
iorism. Actually, both contexts are economic, and Such cases are better understood from a teleological
both are teleological. They are economic in the viewpoint—as patterns of behavior in which the
sense that they both rely on the principle of maximi- value of the pattern may be greater (or less) than the
zation of value; they are both teleological in the sum of the values of its component acts.
sense that they both focus on choice as a function of
outcome—on final rather than efficient causes. The
Economic View
main difference between the two contexts is that
from the economic view, both behavior and out- According to the economic view of choice, an organ-
come are individual discrete events, whereas from ism’s preferences may be expressed in terms of a
the view of teleological behaviorism, both behavior utility function. Utility functions specify the value of
and outcome are patterns of events. Given that any choice alternatives in terms of their amounts, proba-
act, no matter how brief (even a pigeon’s peck), may bilities, and delays. From the economic viewpoint,
be subdivided into subacts and that any pattern, no organisms faced with constraints (contingencies)
matter how extended, may be taken as a unit, the that limit available alternatives maximize utility by
difference is more a matter of perspective than of choosing the highest utility alternative. If the utility
essence. Teleological behaviorism is thus an exten- function and the constraints are known, it should
sion of the basic economic view rather than an be possible—according to the economic view—to
This article was prepared with the assistance of National Institute on Drug Abuse Grant DA02652021.
DOI: 10.1037/13937-020
463
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Locey, Jones, and Rachlin
predict which alternative will be chosen (Kagel, simplicity that V = number of pellets obtained, the
Battalio, & Green, 1995; Rachlin, 1989). According time taken to consume the pellets may be ignored,
to this view, an alternative consists not of a single and kdelay = s = 1.0), the value of the 4p–10s–2p
isolated event but of a series of events strung out in sequence would be [4/(1 + 0) + 2/(1 + 10)] or 4.18
time, each with its own amount, probability, and units; the value of the 3p–10s–8p sequence would
delay. A person with alcoholism deciding whether to be [3/(1 + 0) + 8/(1 + 10)] or 3.73 units. The
drink a glass of scotch considers not just the imme- pigeon is predicted to choose the 4p–10s–2p alter-
diate consequences but also tomorrow’s hangover; native totaling six pellets and forgo the 3p–10s–8p
the effect that drinking this particular scotch will alternative totaling 11 pellets. According to Equa-
have on the tendency to drink another one and tion 1, which roughly predicts choice in such situa-
another one; the enjoyment obtained from those tions, the initial one-pellet difference in reward
future drinks; the series of hangovers they entail; amount overwhelms a six-pellet difference 10 sec-
their effects on health, social acceptance, and work onds later. Although humans are apparently much
performance; and so on for the remainder of the per- less myopic than pigeons (delaying rewards for days,
son’s life (Becker & Murphy, 1988). Fortunately for weeks, and years), when humans are thirsty or hun-
the economist trying to explain choice (if not for the gry and choosing among water or food rewards (as
person with alcoholism), delayed events are reduced pigeon subjects usually are), their myopia (as mea-
in effective amount (discounted) often to the degree sured by the obtained kdelay in Equation 1) is compa-
that in practice only the initial consequences of a rable to that of pigeons (Forzano & Logue, 1994).
given choice (only the initial events within each
alternative) need be considered. That is, delayed Hyperbolic Delay Discounting by Humans
rewards are discounted—the greater the delay, the Rachlin, Raineri, and Cross (1991) obtained delay
greater the discount. For example, a pigeon will discount functions with humans by asking them to
typically choose the sequence of four food pellets, make a series of choices among hypothetical
10-second delay, two food pellets (4p–10s–2p) over rewards of different delays and amounts of money.
the sequence of three food pellets, 10-second delay, For example, Rachlin et al. asked, “Which would
and eight food pellets (3p–10s–8p). The initial you prefer, A) $1,000 now or B) $1,000 one year
rewards count for much more than the delayed ones. from now?” Of course, everyone preferred A. Then
The delay discounting of both humans and they asked, “Which would you prefer, A) $950 now
pigeons may be expressed in terms of this hyper- or B) $1,000 one year from now?” If the person still
bolic discount equation (Mazur, 1987): preferred alternative A, they asked, “Which would
you prefer, A) $900 now or B) $1,000 one year from
V , now?” and so on, going down in steps all the way to
 delay = (1)
1 + kdelay D s “Which would you prefer, A) $0 now or B) $1,000
one year from now?” At this point, people preferred
where V is the value of the reward if it were obtained B. The crucial datum in this procedure was the point
immediately, D is the delay of the reward, kdelay is a at which the person crossed over from preference
constant measuring the degree of delay discounting for A to preference for B. Rachlin et al. assumed that
(the greater kdelay is, the more a given delayed reward the person was indifferent between the crossover
is discounted, i.e., the less it is worth at the present amount now and $1,000 1 year from now. For
moment), and s is a constant measuring subjective example, if a person preferred $800 now to $1,000 a
delay (Green & Myerson, 2004). If an alternative year from now but $1,000 a year from now to $750
consists of more than one reward in sequence, then now, the crossover point was $775—indifference
vdelay is the sum of the values of each reward in the between $775 now and $1,000 a year from now.
sequence discounted by its delay from the present Rachlin et al. then repeated the procedure with a
moment (Mazur, 1986). For the pigeon’s alterna- series of delays ranging from 1 month through 50
tives in the preceding paragraph (assuming for years. In terms of Equation 1, V = $1,000, D = the
464
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
1000
tested delay, vdelay = the obtained crossover point,
and kdelay and s were constants to be estimated from
800
the data. Figure 20.1 shows a delay discount func-
tion obtained by Rachlin et al. (1991) using this
600
$ Amount
method.1 The points are median crossover points (of
40 Stony Brook University undergraduates) at each 400
delay. As one would expect, the longer the delay, the
lower the crossover point. The fit of the line to the 200
points is, as you can see, very good.2
Most laboratory studies of human delay dis- 0
0 100 200 300 400 500 600
counting have used hypothetical monetary rewards
Delay (Months)
(Green & Myerson, 2004), but the results with
small amounts of real cash have been similar to Figure 20.1. Delay discounting: The median
those obtained with equally small hypothetical amount of immediate money equivalent to $1,000 after
rewards (Madden et al., 2003). This is not to say various delays. The solid line is the best-fitting version
of Equation 1 (kdelay = 0.02, s = 0.9, R2 = .996). From
that type or amount of reward makes no difference “Subjective Probability and Delay,” by H. Rachlin, A.
in delay discounting. A weak currency (the Polish Raineri, and D. Cross, 1991, Journal of the Experimental
zloty) was discounted more steeply than a relatively Analysis of Behavior, 55, p. 240. Copyright 1991 by the
Society for the Experimental Analysis of Behavior, Inc.
strong currency (the U.S. dollar; Ostaszewski,
Adapted with permission.
Green, & Myerson, 1998). With noncash rewards,
luxuries were discounted more steeply than necessi- amount effect has not generally been found with
ties (Chapman & Elstein, 1995; Raineri & Rachlin, nonhuman subjects (Green, Myerson, Holt, Slevin,
1993). Large monetary amounts or highly valued & Estle, 2004).
commodities (large V values) are discounted rela-
tively less steeply than small amounts of money or Self-Control in the Economic View
less valued commodities (Green, Myerson, & With such strong preferences for a slightly larger
McFadden, 1997; Kirby, 1997; Raineri & Rachlin, immediate reward and such weak preferences for a
1993); for example, $100 delayed by a year is equiv- much larger delayed reward, how is self-control pos-
alent on average to $58 now (a discount of 42%), sible in the everyday lives of humans and nonhu-
but $10,000 delayed by a year is equivalent to mans? People (and even pigeons) do often choose
$7,500 now (a discount of 25%), and $1 million larger–later over smaller–sooner rewards. Colleges
delayed by a year is equivalent to $900,000 now (a are full of students even though the rewards of
discount of only 10%). This decrease in relative obtaining a college education are delayed by years
delay discounting with increase in amount is called (assuming that going to college is not governed
the amount effect. This effect is highly reliable and is solely by love of knowledge); people do walk past
found with hypothetical money and hypothetical bakeries although tempted to walk in; and people
commodities as well as with small amounts of real with alcoholism do refuse a drink. Given the eco-
money (Kirby, 1997). However, a delay-discounting nomic model, how is such behavior possible?
Although there are problems with hypothetical rewards in the laboratory, real rewards create their own set of problems. With hypothetical rewards,
1
participants are asked to imagine a real-world context and to choose within that context. Real monetary rewards obtained in the laboratory may
impose a context of narrow maximization of monetary amount that may differ from the context, implied by the instructions, of primary interest to
the experimenter. A participant in the laboratory, trying to imagine what she or he would do in a hypothetical real-world situation, may come closer
to that situation than one simply trying to maximize monetary reward. Moreover, comparisons between delay discounting of real and hypothetical
rewards have generally failed to find a difference in either degree or functional form (Kirby, 1997; Madden, Begotka, Raiff, & Kastern, 2003). A com-
parison of real and hypothetical rewards in social discounting also failed to find any significant differences (Locey, Jones, & Rachlin, 2011).
The function drawn in Figure 20.1 is fitted to the median crossover points of the group. However, in all discounting studies with humans reported in
2
this chapter, discount functions were also obtained for each participant individually. In all cases, the median R2 for the individual participant fits was
greater than .90 and the medians of the parameters of the individual functions closely matched the parameters of the median functions (such as those
in Figure 20.1). This concordance suggests that the results are not artifacts of averaging.
465
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
According to the economic view, the answer is by 60 seconds), the pigeon could commit itself to the
prior commitment (Ainslie, 1992). sequence it then prefers (3p–10s–8p). At that point,
According to Equation 1 (assuming again that suppose the pigeon is given the opportunity (by
kdelay = s = 1.0), pigeons prefer the sequence with pecking a third key) to prevent the 4p–10s–2p
the higher immediate reward and the lower overall sequence from being offered later, after the 60-second
reward (4p–10s–2p) to the sequence with the higher delay has elapsed. Research has shown that pigeons
overall reward and the lower immediate reward tend to take this kind of option, thus committing to
(3p–10s–8p). Equation 1, however, predicts that if a the larger overall reward (Rachlin & Green, 1972).
long-enough common delay is added before both Procedures such as this, in which an organism may
sequences, preference will reverse. For example, sup- choose to completely avoid a later alternative, use
pose a 60-second delay is added before both the the strict commitment method.
4p–10s–2p and 3p–10s–8p sequences. Recalculating Another method of commitment studied with
with the extra delay added in, the value of the 60s– pigeons is called punishment commitment (Green &
4p–10s–2p sequence becomes [4/(1 + 60) + 2/(1 + Rachlin, 1996). At the beginning of the common
70)], or 0.10 units, and the value of the 60s–3p–10s– delay, at the point at which the pigeon prefers the
8p sequence becomes [3/(1 + 60) + 8/(1 + 70)], or larger overall reward sequence, the pigeon is offered
0.16. The initial 60-second delay greatly reduced the the option of attaching punishment to the more
value of both sequences, but the relative values of the tempting but smaller overall reward sequence. If the
two have reversed—from a slight (53%) preference pigeon does choose the punishment option, it will
for 4p–10s–2p over 3p–10s–8p to a stronger (62%) not be strictly prevented from choosing the tempting
preference for 60s–3p–10s–8p over 60s–4p–10s– sequence later, but if it does choose and obtain that
2p—and become close to a preference based on the sequence (4p–10s–2p in our example), it will be pun-
overall amount of food, [11/(6 + 11)], or 65%, in ished afterward (by a long blackout in which all
favor of the larger overall reward. Such reversals do lights in the chamber are turned off and no rewards
occur. They are powerful evidence for the hyperbolic are obtainable). The addition of punishment to the
form of Equation 1 as opposed to the exponential tempting sequence makes it much less tempting.
form (v = V/ekD), which predicts no such reversals With the punishment in place, pigeons will usually
(Ainslie, 1992; Rachlin, 2006). choose and obtain the larger overall reward. The
Now suppose a pigeon starts with a 60-second pigeon is essentially in the same position as the per-
delay and time passes so that the common delay son with alcoholism offered the option of taking the
before the two reward sequences diminishes with drug Antabuse before attending a party. If the person
time. At the beginning, with the full 60-second delay takes the drug and then has an alcoholic drink, a very
still to come, the pigeon prefers 60s–3p–10s–8p to painful reaction ensues. Unlike people with alcohol-
60s–4p–10s–2p; however, 60 seconds later, if ism, who typically refuse to take the drug, pigeons,
offered a new choice between the now immediate before the long common delay, tend to choose the
3p–10s–8p and 4p–10s–2p sequences, the pigeon punishment option, the only effect of which is to add
will change its mind and choose the 4p–10s–2p punishment to one alternative. Given their own later
sequence (the sequence with fewer total pellets). preference for the smaller–sooner alternative, the
Similarly, in human life, people promise themselves pigeons can obtain the overall larger reward—the
in the morning to eat and drink moderately at a reward they preferred earlier—only by choosing the
party that evening. When evening comes, however, punishment option. Similarly, humans commit them-
they change their minds and overeat or overdrink. selves to exercise regularly by subscribing to lengthy
(That evening is only part of the sequence; the next health club memberships in which they lose money if
morning when people step on the scale or have the they stop going. People may make bets with their
hangover, they experience the next part.) friends that they will achieve a certain weight at a
Suppose, however, that at the very beginning certain time; they sign contracts with punitive clauses
(at the point at which the common delay was still for failing to complete a job by a certain time.
466
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Commitment in one form or another is the fun- at self-control. If, for example, a person with alco-
damental way the economic view sees self-control holism could decide now how much he or she
(Ainslie, 1992, 2001). Consider the following pair of would drink over the next month rather than
alternatives for a pigeon: a four-pellet reward after a whether to have a drink now—and be strictly com-
4-second blackout versus a one-pellet reward before mitted to that decision—he or she could more easily
a 4-second blackout (4s–4p vs. 1p–4s). According to control the alcoholism. Everyday life, however,
Equation 1 (still assuming that consumption time rarely offers people the opportunity to commit
may be ignored and that s and kdelay = 1), the value themselves in this strict way. In later sections, we
of the former sequence (larger–later reward) is 0.80 outline another kind of commitment, soft commit-
units, and that of the latter sequence (smaller– ment, that is more common in everyday life. First,
sooner) is 1.00 unit (the blackout after the small we must make specific the relationship between dis-
reward is ignored). The pigeon prefers the smaller– counting and self-control.
sooner reward. Now consider a choice between trip-
lets of these alternatives strung together over the Individual Differences in Self-Control
same time period: 4s–4p–4s–4p–4s–4p versus Imagine two people, John and Mary; John’s kdelay
1p–4s–1p–4s–1p–4s. Starting at the beginning of the is 0.1 and Mary’s is 0.3 (delay measured in days).
larger–later sequence, there are three four-pellet Figure 20.2 (axis reversed from that in Figure 20.1
rewards: one after 4 seconds, one after 8 seconds, to show time passing rather than delay) shows two
and one after 12 seconds. Summing the values of the alternative rewards, each with two discount func-
three rewards (0.80 + 0.44 + 0.31), the larger–later tions, one set for John (solid lines) and one set for
sequence increases in value to 1.55 units. Summing
the values of the three one-pellet rewards—one 100
immediately, one after 4 seconds, and one after 8
seconds (1 + 0.20 + 0.11)—the smaller–sooner 80
sequence increases in value to 1.31 units, less than John's crossover point
the value of the larger–later sequence. As the second 60
$ Amount
and third rewards are added, the initial delay to the

larger–later reward weighs less relative to its amount,
40
and the pigeon comes to prefer the larger–later
sequence. Now suppose that it was not possible to
20
choose between the two individual rewards; suppose
that the pigeon was committed to choose between
0
whole sequences. The pigeon would then choose the NOW 2 4 6 8 10 12 14
larger–later sequence—the self-controlled choice. Time (Days)
Ainslie (2001) called commitment to such Figure 20.2. Differences in discount-
sequences bundling. In laboratory experiments, ing: Delay discounting for two different
rats and humans tended to increase choices of monetary rewards: $60 in 7 days (shorter
larger–later over smaller–sooner rewards when bar) and $100 in 15 days (taller bar). The
curved lines indicate the increasing value
rewards were bundled as described in the preceding of those rewards as time passes and the
paragraph, that is, when subjects were required rewards become more imminent. The
to choose not between individual rewards but dashed lines indicate reward values with
a steep discount function (kdelay = 0.3 for
between reward sequences (Ainslie & Monterosso,
Mary). The solid lines indicate reward
2003, with rats, and Kirby & Guastello, 2001, and values with a more shallow discount func-
Kudadjie-Gyamfi & Rachlin, 1996, with humans). tion (kdelay = 0.1 for John). The circle indi-
If, in everyday human life, it was possible to commit cates the point at which the $60 reward
becomes equally as valuable to John as
oneself to choose among bundles rather than on a the $100 reward (2 days before the $60
case-by-case basis, people would certainly do better reward delivery).
467
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Mary (dashed lines). The two alternative rewards & Bickel, 1997; Mitchell, 1999; Petry, 2001;
(the vertical bars) loom in the future—a smaller– Vuchinich & Simpson, 1998; see also the recent
sooner reward ($60 in 1 week) and a larger–later volume edited by Madden & Bickel, 2009). More-
reward ($100 in 15 days). Because John’s kdelay is less over, the kdelays of young children are higher than
than Mary’s, John’s discount functions (the solid those of young adults, and the kdelays of young adults
lines) are shallower than Mary’s (the dashed lines). are higher than those of older adults (Green, Fry, &
At the point marked NOW, John prefers the larger– Myerson, 1994). The simple test of delay discount-
later reward to the smaller–sooner reward; the solid ing described earlier correlates remarkably with fail-
line subtended from the $100 is now higher than the ure of self-control in everyday human life (see
one subtended from the $60 reward. At NOW, he Volume 2, Chapters 7 and 8, this handbook).
can instate a commitment procedure to remove or
attach punishment to the smaller–sooner alterna-
Teleological Behaviorism
tive. If he institutes either of these commitment pro-
cedures, he will actually obtain the larger–later Certainly, it is possible to achieve self-control by
reward, the one he now prefers. However, if he waits strict commitment or punishment commitment.
5 days, the solid lines will have crossed, and he will People can avoid going into the bakery by walking
prefer the smaller–sooner reward. At that point, it down another street; they can avoid eating the pint
will be too late for him to commit himself; he will of ice cream by locking the refrigerator and giving
have no incentive to do it. For Mary, it is already too the key to their spouse. However, another person
late. She now prefers the smaller–sooner reward may actually walk right by the bakery without going
and, given the alternatives as stated, will obtain that in and leave the pint of ice cream untouched in an
reward. (Had she been offered the alternatives unlocked refrigerator. How do people manage to do
2 days earlier, her discount functions would have this? The answer is, by establishing patterns in their
crossed, and she would have been able to commit to lives that are valuable as such—independent of the
the larger reward.) The underlying reason that John sum of the values of their individual components.
can now use commitment and Mary cannot is that The kind of behaviorism that sees patterns of behav-
delay has less of a discounting effect on reward ior as crucial in everyday life as well as in the psy-
value for John than for Mary. chology laboratory is called teleological behaviorism
According to the economic view, the lower a per- (Rachlin, 1992, 1994, 1995b).
son’s kdelay is, the shallower that person’s delay dis-
count functions are, the more time the person has to Teleological View of Self-Control
institute commitment procedures, and the more Consider still another experiment with pigeons
self-control the person should show. On the basis of (Siegel & Rachlin, 1995): A hungry pigeon faces a
this argument, people with low kdelays should be pair of lit keys, one red and one green.3 If the pigeon
expected to have more self-control than people with pecks the red key, it receives two food pellets (2p);
high kdelays. This expectation has been reliably con- if the pigeon pecks the green key, a brief blackout of
firmed in many settings. The kdelays of people 4 seconds ensues, but after the blackout, four food
addicted to heroin, cocaine, or alcohol; people who pellets are delivered (4s–4p). Calculating values
smoke; and people who gamble, as measured by the from Equation 1, as before, the value of the immedi-
procedure described earlier, are significantly higher ate two-pellet reward is 2 units [2/(1 + 0)]; the
than the kdelays of those who do not (Alessi & Petry, value of the delayed four-pellet reward is 0.8 units
2003; Baker, Johnson, & Bickel, 2003; Bickel, [4/(1 + 4)], a strong relative value of 71% [2/(2 +
Odum, & Madden, 1999; Coffey, Gudleski, Saladin, 0.8)] in favor of the smaller–sooner reward over the
& Brady, 2003; Kirby, Petry, & Bickel, 1999; Mad- larger–later one. Now consider the following
den, Bickel, & Jacobs, 1999; Madden, Petry, Badger, change: The pigeon must peck 30 times distributed
In this experiment, as in all other experiments with pigeon subjects described in this chapter, behavior is reported after many weeks of daily sessions
3
and hundreds of exposures to the experimental contingencies.
468
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
in any way between the two keys; after those 30 her job than to be drunk all the time, unhealthy, and
pecks, the very next peck produces a reward. If that socially rejected and to perform poorly at his or her
31st peck was on the red key, the pigeon obtains the job. At the same time, over the next few minutes, he
immediate two-pellet reward; if that 31st peck was or she prefers to have a drink than to not have one.
on the green key, the pigeon obtains the delayed If, over successive brief intervals, the person with
four-pellet reward. It takes about 30 seconds for a alcoholism always does what he or she prefers at the
pigeon to make the initial 30 pecks. Using Equation moment, she or he will always be drinking. A quota-
1 to calculate the value of the two rewards before tion from the comedian Dick Cavett makes a corre-
the pigeon starts pecking, the value of the smaller– sponding point about smoking:
sooner reward (30s–2p) is 0.065 units [2/(1 + 30)],
Once, as [Cavett and Jonathan Miller of
and the value of the larger–later reward (30s–4s–4p)
“Beyond the Fringe”] waited backstage
is 0.11 units [4/(1 + 34)], a relative value of 64%
together at the 92nd St. Y in New York
[0.11/(0.11 + 0.065)] in favor of the larger–later
City, [Cavett] pointed disapprovingly at
reward. Thus, with an added 30-second common
[Miller’s] lit cigarette. [Miller said,] “I
delay, the larger–later reward is nearly twice as valu-
know these will kill me, I’m just not con-
able as the smaller–sooner one—a complete reversal
vinced that this particular one will kill
of the pigeon’s preference without the common
me.” (Cavett, 2009, p. 10)
30-second delay. Indeed, pigeons begin the 30-peck
sequence by preferring the larger–later reward in a The problem for the person with alcoholism as well
ratio of 2:1. However, the economic theory predicts as for the person who smokes is how to make
that as the 30-peck sequence progresses, preference choices over the longer time span and avoid making
for the larger–later reward should weaken and even- choices, as Jonathan Miller did, on a case-by-case
tually reverse. That is, the pigeons should begin by basis. The reason why people have trouble bringing
pecking the green key (30s–4s–4p) but, as the 30 their behavioral patterns into line with abstract and
pecks progress, should eventually switch over, end temporally extended behavioral contingencies is
up pecking the red key, and obtain the smaller– that the value of a desired pattern’s particular com-
sooner reward (2p). However, contrary to economic ponent (refusing the drink or the cigarette) may be
theory, the pigeons almost never switched over. The much less than that of its alternative (drinking the
pigeons persisted in pecking on the key they started drink or smoking the cigarette).4
on and thereby obtained the larger reward. Siegel As Jonathan Miller implied, each cigarette refusal
and Rachlin call this persistence soft commitment. has virtually no value in itself relative to its alterna-
Rachlin (1995a, 2000) argued that it is by means tive. Refusing a particular cigarette is worth nothing
of soft commitment, the persistence of behavioral compared with smoking it. Moreover, individual
patterns, that humans solve most of the self-control cigarette refusals are almost never reinforced—not
problems in their everyday lives (i.e., if they do immediately, not conditionally, not after a delay. If a
solve them). person refuses a single cigarette, he or she does not
The gestalt psychologists said that the whole is wake up 3 weeks later, suddenly a healthier and
greater than the sum of its parts (Koffka, 1935). happier person. To realize the value of a cigarette
Teleological behaviorism extends that gestalt dic- refusal, the person must put together a long string of
tum. It says that the value of an activity may be them. The concept that the value of a behavioral
greater than the sum of the values of its parts. A per- pattern may differ from the sum of the values of its
son with alcoholism prefers to be sober, healthy, parts is not just an empty slogan borrowed from
and socially accepted and to perform well at his or gestalt psychology. It is the very basis of a consistent
Many behavioral patterns are encoded as verbal rules in everyday speech and in texts. Such rules may be nested within metarules such as “Obey your
4
parents.” Occasionally, such rules are enforced wholly by external rewards and punishments of individual components. More frequently, verbal rules
serve as complex discriminative stimuli for the intrinsically valuable patterns of behavior, such as living a healthy life, being discussed here (Baum,
1994; Rachlin, 2000).
469
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
approach to the problem of self-control and, as we Let us take a step back and approach this ques-
show, to altruism. tion from a theoretical perspective. Behaviorists do
not believe that hyperbolic discount functions
Delay Discount Functions in obtained in the laboratory are represented as such in
Teleological Behaviorism people’s nervous systems. Hyperbolic discount func-
In previous sections, we used Equation 1 to calcu- tions are not used by people to evaluate alternatives.
late the value of individual food rewards. Equation 1 Rather, a discount function is a convenient measure
expresses the value of a single reward at a single by which the experimenter or another observer may
delay. Where rewards occurred in sequence, we gauge the temporal extent of the contingencies that
added up their values to arrive at the value of the control a person’s choices.
sequence as a whole. For pigeons in the laboratory In one of Jerry Seinfeld’s stand-up routines, he
presented with simple sequences of discrete and imagines himself as two people: One is the night-
clearly identifiable food rewards, Equation 1 pre- time Jerry who lives carelessly, spends money,
dicts choice well. When the value of a sequence of drinks, and stays up late; the other is next-morning
rewards differs from the sum of the values of its Jerry who has a hangover and needs to get up early
components (as teleological behaviorism asserts is to go to work. Nighttime Jerry has contempt for
often the case), however, Equation 1 cannot be next-morning Jerry; he cares nothing for him and
straightforwardly applied. makes no sacrifices for him. Next-morning Jerry of
We have seen that Equation 1 could not predict course hates nighttime Jerry, the cause of all his suf-
the behavior of pigeons in the Siegel and Rachlin fering. As Seinfeld says, the only way next-morning
(1995) soft commitment procedure (in which 30 Jerry can take revenge against nighttime Jerry is to
pecks were required for reinforcement on either stay in bed, lose his job, and run out of money so
key). The same is true for many complex human that nighttime Jerry will not be able to live his wild
choice situations, in the laboratory as well as in the life. (In real cases of addiction, this is exactly what
real world. Consider a series of rewards arranged in happens.)
either increasing or decreasing order. Equation 1 What would it mean then if this imagined Jerry
predicts that the decreasing order, in which the learned self-control? For a teleological behaviorist,
larger rewards are less delayed than the smaller self-control consists of temporally extended patterns
rewards, will be preferred to the increasing order, in of behavior conforming to abstract environmental
which the smaller rewards are less delayed than contingencies (e.g., “Always try to understand the
larger rewards. Yet people often prefer an increasing other person’s point of view”). Formation of such
series of monetary rewards to a decreasing series patterns involves cooperation between nighttime
(Loewenstein & Prelec, 1992). On the negative side, Jerry, next-morning Jerry, and next-day, next-week,
people prefer a sequence of more intense pain fol- next-month Jerry. These guys need to make friends
lowed by less intense pain to a sequence of less with each other. The wider and more abstract the
intense pain followed by more intense pain (Kahne- pattern necessary to describe Jerry’s behavior is (the
man, 1999), even though Equation 1 predicts the more self-control Jerry has) the shallower his delay
reverse. As one goes from the laboratory to more discount function is. It is important to note, though,
complex real-life situations, Equation 1 breaks down the shallowness of Jerry’s delay discount function is
still further. The order of dishes in a preferred res- not the immediate cause of his self-control but a
taurant meal, of notes in a good song, of movements measure of it (just as the temperature of a gas is not
in a good dance, or of chapters in a good book does the cause of molecular motion but a measure of it).
not go from best to worst over time or more exciting
to less exciting, as Equation 1 predicts. Rather, the
Social Discounting
overall patterns of these events are crucial to their
value. Then what, it may justifiably be asked, is the The view of delay discounting expressed in the previ-
point of measuring people’s kdelays by Equation 1? ous section might be called a social theory of delay
470
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
discounting.5 It sees people at future times connected between delay and social distance, social discount
to their present selves as analogous to different people functions would have the same hyperbolic form as
connected to each other socially. People have overlap- delay discount functions:
ping interests, with their relatives and friends sepa-
rated from them by social space. The greater the social V ,
vsocial = (2)
distance is between a person and any particular other 1 + ksocial N
person and the fewer common interests they have
between them, the less cooperative each will be with where V is the value of an immediate reward given
the other. Similarly, individuals may be seen as hav- to Person A; vsocial is the value to Person A of that
ing overlapping interests with their future selves— reward given to Person B; N is the social distance
their selves separated from them by time; the farther between Person A and Person B; and ksocial is a con-
into the future (or the past) and the fewer common stant that differs among individuals and measures
interests between them and their future selves, the degree of social discounting. (The exponent s in
less cooperative they will be with their future selves Equation 1 has been found to equal 1.0 in social dis-
and the less self-controlled their behavior will be. The count functions.)
fictional Jerry of the previous section would consti- Experiments in the behavioral laboratories at
tute an extreme case in which a few hours is far Stony Brook have reliably obtained hyperbolic social
enough into the future to prevent cooperation. discount functions that conform to Equation 2
This idea, that temporal separation is analogous to (Jones & Rachlin, 2006; 2009; Rachlin & Jones,
social distance, may be found in Aristotle’s Nicoma- 2008a, 2008b, 2009). The general procedure of these
chean Ethics (Rachlin, 1994). In modern philosophy, experiments was as follows: Students (Stony Brook
this idea has been advanced by Parfit (1984); in social University undergraduates) were asked to imagine
psychology, by Trope (Trope & Liberman, 2003) and that they had made a list of the 100 people closest
others; in behavioral psychology, by Ainslie (1992) and to them (but not to actually make the list). Order
Rachlin (1989); and in economics, by Simon (1995). on the list (N, ranging from 1 to 100) was taken as a
Simon suggested extending the economic model to measure of social distance. Students were then asked
three dimensions: (a) current consumption by the per- to choose hypothetically between a smaller amount
son, (b) consumption by the same person at later times of money for themselves and a larger amount for a
(delay discounting), and (c) consumption by other person at a given social distance. For example, a stu-
people (social discounting). For Simon, discount func- dent might prefer $75 for him- or herself to $75 for
tions such as that shown in Figure 20.1 should be three the 10th person on his or her list but prefer $75 for
dimensional: “Instead of a one-dimensional maximiz- the 10th person on the list to $5 for him- or herself.
ing entity, or even the two-dimensional individual who Just as with delay discounting, at some crossover
allocates intertemporally, this model envisages a three- amount between $75 and $5 for him- or herself, the
dimensional surface with an interpersonal ‘distance’ student must be indifferent between an amount for
dimension replacing the concept of altruism” (p. 367). him- or herself and a (usually) larger amount ($X)
For Simon, generosity to another person (or altruism) for the 10th person on the list. In our studies, as N
should be no more surprising than generosity to one’s increased, the crossover amount decreased. That is,
own self at future times (self-control). as social distance increased, students were willing to
forgo less money for themselves to give a fixed
Hyperbolic Social Discounting amount to another person. The function thus
With Humans obtained, relating social distance to amount of
Rachlin and colleagues (Rachlin, 1989; Raineri & money forgone, is a social discount function. Figure
Rachlin, 1993) speculated that given the parallel 20.3 shows functions from two experiments.
The focus of this and the following sections is social discounting and its relation to altruism. Experimental work on social discounting has so far been
5
done almost wholly in our laboratory at Stony Brook University. However, an extensive literature also exists on the experimental analysis of social
cooperation. An excellent review of this work can be found in Schmitt (1998).
471
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
80 of scale used to measure delay and social distance.

Delay is measured on a ratio scale that can be infi-
$ Forgone to Give $75 to Person N
70 Jones & Rachlin

(2006) nitely divided and with which all the operations of
60 arithmetic apply; on a ratio scale, the difference
50
between 1 and 5 equals the difference between 6 and
10. Social distance, however, is measured on an
40
ordinal scale with which only the order of the num-
30 bers has any meaning. The person at N = 1 is closer
Rachlin & Jones to the participant than the person at N = 2, and the
20 (2008a)
person at N = 2 is closer than the person at N = 3,
10
but the difference in closeness between the first and
0 second person may be twice as big as, half as big as,
0 20 40 60 80 100
nearly equal to, a very tiny fraction of, or a gigantic
Social Distance (N)
multiple of the difference between the second and
Figure 20.3. Social discounting: The median amount third person (Stevens, 1946). To get some idea of
of hypothetical money forgone to give $75 to another those differences, Rachlin and Jones (2009) asked
person at various social distances using the Jones and 44 Stony Brook undergraduates as before to imagine
Rachlin (2006) procedure (open squares) and the
Rachlin and Jones (2008b) procedure (filled circles). The they had made a list of the 100 people closest to
dashed and solid lines are the best-fitting versions of them. They then told them to imagine standing on a
Equation 2 for each of those experiments, respectively. vast field with those 100 people, each person at a
From “Social Discounting and Delay Discounting,” by
physical distance proportional to their social dis-
H. Rachlin and B. A. Jones, 2008, Journal of Behavioral
Decision Making, 21, p. 32. Copyright 2008 by John tance. Then, for a series of social distances (1, 2, 5,
Wiley & Sons. Adapted with permission. 10, 20, 50, and 100), we asked them to tell us their
physical distance from that person. The obtained
In Rachlin and Jones’s (2008b) Experiment 1, the relation between ordinal and physical social distances
hypothetical alternatives were $75 for Person N and took the form of the power function F = 0.19N2.2,
nothing for the participant versus $X for the partici- where F is distance in feet.
pant and nothing for Person N (as in the preceding The greater-than-1.0 exponent of this function
example). In Jones and Rachlin’s (2006) earlier indicates that as ordinal distance (N) increased, the
experiment, the hypothetical alternatives were a physical difference between adjacent values of N
shared $150 ($75 for the participant and $75 for increased proportionally. People seem to have tried,
Person N) versus $X (plus $75) for the participant as it were, to push more distant people further away
and nothing for Person N. The points in Figure 20.3 and keep closer people still closer.
are median amounts of money forgone in the Rach- Solving the above equation for N, N = 3.3F0.45.
lin and Jones (2008b) experiment as functions of Then, substituting in Equation 2,
social distance. The solid line is Equation 2 fitted to
those points. The dashed line is Equation 2 fitted in V (3)
v= .
the same way to the data of the Jones and Rachlin 1 + k ' F 0.45
(2006) experiment. The two social discount func- Equation 3 predicts that when social distance is
tions virtually overlap (ksocial ≈ 0.05 in both experi- measured by physical distance on a ratio scale, social
ments). The congruence of the functions of Figure discounting will be hyperbolic, but the social dis-
20.3, obtained with differing procedures, indicates tance term will have a fractional exponent (as does
that hyperbolic social discounting, as with hyper- the ratio-scale delay term in delay discounting). To
bolic delay discounting, is a robust measure. test this prediction, Rachlin and Jones (2009)
obtained social discount functions with a new group
Social Distance and Physical Distance of 64 Stony Brook students. Again, instead of a
One obvious difference between delay discounting series of ordinal distances, Jones and Rachlin (2007)
and social discounting is the difference in the kind asked students to imagine themselves on a vast field
472
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
with all of their friends and relatives and, instead of against a fixed amount of money at a series of delays.
obtaining crossover points at a series of N values, In the social discounting procedure, the amount of
they obtained crossover points for individuals at the an immediate monetary reward is titrated against a
following series of distances on the field: 1 foot, fixed amount of money at a series of social distances.
2 feet, 10 feet, 100 feet, 100 yards, 1 mile, and Rachlin and Jones (2008b) eliminated the immediate
10 miles. Figure 20.4 shows Equation 3 fit to the monetary reward and directly titrated the delay of a
median crossover points with V fixed at 90 (and the fixed amount of money ($75) against that same fixed
exponent fixed at 0.45 as predicted from the prior amount ($75) at a series of social distances. The
experiment).6 crossover point in this experiment represented the
The good fit of the line to the points in Figure delay (D) that reduced the value of $75 as much as
20.4 (R2 = .98) indicates that social discounting is did a social distance of N. In other words, the delay
not an artifact of an ordinal scale and that the social obtained makes the value of a delayed reward equal
discounting measure is precise enough and broad to the value of a reward to another person at a given
enough to predict results in one set of conditions social distance. Setting these values equal to each
from results obtained in another set of conditions. other in Equations 1 and 2,
In the next section, we further extend the predictive
ability of the social discounting measure. On the vdelay = vsocial ,
basis of delay discounting and social discounting
individually, results are predicted of a procedure V V , and
= (4)
that pits one kind of discounting against the other. 1 + kdelay D 1 + ksocial N
s
D = cN 1/ s ,
Social and Delay Discounting
In the typical delay discounting procedure, the where c = (ksocial / kdelay)1/s. The relation between the
amount of an immediate monetary reward is titrated delay and the social distance at the crossover point
in this experiment is predicted to be a power func-
90 tion. A power function plotted in log-log coordi-
80
nates is a straight line with a slope equal to the
$ Forgone to Give $75 to Person N
exponent. Because the exponent s in delay dis-

70
counting has been found to be less than 1.0, the
60
slope of the function with an exponent of 1/s
50 should be greater than 1.0. Figure 20.5 shows the
40 median crossover point as a function of social dis-
30
tance plotted on log-log coordinates. The points
are fitted very well by a straight line (r2 = .96)
20
with a slope greater than 1.0 (1/s = 1.5) as pre-
10 dicted. The obtained value of s of 0.75 (1/1.5) is
0 within the range of s values obtained for under-
1 10 100 1000 10000
graduates and young adults in other delay dis-
Social Distance (feet)
counting studies (Green, Myerson, & Ostaszewski,
Figure 20.4. Social discounting with ratio scale: The 1999; Myerson & Green, 1995). Again, Rachlin
median amount of hypothetical money forgone to give and Jones were able to predict parameters of
$75 to another person at various social distances mea-
behavior in one condition from behavior in other
sured in feet. The curved line is the best-fitting version
of Equation 3 with V fixed at 90 and s fixed at 0.45 conditions, validating the social discounting mea-
(kdelay = 0.21, R2 = .98). Note the logarithmic x-axis. surement procedure.
6 Fixing V at 90 reflects the fact that many participants preferred to give $75 to people at very close social distances (N = 1 or 2) than to receive $75
themselves. This resulted in y-axis intercepts (at N = 0) greater than the nominal $75. See Rachlin and Jones (2008b) for a discussion of this
phenomenon.
473
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
10000 giving is generally regarded as altruistic, but it is not

out of line with the data of social discounting.
1000 Undergraduates typically place a random classmate
Delay (days) at around the 75th person on their list (N = 75).7
Note in Figure 20.3 that the crossover point (the
100
amount subjects were willing to forgo to give $75 to
that person) was around $15, or 20% of the $75.
10
Social discounting results are thus consistent with
dictator game results (see also Chapter 19, this vol-
1 ume, for an alternative interpretation).
The ultimatum game is similar to the dictator
1 10 100 game except that in the ultimatum game, if the sec-
Social Distance (N) ond person (the receiver of the money) refuses the
Figure 20.5. Delay and social money, neither giver nor receiver gets anything.
distance: The median delay at Economic theory says that givers should give receiv-
which $75 was equivalent in ers some minimal amount (say $0.25 out of a $10
value to $75 given immediately endowment) and that receivers should take it; other-
to another person at a social dis-
tance of N. (Note log scale.) From wise, they get nothing. Yet, in these situations,
“Social Discounting and Delay undergraduates usually give about $4 of a $10
Discounting,” by H. Rachlin endowment to classmates. That is a wise decision; if
and B. A. Jones, 2008, Journal of
givers give less than 40% of their endowment,
Behavioral Decision Making, 21,
p. 40. Copyright 2008 by John receivers are likely to refuse the money (Camerer,
Wiley & Sons. Adapted with 2003). The difference between the 40% to 50% of
permission. endowment typically given to classmates in ultima-
tum games and the 20% to 30% given in dictator
Social Discounting in Dictator, games is easily explained as the result of the possi-
Ultimatum, and Public Goods Games bility that low donations will be punished in the
Dictator games are very simple. Two people play. former and the lack of such possibility in the latter.
One player is given an endowment of a certain Rachlin and Jones (2009) found that as social dis-
amount of money and must decide how much of it, tance increased, the amount given in both dictator
if any, to give to the other player. The second player and ultimatum games decreased hyperbolically—as
has no recourse if she or he feels she or he has been Equation 2 predicts—but the slope of the ultimatum
treated unfairly. Economic theory is clear on what hyperbola was less than the slope of the dictator
the first player should do—he or she should give hyperbola; the amount given was greater for ultima-
nothing. A gift of any amount above zero violates tum games than for dictator games across all social
basic economic maximization principles (simply distances tested.
understood). Yet in this game, people (usually Receivers expressed willingness to accept lower
undergraduates) do give significant amounts of their amounts from those close to them than from those
initial endowment to other people. For example, at greater social distances. For example, in an ulti-
with an endowment of $10, the mean amount given matum game, your mother (presumably close to
is roughly $2, and a significant number of players you) would give you a very high percentage of her
split the endowment 50:50 (Camerer, 2003). Such endowment, but you would accept virtually any
Fifty Stony Brook undergraduates were given instructions similar to those given in Rachlin and Jones (2009)—asking them to imagine everyone they
7
knew to be on a vast field in which physical distance was based on social distance. These students were then asked to assign a physical distance mea-
sure for people at various social distances (1, 10, 100) and for a randomly selected classmate. On average, a random classmate was placed at a distance
corresponding to the ordinal social distance (an N value) of 75.
474
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
amount she gave you, no matter how low. However, givers in dictator games and by receivers in ultima-
a distant cousin (presumably not close to you) tum games in terms of an inherited sense of fairness.
would give you much less than your mother would, It is as though each particular act is reinforced by so
but you would quickly punish your cousin (at a cost much money plus so much fairness. This, however,
to yourself) if he or she gave you too much less. is a category mistake (Ryle, 1949)—as with compar-
This phenomenon, called altruistic punishment, is ing a collie with a dog or a dance step with a dance.
pervasive in ultimatum games. It occurs even when It is certainly possible for people to find innate value
the giver is a perfect stranger with whom the in patterns of acts that are fair. We argue in the next
receiver can expect to have no further contact section that such is the case. Individual acts, though,
(Camerer, 2003). What do receivers gain to offset may be fair only provisionally. A 50:50 split might
their own loss from punishing givers for unfair dis- not be fair if the previous split was 100:0 either way.
tributions? The effect that such punishment may In general, fairness is a property of patterns of acts
have on a giver’s behavior may benefit people who (of acts in their temporal as well as social context),
interact with him or her down the line, but how not of individual acts.
would it benefit a receiver who will likely never see As in dictator and ultimatum games, players in
the giver again? public goods games receive an initial endowment
Behavioral theory, simply considered, seems to and may contribute any part of it. In a public goods
predict the same selfish behavior as does economic game, however, there are many players. The contri-
theory. Why choose no reward over a small one? bution goes into a common pool that is augmented
Teleological behaviorism, however, looks beyond proportionally to the total contribution and then
maximization of reward at a narrow time, place, distributed equally to all players. The contingencies
and social space. According to teleological behav- in such games parallel those in everyday life of con-
iorism, people may learn to choose long-term pat- tributing to public radio or television, voting, not
terns of behavior as wholes and refuse to make littering, recycling, and so forth. In a public goods
decisions on a case-by-case basis, not just because game played by 98 Stony Brook undergraduates
it is more efficient to make decisions wholesale (Jones & Rachlin, 2009), each player received (a
than retail, but because if people make decisions hypothetical) $100. Then (again hypothetically) a
on a case-by-case basis, no matter how far into box was passed around the room, and the player
the future they integrate the particular delayed could put any fraction of the $100 (from all to none)
alternatives—even from now to the end of their into the box. Then, as the instructions said, the
lives—they may fail to exhibit the highest valued experimenter would double the amount in the box
behavioral patterns. As we argued previously, a and distribute it equally to all of the players regard-
series of high-valued particular choices may result less of how much money they initially contributed.
in a low-valued pattern of choices; conversely, a The median amount students indicated that they
series of low-valued particular choices may result would put into the box was $25, but a significant
in a high-valued pattern. Thus, in individual cases, number of students contributed $50 or $100, and
people will often behave altruistically (nonmaxi- many indicated $0. The amount contributed may be
mally for them as individuals), but the long-term seen to be a measure of degree of altruism—the
patterns of their behavior may be highly valued same trait measured by social discount functions.
(not just for society, but for them as individuals). A Therefore, it should not be surprising that the
common feature of particular acts of self-control amount contributed by a participant in the public
and particular acts of altruism is that they are both goods game correlated significantly (and negatively)
low-valued components of high-valued abstract with the slope of that participant’s social discount
patterns (such as moderate eating and drinking or function. The more generous a person was in the
obeying the golden rule). public goods game, the shallower that person’s
Decision theorists (e.g., Fehr & Fischbacher, social discount function tended to be. (However,
2003) have explained particular altruistic acts by there was no correlation between public goods game
475
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
contribution and the slope of a person’s individual the kind of things one could directly inherit, people
delay discount function.)8 born with shallow functions would just tend to be
more altruistic than people born with steep func-
Altruism and Self-Control in Evolution tions. However, as we argued previously, this seems
and Learning unlikely. Individual social discounting is not as rigid
Fehr and Fischbacher (2003) defined altruism as as such a tendency would imply. Social discounting
“costly acts that confer economic benefits on other varies with circumstances. A more likely candidate
individuals” (p. 786). Given this definition, and evo- is a tendency to ignore the consequences of individ-
lutionary theory as it stands, it is hard to see how ual acts and to behave in accordance with the conse-
people might have inherited an altruistic tendency. quences of temporally extended patterns of acts.
People with a tendency to sacrifice their own fitness The innate behavior of organisms is often pat-
for that of others must by definition reduce their terned (eating, sleeping, aggression, sex, etc.); these
chances of survival, and hence their chances of patterns can evolve over the lifetimes of individual
reproduction, relative to selfish people. Altruists organisms into highly complex forms in response to
should thus die out. Some evolutionary biologists environmental contingencies (Rachlin, 1995b;
have argued, however, that although, in evolution- Teitelbaum, 1977). Since Thorndike (1911), behav-
ary history, altruists would indeed die out vis-à-vis ior has been noted to evolve over the course of indi-
selfish individuals, groups composed mostly of vidual lifetimes by a process akin to natural
altruists (within which individuals cooperate with selection of species. Staddon and Simmelhag (1971)
each other) would have a better chance of surviving made a modern argument to this effect. Baum
than groups composed mostly of selfish people (just (1994) and Rachlin (2000) extended this argument
as, e.g., basketball teams composed of unselfish to selection, within organisms’ lifetimes, of behav-
players would score more points, all else equal, than ioral patterns (groups of acts). That is, organisms
teams composed only of selfish players). A conflict may learn to choose the highest valued pattern over
thus arises between inherited behavioral patterns the highest valued single alternative through a rein-
beneficial to the individual and inherited behavioral forcement process akin to group selection of organ-
patterns beneficial to the group. The notion that pat- isms in evolutionary biology. Such learning would
terns beneficial to the group may win out over pat- serve for self-control as we conceive it here as well
terns beneficial to the individual is called group as for altruism.
selection. The crucial variable for group selection is
rate of reproduction (or replacement). When the Limitations of the Analogy Between
rate of replacement of individuals within groups is Self-Control and Altruism
low relative to the rate of replacement of groups We have argued here that self-control may be
within the larger population, group selection has viewed as a kind of altruism in which the place of
been shown to work in the evolution of altruism other people is taken by the person at other times. If
(Boyd, Gintis, Bowles, & Richerson, 2005). this was more than a common tendency to group
One may then ask, what is the inherited ten- behavior into patterns, there could exist a single
dency in individuals that results in altruistic behav- fundamental discounting process governing both
ioral patterns? One possibility is innately shallow kinds of discounting, or one kind of discounting
social discount functions. If discount functions were might be fundamental and the other derived from it.
In this experiment, both public goods and social discounting contributions were completely anonymous. Participants made their choices together
8
in a large classroom and were identified only by number. However, one may question whether participants in this and other social discounting
experiments understood that their contributions would be anonymous as regards the receiver. Some unpublished data from our laboratory indicates
that they did understand this (Locey & Rachlin, 2008). Social discounting was measured in three ways: (a) the standard way described previously,
(b) with instructions indicating explicitly that contributions would be anonymous, and (c) with instructions asking participants to imagine that the
receiver was looking over their shoulder while they made their choices. As expected, participants were significantly more generous when they imag-
ined they were observed than when their choices were anonymous. Contributions of the standard group were also significantly less than those of the
observed group and did not differ from those of the anonymous group. This evidence shows that participants in standard social discounting experi-
ments assume that receivers will not know who they are.
476
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
1 Equation 2 had a parameter for sensitivity to social

distance that corresponded to the delay sensitivity
parameter, s, in Equation 1) means that (beyond a
certain point) doubling social distance (N) halves
0.1 SOCIAL vsocial, but because of the fractional s in Equation 1,
doubling delay has less of an effect on vdelay. This
k-value
would seem to indicate a greater sensitivity, on aver-

age, to social distance than to delay. As we said pre-
0.01 DELAY viously, though, arithmetic operations such as
multiplication and division of an ordinal scale have
little meaning. Delays, whether measured on ordinal
or ratio scales, are incommensurate with social
0.001
low medium high distances.
Magnitude Comparing the two types of discounting directly
is therefore not possible. We can compare the two
Figure 20.6. Magnitude effects: The medians of the
best-fitting k values of individual participant discount forms of discounting indirectly, however, in terms
functions at each of three magnitudes ($15, $150, of self-control and social cooperation.
and $150,000 for kdelay in Equation 1 and $7.50, $75, Brown and Rachlin (1999) devised a repeated
and $75,000 for ksocial in Equation 2). From “Social Prisoner’s Dilemma–type game that could be played
Discounting and Delay Discounting,” by H. Rachlin and
B. A. Jones, 2008, Journal of Behavioral Decision Making, in two ways, alone or together (i.e., by one person
21, p. 37. Copyright 2008 by John Wiley & Sons. or two people). In Prisoner’s Dilemma games, a
Adapted with permission. player may cooperate or defect. In a two-player
game, if both players cooperate, both gain moder-
Some lines of evidence, which we summarize in this ately high rewards; if both players defect, both gain
section, have shown that the two kinds of discount- moderately low rewards; if one cooperates and one
ing, although both hyperbolic, measure the extent of defects, the defector gains a very high reward and
different types of behavioral patterns. the cooperator gains a very low reward. The best
First, the steepness of both delay and social dis- pattern of behavior for both players over a series of
counting varies with the magnitude of the undis- Prisoner’s Dilemma trials is to always cooperate and
counted amount (V in Equations 1 and 2). As always earn the moderately high reward. However,
indicated previously, Raineri and Rachlin (1993) because it is to the interest of each player that the
found that smaller undiscounted amounts of money other player cooperates, each player may punish the
and other commodities were discounted more other player’s defection by defecting him- or herself
steeply by delay than were larger amounts. The (as with Punch and Judy, an arms race, or a rocket
exact opposite is the case for social discounting. exchange between Israel and Hamas). In that case,
Rachlin and Jones (2008b) found steeper social dis- both players will end up repeatedly defecting and
counting with larger rewards than with smaller. Fig- earning the moderately low reward.
ure 20.6 shows median k values for delay and social Students in the alone version of the Brown and
discounting as a function of amount (V). Note that Rachlin (1999) game played a Prisoner’s Dilemma
the lines go in opposite directions.9 game against a tit-for-tat strategy. A tit-for-tat strat-
Second, there is a difference of scale in the degree egy simply and mechanically reflects a player’s
to which people cooperate with themselves at later choice on the previous trial; if the player cooperates
times and the degree to which they cooperate with on trial n, the other player will cooperate on trial
others. The unitary exponent s in Equation 2 (i.e., if n + 1; if the player defects on trial n, the other player
9 The same holds for probability discounting vis-à-vis delay discounting. That is, the effect of amount on the steepness of probability discounting, as
with that of social discounting, goes in the reverse direction of the effect of amount on steepness of delay discounting (Green & Myerson, 2004).
477
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
will defect on trial n + 1. When playing against tit- themselves at later times overlap more with their
for-tat, players are essentially playing against them- present interests than with those of even their clos-
selves on the next trial (their next-trial self). Playing est friends or relatives. Rachlin and Jones’s (2008a)
a Prisoner’s Dilemma game against tit-for-tat is thus undergraduate students’ interests and those of their
a self-control problem; the best choice on the pres- later selves would certainly fall out of congruence
ent trial is to defect, but the best choice over a series over time, but it would take a very long time for
of trials is to cooperate. Students playing the alone them to fall as far out of congruence as they must be
version of the game eventually learned to cooperate with those of a random classmate. Thus, it is far eas-
most of the time. In the crucial condition of the ier to cooperate with oneself (even at different
experiment, students first played in the alone condi- times) than it is to cooperate with another person.
tion for a number of trials, eventually learning to Finally, in several of the experiments we have
cooperate most of the time versus their future selves. described, both social and delay discount functions
Then Brown and Rachlin formed pairs of students were obtained for all subjects. One might expect
(each of whom had been cooperating in the alone individuals with steep delay discount functions to
condition) and paired them in a together (two- also have steep social discount functions. To a cer-
person) game. They almost immediately began tain extent, this was the case. Figure 20.7 plots the
defecting against each other at a high rate. In other correlation (on log-log axes) between ksocial and kdelay
words, although students could learn to cooperate for 33 students (Rachlin & Jones, 2008b). Steep-
with their future (next-trial) selves, such learning ness of social discounting correlated significantly
did not transfer to cooperation with other people. (but barely) with steepness of delay discounting
The self-control learned in this situation did not (r = .35). In hindsight, the weakness of the correla-
transfer to social cooperation. tion should have been expected. Delay discounting
In retrospect, such behavior makes sense. Hamil- measures self-control; social discounting measures
ton’s (1964a, 1964b) kin selection theory (1964) social cooperation. In real life, these two traits often
predicts that altruism will be greater with greater do not go together. (Some people are Scrooge-like
genetic overlap (degree of kinship) between giver and abstemious and tend to save their money; they
and receiver. Except for clones or people who are
severely inbred, people’s genetic overlap with them- 1
selves is at least twice as great as their genetic over-

lap with any other person. Kinship correlates
strongly with social distance. The greater the genetic
overlap with someone else is, the closer the person
should feel to the other person (all else being equal),
and the more altruistic the person should be toward 0.1
ksocial
him or her. Rachlin and Jones (2008a) found that

steepness of social discounting varied directly with
genetic distance; social discount functions with non-
relatives as receivers were steeper than social dis-
count functions with relatives as receivers. Students
were willing to forgo significantly more money for 0.01
the benefit of relatives than for the benefit of nonrel-

atives, even at the same social distance. People’s high 0.001 0.01 0.1
generosity to themselves (100% genetic overlap) rel- kdelay
ative to their generosity to other people (50% or less
genetic overlap) might thus have been expected. Figure 20.7. Delay and social discounting: ksocial
as a function of kdelay for each of 33 participants. The
Even ignoring genetic tendencies, it seems safe solid line is the best-fit straight line using simple linear
to assume that people’s common interests with regression.
478
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
are generous to their future selves but not to others. these applications and limitations is a direction for
Some people are generous spendthrifts—buying future work.
drinks for everyone at the bar, say, but careless
about their own future welfare.) References
Even within the areas of self-control and social
Ainslie, G. (1992). Picoeconomics: The strategic interac-
cooperativeness, subpatterns do not completely tion of successive motivational states within the person.
overlap. Not all people with alcoholism are addicted Cambridge, England: Cambridge University Press.
to drugs or bad at saving money or gamble reck- Ainslie, G. (2001). Breakdown of will. Cambridge,
lessly. As Mischel (2004) has pointed out, a person England: Cambridge University Press.
may show a high degree of self-control in one con- Ainslie, G., & Monterosso, J. (2003). Building blocks of
text and be impulsive in another. Similarly, a person self-control: Increased tolerance of delay with bun-
may act altruistically in one social setting and self- dled rewards. Journal of the Experimental Analysis of
ishly in another. Behavioral patterns of self-control
Alessi, S. M., & Petry, N. M. (2003). Pathological gam-
and altruism may be learned and innate to different
bling severity is associated with impulsivity in a
degrees. The contexts and contingencies of the evo- delay discounting procedure. Behavioural Processes,
lutionary processes underlying both largely learned 64, 345–354. doi:10.1016/S0376-6357(03)00150-5
and largely innate patterns may differ within as well Baker, F., Johnson, M. W., & Bickel, W. K. (2003). Delay
as between categories. Different aspects of the envi- discounting in current and never-before cigarette smok-
ronment gain control over behavior at different ers: Similarities and differences across commodity, sign,
and magnitude. Journal of Abnormal Psychology, 112,
times. Nevertheless, steepness of delay discounting 382–392. doi:10.1037/0021-843X.112.3.382
does indeed correlate with many kinds of dysfunc-
Baum, W. B. (1994). Understanding behaviorism: Science,
tional personal behavior (see Volume 2, Chapters 7 behavior, and culture. New York, NY: HarperCollins.
and 8, this handbook), and we expect steepness of Becker, G. S., & Murphy, K. M. (1988). A theory of
social discounting to correlate with many kinds of rational addiction. Journal of Political Economy, 96,
dysfunctional social behavior. 675–700. doi:10.1086/261558
Bickel, W. K., Odum, A. L., & Madden, G. J. (1999).
Impulsivity and cigarette smoking: Delay dis-
Conclusion counting in current, never, and ex-smokers.
Psychopharmacology, 146, 447–454. doi:10.1007/
Another title for this chapter might have been “The PL00005490
Nature and Uses of Delay and Social Discount Func- Boyd, R., Gintis, H., Bowles, S., & Richerson, P. (2005).
tions.” The empirical work examined indicates that The evolution of altruistic punishment. In H. Gintis,
delay and social discount functions are highly useful S. Bowles, R. Boyd, & E. Fehr (Eds.), Moral senti-
in predicting self-control and altruism in the labora- ments and material interests: The foundations of coop-
erative and economic life (pp. 215–228). Cambridge,
tory and (for self-control, at least) in everyday life. MA: MIT Press.
However, this success should not lead researchers to
Brown, J., & Rachlin, H. (1999). Self-control and social
enshrine discount functions as entities in them- cooperation. Behavioural Processes, 47, 65–72.
selves, having an existence as such either in organ- doi:10.1016/S0376-6357(99)00054-6
isms’ nervous systems or in internal cognitive Camerer, C. F. (2003). Behavioral game theory:
structures. Rather, delay and social discount func- Experiments in strategic interaction. New York, NY:
tions are convenient measures of the degree to Russell Sage Foundation.
which patterns of overt behavior extend in time or Cavett, D. (2009, May 31). Week in review. New York
Times, p. 10.
in social space. They enable prediction of behavior
in situations that may differ from those in which Chapman, G. B., & Elstein, A. S. (1995). Valuing the
future: Temporal discounting of health and money.
they were originally measured. However, such pre- Medical Decision Making, 15, 373–386. doi:10.1177/0
dictability has limits from one situation to another 272989X9501500408
both within the realms of self-control and altruism Coffey, S. F., Gudleski, G. D., Saladin, M. E., & Brady,
as well as between those realms. Exploration of K. T. (2003). Impulsivity and rapid discounting
479
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
of delayed hypothetical rewards in cocaine- behavior. Cambridge, England: Cambridge University

dependent individuals. Experimental and Clinical Press. doi:10.1017/CBO9780511664854
Psychopharmacology, 11, 18–25. doi:10.1037/ Kahneman, D. (1999). Objective happiness. In D.
1064-1297.11.1.18 Kahneman, E. Diener, & N. Schwarz (Eds.), Well-
Fehr, E., & Fischbacher, U. (2003). The nature of being: The foundations of hedonic psychology (pp. 3–25).
human altruism. Nature, 425, 785–791. doi:10.1038/ New York, NY: Russell Sage Foundation.
nature02043 Kirby, K. N. (1997). Bidding on the future: Evidence
Forzano, L. B., & Logue, A. W. (1994). Self-control in against normative discounting of delayed rewards.
adult humans: Comparison of qualitatively differ- Journal of Experimental Psychology: General, 126,
ent reinforcers. Learning and Motivation, 25, 65–82. 54–70. doi:10.1037/0096-3445.126.1.54
doi:10.1006/lmot.1994.1004 Kirby, K. N., & Guastello, B. (2001). Making choices
Green, L., Fry, A. F., & Myerson, J. (1994). Discounting in anticipation of future choices can increase self-
of delayed rewards: A life-span comparison. control. Journal of Experimental Psychology: Applied,
Psychological Science, 5, 33–36. doi:10.1111/ 7, 154–164. doi:10.1037/1076-898X.7.2.154
j.1467-9280.1994.tb00610.x Kirby, K. N., Petry, N. M., & Bickel, W. K. (1999).
Green, L., & Myerson, J. (2004). A discounting frame- Heroin addicts have higher discount rates for delayed
work for choice with delayed and probabilistic rewards than non-drug-using controls. Journal
rewards. Psychological Bulletin, 130, 769–792. of Experimental Psychology: General, 128, 78–87.
doi:10.1037/0033-2909.130.5.769 doi:10.1037/0096-3445.128.1.78
Green, L., Myerson, J., Holt, D. D., Slevin, J. R., & Estle, Koffka, K. (1935). Principles of Gestalt psychology. New
S. J. (2004). Discounting of delayed food rewards in York, NY: Harcourt-Brace.
pigeons and rats: Is there a magnitude effect? Journal Kudadjie-Gyamfi, E., & Rachlin, H. (1996). Temporal
of the Experimental Analysis of Behavior, 81, 39–50. patterning in choice among delayed outcomes.
doi:10.1901/jeab.2004.81-39 Organizational Behavior and Human Decision
Green, L., Myerson, J., & McFadden, E. (1997). Rate Processes, 65, 61–67. doi:10.1006/obhd.1996.0005
of temporal discounting decreases with amount Locey, M. L., Jones, B. A., & Rachlin, H. (2011). Real and
of reward. Memory and Cognition, 25, 715–723. hypothetical rewards in self-control and social dis-
doi:10.3758/BF03211314 counting. Judgment and Decision Making, 6, 552–564.
Green, L., Myerson, J., & Ostaszewski, P. (1999). Locey, M. L., & Rachlin, H. (2008). [Altruism and ano-
Discounting of delayed rewards across the life span: nymity]. Unpublished raw data.
Age differences in individual discount functions.
Behavioural Processes, 46, 89–96. doi:10.1016/ Loewenstein, G., & Prelec, D. (1992). Anomalies in
S0376-6357(99)00021-2 intertemporal choice: Evidence and an interpreta-
tion. In G. Loewenstein & J. Elster (Eds.), Choice
Green, L., & Rachlin, H. (1996). Commitment using over time (pp. 119–146). New York, NY: Russell Sage
punishment. Journal of the Experimental Analysis of Foundation.
Madden, G. J., Begotka, A., Raiff, B. R., & Kastern, L. (2003).
Hamilton, W. D. (1964a). The genetical evolution of Delay discounting of real and hypothetical rewards.
social behaviour. I. Journal of Theoretical Biology, 7, Experimental and Clinical Psychopharmacology, 11,
1–16. doi:10.1016/0022-5193(64)90038-4 139–145. doi:10.1037/1064-1297.11.2.139
Hamilton, W. D. (1964b). The genetical evolution of Madden, G. J., & Bickel, W. K. (Eds.). (2009).
social behaviour. II. Journal of Theoretical Biology, 7, Impulsivity: The behavioral and neurological sci-
17–52. doi:10.1016/0022-5193(64)90039-6 ence of discounting. Washington, DC: American
Jones, B. A., & Rachlin, H. (2006). Social discounting. Psychological Association.
Psychological Science, 17, 283–286. doi:10.1111/ Madden, G. J., Bickel, W. K., & Jacobs, E. A. (1999).
j.1467-9280.2006.01699.x Discounting of delayed rewards in opioid-dependent
Jones, B. A., & Rachlin, H. (2007). [A ratio scale for outpatients: Exponential or hyperbolic discounting func-
social discounting]. Unpublished raw data. tions. Experimental and Clinical Psychopharmacology, 7,
284–293. doi:10.1037/1064-1297.7.3.284
Jones, B. A., & Rachlin, H. (2009). Delay, probability, and
Madden, G. J., Petry, N. M., Badger, G. J., & Bickel, W. K.
social discounting in a public goods game. Journal
(1997). Impulsive and self-control choices in opioid-
dependent patients and non-drug-using control par-
doi:10.1901/jeab.2009.91-61
ticipants: Drug and monetary rewards. Experimental
Kagel, J., Battalio, R., & Green, L. (1995). Economic and Clinical Psychopharmacology, 5, 256–262.
choice theory: An experimental analysis of animal doi:10.1037/1064-1297.5.3.256
480
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Mazur, J. E. (1986). Choice between single and mul- Rachlin, H., & Jones, B. A. (2008a). Altruism among rela-
tiple delayed reinforcers. Journal of the Experimental tives and non-relatives. Behavioural Processes, 79,
Analysis of Behavior, 46, 67–77. doi:10.1901/ 120–123. doi:10.1016/j.beproc.2008.06.002
jeab.1986.46-67 Rachlin, H., & Jones, B. A. (2008b). Social discounting
Mazur, J. E. (1987). An adjusting procedure for study- and delay discounting. Journal of Behavioral Decision
ing delayed reinforcement. In M. L. Commons, J. E. Making, 21, 29–43. doi:10.1002/bdm.567
Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative Rachlin, H., & Jones, B. A. (2009). The extended self.
analyses of behavior: The effects of delay and of inter- In G. J. Madden & W. K. Bickel (Eds.), Impulsivity:
vening events on reinforcement value (pp. 55–73). The behavioral and neurological science of discount-
Hillsdale, NJ: Erlbaum. ing (pp. 411–431). Washington, DC: American
Mischel, W. (2004). Toward an integrative science of Psychological Association.
the person. Annual Review of Psychology, 55, 1–22. Rachlin, H., Raineri, A., & Cross, D. (1991). Subjective
doi:10.1146/annurev.psych.55.042902.130709 probability and delay. Journal of the Experimental
Mitchell, S. H. (1999). Measures of impulsivity in ciga- Analysis of Behavior, 55, 233–244. doi:10.1901/
rette smokers and non-smokers. Psychopharmacology, jeab.1991.55-233
146, 455–464. doi:10.1007/PL00005491 Raineri, A., & Rachlin, H. (1993). The effect of temporal
Myerson, J., & Green, L. (1995). Discounting of delayed constraints on the value of money and other com-
rewards: Models of individual choice. Journal of modities. Journal of Behavioral Decision Making, 6,
the Experimental Analysis of Behavior, 64, 263–276. 77–94. doi:10.1002/bdm.3960060202
doi:10.1901/jeab.1995.64-263 Ryle, G. (1949). The concept of mind. Chicago, IL:
Ostaszewski, P., Green, L., & Myerson, J. (1998). Effects University of Chicago Press.
of inflation on the subjective value of delayed and Schmitt, D. R. (1998). Social behavior. In K. A. Lattal &
probabilistic rewards. Psychonomic Bulletin and M. Perone (Eds.), Handbook of research methods in
Review, 5, 324–333. doi:10.3758/BF03212959 human operant behavior (pp. 471–508). New York,
Parfit, D. (1984). Reasons and persons. Oxford, England: NY: Plenum Press.
Oxford University Press. Siegel, E., & Rachlin, H. (1995). Soft commitment: Self-
Petry, N. M. (2001). Delay discounting of money and control achieved by response persistence. Journal of
alcohol in actively using alcoholics, currently absti- the Experimental Analysis of Behavior, 64, 117–128.
nent alcoholics, and controls. Psychopharmacology, doi:10.1901/jeab.1995.64-117
154, 243–250. doi:10.1007/s002130000638 Simon, J. L. (1995). Interpersonal allocation continuous
Rachlin, H. (1989). Judgement, decision, and choice: with intertemporal allocation: Binding commitments,
A cognitive/behavioral synthesis. New York, NY: pledges, and bequests. Rationality and Society, 7,
Freeman. 367–392. doi:10.1177/104346319500700402
Rachlin, H. (1992). Teleological behaviorism. American Staddon, J. E. R., & Simmelhag, V. L. (1971). The
Psychologist, 47, 1371–1382. doi:10.1037/0003- “superstition” experiment: A reexamination of its
066X.47.11.1371 implications for the principles of adaptive behavior.
Psychological Review, 78, 3–43. doi:10.1037/h0030305
Rachlin, H. (1994). Behavior and mind: The roots of mod-
ern psychology. New York, NY: Oxford University Stevens, S. S. (1946). On the theory of scales of mea-
Press. surement. Science, 103, 677–680. doi:10.1126/sci-
ence.103.2684.677
Rachlin, H. (1995a). Self control: Beyond commit-
ment. Behavioral and Brain Sciences, 18, 109–159. Teitelbaum, P. (1977). Levels of integration of the oper-
doi:10.1017/S0140525X00037602 ant. In W. K. Honig & J. E. R. Staddon (Eds.),
Handbook of operant behavior (pp. 125–152).
Rachlin, H. (1995b). The value of temporal patterns in Englewood Cliffs, NJ: Prentice Hall.
behavior. Current Directions in Psychological Science,
4, 188–192. doi:10.1111/1467-8721.ep10772634 Thorndike, E. L. (1911). Animal intelligence. New York,
NY: Macmillan.
Rachlin, H. (2000). The science of self-control. Cambridge,
MA: Harvard University Press. Trope, Y., & Liberman, N. (2003). Temporal con-
strual. Psychological Review, 110, 403–421.
Rachlin, H. (2006). Notes on discounting. Journal of doi:10.1037/0033-295X.110.3.403
doi:10.1901/jeab.2006.85-05 Vuchinich, R. E., & Simpson, C. A. (1998). Hyperbolic
temporal discounting in social drinkers and
Rachlin, H., & Green, L. (1972). Commitment, choice problem drinkers. Experimental and Clinical
and self-control. Journal of the Experimental Analysis Psychopharmacology, 6, 292–305. doi:10.1037/
of Behavior, 17, 15–22. doi:10.1901/jeab.1972.17-15 1064-1297.6.3.292
481
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 21
Behavior in Relation to
Aversive Events: Punishment
and Negative Reinforcement
Philip N. Hineline and Jesús Rosales-Ruiz
We are concerned in this chapter with some topics term: “Those football linemen are taking intense
that generate ambivalence within society at large as punishment, but they keep on coming” equates pun-
well as within the community of behavior analysts— ishment with physical trauma, irrespective of its
ambivalence that arises partly from multiple meanings effect on behavior; it does not satisfy the behavioral
of terms addressing those topics and partly from con- definition. More problematic is the ordinary-language
fusion between the study of a process and the advo- meaning that equates punishment with retribution,
cacy of procedures and practices based on it. When vengeance, or justice: If a person commits a crime, is
such advocacy does occur, the ambivalence some- apprehended, and then serves out a designated term
times appears to arise through disregard of what has of prison or community service, the person may be
been learned from study of the relevant process. said to have “repaid a debt to society.” This usage
Ambivalence concerning research on these topics may suggests retribution, but it says nothing about what
arise from failure to recognize that these processes the person is likely to do next and is thus tangential
often occur outside the laboratory, for better or for to punishment as behavioral procedure or process.
worse, and without having been explicitly arranged Nevertheless, legally imposed consequences of mal-
and thus still require scientific understanding. feasance are characterized forthrightly as punish-
Consider the word punishment, which identifies a ment, presumably to decrease that malfeasance.
topic we address in detail later. In behavior analysis, Meanwhile, as a cultural practice people are discour-
this term has specific technical meanings: A punish- aged from taking justice into their own hands, and
ment procedure is an arrangement whereby some yet some years ago, Governor Dukakis’s presidential
specified behavior produces a consequence, result- campaign suffered serious damage when, on being
ing in a subsequent decrease in the occurrence of asked how he would react in the hypothetical situa-
that behavior. The behavioral process of punishment tion of someone assaulting his wife, he declined to
is a temporally extended sequence that includes the make vengeful assertions. By implication, most citi-
occurrence of some particular behavior, a conse- zens prefer a leader who would be inclined toward
quence that the behavior produces, and a subse- personal vengeance in at least some circumstances.
quent decrease in that behavior. The process can Thus, no matter how carefully one writes or speaks
occur whether the behavior–consequence relation of punishment as defined in behavior analysis, one
has been arranged by design (a procedure), whether cannot prevent people from reading or hearing it in
it is the product of unwitting interaction between other ways. Behavior analysts as well as general
people, or whether it simply arises in the way the audiences have histories of listening, speaking, and
world works. writing as members of the community at large, and
These technical specifications correspond to thus we find muddled disagreements and discus-
only one of the ordinary-language meanings of the sions concerning the term punishment.
DOI: 10.1037/13937-021
483
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Hineline and Rosales-Ruiz
Similarly, the term aversive is a potential red flag; contrasted with a positive alternative, whereby the
in the vernacular, it is understood as including stim- consequence—whether reinforcing or punishing—
uli or situations that are experienced as painful or entails the addition, appearance, or increase in some
unpleasant. Conceptualized behaviorally, aversive event or stimulus as a consequence of specified
events or situations are identified as aversive if behavior. There is one contrasting technical usage,
behavior that produces or increases them decreases whereby in distinguishing between stimuli that are
or ceases (punishment) or if behavior that removes simultaneously present, a stimulus correlated with
or prevents them is maintained or increases (nega- reinforcement (whether positive or negative) is
tive reinforcement). The two types of effect are not deemed the positive stimulus and a stimulus corre-
exactly consistent when intensity of the stimuli is lated with nonreinforcement is deemed the negative
varied, however, so a particular stimulus could be stimulus.
functionally aversive by one definition and not by In ordinary language, positive and negative have
the other (Azrin, Hake, Holz, & Hutchinson, 1965, more problematic and ill-defined connotations in
discussed in the section Change of Situation in the which negative is roughly equated with undesirable
Escape Paradigm later in this chapter). or problematic; thus, one finds references to positive
The use of aversive events is ubiquitous in and negative behaviors and positive and negative
human societies and in nature, but explicitly advo- experiences. The conceptual confusion is exacer-
cating that use can be controversial (e.g. Salzinger, bated by those vernacular usages being adopted
1991; Starin, 1991; Yates, 1991). Still, the contro- apparently for marketing purposes, as in positive
versy does not apply to all aversive procedures. psychology (e.g., Seligman & Csikszentmihalyi,
Time-out procedures and prison sentences are 2000) and positive behavior (or behavioral) support
widely accepted as forms of punishment, on the (e.g., Kern, Koegel, & Dunlap, 1996). Anyone
assumption that they are aversive consequences. teaching behavioral concepts at the introductory
Other aversive procedures are accepted or tolerated level has encountered the degree to which these ver-
as normal practices at home, school, and work— nacular intrusions persist: Despite having been
temporary restraint, yanking a dog on a leash, penal- introduced to the fact that much of criminal behav-
ties in football games, and until recent decades, even ior is maintained by positive reinforcement and that
spanking have been generally noncontroversial aver- negative reinforcement is the defining contingency
sive stimuli. For some people, of course, even most of preventive medicine, many students continue to
of those procedures are unacceptable. imply in their examination answers that positive is
Finally (for the moment), negative is another good and negative is bad. In addition to the result-
loaded term that identifies two of our topics. ing conceptual incoherence, this confusion obscures
Defined technically within behavior analysis, the the fact that negative reinforcement can be funda-
term is algebraic in character, referring to events or mental to generating and maintaining behavior that
situations that are removed or reduced, the removal people value highly. More generally, it should be
or reduction being a consequence of specified evident in what follows that the susceptibility of
behavior. We must emphasize that negative does not people’s behavior to both punishment and negative
characterize the behavior itself. If the specified reinforcement is an important aspect of their adapt-
behavior subsequently increases, we are concerned ability as a species, as individuals, and as a culture.
with negative reinforcement, further subdivided as
escape or avoidance. If the behavior subsequently
Precis
decreases, we are concerned with negative punish-
ment, sometimes identified as penalty, response cost, We begin by discussing punishment, both positive
or time out (although time out can also function as and negative, with a brief sketch of its historical
negative reinforcement or even as positive reinforce- background, leading to a systematic review of the
ment, depending on surrounding circumstances). features known to determine its effectiveness.
These definitions of negative are, of course, each We have selected examples with a translational
484
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Behavior in Relation to Aversive Events
emphasis, illustrating relationships between the Historical Background

basic research that yielded a systematic categoriza- Early conceptions of punishment as a process sym-
tion of the parameters of punishment and the metrical but opposite to reinforcement were for-
applied research that has resulted. We begin our mulated but later rejected by Thorndike (1932),
consideration of negative reinforcement with the who asserted that reinforcers (so-called “satisfi-
escape–avoidance distinction and then emphasize ers”) strengthened behavior, but punishers
some classic experiments introducing a rubric that (“annoyers”) did not weaken behavior. He said,
articulates relationships extending well beyond the “Annoyers do not act on learning” (p. 46). Skinner
purview of traditional avoidance theory. We draw (1953) also minimized the role of punishment in
linkages between phenomena of negative and posi- learning. He said, “The effect of punishment was a
tive reinforcement (intermittent reinforcement, temporary suppression of the behavior, not a
matching law, delay discounting) while also reduction in the total number of responses” (p. 184).
addressing the practical concerns of translational For Skinner, punishment seemed to displace
research. behavior rather than weaken behavior. More
important, however, punishment was seen as estab-
lishing the motivational operations necessary for
Punishment
negative reinforcement to work (somewhat akin to
Punishment as a behavioral process is generally food deprivation in relation to positive reinforce-
defined as an environmental change that produces ment). If the behavior decreased, it was because
a subsequent decrease in the occurrence of behav- other behavior increased through negative rein-
ior that produced that change. The environmental forcement. Thus, the effects of punishment on
change, or punishing consequence, is called a pun- behavior were considered indirect and temporary,
isher and assumes one of two forms: (a) the pre- to be explained by the eliciting and motivational
sentation of a stimulus or situation (positive effects of stimuli. Expressing a similar view, Bolles
punishment) or (b) the removal of a stimulus or (1967) concluded that
situation (negative punishment). In the case of
it does not seem possible to weaken the
stimulus presentation, for example, suppose a
punished behavior itself, nor to produce
child’s receiving ripe cheese for dessert (consid-
by punishment competing behavior other
ered a delicacy by the parents) is made contingent
than the particular response it elicits.
on good manners at the table. If good manners at
Therefore punishment is not effective
the table decrease, the contingency would be con-
in altering behavior unless the reaction
sidered a case of positive punishment and the des-
to punishment itself competes with the
sert would be considered a punisher. In contrast, if
response we wish to punish. (p. 433)
good manners increase, then a positive reinforce-
ment contingency would be indicated. In the Similarly, Estes (1969) proposed that the competi-
case of stimulus removal, suppose the dessert is tion does not occur at the response level, but that
removed contingent on the child’s hitting her sis- shocks create a motivational operation that com-
ter. If hitting decreases, the contingency would be petes with the motivation of the punished behavior.
considered a case of negative punishment. By con- For these authors, then, the reduction in behavior
trast, if hitting increases, the removal of dessert could be accounted for by the eliciting and motiva-
would be considered either a negative reinforce- tional operations; they saw no need for punishment
ment contingency (removal or prevention of an as a behavioral process separate from positive and
aversive stimulus) or perhaps extinction-induced negative reinforcement.
aggression (a result of discontinuing positive rein- That conception has not stood over time, how-
forcement). A punisher, like a reinforcer, is defined ever. On the basis of several converging lines of
functionally—in terms of a contingency and its evidence (e.g., Azrin, 1956; Camp, Raymond, &
subsequent effects on behavior. Church, 1967), punishment came to be recognized
485
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
as a process analogous but opposite to that of rein- been consistent with the effects seen with other
forcement (see Baron, 1991; Fantino, 1973; Hine- animals when electric shock is used (see Crosbie,
line, 1984a). Nevertheless, besides the direct 1998). In applied behavior analysis, focusing on
punishing effect of reducing behavior directly, a socially significant behavior problems, a more varied
punishing stimulus might have additional effects: Its set of operations has been implemented. Positive
removal might negatively reinforce behavior, and its punishers (which entail response-contingent stimu-
presentation might elicit aggression. Further com- lus presentation) rarely involve electric shock; they
plicating matters, if punishment is to be at issue, include presentation of nonharmful noxious sub-
some process or processes must be maintaining the stances, intense or distracting noise, response effort
to-be-punished behavior. Behavior could be main- (overcorrection: restitutional and positive practice),
tained by positive or negative reinforcement, elicited blocking, physical restraint, reprimands, slaps, hair
by a stimulus, or could be a first occurrence that pulls, shaking, and even tickling. Negative punish-
came about through some environmental change. ers (which entail stimulus removal) entail a variety
The effects of a single punishment contingency of procedures that can be grouped into two major
could differ as a result of these surrounding categories: response cost and time out from positive
circumstances. reinforcement. Response cost consists of arrange-
For the operative characteristics of punishment ments such as loss of tokens, money, privileges, free
procedures to be discerned, the to-be-punished time, bonuses, and the like and terms such as fines
behavior must first be stably maintained under well- and penalties. Time out consists of the contingent
specified conditions, with the punishing stimulus presentation of a brief extinction period. In applied
carefully calibrated and controlled. These conditions behavior analysis, time out has been divided into
were first achieved in an experiment by Azrin two types: exclusionary time out and nonexclusion-
(1956) that helped initiate an extensive research ary time out, both of which entail the removal of
program summarized by Azrin and Holz (1966). access to positive reinforcers that would otherwise
This line of research is impressive at many levels be available. The nonexclusionary type of time out
and stands as one of the most significant contribu- consists of planned ignoring, contingent observa-
tions to the field of behavior analysis. It not only tion, and a time-out ribbon. Exclusionary time out
gave clarity and direction to the study of punish- entails special arrangements: a partition, a separate
ment, it also provided a framework for the study of room, or a hallway.
punishment and paved the way for an informed Clearly, the list and variety of specific punish-
technology of behavior reduction. Theoretical, ment procedures is long. Commenting on the num-
experimental, and applied treatments (including the ber and variety of procedures, Kazdin (2003) noted
present one) continue to borrow from the Azrin and the irony of this, because in many venues punish-
Holz analysis (e.g., Axelrod & Apsche, 1983; Baron, ment has been relegated to a category of procedures
1991; Crosbie, 1998; Fantino, 1973; Johnston, 1972; that should not be used at all or, if used, should be
Matson & DiLorenzo, 1984). More research is used only as a supplement to reinforcement pro-
surely needed (cf. Lerman & Vorndran, 2002; Per- grams. Even a cursory review covering all of these
one, Galizio, & Baron, 1988), but it is also clear that procedures would be beyond the scope of this chap-
much is already known about punishment as a dis- ter. Descriptions and evaluations of these proce-
tinct behavioral process. dures, addressed to an equally large variety of target
behaviors—ranging from getting out of one’s seat
Punishment Procedures during school to life-threatening behaviors such as
In basic research, a variety of punishment proce- chronic infant rumination—can be found in text-
dures have been used with humans, including the books such as Applied Behavior Analysis by Cooper,
use of loud noise, increased response effort, point Heron, and Heward (2007) and Behavior Modifica-
loss, and time out from positive reinforcement as tion in Applied Settings by Kazdin (2003). Two
well as contingent electric shock. The effects have other books exclusively dedicated to the topic of
486
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
punishment are The Effects of Punishment on Human opposite in their effects, a proposal that has gained
Behavior, edited by Axerold and Apsche (1983), and favor over time (see Baron, 1991; Crosbie, 1998;
Punishment and Its Alternatives, by Matson and Fantino, 1973). To the extent that reinforcement and
DiLorenzo (1984). punishment are symmetrical, the proposal offers an
economy of analysis, because understanding of each
Parameters Determining the Effects domain has implications for the other (see Hineline,
of Punishment on Behavior 1984a).
By definition, punishment, as with reinforcement,
Immediacy. A punishment procedure may fail
always works, for both are functional concepts,
because the punisher is delayed. Camp et al. (1967)
defined in terms of how they change behavior. How-
systematically studied several delays of punish-
ever, their effects on behavior are not fixed:
ment (0, 2, 7.5, and 30 seconds) and confirmed that
Research has shown that punishment effects depend
immediate delivery is more effective than delayed
on the characteristics of the punisher, the immedi-
punishment and that at long delays, the effect of
acy, the program, the maintaining variables of the
punishment does not differ from response-independent
punished behavior, availability of alternative behav-
delivery of the punisher. In the laboratory, imme-
ior, and alternative sources of reinforcement. The
diate delivery of the punisher is often ensured by
decreases in behavior are modulated by all of those
the use of automatic apparatus, but this is not the
factors, and punishment effects can be reliably pre-
case when a human practitioner or parent applies a
dicted if one knows those conditions, which are
punishment procedure in the natural environment.
especially important for ethical implementation of a
The human observer may be late in detecting the
behavioral technology.
response and slow to apply the punisher. Although
Azrin and Holz (1966) in their classic review
the body of research on procedures to deal with
listed 14 variables that modify the effectiveness of
delayed punishment is small, and more research is
punishment. These variables have provided the
needed, a few methods have been shown to mitigate
basis for continued experimental analysis and have
this problem (see Lerman & Vorndran, 2002). The
guided the development of behavior intervention
problem of delay is mainly technological and can be
programs to reduce unwanted behavior. Generally
solved by means of devices that detect and deliver
speaking, punishment applications should work
the punishment in a timely manner and by means of
fairly quickly. If they do not, one or more of the 14
procedures such as the use of conditioned punishers
variables needs adjustment. The adjustments can
that link the response to the delayed punishment.
be grouped as having to do with the administra-
This problem is not different for delayed positive
tion of the punisher, with the characteristics of the
reinforcement, so procedures that effectively link the
to-be-punished behavior, with the availability of
response with the delayed reinforcement can also be
alternative reinforced responses, and with stimu-
applied to delayed punishment.
lus control.
Intensity. The punishing stimulus should be as
Administration of punishment. To be effective, a intense as possible. Although the relation between
punisher should be delivered immediately, consis- intensity of punishment and its effects has been well
tently, suddenly, at maximum value, briefly, and on established for stimuli whose physical parameters
every occurrence of the targeted behavior. Faults can be measured, such as electric shock, noise, air
in any of those variables will affect the degree of blast, and point or money loss, intensity is an impre-
behavior reduction. We should note that these vari- cise term when applied to other stimuli. Louder
ables are not unique to punishment: Immediacy, reprimands do not necessarily translate into more
potency, and consistency are also relevant to positive intense punishers. However, larger penalties (e.g.,
reinforcement in the acquisition or enhancement of money or point loss) are more effective than smaller
behavior. Azrin and Holz (1966) suggested that pun- ones (Crosbie, 1998; Kazdin, 1972). We should note
ishment and reinforcement are similar processes but that longer durations do not necessarily mean larger
487
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
intensities and do not necessarily translate into bet- other than continue to detract from the
ter punishers. Given these difficulties, practitioners time available for good programming.
advised beginning with an intense but safe, practi- If the reinforcement and punishment
cal, and accepted punisher and then withholding contingencies of the time-out environ-
further increases in intensity until the other vari- ment are better than those of the time-in
ables are ruled out. environment time-out will strengthen
In the case of time out from reinforcement, one the behavior on which it is systematically
way to look at the intensity of punishment focuses contingent, but only if the contingency is
on the duration of the extinction period. Here again, managed well. (p. 259)
however, longer time outs do not necessarily mean
more effective punishers. Time-out periods as brief Manner of introduction. On one hand, introduc-
as 2 minutes have been shown to be effective in ing punishers at a low intensity followed by gradual
reducing disruptive and undesirable behavior among increases is likely to result in adaptation and render
children (e.g., Bostow & Bailey, 1969). Furthermore, higher levels of intensity ineffective. On the other
the effectiveness of time out depends on the interac- hand, the sudden presentation of an intense stimu-
tion of the time-in and time-out environments (e.g., lus is known to disrupt behavior. The combination
Solnick, Rincover, & Peterson, 1977; see also Hack- of a sudden high-intensity stimulus maximizes the
enberg & Defulio, 2007) and variables concerning stimulus change. Thus, counterintuitive as it may
the maintenance of behavior (Plummer, Baer, & seem, the punishing stimulus should not be intro-
Leblanc, 1977). If the behavior is maintained by pos- duced gradually but should instead be introduced at
itive reinforcement, time out from that reinforce- maximum feasible intensity (Azrin, 1960).
ment should work (given that the other parameters
Schedule of punishment. In general, the greater
are fulfilled). If the behavior is maintained by nega-
the proportion of punished to unpunished behavior
tive reinforcement, however, time out might increase
is, the greater the reduction of the punished behav-
the behavior. The interplay of time in and time out
ior. Similarly, the more frequent the punishment is,
was well summarized by Baer (2001):
the less frequent the punished behavior (e.g., H. B.
Clark, Rowbury, Baer, & Baer, 1973). Thus, the prob-
Time out is probably a widely used and
ability of punishment should be as high as possible;
largely misused procedure in our society
ideally, every occurrence of the targeted response
today. It is often prescribed as an accept-
should result in the punishing stimulus. However,
able way to reduce undesirable behavior,
when punishment is discontinued, continuously
and the behavior therapies often pack-
punished behavior recovers faster than intermittently
age it with an attempt at cognitive reor-
punished behavior, presumably because the absence
ganization. But behavior-analytic logic
of punishment is more discriminable. A likely strat-
teaches that time out has no necessary or
egy, then, is to begin with consistent, intense punish-
fixed function for behavior. If the rein-
ment and then to gradually introduce intermittency,
forcement and punishment schedules of
just as one would do with positive reinforcement
the time-out environment are worse than
in maintaining targeted responses. In addition,
those of the time-in environment, time
increased intensity of the punisher can enhance the
out will weaken the behavior on which it
effectiveness of intermittent punishment.
is systematically contingent, but only if
the contingency is managed well. If the Consistency. Failure of punishment may be the
reinforcement and punishment schedules result of uncontrolled variations of punishment
of the time-out environment are equal intensity. Again, this is less of a problem when auto-
to those of the time-in environment, mated procedures are used but a greater problem
time out will not change the behavior when human activity is required for the punishment
on which it is systematically contingent, administration. However, even in the laboratory the
488
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
intensity of stimuli can also be affected by nontar- a known positive reinforcer (praise plus food) pro-
geted behavior that reduces it. The classic example duced an increase in the-to-be punished behavior.
is behavior called breakfast in bed. The term was In this case, the procedural time out functionally
inspired by a rat that learned to avoid the grid shock constituted negative reinforcement by reducing the
by lying on its back while pressing the lever with its demands of the time-in environment.
hind foot to produce food. In the natural environ- A given punishment procedure will produce a
ment, individuals can resist, plea, apologize, and greater reduction in responding if that responding is
show remorse either to reduce the intensity of the maintained under a lean reinforcement schedule
punisher or to avoid it altogether. Treatment integrity rather than a rich one. Thus, to maximize the pun-
is the more contemporary term for the accuracy and ishment effect, the frequency of positive reinforce-
consistency with which interventions are implement for the punished response should be reduced.
mented (e.g., McIntyre, Gresham, DiGennaro, & Similarly, the degree of motivation to emit the pun-
Reed, 2007). ished response should be reduced because punish-
ment effectiveness has been found to be inversely
Characteristics of the to-be-punished behavior. related to the levels of deprivation of reinforcers
In addition to the administration of the punish- maintaining the to-be-punished behavior: The
ment procedure, the effectiveness of punishment higher the level of deprivation, the less effective the
depends on current variables operating on the punishment procedure (Azrin, 1960). However,
to-be-punished behavior as well as its history (cf. behavior suppressed under low levels of deprivation
Lerman & Vorndran, 2002). Except for first occur- does not recover by increasing the levels of depriva-
rences of the to-be-punished behavior, the delivery tion afterward. Another way to minimize the effects
of punishment is contingent on ongoing behavior of the schedule maintaining the to-be-punished
that is maintained by positive reinforcement or behavior is to make available a concurrent alterna-
negative reinforcement or is induced by other pro- tive response that will not be punished but will pro-
cesses. Positive punishment of behavior maintained duce the same or greater reinforcement as the
by positive reinforcement is by far the most studied punished response (Herman & Azrin, 1964). For
case. However, studies of behavior maintained by example, punishment of criminal behavior can be
negative reinforcement or behavior elicited by shock expected to be more effective if noncriminal behav-
are generally in agreement with the results found ior that produces the same advantages as the crimi-
with punishment of behavior maintained by posi- nal behavior is available.
tive reinforcement. Nonetheless, knowledge of the
variables maintaining the-to-be punished behavior Stimulus Control and Punishment Effects
can guide the choice of procedures. The first choice, As we discussed in the previous section, providing
of course, is to manipulate the maintaining variables for an alternative reinforced response can make
directly and thus obviate the need for punishment. punishment more effective. However, if no alterna-
For example, if the behavior is maintained by positive response is available the subject should be given
tive or negative reinforcement, shaping procedures access to a different situation in which the same
could be used. However, sometimes those maintain- reinforcement is available without punishment. This
ing variables cannot be controlled, leaving a punish- recommendation basically amounts to discrimina-
ment procedure as the best course of action. tion training, and it is used when the behavior is
Identification of the maintaining variables is appropriate only under certain conditions and not
especially crucial for the application of time out. If under others. However, the stimulus control of
the to-be-punished behavior is maintained by nega- behavior can enhance the effects of mild punish-
tive reinforcement, response-contingent time out ment. Doughty, Anderson, Doughty, Williams, and
will increase that behavior rather than decreasing it, Saunders (2007) provided a relevant example. In
as mentioned earlier. Plummer et al. (1977) pro- their study, dysfunctional stereotypic behavior
vided an illustrative example in which time out from occurred frequently in the presence of a stimulus
489
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
correlated with nonpunishment of that behavior. behavior, the ribbon was removed from the child, all
Presentation of a stimulus correlated with mild forms of social interaction were terminated, and no
punishment of stereotypy produced immediate edibles were delivered for 3 minutes. As a result, the
decreases, with stereotypy occurring rarely, if ever, disruptive behavior of all four children was sud-
in the presence of that stimulus. Another study, denly and substantially reduced.
illustrating a similar effect with pica (ingestion of Laraway, Snycersky, Michael, and Poling (2003)
non-nutritive substances), is that of Piazza, Hanley, attributed the effects of this procedure not to pun-
and Fisher (1996). One condition involved ishment but to an establishing operation. They
response-independent delivery of food, coupled argued that “the [establishing operations] for the
with an instruction not to touch the cigarette butts programmed events during the time-in also estab-
present in the room; no consequences were pro- lished the punishing effect of the ribbon loss (i.e.,
vided for picking up or consuming the cigarette functioned as [establishing operations] for ribbon
butts. Another condition involved taping a sheet of loss as a punishing event) and abated misbehaviors
purple construction paper to the wall along with the that resulted in ribbon loss” (p. 410). Alternatively,
initial instructions. In this condition, however, each one could argue that the ribbon signaled periods of
time the subject touched a cigarette butt, the thera- reinforcement, and absence of ribbon signaled peri-
pist entered the room and delivered the reprimand, ods of time out from reinforcement. Also notice that
‘’No butts.” If the subject did not comply with this the differential reinforcement procedure before and
reprimand, the therapist physically guided the sub- after the introduction of time out did not have any
ject to drop the butt. The two procedures were alter- effect on replacing the misbehavior with good
nated in a multiple-element design, thus providing behavior. Was it the case that during those phases
for the differential correlation of the purple piece of the social praise and edibles were not potent? The
paper with the punishment contingency. The results effects of punishment were more likely enhanced by
show that response-independent presentation of the stimulus control provided by the presence and
food had virtually no effect, but the discrimination the absence of the ribbon (see Azrin & Holz, 1966;
training produced dramatic effects on both cigarette Azrin, Holz, & Hake, 1963). DeFulio and Hacken-
butt pick-ups and cigarette butt pica. Similar effects berg’s (2007) study of the roles of added stimuli in
were found by McKenzie, Smith, Simmons, and avoidance time out showed similar stimulus-control
Soderlund (2008), who were able to completely effects.
eliminate chronic eye poking by correlating wrist- Another way to establish stimulus control in the
bands with the simple reprimand “Stop poking” in absence of external stimuli is to alternate periods of
the punishment condition. reinforcement with periods of extinction plus pun-
Stimulus-control techniques for enhancing punishment. This procedure establishes the delivery
ishment effects have also been documented for the of punishment as a discriminative stimulus for
case of negative punishment. A variation on time- extinction. Under these conditions, an otherwise
out procedures termed time-out ribbon, developed by ineffective punisher will produce large decreases in
Foxx and Shapiro (1978), is an example. During behavior during extinction periods. Thus, if possi-
their baseline phase, a teacher continued to use rep- ble, the delivery of the punishing stimulus should be
rimands (as had occurred before the experiment) in made a signal or discriminative stimulus that a
attempts to discourage the disruptive behavior of period of extinction is in progress.
four elementary students with mental retardation. In The principle of using enhanced stimulus control
the next phase, social praise and edibles were pro- to increase the effects of punishment offers an excit-
vided for good behavior, and the misbehavior was ing area for further applied and translational research.
ignored while the child was wearing the ribbon. Azrin and Holz’s (1966) elegant and systematic
This was done to increase the density of reinforce- strategies provide an excellent model for such work.
ment during the time in. Contingent on inappropriate Also, these results invite the reconsideration of
490
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
events such as don’t-do-it reprimands common in

everyday situations despite their typical ineffective-
ness. A great contribution of behavior analysis could
be to link the determinants of effectiveness to such
common procedures.
Punishment can also be established as a discrimi-
native stimulus for reinforcement. Thus, great care
should be taken to see that the delivery of the pun-
ishment stimulus is not correlated with the delivery
of reinforcement. Otherwise, the punishing stimulus
may acquire conditioned reinforcing properties.
Both parents and pet owners often unwittingly pro-
duce this effect by making their deliveries of punish-
ment the occasions that are most reliably followed
by expressions and gestures of affection. Holz and Figure 21.1. Illustration of punishment as a
discriminative stimulus for positive reinforcement:
Azrin (1961) provided a compelling experimental Cumulative responding during no punishment as
analogue of this, demonstrating unambiguously that a discriminative stimulus for positive reinforce-
an effective punisher can lose its effectiveness by ment and punishment as a discriminative stimulus
for positive reinforcement. From “Discriminative
establishing it as discriminative for positive rein-
Properties of Punishment,” by W. C. Holz and N. H.
forcement. To establish shock as a discriminative Azrin, 1961, Journal of the Experimental Analysis of
stimulus, they alternated periods in which respond- Behavior, 4, p. 228. Copyright 1961 by the Society for
ing produced positive reinforcement on an intermit- the Experimental Analysis of Behavior, Inc. Reprinted
with permission.
tent schedule and punishment on a fixed-ratio 1
schedule with periods of extinction in which
responding produced neither reinforcement nor For example, Matson and DiLorenzo (1984) and
punishment. Under these conditions, punishment Miltenberger (2001) proposed using the types of
increased responding instead of decreasing it. The generalization-promoting procedures that were sug-
results are shown in Figure 21.1. gested long ago by Stokes and Baer (1977). In a
This type of stimulus control over behavior is recent review of the applied literature, Lerman and
considered a side effect of punishment because it Vorndran (2002) acknowledged the value of that
could be established unintentionally during the suggestion but urged caution: “Current knowledge
administration of punishment. about punishment, however, is insufficient to guide
the application of such strategies” (p. 452). As sup-
Maintenance and Generalization of porting evidence, they pointed to Stokes and Baer’s
Punishment Effects recommendation to use intermittent and delayed
Two enduring rationales for favoring reinforcement reinforcement as one way to program indiscrim-
over punishment procedures are the specificity of inable contingencies so as to maintain behavior for
punishment and its transient nature. Although it has long periods of time. They rightly pointed out that
been pointed out repeatedly that the same is true for these contingencies undermine the efficacy of pun-
positive reinforcement, the argument has been resis- ishment. Nevertheless, there are some promising
tant to change. Unfortunately, behavior analysts studies along these lines. Lerman, Iwata, Shore, and
have perhaps inadvertently contributed to this mis- DeLeon (1997) were able to maintain punishment
understanding by treating reinforcement and pun- effects by slowly changing from punishing every
ishment as entirely separate domains, instead of response to punishing as infrequently as once every
pointing out that problems of maintenance and gen- 300 seconds. H. B. Clark et al. (1973) were able to
eralization are common to both. maintain punishment effects by transitioning from
491
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
punishing every response to punishing every third infant learning to walk illustrates this difficulty.
or fourth response (on average, in variable sched- Noting that the process will certainly involve falling,
ules) without the need for prolonged gradual he asked,
transitions. Clearly, more research is needed here;
Do these aversive stimuli [falling] influ-
perhaps the intermittent punishment rule applies
ence the infant’s learning by suppressing
only to the initial decrease, as it does for reinforce-
erroneous responses such as looking
ment during the initial increase, or perhaps there are
past obstacles, placing the feet too close
ways to accomplish intermittency of punishment
together, or letting the body tilt too far to
without compromising the effects.
one side?
Assuming that what is known about reinforce-
Or, are the necessary adjustments
ment is very likely to apply to punishment as well,
such as scanning the oncoming terrain,
we recommend exploring Stokes and Baer’s (1977)
spreading the feet, and promptly correct-
analysis to promote generalization of punishment
ing minor deviations in balance learned
effects. Within this general framework, program-
because they avoid untoward conse-
ming common stimuli is a very promising approach,
quences? (p. 194)
one that was adopted by Piazza et al. (1996) and by
McKenzie et al. (2008). With appropriate adjust- Dinsmoor preferred the second alternative, but it is
ments, both punishing loosely and punishing suffi- difficult to imagine how these competing alternative
cient exemplars should also be quite effective. Even accounts can be empirically separated. Notice that
more broadly, we propose that the effectiveness, gen- the role of positive reinforcement is not mentioned.
eralization, and maintenance of punishment should One could easily ask, does walking increase because
be systematically programmed, as is commonly done one is avoiding falling or because one is getting
in the applications of positive reinforcement. somewhere? The conundrum is similar to the issue
(discussed later) of ascertaining the roles of discrim-
Punishment as a motivational effect. The sup- inative versus establishing stimuli in avoidance and
pressive effects of punishment have sometimes been the distinction between positive and negative
attributed to the development of avoidance behavior reinforcers.
that competes with the punished response; this is To be sure, the presentation of aversive stimuli
called the avoidance hypothesis of punishment (see such as an electric shock can have motivational
Dinsmoor, 1954, 1998). In this view, the suppressive effects. That is, it establishes shock removal as a
effects of punishment are seen as secondary effects negative reinforcer (Michael, 1982, 1993). How-
of negative reinforcement. That is, the decrease in ever, the reduction of behavior is only partially
behavior is accounted for as an increase in alterna- explained by the avoidance and escape behavior
tive behavior maintained by negative reinforcement maintained by negative reinforcement. Azrin et al.
(see also Skinner, 1953). In that view, the role of the (1965) showed (in an experiment we describe in
punishment is relegated to potentiate negative rein- the section Change of Situation in the Escape Para-
forcers only. Although this view is not the dominant digm later in this chapter; see Figure 21.4) that the
position in behavior analysis, it is still occasionally motivational effects of shock presentation, as mea-
acknowledged as an alternative interpretation (e.g., sured by escape or avoidance, are independent of
Sidman, 2006; Spradlin, 2002). their suppressive effects. Shock at low values had
The continuing appeal of the avoidance hypothe- no apparent role; at intermediate values, it served as
sis regarding punishment effects may be attributable an establishing stimulus operation only; and at
to the difficulty of discerning, both conceptually and higher values, it could also serve as a punisher. A
experimentally, the multiple effects of stimuli that parallel effect of positive reinforcement would be
are involved (e.g., Arbuckle & Lattal, 1987; Gal- that certain types of reinforcers are good enough to
bicka & Branch, 1981; Galbicka & Platt, 1984). An approach (consume) but ineffective for reinforcing
example provided by Dinsmoor (1998) about an other behaviors.
492
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Negative Reinforcement A second resolution of the arbitrariness is based

on the fact that if something is to be removed as a
As we noted earlier, the negative of negative rein-
consequence of specified behavior, that behavior
forcement denotes the reduction of some event as a
must occur in the presence of the to-be-removed
consequence of behavior, resulting in a subsequent
stimulus or situation. Many such stimuli or situa-
increase in the occurrence of that behavior. That
tions will elicit or otherwise evoke behavior that
reduction can entail any of several characteristics,
competes with the to-be-reinforced behavior (Hine-
including complete prevention, reduction in inten-
line, 1984a). Thus, if a person is chilled while in
sity, postponement, or reduction in the event’s fre-
bed, shivering and curling up to conserve heat are
quency of occurrence. Thus, a person might take a
likely to compete with getting up and finding a
medication that eliminates pain, follow a particular
warm garment or blanket. Laboratory experiments
diet that reduces blood pressure, reduce the fre-
on avoidance have frequently been complicated by
quency of illness by arranging to get more sleep, or
the competing behavior (immobility, biting, and
postpone the payment of income tax while continu-
other stereotyped behavior patterns) that is elicited
ing to pay it once per year. Although ancillary prin-
by to-be-avoided shocks (e.g., Bolles, 1970; Dins-
ciples can be involved, each example comes under
moor, 1977; Forgione, 1970; Hake & Campbell,
the heading of negatively reinforced behavior. We
1972; Keehn, 1967). These eliciting and other
identify and describe experimental procedures that
response-inducing effects have generally been called
isolate and illustrate variations such as these.
side effects or by-products of aversive control (e.g.,
Hutchinson, 1977; Newsom, Favell, & Rincover,
Basic Terms and Distinctions 1983; Ulrich, Dulaney, Kucera, & Colasacco, 1972).
Concerning Negative Reinforcement Such effects are not confined to stimuli that produce
A distinction consistently maintained in the litera- pain or discomfort, however, for competing, aggres-
ture is that between escape, whereby behavior results sive behavior can also be produced by events such as
in removal of something already present, and avoid- response-independent point loss, which, as we
ance, whereby behavior postpones or prevents some- show, can also generate avoidance.
thing from occurring. However, ambiguity can arise A third consideration that distinguishes negative
despite these clear definitions because as Michael from positive reinforcement is more complex because
(1975) has argued, removal of one thing inherently it concerns the sequential characteristics and
implies the addition of its converse. Thus, putting extended time scales of motivating, or establishing,
on a coat can be characterized as accomplishing operations, conditions, or stimuli. As Michael (1982)
either the removal of cold or the facilitation of has shown, for situations involving positive reinforce-
warmth; removal of a threat can be understood as ment it can be a relatively straightforward matter to
the production of safety. Although there will always distinguish between discriminative and motivational
be borderline cases in which the specification of aspects of stimuli or of the situations that those stim-
addition versus removal seems arbitrary, one of the uli delineate. In contrast, and as we illustrate, dis-
two usually has the advantage of precision: The criminative and establishing stimuli are often
threat is likely to be specific, whereas safety is conflated when negative reinforcement is at issue.
defined only indirectly by the absence not only of An establishing condition or establishing stimulus
that threat, but of all threats. Thus, a limited range is a condition or stimulus that potentiates some
of responses is likely to be involved in removal of a other event as a reinforcer. An establishing operation
threat, whereas a wide and largely unspecified range is the explicit manipulation of such a stimulus or
of responses is likely to be accompanied by safety. the production of such a condition. Thus, a bar-
However, particular circumstances can exist in tender potentiates beverages as positive reinforcers
which safety is more clearly delineated than its by freely supplying salted snack foods. Establishing
corresponding range of threats, so the interpretive conditions can also arise without having been
language should adjust accordingly. explicitly arranged—as when a group of hikers
493
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
arrives at the pub after a long hike on a warm day. defined by the prevention of some event or situa-
In the case of positive reinforcement, the most com- tion: How can nonoccurrence be a consequence of
monly recognized establishing conditions vary on behavior? This problem appears to arise from a pre-
time scales quite different from the behavior that sumed necessity of consequences being contiguous
affects the related reinforcers: Hunger increases or with behavior if they are to reinforce it. Thus, avoid-
decreases over hours, or at least many minutes, ance theory has been viewed as a distinct domain,
whereas the behavior enhanced by its producing the with interpretations focused on identifying a plausi-
potentiated reinforcer (food) is often analyzed ble process whereby some event can be construed as
moment-to-moment. The salted pretzels have their supplying a contiguous consequence (Bolles, 1970;
effect gradually, whereas the episode of procuring a Dinsmoor, 2001; Herrnstein, 1969; Hineline, 1976,
drink transpires quickly. Discriminative stimuli, 1984a). Most avoidance theories, which is to say,
however, denote whether the behavior of concern theories posited as solving the avoidance problem,
can produce a given consequence, irrespective of have been anchored in procedures that include a
whether the consequence is a potent reinforcer. For warning stimulus—a stimulus that accompanies but
example, the presence of a bartender looking at you occurs in advance of the to-be-avoided stimulus or
is a discriminative stimulus that occasions your situation. Indeed, in the history of avoidance stud-
requesting a beverage. In the past, beckoning to bar- ies, the warning stimulus was ubiquitous in experi-
tenders who are looking elsewhere has not been mental procedures and was initially viewed as
reinforced; thus, people come to beckon and request essential to the behavior of avoiding, partly because
a beverage only when the bartender is looking at it fits so comfortably with a theoretical account
them. Discriminative stimuli commonly come and known as two-factor, or two-process, avoidance
go moment to moment, thus varying on time scales theory (for reviews, see Bolles, 1973; Herrnstein,
commensurate with the behavior of concern. 1969). The two putative processes are Pavlovian (or
In situations involving negative reinforcement, respondent) conditioning and operant conditioning.
however, both establishing and discriminative stim- By that account, the warning stimulus begins as a
uli often change from moment to moment, often neutral event, but through pairing with the primary
simultaneously, and sometimes with both roles even aversive stimulus it becomes a conditioned aversive
devolving onto a single stimulus, which conflates (or fear-eliciting) stimulus whose removal can serve
the establishing and discriminative functions. As a to reinforce behavior. According to that account,
result, interpretations of avoidance have tended to people do not avoid rain directly; instead, they avoid
emphasize the motivational aspect of relevant rain by escaping from warning stimuli such as
stimuli—denoting them as conditioned aversive clouds, which in their past experience have accom-
stimuli, in the behavior-analytic tradition (e.g., panied rain.
Anger, 1963), or as elicitors of conditioned fear in Although that concatenation of two processes
contrasting interpretive traditions (e.g., Kamin, can account for some of the behavior that is com-
Brimer, & Black, 1963). Interpretations with this monly characterized as avoidance, the theory is
emphasis have given insufficient recognition to the silent concerning many features of such behavior. In
discriminative aspects of those stimuli. As we show, addition, a major contribution of behavior-analytic
basic behavior-analytic research on avoidance has research has been to show not only that avoidance
revealed that in many cases, the function of a puta- can occur without the aid of warning stimuli, but
tively motivational stimulus is mainly discriminative even more tellingly that warning stimuli often play a
in character. role different from that posited by two-process the-
Conflation of those two functions has tangled the ory. Warning stimuli have often proved to be more
study of negative reinforcement throughout the his- discriminative than motivational. Thus, although
tory of its psychological interpretation. On one many textbooks of conditioning and learning con-
hand, a traditional issue has been “the avoidance tinue to feature two-process theory as the major
problem,” whereby avoidance, as noted earlier, is explanation of avoidance, studies accumulated over
494
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
more than a half-century have shown that theory to A

be untenable as a comprehensive account (for S
reviews supporting this assertion, see Hineline, R

1976, 1981). We describe representative studies of SS = RS Interval
that kind here, along with an alternative conception SS Interval
of the behavior called avoidance. B

S
Classic, and Still Relevant, Experiments

R
The first, and best-known, experiment questioning RS Interval
the importance of warning stimuli in avoidance was
done long ago by Sidman (1953a), who devised a Figure 21.2. Schematic diagram
showing two variants of Sidman’s
procedure that carries his name. It was first accom- (1953a) avoidance procedure,
plished with laboratory rats but was soon replicated which does not include warning
with other species, including humans (e.g., Ader & stimuli. In both diagrams, the pas-
Tatum, 1961; Behrend & Bitterman, 1963; Black & sage of time (and the progress
of the timer that can deliver an
Morse, 1961; F. C. Clark, 1961). The simplest ver- impending shock (S) is indicated
sion of Sidman’s procedure is arranged by means of as progressing from left to right, as
a clock, a response lever, and a device that can indicated by the solid arrowed line.
Instantaneous transitions (initiated
deliver brief (uncomfortable but harmless) electric by responses that occur at any time,
shocks; the relationships among them is presented as indicated by the braces, or by
schematically in Panel A of Figure 21.2. In the the delivery of shocks, as indicated
absence of the subject’s lever pressing, the timer at the right) are indicated by the
dashed lines. A: a version based
delivers a brief shock when its specified time has on a single timer; B: a version with
elapsed, and then it starts over. If the subject fails to two timers, one of which controls
respond, the brief shocks will continue to occur at the S–S interval (the time between
the specified intervals, typically 20 seconds. Any shocks in which no response
occurs), and the response R–S
lever press resets the timer, starting the interval over interval, which is initiated by, and
and preventing a shock. also reset by, a response. From
This procedure readily established and main- Advances in Analysis of Behavior:
Vol. 2. Predictability, Correlation,
tained lever pressing. Sidman (1953b) also systemat- and Contiguity (p. 205), by P.
ically varied the time between shocks when no Harzem and M. D. Zeiler (Eds.),
responses intervened and, by adding a second 1981, Chichester, England: Wiley.
response–shock timer, independently assessed the Copyright 1981 by John Wiley &
Sons. Adapted with permission.
time by which a response could postpone a shock,
as diagrammed in Panel B of Figure 21.2 (Sidman,
1953b). Although these procedures did not supply again with no warning stimuli supplied. In the
exteroceptive warning stimuli, Anger (1963) offered absence of responding, shocks were delivered at one
a plausible account of how the temporal regularities probability, sampled every 2 seconds; a single
of those procedures still allowed for two-process response lowered the probability, still sampled every
avoidance theory to account for the results, propos- 2 seconds, where it stayed until a shock resulted,
ing that the passage of time functions in a manner thus returning to the higher probability distribution.
comparable to stimuli such as tones and lights. Herrnstein and Hineline argued, then, that shock-
A still stronger challenge to two-factor theory frequency reduction per se constituted the reinforc-
was provided by Herrnstein and Hineline (1966), ing consequence of behavior—an interpretation that
who found that the lever-press responding of labora- Sidman (1962) had tentatively proposed on the basis
tory rats was readily established and maintained by of an experiment that had not been designed to
changes in the frequency of probabilistic events, directly manipulate the shock frequencies.
495
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Addressing a converse interpretation, Hineline onset. However, the animals were still capable of
(1970) subsequently found that short-term post- preventing the warning stimulus, as was shown by
ponement of shocks could also generate and main- arranging for responses to be ineffective only during
tain responding even when overall shock frequency the warning stimulus (Sidman & Boren, 1957a).
was held constant. In that experiment, brief shocks A still stronger case for a differing role of warning
were arranged to occur at the 10th second within stimuli was presented by G. E. Field and Boren
repeating 20-second cycles and with a retractable (1963), who built on Sidman’s (1962) ingenious
response lever available during the first 8 seconds. A escalator procedure, in which the subject’s respond-
single response retracted the lever for the remainder ing was analogous to running up a down-escalator
of the cycle and postponed the shock from the 10th (Sidman, 1962). That is, with no responding, brief
to the 18th second. Thus, with one shock per shocks occurred every 5 seconds. Each response
20-second cycle, shock frequency was constant at postponed the next shock by an increment of 10 sec-
three per minute and yet responding was established onds, up to a maximum of 110 seconds, while the
and maintained. In a second experiment, the clock kept running, thus continually subtracting
response postponed the shock from the 10th second from the accumulated shock-free time. G. E. Field
to the point 8 seconds after that response and with a and Boren then superimposed warning stimuli that
new cycle beginning 2 seconds after the shock. This varied in synchrony with the current amount of
procedure did not maintain responding—a result response-produced postponement—a row of 11
attributable to the fact that responses shortened the lights that were illuminated successively or an audi-
cycle, thus increasing shock frequency. Warning tory clicker that produced varying numbers of clicks
stimuli are irrelevant here, making two-factor theory per second. When lights alone were supplied, most
silent concerning the results, but the results suggest responses occurred when shocks were 40 to 70 sec-
two distinguishable independent variables operating onds away. When the clicks alone were supplied, the
on different time scales—short-term, moment-to- subjects allowed the shocks to come closer, with
moment postponement and changes in overall fre- nearly half of the responses occurring when shocks
quency of aversive events. were 30 to 40 seconds away. The most informative
Ironically, some of the strongest evidence that result occurred when the lights and clicks were sub-
challenges the two-process interpretation of warning sequently presented in combination; the resulting
stimuli came from the insertion of warning stimuli response patterns were similar to those of clicks
into the very procedures that had shown them to be alone. By the logic of two-process theory, the fact
unnecessary. Thus, Sidman (1955) superimposed that the added lights alone had maintained greater
the presentation of a light (in a procedure for rats) postponement than clicks alone would mean that
and a tone (for cats) during the 5-second period lights were more aversive than the corresponding
before a shock was due. Lever presses in advance of click rates. It would follow that adding clicks to light
the warning stimulus reset the timer, thus postpon- would have relatively little effect, perhaps resulting
ing both the warning stimulus and the shock, in slightly greater postponement. Instead, the oppo-
whereas responses in the presence of the warning site occurred: Adding the click to the light resulted
stimulus terminated that stimulus while also reset- in the animals maintaining a closer temporal dis-
ting the timer. One might have thought that the tance to shock. It is also relevant that when neither
warning stimulus, having been paired with shock, the click nor the lights were supplied, the animals
would function as an aversive stimulus to be maintained the maximal, 110-second distance from
avoided. Instead, the animals typically waited for the shock. Thus, the role of the warning stimuli was pri-
warning stimulus to come on before responding. In marily discriminative rather than motivational.
the light of this, one might suggest that the warning Dinsmoor and Sears (1973) demonstrated that
stimulus was sufficiently aversive to maintain escape response-produced stimuli correlated with the
responding in its presence, but not sufficiently aver- absence of shock could also play a potent role in
sive to maintain behavior that would prevent its avoidance situations. They used pigeons as subjects
496
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
and a Sidman-type shock postponement procedure A variety of experiments have revealed the roles
contingent on treadle pressing. In addition, during of stimuli that delineate such situations. The first of
initial training each response produced a 5-second these, by Sidman and Boren (1957b), entails yet
1,000-Hz tone. During subsequent testing in every another elaboration of Sidman’s shock-postponement
third session, the frequency of the response- procedures, using two timers and an added light, as
produced tone was varied from session to session, diagrammed in Figure 21.3. The first timer specified
from 250 to 4,000 Hz, including some sessions with a response–light (R-L) interval and thus controlled
no tones. This testing yielded distinct generalization the onset of a light; lever presses could reset that
gradients that were fairly symmetrical on a logarith- timer, thus postponing the presentation of the light.
mic scale, thus revealing control by these safety sig- If the light was allowed to come on, a different timer
nals. These manipulations did not differentiate became operative that specified a response–shock
between discriminative and conditioned reinforcing (R-S) interval; now lever presses could reset this
roles of those stimuli, but they did indicate effects timer, thus postponing a shock, but the light stayed
distinct from the motivational and energizing role on. The light went off only if the subject (a labora-
commonly attributed to warning stimuli. tory rat) paused long enough for a shock to occur,
accompanied by reactivation of the first timer and
Situational Change as Reinforcing the R-L interval that it controlled. When the R-L
Avoidance and R-S intervals were equal at 10 seconds, nearly
Subsequent and increasingly complex experiments all responding occurred in the presence of the light,
have made the two-process, primarily motivational directly postponing shocks; on the few occasions
account less tenable. To make sense of them, it is when shocks occurred, the subjects typically
useful to begin by developing the concept of behav- waited out the R-L interval and then resumed the
ioral situation. A behavioral situation is a bounded
period of time that can be functionally contrasted
with other such periods. Its boundaries are typically
delineated with accompanying stimuli—in experi- //////////////// S
ments, by the temporary presence or absence of
lights or tones or even of the specific chamber or R1 R2
room in which a specific procedure is implemented.
Boundaries of the situation can also arise directly Figure 21.3. Diagram of a procedure that illus-
trates the aversiveness of behavioral situations, as
from events or stimuli that yield its functional
distinct from aversive events per se within those
properties. Thus, a rainstorm begins when droplets situations. The passage of time is indicated by the
begin to arrive and ends when they cease, but if solid line, from left to right, and instantaneous
those transitions are gradual, the storm can be changes are indicated by the dashed lines. The
hatched area indicates the presence of a warning
made more unitary if accompanied by abrupt stimulus. Responses in advance of the warning light
changes from sunshine to cloudiness and back to (R1 in the diagram) reset a timer controlling the
sunshine. For the analysis of negative reinforce- response–light (R–L) interval, thus postponing the
ment, our main concern is with goings on within onset of that stimulus. If the warning stimulus was
allowed to come on, responding (R2 in the diagram)
a situation that affect the aversiveness of that could still postpone shock, but only by resetting the
situation—aversiveness being revealed by whether timer that controlled the response–shock interval
response-contingent termination or prevention of (also the duration of the warning stimulus), which
stayed on. Only by taking a shock (represented by
the situation will serve to maintain behavior. One S in the diagram) could the animal return to the
might think that the major determinant of this aver- situation in which R1 and the R-L interval was in
siveness would mainly be the frequency of primary effect. From Advances in Analysis of Behavior: Vol. 2.
aversive events within the situation. It turns out Predictability, Correlation, and Contiguity (p. 213), by
P. Harzem and M. D. Zeiler (Eds.), 1981, Chichester,
that several other and more interesting features England: Wiley. Copyright 1981 by John Wiley &
contribute to that aversiveness. Sons. Adapted with permission.
497
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
responding that reset the R-S clock. To this point, which Sd is a discriminative stimulus that sets the
this result was reminiscent of Sidman’s (1955) occasion for a particular response to be reinforced
experiment, showing little that was new. The most and SΔ is a stimulus that occasions nonreinforce-
important results occurred when the R-L interval ment (extinction) of that response; responding is
was increased, with the RS interval held constant at effective in the presence of the Sd and ineffective
10 seconds. As the R-L interval (the interval during in its absence. However, not only is responding inef-
the dark, whereby responding could postpone the fective in the absence of the Sd, the establishing
light) was increased over blocks of experimental condition is absent as well. As noted earlier, this
sessions, responding increased during that interval illustrates a major difference between positive and
even though the animals could have responded negative reinforcement; in the case of positive rein-
more slowly without cost. Even more significant, forcement, the discriminative and establishing func-
during the light, which accompanied the constant tions typically occur out of synchrony and on
10-second R-S interval, response rates decreased differing time scales.
from more than 12 responses per minute to fewer Using basic schedules—variable intervals and
than three. That is, the largest effect on response fixed ratios operative in the presence of shock—with
rate occurred in a situation in which the response the scheduled reinforcement being termination of
contingency was held constant. In addition, it was shock, Dinsmoor and Winograd (1958) and Wino-
clear that the animals were waiting out the interval grad (1965) were able to identify schedule effects
and taking a shock, with the result being the oppor- suggestive of those obtained with positive rein-
tunity to more easily avoid other shocks. Krasnegor, forcement but with limited ranges of schedule
Brady, and Findley (1971) demonstrated closely requirements. The limitations appeared to arise sub-
similar relationships with monkeys instead of rats as stantially from competition, with behavior patterns
subjects and with ratio schedules under time limita- directly elicited by the shocks, thus illustrating the
tions instead of shock-postponement procedures. asymmetry between negative and positive reinforce-
ment, which we have noted.
Change of Situation in the Escape It was a short step beyond escape from continu-
Paradigm ously available shock to arrange for escape from sit-
Conceptualizing change of situation as a potentially uations delineated by continuous tones or lights that
reinforcing consequence of behavior enables us to accompany brief intermittent shocks, thus effec-
tie some of these complex elaborations of avoidance tively aggregating those shocks plus delineating
back to the putatively simpler escape relations and stimuli into functionally unitary situations (Hine-
then to elaborate the escape relation itself in terms line, 1984a). The salience of such delineating stim-
of intermittent reinforcement contingencies that are uli can be important, as shown by Baron, DeWaard,
analogous to the familiar schedules of positive and Lipson (1977), who found that when an ele-
reinforcement. vated shelf was made available during the time-out
First, to review, the escape contingency is one in periods from an avoidance procedure, which
which a response occurring in the presence of an enabled the rats to get off the grid floor, response
aversive stimulus can directly eliminate that stimu- rates on the variable-interval schedule whereby they
lus. Furthermore, in a pure escape procedure, addi- could produce the time-out periods were elevated—
tional responding, which occurs in the absence of even though no shocks were delivered if they stayed
the aversive stimulus, has no effect on the subse- on the grid floor during the time outs. Interpreting
quent likelihood of that stimulus occurring; other- this result as enhanced discriminability of safe peri-
wise, it would be a combination of escape and ods was supported by an extensive series of studies
avoidance contingencies. Removal of an aversive by Badia and his colleagues, who examined animals’
stimulus surely constitutes a highly discriminable preferences for situations in which brief shocks were
consequence of the response. Thus, an escape proce- preceded by signals of a few seconds’ duration in
dure has properties of an Sd–SΔ arrangement, in comparison with situations in which comparable
498
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
shocks were unsignaled. Some manipulations indi- maintenance of responding on a variety of variable-
cated that rats preferred situations in which shocks interval schedules. In a tour de force demonstrating
were signaled over situations in which the shocks the potency of reinforcement schedules based on
were unsignaled, even when the signaled shocks clearly discriminable time-out periods as reinforcers,
were more intense or of longer duration (Badia, Cul- Kelleher and Morse (1964; also see Hineline, 1976)
berson & Harsh, 1973) or occurred more frequently showed that when surrounding parameters (shock
(Badia, Coker, & Harsh, 1973). intensity, potency of food reinforcers, etc.) were
Azrin et al. (1965) used the addition of a delineat- appropriately adjusted, they were able to obtain
ing stimulus when studying removal of a punishment identical positively reinforced and negatively rein-
contingency as the basis for negative reinforcement. forced behavior patterns on fixed-interval 10-minute
That is, a pigeon’s pecks on one response key resulted and fixed-ratio 30 schedules of reinforcement that
in escape from a situation in which the food-reinforced were presented in alternation. Furthermore, when
responses on a second key produced shocks. The chlorpromazine and d-amphetamine were adminis-
experimenters accomplished this by first maintaining tered (in separate experiments) with systematic vari-
one response with a schedule of positive reinforce- ation of doses, the two drugs had differing effects on
ment, then arranging for those responses to produce behavior maintained by the differing schedules, but
brief shocks, and finally arranging for a second similar effects on a given schedule irrespective of
response to produce a change of illumination, which whether that schedule implemented positive rein-
accompanied the disabling of the punishment contin- forcement (food delivery) or negative reinforcement
gency until the next positive reinforcer had been pro- (termination of the stimulus that accompanied
duced. Responding that produced this change of intermittent brief shocks). This finding runs counter
situation was readily established and maintained. to the pervasive practice of using hedonic terms to
Variations in punishment intensity yielded a finding characterize psychoactive drugs.
of great practical importance: As shown in Figure Baum (1973) studied behavioral allocation as a
21.4, when escape was possible, a much less intense function of the frequency of discriminable time-out
punishment was required to suppress the positively periods in a situation that entailed brief response-
reinforced behavior; with no escape available, the independent shocks. The subjects in this experiment
punished behavior was much more persistent. were pigeons in a chamber whose floor was divided
Arranging for discriminable shock-free periods into two platforms. Standing on one platform illumi-
thus delineated as intermittent consequences of nated a red light; standing on the other yielded a
responding, Dinsmoor (1962) reported stable green light. Depending on which platform a bird was
standing on, one of two variable-interval timers could
deliver a 2-minute shock-free period with the lights
off. Systematic changes in the mean intervals gener-
ated by the two timers resulted in corresponding vari-
ations in the birds’ relative time spent on the two
platforms—variations that on average corresponded
to the matching law that is familiar in studies of posi-
tive reinforcement (e.g., Davison & McCarthy, 1988).
Logue and DeVilliers (1978) also obtained results
consistent with the matching law, using a procedure
Figure 21.4. Frequency of punished respond-
that required more active responding (than merely
ing under escape versus no-escape conditions. From
“Motivational Aspects of Escape From Punishment,” standing on a platform) that could cancel shocks that
by N. H. Azrin, D. F. Hake, W. C. Holz, and R. R. were scheduled to occur at various frequencies.
Hutchinson, 1965, Journal of the Experimental Analysis Reduction in the duration of a situation can
of Behavior, 8, p. 34. Copyright 1965 by the Society for
the Experimental Analysis of Behavior, Inc. Reprinted function as negative reinforcement, even when the
with permission. reinforced behavior does not immediately produce
499
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
that reduction, as illustrated by Mellitz, Hineline,

Whitehouse, and Laurence (1983). In their experi-
ment, responding on either of two adjacent and con-
currently equivalent levers could postpone shock in
Sidman’s basic procedure. Each rat proved to have an
initially preferred lever when performance had
become stable but did not respond exclusively on
that lever. With this accomplished, the Sidman pro-
cedure remained in effect as before, but in addition,
each response on the nonpreferred lever subtracted 1
minute from the daily 120-minute session. This con-
tingency was discontinued when there were only 2
Figure 21.5. Percentage of session time
minutes remaining in the session, whereas the Sid- spent in the 3-minute response-produced alter-
man procedure continued in effect until the end. Sys- nate situations for subjects in each of three
tematically over time, all five animals switched to a groups, whose responses yielded transitions
from two-per-minute irregularly spaced shocks
preponderance of responding on what had been their to six closely spaced shocks with 10 seconds,
nonpreferred lever. For four of the five animals, this 88 seconds, and 165 seconds delay, respec-
effect was validated by reversal procedures, repeat- tively. From “Negative Reinforcement With
edly shifting the preferences from one lever to the Shock-Frequency Increase,” by E. T. Gardner
and P. Lewis, 1976, Journal of the Experimental
other and back. The clear message here is that reduc- Analysis of Behavior, 25, p. 5. Copyright 1976
tion of situation duration can function as negative by the Society for the Experimental Analysis of
reinforcement, and the consequence need not be con- Behavior, Inc. Reprinted with permission.
tiguous with the occurrence of the affected response.
However, distribution of events within a situa- situations entailed more shocks than the six that
tion can affect the degree to which transitions into would be received in a comparable period of the
that situation will be reinforcing. Gardner and Lewis imposed situation. For one group, a train of 9 one-
(1976) examined this feature while clarifying and per-second shocks began 161 seconds into the
enhancing the dissociation of reinforcement via imposed period. For another group, a train of 12
shock postponement versus reinforcement via shocks occurred 158 seconds into that period, and
shock-frequency reduction. The Gardner and Lewis for the third group, a train of 18 shocks began 152
procedures entailed an imposed situation in which seconds into that period. In each case, after the last
brief shocks occurred unpredictably on the average shock in the alternate period, 10 seconds elapsed
of twice per minute. The animal’s lever press in this before a return to the imposed period. All three rats
situation initiated a 3-minute alternate situation that in each of the first two groups responded sufficiently
was accompanied by light and auditory click. The to spend most of their time in the alternate situa-
six shocks that would have been distributed irregu- tion, thus receiving nine or 12 postponed shocks
larly throughout the 3 minutes if the response had during periods in which they could have received
not occurred were now closely packed, one per sec- six evenly distributed shocks. Of the third group,
ond beginning 10, 88, or 165 seconds into the only one of the three animals responded apprecia-
3-minute alternate period, for different groups of bly, producing 18 postponed instead of six evenly
animals. As shown in Figure 21.5, both the 88- and distributed shocks. In a follow-up experiment,
the 165-second delays resulted in the animals Lewis, Gardner, and Hutton (1976) controlled for
spending substantial percentages of time in the alter- several extraneous features while arranging that in
nate situation, whereas those with the 10-second the alternate condition, the first shock occurred at
delays spent most of their time in the imposed con- the same time as would have been delivered by the
dition. In a second experiment, the imposed situation imposed condition if the animal had not responded.
was the same as before, but the 3-minute alternate Again, the animals responded so as to shift the
500
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
preponderance of shocks toward the end of the contingencies, time out from positive reinforcement
alternate period, despite there being no reductions can also provide a basis for negative reinforcement,
in overall frequency of shocks. and thus prevention of loss of food situations can
More conventionally, change of situation constitute avoidance. Baer (1962) provided an early
arranged as an intermittent but immediate behavioral demonstration of such an effect with a procedure in
consequence (as in typical schedules of positive rein- which young children could postpone the interrup-
forcement) has provided a flexible paradigm serving tion of animated cartoons. His first procedure was
to reveal increasingly multifaceted aspects of negative analogous to escape avoidance in that any time the
reinforcement. Thus, Perone and Galizio (1987) cartoon was off, it could be reinstated by a response.
arranged for rats’ responses on one lever to postpone Additional responses could postpone a subsequent
shocks according to Sidman’s procedure, whereas interruption of the cartoon. This procedure failed to
responses on a second, concurrently effective lever establish postponement responding. In Baer’s sec-
could occasionally (via conventional variable-interval ond procedure, responses during the cartoon could
schedules) produce discriminable 2-minute periods accumulate time, as in Sidman’s (1962) escalator
of time out from the postponement procedure. procedures. This version maintained responding
Responding was robustly and independently main- that postponed the interruption of the cartoon. Baer
tained on both levers. Galizio and Perone (1987) (1960) found that preschool children’s responding
then used this preparation as a baseline for studying on the time-accumulating procedure could also be
the effects of psychoactive drugs. Chlordiazepoxide maintained by preventing loss of access to conversa-
had little effect on shock-postponement responding tion with a puppet.
at doses that enhanced responding on the time-out Baron and Kaufman (1966) implemented a some-
lever. In contrast, morphine depressed time-out what similar procedure with college students as sub-
responding at doses that either increased or had no jects, whereby money could be accumulated except
effect on shock-postponement responding. Galizio, during time-out periods that could be prevented by
Robinson, and Ordronneau (1994) subsequently rep- button pressing. Initial training entailed an escape-
licated the dissociative effect of morphine, using avoidance procedure supplemented by verbal
variable-ratio schedules in place of variable-interval instructions. Thereafter, a Sidman-type postpone-
schedules. Interpretations that appeal to general ment contingency was implemented, yielding results
emotional or motivational states, which are com- similar to those Sidman reported with rats as sub-
monly invoked to categorize psychoactive drugs, jects. Also using rats as subjects, D’Andrea (1971)
have little to say about such differences between neg- arranged a procedure more closely analogous to the
atively reinforced repertoires. Courtney and Perone basic Sidman procedure in virtually all respects,
(1992) also used a similar arrangement, but with with the to-be-postponed event being a 1-minute
response-produced deletion of noncontingent shocks time out (lights off) from the situation in which
in place of shock postponement. This arrangement food was delivered irrespective of behavior. Holding
enabled them to evaluate the relative contributions of constant the periods separating time outs in the
shock frequency and response effort to the effective- absence of responding, D’Andrea varied the amount
ness of time out from an aversive situation. They by which a response could postpone the time out
found that in some circumstances, response effort and obtained results analogous to those Sidman had
required to avoid shocks contributed more to the obtained when manipulating the response–shock
aversiveness of a situation than was contributed by interval. Galbicka and Branch (1983) replicated the
relative frequencies of the shocks themselves. basic procedure, but with pigeons as subjects. The
results were analogous provided that one took into
Avoidance of Time Out From account some elicitation of the birds’ pecking that
Situations of Positive Reinforcement occurred with transitions from time-out to time-in
Just as time out from situations containing posi- periods. Again with rats, Richardson and Baron
tive reinforcement can be a basis for punishment (2008) extended the range of a similar procedure,
501
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
systematically varying the frequency with which the immediately impending 60-second time-out period
response-independent food was delivered during to the latter half of the 125-second trial but could
time in, and found that rates of postponement varied not eliminate it. The birds reliably postponed those
in a way indicating that time outs from high food- time-out periods. Similar responding was main-
delivery frequencies were more aversive, producing tained in a second experiment when the postpone-
effects analogous to higher shock intensities previ- ment was accomplished at the cost of the postponed
ously studied with analogous procedures. time out being longer than the immediate time out
More subtle features of negative reinforcement that would occur if the bird had not responded
contingencies and their effects, originally demon- on the postponement key. Additional manipulations
strated with electric shock as the to-be-avoided of the length of postponement intervals verified this
stimulus, have been replicated in experiments to be a robust effect, and potential artifacts, includ-
whereby responding could avoid or postpone time ing adventitious positive reinforcement of the post-
out from situations of positive reinforcement. For ponement response, were ruled out.
example, DeFulio and Hackenberg (2007) arranged
for concurrent contingencies of food production and Summarizing Principles
postponement of time outs from that contingency of A reader new to the topics and the approach repre-
food production. In addition, they supplied accom- sented in this chapter may well have found the novel
panying stimuli that were distinctively correlated and complex procedures and relationships to be
with the passage of time within the R–time out confusing. To mitigate this, Hineline (1984a) pro-
(Sidman-type postponement) interval. When the posed a set of summarizing principles that have
stimuli and corresponding intervals were manipu- remained useful even as the procedures and research
lated, the results were similar to those we described questions have become still more complex:
earlier, concerning discriminative stimuli superim-
posed on shock-postponement procedures. That is, 1. Negative reinforcement is to be understood in
in general, with the addition of such stimuli, avoid- terms of transitions between situations as well as
ance responding shifted toward the situation closest by postponement or prevention of events within
to the to-be-avoided event (e.g., G. E. Field & a situation. In some cases, a continuously pres-
Boren, 1963; Sidman, 1955), and when the relative ent aversive stimulus defines such a situation; in
efficiency of responses was varied differentially, other cases, the situations will be partly defined
responding gravitated toward the situations in by additional, delineating stimuli and by operant
which it was most efficient (Krasnegor et al., 1971; contingencies that are in effect only during the
Sidman & Boren, 1957a, 1957b). situation.
Also using pigeons as subjects, Pietras and Hack- 2. Relative aversiveness of a situation (the degree
enberg (2000) played postponement-of-time-out to which transitions away from it will reinforce
contingencies off against the resulting frequencies of behavior) depends only partly on primary aver-
food delivery that would result from the time outs, sive stimuli that occur within the situation. Even
in a manner similar to the much earlier experiments when those stimuli do contribute to aversiveness,
by Hineline (1970), Gardner and Lewis (1976), and a relevant feature is the relation between their
Lewis et al. (1976), who had played shock post- short-term versus longer term distributions over
ponement off against shock frequency. The back- time.
ground condition of Pietras and Hackenberg’s 3. Relative aversiveness of a situation depends sub-
discrete-trial procedure was a random-interval stantially on contingencies (work requirements)
schedule of response-produced food, effective within that situation, in comparison to contin-
throughout 125-second trials except during time- gencies in alternative situations.
out periods. Each trial began with a 5-second oppor- 4. Most important, the role of the alternative situ-
tunity during which a bird’s peck on a separate ation or situations depends on contingencies
response key could postpone by 45 seconds an regarding change of situation—that is, on what
502
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
is involved in getting from one situation to the and maintain alternative, more acceptable behavior
other. (such as asking for help).
5. All things being equal, performance tends to That strategy has not been common practice
allow persistence of the situation closer to pri- when addressing problematic behavior of pets, how-
mary aversive events. (p. 505) ever. Instead, aggressive and other problematic
behavior is more likely to be addressed with repri-
This rubric encompasses the classical procedures
mands and other attempts at punishment, which are
that were derived from and addressed to traditional
for the most part ineffective as well as stressful
avoidance theory. It also provides a systematic way
(Merck Veterinary Manual, 2006; Schilder & van der
to approach the additional procedures and phenom-
Borg, 2004). A notable, somewhat exotic alternative
ena that have been described here, which are clearly
was suggested by Pryor (1999), who provided an
relevant to behavior termed avoidance and about
anecdotal account of her work with timid llamas;
which traditional avoidance theories are silent.
more recently, Snider (2007) provided a systematic,
Applied and translational research involving data-based report of a similar technique with aggres-
negative reinforcement. Even the most basic, sive dogs. The dogs all had long-standing repertoires
conceptually driven behavior-analytic research will of problematic aggressive behavior. The behavior of
often have direct practical implications. Thus, Iwata five of the six was dramatically changed within min-
(1987) identified several ways in which negative utes, or at most a few hours, of intervention; the
reinforcement is inextricably involved in practical sixth served as an unplanned control, whereby
situations. He distinguished among (a) undesir- household circumstances resulted in extended expo-
able behavior acquired and maintained through sure to the baseline, preintervention condition.
negative reinforcement, (b) treatment of negatively The technique is predicated on the assumption
reinforced behavior, and (c) negative reinforce- that the dogs’ aggressive behavior is maintained by
ment as therapy. Examples in the first category the termination of threat—from an unfamiliar per-
included tantrums, aggressive behavior, and self- son, a specific person, another dog, and so forth—
injurious behavior maintained by the termination characterized as a decoy in the treatment. A
of instructional demands. Promising examples of threshold distance is empirically identified as the
the second category entailed differential reinforce- maximal distance that yields signs of aggressive
ment of alternative, even incompatible behavior behavior when the decoy approaches the potential
that could be maintained by the same consequence aggressor. Then, in repeated trials, the decoy is
and thus still be negatively reinforced. Termination advanced steadily from a safe distance to just short
of moderately intense tones were included in the of the threshold distance, pausing there until the
third category, providing negative reinforcement of dog exhibits relaxed or other distinctly nonaggres-
behaviors such as improved posture, taking pills at sive behavior. At that point, the decoy quietly moves
appropriate times, and eliminating nocturnal enure- back to the safe area. With repeated trials, the alter-
sis. In recent years, negative reinforcement, as in native behavior, having thus been negatively rein-
escape from demands, has become a standard com- forced, comes to occur more quickly and reliably, at
ponent of functional analyses (e.g., Iwata, Dorsen, which point the decoy is advanced slightly closer.
Slifer, Jauman, & Richman, 1994; Repp & Horner, Again, only when the specified alternative behavior
1999) and descriptive analyses (e.g., Pence, Roscoe, occurs does the decoy return to a safe area. Once the
Bourret, & Ahearn, 2009), both of which are now relaxed, nonaggressive behavior occurs quickly and
in common use for identifying the sources of prob- reliably with the decoy in close proximity, positive
lematic behavior. When such behavior has thus been reinforcement is brought into play. Although the
identified as being maintained by negative reinforce- initial intervention was consistently and quickly
ment, as in a child’s tantrums yielding escape from effective, it was also necessary to train for general-
academic tasks (e.g. Iwata et al., 1994), an effective ization, repeating it in various settings (except in
strategy uses the same negative reinforcer to shape a case in which the aggressive behavior was very
503
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
situation specific) or with various decoys, except for more problematic behavior such as rough handling
when the target of the aggressive behavior was a par- that is known to occur in such situations.
ticular person. Superficially, the whole arrangement Studies of humans’ reactions to potential money
could be mistaken for one based on habituation or loss have provided another paradigm for transla-
systematic desensitization, but as Snider (2007) tional research based on negative reinforcement. For
carefully delineated, crucial details are different. example, Magoon and Critchfield (2008) used the
Thus, the procedure entails differential reinforce- generalized matching law (Baum, 1974), which has
ment and shaping of incompatible behavior, com- proved to be broadly applicable in basic behavior-
bined with stimulus fading and initially using analytic research (e.g., Davison & McCarthy, 1988),
negative instead of positive reinforcement. A possi- as a basis for assessing the degree of asymmetry
ble source of its effectiveness is the resemblance of between positive and negative reinforcement. When
this procedure to the fading procedures known as quantitatively equated positive and negative rein-
errorless discrimination techniques, which have pre- forcement contingencies of intermittent reinforce-
viously been based only on positive reinforcement ment were concurrently in effect, no consistent bias
(Terrace, 1963; see also Etzel, 1997). was evident. This arrangement, which provided tan-
Translational research—research that explicitly gible consequences for actual choices, contrasts with
builds on basic research to address issues of practical studies that merely ask people what they would do
and social importance—is a current priority of the in hypothetical situations. Whether the lack of bias
national scientific agenda. The best of such work commonly reported in these studies hinges on the
advances both the applied and the basic agendas; it is difference between actual versus hypothetical
work with clear practical relevance that informs a basic choices is an important matter for further study.
conceptual issue. Bruzek, Thompson, and Peters Building on experiments by Baron and Galizio
(2009) used termination of the (recorded) sounds of (1976) that had demonstrated reinforcement of
infant crying in a study that nicely illustrates that strat- observing responses—responses maintained by the
egy in the domain of negative reinforcement. The basic production of discriminative stimuli—that func-
issue concerned the phenomenon of resurgence—the tioned analogously in both positive and negative
recurrence of a previously, but not currently, rein- reinforcement situations, Galizio (1979) used avoid-
forced response during extinction of a more recently ance of monetary loss as the basis for studying the
reinforced response. Bruzek et al.’s practical issue was origins of instructions as discriminative stimuli.
the behavior of adults in caring for young children. Although, on one hand, accurate instructions were
The experimental arrangement entailed a life-sized seen to facilitate a person’s initially behaving in con-
baby doll, various toys, and audio recordings of an cert with the contingencies of loss prevention, subse-
infant’s crying, which could be played by means of an quent exposure to consequences of following
audio speaker under the crib. With no special instruc- inaccurate instructions resulted in the instructions
tions, the undergraduate subjects readily acquired first being ignored. Still later exposure to accurate instruc-
one and then another response—engaging the baby tions resulted in presentation of the instructional
doll with each of two toys—when one or the other stimuli themselves having the effects of reinforcers
response could terminate or prevent the recorded cries. for observing responses. This sequence of experi-
After extinguishing both of those responses, a third ments documented a role of reinforcement history in
response was reinforced and then extinguished; during determining the ways in which and the degrees to
that final phase, the initially reinforced response which instruction following is maintained.
recurred. In general, the results indicated that a
response with a longer history of reinforcement was Indirect Origins of Stimulus
more likely to resurge than a more recently but briefly Relations in Avoidance
reinforced response. As the authors pointed out, this We have identified various functions of stimuli in
finding has implications not only for relatively benign situations that involve negative reinforcement—
methods for dealing with an infant’s crying, but also discriminative, as related to contingencies; delineating,
504
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
as affecting aggregation of dispersed events or defin- is different from B, C may or may not be different
ing the boundaries of distinct situations; and nega- from A). Intermediate to these are opposites (if B is
tively reinforcing, in various time relations. the opposite of A, and C is the opposite of B, then
Appropriately, those functions have been demon- C = A). Using human subjects, Dymond, Roche,
strated as being generated directly by their place- Forsyth, Whelan, and Rhoden (2007, 2008) built on
ment within those situations, and indeed, that this conception to demonstrate avoidance based on
genesis has often been the focus of basic research. In such relations with aversive images and sounds as
applied settings, however, one often does not have the to-be-avoided stimuli. Prior training that estab-
sufficient access to an individual’s history to identify lished sets of opposites, then, yielded the predicted
the origins of particular stimulus functions, and a transfers or nontransfers of avoidance reactions to
particular stimulus with an evident role in aversively other stimuli when avoidance was taught with
maintained behavior may not even have a plausible respect to a particular member of the set. Dymond
basis for participating in a history that produced and Roche (2009) have proposed a pragmatic
that function. In such cases, ad hoc appeals to stim- approach to clinical anxiety by using relational
ulus generalization do not satisfy. What is needed is frame theory when devising strategies for the extinc-
a principle whereby purely arbitrary stimulus relation of discriminative aspects of anxiety-producing
tions can arise and whereby one stimulus can stand stimuli.
in place of another despite their lacking any physical
resemblance—a scientifically grounded basis for
Summing Up: Controversies and
symbolic relations.
Cautions
That grounding was initially provided by Sidman
and his colleagues in their experimental demonstra- Punishment and negative reinforcement contingen-
tions of equivalence relations that entail symmetri- cies abound in nature and in social communities
cal and transitive relations between arbitrarily and thus constitute important domains for study. In
chosen stimuli (e.g., Sidman, 1986, 1994; Sidman each domain, we have offered a rubric for systematic
et al., 1982; Sidman, Kirk, & Willson-Morris, 1985). understanding. In one domain, our focus has been
Augustson and Dougher (1997) then built on this on the variables whereby punishment is effective in
conception by showing that the function of a warn- reducing behavior, including immediacy and consis-
ing stimulus in an avoidance procedure could trans- tency, intensity, availability of alternative unpun-
fer to a physically dissimilar, arbitrarily chosen ished repertoires, and the separate variables tending
stimulus solely by virtue of independently estab- to maintain the punished behavior. In the other
lished equivalence relations. domain, our focus has been on the varied time scales
Equivalent or symbolic relations, whereby one whereby negative reinforcement is operative: short-
entity stands in place of another, are transparent to term postponement of, immediate but sometimes
the extent that all interrelated stimuli are inter- intermittent escape from, and reduction in fre-
changeable (see Chapter 16, this volume). These can quency of aversive events. Sometimes two or more
be viewed as special cases of arbitrary relational of these occur in alternative or nested relationships
responding, conceptualized as higher order operant whereby situational change is the best way to char-
classes (Catania, 2007, pp. 120, 155–158) or in acterize the consequence of behavior. Further com-
terms of relational frame theory (e.g., Dymond & plicating matters, the aversive event can be time out
Rehfeldt, 2000; Hayes, 1991; Steele & Hayes, 1991). from positive reinforcement. In both domains, our
Most of these sets of arbitrary relations might be basic understanding translates readily into practical
characterized as making up structured rather than applications, especially with close attention to dis-
transparent networks. Thus, the network of rela- criminative effects.
tions characterized as different from entails symme- Despite this understanding and the effective
try (if B is different from A, then A is different from technology that it yields, some controversies endure.
B) but not transitivity (if B is different from A, and C Behavior analysis, in general, has sometimes
505
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
encountered resistance arising from its distinctive From a scientific perspective, which we have
prose patterns (Hineline, 1980, 1990) and its attempted to represent here, one could wish that
emphasis on interpretive concepts anchored directly these controversies could be put aside, recognizing
in behavior–environment interactions rather than in the processes of punishment and negative reinforce-
dispositional characteristics of the behaving organ- ment as important to both humans’ and other organ-
ism (D. P. Field & Hineline, 2008; Hineline, 1984b, isms’ adaptiveness. Those who study those processes
1992; Hineline & Wanchisen, 1989). In addition, as take no pleasure in exposing experimental subjects
we noted earlier, the topics of this chapter have been to pain, discomfort, or even inconvenience; instead,
the focus of their own particular controversies. Less they recognize behavior in relation to aversive
parochial and more volatile controversies continue events as inevitably involved in people’s lives, mak-
to swirl around punishment—the term, with its var- ing those processes important to understand, that
ious meanings, and the practices that are taken as they might be minimized when alternative tech-
corresponding to it but that do not always match up niques are not feasible and used effectively when
with those meanings. On one hand, in many con- that use is deemed important and appropriate.
texts and forums, aversive control is not viewed as Nevertheless, scientific study does legitimize
controversial. In discussions at the philosophical some concerns that are embedded in these contro-
and cultural levels that focus on what is deemed versies. An example concerns people’s understand-
good or bad for the individual and society, there ing of the behavior of punishing as well as of the
seems to be agreement that mild and temporary behavior targeted for punishment, and the impor-
pain, loss of ability, loss of freedom, or loss of plea- tance of discerning whether the former is a vengeful
sure are acceptable if justly applied. Unjust proce- action rather than an action predicated on a decrease
dures, besides “cruel and unusual,” would include in the targeted behavior. First, basic research with
procedures that are thought to humiliate, dehuman- both human and nonhuman subjects has shown that
ize, or inflict extreme pain (TASH, 2011). Less aggressive behavior can be a by-product of extinc-
mildly, as we noted earlier, justice is understood as tion (e.g. Flory, Smith, & Ellis, 1977; Kelly & Hake,
retribution—a sanitized term for vengeance, which 1970) or of intermittent positive reinforcement
can be more baldly stated (while biblically justified) (May & Kennedy, 2009). That is, when an individual
as “an eye for an eye and a tooth for a tooth.” with a history of consistently reinforced behavior
Punishment as explicitly and contingently encounters a period or situation of nonreinforce-
applied seems to be controversial in ways that pun- ment, that individual has an increased likelihood
ishment as haphazard practice may not be. Thus, of attacking whatever or whoever is available—
whether it is implemented by design appears to be conspecifics or even inanimate objects. In addition,
important, which is ironic, for implemented as pro- aggressive behavior can also be generated by painful
cedures explicitly based on behavioral principles, stimulation (Hutchinson & Emley, 1977), by injury
punishment is defined on the basis of its producing suffered at the hands of another person, or even by
a decrease in behavior, not on its characteristics circumstances in which a team of people is working
related to pain or trauma. Still, it is mainly the char- to prevent monetary loss (Emurian, Emurian, &
acteristics of the punishing consequence, rather Brady, 1985; Hake & Campbell, 1972). One might
than the contingencies and effects of that conse- argue that humans have transcended this sort of
quence, that gain the most attention and generate thing, but the nasty little fact crops up when, for
the most heat in the controversies, with opponents example, a computer or a soft-drink machine sud-
viewing the latter as ends that do not justify the denly ceases to yield its usual reinforcers or some-
means. For them, the consequent stimuli, irrespec- one violates a person’s home, injures a person’s
tive of the effects of their use, are considered by friend, or steals a person’s property.
some to be in violation of human rights or to con- Second, the circumstances in which a punish-
flict with values or moral standards (Singer, Gert, & ment procedure might be proposed concern situa-
Koegel, 1999; Van Houten et al., 1988). tions in which an individual’s behavior is aversive to
506
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
other people or situations in which reinforcement- Azrin, N. H., & Holz, W. C. (1966). Punishment. In W.
based interventions have been unsuccessful—that is, K. Honig (Ed.), Operant behavior: Areas of research
and application (pp. 380–447). Englewood Cliffs, NJ:
the behavior of caregivers, teachers, and perhaps Prentice-Hall.
consultants is under extinction. It follows, then, that Azrin, N. H., Holz, W. C., & Hake, D. F. (1963). Fixed-
punishment procedures should be more tightly reg- ratio punishment. Journal of the Experimental
ulated than other procedures to ensure that punish- Analysis of Behavior, 6, 141–148. doi:10.1901/
ment is being implemented purely as a best feasible jeab.1963.6-141
alternative for decreasing the behavior of concern Badia, P., Coker, C., & Harsh, J. (1973). Choice of higher
density signaled shock over lower density unsig-
and does not constitute vengeful or aggressive
nalled shock. Journal of the Experimental Analysis of
behavior on the part of those implementing the pro- Behavior, 20, 47–55. doi:10.1901/jeab.1973.20-47
cedure. This illustrates a feature of behavior analysis Badia, P., Culberson, S., & Harsh, J. (1973). Choice of
that is often overlooked by those seeing it from longer or stronger signaled shock over shorter or
other viewpoints as well as occasionally by behavior weaker unsignalled shock. Journal of the Experimental
Analysis of Behavior, 19, 25–32. doi:10.1901/jeab.
analysts themselves: Behavior analysts properly view
1973.19-25
the principles that they study as applicable to their
Baer, D. M. (1960). Escape and avoidance response of
own behavior even as they address the behavior of preschool children to two schedules of reinforcement
others. withdrawal. Journal of the Experimental Analysis of
References Baer, D. M. (1962). A technique of social reinforcement
for the study of child behavior: Behavior avoiding
Ader, R., & Tatum, R. (1961). Free-operant avoid-
reinforcement withdrawal. Child Development, 33,
ance conditioning in human subjects. Journal of
847–858.
doi:10.1901/jeab.1961.4-275 Baer, D. M. (2001). A small matter of proof. In W. T.
O’Donohue, D. A. Henderson, S. C. Hayes, J. E. Fisher,
Anger, D. (1963). The role of temporal discriminations & L. J. Hayes (Eds.), A history of the behavioral thera-
in the reinforcement of Sidman avoidance behavior. pies: Founders’ personal histories (pp. 253–265). Reno,
Journal of the Experimental Analysis of Behavior, 6(3, NV: Context Press.
Pt. 2, Suppl.), 477–506. doi:10.1901/jeab.1963.6-s477
Baron, A. (1991). Avoidance and punishment. In I. H.
Arbuckle, J. L., & Lattal, A. (1987). A role for negative Iversen & K. A. Lattal (Eds.), Experimental analy-
reinforcement of response omission in punishment? sis of behavior: Part 1. Techniques in the behavioral
Journal of the Experimental Analysis of Behavior, 48, and neural sciences (pp. 173–217). Amsterdam, the
407–416. doi:10.1901/jeab.1987.48-407 Netherlands: Elsevier Science.
Augustson, E. M., & Dougher, M. J. (1997). The transfer Baron, A., DeWaard, R. J., & Lipson, J. (1977).
of avoidance evoking functions through stimulus Increased reinforcement when time-out from avoid-
equivalence classes. Journal of Behavior Therapy and ance includes access to a safe place. Journal of the
Experimental Psychiatry, 28, 181–191. doi:10.1016/ Experimental Analysis of Behavior, 27, 479–494.
S0005-7916(97)00008-6 doi:10.1901/jeab.1977.27-479
Axelrod, S., & Apsche, J. (Eds.). (1983). The effects Baron, A., & Galizio, M. (1976). Clock control of human
of punishment on human behavior. New York, NY: performance on avoidance and fixed-interval sched-
Academic Press. ules. Journal of the Experimental Analysis of Behavior,
Azrin, N. H. (1956). Some effects of two intermittent 26, 165–180. doi:10.1901/jeab.1976.26-165
schedules of immediate and non-immediate punish- Baron, A., & Kaufman, A. (1966). Human, free-operant
ment. Journal of Psychology, 42, 3–21. doi:10.1080/00 avoidance of “time out” from monetary reinforce-
223980.1956.9713020 ment. Journal of the Experimental Analysis of Behavior,
Azrin, N. H. (1960). Effects of punishment intensity 9, 557–565. doi:10.1901/jeab.1966.9-557
during variable interval reinforcement. Journal of Baum, W. M. (1973). Time allocation and negative rein-
the Experimental Analysis of Behavior, 3, 123–142. forcement. Journal of the Experimental Analysis of
doi:10.1901/jeab.1960.3-123 Behavior, 20, 313–322. doi:10.1901/jeab.1973.20-313
Azrin, N. H., Hake, D. F., Holz, W. C., & Hutchinson, Baum, W. M. (1974). On two types of deviation from the
R. R. (1965). Motivational aspects of escape from matching law: Bias and undermatching. Journal of
punishment. Journal of the Experimental Analysis of the Experimental Analysis of Behavior, 22, 231–242.
Behavior, 8, 31–44. doi:10.1901/jeab.1965.8-31 doi:10.1901/jeab.1974.22-231
507
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Behrend, E. R., & Bitterman, M. E. (1963). Sidman avoid- Davison, M., & McCarthy, D. (1988). The matching law:
ance in the fish. Journal of the Experimental Analysis A research review. Hillsdale, NJ: Erlbaum.
of Behavior, 6, 47–52. doi:10.1901/jeab.1963.6-47 DeFulio, A., & Hackenberg, T. D. (2007). Discriminated
Black, A. H., & Morse, P. (1961). Avoidance learn- time-out avoidance in pigeons: The roles of added
ing in dogs without a warning stimulus. Journal stimuli. Journal of the Experimental Analysis of
of the Experimental Analysis of Behavior, 4, 17–23. Behavior, 88, 51–71. doi:10.1901/jeab.2007.59-06
doi:10.1901/jeab.1961.4-17
Dinsmoor, J. A. (1954). Punishment: I. The avoid-
Bolles, R. C. (1967). Theory of motivation. New York, NY: ance hypothesis. Psychological Review, 61, 34–46.
Harper & Row. doi:10.1037/h0062725
Bolles, R. C. (1970). Species-specific defense reactions Dinsmoor, J. A. (1962). Variable-interval escape
and avoidance learning. Psychological Review, 77, from stimuli accompanied by shocks. Journal of
32–48. doi:10.1037/h0028589 the Experimental Analysis of Behavior, 5, 41–47.
Bolles, R. C. (1973). The avoidance learning problem. doi:10.1901/jeab.1962.5-41
In G. H. Bower (Ed.), The psychology of learning Dinsmoor, J. A. (1977). Escape, avoidance, punishment:
and motivation (Vol. 6, pp. 97–145). New York, NY: Where do we stand? Journal of the Experimental
Academic Press. Analysis of Behavior, 28, 83–95. doi:10.1901/jeab.
Bostow, D. E., & Bailey, J. B. (1969). Modification of 1977.28-83
severe disruptive and aggressive behavior using brief Dinsmoor, J. A. (1998). Punishment. In W. O’Donohue
time-out and reinforcement procedures. Journal of (Ed.), Learning and behavior therapy (pp. 188–204).
Applied Behavior Analysis, 2, 31–37. doi:10.1901/ Boston, MA: Allyn & Bacon.
jaba.1969.2-31
Dinsmoor, J. A. (2001). Stimuli inevitably generated by
Bruzek, J. L., Thompson, R. H., & Peters, L. C. (2009). behavior that avoids electric shock are inherently
Resurgence of infant caregiving responses. Journal of reinforcing. Journal of the Experimental Analysis of
the Experimental Analysis of Behavior, 92, 327–343. Behavior, 75, 311–333. doi:10.1901/jeab.2001.75-311
doi:10.1901/jeab.2009-92-327
Dinsmoor, J. A., & Sears, G. W. (1973). Control of avoid-
Camp, D. S., Raymond, G. A., & Church, R. M. (1967). ance by a response-produced stimulus. Learning and
Temporal relationship between response and pun- Motivation, 4, 284–293. doi:10.1016/0023-9690
ishment. Journal of Experimental Psychology, 74, (73)90018-0
114–123. doi:10.1037/h0024518
Dinsmoor, J. A., & Winograd, E. (1958). Shock inten-
Catania, A. C. (2007). Learning (Interim [4th] ed.). sity in variable-interval escape schedules. Journal of
Cornwall-on-Hudson, NY: Sloan. the Experimental Analysis of Behavior, 1, 145–148.
Clark, F. C. (1961). Avoidance conditioning in the doi:10.1901/jeab.1958.1-145
chimpanzee. Journal of the Experimental Analysis of Doughty, S. S., Anderson, C. M., Doughty, A. H., Williams,
Behavior, 4, 393–395. doi:10.1901/jeab.1961.4-393 D. C., & Saunders, K. J. (2007). Discriminative con-
Clark, H. B., Rowbury, T., Baer, A. M., & Baer, D. M. trol of punished stereotyped behavior in humans.
(1973). Time-out as a punishing stimulus in con- Journal of the Experimental Analysis of Behavior, 87,
tinuous and intermittent schedules. Journal of 325–336. doi:10.1901/jeab.2007.39-05
Applied Behavior Analysis, 6, 443–455. doi:10.1901/ Dymond, S., & Rehfeldt, R. (2000). Understanding com-
jaba.1973.6-443 plex behavior: The transformation of stimulus func-
Cooper, J. O., Heron, T. E., & Heward, W. L. (2007). tions. Behavior Analyst, 23, 239–254.
Applied behavior analysis. Columbus, OH: Pearson. Dymond, S., & Roche, B. (2009). A contemporary behav-
Courtney, K., & Perone, M. (1992). Reductions in shock ior analysis of anxiety and avoidance. Behavior
frequency and response effort as factors in rein- Analyst, 32, 7–27.
forcement by time-out from avoidance. Journal of Dymond, S., Roche, B., Forsyth, J. P., Whelan, R., &
the Experimental Analysis of Behavior, 58, 485–496. Rhoden, J. (2007). Transformation of avoidance
doi:10.1901/jeab.1992.58-485 response functions in accordance with same and
Crosbie, J. (1998). Negative reinforcement and punish- opposite relational frames. Journal of the Experimental
ment. In K. A. Lattal & M. Perone (Eds.), Handbook Analysis of Behavior, 88, 249–262. doi:10.1901/
of research methods in human operant behavior (pp. jeab.2007.22-07
163–189). New York, NY: Plenum Press. Dymond, S., Roche, B., Forsyth, J. P., Whelan, R., &
D’Andrea, T. (1971). Avoidance of time-out from Rhoden, J. (2008). Derived avoidance learning:
response-independent reinforcement. Journal of Transformation of avoidance response functions in
the Experimental Analysis of Behavior, 15, 319–325. accordance with same and opposite relational frames.
doi:10.1901/jeab.1971.15-319 Psychological Record, 58, 269–286.
508
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Emurian, H. H., Emurian, C. S., & Brady, J. V. (1985). Galizio, M., & Perone, M. (1987). Variable-interval
Positive and negative reinforcement effects on schedules of time-out from avoidance: Effects of
behavior in a three-person microsociety. Journal of chlordiazepoxide, CGS 8216, morphine, and naltrex-
the Experimental Analysis of Behavior, 44, 157–174. one. Journal of the Experimental Analysis of Behavior,
doi:10.1901/jeab.1985.44-157 47, 115–126. doi:10.1901/jeab.1987.47-115
Estes, W. K. (1969). Outline of a theory of punish- Galizio, M., Robinson, E. G., & Ordronneau, C. (1994).
ment. In B. A. Campbell & R. M. Church (Eds.), Opioid drugs and time-out from avoidance.
Punishment and aversive behavior (pp. 57–82). New Behavioural Pharmacology, 5, 125–130. doi:10.1097/
York, NY: Appleton-Century-Crofts. 00008877-199404000-00003
Etzel, B. (1997). Environmental approaches to the devel- Gardner, E. T., & Lewis, P. (1976). Negative reinforce-
opment of conceptual behavior. In D. M. Baer & ment with shock-frequency increase. Journal of
E. M. Pinkston (Eds.), Environment and behavior the Experimental Analysis of Behavior, 25, 3–14.
(pp. 52–79). Boulder, CO: Westview Press. doi:10.1901/jeab.1976.25-3
Fantino, E. (1973). Aversive control. In J. A. Nevin & Hackenberg, T. D., & Defulio, A. (2007). Time-out from
G. S. Reynolds (Eds.), The study of behavior: Learning, reinforcement: Restoring the balance between analy-
motivation, emotion, and instinct (pp. 239–279). sis and application. Revista Mexicana de Análisis de la
Glenview, IL: Scott, Foresman. Conducta, 33, 37–44.
Field, D. P., & Hineline, P. N. (2008). Dispositioning and Hake, D. F., & Campbell, R. L. (1972). Characteristics
the obscured roles of time in psychological explana- and response-displacement effects of shock-generated
tion. Behavior and Philosophy, 36, 5–69. responding during negative reinforcement procedures:
Pre-shock responding and post-shock aggressive
Field, G. E., & Boren, J. J. (1963). An adjusting avoidance responding. Journal of the Experimental Analysis of
procedure with multiple auditory and visual warn- Behavior, 17, 303–323. doi:10.1901/jeab.1972.17-303
ing stimuli. Journal of the Experimental Analysis of
Behavior, 6, 537–543. doi:10.1901/jeab.1963.6-537 Hayes, S. C. (1991). A relational control theory of stimu-
lus equivalence. In L. J. Hayes & P. N. Chase (Eds.),
Flory, R. K., Smith, E. L., & Ellis, B. B. (1977). The effects Dialogues on verbal behavior: The First International
of two response-elimination procedures on reinforced Institute on Verbal Relations (pp. 19–40). Reno, NV:
and induced aggression. Journal of the Experimental Context Press.
Analysis of Behavior, 27, 5–15. doi:10.1901/jeab.
Herman, R. L., & Azrin, N. H. (1964). Punishment by
1977.27-5
noise in an alternative response situation. Journal of
Forgione, A. G. (1970). The elimination of interfering the Experimental Analysis of Behavior, 7, 185–188.
response patterns in lever-press avoidance situations. doi:10.1901/jeab.1964.7-185
Herrnstein, R. J. (1969). Method and theory in the study
51–56. doi:10.1901/jeab.1970.13-51
of avoidance. Psychological Review, 76, 49–69.
Foxx, R. M., & Shapiro, S. T. (1978). The time-out doi:10.1037/h0026786
ribbon: A nonexclusionary time-out procedure. Herrnstein, R. J., & Hineline, P. N. (1966). Negative
Journal of Applied Behavior Analysis, 11, 125–136. reinforcement as shock-frequency reduction. Journal
doi:10.1901/jaba.1978.11-125 of the Experimental Analysis of Behavior, 9, 421–430.
Galbicka, G., & Branch, M. N. (1981). Selective punish- doi:10.1901/jeab.1966.9-421
ment of interresponse times. Journal of the Experimental Hineline, P. N. (1970). Negative reinforcement without
Analysis of Behavior, 35, 311–322. doi:10.1901/ shock reduction. Journal of the Experimental Analysis of
jeab.1981.35-311 Behavior, 14, 259–268. doi:10.1901/jeab.1970.14-259
Galbicka, G., & Branch, M. N. (1983). Stimulus-food Hineline, P. N. (1976). Negative reinforcement and
relations and free-operant postponement of time-out. avoidance. In W. K. Honig & J. E. R. Staddon
Journal of the Experimental Analysis of Behavior, 40, (Eds.), Handbook of operant behavior (pp. 364–414).
153–163. doi:10.1901/jeab.1983.40-153 Englewood Cliffs: Prentice-Hall.
Galbicka, G., & Platt, J. R. (1984). Interresponse-time Hineline, P. N. (1980). The language of behavior analy-
punishment: A basis for shock-maintained behavior. sis: Its community, its function, and its limitations.
Journal of the Experimental Analysis of Behavior, 41, Behaviorism, 8, 67–86.
291–308. doi:10.1901/jeab.1984.41-291
Hineline, P. N. (1981). The several roles of stimuli in
Galizio, M. (1979). Contingency-shaped and rule- negative reinforcement. In P. Harzem & M. D.
governed behavior: Instructional control of human Zeiler (Eds.), Advances in analysis of behavior: Vol. 2.
loss avoidance. Journal of the Experimental Analysis of Predictability, correlation, and contiguity (pp. 203–246).
Behavior, 31, 53–70. doi:10.1901/jeab.1979.31-53 Chichester, England: Wiley.
509
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Hineline, P. N. (1984a). Aversive control: A separate Experimental Analysis of Behavior, 10, 461–465.
domain? Journal of the Experimental Analysis of doi:10.1901/jeab.1967.10-461
Kelleher, R. T., & Morse, W. H. (1964). Escape behavior
Hineline, P. N. (1984b). What, then, is Skinner’s opera- and punishment behavior. Federation Proceedings, 23,
tionism? [Commentary]. Behavioral and Brain 808–817.
Sciences, 7, 560.
Kelly, J. F., & Hake, D. F. (1970). An extinction-induced
Hineline, P. N. (1990). The origins of environment-based increase in an aggressive response with humans.
psychological theory. Journal of the Experimental Journal of the Experimental Analysis of Behavior, 14,
Analysis of Behavior, 53, 305–320. doi:10.1901/jeab. 153–164. doi:10.1901/jeab.1970.14-153
1990.53-305
Kern, L., Koegel, R. L., & Dunlap, G. (Eds.). (1996).
Hineline, P. N. (1992). A self-interpretive behavior Positive behavioral support: Including people with
analysis. American Psychologist, 47, 1274–1286. difficult behavior in the community. Baltimore, MD:
doi:10.1037/0003-066X.47.11.1274 Brookes.
Hineline, P. N., & Wanchisen, B. A. (1989). Correlated Krasnegor, N. A., Brady, J. V., & Findley, J. D. (1971).
hypothesizing, and the distinction between Second-order optional avoidance as a function of
contingency-shaped and rule-governed behavior. In fixed-ratio requirements. Journal of the Experimental
S. C. Hayes (Ed.), Rule-governed behavior: Cognition, Analysis of Behavior, 15, 181–187. doi:10.1901/
contingencies, and instructional control (pp. 221–268). jeab.1971.15-181
New York, NY: Plenum Press.
Laraway, S., Snycersky, S., Michael, J., & Poling, A.
Holz, W. C., & Azrin, N. H. (1961). Discriminative prop- (2003). Motivating operations and terms to describe
erties of punishment. Journal of the Experimental them: Some further refinements. Journal of Applied
Analysis of Behavior, 4, 225–232. doi:10.1901/jeab. Behavior Analysis, 36, 407–414. doi:10.1901/jaba.
1961.4-225 2003.36-407
Hutchinson, R. R. (1977). By-products of aversive con- Lerman, D. C., Iwata, B. A., Shore, B. A., & DeLeon, I. G.
trol. In W. K. Honig & J. E. R. Staddon (Eds.), (1997). Effects of intermittent punishment on
Handbook of operant behavior (pp. 415–431). self-injurious behavior: An evaluation of schedule
Englewood Cliffs, NJ: Prentice Hall. thinning. Journal of Applied Behavior Analysis, 30,
Hutchinson, R. R., & Emley, G. S. (1977). Electric shock 187–201. doi:10.1901/jaba.1997.30-187
produced drinking in the squirrel monkey. Journal Lerman, D. C., & Vorndran, C. M. (2002). On the status
of the Experimental Analysis of Behavior, 28, 1–12. of knowledge for using punishment: Implications
doi:10.1901/jeab.1977.28-1 for treating behavior disorders. Journal of Applied
Iwata, B. A. (1987). Negative reinforcement in applied Behavior Analysis, 35, 431–464. doi:10.1901/jaba.
behavior analysis: An emerging technology. 2002.35-431
Lewis, P., Gardner, E. T., & Hutton, L. (1976). Integrated
doi:10.1901/jaba.1987.20-361
delays to shock as negative reinforcement. Journal of
Iwata, B. A., Dorsen, M. F., Slifer, K. J., Jauman, K. E., & the Experimental Analysis of Behavior, 26, 379–386.
Richman, G. S. (1994). Toward a functional analysis doi:10.1901/jeab.1976.26-379
of self-injury. Journal of Applied Behavior Analysis, 27,
Logue, A. W., & De Villiers, P. A. (1978). Matching in
197–209. doi:10.1901/jaba.1994.27-197
concurrent variable-interval avoidance schedules.
Johnston, J. M. (1972). Punishment of human behavior. Journal of the Experimental Analysis of Behavior, 29,
American Psychologist, 27, 1033–1054. 61–66. doi:10.1901/jeab.1978.29-61
Kamin, K. J., Brimer, C. J., & Black, A. H. (1963). Magoon, M. A., & Critchfield, T. S. (2008). Concurrent
Conditioned suppression as a monitor of fear of the schedules of positive and negative reinforcement:
CS in the course of avoidance training. Journal of Differential-impact and differential-outcomes
Comparative and Physiological Psychology, 56, 497–501. hypotheses. Journal of the Experimental Analysis of
doi:10.1037/h0047966 Behavior, 90, 1–22. doi:10.1901/jeab.2008.90-1
Kazdin, A. E. (1972). Response cost: The removal of Matson, J. L., & DiLorenzo, T. M. (1984). Punishment and
conditioned reinforcers for therapeutic change. its alternatives: New perspectives for behavior modifica-
Behavior Therapy, 3, 533–546. doi:10.1016/S0005- tion. New York, NY: Springer.
7894(72)80001-7
May, M. E., & Kennedy, C. H. (2009). Aggression as
Kazdin, A. E. (2003). Behavior modification in applied set- positive reinforcement in mice under various ratio-
tings. Belmont, CA: Wadsworth. and time-based reinforcement schedules. Journal of
Keehn, J. D. (1967). Bar-holding with negative reinforce- the Experimental Analysis of Behavior, 91, 185–196.
ment: Preparatory or perseverative? Journal of the doi:10.1901/jeab.2009.91-185
510
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
McIntyre, L. L., Gresham, F. M., DiGennaro, F. D., & frequency. Journal of the Experimental Analysis of
Reed, D. D. (2007). Treatment integrity of school- Behavior, 74, 147–164. doi:10.1901/jeab.2000.74-147
based interventions with children in the Journal Plummer, S., Baer, D. M., & Leblanc, J. (1977).
of Applied Behavior Analysis 1991–2005. Journal of Functional considerations in the use of procedural
Applied Behavior Analysis, 40, 659–672. time-out and an effective alternative. Journal of
McKenzie, S. D., Smith, R. G., Simmons, J. N., & Applied Behavior Analysis, 10, 689–705. doi:10.1901/
Soderlund, M. J. (2008). Using a stimulus corre- jaba.1977.10-689
lated with reprimands to suppress automatically Pryor, K. (1999). Don’t shoot the dog! The new art of teach-
maintained eye poking. Journal of Applied Behavior ing and training (Rev. ed.). New York, NY: Bantam
Analysis, 41, 255–259. doi:10.1901/jaba.2008.41-255 Books.
Mellitz, M., Hineline, P. N., Whitehouse, W. G., & Repp, A. C., & Horner, R. H. (1999). Functional analysis
Laurence, M. T. (1983). Duration-reduction of of problem behavior. Belmont, CA: Wadsworth.
avoidance sessions as negative reinforcement. Journal
Richardson, J. V., & Baron, A. (2008). Avoidance of
time-out from response-independent food: Effects of
doi:10.1901/jeab.1983.40-57
delivery rate and quality. Journal of the Experimental
Merck veterinary manual. (9th ed.). (2006). Whitehouse Analysis of Behavior, 89, 169–181. doi:10.1901/jeab.
Station, NJ: Marial Ltd. 2008.89-169
Michael, J. (1975). Positive and negative reinforcement, Salzinger, K. (1991). Definitions and usage, or a rose by
a distinction that is no longer necessary; or a better any other name smells as sweet. Behavior Analyst,
way to talk about bad things. Behaviorism, 3, 33–44. 14, 213.
Michael, J. (1982). Distinguishing between discrimina- Schilder, M. B. H., & van der Borg, J. A. M. (2004). Training
tive and motivating functions of stimuli. Journal of dogs with the help of the shock collar: Short and long
the Experimental Analysis of Behavior, 37, 149–155. term behavioural effects. Applied Animal Behaviour
doi:10.1901/jeab.1982.37-149 Science, 85, 319–334. doi:10.1016/j.applanim.2003.
10.004
Michael, J. (1993). Establishing operations. Behavior
Analyst, 16, 191–206. Seligman, M. E. P., & Csikszentmihalyi, M. (2000).
Positive psychology: An introduction. American
Miltenberger, R. G. (2001). Behavior modification: Psychologist, 55, 5–14. doi:10.1037/0003-066X.55.1.5
Principles and procedures (2nd ed.). Belmont, CA:
Sidman, M. (1953a). Avoidance conditioning with brief
Wadsworth.
shock and no exteroceptive warning signal. Science,
Newsom, C., Favell, J. E., & Rincover, A. (1983). Side 118, 157–158. doi:10.1126/science.118.3058.157
effects of punishment. In S. Axelrod & J. Apsche Sidman, M. (1953b). Two temporal parameters in the
(Eds.), The effects of punishment on human behavior maintenance of avoidance behavior by the white rat.
(pp. 285–316). New York, NY: Academic Press. Journal of Comparative and Physiological Psychology,
Pence, S. T., Roscoe, E. M., Bourret, J. C., & Ahearn, 46, 253–261. doi:10.1037/h0060730
W. H. (2009). Relative contributions of three Sidman, M. (1955). Some properties of the warning stim-
descriptive methods: Implications for behavioral ulus in avoidance behavior. Journal of Comparative and
assessment. Journal of Applied Behavior Analysis, 42, Physiological Psychology, 48, 444–450. doi:10.1037/
425–446. doi:10.1901/jaba.2009.42-425 h0047481
Perone, M., & Galizio, M. (1987). Variable-interval Sidman, M. (1962). Reduction of shock frequency as
schedules of time-out from avoidance. Journal of reinforcement for avoidance behavior. Journal of
the Experimental Analysis of Behavior, 47, 97–113. the Experimental Analysis of Behavior, 5, 247–257.
doi:10.1901/jeab.1987.47-97 doi:10.1901/jeab.1962.5-247
Perone, M., Galizio, M., & Baron, A. (1988). The relevance Sidman, M. (1986). Functional analysis of emergent ver-
of animal-based principles in the laboratory study bal classes. In T. Thompson & M. D. Zeiler (Eds.),
of human operant conditioning. In G. Davey & C. Analysis and integration of behavioral units (pp. 213–245).
Cullen (Eds.), Human operant conditioning and behav- Hillsdale, NJ: Erlbaum.
ior modification (pp. 59–85). New York, NY: Wiley. Sidman, M. (1994). Equivalence relations and behavior: A
Piazza, C. C., Hanley, G. P., & Fisher, W. W. (1996). research story. Boston, MA: Authors Cooperative.
Functional analysis and treatment of cigarette pica. Sidman, M. (2006). The distinction between positive and
Journal of Applied Behavior Analysis, 29, 437–450. negative reinforcement: Some additional consider-
doi:10.1901/jaba.1996.29-437 ations. Behavior Analyst, 29, 135–139.
Pietras, C. J., & Hackenberg, T. D. (2000). Time-out Sidman, M., & Boren, J. J. (1957a). A comparison of two
postponement without increased reinforcement types of warning stimulus in an avoidance situation.
511
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Journal of Comparative and Physiological Psychology, Starin, S. (1991). “Nonaversive” behavior management: A
50, 282–287. doi:10.1037/h0046474 misnomer. Behavior Analyst, 14, 207–209.
Sidman, M., & Boren, J. J. (1957b). The relative aversive- Steele, D., & Hayes, S. C. (1991). Stimulus equivalence
ness of signal and shock in an avoidance situation. and arbitrarily applicable relational responding.
Journal of Abnormal and Social Psychology, 55, 339–344. Journal of the Experimental Analysis of Behavior, 56,
doi:10.1037/h0043237 519–555. doi:10.1901/jeab.1991.56-519
Sidman, M., Kirk, B., & Willson-Morris, M. (1985). Stokes, T. F., & Baer, D. M. (1977). An implicit technol-
Six-member stimulus classes generated by ogy of generalization. Journal of Applied Behavior
conditional-discrimination procedures. Journal of Analysis, 10, 349–367. doi:10.1901/jaba.1977.10-349
the Experimental Analysis of Behavior, 43, 21–42. TASH. (2011). TASH letter to Massachusetts Department
doi:10.1901/jeab.1985.43-21 of Developmental Services. Retrieved from http://tash.
Sidman, M., Rauzin, R., Lazar, R., Cunningham, S., org/tash-letter-to-massachusetts-department-of-
Tailby, W., & Carrigan, P. (1982). A search for developmental-services
symmetry in the conditional discriminations of Terrace, H. S. (1963). Discrimination learning with and
rhesus monkeys, baboons, and children. Journal of without “errors.” Journal of the Experimental Analysis
the Experimental Analysis of Behavior, 37, 23–44. of Behavior, 6, 1–27. doi:10.1901/jeab.1963.6-1
doi:10.1901/jeab.1982.37-23
Thorndike, E. L. (1932). Fundamentals of learning. New
Singer, G. H., Gert, B., & Koegel, R. I. (1999). A moral York, NY: Columbia University, Teachers College.
framework for analyzing the controversy over aversive doi:10.1037/10976-000
behavioral interventions for people with severe mental
retardation. Journal of Positive Behavior Interventions, Ulrich, R. E., Dulaney, S., Kucera, T., & Colasacco, A.
1, 88–100. doi:10.1177/109830079900100203 (1972). Side effects of aversive control. In R. M.
Gilbert & J. D. Keehn (Eds.), Schedule effects: Drugs,
Skinner, B. F. (1953). Science and human behavior. New drinking, and aggression (pp. 203–242). Toronto,
York, NY: Macmillan. Ontario, Canada: University of Toronto Press.
Snider, K. S. (2007). A constructional canine aggression Van Houten, R., Axerold, S., Bailey, J. S., Favell, J. E.,
treatment: Using a negative reinforcement shaping Foxx, R., Iwata, B., & Lovaas, O. I. (1988). The
procedure with dogs in home and community settings. right to effective behavioral treatment. Journal of
Unpublished master’s thesis, University of North Applied Behavior Analysis, 21, 381–384. doi:10.1901/
Texas, Denton. jaba.1988.21-381
Solnick, J. V., Rincover, A., & Peterson, C. R. (1977). Winograd, E. (1965). Escape behavior under differ-
Some determinants of the reinforcing and punish- ent fixed ratios and shock intensities. Journal of
ing effects of time-out. Journal of Applied Behavior the Experimental Analysis of Behavior, 8, 117–124.
Analysis, 10, 415–424. doi:10.1901/jaba.1977.10-415 doi:10.1901/jeab.1965.8-117
Spradlin, J. E. (2002). Punishment: A primary process? Yates, C. (1991). A response to nonaversive behavior
Journal of Applied Behavior Analysis, 35, 475–477. management and “default” technologies. Behavior
doi:10.1901/jaba.2002.35-475 Analyst, 14, 217–218.
512
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 22
Operant Variability
Allen Neuringer and Greg Jensen
During almost the entirety of a documentary film world, including mind and behavior, scientists must
featuring Pablo Picasso (Clouzot, 1956), the camera appreciate the reality of unpredictability. From Epi-
is focused on the rear of a large glass screen that curus in the 3rd century BC, who hypothesized ran-
serves as a canvas. As viewers watch, each paint dom swerves of atoms, to contemporary quantum
stroke appears and a theme emerges, only to be physicists, some have posited that nature contains
transformed in surprising ways. In many of the within it unpredictable aspects that cannot be
paintings, what started out as the subject is modi- explained by if-A-then-B causal relationships, no
fied, sometimes many times. Erasures; new colors; matter how complex those relationships might be.
alterations of size; and, indeed, the very subject of In this chapter, we discuss the unpredictability
the painting flow into one another. Each painting of behavior and focus on one aspect of it. When
can readily be identified as a Picasso, but the process reinforcers are contingent on variability, or more
seems to be filled with unplanned and unpredictable precisely on a level of operant–response variability
turns. Uncertainty and surprise characterize other (with levels ranging from easily predictable
arts as well. A fugue may be instantly recognizable responding to randomlike), the specified level will
as a composition by J. S. Bach, and yet the transi- be generated and maintained. Stated differently,
tions within the fugue may astound the listener, response unpredictability can be reinforced. Stated
even after many hearings. Leonard Bernstein wrote yet another way, variability is an operant dimen-
of the importance of establishing expectancies in sion of behavior. Operant dimension implies a bidi-
musical compositions and then surprising the lis- rectional relationship between behavior and
tener. Fiction writers describe their desires to com- reinforcer. Responses influence (or cause) the rein-
plete novels to find out what the characters will do forcers, and reinforcers influence (or cause) reoc-
because the authors can be as uncertain about their currence of the responses. The same bidirectional
creations as are their readers. relationship is sometimes true of response dimen-
Everyday behaviors may similarly be described in sions as well. For example, when food pellets are
terms of generativity and unpredictability. Listen contingent on rats’ lever presses, a minimum force
carefully to the meanderings of a conversation. must be exerted in a particular direction at a par-
Watch as an individual walks back and forth in his ticular location. Force, direction, and location are
or her office, the tilt of the head, the slight changes response dimensions that are involved in the con-
in pace or direction. Monitor the seemingly unpre- trol of reinforcers and come to be controlled by the
dictable transitions in one’s daydreams or images. reinforcers. Variability is related to reinforcement
Science is generally thought to be a search for pre- in the same way. We refer to this capacity as the
dictable relationship—if A then B—but throughout operant nature of variability or by the shorthand
history, some have argued that to understand the operant variability.
DOI: 10.1037/13937-022
513
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Neuringer and Jensen
The very idea of operant variability is surprising sequence of eight L and R responses in that trial dif-
to many and, at first blush, seems counterintuitive. fered from those in each of the previous 50 trials (as
Does variability not indicate noise? How can noise evaluated across a moving window). This contin-
be reinforced? In fact, does reinforcement not con- gency was referred to as lag 50. If the current
strain and organize responses—by definition—and sequence repeated any one (or more) of the previous
is that definition not confirmed by observation? As 50, then a brief time out (darkening of all lights)
we show, the answers are sometimes yes, but not resulted, and food was withheld. After food or time
always. Operant variability provides an important out, the keylights were again illuminated, and
exception, one that may be a factor in the emission another trial initiated. Sequences during the first 50
of voluntary operant responses generally. trials of a session were checked against the trials at
The chapter is organized broadly as follows. We the end of the previous session, that is, performance
discuss was evaluated continuously across sessions. Approx-
imately 25 sessions were provided under these VAR
■■ Experimental evidence showing that reinforcers contingencies.
and discriminative stimuli control behavioral Let us consider some possible outcomes. One
variability; would be that the birds stopped responding—
■■ Relationships between reinforcement of variabil- responding extinguished—because the lag 50
ity and other influences; requirement was too demanding. At the other end of
■■ Explanations: When variability is reinforced, the possibility spectrum, the birds cycled across at
what in fact is being reinforced? least 51 patterns and by so doing were reinforced
■■ How operant variability applies in such areas as 100% of the time, with each sequence being differ-
creativity, problem solving, and psychopathol- ent from every one of the previous 50. Although
ogy; and unlikely for pigeons, one way to solve lag contingen-
■■ How reinforced variability helps to explain the cies is to count in binary, with L = 0 and R = 1,
voluntary nature of operant behavior generally. then LLLLLLLL followed by LLLLLLLR, followed by
LLLLLLRL, then LLLLLLRR, and so on. Lag proce-
dures have also been used with human participants,
Reinforcement of Variability
and therefore such sophisticated counting behavior
As a way to describe the phenomenon, we begin must be considered. A third possible result would be
with descriptions of some of the methods that have alternations between a few preferred sequences
been successfully used to reinforce variability. (these would not be reinforced because of their high
frequencies but would fill the lag window) and an
Recency-Based Methods occasional “do something else” (leading to rein-
Imagine that a response earns a reinforcer only if it forcement). The fourth possibility would be that L
has not been emitted recently. Page and Neuringer and R responses were generated in randomlike fash-
(1985) applied this recency method, based on ion, or stochastically,1 as if the birds were flipping a
Schwartz (1980, 1988), to pigeons’ response coin.
sequences across two illuminated keys, left (L) and This last alternative best describes the results.
right (R). Each trial consisted of eight responses, One piece of evidence was that reinforcement
yielding 256 (or 28) different possible patterns of occurred on approximately 70% of the trials (with
L and R, for example, LLRLRRRR. In the initial the other 30% leading to time-outs; Page & Neu-
variability-reinforcing (or VAR) phase of the experi- ringer, 1985). The pigeons’ performances were com-
ment, a pattern was reinforced only if it had not pared with the results of a simulated model in which
occurred for some number of trials, referred to as a computer-based random-number generator pro-
the lag. A trial terminated with food only if the duced L and R responses under exactly the same
Throughout this chapter, we use stochastic and random interchangeably.

1
514
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
reinforcement contingencies experienced by the If Page and Neuringer’s (1985) experiment had
pigeons. The simulation showed that the model was stopped at this point, there would be uncertainty as
reinforced on 80% of trials because, by chance, to why responses varied. The issue is tricky because
response sequences were repeated within the win- it involves more than whether the lag procedure
dow of the lag 50 approximately 20% of the time. resulted in response variability (which it clearly
Thus, the pigeons’ performances were similar to, did). Rather, was variability directly reinforced, or
although not quite as good as, that of a random could the results be explained differently? Variabil-
response model. ity could have resulted from extrinsic sources (noise
A second source of support for approximations in the environment) or intrinsic sources (within the
to random generation was provided by statistical organism), or it could have been caused by experi-
analyses of the sequences, namely by the U statistic mental error, an insufficient flow of reinforcers, or
(Page & Neuringer, 1985). That statistic is a mea- any number of other things. To show that variability
sure of uncertainty or entropy and is calculated from depended on the “if vary, then reinforce” contin-
the relative frequencies of a set of items using the gency, a control procedure provided reinforcers after
equation some eight-response trials and time outs after others,
just as in the VAR condition, but these reinforcers
RFi • log ( RFi ) were now unrelated to the pigeon’s sequence varia-
U = −∑ . (1)
n log ( n ) tions. Under this control, reinforcers and time outs

were yoked to those delivered during the VAR phase.
Here, RFi refers to the relative frequency of element In other words, the yoke condition was identical to
i, out of n total elements. As a convention, every the VAR condition (eight responses per trial, and trials
RFi = 0.0 is considered to contribute a value of 0.0 were followed by food or time out at exactly the same
to the sum, without an attempt to resolve log(RFi). rates as during VAR), except that the pigeon received
When all elements occur with equal frequency, U is food and time outs whether or not the variability con-
maximal with a value of 1.0; if any single element tingency had been met. Each pigeon’s terminal six ses-
has a frequency of 1.0 (and all others are 0.0), then sions under the lag 50 VAR contingencies provided
U is minimal with a value of 0.0. the trial-by-trial schedule for its reinforcements and
In the Page and Neuringer (1985) study, three time outs under the yoke condition.
levels of U value were analyzed at the end of each The yoke procedure produced large and consis-
session. U value was calculated for the relative fre- tent effects. Levels of variability fell rapidly and
quencies of L and R; for relative frequencies of dyads remained low under the yoke condition. U values
(namely LL, LR, RR, and RL); and for triads (e.g., that had approached the 1.0 of a random model in
LLL, LLR, LRL . . .). The birds’ U values were com- the VAR condition dropped to values closer to 0.50,
pared with those from the random model. As indicating substantially more sequence repetition.
expected, the random model produced U values Other statistics confirmed the increased repetitive-
close to 1.0 at each level of analysis, and the pigeons’ ness and predictability of responding. These effects
U values also approached 1.0, although not quite as were replicated with an A-B-A-B design (VAR–yoke–
closely as the random model. Thus, in this case, the VAR–yoke), yielding a conclusion that direct rein-
results can best be described as randomlike but dis- forcement of variability was responsible for the high
criminably different from a true random source. variability (see Figure 22.1).
Rather than responding equiprobably, the birds Lag contingencies have been used with other
demonstrated biases, for example, favoring one key species, including humans, monkeys, rats, budgeri-
over another or favoring repetition over switching. gars, and fish (see Neuringer, 2002, for a review).
We return later to a more detailed discussion of With rats, four-response trials across L and R levers
whether operant responses can be generated sto- are often used, in which case 16 (or 24) different
chastically when the reinforcement contingencies sequences are possible. Here, too, high sequence
are more demanding. variability is observed. In an example of a human
515
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
response frequencies (rather than recency) provides

a partial solution, and we describe it next.
Frequency-Based Methods
In these procedures, reinforcement is contingent on
low overall relative frequencies. As one example,
rats’ responses were reinforced on the basis of four-
response trials (across L and R levers), with, as we
indicated, 16 different possible sequences (LLLL,
LLLR, LLRL, LLRR, etc.; Denney & Neuringer,
1998). Frequencies of these sequences were updated
throughout each session in 16 separate counters, or
bins, and a sequence was reinforced only if its rela-
tive frequency—the number of times that it was
emitted divided by the total number of sequences—
was less than some designated threshold value.
Reinforcement of low relative frequency sequences
Figure 22.1. Three levels of the U statistic, an has the advantage of permitting occasional rein-
index of behavioral variability (U1 based on number forcement of repetitions. This procedure has several
of left [L] and right [R] responses; U2 on LL, LR, RR, technical aspects. For example, after each trial, all
and RL dyads; and U3 based on triads), during the
lag 50 reinforcement-of-variability phases and yoke
bins are multiplied by an exponent, for example,
d-VR phases. VR = variable ratio; F = first session in 0.95, which results in recent sequences having more
each phase; L = last session in each phase. Adapted weight than those emitted in the past, a kind of
from “Variability Is an Operant,” by S. Page and A. memory decay (see Denney & Neuringer, 1998, for
Neuringer, 1985, Journal of Experimental Psychology:
Animal Behavior Processes, 11, p. 445. Copyright 1985 details). The important point is that, as with lag,
by the American Psychological Association. highly variable responding was generated. The pro-
cedure has been used in many experiments with
procedure, Stokes and Harrison (2002) presented on yoke serving as the control (Neuringer, 2002).
a computer screen a triangle consisting of one loca- In an interesting variant of the procedure, Don-
tion at the top, two locations in the next row down, ald Blough (1966) reinforced variable interresponse
three in the third, and so on until the sixth row, times (IRTs). Blough’s goal was a difficult one,
which contained six locations. A trial involved mov- namely, to see whether pigeons could learn to
ing from the top row to the bottom, thereby requir- behave like an emitter of atomic particles, the most
ing five responses, with 32 possible patterns. These random of physical phenomena (or, put another
five-response sequences were reinforced under lag way, to respond as would a Geiger counter). For a
contingencies, and high levels of variability often random-in-time responder, the likelihood of a
resulted. In this procedure, however, as well as oth- response is independent of whether a previous
ers with humans, some (albeit rarely observed) par- response had occurred recently. To accomplish this,
ticipants use a different strategy of cycling through a Blough created a series of IRT bins, such that a ran-
subset of sequences, such as a binary counting strat- dom responder would be expected to have an equal
egy. Another problem with lag procedures is that number of IRTs in each bin. A moving window of
they never reinforce repetitions, and a random gen- 150 responses was analyzed in real time, with each
erator sometimes repeats (e.g., if 16 sequences are response allocated into one of 16 bins, depending on
possible, then for a random generator, the probabil- its IRT. Blough then only reinforced an IRT falling
ity of two identical back-to-back sequences is .0625). in the bin with the lowest current relative frequency,
An alternative method that bases reinforcement on that is, he reinforced only for the least frequent2 IRT
As such, this contingency is called a least frequent contingency.
2
516
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
in a given window. The procedure resulted in the all behaviors are strictly determined—by inheri-
pigeons learning to approximate the IRT distribu- tance, experiences, stimuli, responses, and the
tions of a truly random generator, that is, to distrib- like—and therefore that random behavior is not
ute pecks (with some minor exceptions resulting possible, certainly not when voluntarily attempted.
from double pecks) much as a Geiger counter would However, none of the earlier studies tried to rein-
respond. force randomlike behavior directly. This approach
was accomplished with a procedure that required
Statistical Feedback Methods students to enter tens of thousands of responses at a
If the goal is to test whether animals and people can computer terminal (Neuringer, 1986). A trial con-
respond truly randomly, then both recency and fre- sisted of 100 responses across two keys (which we
quency methods have a potential weakness. As indi- refer to as 1 and 2) with feedback, based on com-
cated earlier, systematic strategies, such as binary mon statistical tests of randomness, presented at the
counting, can provide higher frequencies of reinforce- end of each trial. At first, satisfying one statistical
ment than responding stochastically. Although some- test was reinforced, then two tests had to be satis-
times present in animals, this type of strategy is most fied, and then three, and so on—with graphical
commonly observed in human participants. Note that feedback showing each statistic relative to that
exploiting weaknesses in the variability-reinforcing expected from a true random source—until partici-
contingencies—by responding in a systematic way pants were passing 10 evaluations of random
that maximizes reinforcement—is not a sign of insen- sequences.
sitivity to the schedule requirements. If anything, it is The challenge confronting the participants was
precisely the opposite. An additional problem is that even greater than just described. Participants were
all statistical tests of randomness (and therefore rein- required to generate a distribution of statistical val-
forcement contingencies based on randomlike ues that would be expected from a random source
responding) have certain blind spots that result in (Neuringer, 1986). To take a simple example, across
false positives. Therefore, researchers sought an alter- many trials of 100 responses each, random genera-
native procedure, one that was better at detecting, tion of 1s and 2s shows a distribution of proportions
and therefore not reinforcing, strategic or patterned of 1s and 2s. The most likely outcome would be
responding. It seemed reasonable to hypothesize that approximately equal numbers of 1s and 2s, or 50%
if a reinforcement contingency was based on a multi- each, but some trials would occur in which, for
plicity of different measures of variability, it might be example, there were 40% 1s and 60% 2s, or vice
less likely to reward exploitative strategies and poten- versa, and fewer trials of, say, 30% 1s (or 2s). The
tially lead more reliably to approximations to random participants’ task, therefore, was not simply to
outputs, especially in human participants. match the average of a random distribution but
Before we describe the relevant experiment, note more precisely to approximate the randomly gener-
that the attempt to reinforce randomness flies in the ated distributions. This task required many weeks of
face of more than 50 years of research in which peo- training, but all participants learned to approximate
ple were asked to generate random sequences (e.g., the random model according to 10 simultaneously
“Pretend you are flipping a coin”). The consistent applied statistics, and some participants (but not all)
conclusion from this large body of studies was that were able to pass additional tests as well.
people do not respond randomly when so requested How should this research be interpreted?
(Brugger, 1997), and indeed, some researchers con- Because so much training was necessary, it would
cluded that people cannot respond randomly. (The seem that the ability to respond unpredictably is
literature on human randomness rarely references unnatural, but that would be a misinterpretation. If
nonhuman animal studies.) This conclusion is fun- you, the reader, were to call out 100 instances of
damentally important because randomness implies heads and tails and try to do so unpredictably, it is
absence of identifiable causes and independence unlikely that an observer of your behaviors, even
from determination. Most psychologists assume that with access to sophisticated statistical analyses,
517
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
could predict your responses with a high degree of difficult for the subject to produce never-before-seen
accuracy. You can, without training, respond quite behaviors and increasingly difficult for the observer
unpredictably. The requirement to pass 10 statistical to discriminate among the various behaviors being
tests, however, demands equivalence, over the long emitted. At least over the short run, however, rein-
run, of instances, dyads, triads, and the like and forcing novel responses led to an exceedingly high
absence of all biases. To use a rat’s operant response level of unpredictable behaviors.
as an analogy, it is quite easy to train a rat to press a The Pryor et al. (1969) study was followed by an
lever for food pellets. Indeed, that can often be analogous one with preschool children (Goetz &
accomplished in an hour-long session. To train a rat Baer, 1973). The children were rewarded for block
to respond precisely at some given force, or with constructions that differed from any that had previ-
precise interresponse intervals, may take weeks or ously been observed during the session. As training
months of training. In similar fashion, approximat- proceeded, the children built increasingly varied
ing true randomness is difficult to attain, but forms, including ones never before made by the
responding variably, and to large extent unpredict- child. Similar results were obtained with the draw-
ably, is readily achieved. In rats, for example, highly ing of color pictures as the target behavior (Holman,
variable operant responding is obtained within a few Goetz, & Baer, 1977).
sessions (McElroy & Neuringer, 1990). The evidence from many methods has therefore
shown control over response variability by directly
Novelty-Based Methods contingent reinforcers (see also Hachiga & Saka-
To evaluate variability, each of the methods dis- gami, 2010; Machado, 1989, 1992, 1997). Variabil-
cussed to this point requires that a set of possibili- ity is highest when reinforcers follow high
ties be explicitly defined—responses, sequences, variability. In the next section, we show that rein-
paths, or times. Mathematical definitions of random- forcers exert even more precise control than that:
ness, statistical analyses, and reinforcement contin- Levels of variability can be specified, levels that span
gencies depend on such specification. However, in the range from response repetitions (or stereotypy)
many outside-of-lab situations, the set may not be to response unpredictability. As such, variability
known, and an alternative method allocates rein- parallels other operant dimensions in which rein-
forcers for novel, or not previously emitted (e.g., forcers influence exactly how fast to respond or
within a session or ever) responses. Reinforcement when, with what force, or at which location.
of novel responses was first used by Pryor, Haag,
and O’Reilly (1969) in research with porpoises. At Levels of Variability
the beginning of each session, Pryor et al. waited Other experiments in the Page and Neuringer
until they observed some behavior not previously (1985) article described earlier applied different lag
emitted by the porpoise and then selected the new values in different phases, from lag 1 (the current
behavior for consistent reinforcement during the sequence of eight responses had to differ from the
session. This procedure resulted in the porpoise single previous sequence) to lag 50 (the current
sequence had to differ from each of the previous 50
emitting an unprecedented range of
sequences). As the lag increased, requiring that
behaviors, including aerial flips, gliding
sequences be withheld for an increasing number of
with the tail out of the water, and “skid-
trials, responses generally became increasingly
ding” on the tank floor, some of which
unpredictable (as assessed by U values, number of
were as complex as responses normally
different sequences per session, and other statistics;
produced by shaping techniques, and
see also Machado, 1989). Frequency-based methods
many of which were quite unlike anything
show similar control over levels. For example,
seen in . . . any other porpoise. (p. 653)
Grunow and Neuringer (2002) used a different
One problem with long-term use of this procedure, threshold reinforcement criterion with each of four
however, is that, over time, it becomes increasingly groups of rats: one that required rats to distribute
518
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
three-response sequences (across three different variability as a function of the reactions of their play-
operanda) in a way that paralleled a random genera- mates. When entertaining a child, the actions of an
tor (high variability), another that required adult sometimes include surprises, such as tickling,
medium-high variability, another that required as well as repetitions, such as repeatedly bouncing a
medium-low variability, and the last that permitted child on one’s lap, and the child’s reactions influence
frequent repetitions. Levels of variability were again the action’s (un)predictability. Similarly, in conver-
controlled by these specific requirements, as shown sations, the speaker is (often) sensitive to the reac-
by the leftmost points in Figure 22.2 (the other tion of the listener with variations in topic as well as
points in the figure are discussed later). Several in prosody, loudness, and speed. Unpredictability is
additional studies have demonstrated reinforcement particularly important in competitive situations.
control over precise levels of variability in pigeons Consider the example of table tennis: When a skilled
(Neuringer, 1992) and people (G. Jensen, Miller, & player (S) plays with a beginner (B), S will often
Neuringer, 2006). return the ball in a way that B can easily predict, but
Precisely controlled levels of behavioral as B becomes increasingly capable, S will vary ball
(un)predictability can be observed in many natural placement and speed until a high level of unpredict-
situations. Variable behaviors are used to attract ability is (sometimes) manifest. Precise control of
attention, as when male songbirds increase the vari- levels of unpredictability plays a substantial role in
ability of their songs in the presence of a receptive game theory, under the rubric of mixed strategies
female (Catchpole & Slater, 1995). During play (see Glimcher, 2003; Smith, 1982). These examples
and games, animals and people modulate levels of are only a few of the commonplace variations in
Figure 22.2. U value as a function of reinforcement frequencies. Each line rep-

resents a different group: .037 = very high variability (var) required for reinforce-
ment; .37 = very low variability required; .055 and .074 = intermediate levels
required. CRF = continuous reinforcement, or reinforcement every time variability
contingencies were met; VI 1 = variable-interval reinforcement for meeting vari-
ability contingencies no more than once per minute, on average; VI 5 = variable-
interval reinforcement no more than once every 5 minutes. From “Learning to Vary
and Varying to Learn,” by A. Grunow and A. Neuringer, 2002, Psychonomic Bulletin
and Review, 9, p. 252. Copyright 2002 by the Psychonomic Society, Inc. Adapted
with permission.
519
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
levels of response (un)predictability that character- of the rectangles were evaluated: area (the number
ize many real-world operant behaviors, variations of pixels enclosed by the rectangle), location (the
that are controlled by consequences. We discuss position of its center point), and shape (its height-
additional real-world applications later. to-width ratio). To be reinforced, the rectangles had
to vary along two of these dimensions while repeat-
Orthogonal Dimensions ing along the third. The participants were told noth-
As indicated in the introduction, reinforcement ing about these criteria, and the only instructions
often depends on a combination of many aspects of were to gain points by drawing rectangles. Partici-
a response. For example, a child may receive a rein- pants were randomly assigned to one of three
forcer for saying “thank you” but only when the groups, with rewards delivered in one group when
child (a) speaks slowly and (b) makes eye contact. the areas of the drawn rectangles were approxi-
Because responses can vary across many dimensions mately the same, trial after trial, but locations and
independently from one another, one can readily shapes varied. The other two groups had analogous
imagine circumstances in which it might be func- contingencies, but for one, locations had to repeat,
tional to vary some dimensions of behavior while and for the other, shapes had to repeat. All partici-
keeping others highly predictable. pants learned to meet their respective three-part
A demonstration of the independent reinforce- contingencies, varying and repeating as required
ment of variability and predictability along indepen- (Figure 22.3). Thus, binary feedback—reinforcement
dent dimensions was provided by Ross and or not—influenced variability and repetitions along
Neuringer (2002). They instructed college students three orthogonal dimensions and did so indepen-
to earn points in a video game involving drawing dently, thereby highlighting the precise, multifac-
rectangles on a computer screen. Three dimensions eted way in which reinforcers control variability.
Figure 22.3. U values for each of three dimensions of rectangles drawn by

participants in three separate groups. One group was required to repeat the
areas of their rectangles while varying shapes and locations (left set of bars), a
second group was required to repeat shape while varying areas and locations
(middle set of bars), and a third group was required to repeat location while
varying areas and shapes (right set of bars). Error bars indicate standard errors.
From “Reinforcement of Variations and Repetitions Along Three Independent
Response Dimensions,” by C. Ross and A. Neuringer, 2002, Behavioural
Processes, 57, p. 206. Copyright 2002 by Elsevier B.V. Adapted with permission.
520
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
As we show in the next section, reinforcers exert sequences were reinforced in the presence of red
other simultaneous influences: They select the set or keylights (lag schedule). Blue and red alternated
class of responses from which instances emerge and, after every 10 reinforcers under what is referred to as
simultaneously, the required level of variation. a multiple schedule (two different reinforcement con-
tingencies presented successively, each correlated
Response Sets and Variations with a distinct stimulus). The birds learned to repeat
Whenever variability is reinforced, a set of appropri- in the presence of blue and to vary in the presence of
ate responses is also strengthened. Reinforcers select red, and when the stimulus relationships were
the set from which variations emerge. Mook and reversed, the birds varied in the presence of blue
Neuringer (1994) provided experimental evidence while repeating in the presence of red. In another
for this point. In the first phase, rats’ variable four- experiment, rats learned to emit variable four-
response sequences across L and R levers (lag sched- response sequences across L and R levers in the pres-
ule) were reinforced. In the second phase, only ence of one set of lights and tones and repeated a
sequences that began with two right responses, RR, single pattern, LLRR, in the presence of different
were reinforced. Thus, now only RRLL, RRLR, stimuli (Cohen, Neuringer, & Rhodes, 1990). In an
RRRL, and RRRR patterns were effective. In the first even more stringent test by Denney and Neuringer
phase, all 16 possible sequences were emitted, (1998), rats’ variable sequences were reinforced in
whereas in the second phase, most sequences began one stimulus, whereas in a yoke stimulus, reinforcers
with two right responses, RR. Thus, the reinforce- were delivered at exactly the same rate and distribu-
ment contingency generated behaviors that satisfied tion but independent of variability. The cues came to
the appropriate set definition while simultaneously exert strong differential control, and when variability
producing a required level of variability within that was required, the animals varied; when variability
set. In another experimental example (Neuringer, was not required but permitted in yoke, response
Kornell, & Olufs, 2001), rats responded in cham- sequences became more repetitive and predictable.
bers containing five operanda: left lever, right lever, These results indicate that an individual may behave
left key, center key, and right key. In one phase, in a habitual and predictable manner in one context,
reinforcers were contingent on variations across whereas in a different context, perhaps occurring
only three of the operanda (left and right levers and only a few moments later, the same individual will
center key), and the rats learned to respond variably respond unpredictably or in novel ways. The results
across only those three. A binary event—reinforce further indicate (along with other VAR–yoke com-
or not—can function simultaneously to define a parisons described earlier) that to engender highly
response class and levels of (un)predictability along variable behaviors, it may be necessary to reinforce
multiple dimensions of the within-class instances. variability explicitly rather than, as in laissez-faire
This result shows an extraordinary diversity of con- environments, simply permit individuals the free-
trol by simple reinforcement operations. dom to vary. To the extent that individual freedom
depends on the possibility of variation, reinforce-
Discriminative Stimuli ment plays an important role (a topic to which we
Operant responses are generally influenced by dis- return in the final sections of this chapter).
criminative stimuli, that is, cues that indicate rein-
forcer availability. If pigeon pecks are intermittently Endogenous Stimulus Control
reinforced when the keylight is red but not when it is The discriminative stimuli described in the previous
green, the birds learn to peck almost exclusively section were external to the organism and publicly
when the keylight is red. Discriminative stimuli con- observable. Another form of discriminative control
trol levels of variability as well. For example, Page depends on the interactions of an organism with a
and Neuringer (1985, Experiment 6) reinforced rep- reinforcement schedule. An example of such endog-
etitions of a single sequence of key pecks, LRRLL, in enous stimulus control is seen in the pauses that
the presence of blue keylights, whereas variable follow reinforcers under fixed-interval schedules.
521
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
The reinforcers serve as indicators that for some noncontingent influences often interact in important
period of time, reinforcement is not possible. Hopson, ways with variability-contingent reinforcers.
Burt, and Neuringer (2002) showed that response–
reinforcer relationships exert discriminative control Random Events
over levels of variability as well (see also Neuringer, Many behavioral trajectories are initiated by the
2002). Rats’ responses were reinforced under a accidental confluence of the organism with one or
schedule in which two periods alternated, VAR and more environmental happenings. Hurricanes, earth-
repetition (REP), but these periods were not cued by quakes, and wars change behaviors in ways that can-
external stimuli (technically, a mixed schedule). In not readily be anticipated. Winning a lottery is a
the VAR period, four-response sequences of L and R happier example. Another might be happening to sit
lever presses were reinforced if they met a threshold next to a particular individual on a cross-country
variability contingency; in the REP period, only rep- flight, which leads to a long-term romantic relation-
etitions of LLLL were reinforced. (Probabilities of ship (see Bandura, 1982; Taleb, 2007). These events,
reinforcement were equalized in the two periods by although randomly related to the individual’s behav-
intermittent reinforcement of LLLL in REP.) After ior, have important long-term influences.
the schedule transitioned into the VAR component, Random events have been purposively used
responding began to vary within a few trials, and throughout history to guide behaviors, for example,
variations continued until the schedule transitioned throws of dice, randomly selected sticks, cards,
into REP, with responding soon reverting to LLLL. bones, or organs. Today, a referee flips a coin at the
These results indicate that the variability produced beginning of a football game to decide which team
when reinforcement is withheld for short periods, as can choose whether to kick the ball; a computer’s
when a new response is being shaped, may partly be random-number generator assists scientists with
discriminatively controlled despite absence of exter- avoiding biases in assigning subjects to experimental
nal cues; that is, animals and people may learn when groups; and alleotoric events are used in modern art,
it is functional to vary, and some of the cues may music, and literature. The Dice Man by Rhinehart
come from response–outcome relationships. (1998) provides a fictional example of intentional
use of random artifacts. The protagonist, bored with
life, writes a number of possible actions on slips of
Noncontingent Effects
paper and then periodically selects one blindly and
Until this point, we have focused on contingencies behaves accordingly. These examples show that ran-
that directly relate reinforcers to variability. All dom events that are independent of an individual’s
operant responses are also influenced by events that actions may be used to avoid biases, engender
are not directly contingent on the responses, some- unlikely responses, and break out of behavioral ruts.
times referred to as eliciting or inducing influences,
respondents, or establishing operations. For exam- Evolved Responses
ple, levels of deprivation, injections of drugs, and Modes of unpredictable behavior have evolved that
ambient room temperature can all influence learning permit organisms to escape from or avoid predators
and maintenance of operant responses. Even non- or aggressors. These behaviors have been referred to
contingent aspects of the reinforcement operation as protean behaviors that are “sufficiently unsystem-
itself may have important effects, for example, attri- atic in appearance to prevent a reactor predicting in
butes such as the quality and quantity of food (refer- detail the position or actions of the actor” (Driver &
ring here to when these do not change as a function Humphries, 1988, p. 36). Examples include the ran-
of behavior). Thus, to understand operant respond- dom zigzags of butterflies, stickleback fish, rabbits,
ing, including operant variability, these other influ- and antelopes when being attacked. One conse-
ences must be considered. We turn to a discussion quence of evolved protean behavior is that it inter-
of effects of noncontingent events on operant vari- feres with a predator species’ evolving a response to
ability. As we describe at the end of this section, a specific escape or avoidance pattern. In brief,
522
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
protean behaviors demonstrate evolved randomlike interpretation of these effects is that low expectation
responses to eliciting stimuli. (or anticipation) of reinforcers induces variability
(Gharib et al., 2004). Whatever the explanation, it is
Schedules of Reinforcement and important to be able to identify whether variability
Expectancy is selected by reinforcers or pushed by states of the
Both in the laboratory and in the real world, it is body (endogenous inducers) or environmental
common for responses to be intermittently (or occa- events, including noncontingent effects of reinforc-
sionally) reinforced. Much operant conditioning ers. Discriminating between selection and induction
research is devoted to documenting the effects of will facilitate the modification of variability when
such schedules. To take one example, under a fixed- that is desirable.
ratio schedule of reinforcement, a fixed number of
responses (say, 30) is required to gain access to a Experience
pellet of food. After receipt of each pellet, it is Thorndike (1911) and Guthrie and Horton (1946)
impossible to obtain another immediately, because described the responses of cats that had been con-
30 additional responses are required. As was the fined in a puzzle box and who received food contin-
case for the fixed-interval schedules mentioned ear- gent on escape. Response topographies were highly
lier, pauses are generally observed after reinforce- variable at first but, over trials and rewards, became
ment, or lower rates of responding, as compared increasingly predictable and stereotyped. Antonitis
with later in the ratio, when access to reinforcement (1951) studied nose pokes by rats along a long hori-
is possible. zontal slit. When pokes produced access to food,
In addition to these effects on response rate, location variability decreased across trials. Notter-
response variability is also found to change under man and Mintz (1965) measured the force exerted
similar reinforcement schedules. In the cases dis- by rats on a response lever and found that across
cussed here, variability plays no role in the contin- training, force decreased, approaching the minimum
gency, that is, reinforcers do not depend on level necessary to operate the lever, with force vari-
response variations. However, responding tends to ability decreasing as well. Brener and Mitchell
become increasingly repetitive and predictable as an (1989) extended these results to the total energy
anticipated reinforcer is approached in time or num- expended by a rat in an operant conditioning cham-
ber. This tendency was shown for variability across ber. A last example comes from Vogel and Annau
two levers when a fixed sequence of responses was (1973), who reinforced pecking three times on a left
the operant (Cherot, Jones, & Neuringer, 1996), for key and three times on a right key, in any order.
variability of lever-press durations also under ratio Across sessions, a marked increase occurred in the
schedules (Gharib, Gade, & Roberts, 2004), and for predictability (stereotypy) of the pigeons’ patterns of
variability of movements across a chamber space response. A general consensus has therefore
when access to a potential sexual reinforcer is emerged: Variability of operant behavior decreases
approached (Atkins, Domjan, & Gutierrez, 1994). with experience. This conclusion, however, may
In each of these cases, variability is relatively high apply mainly to situations in which every response
when reinforcers are distant with respect to effort, or sequence leads to a reinforcing consequence and
time, or space, and responding becomes more pre- to situations in which high variability is not differ-
dictable as reinforcers are neared (see also Craig, entially reinforced.
1918). These changes in response predictability are
said to be induced by the schedule of reinforcement. Extinction
Another variable shown to induce differences in After long-term experience in which responses pro-
response variability is reinforcement frequency. In duce reinforcers, suddenly withholding reinforcers—
general, response variability is high when reinforc- referred to as extinction of responding—increases
ers are infrequent and low under high-density rein- variability. In the experiment by Antonitis (1951)
forcement (Lee, Sturmey, & Fields, 2007). One noted earlier, after the rats were accustomed to
523
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
producing food reinforcers by poking their noses

anywhere along a horizontal opening, food was
withheld, which caused an increase in location
variability. Extinction-induced variability has been
seen along many other response dimensions: loca-
tion (Eckerman & Lanson, 1969), force (Notterman
& Mintz, 1965), topography (Stokes, 1995), and
number (Mechner, 1958). One contrary result is
often cited, namely a study by Herrnstein (1961) in
which variability of the location of pigeon pecks
along a continuous strip was reported to decrease
during a period of extinction. However, the extinc-
tion in that study followed a phase in which every
response was reinforced (continuous reinforce-
ment) in an A-B design (first a reinforcement phase,
then extinction, without return to the first phase).
Experience may therefore have confounded the
results. In general, extinction causes variability to
increase.
The variations induced by extinction generally
emerge from the class of responses established dur-
ing original learning. For example, if lever pressing
produced food pellets, a rat may vary the ways in
which it presses when food is withheld, but much of
the behavior will be directed toward the lever (e.g.,
Figure 22.4. The top graph shows the propor-
Stokes, 1995). Neuringer et al. (2001) quantified the tion (or probability) of occurrences of the three-
bounded nature of extinction-induced variability response patterns shown along the x-axis during
that was observed after rats had been rewarded for a period when a single sequence (left lever, key,
repeating a single sequence across two levers and a right lever) was being reinforced (filled circles)
and during a period of extinction, when reinforc-
key: left lever, key, right lever (LKR), in that order. ers were withheld completely (open circles). The
The top panel of Figure 22.4 shows the distribution bottom graph shows the ratio of responding during
of the relative frequencies of each of the possible extinction (EXT) to responding during reinforce-
ment (REIN; i.e., the ratio of the two curves in the
sequences (proportions of occurrences) during the upper graph). Together, the graphs show that pat-
conditioning, or reinforcement, phases (filled cir- terns of responding during extinction were similar
cles) and during extinction (open circles). The LKR to those during reinforcement, but high-frequency
sequence was, of course, most frequent during the sequences decreased and low-frequency sequences
increased during the extinction phase. Adapted
reinforcement phase, with other, somewhat similar from “Stability and Variability in Extinction,” by
sequences falling off in terms of their frequencies. A. Neuringer, N. Kornell, and M. Olufs, 2001,
The LKR sequence was also most frequent through- Journal of Experimental Psychology: Animal Behavior
out the extinction phase—during which time Processes, 27, p. 89. Copyright 2001 by the
American Psychological Association.
response rates fell to low levels—with the two
curves being quite similar. (Note that these curves
show relative frequencies. Absolute rates of response the ratio of the two curves in the upper graph). The
were much lower during extinction than during the take-home message is that the basic form of the
reinforcement phase.) Also shown at the bottom of behavior was maintained during extinction, and
the figure are the ratios of response proportions dur- variability increased because of the generation of
ing the reinforcement and extinction phases (i.e., unusual or highly unlikely sequences (for related
524
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
findings, see Bouton, 1994). Extinction was there- variability, indicated by the U values in Figure 22.2.
fore characterized as resulting in a “combination of The individual curves represent the four variability
generally doing what worked before but occasion- thresholds, and the x-axis represents frequencies of
ally doing something very different. . . . [This] may reinforcement. The four thresholds exerted primary
maximize the possibility of reinforcement from a control, that is, the groups differed in variability
previously bountiful source while providing neces- throughout the experiment. Effects of reinforcement
sary variations for new learning” (Neuringer et al., frequency were more subtle and depended on the
2001, p. 79). Knowledge of these effects can be threshold requirements, an interaction effect. When
applied to one’s own behavior as well as to others. the contingencies were lenient and low levels of
When in a rut, or unproductive or dissatisfied, variability sufficed for reinforcement, variability
avoiding those reinforcers that had been produced increased as reinforcement rates fell (from continu-
by habitual behaviors may help. ous reinforcement to VI 1 to VI 5). When the con-
tingencies were demanding and high levels of
Interactions variability were reinforced, the opposite occurred,
Noncontingent inducers often interact with that is, variability decreased with decreasing rein-
variability-contingent reinforcers to control levels of forcements. The intermediate groups showed inter-
response variability. Additional phases in the mediate effects. A similar interaction was obtained
Grunow and Neuringer (2002) experiment described when delays were introduced between the end of a
in the Levels of Variability section provide one exam- varying sequence and reinforcement (Wagner &
ple. In the first phase of that experiment, recall that Neuringer, 2006). Thus, when reinforcers are con-
high, medium-high, medium-low, and low levels of tingent on variability, the contingency exerts a
response-sequence variability were reinforced across strong—and often primary—effect, but that effect is
groups of rats, resulting in different levels of modified by noncontingent influences, including
response variability across the four groups. Two reinforcement rates and delays. Levels of response
additional phases followed in which, although the variability depend on both contingent and noncon-
four different variability criteria were unchanged, tingent influences.
overall frequencies of reinforcement were systemati- Interactions between variability-contingent and
cally lowered by providing reinforcement only inter- variability-noncontingent reinforcement may help to
mittently. In particular, a variable-interval (VI) explain effects seen outside of the lab. Repetitive
schedule of reinforcement was superimposed on the behaviors are required for many workers (e.g., factory
variability contingency: first a VI 1 minute (such that workers, mail carriers, fare collectors), but for others,
food pellets were limited to an average of once per variable (and unpredictable) behaviors are the norm
minute, with unpredictable gaps of time between (e.g., inventors, fashion designers, artists). Lowering
food deliveries) and then VI 5 minute (limiting food pay or withholding positive feedback may affect
pellets to no more than once, on average, every 5 behaviors differently in these two cases. Thus, to pre-
minutes). Under the VI schedules, after an interval dict effects on behavioral variability, one must know
elapsed, the first trial to meet the variability contin- both contingent and noncontingent relationships.
gency ended with a reinforcer. All other trials ended Cherot et al. (1996) described a different interac-
with a brief time out (whether or not the variability tion that may also help to illuminate real-world
requirement had been satisfied). effects. In that experiment, repeated response
As reinforcement frequencies were lowered, sequences across two levers were reinforced in one
response rates fell in all four groups and did so group of rats (REP) and sequence variability was
equally, that is, all groups responded much more reinforced in another group (VAR). Not every
slowly when varying sequences were reinforced on sequence that met the VAR or REP contingency
average once every 5 minutes than when they were gained reinforcement, however; rather, a superordi-
reinforced each time that they met the contingen- nate fixed-ratio 4 also had to be satisfied. That is, the
cies. However, different results were obtained for REP group had to successfully repeat a sequence
525
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
four times to get a single reinforcer, and the VAR

group had to successfully vary a sequence the same
number of times. For example, in the REP group,
one animal may have emitted LLLR in the first trial
after a reinforcer, but that correct sequence caused
only a signal indicating correct. No food was given.
If the next trial also contained LLLR, the correct sig-
nal was again provided. If the following trial was
LLLL, then a brief time out was given. This process
continued until the fourth correct LLLR sequence
produced the signal plus a food pellet. Exactly the
same procedure was in place for the VAR animals,
except they were correct only when they met a lag
variability contingency.
As shown in Figure 22.5 (bottom), the main
result was that the VAR animals responded much
more variably overall than did the REP animals,
again indicating primary control by the variability
and repetition contingencies. However, as reinforce-
ment was approached (i.e., as the last of the four
successful sequences was neared), levels of variabil-
ity decreased for both VAR and REP groups. Recall
the expectancy-of-reinforcement effects described in
the section Schedules of Reinforcement and Expec-
tancy earlier in this chapter. In this case as well,
variability decreased as reinforcers were approached,
thereby facilitating correct responding in the REP
group but interfering with it in the VAR group (Fig-
ure 22.5, top panel). Let us pause for a moment to Figure 22.5. The top graph shows per-
consider this surprising finding. Despite the fact centages of sequences that met variability
(VAR) or repetition (REP) contingencies
that variability was being reinforced in the VAR as a function of location within a fixed-
group, as the reinforcer was neared, the likelihood ratio 4 (FR 4) schedule. The lines connect
of varying decreased. It is important to note again means for groups of rats, and the error bars
that reinforcement of variability generated much indicate standard deviations. The lower
graph shows U values, an index of sequence
higher levels of variation overall than did reinforce- variability, for the two groups across the
ment of repetitions, a variability-contingent effect, FR schedule. Adapted from “Reinforced
but superimposed was an expectancy-inducing Variability Decreases With Approach to
Reinforcers,” by C. Cherot, A. Jones, and
decrease in variability. Similar interactions may help
A. Neuringer, 1996, Journal of Experimental
to explain effects of reinforcers on other types of Psychology: Animal Behavior Processes, 22,
behavior, including creative behaviors, a topic that p. 500. Copyright 1996 by the American
we discuss in the Applications section. Psychological Association.
discussed explanations of operant variability, namely

Memorial and Stochastic
memorial and stochastic processes. According to the
Explanations
memorial explanation, each response can be related
We have described types of events that result in vari- to or predicted from prior stimuli or responses.
able behaviors. Now, we examine two commonly According to the stochastic-generator hypothesis,
526
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
individual responses are unpredictable because of reinforced. By alternating, LRLRLR . . . , every

the nature of a random process. That is, individual response is reinforced, and birds developed just
responses do not have identifiable individual causes, such an alternation strategy. When the sequence
a hypothesis that many consider problematic. We consisted of two responses, the birds again devel-
consider each of these explanations and argue that oped memory-based sequences, for example, repeat-
the evidence for both of these hypotheses is good ing RRLLRRLL. However, when the sequence was
and therefore that behaving (more or less) unpre- increased to three responses, such that reinforce-
dictably derives from multiple sources. ment was given for responses in the least frequent
three-response bin, the birds apparently could not
Memory-Based Variability develop the optimal fixed pattern of RRRLRLLL . . .
Memory is a shorthand way to refer to the influence but instead reverted to randomlike behavior (Mach-
of past events that are separated in time from a cur- ado, 1993, p. 103). Thus, a memory-based strategy
rent response. The term is not intended to connote was used when that was possible, but when the
conscious awareness (although that might be memory demands became too high, stochastic
involved) but rather potentially identifiable influ- responding emerged. A similar pattern was seen
ences (or causes). To the extent that memorial pro- with songs generated by a songbird, a budgerigar,
cesses are responsible for variability generation, under lag contingencies of reinforcement (Manabe,
prediction of individual responses is possible, even Staddon, & Cleaveland, 1997). Under lag 1, the
when the overall output is variable; thus, each mem- birds tended to alternate between two songs; under
ber of a variable sequence could be said to be deter- lag 2, they cycled among three songs. When the lag
mined by prior events. was increased to 3, however, song diversity and vari-
At the outset of this chapter, we indicated that ability increased appreciably. Thus, under recency-
under lag 50 schedules, in which the current and frequency-based methods of variability
response sequence must differ from each of the pre- reinforcement, variable responses are generated via
vious 50 sequences, responding was highly variable memorial processes when possible, but reversion to
and, indeed, approached that expected of a stochas- stochastic-like emission is seen when memory
tic generator. However, behaviors are often quite requirements exceed the organism’s capacity.
different under lag 1 or 2 schedules. In these cases,
the current sequence must differ from only the pre- Chaotic Responding
vious one or two, and memory-based response strat- Memory-based strategies can be used in other ways
egies frequently emerge: Animals and people as well. For example, chaotic processes generate out-
sometimes cycle repeatedly through two or three puts that are so noisy that it is exceedingly difficult
sequences, apparently basing a current response to distinguish them from stochastically generated
sequence on the just-emitted sequences. The cycling ones. “Chaos [is] a technical term . . . refer[ring] to
strategy produces reinforcement for every sequence, the irregular, unpredictable behavior of determinis-
which is a better rate of return than responding in a tic, nonlinear systems” (R. V. Jensen, 1987, p. 168).
stochastic-like manner.3 The advantage is, however, Chaotic behavior is both highly variable and pre-
only conferred when the memory demands are cisely controlled by prior events (Hoyert, 1992;
within the subject’s capacity. Mosekilde, Larsen, & Sterman, 1991; Townsend,
In a demonstration of the latter point, Machado 1992). A study by Neuringer and Voss (1993) asked
(1993) studied pigeons pecking L and R keys under whether people could learn to generate chaotic-like
a frequency-dependent variability contingency. responses. They used one example of a chaotic func-
Under this schedule, if the sequence is composed tion, the logistic difference function:
of just one response, then pecking the key that
had been pecked least frequently in the past will be R n = t • R n−1 • (1 − R n−1 ) . (2)

For example, stochastic responding on two alternatives under a lag 1 contingency earns 50% reinforcement, whereas alternating earns 100%.
3
527
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Here, Rn refers to the nth iteration in a series, each R

is a value between 0.0 and 1.0, and t is a constant
between 1.0 and 4.0. The current value of the logis-
tic difference function (Rn) is based on the previ-
ously generated value (Rn −1). The process begins
with an arbitrary seed value for the R0 between 0 and
1, which is used to calculate R1. Apart from the initial
seed value, the function is completely self-generated,
with each value determined by the just-prior value,
together with the constant parameters.
Chaotic outputs have two identifying characteris-
tics. First, given a constant value for t that approaches
4, for example, 3.98, the generated sequence approxi-
mates a random one, that is, it passes many tests for
randomness. Outputs are noisy and apparently
unpredictable. However, second, if the current value
of Rn is plotted as a function of the just prior value,
Rn− 1, a predictive structure can be identified. In the
particular case of the logistic difference function, the
form of this autocorrelated relationship is a parabola
(different chaotic functions show different types of
internal structures). Thus—and this is the identifying
attribute of chaotic processes—a deterministic math-
ematical function can generate randomlike outputs
with prediction of each value of the function possible
given precise knowledge of parameters and prior val-
ues. The outputs are extremely noisy and, at the same
time, identifiably determined.
In the Neuringer and Voss (1993) study, college
students were shown, after each response, the differ-
ence between their responses and that of the iterated
logistic difference model. With training, the students
became increasingly adept at responding in chaotic-
like fashion—the students’ responses matched
closely the iterations of the logistic function—and
their autocorrelations increasingly approximated a
parabola (Figure 22.6). Because each iteration in the Figure 22.6. Responses in trial n as a function of
logistic difference sequence is based on the prior responses in trial n − 1 during the first 120 responses
output, Neuringer and Voss hypothesized that the of the experiment (left column) and final 120 responses
human subjects also remembered prior responses. (right column). Each row of graphs represents data
from a single participant. The drawn lines show the
Put simply, the subjects may have learned (or mem- best-fitting parabolas. From “Approximating Chaotic
orized) a long series of “if the previous response was Behavior,” by A. Neuringer and C. Voss, 1993,
value A, then the current response must be value B” Psychological Science, 4, p. 115. Copyright 1993 by the
Association for Psychological Science. Adapted with
pairs, a suggestion made by Metzger (1994) and by permission.
Ward and West (1994).
To test this memory hypothesis, Neuringer and
Voss (2002) interposed pauses (IRTs) between each
528
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
response, that is, they slowed responding (see also of the items (the two colors, in our example). It is
Neuringer, 2002). As IRT durations increased, the dif- also true that the greater the number of different
ference between the subjects’ sequences and the mod- item classes, for example, different colors, the less
el’s chaotic output increased, and the underlying predictable any given instance will be. If the urn
parabolic structure was disrupted, providing evidence contained equal numbers of 20 different colors, for
that the highly variable responding was memory based. example, then the chance level of prediction would
be .05 (rather than .50 in the two-color case). Dis-
Stochastic Generation cussion of these concepts in historical context can
Stochastic generation has been hypothesized at be found in Gigerenzer et al. (1989).
numerous points in the chapter. Here we discuss in When trying to ascertain whether a finite
more detail what the stochastic hypothesis involves sequence of outputs was randomly generated, the
and possible ways to test it. The issue is complex, best one can do is to estimate the probability that a
difficult, and important. If variable operant random process is involved. For example, if 100
responses are generated stochastically, then it may selected balls were all blue, it would be unlikely but
not be possible to predict individual responses at not impossible that the balls were selected randomly
greater than chance levels. Stochastic generation from an urn containing an equal number of red and
may also be relevant to operant responses generally blue balls. Given a random generating process, any
and to explanations of their voluntary nature, as we subsequence of any length is possible, and every par-
discuss later. A researcher confronts many prob- ticular sequence of outcomes of a given length is
lems, however, in attempting to decide whether a exactly as likely as any other (see Lopes, 1982).
particular response stream is random or not and These considerations indicate the impossibility of
confronts additional difficulties when trying to proving that a particular finite sequence deviates
determine whether it has been generated by a ran- from random: The observed sequence may have been
dom process (see Nickerson, 2002). selected from an infinite random series (see Chaitin,
To get an intuitive sense of what random implies, 1975). However, the probability of 100 blue balls is
imagine an urn filled with 1,000 colored balls. The extremely low in our example, and the probability is
urn is well shaken, and one ball is blindly selected. much higher for sequences that contain approxi-
After selection, the ball’s color is noted, the ball is mately 50% red and 50% blue. Thus, one can evalu-
returned to the urn, and the selection process is ate the probability that a given output was generated
repeated. If the urn contains an equal number of by a stochastic process having particular parameters.
blue and red balls, then prediction of each ball’s A second problem is that as demonstrated by
color will be no better than chance; that is, the prob- chaos theory, seemingly random outputs may be
ability of a correct prediction would be .50. The generated by nonrandom processes. Another exam-
repeated selections represent a random process4 ple is given by iteration of the digits of pi. Someone
with the resulting output being a random sequence. could memorize the first 100 digits of pi and use
Note that predictions can be better than 50% for those to generate a randomlike sequence. Thus,
random processes, as shown by the following: If the behavioral outputs can be highly variable but (given
urn was filled with an uneven number of different sufficient knowledge by an observer) predictable.
colored balls, prediction could become increasingly How, then, can one test whether highly variable
accurate. For example, if the urn contained 900 red operant responses derive from a stochastic process?
balls and 100 blue balls, then prediction accuracy The test must involve a comparison—which of two
would rise to .90 (if one always predicted red). hypotheses is most likely to account for the data?—
However, the process and output are still referred to and the most likely alternative is the one already
as stochastic. Thus, stochastic outputs are more or discussed, namely, memory-based processes
less predictable depending on the relative frequencies under which each response can be predicted from
Specifically, this process is “random with replacement” on account of the act of returning the ball to the urn. All discussions of randomness in this
4
chapter refer to this type of randomness.
529
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
knowledge of prior stimuli or responses. A powerful sequence depended in part on memory for the just-
tool for making this comparison is memory interfer- prior response.
ence, that is, the degrading of control by prior events. Effects of the same blackouts were assessed in a
Availability of memory-interfering procedures leads second group of rats that obtained reinforcers for
to the following reasoning: When approximation varying four-response sequences under lag contin-
to a stochastic output is reinforced, if a memory- gencies. Neuringer (1991) reasoned that if variable
interfering event degrades performance, it provides responses were generated by a memory-based pro-
evidence against a stochastic generation process. cess, then performances would be degraded as black-
Absence of memory-interfering effects provides evi- out durations increased, as was the case for the
dence consistent with stochastic generation. We LLRR group. In fact, performances by the variability
have already seen evidence for stochastic-like group actually improved with increasing blackout
responding when demands on memory were high durations, resulting in higher rates of reinforcement.
(Machado, 1993; Manabe et al., 1997) and turn next Some have suggested that absence of memory for
to additional tests of the stochastic hypothesis. prior events is necessary for random responding
Neuringer (1991) compared the effects of mem- (Weiss, 1965), implying that memory interferes with
ory interference on responding by two groups of random generation. In any event, the results were
rats. One group obtained reinforcers by repeating a clearly inconsistent with the memory hypothesis.
single pattern, LLRR. Once that pattern was well In a related study, alcohol was administered to a
learned, blackouts were introduced between each single group of rats that had learned to respond vari-
response, the durations ranging from 0.1 second to ably when one stimulus was present and repeat
20 seconds across different phases of the experi- LLRR sequences given a second stimulus (Cohen et
ment. Responses were ineffective during the black- al., 1990). The two stimuli alternated throughout
out periods. As blackout durations increased, errors each session under a multiple schedule. As alcohol
increased and reinforcement rates fell. Neuringer doses increased, performance of the LLRR sequence
hypothesized that the interposed blackouts degraded was seriously impaired, whereas varying under the
performance because each response in the LLRR lag contingency was unaffected (Figure 22.7; see
Figure 22.7. Percentages of reinforced, or correct, sequences as a function of etha-

nol dosage for each of five rats. Varying sequences were reinforced under one stimu-
lus condition (left graph), and repetitive LLRR sequences were reinforced under
another stimulus (right graph). The lines connect averages of the five subjects (BG,
BR, RG, B, and P). From “Effects of Ethanol on Reinforced Variations and Repetitions
by Rats Under a Multiple Schedule,” by L. Cohen, A. Neuringer, and D. Rhodes,
1990, Journal of the Experimental Analysis of Behavior, 54, p. 5. Copyright 1990 by
Society for the Experimental Analysis of Behavior, Inc. Adapted with permission.
530
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
also Doughty & Lattal, 2001). Thus, within a single and the underlying processes. Results from these
session, the drunk rats failed to repeat accurately laboratory experiments may help to explain unpre-
but were highly proficient when required to vary. dictable operant behaviors in many nonlaboratory
Both interposed time delays and alcohol, two ways cases in which variability contingencies occur
to affect memory for prior responses; degraded per- naturally. In this section, we continue to describe
formances of fixed-pattern sequences; and either laboratory-based studies but ones with direct rele-
improved operant variability or left it unaffected. vance to real-world conditions.
Additional evidence for the difference between
memory-based and stochastic responding was pro- Training New Responses
vided by Neuringer and Voss (2002; see also Neu- Skinner (1981) hypothesized a parallel between evo-
ringer, 2002). College students learned to generate lutionary processes and selection by reinforcers of
chaotic-like sequences (according to the logistic differ- operant responses from a substrate of varying behav-
ence chaos function described in Equation 2) as well iors (see also Baum, 1994; Hull, Langman, & Glenn,
as to generate stochastic-like sequences (given feed- 2001; Staddon & Simmelhag, 1971). As described in
back from eight statistical tests, as in Neuringer, the Reinforcement of Variability section earlier in
1986). These two ways of responding variably were this chapter, variable behaviors can be generated by
alternated, under stimulus control, throughout each reinforcers whose delivery is directly contingent on
session. Memory interference was later introduced in a that variability, something not anticipated by Skin-
rather complex way. In the chaos phase of the experi- ner. One question that has potential importance
ment, subjects were required to generate four separate for applications is whether reinforced variability
chaotic functions, each differing from the other. In the facilitates acquisition of new responses, especially
stochastic phase, four uncorrelated response sequences difficult-to-learn ones.
were required. In essence, chaotic responses were used Neuringer, Deiss, and Olson (2000) reinforced
to interfere with one another, and stochastic responses variable five-response sequences across L and R
were used similarly. Results showed that performances levers while concurrently reinforcing a target
were significantly degraded during the four-segment sequence that rats find difficult to learn, namely
chaotic phases: Chaotic responses interfered with one RLLRL. Reinforcement for varying was limited to
another. A different result was obtained from the sto- once per minute, whereas the target sequence
chastic portion of the experiment. For one subject, RLLRL was reinforced whenever it occurred. Thus,
seven of eight statistics were closer to a random model if the rats learned to emit the target, reinforcement
during the interference phase than at the end of the was much more frequent than if they only varied.
original training with a single sequence; for the second The question was whether concurrent reinforcement
subject, all eight statistics were closer. These results of variations would facilitate acquisition of the target
are consistent with the hypothesis that a memory- sequence, and the answer was obtained through a
based process controls chaotic variability and that a comparison with two other groups of rats. In one
stochastic process, not dependent on memory, con- group, the same RLLRL target sequence was rein-
trols stochastic variability. (For additional evidence, forced whenever it occurred, but varying was never
see G. Jensen et al., 2006; Page & Neuringer, 1985.) reinforced. These target-only animals stopped
The importance of this experiment is that it demon- responding altogether—responses extinguished
strated the two modes of variability generation, one because reinforcer frequencies were low or zero—
memory based, the other stochastic, under a procedure and they did not learn the target (shown by the con-
that controlled for extraneous variables. trol [CON] data in Figure 22.8, top panel). In a
second control group, the RLLRL target was also
reinforced whenever it occurred. In addition, five-
Applications
response sequences were reinforced at a rate yoked
We have described experiments on the reinforce- to that obtained by the experimental animals; note
ment of predictable and unpredictable responding that these reinforcers could follow any sequence and
531
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
panel of Figure 22.8, these rats too did not learn the
target. Only the experimental animals who concur-
rently received reinforcers for variable responding
(VAR) learned to emit the RLLRL sequence at high
rates. Thus, it appeared that concurrent reinforce-
ment of variations facilitated acquisition of a
difficult-to-learn sequence, a potentially important
finding. The experiment was replicated with a sec-
ond difficult sequence with the same results (bottom
panel of Figure 22.8) and in a separate study with
rats as well (Neuringer, 1993).
However, attempts in two laboratories to repli-
cate these effects with human participants failed
(Bizo & Doolan, 2008; Maes & van der Goot, 2006).
In both cases, the target-only group (with no addi-
tional reinforcers presented) learned most rapidly.
Several possible explanations have been suggested,
including differences in relative frequencies of rein-
forcements for target responses versus variations,
differences in levels of motivation in the animal ver-
sus human studies, and the “figure out what’s going
on” type of instructions provided to the human par-
ticipants, but why or when concurrent reinforce-
ment of variations facilitates versus interferes with
Figure 22.8. Rates of emission of a learning of new responses is not yet clear (see
difficult-to-learn target sequence (RLLRL Neuringer, 2009).
on top and LLRRL on bottom) for three
groups of rats as a function of blocks of
sessions (each session block shows the Problem Solving
average of five sessions). In all groups, Arnesen (2000; see also Neuringer, 2004) studied
the target sequence was reinforced whether a history of explicit reinforcement of varia-
whenever it occurred. For one group, tions would facilitate later problem solving. Using a
reinforcement was additionally arranged
for varying sequences (VAR); for a sec- rat model, she provided food pellets to rats in an
ond group, the additional reinforcers experimental group for varying their responses to
occurred at the same rate as in VAR but arbitrarily selected objects. For example, a soup can
independent of variability (ANY); a third
was placed in the chamber, and responding to it in a
group did not receive additional rein-
forcement for any sequence other than variety of ways was reinforced. Each session pro-
the target sequences (CON). Adapted vided a different object, with response variability
from “Reinforced Variability and Operant being reinforced throughout. Members of a yoked
Learning,” by A. Neuringer, C. Deiss, and
G. Olson, 2000, Journal of Experimental control group experienced the same objects but
Psychology: Animal Behavior Processes, received food pellets independent of their interac-
26, p. 107. Copyright 2000 by the tions. A second control group was simply handled
American Psychological Association. for a period each day. After training, each rat was
placed alone in a problem space, a room approxi-
did not depend on variations. These animals contin- mately 6 feet by 8 feet, on the floor of which were
ued to respond throughout (the yoke reinforcers 30 objects—for example, a toy truck, metal plumb-
maintained high response strength) but as shown by ing pipes, a hair brush, a doll’s chest of drawers—
the independent-of-variability (ANY) data in the top arbitrarily chosen but different from those used
532
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
during the training phase. Hidden in each object Psychopathology

was a small piece of food, and the hungry rats were Behavioral and psychological disabilities are some-
permitted to explore freely for 20 minutes. The times associated with reduced control of variability.
question was how many food pellets would be dis- In autism and depression, for example, behaviors
covered and consumed. The experimental animals tend to be repetitive or stereotyped even when varia-
found significantly more pellets than either of the tions are desirable. In attention-deficit/hyperactivity
control groups, which did not differ from one disorder (ADHD), the opposite is true, with abnor-
another. Furthermore, the experimental rats explored mally high variability observed when focused and
more—they seemed bolder—and interacted more repetitive responses are adaptive. All three of these
with the objects than did the control rats, many of disorders share a common characteristic, however:
whom showed signs of fear. Thus, prior reinforce- an apparent inability to move from one end or the
ment of response variations transferred to a novel other of the variability continuum. One question is
environment and facilitated exploration of novel whether reinforcement contingencies can modify
objects and discovery of reinforcers. The advantages abnormal levels of variability. The answer to this
incurred by variations are discussed in the human question may differ with respect to depression and
literature (e.g., brainstorming), but tests of direct autism, on the one hand, and ADHD, on the other.
reinforcement-of-variability procedures for problem
Depression. Hopkinson and Neuringer (2003)
solving more generally have been few.
asked whether the low behavioral variability asso-
ciated with depression (Channon & Baker, 1996;
Creativity
Horne, Evans, & Orne, 1982; Lapp, Marinier, &
Although creative production requires more than
Pihl, 1982) could be increased by direct reinforce-
variation, Donald Campbell (1960) argued that vari-
ment. College students were separated into mildly
ations, and indeed random variations, are necessary.
depressed and not depressed on the basis of Center
If so, then operant variability may make important
for Epidemiological Studies Depression Scale scores
contributions to creativity. Support comes from
(Radloff, 1991). Each participant played a computer
studies in which creativity was directly reinforced
game in which sequences of responses were first
(e.g., Eisenberger & Armeli, 1997; Holman et al.,
reinforced independently of variability or proba-
1977; Pryor et al., 1969; see also Stokes, 2001).
bilistically (PROB), as in the yoke procedures we
Other studies, however, have indicated that rein-
have described, after which variable sequences were
forcement interferes with, or degrades, creative out-
directly reinforced (VAR). Figure 22.9 shows that
put (e.g., Amabile, 1983). This literature is deeply
under PROB, the depressed students’ variability
controversial and has been reviewed in several arti-
(U values) was significantly lower than that of the
cles (e.g., Cameron & Pierce, 1994; Deci, Koestner,
nondepressed students. When variability was explic-
& Ryan, 1999; Lepper & Henderlong, 2000), but the
itly reinforced, however, levels of variability increased
research listed earlier may contribute to a resolution.
in both groups and to the same high levels. This
As shown by Cherot et al. (1996) and others (Wag-
result, if general, is important because it indicates
ner & Neuringer, 2006), reinforcement of variations
that variability can be explicitly reinforced in people
has two effects. As a reinforcer is approached, vari-
manifesting mild depression (see also Beck, 1976).
ability declines. Thus, situations that potentiate the
anticipation of consequences on the basis of comple- Autism. In an experiment conducted by Miller
tion may interfere with creative activities. The con- and Neuringer (2000), five individuals diagnosed
tingencies may at the same time, however, maintain with autism and nine control subjects received rein-
high overall levels of creativity. Consideration of forcers independent of variability in a baseline phase
both induced effects (anticipation of reinforcement) (PROB), followed by a phase in which sequence
and contingency effects (reinforced variability and variations were directly reinforced. Subjects with
creativity) may help explain reinforcement’s contri- autism behaved less variably than the control sub-
bution to creativity (see Neuringer, 2003). jects in both phases; however, variability increased
533
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Rubia, Smith, Brammer, & Taylor, 2007). A sec-

ond common identifier is lack of inhibitory control
(Nigg, 2001). Can such behavior be influenced by
direct reinforcement? The evidence has indicated
that unlike the case for autism, variability may result
mainly from noncontingent (i.e., inducing) influ-
ences. One example is provided by the beneficial
effects of drugs such as methylphenidate (Ritalin).
Another is the fact that variability in individuals
with ADHD is higher than in control subjects when
reinforcement is infrequent, but not when it is fre-
quent (Aase & Sagvolden, 2006). Methylphenidate
Figure 22.9. Levels of variability
(indicated by U values) for depressed reduces variability. Low reinforcement frequencies
and nondepressed college students induce high variability, and the effects on those with
when reinforcers were provided inde- ADHD may be independent of direct reinforcement-
pendent of response variability (PROB of-variability contingencies. Similarly, when rein-
phase) versus when variations were
required (VAR phase). Standard errors forcement is delayed, the responses of subjects with
are shown by the error bars. From ADHD weaken more than those of control subjects,
“Modifying Behavioral Variability in possibly because of induced increases in variability
Moderately Depressed Students,” by
(Wagner & Neuringer, 2006). Thus, variability may
J. Hopkinson and A. Neuringer, 2003,
Behavior Modification, 27, p. 260. be induced in individuals diagnosed with ADHD
Copyright 2003 by Sage Publications, by different attributes of reinforcement, but to date
Inc. Adapted with permission. little evidence has indicated sensitivity to variability-
reinforcing contingencies.
significantly in both groups when it was reinforced.
Thus, individuals with autism, although relatively
Operant Variability and the
repetitive in their responding, acquired high levels
Emitted Operant
of operant varying. Ronald Lee and coworkers (Lee,
McComas, & Jawor, 2002; Lee & Sturmey, 2006) Reinforced variability may help to explain some
extended this work. Under a lag schedule, indi- unique attributes of operant behavior. Operants are
viduals with autism received reinforcers for vary- often compared with Pavlovian reflexes, and the two
ing verbal responses to questions, and two of three can readily be distinguished at the level of the proce-
participants in each of two experiments learned to dures used to establish them. In Pavlovian condi-
respond appropriately and nonrepetitively. Thus, tioning, a conditional relationship exists between a
the experimental evidence, although not extensive, previously neutral stimulus, such as a bell, and an
has indicated that the behavior of individuals with unconditioned stimulus, such as food. The result is
autism can benefit from reinforcers contingent on that the neutral stimulus becomes a conditioned
variability. Stated differently, the abnormally low stimulus that elicits a conditioned response. One
levels of variability characteristic of individuals with view is that operant responses differ in that they
autism may at least in part be under the influence of depend on a conditional relationship between
operant contingencies. response and reinforcer. Thus in one case, a condi-
tional relationship exists between two stimuli (if
Attention-deficit/hyperactivity disorder. Things conditioned stimulus, then unconditioned stimu-
may differ for individuals diagnosed with ADHD. lus), whereas in the other, the relationship is
Here, the abnormal levels of variability are at the between response and reinforcer.
opposite end of the continuum, with high variability However, according to Thorndike, Guthrie, and
a defining characteristic (Castellanos et al., 2005; others, when a response is made in the presence of a
534
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
particular stimulus and the response is reinforced, of these responses is most likely to be reinforced.
then over trials, the stimulus takes on the power of For example, if Jackie’s mother is nearby, the “Mom-
an elicitor (Bower & Hilgard, 1981). This finding mie, get my toy” response might be most likely.
led some researchers to conclude that both operant Alternatively, if the toy is just beyond reach, the
and Pavlovian responses were elicited by prior stim- child might be most likely to jump to get it. In many
uli. That is, in both cases stimulus–response rela- cases, however, the behavior appears to be selected
tionships were critical to predicting and explaining with equal probabilities, and prediction of the
the observed behaviors. instance becomes difficult.
Skinner (1935/1959) offered a radically different As just suggested, members of a particular class
view of the operant. Skinner’s position is difficult to of behaviors may be divided into subclasses, and
grasp, partly because at times he assumed the point even here variability may characterize aspects of the
of view of an environmental determinist, whereas at response. For example, if “ask for the toy” is the
other times he proposed probabilistic (and possibly activated subclass, the exact moment of a verbal
indeterministic) outcomes. According to Skinner, request, the particular words used, or the rhythm or
eliciting stimuli could not be identified for the oper- loudness may all be difficult to predict. Similarly,
ant. Although discriminative stimuli signaled the when a rat is pressing a lever to gain food pellets,
opportunity for reinforcement, no discrete environ- the characteristics of the press (one paw vs. both,
mental event could be identified to predict the exact with short or long latency, with high or low force,
time, topography, or occurrence of the response. etc.) are sometimes predictable, but often are not.
Skinner described operants as emitted to distinguish Thus, according to a Skinnerian model, functionally
them from elicited Pavlovian reflexes. equivalent instances emerge unpredictably from
But how is one to understand emission? The within a class or subclass, as though generated by a
term is ambiguous, derived from the Latin emittere, stochastic process (Skinner, 1938; see also Moxley,
meaning “to send out.” To be sent out might imply 1997). To state this differently, there is variance
being caused to leave, but there is a sense of emer- within the operant, manifested as the emission of
gence, rather than one-to-one causation, as in the instances from a set made up of functionally related
emission of radioactive particles. More important, but often physically dissimilar behaviors.
the term captures, for Skinner and others, the mani- Behavioral variability occurs for many reasons, as
fest variability of all operant behaviors. Skinner we have discussed. It decreases with training and
interpreted that variability as follows. experience. It is low when reinforcers are frequent
An individual operant response is a member of a and higher under intermittent schedules of rein-
class C of instances, a generic class, made up of forcement. It decreases with expectancy of and
functionally similar (although not necessarily physi- proximity to reinforcement. However, consequence-
cally similar) actions (Skinner, 1935/1959). An controlled variability may play a special role in
example may help to explain this point. Jackie, a explaining the emitted nature of the operant. To see
young child, desires a toy from a shelf that is too why, we next turn to experiments on volition. The
high for her to reach. Jackie might ask her mom to operant is often referred to as the voluntary operant,
get the toy, jump to try to reach it, push a chair next in contrast to the Pavlovian reflex. The question is
to the shelf to climb up to the toy, take a broom what about the operant indicates (and helps to
from the closet and try to pull the toy from the shelf, explain) volition.
or cry. Each of these acts, although differing in
physical details, is a member of the same operant
Operant Variability and
class because each potentially serves the same func-
Voluntary Behavior
tional relationship between the discriminative stim-
ulus (out-of-reach toy) and the goal (toy in hand). Attempts to explain volition have been ongoing for
Some responses may be more functional than other more than 2,000 years, and heated debates continue
members of the class, and cues may indicate which to this day in philosophy (Kane, 2002), psychology
535
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
(Maasen, Prinz, & Roth, 2003; Sebanz & Prinz, then together with knowledge of the individual’s
2006; Wegner, 2002), and physiology (Glimcher, past experiences and current circumstances, at least
2005; Libet, Freeman, & Sutherland, 1999). These somewhat accurate predictions can be made about
debates often concern the reality of volitional behav- the individual’s future goal-directed actions. Thus,
ior or lack thereof and, if real, how to characterize because functionality is thought to require an
it. Research on operant variability has suggested orderly relationship to environmental variables, pre-
that the descriptive term voluntary can be usefully dictions must be (at least theoretically) possible.
applied; that is, voluntary behaviors can be distin- Again, though, voluntary acts are often character-
guished from accidental reactions, such as stumbles; ized by their unpredictability, with this serving as a
from elicited responses, such as reflexes, both sign of autonomous control.
unconditioned and Pavlovian; from induced ones, An added complication is that unpredictability
such as those caused by drinking alcohol or antici- alone does not characterize voluntary actions.
pating a reinforcer; and many other cases. The Researchers do not attribute volition to random
research has also indicated important ways in which events, such as the throw of dice or emission of
voluntary actions differ from these others. atomic particles (Dennett, 2003; Popper & Eccles,
In large part, the difficulty surrounding attempts 1977), and truly random responding would often be
to explain voluntary behavior comes from an appar- maladaptive. Yet another problem is that voluntary
ent incompatibility between two often-noted charac- behaviors are not always unpredictable—they are
teristics. On the one hand, voluntary acts are said to quite predictable some of the time and, indeed, exist
be intentional, purposeful, goal directed, rational, or across the range of predictability. For example,
adaptive. These characteristics indicate the function- when the traffic light turns red, a driver is likely to
ality of voluntary behaviors, and we use that term as step on the brake. When you are asked for your
a summarizing descriptor. On the other hand, vol- name, you generally answer veridically, and so on.
untary actions are described as internally motivated But even in cases of predictable behaviors, if volun-
and autonomously controlled. Unpredictability, tary, these responses can be—and sometimes are—
demonstrated or potential, is offered as empirical emitted in more or less unpredictable fashion. The
evidence for such hypothesized autonomous con- red light can cause speeding up, slowing down, or
trol. Thus, unpredictability is thought to separate cursing. The name offered might be made up so as
voluntary acts from other functional behaviors (e.g., to fool the questioner, for example, during a game.
reflexes) and to separate their explanation from In brief, voluntary responses have the potential to
Newtonian causes and effects. Proposed explana- move along a variability continuum from highly pre-
tions of the unpredictability run the gamut from a dictable to unpredictable. A characteristic of all vol-
soul or a mind that can function apart from physical untary behaviors is real or potential variations in
causes to quantum-mechanical random events, but levels of variability.
they are all ultimately motivated by the presumed Operant variability helps to explain volition by
inability of a knowledgeable (perhaps even combining functionality with variations in levels of
supremely knowledgeable) observer to anticipate variability. Operant responses are goal directed and
the particulars of a voluntary act. functional, and the same holds for voluntary behav-
How can unpredictability (perhaps even unpre- iors. (In some cases, researchers say that the volun-
dictability in principle) be combined with function- tary response—and the operant—is intended to be
ality? That is the critical question facing those of us functional because it is governed by previous experi-
who would argue that voluntary is a useful classifica- ences and because in a variable or uncertain environ-
tion. The problem derives from the (erroneous) ment, what was once functional may no longer be so.)
assumption that functionality necessarily implies Operant responses are more or less variable, depend-
potential predictability. That assumption goes some- ing on discriminative stimuli and reinforcement con-
thing like this: If an observer knows what an indi- tingencies, and the same is true for voluntary
vidual is striving for, or attempting to accomplish, behaviors. Thus, for both operant and voluntary
536
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
behaviors, the ability of a knowledgeable observer to reinforcement is unpredictable, and the two alterna-
predict future occurrences will depend on the circum- tives are independent of one another, every response
stances. Voluntary behavior is behavior that is func- has the possibility of producing a reinforcer. How-
tional (or intended to be so) and sometimes highly ever, in general, the left alternative is three times
predictable, other times unpredictable, with predict- more likely to have reinforcement waiting than the
ability governed by the same functionality require- right alternative.
ment as other attributes of operant behavior. We have The VI values (or average times between rein-
just summarized a theory of volition referred to as the forcer setups) generally differ across phases of an
operant variability and voluntary action (OVVA) theory experiment. For example, a 1:3 ratio of setup time
(Neuringer & Jensen, 2010). In the following sec- left to right in one phase might be followed by a 3:1
tions, we provide experimental evidence consistent ratio in another, and a third might use a 2:2 ratio.
with OVVA theory. We begin with a discussion of When the ratios across these alternatives are system-
choices under conditions of uncertainty, partly atically manipulated, an often observed finding is
because choices are generally thought to be voluntary that the overall ratios of left-to-right choices are
and partly because concurrent schedules of reinforce- functionally related to ratios of left-to-right obtained
ment, a method used to study choice, provided the reinforcers, a relationship commonly described as a
means to test OVVA theory. power function and referred to as the generalized
matching law (Baum, 1974):
Choice Under Uncertainty s
In some choice situations, one (and only one) of CX  kX   R X 

=   •  . (3)
many options provides reinforcement (e.g., the third CY  k y   R y 

key from the left in a row of eight keys), and both
people and other animals learn to choose correctly In Equation 3, CX refers to observed choices of alter-
and to do so repeatedly. In other cases, a particular native X, and RX corresponds to delivered reinforcers
pattern of choices is required (e.g., LLRR in a two- (CY and RY correspond to alternative Y, accordingly).
lever chamber), and that pattern is learned and The parameter kX refers to bias for X, with biases—
repeated. Individual choices in these situations are because of side preferences, differences in the oper-
readily predicted. In many situations, though, fixed anda, or any number of variables—not thought to be
choices and patterns are not reinforced, and rein- influenced by the reinforcer ratios. The s parameter
forcer availability is uncertain, both in time and refers to the sensitivity of choice ratios to reinforce-
place. As we discuss, these conditions often result in ment ratios. When s = 1.0, choice ratios exactly
stochastic responding. match (or equal) reinforcement ratios. With s
Choices under conditions of reinforcement parameter values less than 1.0, choice ratios are not
uncertainty have commonly been studied in behav- as extreme as the ratio of reinforcers, with the oppo-
ioral laboratories with concurrent schedules of site for s more than 1.0 (see the Psychophysical Test
reinforcement. Reinforcers are independently pro- section later in this chapter). To the extent that the
grammed for two (or sometimes more) options, and generalized matching law provides an accurate
subjects choose freely among them. Consider the description (and there is much support for it), it
example of concurrent VI schedules. In a VI 1 minute– permits predictions of the molar distribution of
VI 3 minute procedure, each schedule is applied to choice allocation; that is, overall ratios of choices
one of two response alternatives, left and right. can accurately be described as a function of obtained
Under this procedure, a reinforcer becomes available reinforcer ratios (Davison & McCarthy, 1988).
(or “sets up”) on average once per minute for Another observation from studies of concurrent
responses on the left and independently on average VI schedules, however, is that individual choices are
every 3 minutes for choices of the option on the difficult to predict. Even when they conform to
right. Once a reinforcer has set up, it is delivered on Equation 3, they often appear to be emitted stochas-
the next response to that alternative. Because time to tically (Glimcher, 2003, 2005; G. Jensen & Neuringer,
537
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
2008; Nevin, 1969; see also Silberberg, Hamilton, functionally varying behaviors yielded a perception
Ziriax, & Casey, 1978, for an alternative view). In of voluntary action.
the VI 1 minute–VI 3 minute example given earlier,
an observer might accurately predict that the left Psychophysical Test
option will be chosen three times more frequently OVVA theory predicts that responses will appear to
than the right but be unable to accurately predict be voluntary when levels of (un)predictability vary
any given choice. A recent example of such stochas- functionally (purposefully, adaptively). Choices
ticity was observed when pigeons’ choices were allo- under concurrent schedules of reinforcement pro-
cated across three concurrently available sources vided a way to test this claim. Neuringer, Jensen,
of reinforcement (G. Jensen & Neuringer, 2008). and Piff (2007) had human participants observe six
Figure 22.10 shows that run lengths—defined as the different virtual actors (hereinafter called agents) as
average number of choices on one key before each agent made thousands of choices. The agents
switching to a different key—approximated those differed in how they went about choosing among
expected from a stochastic process.5 Thus, at the the available options (the strategies are described in
same time that overall choice proportions can read- a subsequent paragraph). Each agent’s choices were
ily be predicted, individual choices cannot. This shown separately on an individual computer, with
combination of functionally related choice propor- six computers located close to one another on small
tions and stochastic emission provided the means desks in a laboratory. The participants were free to
to assess the relationship between operant variabil- walk among the computers to compare the agents’
ity and volition. In particular, we asked whether choice strategies.
To minimize extraneous cues, such as whether
the agent resembled a human figure, choices were
represented in a simple manner, namely as dots
moving around the screens. Participants were
instructed that the agents were choosing among
three alternative gambles, similar to slot machine
gambles, with each gamble’s choice represented by
the dot’s movement in one of three directions.
Whenever a choice led to reinforcement—the agent
won that gamble—the dot’s color changed as a sign
of success. Thus, participants could observe how
choices were made in relationship to the reinforcers
received. Participants were asked to judge how well
the choices made by the agents represented volun-
tary choices made by a real human player.
Unknown to the participants, the agents’ choices
were controlled by iterating the generalized match-
Figure 22.10. Mean run lengths by pigeons on each ing power function (Equation 3) that was extended
of three response keys as a function of the propor-
to a three-alternative situation (G. Jensen & Neu-
tion of responses to that key. The drawn line is the
expected function if responses were emitted stochasti- ringer, 2008). Thus, the agents chose probabilisti-
cally. Adapted from “Choice as a Function of Reinforcer cally among the three options on the basis of the
‘Hold’: From Probability Learning to Concurrent proportions of reinforcers that they had received
Reinforcement,” by G. Jensen and A. Neuringer, 2008,
Journal of Experimental Psychology: Animal Behavior from the three alternatives. These calculations were
Processes, 34, p. 44. Copyright 2008 by the American done in real time, with current choice probabilities
Psychological Association. depending on previously obtained reinforcers.
See Jensen and Neuringer (2008) for discussion of these findings, including the small divergence of data from the theoretical curve.
5
538
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
Choices by the six agents differed only with respect among the three options with more equal probabili-
to the s exponent of the power functions governing ties than indicated by the reinforcement ratios
the choices: Some agents had high values for their throughout the six games. In the preceding example,
sensitivity parameters, and others had low values. this agent would choose X with probability of .399
Participants were told only that the dot movements (rather than .5 for the exact matcher), choose Y with
represented choices of gambles and that their objec- probability of .325 (rather than .3), and choose Z
tive was to rate how closely those movements with probability of .276 (rather than .2). In general,
resembled the voluntary choices of real human algorithms with s values less than 1.0 are referred to
players. Next, we describe how reinforcer availabil- as undermatchers: They distribute choices more
ity was programmed and the effects of s values on equally—and therefore more unpredictably—across
the generated choices. the available options than the exact matcher. The
Reinforcers set up probabilistically (and remained opposite was the case for agents with s values more
available until collected, as in concurrent VI sched- than 1.0, whose preferences were more extreme
ules) for each of the three gambles. There were six than indicated by the reinforcer ratios and were
different combinations of set-up rates, which partici- referred to as overmatchers. Over the course of sev-
pants were told constituted six different games. eral experiments, a wide range of s values was pre-
Thus, in some games, the agent’s X choices were sented, spanning a range from 0.0 (extreme
most frequently reinforced; in other games, Y choices undermatcher) to 6.0 (extreme overmatcher) in one
were most frequently reinforced; in others, the rein- experiment, a range from 0.1 to 2.0 in another, and
forcers were more equally distributed; and so on. a range from 0.1 to 1.9 in a third.
Participants were free to observe each agent playing Results were consistent and clear: The strict
each of the six games for as long as needed to make matcher (s = 1.0) was judged to best represent voli-
their evaluation. After observing the choices in all tional choices. Figure 22.11 shows data from two of
games, the participants judged the degree to which the experiments. In one experiment, participants
each agent’s responses appeared to be those of a were informed in advance that all of the agents’
human player who was voluntarily choosing among choices were generated by computer algorithms, and
the options. The key question was whether the they were asked to rate the algorithms in terms of
agents’ different choice strategies—caused by differ- volitional appearance. In the second, participants
ences in the s exponents—generated systematic dif- were told that some agents’ choices were based on
ferences in participants’ judgments of volition. computer algorithms, that others depicted voluntary
The s values, and their effects on the agents’ choices of real humans, and that their task was to
choice allocations, were as follows: For one agent, s identify the humans.6
equaled 1.0, and choice proportions therefore As s values approached 1.0, the agents were rated
strictly matched proportions of received reinforcers. as providing increasingly good representations of
Assume, for example, that this agent had gained a voluntary human choice, suggesting a continuum of
total of 100 reinforcers at some point in the game: more or less apparent volition. From the perspective
50 reinforcers for option X, 30 for Option Y, and 20 of the participants, the s = 1.0 strict matcher some-
for Option Z. The probability of the agent’s next X times responded unpredictably (when reinforcers
choice would therefore equal 0.5 (50/100); a Y were equally allocated across the three alternatives),
choice, 0.3 (30/100); and a Z choice, 0.2 (20/100). at other times highly predictably (when most rein-
The s = 1.0 actor therefore distributed its choices forcers were obtained from one alternative), and at
probabilistically in exact proportion to its received yet other times at intermediate levels. In each case,
reinforcers. however, the agent’s choices seemed to be governed
Another agent was assigned an s value of 0.4, the by the reinforcement distribution in a particular
consequence of which was that it tended to choose game environment, an indicator of functional
This task was inspired by the Turing test, considered by many to be the gold standard of artificial intelligence.
6
539
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Figure 22.11. Judgments of how closely agents’ responses approximated

voluntary human choices (on left y-axis) and probabilities (prob.) of identifying
agents as a voluntarily choosing human player (on right y-axis) as functions of
the agents’ s-value exponents. From “Stochastic Matching and the Voluntary
Nature of Choice,” by A. Neuringer, G. Jensen, and P. Piff, 2007, Journal of the
Experimental Analysis of Behavior, 88, pp. 7, 13. Copyright 2007 by Society for
the Experimental Analysis of Behavior, Inc. Adapted with permission.
changes in behavior. The undermatchers tended to Another control experiment tested whether
respond less predictably throughout, as we indicated matching alone implied volition (Neuringer et al.,
earlier, and the overmatchers more predictably. 2007). The question was whether the more or less
Thus, the undermatchers demonstrated that unpre- (un)predictable responding contributed at all to the
dictability alone was not sufficient for apparent voli- judgments. Stated differently, did matching or pre-
tion: It was necessary that agents display functional dictability or both generate the volitional judg-
variations in levels of (un)predictability to receive ments? Participants were therefore asked to
the highest volitional ratings. compare two agents, both of which exactly matched
A series of control experiments evaluated alterna- choice proportions to reinforcer proportions; how-
tive explanations. For example, rates of reinforce- ever, one agent matched by stochastically allocating
ment were overall slightly higher (across games) for its choices (as was done in all of the experiments
the s = 1.0 matcher than for any of the other agents, described to this point), whereas the other agent
and one control showed that differences in rein- allocated its choices in an easily predictable fashion.
forcement rate were not responsible for the voli- For example, if the stochastic matcher had received
tional judgments. In the experiment, agents who reinforcers in a ratio of 5:3:2, it responded to the left
cheated (i.e., those who appeared to know where to alternative with a .5 probability, to the center with a
respond for reinforcers) were compared with the .3 probability, and to the right with a .2 probability.
strict—probabilistically choosing—matcher, and the Because they were emitted stochastically, individual
matcher was evaluated as substantially more voli- choices could not be predicted above chance levels.
tional in appearance, despite obtaining fewer rein- By contrast, the patterned matcher also matched
forcers than the cheaters. An observer might exactly but did so in a patterned and therefore read-
appreciate the individual who gains more reinforce- ily predictable way. In the example just given, it
ment than another, but that fact alone will not con- would respond LLLLLCCCRR, again and again
vince the observer that the individual is choosing in cycling through the same 5:3:2 strings of respond-
a voluntary manner. ing until there was a change in obtained reinforcer
540
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
proportions, at which point it would adjust the instance may be difficult or impossible to predict,
length of its strings accordingly. Because both especially for large response classes.
agents matched, both received identical rates of Unpredictability, real or potential, is emphasized
reinforcement. The participants judged the stochas- in many discussions of volition. Indeed, the size of
tic matcher to significantly better represent a volun- the active set can be exceedingly large—and func-
tary human player than the patterned one, showing tionally so—because if someone was attempting to
that both functionality (matching, in this case) and prove that he or she is a free agent, the set of possibil-
stochasticity were jointly necessary for the highest ities might consist of all responses in that person’s
ratings of volition. repertoire (see Scriven, 1965). We return to the fact,
The combination of choice distributions (match- though, that voluntary behaviors can be predictable
ing) and choice variability (more or less predictabil- as well as not predictable. The most important char-
ity) provided evidence for voluntary behavior. acteristic is functionality of variability, or ability to
Choice distributions alone did not lead responses to change levels of predictability in response to environ-
be evaluated as highly voluntary, nor did choice mental demands. This is equally an identifying char-
unpredictability alone. Choices were most voluntary acteristic of operant behavior in which responses are
in appearance when probabilities and distributions functional and stochastically emitted. Thus, with
of stochastic responses changed with distributions Skinner, we combine voluntary and operant in a sin-
of reinforcers. According to OVVA theory, function- gle phrase, but research has now shown why that is
ally changing variable behaviors are voluntary appropriate. Operant responses are voluntary because
behaviors. Stated differently, voluntary behaviors they combine functionality with (un)predictability.
are members of a class characterized by ability to
vary levels of response (un)predictability in a func-
Conclusion
tional manner. The psychophysical evidence just
reviewed is consistent with OVVA theory. Aristotle anticipated what many have referred to as
To review, the facts of operant variability show the most influential law in psychology (Murray,
that levels, or degrees, of behavioral (un)predictabil- 1988). When two events co-occur, presentation of
ity are guided by environmental consequences. A one will cause recollection or generation of the
theory of volition, OVVA, proposes that exactly the other. Although he and many others were wrong in
same is true for voluntary actions. Voluntary behav- the details, laws of association have been the foun-
iors are sometimes readily predictable, sometimes dation of theories of mind and behavior throughout
less predictable, and sometimes quite unpredictable. the history of Western thought, and the science of
In all cases, the reasons for the predictability can be psychology and behavior has been well served by
identified (given sufficient knowledge), but the pre- the search for them. From the British Association-
cise behaviors may still remain unpredictable. For ists, to Pavlov, to Hebb and Rescorla, theoreticians
example, under some circumstances, the response to and researchers have documented laws of the form
“How are you?” can readily be predicted for a given “if A, then B” that help to explain thoughts and
acquaintance. Even when the situation warrants behaviors. Evolutionary theory offered a distinctly
unpredictable responses, as when responders wish different type of behavioral law, involving selection
to conceal their feelings, some veridical predictions from variations, laws that were developed by Skinner
can be made: that the response will be verbal, that it (1981) and others (Hull et al., 2001). In this chap-
will contain particular parts of speech, and so on. ter, we provided evidence of how selection interacts
The functionality of variability implies a degree of with variation: Parameters of variation are selected
predictability in the resulting behaviors that is (via reinforcement of variability), and selections
related to the activated class of possibilities from emerge from variations (via stochastic emission).
which the response emerges. The class can often be This interaction, of equal importance to that of asso-
predicted on the basis of knowledge of the organism ciation, must be deciphered if researchers are to
and environmental conditions. However, the explain, at long last, voluntary behavior.
541
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
References Motor Skills, 84, 627–661. doi:10.2466/pms.1997.

84.2.627
Aase, H., & Sagvolden, T. (2006). Infrequent, but not
frequent, reinforcers produce more variable respond- Cameron, J., & Pierce, W. D. (1994). Reinforcement,
ing and deficient sustained attention in young chil- reward, and intrinsic motivation: A meta-analysis.
dren with attention-deficit/hyperactivity disorder Review of Educational Research, 64, 363–423.
(ADHD). Journal of Child Psychology and Psychiatry, doi:10.3102/00346543064003363
47, 457–471. doi:10.1111/j.1469-7610.2005.01468.x Campbell, D. T. (1960). Blind variation and selective
Akins, C. K., Domjan, M., & Gutierrez, G. (1994). retention in creative thought as in other knowl-
Topography of sexually conditioned behavior in edge processes. Psychological Review, 67, 380–400.
male Japanese quail (Coturnix japonica) depends doi:10.1037/h0040373
on the CS–US interval. Journal of Experimental Castellanos, F. X., Sonuga-Barker, E. J. S., Schere, A., Di
Psychology: Animal Behavior Processes, 20, 199–209. Martino, A., Hyde, C., & Walters, J. R. (2005). Varieties
doi:10.1037/0097-7403.20.2.199 of attention-deficit/hyperactivity disorder-related intra-
Amabile, T. M. (1983). The social psychology of creativity. individual variability. Biological Psychiatry, 57, 1416–
New York, NY: Springer-Verlag. 1423. doi:10.1016/j.biopsych.2004.12.005
Antonitis, J. J. (1951). Response variability in the white Catchpole, C. K., & Slater, P. J. (1995). Bird song:
rat during conditioning, extinction, and recondition- Biological themes and variations. Cambridge, England:
ing. Journal of Experimental Psychology, 42, 273–281. Cambridge University Press.
doi:10.1037/h0060407 Chaitin, G. J. (1975). Randomness and mathematical
Arnesen, E. M. (2000). Reinforcement of object manipula- proof. Scientific American, 232, 47–52. doi:10.1038/
tion increases discovery. Unpublished undergraduate scientificamerican0575-47
thesis, Reed College, Portland, OR. Channon, S., & Baker, J. E. (1996). Depression and
Bandura, A. (1982). The psychology of chance encoun- problem-solving performance on a fault-diagnosis
ters and life paths. American Psychologist, 37, task. Applied Cognitive Psychology, 10, 327–336.
747–755. doi:10.1037/0003-066X.37.7.747 doi:10.1002/(SICI)1099-0720(199608)10:4<327::AID-
ACP384>3.0.CO;2-O
Baum, W. M. (1974). On two types of deviation from the
matching law: Bias and undermatching. Journal of Cherot, C., Jones, A., & Neuringer, A. (1996). Reinforced
the Experimental Analysis of Behavior, 22, 231–242. variability decreases with approach to reinforcers.
doi:10.1901/jeab.1974.22-231 Journal of Experimental Psychology: Animal Behavior
Processes, 22, 497–508. doi:10.1037/0097-7403.22.4.497
Baum, W. M. (1994). Understanding behaviorism. New
York, NY: HarperCollins. Clouzot, H.-G. (Producer & Director). (1956). Le mystère
Picasso [The mystery of Picasso; Motion picture].
Beck, A. T. (1976). Cognitive therapy and the emotional France: Filmsonor.
disorders. New York, NY: International Universities
Press. Cohen, L., Neuringer, A., & Rhodes, D. (1990). Effects
of ethanol on reinforced variations and repeti-
Bizo, L. A., & Doolan, K. (2008). Reinforced behavioural tions by rats under a multiple schedule. Journal
variability. Paper presented at the meeting of the of the Experimental Analysis of Behavior, 54, 1–12.
Association for Behavior Analysis, Chicago, IL. doi:10.1901/jeab.1990.54-1
Blough, D. S. (1966). The reinforcement of least frequent Craig, W. (1918). Appetites and aversions as constitu-
interresponse times. Journal of the Experimental ents of instincts. Biological Bulletin, 34, 91–107.
Analysis of Behavior, 9, 581–591. doi:10.1901/jeab. doi:10.2307/1536346
1966.9-581
Davison, M., & McCarthy, D. (1988). The matching law:
Bouton, M. (1994). Context, ambiguity, and classical con- A research review. Hillsdale, NJ: Erlbaum.
ditioning. Current Directions in Psychological Science,
Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A
3, 49–53. doi:10.1111/1467-8721.ep10769943
meta-analytic review of experiments examin-
Bower, G. H., & Hilgard, E. R. (1981). Theories of learning the effects of extrinsic rewards on intrinsic
ing (5th ed.). Englewood Cliffs, NJ: Prentice-Hall. motivation. Psychological Bulletin, 125, 627–668.
Brener, J., & Mitchell, S. (1989). Changes in energy doi:10.1037/0033-2909.125.6.627
expenditure and work during response acquisition Dennett, D. (2003). Freedom evolves. New York, NY:
in rats. Journal of Experimental Psychology: Animal Viking Adult.
Denney, J., & Neuringer, A. (1998). Behavioral variabil-
7403.15.2.166
ity is controlled by discriminative stimuli. Animal
Brugger, P. (1997). Variables that influence the genera- Learning and Behavior, 26, 154–162. doi:10.3758/
tion of random sequences: An update. Perceptual and BF03199208
542
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
Doughty, A. H., & Lattal, K. A. (2001). Resistance to students. Behavior Modification, 27, 251–264.
change of operant variation and repetition. Journal of doi:10.1177/0145445503251605
the Experimental Analysis of Behavior, 76, 195–215. Hopson, J., Burt, D., & Neuringer, A. (2002). Variability
doi:10.1901/jeab.2001.76-195 and repetition under a multiple schedule. Unpublished
Driver, P. M., & Humphries, D. A. (1988). Protean behav- manuscript.
ior: The biology of unpredictability. Oxford, England:
Horne, R. L., Evans, F. J., & Orne, M. T. (1982). Random
Oxford University Press.
number generation, psychopathology, and therapeu-
Eckerman, D. A., & Lanson, R. N. (1969). Variability of tic change. Archives of General Psychiatry, 39, 680–
response location for pigeons responding under con- 683. doi:10.1001/archpsyc.1982.04290060042008
tinuous reinforcement, intermittent reinforcement
Hoyert, M. S. (1992). Order and chaos in fixed-
and extinction. Journal of the Experimental Analysis of
interval schedules of reinforcement. Journal of the
Eisenberger, R., & Armeli, S. (1997). Can salient reward doi:10.1901/jeab.1992.57-339
increase creative performance without reducing
Hull, D. L., Langman, R. E., & Glenn, S. S. (2001). A gen-
intrinsic creative interest? Journal of Personality and
eral account of selection: Biology, immunology and
Social Psychology, 72, 652–663. doi:10.1037/0022-
behavior. Behavioral and Brain Sciences, 24, 511–528.
3514.72.3.652
doi:10.1017/S0146525X0156416X
Gharib, A., Gade, C., & Roberts, S. (2004). Control
Jensen, G., Miller, C., & Neuringer, A. (2006). Truly ran-
of variation by reward probability. Journal of
dom operant responding: Results and reasons. In
Experimental Psychology: Animal Behavior Processes,
E. A. Wasserman & T. R. Zentall (Eds.), Comparative
30, 271–282. doi:10.1037/0097-7403.30.4.271
cognition: Experimental explorations of animal intel-
Gigerenzer, G., Swijtink, Z., Porter, T., Daston, L., ligence (pp. 459–480). Oxford, England: Oxford
Beatty, J., & Kruger, L. (1989). The empire of chance: University Press.
How probability changed science and everyday life.
Cambridge, England: Cambridge University Press. Jensen, G., & Neuringer, A. (2008). Choice as a function
of reinforcer “hold”: From probability learning to
Glimcher, P. W. (2003). Decisions, uncertainty, and the concurrent reinforcement. Journal of Experimental
brain. Cambridge, MA: MIT Press. Psychology: Animal Behavior Processes, 34, 437–460.
Glimcher, P. W. (2005). Indeterminacy in brain and doi:10.1037/0097-7403.34.4.437
behavior. Annual Review of Psychology, 56, 25–56. Jensen, R. V. (1987). Classical chaos. American Scientist,
doi:10.1146/annurev.psych.55.090902.141429 75, 168–181.
Goetz, E. M., & Baer, D. M. (1973). Social control of form Kane, R. (Ed.). (2002). The Oxford handbook of free will.
diversity and emergence of new forms in children’s Oxford, England: Oxford University Press.
blockbuilding. Journal of Applied Behavior Analysis, 6,
209–217. doi:10.1901/jaba.1973.6-209 Lapp, J. E., Marinier, R., & Pihl, R. O. (1982). Correlates
of psychotropic drug use in women: Interpersonal
Grunow, A., & Neuringer, A. (2002). Learning to vary personal problem solving and depression. Women
and varying to learn. Psychonomic Bulletin and and Health, 7, 5–16. doi:10.1300/J013v07n02_02
Review, 9, 250–258. doi:10.3758/BF03196279
Lee, R., McComas, J. J., & Jawor, J. (2002). The effects of
Guthrie, E. R., & Horton, G. P. (1946). Cats in a puzzle differential reinforcement on varied verbal respond-
box. New York, NY: Rinehart. ing by individuals with autism to social questions.
Hachiga, Y., & Sakagami, T. (2010). A runs-test algo- Journal of Applied Behavior Analysis, 35, 391–402.
rithm: Contingent reinforcement and response run doi:10.1901/jaba.2002.35-391
structures. Journal of the Experimental Analysis of Lee, R., & Sturmey, P. (2006). The effects of lag sched-
Behavior, 93, 61–80. doi:10.1901/jeab.2010.93-61 ules and preferred materials on variable respond-
Herrnstein, R. J. (1961). Stereotypy and intermittent rein- ing in students with autism. Journal of Autism and
forcement. Science, 133, 2067–2069. doi:10.1126/ Developmental Disorders, 36, 421–428. doi:10.1007/
science.133.3470.2067-a s10803-006-0080-7
Holman, J., Goetz, E. M., & Baer, D. M. (1977). The Lee, R., Sturmey, P., & Fields, L. (2007). Schedule-induced
training of creativity as an operant and an examina- and operant mechanisms that influence response vari-
tion of its generalization characteristics. In B. Etzel, ability: A review and implications for future investiga-
J. LeBland, & D. Baer (Eds.), New development in tions. Psychological Record, 57, 429–455.
behavior research: Theory, method and application Lepper, M. R., & Henderlong, J. (2000). Turning “play”
(pp. 441–471). Hillsdale, NJ: Erlbaum. into “work” and “work” into “play”: 25 years of
Hopkinson, J., & Neuringer, A. (2003). Modifying research on intrinsic versus extrinsic motivation. In
behavioral variability in moderately depressed C. Sansone & J. M. Harackiewicz (Eds.), Intrinsic and
543
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
extrinsic motivation: The search for optimal motiva- Behavior Analysis, 33, 151–165. doi:10.1901/
tion and performance (pp. 257–307). San Diego, CA: jaba.2000.33-151
Academic Press. doi:10.1016/B978-012619070-0/ Mook, D. M., & Neuringer, A. (1994). Different effects
50032-5 of amphetamine on reinforced variations versus rep-
Libet, B., Freeman, A., & Sutherland, K. (Eds.). (1999). etitions in spontaneously hypertensive rats (SHR).
The volitional brain: Towards a neuroscience of free Physiology and Behavior, 56, 939–944. doi:10.1016/
will. Thorverton, England: Imprint Academic. 0031-9384(94)90327-1
Lopes, L. L. (1982). Doing the impossible: A note on Mosekilde, E., Larsen, E., & Sterman, J. (1991). Coping
induction and the experience of randomness. Journal with complexity: Deterministic chaos in human deci-
of Experimental Psychology: Learning, Memory, and sion making behavior. In J. L. Casti & A. Karlqvist
Cognition, 8, 626–636. doi:10.1037/0278-7393.8.6.626 (Eds.), Beyond belief: Randomness, prediction and
explanation in science (pp. 199–229). Boca Raton, FL:
Maasen, S., Prinz, W., & Roth, G. (Eds.). (2003).
CRC Press.
Voluntary action: Brains, minds, and sociality. New
York, NY: Oxford University Press. Moxley, R. A. (1997). Skinner: From determinism to ran-
dom variation. Behavior and Philosophy, 25, 3–28.
Machado, A. (1989). Operant conditioning of behavioral
variability using a percentile reinforcement schedule. Murray, D. J. (1988). A history of Western psychology (2nd
Journal of the Experimental Analysis of Behavior, 52, ed.). Englewood Cliffs, NJ: Prentice Hall.
155–166. doi:10.1901/jeab.1989.52-155 Neuringer, A. (1986). Can people behave “randomly?”:
Machado, A. (1992). Behavioral variability and frequency- The role of feedback. Journal of Experimental
dependent selection. Journal of the Experimental Psychology: General, 115, 62–75. doi:10.1037/0096-
Analysis of Behavior, 58, 241–263. doi:10.1901/ 3445.115.1.62
jeab.1992.58-241 Neuringer, A. (1991). Operant variability and repeti-
Machado, A. (1993). Learning variable and stereotypical tion as functions of interresponse time. Journal of
sequences of responses: Some data and a new model. Experimental Psychology: Animal Behavior Processes,
Behavioural Processes, 30, 103–129. doi:10.1016/ 17, 3–12. doi:10.1037/0097-7403.17.1.3
0376-6357(93)90002-9 Neuringer, A. (1992). Choosing to vary and repeat.
Machado, A. (1997). Increasing the variability of Psychological Science, 3, 246–250. doi:10.1111/
response sequences in pigeons by adjusting the j.1467-9280.1992.tb00037.x
frequency of switching between two keys. Journal Neuringer, A. (1993). Reinforced variation and selec-
of the Experimental Analysis of Behavior, 68, 1–25. tion. Animal Learning and Behavior, 21, 83–91.
doi:10.1901/jeab.1997.68-1 doi:10.3758/BF03213386
Maes, J. H. R., & van der Goot, M. (2006). Human oper- Neuringer, A. (2002). Operant variability: Evidence,
ant learning under concurrent reinforcement of functions, and theory. Psychonomic Bulletin and
response variability. Learning and Motivation, 37, Review, 9, 672–705. doi:10.3758/BF03196324
79–92. doi:10.1016/j.lmot.2005.03.003 Neuringer, A. (2003). Reinforced variability and creativ-
Manabe, K., Staddon, J. E. R., & Cleaveland, J. M. (1997). ity. In K. A. Lattal & P. N. Chase (Eds.), Behavior
Control of vocal repertoire by reward in budgerigars theory and philosophy (pp. 323–338). New York, NY:
(Melopsittacus undulatus). Journal of Comparative Kluwer Academic/Plenum.
Psychology, 111, 50–62. doi:10.1037/0735-7036. Neuringer, A. (2004). Reinforced variability in animals
111.1.50 and people. American Psychologist, 59, 891–906.
McElroy, E., & Neuringer, A. (1990). Effects of alcohol doi:10.1037/0003-066X.59.9.891
on reinforced repetitions and reinforced variations in Neuringer, A. (2009). Operant variability and the power
rats. Psychopharmacology, 102, 49–55. doi:10.1007/ of reinforcement. Behavior Analyst Today, 10, 319–
BF02245743 343. Retrieved from http://www.baojournal.com/
Mechner, F. (1958). Sequential dependencies of the BAT%20Journal/VOL-10/BAT%2010-2.pdf
lengths of consecutive response runs. Journal of Neuringer, A., Deiss, C., & Olson, G. (2000). Reinforced
the Experimental Analysis of Behavior, 1, 229–233. variability and operant learning. Journal of
Metzger, M. A. (1994). Have subjects been shown to gen- 26, 98–111. doi:10.1037/0097-7403.26.1.98
erate chaotic numbers? Commentary on Neuringer Neuringer, A., & Jensen, G. (2010). Operant variability
and Voss. Psychological Science, 5, 111–114. and voluntary action. Psychological Review, 117,
doi:10.1111/j.1467-9280.1994.tb00641.x 972–993. doi:10.1037/a0019499
Miller, N., & Neuringer, A. (2000). Reinforcing vari- Neuringer, A., Jensen, G., & Piff, P. (2007). Stochastic
ability in adolescents with autism. Journal of Applied matching and the voluntary nature of choice. Journal
544
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Operant Variability
of the Experimental Analysis of Behavior, 88, 1–28. Schwartz, B. (1988). The experimental synthesis of
doi:10.1901/jeab.2007.65-06 behavior: Reinforcement, behavioral stereotypy and
Neuringer, A., Kornell, N., & Olufs, M. (2001). Stability problem solving. In G. H. Bower (Ed.), The psychol-
and variability in extinction. Journal of Experimental ogy of learning and motivation (Vol. 22, pp. 93–138).
Psychology: Animal Behavior Processes, 27, 79–94. New York, NY: Academic Press.
doi:10.1037/0097-7403.27.1.79 Scriven, M. (1965). An essential unpredictability in
Neuringer, A., & Voss, C. (1993). Approximating cha- human behavior. In B. Wolman (Ed.), Scientific psy-
otic behavior. Psychological Science, 4, 113–119. chology (pp. 411–425). New York, NY: Basic Books.
doi:10.1111/j.1467-9280.1993.tb00471.x Sebanz, N., & Prinz, W. (Eds.). (2006). Disorders of voli-
Neuringer, A., & Voss, C. (2002). Approximations to tion. Cambridge, MA: MIT Press.
chaotic responding depends on interresponse time.
Unpublished manuscript. Silberberg, A., Hamilton, B., Ziriax, J. M., & Casey, J.
(1978). The structure of choice. Journal of
Nevin, J. A. (1969). Interval reinforcement of choice Experimental Psychology: Animal Behavior Processes,
behavior in discrete trials. Journal of the Experimental 4, 368–398. doi:10.1037/0097-7403.4.4.368
jeab.1969.12-875 Skinner, B. F. (1938). The behavior of organisms. New
York, NY: Appleton-Century.
Nickerson, R. S. (2002). The production and perception
of randomness. Psychological Review, 109, 330–357. Skinner, B. F. (1959). Cumulative record. New York, NY:
doi:10.1037/0033-295X.109.2.330 Appleton-Century-Crofts. (Original work published
Nigg, J. T. (2001). Is ADHD a disinhibitory disorder? 1935)
Psychological Bulletin, 127, 571–598. doi:10.1037/ Skinner, B. F. (1981). Selection by consequences. Science,
0033-2909.127.5.571 213, 501–504. doi:10.1126/science.7244649
Notterman, J. M., & Mintz, D. E. (1965). Dynamics of Smith, J. M. (1982). Evolution and the theory of games.
response. New York, NY: Wiley. Cambridge, England: Cambridge University Press.
Page, S., & Neuringer, A. (1985). Variability is an oper-
ant. Journal of Experimental Psychology: Animal Staddon, J. E. R., & Simmelhag, V. L. (1971). The
Behavior Processes, 11, 429–452. doi:10.1037/0097- “superstition” experiment: A reexamination of its
7403.11.3.429 implications for the principles of adaptive behav-
ior. Psychological Review, 78, 3–43. doi:10.1037/
Popper, K. R., & Eccles, J. C. (1977). The self and its h0030305
brain. New York, NY: Springer-Verlag.
Stokes, P. D. (1995). Learned variability. Animal Learning
Pryor, K. W., Haag, R., & O’Reilly, J. (1969). The creative and Behavior, 23, 164–176. doi:10.3758/BF03199931
porpoise: Training for novel behavior. Journal of
the Experimental Analysis of Behavior, 12, 653–661. Stokes, P. D. (2001). Variability, constraints, and cre-
doi:10.1901/jeab.1969.12-653 ativity: Shedding light on Claude Monet. American
Radloff, L. S. (1991). The use of the Center for Psychologist, 56, 355–359. doi:10.1037/0003-066X.
Epidemiological Studies Depression Scale in ado- 56.4.355
lescents and young adults. Journal of Youth and Stokes, P. D., & Harrison, H. M. (2002). Constraints have
Adolescence, 20, 149–166. doi:10.1007/BF01537606 different concurrent effects and aftereffects on vari-
Rhinehart, L. (1998). The dice man. Woodstock, NY: ability. Journal of Experimental Psychology: General,
Overlook Press. 131, 552–566. doi:10.1037/0096-3445.131.4.552
Ross, C., & Neuringer, A. (2002). Reinforcement of Taleb, N. N. (2007). The black swan. New York, NY:
variations and repetitions along three independent Random House.
response dimensions. Behavioural Processes, 57,
199–209. doi:10.1016/S0376-6357(02)00014-1 Thorndike, E. L. (1911). Animal intelligence. New York,
NY: Macmillan.
Rubia, K., Smith, A. B., Brammer, M. J., & Taylor, E.
(2007). Temporal lobe dysfunction in medication- Townsend, J. T. (1992). Chaos theory: A brief tutorial
naïve boys with attention-deficit/hyperactivity disor- and discussion. In A. F. Healy, S. M. Kosslyn, & R.
der during attention allocation and its relation M. Shiffrin (Eds.), From learning theory to connection-
to response variability. Biological Psychiatry, 62, ist theory: Essays in honor of William K. Estes (Vol. 1,
999–1006. doi:10.1016/j.biopsych.2007.02.024 pp. 65–96). Hillsdale, NJ: Erlbaum.
Schwartz, B. (1980). Development of complex stereo- Vogel, R., & Annau, Z. (1973). An operant discrimina-
typed behavior in pigeons. Journal of the Experimental tion task allowing variability of reinforced response
Analysis of Behavior, 33, 153–166. doi:10.1901/ patterning. Journal of the Experimental Analysis of
jeab.1980.33-153 Behavior, 20, 1–6. doi:10.1901/jeab.1973.20-1
545
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Wagner, K., & Neuringer, A. (2006). Operant variabil- Wegner, D. M. (2002). The illusion of conscious will.
ity when reinforcement is delayed. Learning and Cambridge, MA: MIT Press.
Behavior, 34, 111–123. doi:10.3758/BF03193187 Weiss, R. L. (1965). “Variables that influence random-
Ward, L. M., & West, R. L. (1994). On chaotic generation”: An alternative hypothesis. Perceptual
behavior. Psychological Science, 5, 232–236. and Motor Skills, 20, 307–310. doi:10.2466/
doi:10.1111/j.1467-9280.1994.tb00506.x pms.1965.20.1.307
546
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Chapter 23
Behavioral Pharmacology
Gail Winger and James H. Woods
In 1968, Roger Kelleher and William Morse wrote a first given in clinical trials in 1961 and which
review of behavioral pharmacology, describing the became among the most widely used drugs in the
field as “a relatively new branch of pharmacology practice of medicine in the 1970s, were discussed
concerned with the effects of drugs on behavior” for the treatment of anxiety. Early but still useful
and an area that “has been stimulated by the discov- treatments for depression were described in the
eries of new drugs” (p. 2) between the mid-1950s form of monoamine oxidase inhibitors and tricyclic
and the mid-1960s. A comparison of the 1955 edi- antidepressants. A section in this chapter on psy-
tion with the 1965 edition of a medical pharmacol- chotogenic drugs described the behavioral and sub-
ogy text (Goodman & Gilman, 1955, 1965) makes jective effects of LSD and mescaline. In an update of
this latter point convincingly. In the 1955 edition, the chapter “Narcotic Analgesics” (Jaffe, 1965b), the
the number of drugs that acted on the central ner- number of opiate drugs had increased to 20,
vous system was limited. Many of these are no lon- although a relatively pure opioid antagonist (nalox-
ger used today, but the list of those that are in one) would not be mentioned in this text for
current use includes barbiturates as hypnotics, can- another 5 years, and then only briefly (Jaffe, 1970).
nabis as a miscellaneous agent, morphine and The first chapter, “Drug Abuse and Drug Addiction”
codeine as narcotic analgesics, and methadone as a (Jaffe, 1965a), emphasized the behavioral aspects of
treatment for narcotic withdrawal. Nalorphine was the use of psychoactive drugs for recreational
included as a narcotic antagonist. Cocaine was purposes.
described as a local anesthetic, its only approved Both of these chapters included a description of
use, and amphetamine was included as a sympatho- some of the effects the drugs of interest had on
mimetic, although its central nervous system stimu- intact, behaving animals. In other words, presenta-
lant actions were acknowledged as the primary tion of the effects of these psychoactive drugs
therapeutic use. included descriptions of their behavioral pharmacol-
By Goodman and Gilman’s 1965 edition, central ogy and how they related to clinical use of the
nervous system drug use had been transformed. drugs. For example, although chlorpromazine and
Two new chapters were added that dealt exclusively the tricyclic antidepressant imipramine have very
with psychoactive drugs. One, “Drugs Used in the different clinical uses, they both have sedative
Treatment of Psychiatric Disorders,” described the effects among humans. Animal studies of these
antipsychotic chlorpromazine, which had been drugs would be of limited utility if they showed only
introduced to Western medicine in 1954 as “among the common, sedative effects of these drugs. How-
the most widely used drugs in the practice of medi- ever, an important experiment conducted by Peter
cine today” (Jarvik, 1965, p. 163). Benzodiazepines, Dews (1962, as cited in Jarvik, 1965) demonstrated
Writing of this chapter was supported substantially by National Institute on Drug Abuse Grant DA 015449 to Gail Winger.
DOI: 10.1037/13937-023
547
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
that chlorpromazine and imipramine had markedly organism, made in response to a stimulus that is
different effects on the behavior of pigeons under either internally derived or externally applied. Drugs
specific experimental conditions. Dews’s findings in this case are nearly always chemicals that enter
indicated that behavioral pharmacology experiments and act primarily on the central nervous system to
using animal subjects could reveal much of rele- modify ongoing behavior, although they may have
vance about the burgeoning number of new drugs prominent action on peripheral neurons as well. As
to treat psychiatric symptoms in humans. They described by Kelleher and Morse (1968), behavioral
also suggested that a science that explores drug– pharmacologists are typically involved in one of two
environment–behavior interactions can help deter- tasks. By far the more frequent task is to use well-
mine the conditions under which relevant behavioral studied behavioral preparations to assist in explicat-
effects of drugs can be observed and can increase ing the effects of interesting, perhaps novel drugs.
researchers’ understanding of how pharmacological This task is the more frequent one in large part
and contextual variables combine to affect behavior. because the number of established, validated behav-
Two additional factors contributed to the emer- ioral preparations are, or should be, fairly limited;
gence and development of behavioral pharmacology. however, the number of drugs are almost limitless.
First, the 1960s saw an increase in the illicit use of Less often, behavioral pharmacologists use a thor-
psychoactive drugs. The health and cultural prob- oughly studied drug to better understand a novel
lems associated with these illicit activities motivated behavioral preparation. Occasionally, a scientist is
further scientific study of the behavioral effects of ignorant of the actions of both the drug of interest
these drugs. Second, laboratory procedures devel- and the behavior that seems best to reflect the drug’s
oped within the experimental analysis of behavior actions. This undertaking is much riskier, but it can
were proving useful in identifying and categorizing be quite useful if done carefully. It is also often the
psychoactive drugs, both in general and in the con- case that behavioral pharmacologists undertake a
text of their therapeutic use. The combination of study in which both the behavior and the drug have
these factors made the birth of behavioral pharma- been thoroughly investigated separately, but inter-
cology an easy and natural one. esting questions remain as to how they interact.
Our goals in this chapter are first to define the
field of behavioral pharmacology and place it in the Distinguishing Behavioral Pharmacology
context of other areas of research that involve stud- From Psychopharmacology
ies of behavior–drug interactions. Our second goal is Psychopharmacology is focused on the investigation
to provide the reader with some of the pharmacolog- of drugs that have or might have effects in treating
ical concepts that are important to behavioral phar- psychiatric or neurological diseases such as schizo-
macology, with a few examples of how behavioral phrenia or movement disorders. Psychopharmacolo-
pharmacology has benefitted from and contributed gists are primarily interested in describing how
to the science of pharmacology. Finally, we describe these drugs modify human psychopathological
the conditioning approaches that have contributed behavior. Psychopharmacologists are likely to use
most to the field of behavioral pharmacology, with large groups of patients, all having a particular psy-
an emphasis on the consideration of drugs as stimuli chiatric disease. Only one or two doses of the drug
that can interact with, modify, and be modified by in question are typically given. If more than one
behavior. dose is given, each dose is typically given to differ-
ent groups of patients, and a separate group will be
given a placebo or other control preparation. Psy-
What is Behavioral Pharmacology?
chopharmacologists generally want to know
What are the aspects, limits, and range of study of whether the disease gets better as a function of drug
this branch of pharmacology concerned with the administration, and they use a large number of sub-
effects of drugs on behavior? Behavior is the ongo- jects so that even if the therapeutic effect is small,
ing, typically motoric behavior of an intact, living they will be able to detect it.
548
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Behavioral pharmacologists can and do study the studied. For example, central stimulants, excitatory
same drugs as psychopharmacologists, but they are amino acid agonists and antagonists, opioids, canna-
likely to do so by giving the drugs to normal animals binoids, and depressant drugs are used extensively
and observing how the animal’s behavior is changed. by both behavioral pharmacologists and neuroscien-
The animal’s behavior might be rendered abnormal tists. Investigators in both areas may use similar
by placing it in a swim tank (an animal model of behavioral tools to study these drugs. The difference
depression) or in an elevated plus maze (an animal lies primarily in where the two types of scientists
model of anxiety) or by administering a central look for explanatory principles (see Chapter 15, this
stimulant (an animal model of psychosis). In these volume). Neuroscientists are usually searching for
ways, the behavioral pharmacologist might be inter- an underlying brain mechanism that is related to a
ested in some of the same questions that interest the drug’s action. Their search often leads them to look
psychopharmacologist; that is, how does this drug for neural changes or other changes in the brain that
modify behavior that reflects a psychopathology? develop as a consequence of drug administration.
The scientific difference may be primarily in experi- Even if they do not actually study neural changes
mental methodology or in the type of inferences that accompany drug effects, neuroscientists are
drawn from the research. In contrast to the psycho- quite willing to suggest what these changes might
pharmacologist, the behavioral pharmacologist typi- be. By contrast, behavioral pharmacologists typically
cally uses a relatively small number of subjects, do not have a strong investment in describing neural
establishes stable predrug behavior patterns, and activity or neural changes that accompany drug
evaluates the effect of a number of doses of the test action. They focus on how the drug alters ongoing
drug, including a dose that produces no effect (a behavior and find sufficient explanatory value in
placebo or nondrug control). Although this compar- that behavioral change.
ison suggests that behavioral pharmacologists study
animal behavior and psychopharmacologists study
Pharmacological Aspects of
the effects of drugs in humans, behavioral pharma-
cologists can study human behavior as well. Human
behavioral pharmacologists, however, have the same Pharmacology has been defined as the science of the
fundamental approach to their investigation as do source, properties, physiological action, absorption,
animal behavioral pharmacologists. They establish fate, excretion, and therapeutic uses of drugs (Gil-
rigorous measures of behavior in a relatively small man, 2001). There is nothing obvious in this defini-
number of human subjects, administer several doses tion, used since 1941, that is of direct relevance to
of the drug under study, carefully observe as many behavioral pharmacology as we have defined it thus
changes in the behavior as possible, and reestablish far. Nevertheless, some of the most interesting and
the non–drug-influenced behavior between drug important aspects of behavioral pharmacology have
administrations. Each subject is likely to receive come from the study of drug action on isolated
each dose of the drug and placebo or a no-effect organs or on molecular events.
drug dose during the course of the study; the sub-
jects may or may not have a psychiatric disease; and Pharmacodynamics
the outcomes may or may not be related directly to a Most of the drugs used to study behavioral or physi-
potential therapeutic effect of the drug. ological systems produce effects by acting directly or
indirectly on receptors. The receptors with which
Distinguishing Behavioral Pharmacology the drugs interact normally transduce stimuli pro-
From Neuroscience vided by endogenous neurotransmitters, neuromod-
The overlap between some aspects of behavioral ulators, or hormones and are usually located in the
pharmacology and the neuroscience of drug action area in which synaptic connections between neu-
is sufficient to comment on how these two areas dif- rons occur. This area is where the various neuro-
fer. The overlap is primarily in the drugs that are chemicals are released, have their actions, and are
549
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
quickly metabolized or removed. The endogenous subunits, and these subunits may have a slightly dif-
chemicals either increase or decrease the chances ferent protein structure. Subunits are typically given
that the downstream neuron will fire in response to Greek labels. Thus, the nicotinic acetylcholine recep-
an incoming stimulus, or occasionally even in the tor is an ion pore that has a total of five subunits that
absence of such a stimulus. Drugs typically modify may consist of alpha, beta, delta, and gamma sub-
the probability of neuronal firing by acting directly units; the alpha and beta subunits themselves may
on a specific receptor (e.g., opioid drugs), by alter- exist in slightly different forms, which are distin-
ing the rate of removal of a specific endogenous guished numerically. For example, the nicotinic ace-
neurotransmitter (e.g., cocaine and many other tylcholine receptor, which appears to be responsible
stimulant drugs), by changing the shape of a partic- for the reinforcing effects of nicotine, is constituted
ular receptor and thereby modifying its sensitivity to with two alpha4 and three beta2 subunits and is
the endogenous agent (e.g., benzodiazepines and often referred to as the alpha4beta2 nicotine receptor.
barbiturates), or through one or more other chemi- These receptor form distinctions are important
cal or physical interactions. because receptors with specific subunit composition
Receptors are proteins located on cellular mem- are located in distinct areas of the brain and likely
branes. As suggested by their name, receptors have subserve slightly different functions, even though
active sites that are sensitive to specific neurochemi- they may be activated by the same endogenous agent.
cals. They alter their structure when the active site is Some drugs have been synthesized to interact more
bound by the appropriate neurochemicals, which or less selectively with receptors of different type or
subsequently leads to a modification of the electrical subtype composition. The collaboration between
or biochemical sensitivity of the cell. There are two chemists and behavioral pharmacologists to design,
fundamentally different types of receptors. One type synthesize, and evaluate these types of receptor-
is an ion pore in the cell membrane (an ionotropic subtype–selective drugs has tremendous potential in
receptor; Figure 23.1). As shown in Figure 23.1, furthering understanding of the specific behavioral
ionotropic receptors are made up of several different function of that particular receptor type or subtype.
The action of the neurochemical can result in the
ion pore being open more or less frequently. When
it is open, the pore allows ions to pass in or out of
the cell, changing the neuron’s likelihood of firing.
Action at ion channel receptors generally results in a
fairly localized change in neuronal response through
a brief change in the influx of charged ions through
the receptor. The behavioral effects of drugs that act
on specific ionotropic receptors often give important
clues as to the function of that receptor class. The
dissociative anesthetics phencyclidine and ketamine,
for example, produce their effects by entering and
blocking the ion pore of a specific type of ionotropic
receptor that responds to actions of the major excit-
atory neurotransmitter, glutamate. Benzodiazepines
and barbiturates, however, modify the shape of
Figure 23.1. A drawing of an ionotropic receptor,
ionotropic GABA receptors so that they bind less
located in the membrane of a postsynaptic neuron.
This receptor has five sections, some of which are readily to the primary endogenous inhibitory neu-
likely to have a different protein structure. Agonist rotransmitter, GABA.
interaction causes the receptor to modify its struc- The second type of receptor is the G-protein-
ture in such a way as to allow an ion to pass through
to the inside of the neuron, which alters the possibil- coupled receptor (GPCR), also called the metabo-
ity that the neuron will be activated. tropic receptor. The GPCR has seven areas that pass
550
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
through the cell membrane (seven transmembrane serotonin. As with ionotropic receptors, there are
domains), forming a circular structure somewhat frequently several subtypes of GPCR receptors, and
like the ion pore (see Figure 23.2). GPCRs do not evaluating drugs that have been designed to interact
have distinct subunits; rather, they differ from each specifically with certain receptor subtypes has
other in the amino acid arrangement of the receptor proven to be one of the most valuable contributions
protein. The action of the drug or endogenous trans- of behavioral pharmacology.
mitter on this receptor initiates a biochemical cas- The receptors that drugs act on are present in
cade that differs depending on the particular membranes to carry out the normal processes of the
receptor type. This cascade, in which each change in cell. In a few cases, the endogenous chemical that
protein structure causes a change in another, down- acts on these drug receptors has been identified.
stream protein structure until a final modification of Frequently, the chemical was known before the
neuronal activity is produced, permits more amplifi- receptor was discovered. For example, many small-
cation and modulation in outcome than does the molecule neurotransmitters such as norepinephrine,
ionotropic receptor. This presents the possibility of dopamine, and serotonin were identified as neu-
a much more nuanced and widespread outcome of rotransmitters before their various receptors and
the drug effect, which is especially important in rela- receptor subtypes were discovered. These particular
tion to the heterogeneous distribution of GPCR neurotransmitters are especially interesting to
receptors in the brain and spinal cord. behavioral pharmacologists because the behavioral
All four types of opioid receptors (mu, kappa, effects of many centrally acting drugs, including
delta, and nociception receptors) are GPCRs, as central nervous system stimulants (e.g., cocaine and
are receptors for dopamine and most receptors for amphetamine), hallucinogens (e.g., LSD and mesca-
line), and antidepressants (e.g., fluoxetine and ser-
traline) involve actions directly on one or more of
these neurotransmitters’ receptors or blocking the
reuptake and removal of one or more of these
neurotransmitters.
In other cases, the endogenous chemical was
searched for and located after a receptor was found
for a drug action. For example, researchers began
looking for endogenous chemicals that interacted
with the opioid receptor after these receptors were
identified as the locus of action of morphine and
morphinelike drugs. Identification of the various
endogenous opioid peptides, endorphins, and
enkephalins resulted from this investigation
(Hughes, 1975). In still other instances, the drug
receptor has been reasonably well described, but the
endogenous ligand remains elusive. Endogenous
ligands for the benzodiazepine receptor, for exam-
Figure 23.2. A drawing of a G-protein-coupled ple, have not yet been definitively identified and
receptor, located in the membrane of a postsynaptic may not exist.
neuron. Also known as a metabotropic receptor, it
Description of the actions of drugs on receptors
contains seven transmembrane domains, connected
by intracellular and extracellular amino acid loops. is the cornerstone of the development of receptor
Agonist interaction at a specific extracellular site theory, which in turn is a cornerstone of the science
causes a change in protein structure, which causes an of pharmacology. The description takes the form of
intracellular G-protein to detach from the receptor,
activate a second messenger, and modify cell signal- dose–response functions wherein the effects of a
ing processes. range of doses of a particular drug are evaluated in a
551
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
specific assay (Kenakin, 1997). In most in vitro

assay systems, such as smooth muscle preparations,
the more drug that is given, the greater the effect, up
to a certain maximum that usually reflects the limits
of the assay’s response. When the dose is plotted
logarithmically and the effect is plotted linearly, the
resulting dose–response curve is sigmoidally shaped
(Figure 23.3). When behavioral measures are used,
dose–response curves of this shape are also fre-
quently obtained. With many behavioral assays,
Figure 23.4. Increasing doses of the
however, across a substantial range of doses, target dopamine receptor agonist ropinirole
behavior may increase and then decrease, and the produces an increase in yawning in
resulting dose–response curve has a bell shape or rats. Further increases in dose yields
inverted-V shape (Figure 23.4). These curves may a decrease in yawns, resulting in an
inverted-V shaped dose–response curve.
reflect several aspects of drug–behavior interaction. These previously unpublished data are
Action at one receptor may be responsible for the courtesy of Greg Collins. Statistical
initial increasing function, and action at another significance of results versus vehicle
controls according to a Dunnett’s test are
receptor may account for the decreasing function. indicated as * p .05, *** p .01.
Alternatively, action at a single receptor may pro-
duce the increasing function, and continued action
administration of agonist and antagonist com-
at this receptor may produce other behavior that is
pounds, dose–response curves of whatever shape are
incompatible with the target behavior, which then
critical in understanding precisely the same actions
decreases.
and interactions using behavioral preparations.
The fact that behavioral dose–response curves
may not precisely mimic the shape of the curves that Drug–receptor interactions. Receptor theory
have been used to establish and elaborate receptor emphasizes three general parameters of drug–
theory should not detract from the necessity of receptor interactions: affinity, or the tendency to
obtaining complete dose–response curves for behav- bind to the receptor; efficacy, or the ability to pro-
ioral measures. Just as these curves are essential to duce a response once binding has occurred; and
understanding drug actions in nonbehavioral sys- a complex interaction between affinity and effi-
tems, permitting explication of the nature of the cacy called potency, which translates roughly into
effect, allowing comparison of one drug’s effects how much drug is required to produce an effect.
to another drug’s effects, and confirming the Although affinity and efficacy can be compared in
receptor bases of the outcome through concurrent a meaningful way only among drugs that act at the
same receptor, useful potency comparisons can be
made among all drugs that produce the same effect.
As an example, mu and kappa opioid agonists pro-
duce analgesia by actions on their separate recep-
tors. Therefore, it is proper to compare the efficacies
of the individual mu agonists by observing the
amount of analgesia each produces (greater peak
analgesia may indicate greater efficacy). However,
comparisons between mu and kappa agonist effica-
cies cannot be made using this or any other assay.
The potencies of these drugs, however, can be com-
Figure 23.3. A sigmoidally shaped
dose–response function. Note the linear- pared. If the dose–response function of a kappa ago-
log plot. nist is located to the left of the function generated by
552
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
a mu agonist (i.e., the same responses are produced highest affinity among these drugs, whereas mor-
by smaller doses of the kappa agonist), a correct phine has the lowest.
conclusion is that this particular kappa agonist is Agonists are drugs that have both affinity and
more potent than this particular mu agonist. efficacy at their receptors; antagonists are drugs that
Most drug–receptor interactions are quite tempo- have affinity but no efficacy at their receptors.
rary, with the drug attaching to and then rapidly Antagonists bind to the receptor in such a way as to
coming off the receptor. Affinity, therefore, does not reduce the likelihood of binding of both endogenous
reflect how tightly or how long the drug attaches to and exogenous ligands, but they may not cause a
the receptor but rather the likelihood that the drug response themselves. There are three mechanisms
will attach to the nearby receptor. Affinity is simple by which apparent behavioral responses can be pro-
to measure in binding assays, in which the ability of duced by established antagonists. First, a behavioral
a test compound to displace a radioactive ligand response may result if there is constitutive activity at
from the receptor is determined. Affinity is usually the receptor (i.e., receptor activity in the absence of
established by determining the drug’s ability to dis- a stimulating agonist). If this constitutive activity is
place a labeled standard, and it is expressed as the prevented by the antagonist, it could result in a
Ki. The smaller this number is, the fewer drug mole- behavioral response. Second, an antagonist response
cules are necessary to inhibit binding of the stan- could result if tone is produced by the endogenous
dard, and the higher the affinity of the drug is. Table agonist at the receptor. An antagonist drug with no
23.1 displays the affinity of a group of opioids as efficacy itself at this site could cause a response if it
reflected in their relative ability to displace radiola- prevents full access to the receptor by the endoge-
beled DAMGO, a mu-selective agonist (Emmerson, nous ligand (i.e., a reduction in tone). Finally, a
Liu, Woods, & Medzihradsky, 1994). The agonist drug that appears to be an efficacy-free ligand could
etonitazene and the antagonist quadazocine have the produce a response if it acts as an inverse agonist at
the site of action (vide infra). An inverse agonist is a
drug that produces the opposite effect to the agonist
Table 23.1 or endogenous ligand.
Affinity of Several Opioid Agonists and Antagonists come in a variety of types. Competi-
Antagonists at the Mu Opioid Receptor, as tive antagonists, as do most agonists, bind to their
Determined by Their Ability to Displace the receptors only briefly, and through the process
Tritiated Mu Ligand DAMGO of repeatedly attaching to and disengaging from the
receptor, they prevent the agonist from having as
Compound [3H]DAMGO 𝛍Ki; nM
much access to that receptor. However, agonists
DAMGO 1.23 administered at large enough doses can produce
Etonitazene 0.02 their maximal effect by overwhelming the effects of
Fentanyl 1.48 the competitive antagonist, which is illustrated in
Levorphanol 1.23 Figure 23.5. Here, a single dose of the competitive
Morphine 2.66
Nalmefene 0.13
mu opioid receptor antagonist nalmefene is given
Naloxone 0.62 (±0.04) before administration of several intrathecal doses of
Naltrexone 0.11 (±0.01) the mu opioid receptor agonist morphine. Intrathe-
Quadazocine 0.03 cal morphine elicits scratching in the monkey, and
Sufentanil 0.19 (±0.01) nalmefene produces a systematic decrease in the
potency of morphine in eliciting this response.
Note. From “Binding Affinity and Selectivity of Opioids
at Mu, Delta, and Kappa Receptors in Monkey Brain Noncompetitive, or uncompetitive, antagonists
Membranes,” by P. J. Emmerson, M.-R. Liu, J. H. Woods, bind to the receptor more tenaciously and thereby
and F. Medzihradsky, 1994, Journal of Pharmacology restrict access to agonists in a more absolute fashion
and Experimental Therapeutics, 271, p. 1634. Copyright
1994 by the American Society for Pharmacology and than a competitive antagonist. Uncompetitive antag-
Experimental Therapeutics. Used with permission. onism can also occur if the antagonist reduces the
553
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
Figure 23.5. Intrathecal morphine produces increased scratching in rhesus

monkeys, which can be antagonized by administration of the opioid receptor
antagonist nalmefene, which is shown by a rightward shift in the morphine dose–
response curve. These previously unpublished data are courtesy of Mei-Chuan Ko.
effect of the agonist by distorting the shape or func- among drugs that act on the same receptor to pro-
tion of the receptor, as occurs with some N-methyl duce their effects. Highly efficacious agonists (possi-
D-aspartate (NMDA) antagonists that enter and ble full agonists) produce larger effects than less
physically block a glutamate ionotropic receptor efficacious agonists, and a group of drugs that act as
channel. Typically, the interaction between an ago- agonists at the same receptor can be ranked with
nist and its uncompetitive antagonist is described by respect to their efficacy. An important rule is that
a curve in which the maximal effect of the agonist the rank order of efficacy for a group of agonists act-
can no longer be produced, even after administra- ing at a receptor will not change across different
tion of very large doses. preparations that measure an effect mediated
Just as drugs can vary in their affinity for their through transduction of that particular receptor’s
receptor, they can also vary in their efficacy. Efficacy actions.
refers to a drug-based ability to activate a response An example follows: There are many mu opioid
once binding has occurred. A drug that produces the agonists, and the rank order of efficacy for three of
maximum possible response in all the systems in them is first fentanyl, then morphine, then nalbu-
which it has been tested is usually called a full ago- phine. Thus, fentanyl will produce a response
nist. This label is slightly tenuous because the possi- greater than or equal to that of morphine, which will
bility always exists that a new drug will be have a response greater than or equal to nalbuphine
synthesized that produces an even larger response in all behavioral, physiological, or biochemical sys-
than any previous drug and will become the new tems that reflect mu opioid receptor activity. If a
definition of a full agonist. If there is a maximum system were found that ranked these drugs in any-
response that a given preparation is capable of, there thing other than this order, the drugs would almost
may be several putative full agonists in this prepara- certainly be acting on something other than or in
tion, each producing the maximum response. To addition to the mu opioid receptor alone.
rank these drugs on the basis of their efficacy,
another preparation is needed that is less sensitive to Test system sensitivity. Why, one might ask,
the effects of agonists at this receptor. The agonist would nalbuphine ever produce a response equal to
with the highest efficacy is the one that produces the that of fentanyl if fentanyl is an agonist with higher
largest response in the least sensitive assay. efficacy? The answer to this question raises another
This description should convey the notion that important point in drug–receptor pharmacology.
efficacy is a relative matter and involves comparisons The nature of the test system, behavioral or not,
554
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
plays a critical role in assessing efficacy. A very sen- analgesia by binding to 50% of the available receptor
sitive test system may reveal that one drug is more population, and nalbuphine can produce full analge-
effective than another, whereas a less sensitive sys- sia by binding to 90% of the available receptors. In
tem may suggest that both drugs have full efficacy. this assay, at this water temperature, there is an
For example, in analgesia evaluations using rhe- increase in receptor reserve and an increase in the
sus monkeys, fentanyl increases the latency of tail apparent relative efficacy of nalbuphine.
removal from warm water (55 °C). At a sufficiently Because each of these drugs is binding to the
large dose of fentanyl, latencies reach the maximum same receptor, a drug with less efficacy is able to
allowable time of 20 seconds. Nalbuphine also antagonize the effects of drugs with greater intrinsic
increases tail removal latencies but does not produce efficacy. This is the case only in assays in which the
the maximum analgesic response when the water partial agonist is unable to produce as much
is 55 °C. After administration of substantial doses response as a full agonist. If both agonists produce a
of nalbuphine, the monkey leaves its tail in very full effect, the less efficacious agonists will not block
warm water for only about 1 second. This difference the effect of the agonist with greater efficacy. When
reflects the lower efficacy of nalbuphine relative to a partial agonist produces a partial effect, it can
fentanyl. However, if the tests are done with water reduce the activity of a full agonist only to the level
that is 48 °C, both drugs produce the full 20-second of that shown by the partial agonist. In the example
analgesic response. One cannot therefore deter- given earlier of tail withdrawal analgesia, nalbu-
mine any difference in the efficacy of nalbuphine phine should be able to reduce the effect of fentanyl
and fentanyl using a 48 °C water analgesia assay to that produced by nalbuphine alone when 55 °C
(Gerak, Butelman, Woods, & France, 1994; Walker, water temperature is used as the noxious stimulus.
Butelman, DeCosta, & Woods, 1993). Drug–receptor interactions have an additional
What might sensitivity mean mechanistically? complication: the occasional existence of drugs that
One way of thinking about this involves the concept act at a common receptor and produce effects that
of receptor reserve. Most drugs do not have a one-to- are opposite to those produced by the agonist. These
one correspondence between their effect and bind- drugs are termed inverse agonists. They have been
ing to their receptor. Drugs with high efficacy can well described primarily in the benzodiazepine sys-
produce a full effect in an assay by binding to a frac- tem, although inverse agonist properties may
tion of the total available receptors. For these drugs, account for some of the actions of nominal dopa-
there is a surplus of receptors, or a receptor reserve. mine antagonists as well. Suppose the receptor is
Drugs with less efficacy can produce a full effect by activated naturally only under certain conditions.
binding to a greater proportion of the total available For example, suppose natural ligands for the opioid
receptors, leaving fewer receptors in reserve. Partial receptor are released and bind to the receptor only
agonists, however, may not be able to produce a full after a painful stimulus or that natural ligands for
effect even when they bind to all of the receptors. the benzodiazepine receptor activate this receptor
This concept applies to behavioral assays. only when the individual is anxious. For these
Assume there are 1,000 mu opioid receptors receptors, administration of the antagonist should
involved in producing opioid-induced analgesia. have little effect as long as the organism is not in
When the water temperature is 55 °C, a high-efficacy pain or anxious. Contrast this to receptors that are
agonist such as fentanyl may need to bind to 900 of activated by their natural ligands on a regular basis.
these receptors to produce full analgesia. The most Dopamine receptors in the striatum, for example,
effective dose of nalbuphine in this assay may bind are activated in the process of motor movements. In
to all 1,000 of the receptors, but because it does not this case, an antagonist might have activity in all but
have fentanyl’s efficacy, it is unable to produce full sleeping or paralyzed organisms by blocking the nat-
analgesia. When the water temperature is decreased urally active dopamine.
to 48 °C, full analgesia can be produced with less Administration of many dopamine antagonists
drug stimulus. Therefore, fentanyl may produce full has profound effects on the behavior of the organism.
555
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
These antagonist effects can be explained in at least from what it would be in the more basic environ-
two ways. One is that the antagonists block the nat- ment of the blood.
urally occurring tone in their particular system, and The blood–brain barrier is simply the membrane
the resulting behavior change is the result of a that surrounds the blood vessels that are located in
reduced endogenous receptor occupation. The other the central nervous system. This membrane has rela-
possibility is that these drugs are not simply antago- tively fewer fenestrations and is supported by addi-
nists but are actually inverse agonists, producing tional cells that resist the passage of molecules.
effects that are opposite to those of the naturally Therefore, for a drug to enter the brain from the
occurring agonist. Separating these two possibilities blood, it must be even smaller or more fat soluble
can be a bit complicated in some systems and less than drugs that pass through blood vessels in the
difficult in others. Answering some of these ques- rest of the body. The blood–brain barrier is far from
tions about the mechanism of action of dopamine absolute, and a number of active transport systems
antagonists is an ongoing, important challenge, in are designed to ensure that large, charged, or fat-
part because of their importance in basic behavioral insoluble compounds that the brain requires are
pharmacology and their increasing importance in given rapid access to the central nervous system.
clinical populations These compounds aside, this barrier restricts access
of many compounds that are quite large or highly
Pharmacokinetics charged (e.g., peptides and protein-based drugs).
Drugs reach their receptors only after passing In addition to studying how drugs pass through
through various membranes that exist in the gastro- membranes, pharmacokinetics also involves studies
intestinal tract (if the drug is taken orally), that are of how drugs are absorbed, sequestered, and elimi-
located between the blood and subcutaneous or nated from the body. Behavioral pharmacologists
muscle tissue (if the drug is given subcutaneously or frequently evaluate the duration of the behavioral
intramuscularly), and that line blood vessels (if the effect they are studying, and their procedures are
drug is given by any route other than direct injec- well suited to this task. Sometimes there are con-
tion into the central nervous system). In addition, cerns that the duration of drug action does not last
because centrally active drugs must access the brain, long enough for the behavior of interest to be mea-
they must have the capacity to cross the extratight sured in an experimental session. Under these con-
membrane (blood–brain barrier) that lines blood ditions, time–action studies are appropriate. In these
vessels throughout most of the central nervous studies, behavior is plotted much like a dose–
system. response curve but with time rather than dose on
Some drugs are more adept at crossing mem- the x-axis. The duration of action of a drug is a criti-
branes than others, and the principles that govern cal aspect of its behavioral effect. Drugs with thera-
crossing one membrane generally apply to all mem- peutic actions—an antipsychotic or analgesic, for
branes. These principles are small size, fat solubility, example—will be more useful if they last more than
and lack of an electrical charge. Because membranes 24 hours and can therefore be taken once per day.
have a central core of fat cells, fat-soluble drugs are Drugs used to treat drug abuse will have a distinct
more permeable than drugs that are largely water advantage if they have extremely long durations of
soluble. Membranes contain small pores (fenestra- action, and this determination is one that behavioral
tions) that are more resistant to large-molecule pas- pharmacologists can easily make.
sage than small-molecule passage. In addition,
membranes tend to reject the passage of ionized or
Behavioral Aspects of Behavioral
charged drugs more than that of unionized drugs.
Pharmacology
The amount of ionization of a particular drug
depends on whether it is in an acidic or basic envi- Drugs are capable of modifying virtually any type of
ronment, making passage of a drug through the behavior of an intact organism. Useful categoriza-
highly acidic environment of the stomach different tion of drug effects has been obtained from studies
556
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
that simply give drugs to animals and observe CS and the delivery of the US is either very short or
changes in ongoing behavior of whatever type is quite long (two conditions that typically produce
occurring (Koek, Woods, & Ornstein, 1987), which slow learning; Harvey, 2003).
might be a recommended place to start with a drug Harvey (2003) showed that this effect of LSD was
whose behavioral effects are completely unknown, mediated through 5HT2A receptors by evaluating the
before selecting specific behavior to focus on more ability of 5HT2A antagonists to block the LSD-
intensively, for example, a particular form of elicited induced enhancement of the acquisition of the con-
behavior. More typically, the effects of drugs are ditioned response. Some of these antagonists had no
studied on one of two types of conditioned effects of their own (so-called neutral antagonists),
behavior. whereas others produced a decrease in the rate of
acquisition of the conditioned response (relative to a
Drug Effects on Classically no-LSD baseline), suggesting that they were inverse
Conditioned Behavior agonists. As predicted by receptor theory, the effects
Classically (Pavlovian) conditioned behavior, also of both the agonist and the inverse agonists were
known as respondent behavior or stimulus–stimulus blocked by a neutral antagonist, whereas the inverse
learning, is a rich source of behavioral pharmacolog- agonist was able to block the effect of the agonist.
ical study (see Chapter 13, this volume). When a Clearly, these drugs produce their effects on learn-
previously neutral stimulus has been paired with ing through the same receptor, and their categoriza-
another stimulus, an unconditioned stimulus (US), tion as agonists, inverse agonists, and antagonists
with positive or negative valence (e.g., food or elec- was nicely made using a behavioral preparation.
tric shock), behavior similar to that elicited by the This is an example of using a well-established
US is now elicited by the neutral stimulus (now behavioral preparation to evaluate the effects of a
termed a conditioned stimulus, or CS). Pavlov’s particular drug whose effects on learning have not
original example remains instructive: Dogs that been thoroughly documented. It is also an example
originally salivated when food was presented but not of the use of selective antagonists to confirm the
when a tone was presented came to salivate to the receptor-selective nature of the effect of the agonist.
tone after it had been paired several times with the These are fundamental demonstrations of these
delivery of food. Those of us who salivate while important procedures.
standing in front of a food-vending machine are
demonstrating conditioning of exactly this type. Drug Effects on Operant Behavior
Studies of the interaction of drugs with classi- A second type of conditioned behavior is operant
cally conditioned behavior often focus on the drugs’ behavior, or behavior affected by its consequences.
ability to modify learning of the CS–US association. In contrast to studies of drug effects on the acquisi-
Studies of the interaction of various serotonin ago- tion of classically conditioned behavior, studies of
nists on rabbits’ learning classically conditioned the effects of drugs on operant behavior usually
responses have been particularly interesting. There evaluate the interaction of the drug with stable
are several distinct serotonin (5HT) receptors, and behavior. The early studies in behavioral pharmacol-
many of these have receptor subtypes. The halluci- ogy stressed the interactions of drugs with well-
nogen LSD is an agonist at the 5HT2A receptor, for established, carefully controlled operant behavior,
example. LSD and other 5HT2A agonists enhance which continues to be the primary emphasis of a
acquisition of several classically conditioned great deal of behavioral pharmacological research.
responses: an eyeblink response to a tone previously The review mentioned at the beginning of this
paired with an airpuff or a shock or jaw movement chapter (Kelleher & Morse, 1968) is a classic over-
responses to a tone that was paired with water deliv- view of the interaction of drugs and operant behav-
ery (Gormezano, Harvey, & Aycock, 1980; Harvey, ior and continues to instruct on this interaction.
2003). It is interesting that the drug enhances rates Kelleher and Morse (1968) began this review by
of learning when the time between the onset of the making the important point that it is a mistake to
557
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
draw generalizations about drugs’ behavioral mecha- produces it, makes a tremendous difference in deter-
nisms of action on the basis of common notions mining the effects of various classes of drugs
about why behavior is maintained. As an example, (Kelleher & Morse, 1964).
consider a preparation used to examine the effects of A similar caution should be applied when con-
drugs on avoidance behavior. In this preparation, a sidering the effects of drugs on behavior maintained
specific stimulus is followed by brief shock. A by positive reinforcers such as food. Common
response in the presence of the shock turns it off notions about drug effects often do not, and should
(escape), and a response in the presence of the pre- not, be used to explain their behavioral effects.
shock stimulus turns off the stimulus and prevents Amphetamine, for example, increases locomotor
the shock (avoidance). Drugs such as chlorproma- activity across a limited dose range and is therefore
zine, a major tranquilizer, and morphine, an analge- considered a central stimulant (e.g., Lehr, Morse, &
sic, both reduce avoidance (behavior in the presence Dews, 1985). However, amphetamine usually
of the stimulus) but do not prevent the animal from decreases food consumption in a free-feeding situa-
making the same response to escape shock (Cook & tion (Siegel & Sterling, 1959), and stimulants have
Catania, 1964). A natural tendency might be to been used for weight reduction because of this
interpret this drug effect on the basis of the common effect. A common-notion account of these suppress-
notion that avoidance behavior is maintained by the ing effects of amphetamine involves its ability to
termination of fear, anxiety, or both; that is, the produce anorexia, or a decreased motivation for
drugs decrease these emotions, which reduces the food. Extensive evaluation of the behavioral effects
motivation to engage in fear- or anxiety-terminating of amphetamine, however, has shown that its effects
behavior. If this were the case, then chlorpromazine depend on the rate at which behavior is occurring
and morphine should also increase responding sup- before administration of the drug. If the rate is low,
pressed by the response-contingent presentation of amphetamine tends to increase responding; if the
electric shock (punishment). But neither of these rate is high, amphetamine tends to decrease
drugs attenuates punished responding under most responding (e.g., Dews, 1958). This rate-dependent
conditions (Geller, Bachman, & Seifter, 1963; Geller effect holds true regardless of whether a positive
& Seifter, 1960; McMillan, 1973a, 1973b; McMillan reinforcer such as food or a negative reinforcer such
& Leander, 1975; Morse, 1964), suggesting that as shock termination are maintaining the behavior
their behavioral effects are not the result of a reduc- (Davis, Kensler, & Dews, 1973).
tion in fear or anxiety (or of a reduction in the pain Under temporally defined schedules such as fixed-
experienced with each shock). interval schedules, in which the first response after a
There are drugs that do increase rates of pun- specified time interval produces reinforcement, rates of
ished responding. Benzodiazepines, barbiturates, responding are typically low at the beginning of each
and to a lesser extent ethanol can produce increases interval and increase as the time to reinforcer delivery
in behavior suppressed by response-contingent approaches. At appropriate doses, amphetamine
shock or other aversive stimuli (e.g., Mansbach increases the low initial rates of responding and
et al., 1988). However, benzodiazepines do not decreases the higher terminal rates of responding
decrease shock avoidance in the same manner as (Dews, 1958). Not all low rates of responding are
chlorpromazine and morphine: The dose of a benzo- increased by amphetamine, however. Rates of respond-
diazepine required to suppress escape behavior is ing suppressed by punishment are not enhanced by
the same dose necessary to suppress avoidance doses of amphetamine that increase low fixed-interval
behavior. In these situations, shock per se does not rates of responding (Kelleher & Morse, 1968). Neither
determine the effect of specific drugs but rather the does amphetamine increase rates that are low because
way in which the shock is related to behavior. of extinction or because they have never resulted in
Whether an aversive stimulus serves as a negative reinforcer delivery (Kelleher & Morse, 1968).
reinforcer, increasing behavior that prevents it, As noted by Kelleher and Morse (1968), the sed-
or serves as a punisher, decreasing behavior that ative barbiturates also have rate-dependent effects,
558
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
but these effects are somewhat different from those These interoceptive stimuli are similar in function
of amphetamine. Across a fairly narrow dose range, to an external stimulus, such as a light or tone, that
barbiturates can increase even rapid response rates, may be used to control operant behavior. These drug
such as those that occur during the terminal parts of discrimination procedures have proven useful in the
fixed intervals or during analogous segments of pharmacological classification of drugs.
fixed-ratio schedules (in which reinforcement Because behaviorally active drugs usually pro-
depends on a specified number of responses; Kelle- duce a monotonic dose–response curve rather than a
her & Morse, 1968). An even more characteristic biphasic curve in a drug discrimination assay, the
effect of barbiturates, however, and especially the ability of a receptor-selective competitive antagonist
sedative benzodiazepine agonists, with similar but to produce rightward shifts in an agonist drug dis-
more nuanced sedative effects, is their ability to crimination curve can be used to determine the
increase response rates suppressed by punishment selective nature of the drug effect. Figure 23.6 pro-
(Kelleher & Morse, 1968). This antipunishment vides an example of the ability of the selective nico-
effect of benzodiazepines is sufficiently profound tine antagonist, dihydroβerythroidine (DHβE), to
that it is an effective method of evaluating the poten- competitively antagonize the discriminative stimu-
tial of a drug to produce anxiolytic effects in a clini- lus effects of nicotine in rats.
cal population (Witkin, Morrow, & Li, 2004). This Using drug discrimination procedures to classify
is an example of using a drug to establish the diag- drugs was particularly helpful in early studies of var-
nostic effectiveness of a procedure. Even a very ious opioid agonists. Even before mu and kappa opi-
mild, nonbenzodiazepine antianxiety compound, oid receptors had been differentiated through
buspirone, has the ability to attenuate punished receptor-binding studies, studies of the differences
responding under fairly restricted conditions in the discriminative stimulus effects of these drugs
(Wojnicki & Barrett, 1993). left little room to doubt that these were distinct
receptors. The drugs ethylketocyclazocine and mor-
Drugs as Stimuli phine were designated as opioids because the behav-
Another approach to the interaction of drugs and ioral effects of both could be reversed by the opioid
behavior is to consider psychoactive drugs as stimuli antagonist naloxone. However, ethylketocyclazocine
that occasion and reinforce behavior. Drugs may be
conceived of as stimuli in much the same way as
other environmental events—more specifically, as
discriminative stimuli and reinforcing stimuli.
Drugs as discriminative stimuli. The stimulus

functions of drugs have been widely studied (e.g.,
Colpaert & Rosecrans, 1978). In fact, the psychoac-
tive drugs that have not been established as discrim-
inative stimuli and whose discriminative stimulus
functions have not been compared across a range of
doses and drug classes are very few. A database has
been established and maintained that nicely catalogs
studies of drugs as discriminative stimuli (http://
Figure 23.6. The discriminative stimulus effects
www.dd-database.org). In these studies, administra-
of nicotine are shifted in a parallel fashion to the
tion of a psychoactive drug is often used to direct right by increasing doses of the nicotine competitive
behavior to one of two response options. When the antagonist dihydroβerythroidine (DHβE). Ordinate:
drug has been given, behavior toward one option Percentage of responding on the nicotine-appropriate
lever. Abscissa: Dose of nicotine, given cumulatively,
is reinforced; when the drug has not been given, in milligrams per kilogram. These previously unpub-
behavior toward the other option is reinforced. lished data are courtesy of Emily Jutkiewicz.
559
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
and morphine did not have the same interoceptive subtypes; there are four different alpha subunits, for
stimulus effects in monkeys (Hein, Young, Herling, example.
& Woods, 1981). Both drugs were found to function The experimental drug L-838,417 was found to
as discriminative stimuli (demonstrating agonist have selective affinity for GABAA receptors with cer-
action), but one drug would not substitute for the tain alpha subunit types (McKernan et al., 2000).
other. Therefore, the difference between these drugs Animal studies revealed that L-838,417 did not have
were unlikely to be simply one of different efficacies benzodiazepine-like discriminative stimulus effects,
at a single receptor. A thorough study of the dis- although it did have sedative and antianxiety effects
criminative stimulus effect of a variety of opioids (Rowlett, Platt, Lelas, Atack, & Dawson, 2005). Not
with suggested mu and kappa selectivity was effec- only did these results suggest that it might be possi-
tive in confirming the distinction (Negus, Picker, & ble to separate the discriminative stimulus effects of
Dykstra, 1990). Among the important experiments benzodiazepines from the sedative effects, it also
were those involving administration of mu agonists indicated that the discriminative stimulus effects of
and kappa agonists in combination. Several of the benzodiazepines might be mediated through actions
drugs that were discriminated as kappa agonists did at distinct subtypes of the GABAA receptor (Rowlett
not modify the discrimination produced by the mu et al., 2005).
agonist morphine, demonstrating that the kappa As noted, however, other interpretations must
agonists were not antagonists or partial agonists at be ruled out before action at a distinct receptor is
the mu receptor at which morphine acts. This find- demonstrated. McMahon and France (2006) repli-
ing confirmed that the effects of the two drug classes cated the findings that L-838,417 did not produce
were mediated through distinct receptors and made benzodiazepine-like discriminative stimulus effects.
the case that drug discrimination studies provide They then determined, however, that this drug was
useful data to supplement and to stimulate analyses a competitive inhibitor of the discriminative stimu-
at more molecular levels (Negus et al., 1990). lus effects of the benzodiazepine midazolam. This
Although these studies on the differential dis- determination provided evidence that L-838,417
criminative stimulus effects of drugs acting on either was not acting on a distinct GABAA receptor but
mu or kappa opioid receptors supported the concept was a low-efficacy agonist at the same site at which
that these drugs were acting on distinct receptors, typical benzodiazepines act. The unusual pattern
keep in mind that a new receptor should not be the of effects of this drug (sedative effects, but no
first option to consider when two related drugs have benzodiazepine-like discriminative stimulus effects)
different discriminative stimulus effects. Thorough was because of the possibility that sedative effects
assessment of potential differences in efficacy or may be a less sensitive behavioral outcome and can
affinity is required before reaching this conclusion. develop after administration of lower efficacy com-
An example of more parsimonious options is found pounds, similar to the ability of the lower efficacy
in studies of the behavioral effects of some drugs opioids described earlier to produce full analgesia in
that bind to the benzodiazepine receptor. lower water temperature.
Benzodiazepines, including diazepam, chlordiaz- There are several lessons here. One is that studies
epoxide, oxazepam, and midazolam, among many of the discriminative stimulus effects of drugs can
others, have important uses as antianxiety or hyp- provide a wealth of information about the pharma-
notic agents, but with side effects of some concern, cological profiles of drugs. Another is that careful
including abuse liability. These drugs produce their attention should be paid to the possibility that differ-
behavioral effects by binding to a site on the GABAA ent discriminative stimulus effects of drugs that bind
receptor and increasing the potency of GABA, the to the same class of receptors may be the result of
primary inhibitory neurotransmitter in the central their greater or lesser efficacies rather than of actions
nervous system. The ionotropic GABAA receptor on unique receptors. The interactions between these
on which benzodiazepines have a binding site con- drugs can be studied easily by administering them
sists of five subunits, each with several different together and observing quantitative changes, or lack
560
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
thereof, that one drug produces on the effect of equally as well as reinforcing stimuli. This outcome
another. If a drug with reduced or no effect in a spe- might be modified depending on the drug self-
cific assay is able to antagonize the effect of the other administration procedure used (e.g., drug history or
drug in a competitive fashion, it is virtually certain fixed-ratio size), but generally there are only two
that the two drugs are acting on the same receptor, categorical outcomes.
albeit with important differences in efficacy at differ- Drug self-administration procedures have proven
ent subtypes. This was the case with L-838,417. useful in evaluating agonist–antagonist interactions.
Pretreatment with increasing doses of an opioid
Drugs as reinforcing stimuli. Whereas the dis- antagonist such as quadazocine or naltrexone will
criminative stimulus effects of a drug refer to stimuli produce parallel rightward shifts in the reinforcing
that precede responding (i.e., antecedent stimuli), stimulus effects of alfentanil (Bertalmio & Woods,
the reinforcer effects of a drug refer to stimuli that 1989), just as it will produce quantitatively identical
follow a response (i.e., consequences). The rein- rightward shifts in the analgesic and discriminative
forcing stimulus effects of drugs, particularly as stimulus effects of opioid agonists (Bertalmio &
models of human drug taking and abuse, have also Woods, 1987; Ko, Butelman, Traynor, & Woods,
been widely and usefully studied. The technique 1998). Thus, the same receptor type is responsible
is typically referred to as a drug self-administration for each of these effects.
preparation and often involves implanting an intra- Some interesting observations have been made
venous catheter and allowing the animals to make regarding drug interactions while the effects of poten-
an operant response, such as a lever press, that tial pharmacotherapies on drug self-administration
causes a small dose of the drug to be infused. This have been studied. As indicated throughout this
opportunity is repeatedly presented over daily ses- chapter, mu opioid antagonists can reduce the
sions in the same way that the reinforcing effects of potency of mu opioid agonists in virtually all behav-
small portions of food or small amounts of water are ioral assays that have been studied, which conforms
studied in animals. Most drugs that are used exten- with what is known about the sites of actions of
sively by humans for recreational purposes serve these drugs and with established receptor theory. As
as reinforcing stimuli in rats or monkeys (Griffiths, described in the previous paragraph, this interaction
Brady, & Bradford, 1979), and pharmacotherapies is observed with mu opioid receptor antagonism of
that reduce drug use in humans may also reduce the the reinforcing effects of mu opioid receptor ago-
reinforcing stimulus functions of drugs in experi- nists, and it is also the case that mu opioid receptor
mental animals (Winger & Woods, 1996). Thus, antagonists provide exceptional treatment for abuse
the drug self-administration model has potential of heroin in humans, as long as full compliance with
in developing treatments for drug abuse, insofar antagonist administration can be obtained. As a fur-
as these treatments aim to reduce the reinforcing ther example, buprenorphine, a partial mu agonist,
effects of drugs of abuse (Mello & Negus, 1996). is very effective in reducing heroin abuse. As a par-
Data collected in drug self-administration studies tial agonist, buprenorphine has both agonist and
are typically not as useful for categorizing psychoac- antagonist effects, and because both opioid agonists
tive drugs as data on discriminative stimulus effects and opioid antagonists are able to reduce heroin
of drugs. Whereas drug discrimination studies can abuse, discussion has been ongoing regarding which
group drugs into pharmacological classes (e.g., mu of these actions of buprenorphine is responsible for
or kappa opioids, dopamine reuptake inhibitors, its therapeutic effects. Although the answer has not
benzodiazepines, serotonin 2A agonists, or nicotinic been clearly determined for human heroin abusers,
agonists), self-administration studies usually have animal studies make it quite clear that buprenor-
only one of two outcomes: The drug in question is phine reduces the reinforcing effects of mu opioid
either a reinforcer or it is not. Often, drugs that have agonists through its actions as an antagonist. Preses-
clearly distinct discriminative stimulus and receptor- sion administration of buprenorphine produces
based effects (e.g., cocaine and alfentanil) serve dose-related rightward shifts in the dose–effect
561
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
curves described by opioid self-administration antagonist, NGB 2904, was unable to attenuate
(Winger & Woods, 1996). This clear interaction either the reinforcing or the discriminative stimulus
between opioid receptor agonists and antagonists effects of cocaine in rhesus monkeys (Martelle et al.,
does not prevail when studying cocaine and dopa- 2007). The reasons why it is difficult to block
mine antagonists, and nowhere is this more clear cocaine’s behavioral effects selectively with dopa-
than in studies of these drugs in self-administration mine receptor antagonists are unknown, but these
assays. One of the most likely mechanisms of findings are sufficiently consistent to have sent those
cocaine’s behavioral effects, including its reinforcing seriously interested in cocaine abuse pharmacother-
effect, is blocking transporter reuptake of the neu- apies to dramatically different targets, including
rotransmitter dopamine, thereby increasing the con- modifying the pharmacokinetics of cocaine (Collins
centration of dopamine in the synaptic cleft. et al., 2009).
Dopamine acts on five different receptors, and these
receptors are located in distinct areas and in differ- Other stimulus effects of drugs. When consider-
ent densities in the central nervous system. Theoret- ing drugs as stimuli, their ability to serve as discrim-
ically, administration of selective antagonists at inative and reinforcing stimuli has received by far
each of these receptors to animals that are self- the most attention, and an extremely large literature
administering cocaine would provide an answer to has developed for both of these functions. However,
the question of which receptor is most critical to the these situations are certainly not the only ones in
reinforcing effects of cocaine and also put forward which drugs have stimulus properties. We describe
potential pharmacotherapies for cocaine abuse. Ago- five of these next; many more have been overlooked
nists at this particular receptor should also serve as in the interests of time and space.
reinforcing stimuli. First, drugs can act as stimuli that elicit specific
Unfortunately, the interaction between cocaine behavior patterns. We described several examples
and dopamine antagonists has not been as clear cut earlier, usually in the form of drug-produced effects.
as those between opioid agonists and antagonists. As Other examples include mu opioid receptor ago-
reviewed by Witkin (1994), antagonists at the nists, which elicit analgesia and scratching (Ko &
D1-like (D1 and D4) and D2-like (D2, D3, and D4) Naughton, 2000); dopamine D3–preferring agonists,
receptors modified some actions of cocaine, includ- which elicit yawning (Depoortère et al., 2009); D2
ing apparent antagonism of its reinforcing effects. receptor agonists, which elicit decreases in body
However, these findings had two consistent draw- temperature (Collins et al., 2007); amphetamine,
backs. First, the effects were not robust, in that which elicits increases in locomotor activity (e.g.,
dose–effect curve shifts were small and limited; Lehr, Morse & Dews, 1985); and d-amphetamine,
second, the drug doses that suppressed cocaine- which elicits focused stereotyped head movements
maintained responding also frequently suppressed in rats (Fowler, Pinkston, & Vorontsova, 2007).
food-maintained responding. Thinking of these effects as drug stimulus–elicited
More recent studies have not been much more effects rather than as drug-produced effects is a sub-
promising, despite the synthesis of newer, more tle modification, but one that can influence the gen-
selective drugs. A D3 partial agonist, CJB090, pro- eral approach to behavioral pharmacology. If drugs
duced parallel shifts to the right in cocaine’s dis- are considered to act as stimuli, investigators and
criminative stimulus effect and suppressed students are more likely to be challenged to consider
self-administration of a single dose of cocaine (no their effects as subject to modification and condi-
cocaine dose–response curves were obtained). tioning, for example, rather than as immutable
However, this drug also suppressed food-maintained outcomes.
responding at doses that suppressed cocaine- Second, drugs can act as punishing stimuli to
maintained responding, indicating that its rate- reduce behavior. Most studies of drug-contingent
depressing effects may not have been selective behavior have focused on drugs as positive reinforc-
(Martelle et al., 2007). A relatively selective D3 ers. As mentioned earlier, this work has tended to
562
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
categorize drugs as either reinforcers or nonrein- when the drug is given repeatedly (sensitization).
forcers. Very little attention has been paid to a third After haloperidol sensitization, if haloperidol is
possibility: that a drug may be aversive. This possi- tested in the presence of the NMDA antagonist
bility can be evaluated in studies to determine MK-801, it reverts to producing the amount of cata-
whether a specific drug will serve as a punishing lepsy that occurred on the first administration. This
stimulus and, when delivered contingently on a response was originally interpreted as an ability of
response, reduce the rate of behavior maintained MK-801 to block sensitization to haloperidol. How-
by another reinforcer. For example, response- ever, another interpretation is that MK-801 func-
contingent delivery of intravenous histamine sup- tions as a novel environmental context, and as
pressed behavior maintained by food (Goldberg, previously noted, sensitization effects are reduced in
1980). Many drugs that are not self-administered by novel contexts. Subsequent research showed that if
laboratory animals, including kappa opioid receptor MK-801 is given on every occasion on which halo-
agonists, many antipsychotic drugs, and hallucino- peridol is given (making MK-801 the environmental
gens such as LSD, should be evaluated in punish- context reliably correlated with haloperidol admin-
ment paradigms so that their behavioral actions can istration), sensitization in the form of increasing cat-
be more thoroughly characterized. alepsy occurred just as it would if haloperidol was
Third, drugs can modify the function of environ- given alone. If haloperidol is then given without
mental contexts. When a specific environment (e.g., MK-801 (a novel context), the cataleptic response
the inside of an experimental chamber) is correlated is reduced to the initial response (Schmidt,
with the interoceptive stimulus effects of the drug, Tzschentke, & Kretschmer, 1999). Clearly, sensi-
then the behavioral effects of the drug may be tization to haloperidol is observed only under the
affected by the presence or absence of the context. environmental conditions in which it developed.
For example, several effects of chronic drug admin- Finally, drugs may serve as unconditioned stimuli
istration are more robust in the environmental con- in a classical conditioning context. The phenomenon
text in which they were produced. Cocaine, for of conditioned taste avoidance has received a large
example, produces an increase in locomotor activity amount of research and thought. In conditioned
that is facilitated when the drug is given over a taste avoidance, animals, rodents in particular, given
period of days. This facilitation (or sensitization) is any one of many drug types after exposure to a
greater in the environment in which the drug has novel-tasting fluid, will subsequently decrease con-
been given in the past; if cocaine is given in a novel sumption of that fluid (Garcia & Koelling, 1966).
environment, the locomotor facilitation effect is Although there are many theories about the mecha-
reduced (e.g., Post, Lockfeld, Squillace, & Contel, nism of this phenomenon, there is considerable, but
1981). Similar outcomes have been reported with not complete, consensus that the reduced intake of
some measures of tolerance to the effects of mor- the novel-tasting fluid is the result of pairing it
phine. With repeated administration of morphine, with the drug—a classical conditioning effect. The
the analgesic effects of the drug are reduced, sug- drug is the US, and the novel fluid is the CS. There
gesting tolerance (the opposite of sensitization). are several arguments for and against these designa-
However, tolerance occurs to a much greater extent tions as well as for and against consideration of the
when analgesia is tested repeatedly in the same envi- conditioned taste avoidance paradigm as classical
ronment and is less pronounced when analgesia is conditioning. This topic is far too complex to include
tested in a novel environment (Tiffany, Drobes, & in this discussion of behavioral pharmacology, but
Cepeda-Benito, 1992). one side issue is of sufficient interest to include.
Fourth, and related to the third, drugs them- That side issue is a modification of the typical
selves can function as the environmental context conditioned taste avoidance situation to include an
that modifies the action of other drugs. Haloperidol, operant aspect. Here, animals are deprived of and
for example, produces catalepsy the first time it is respond for a consummatory stimulus such as fla-
given, and this cataleptic response is facilitated vored water and are given a drug after the session.
563
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
Two studies by the same investigators (D’Mello & techniques for evaluating drug–behavior interactions
Stolerman, 1978; Stolerman & D’Mello, 1978) sug- and by chemical synthesis programs that provide
gested an interesting process underlying the find- more and more selective drugs for study using these
ings, and it warrants more thorough investigation. techniques. The ultimate goal of behavioral pharma-
In one of these studies (Stolerman & D’Mello, cology is to understand and treat human disease.
1978), water-restricted rats were trained to respond With its distinct approach, it can provide informa-
on a fixed-ratio schedule, in which a specific num- tion about drug–behavior interactions that are of
ber of responses produced water. In the second considerable value in and of themselves (e.g., identi-
study (D’Mello & Stolerman, 1978), the conditions fication of drugs that can reduce the reinforcing
were the same, but a fixed-interval schedule was effects of drugs of abuse) as well as procedures and
used. The water reinforcer was then replaced on information that can benefit neuroscience, medicinal
occasion with a distinctively flavored solution, chemistry, and human psychopharmacology. Behav-
either salty or sour. For one group, when a salty ioral pharmacology has shown itself to be a support-
solution replaced the water, the session was fol- ive member of the pharmacology community; rules
lowed by administration of 1.0 milligrams per kilo- and theories that were developed for other branches
gram amphetamine; when the sour solution replaced of pharmacology often apply well to studies of drugs
the water, the session was followed by administra- and behavior. When they do not, investigations into
tion of saline. When water was the reinforcer, no why behavioral actions of drugs differ from actions
injections were given. Only a temporary decrease measured at more molecular levels will certainly
was observed in responding maintained by the sour provide a deeper understanding of the drugs and the
solution paired with saline injections. However, diseases they were designed to treat.
responding maintained by the salty solution paired There remains a lack of underlying principles to
with amphetamine administration was disrupted define and focus research on drug–behavior interac-
after the first paired amphetamine injection and tions. Various attempts over the years to develop
declined further with additional pairings. The pat- organizing principles (Barrett, 2002; Branch, 2006;
tern of the decrease was that responding early in the Thompson & Schuster, 1968) have provoked seri-
session was unchanged, but response rates ous thought but are ultimately lacking. We hope
decreased over the course of the session. With that organizing principles may develop as behavioral
repeated amphetamine–tastant pairings, responding pharmacology continues to grow and contribute to a
decreased earlier and earlier in the session. deeper understanding of how the central nervous
As with studies of conditioned taste avoidance in system functions in health and disease.
general, arguments can be made about whether this
is a demonstration of a drug serving as US in a Pav- References
lovian sense to suppress responding in an operant Barrett, J. E. (2002). The emergence of behavioral phar-
paradigm. Whatever the theoretical interpretation, macology. Molecular Interventions, 2, 470–475.
the findings are provocative and clearly show the doi:10.1124/mi.2.8.470
power of drugs to act as stimuli. Many possibilities Bertalmio, A. J., & Woods, J. H. (1987). Differentiation
between mu and kappa receptor-mediated effects
exist for further study of the interaction of condi-
in opioid drug discrimination: Apparent pA2
tioned responses, their contingencies, and drug analysis. Journal of Pharmacology and Experimental
administration. Therapeutics, 243, 591–597.
Bertalmio, A. J., & Woods, J. H. (1989). Reinforcing
effect of alfentanil is mediated by mu opioid recep-
Conclusions and Summary tors: Apparent pA2 analysis. Journal of Pharmacology
and Experimental Therapeutics, 251, 455–460.
Behavioral pharmacology is a relatively new field,
Branch, M. N. (2006). How research in behavioral phar-
born of the need to evaluate the burgeoning number
macology informs behavioral science. Journal of
of drugs used to treat behavioral disorders and sus- the Experimental Analysis of Behavior, 85, 407–423.
tained both by increasingly powerful behavioral doi:10.1901/jeab.2006.130-04
564
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Collins, G. T., Brim, R. L., Narasimhan, D., Ko, M.-C., punishment. Life Sciences, 2, 226–231. doi:10.1016/
Sunahara, R. K., Zhan, C.-G., & Woods, J. H. (2009). 0024-3205(63)90002-X
Cocaine esterase prevents cocaine-induced toxic- Geller, I., & Seifter, J. (1960). The effects of meprobam-
ity and the ongoing intravenous self-administration ate, barbiturates, d-amphetamine and promazine
of cocaine in rats. Journal of Pharmacology and on experimentally induced conflict in the rat.
Experimental Therapeutics, 331, 445–455. doi:10.1124/ Psychopharmacology, 1, 482–492. doi:10.1007/
jpet.108.150029 BF00429273
Collins, G. T., Newman, A. H., Grundt, P., Rice, K. C., Gerak, L. R., Butelman, E. R., Woods, J. H., & France,
Husbands, S. M., Chauvignac, C., . . . Woods, J. H. C. P. (1994). Antinociceptive and respiratory
(2007). Yawning and hypothermia in rats: Effects effects of nalbuphine in rhesus monkeys. Journal of
of dopamine D3 and D2 agonists and antagonists. Pharmacology and Experimental Therapeutics, 271,
Psychopharmacology, 193, 159–170. doi:10.1007/ 993–999.
s00213-007-0766-3
Gilman, A. G. (2001). Introduction. In J. G. Hardman,
Colpaert, F. C., & Rosecrans, J. A. (1978). Stimulus prop- L. E. Limbird, & A. G. Gilman (Eds.), Goodman and
erties of drugs: Ten years of progress. Amsterdam, the Gilman’s the pharmacological basis of therapeutics
Netherlands: Elsevier/North Holland Biomedical (10th ed., pp. 1–2). New York, NY: McGraw-Hill.
Press.
Goldberg, S. R. (1980). Histamine as a punisher in squir-
Cook, L., & Catania, A. C. (1964). Effects of drugs rel monkeys: Effects of pentobarbital, chlordiaz-
on avoidance and escape behavior. Federation epoxide and H1- and H2-receptor antagonists on
Proceedings, 23, 818–835. behavior and cardiovascular responses. Journal of
Davis, T. R. A., Kensler, C. J., & Dews, P. B. (1973). Pharmacology and Experimental Therapeutics, 214,
Comparison of behavioral effects of nicotine, d- 726–736.
amphetamine, caffeine, and dimethylheptyl tetrahydro- Goodman, L. S., & Gilman, A. (1955). The pharmacologi-
cannabinol in squirrel monkeys. Psychopharmacology, cal basis of therapeutics (3rd ed.). New York, NY:
32, 51–65. doi:10.1007/BF00421707 Macmillan.
Depoortère, R., Bardin, L., Rodrigues, M., Abrial, E., Goodman, L. S., & Gilman, A. (Eds.). (1965). Goodman
Aliaga, M., & Newman-Tancredi, A. (2009). Penile and Gilman’s the pharmacological basis of therapeutics
erection and yawning induced by dopamine D2-like (3rd ed.). New York, NY: Macmillan.
receptor agonists in rats: Influence of strain and
contribution of dopamine D2, but not D3 and D4 Gormezano, I., Harvey, J. A., & Aycock, E. (1980).
receptors. Behavioural Pharmacology, 20, 303–311. Sensory and associative effects of LSD on classical
doi:10.1097/FBP.0b013e32832ec5aa appetitive conditioning of the rabbit jaw move-
ment response. Psychopharmacology, 70, 137–143.
Dews, P. B. (1958). Studies on behavior: IV. Stimulant doi:10.1007/BF00435304
actions of methamphetamine. Journal of Pharmacology
and Experimental Therapeutics, 122, 137–147. Griffiths, R. R., Brady, J., & Bradford, L. (1979).
Predicting the abuse liability of drugs with animal
D’Mello, G. D., & Stolerman, I. P. (1978). Suppression self-administration procedure: Psychomotor stimu-
of fixed-interval responding by flavor-amphetamine lants and hallucinogens. In T. Thompson & P. B.
pairings in rats. Pharmacology, Biochemistry and Dews (Eds.), Advances in behavioral pharmacology
Behavior, 9, 395–398. doi:10.1016/0091-3057(78) (pp. 163–208). New York, NY: Academic Press.
90304-0
Harvey, J. A. (2003). Role of the serotonin 5-HT2A recep-
Emmerson, P. J., Liu, M.-R., Woods, J. H., & tor in learning. Learning and Memory, 10, 355–362.
Medzihradsky, F. (1994). Binding affinity and selec- doi:10.1101/lm.60803
tivity of opioids at mu, delta and kappa receptors in
Hein, D. W., Young, A. M., Herling, S., & Woods, J. H.
monkey brain membranes. Journal of Pharmacology
(1981). Pharmacological analysis of the discrimina-
and Experimental Therapeutics, 271, 1630–1637.
tive stimulus characteristics of ethylketazocine in
Fowler, S. C., Pinkston, J. W., & Vorontsova, E. (2007). the rhesus monkey. Journal of Pharmacology and
Clorpromazine and prazosin slow the rhythm of Experimental Therapeutics, 218, 7–15.
head movements during focused stereotypy induced
Hughes, J. (1975). Isolation of an endogenous compound
by d-amphetamine in rats. Psychopharmacology, 192,
from the brain with pharmacological properties
219–230. doi:10.1007/s00213-007-0705-3
similar to morphine. Brain Research, 88, 295–308.
Garcia, J., & Koelling, R. A. (1966). Relation of cue to doi:10.1016/0006-8993(75)90391-1
consequence in avoidance learning. Psychonomic
Jaffe, J. H. (1965a). Drug addiction and drug abuse. In L.
Science, 4, 123–124.
S. Goodman & A. Gilman (Eds.), The pharmacologi-
Geller, I., Bachman, E., & Seifter, J. (1963). Effects of cal basis of therapeutics (3rd ed., pp. 285–311). New
reserpine and morphine on behavior suppressed by York, NY: Macmillan.
565
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
Winger and Woods
Jaffe, J. H. (1965b). Narcotic analgesics. In L. S. Goodman of Pharmacology and Experimental Therapeutics, 321,
& A. Gilman (Eds.), The pharmacological basis of 573–582. doi:10.1124/jpet.106.113571
therapeutics (3rd ed., pp. 247–284). New York, NY: McKernan, R. M., Rosahl, T. W., Reynolds, D. S., Sur, C.,
Macmillan. Wafford, K. A., Atack, J. R., . . . Whiting, P. J. (2000).
Jaffe, J. H. (1970). Narcotic analgesics. In L. S. Goodman Sedative but not anxiolytic properties of benzodiaz-
& A. Gilman (Eds.), The pharmacological basis of epines are mediated by the GABAA receptor α1 sub-
therapeutics (4th ed., pp. 276–313). New York, NY: type. Nature Neuroscience, 3, 587–592.
Macmillan. McMahon, L. R., & France, C. P. (2006). Differential
Jarvik, M. E. (1965). Drugs used in the treatment of psy- behavioral effects of low efficacy positive GABAA
chiatric disorders. In L. S. Goodman & A. Gilman modulators in combination with benzodiazepines
(Eds.), The pharmacological basis of therapeutics (3rd and a neuroactive steroid in rhesus monkeys. British
ed., pp. 159–214). New York, NY: Macmillan. Journal of Pharmacology, 147, 260–268. doi:10.1038/
sj.bjp.0706550
Kelleher, R. T., & Morse, W. H. (1964). Escape behavior
and punished behavior. Federation Proceedings, 23, McMillan, D. E. (1973a). Drugs and punished respond-
808–817. ing: I. Rate-dependent effects under multiple sched-
ules. Journal of the Experimental Analysis of Behavior,
Kelleher, R. T., & Morse, W. H. (1968). Determinants 19, 133–145. doi:10.1901/jeab.1973.19-133
of the specificity of the behavioral effects of drugs.
Ergebnisse der Physiologie, Biologischen Chemie und McMillan, D. E. (1973b). Drugs and punished respond-
Experimentellen Pharmakologie, 60, 1–56. ing: III. Punishment intensity as a determinant
of drug effect. Psychopharmacology, 30, 61–74.
Kenakin, T. P. (1997). Pharmacological analysis of drug- doi:10.1007/BF00422794
receptor interaction. Philadelphia, PA: Lippincott-Raven.
McMillan, D. E., & Leander, J. D. (1975). Drugs and pun-
Ko, M.-C., Butelman, E. R., Traynor, J. R., & Woods, ished responding: IV. Effects of drugs on responding
J. H. (1998). Differentiation of kappa opioid suppressed by response-dependent and response-
agonist-induced antinociception by naltrexone: independent electric shock. Archives Internationales
Apparent pA2 analysis in rhesus monkeys. Journal de Pharmacodynamie et de Therapie, 213, 22–27.
of Pharmacology and Experimental Therapeutics, 285,
Mello, N. K., & Negus, S. S. (1996). Preclinical evalua-
518–526.
tion of pharmacotherapies for treatment of cocaine
Ko, M.-C., & Naughton, N. N. (2000). An experimental and opioid abuse using drug self-administration
itch model in monkeys. Anesthesiology, 92, 795–805. procedures. Neuropsychopharmacology, 14, 375–424.
doi:10.1097/00000542-200003000-00023 doi:10.1016/0893-133X(95)00274-H
Koek, W., Woods, J. H., & Ornstein, P. (1987). A simple Morse, W. H. (1964). Effects of amobarbital and chlor-
and rapid method for assessing similarities among promazine on punished behavior in the pigeon.
directly observable behavioral effects of drugs: PCP- Psychopharmacology, 6, 286–294. doi:10.1007/
like effects of 2-amion-5-phosphonovalerate in rats. BF00413158
Psychopharmacology, 91, 297–304. doi:10.1007/ Negus, S. S., Picker, M. J., & Dykstra, L. A. (1990).
BF00518181 Interactions between mu and kappa opioid ago-
Lehr, E., Morse, W. H., & Dews, P. B. (1985). Effects of nists in the rat drug discrimination procedure.
drugs on schedule-controlled running of mice in a Psychopharmacology, 102, 465–473. doi:10.1007/
circular runway. Arzneimittelforschung, 34, 432–434. BF02247126
Mansbach, R. S., Harrod, C., Hoffman, S. M., Nader, Post, R. M., Lockfeld, A., Squillace, K. M., & Contel, N.
M., Lei, Z., Witkin, J. M., & Barrett, J. E. (1988). R. (1981). Drug-environment interaction: Context
Behavioral studies with anxiolytic drugs: V. Behavior dependency of cocaine-induced behavioral sensitiza-
and in vivo neurochemical analysis in pigeons of tion. Life Sciences, 28, 755–760. doi:10.1016/0024-
drugs that increase punished responding. Journal of 3205(81)90157-0
Pharmacology and Experimental Therapeutics, 246, Rowlett, J. K., Platt, D. M., Lelas, S., Atack, J. R., &
114–120. Dawson, G. R. (2005). Different GABAA recep-
Martelle, J. L., Claytor, R., Ross, J. T., Reboussin, B. A., tor subtypes mediate the anxiolytic, abuse-related,
Newman, A. H., & Nader, M. A. (2007). Effects and motor effects of benzodiazepine-like drugs in
of two novel D3-selective compounds, NGB 2904 primates. Proceedings of the National Academy of
[N-(4-(4-(2,3-dichlorophenyl)piperazin-1-yl)butyl)- Sciences of the United States of America, 102, 915–920.
9H-fluorene-2-carboxamide] and CJB 090 [N-(4-(4-(2,3- doi:10.1073/pnas.0405621102
dichlorophenyl)piperazin-1-yl)butyl)-4-(pyridin-2-yl) Schmidt, W. J., Tzschentke, T. M., & Kretschmer, B. D.
benzamide], on the reinforcing and discriminative (1999). State-dependent blockade of haloperidol-
stimulus effects of cocaine in rhesus monkeys. Journal induced sensitization of catalepsy by MK-801.
566
INDEX BOOKS GROUPS

INDEX BOOKS GROUPS
European Journal of Neuroscience, 11, 3365–3368. rhesus monkeys: Receptor mechanisms and tem-
doi:10.1046/j.1460-9568.1999.00794.x perature dependency. Journal of Pharmacology and
Siegel, P. S., & Sterling, T. D. (1959). The anorexi- Experimental Therapeutics, 267, 280–286.
genic action of dextro-amphetamine sulphate upon Winger, G., & Woods, J. H. (1996). Effects of buprenor-
feeding responses of differing strength. Journal of phine on behavior maintained by heroin and alfent-
Comparative and Physiological Psychology, 52, anil in rhesus monkeys. Behavioural Pharmacology, 7,
179–182. doi:10.1037/h0045670 155–159. doi:10.1097/00008877-199603000-00006
Stolerman, I. P., & D’Mello, G. D. (1978). Amphetamine- Witkin, J. M. (1994). Pharmacotherapy of cocaine
induced taste aversion demonstrated with operant abuse: Preclinical development. Neuroscience and
behavior. Pharmacology, Biochemistry and Behavior, Biobehavioral Reviews, 18, 121–142. doi:10.1016/
8, 107–111. doi:10.1016/0091-3057(78)90324-6 0149-7634(94)90042-6
Thompson, T., & Schuster, C. R. (1968). Behavioral phar- Witkin, J. M., Morrow, D., & Li, X. (2004). A rapid pun-
macology. Englewood Cliffs, NJ: Prentice-Hall. ishment procedure for detection of anxiolytic com-
Tiffany, S. T., Drobes, D. J., & Cepeda-Benito, A. (1992). pounds in mice. Psychopharmacology, 172, 52–57.
Contribution of associative and nonassociative pro- doi:10.1007/s00213-003-1618-4
cesses to the development of morphine tolerance. Wojnicki, F. H. E., & Barrett, J. E. (1993). Anticonflict
Psychopharmacology, 109, 185–190. doi:10.1007/ effects of buspirone and chlordiazepoxide in pigeons
BF02245498
under a concurrent schedule with punishment and
Walker, E. A., Butelman, E. R., DeCosta, B. R., & Woods, a changeover response. Psychopharmacology, 112,
J. H. (1993). Opioid thermal antinociception in 26–33. doi:10.1007/BF02247360
567
INDEX BOOKS GROUPS

Madden (2013) APA Handbook of Behavior Analysis (v.1)

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Madden (2013) APA Handbook of Behavior Analysis (v.1)

Uploaded by

Copyright:

Available Formats

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

APA Handbooks in Psychology

Gregory J. Madden, Editor-in-Chief

American Psychological Association • Washington, DC

INDEX BOOKS GROUPS

American Psychological Association Staff

Typeset in Berkeley by Cenveo Publisher Services, Columbia, MD

Printer: United Book Press, Baltimore, MD

Library of Congress Cataloging-in-Publication Data

APA handbook of behavior analysis / Gregory J. Madden, editor-in-chief ;

British Library Cataloguing-in-Publication Data

Printed in the United States of America

INDEX BOOKS GROUPS

Volume 1: Methods and Principles

Part II. Single-Case Research Designs����������������������������������������������������������������������������� 105

INDEX BOOKS GROUPS

Chapter 11. Time-Series Statistical Analysis of Single-Case Data���������������������������������������� 251

Part III. The Experimental Analysis of Behavior������������������������������������������������������������� 281

INDEX BOOKS GROUPS

HANDBOOK EDITORIAL MANAGER

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

About the Editor-in-Chief

Gregory J. Madden, PhD, is a professor in the Department of Psychology at Utah State

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

William B. Abernathy, PhD, Department of Psychology, Southeastern Louisiana University,

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

Stephanie Stolarz-Fantino, PhD, Department of Psychology, University of California San

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

(Original work published 1960)

INDEX BOOKS GROUPS

Murray Sidman, PhD

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

Gary R. VandenBos, PhD

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

arranging reinforcement contingencies as a means of addressing addictions. Extensive

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

Single-Case Research Methods:

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

S­ ubject 42) compared with the early test. Thorndike

INDEX BOOKS GROUPS

INDEX BOOKS GROUPS

of the single-case method, modified their apparatus A-B Design

Part II. Single-Case Research Designs�� 105

Chapter 11. Time-Series Statistical Analysis of Single-Case Data�� 251

Part III. The Experimental Analysis of Behavior�� 281

S ubject 42) compared with the early test. Thorndike

rogram for safety behaviors in two open-pit mines.

anagement interventions based on single-case