High Performance Computing: Modern

Systems and Practices - eBook PDF

Visit to download the full and correct content document:
https://ebooksecure.com/download/high-performance-computing-modern-systems-an
d-practices-ebook-pdf/
 High Performance
Computing
Modern Systems and Practices

Thomas Sterling
Matthew Anderson
Maciej Brodowicz
School of Informatics, Computing, and Engineering
Indiana University, Bloomington

Foreword by C. Gordon Bell

Acknowledgments

This textbook would not have been possible in either form or quality without the many contributions,
both direct and indirect, of a large number of friends and colleagues. It is derivative of first-year grad-
uate courses taught at both Louisiana State University (LSU) and Indiana University (IU). A number of
people contributed to these courses, including Chirag Dekate, Daniel Kogler, and Timur Gilmanov.
Amy Apon, a professor at the University of Arkansas, partnered with LSU and taught this course in
real time over the internet and helped to develop pedagogical material, including many of the exercises
used. Now at Clemson University, she continued this important contribution using her technical and
pedagogical expertise. Andrew Lumsdaine, then a professor at IU, cotaught the first version of this
course at IU. Amanda Upshaw was instrumental in the coordination of the process that resulted in
the final draft of the book, and directly developed many of the illustrations, graphics, and tables.
She was also responsible for the glossary of terms and acronyms. Her efforts are responsible in part
for the quality of this textbook.
A number of friends and colleagues provided guidance as the authors crafted early drafts of the book.
These contributions were of tremendous value, and helped improve the quality of content and form to be
useful for readers and students. David Keyes of KAUST reviewed and advised on Chapter 9 on parallel
algorithms. Jack Dongarra provided important feedback on Chapter 4 on benchmarking.
This textbook reflects decades of effort, research, development, and experience by uncounted num-
ber of contributors to the field of high performance computing. While not directly involved with the
creation of this text, many colleagues have contributed to the concepts, components, tools, methods,
and common practices associated with the broad context of high performance computing and its value.
Among these are Bill Gropp, Bill Kramer, Don Becker, Richard and Sarah Murphy, Jack Dongarra and
his many collaborators, Satoshi Matsuoka, Guang Gao, Bill Harrod, Lucy Nowell, Kathy Yelick, John
Shalf, John Salmon, and of course Gordon Bell. Thomas Sterling would like to acknowledge his thesis
advisor (at MIT) Bert Halstead for his mentorship to become the contributor that he has become.
Thomas Sterling also acknowledges Jorge Ucan, Amanda Upshaw, co-authors who made this book
possible, and especially Paul Messina who is his colleague, role model, mentor, and friend without
whom this book would never occurred. Matthew Anderson would like to thank Dayana Marvez, Oliver
Anderson, and Beltran Anderson. Maciej Brodowicz would like to thank his wife Yuko Prince Brodo-
wicz. The authors would like to thank Nate McFadden of Morgan-Kaufmann who provided enormous
effort, guidance, and patience that made this textbook possible.

DEDICATION TO PAUL MESSINA, WRITTEN BY THOMAS STERLING

The authors are pleased to dedicate this book to Dr Paul C. Messina, in acknowledgment of and grat-
itude for his exceptional contributions to and leadership in the field of high performance computing
over a career of more than 4 decades. It is impossible to capture fully the importance of his impact,
but many of the significant national programs have benefited from his guidance. Dr Messina has
been a visionary, a strategist, and a leader of programs, projects, organizations, initiatives, and, perhaps
most importantly, the careers of individual scientists who would come to deliver technical accomplish-
ments and leadership of their own. Dr Messina was the founding director of the Mathematics and
Computer Science Division at Argonne National Laboratory, a leading institution applying

xxvii
xxviii ACKNOWLEDGMENTS

high performance computing to mission-critical problem domains of the US Department of Energy

(DOE). He then founded and directed the Caltech Concurrent Supercomputing Facilities that staged
the Intel Touchstone Delta massively parallel processor, the fastest computer in the world in 1991
and the prototype of a family of massively parallel processors that determined the future direction
of high performance computing for the next 30 years. Caltech Concurrent Supercomputing Facilities
evolved into the Caltech Center for Advanced Computing Research, at which two of the authors spent
some of their most formative years. Paul was particularly instrumental and served as co-principal
investigator for the pioneering NSF Teragrid and the National Virtual Observatory. He directed the
DOE ASCI program for almost 3 years, building up the nation’s high performance computing capa-
bilities toward the leadership-scale computing it currently demonstrates. Most recently Paul Messina
led the Exascale Computing Project, America’s biggest undertaking in achieving exascale computing
performance by the beginning of the 2020s. For some Paul has had a direct and meaningful effect on
their individual careers. To author Thomas Sterling Paul has been a colleague, leader, mentor, and
friend for many years.
Morgan Kaufmann is an imprint of Elsevier
50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States
Copyright © 2018 Elsevier Inc. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or any information storage and retrieval system, without
permission in writing from the publisher. Details on how to seek permission, further information about the
Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance
Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.
This book and the individual contributions contained in it are protected under copyright by the Publisher (other
than as may be noted herein).
Notices
Knowledge and best practice in this field are constantly changing. As new research and experience broaden our
understanding, changes in research methods, professional practices, or medical treatment may become
necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using
any information, methods, compounds, or experiments described herein. In using such information or methods
they should be mindful of their own safety and the safety of others, including parties for whom they have a
professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any
liability for any injury and/or damage to persons or property as a matter of products liability, negligence or
otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the
material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress
British Library Cataloguing-in-Publication Data
A catalogue record for this book is available from the British Library
ISBN: 978-0-12-420158-3

For information on all Morgan Kaufmann publications visit

our website at https://www.elsevier.com/books-and-journals

Publisher: Katey Birtcher

Acquisition Editor: Steve Merken
Developmental Editor: Nate McFadden
Production Project Manager: Punithavathy Govindaradjane
Designer: Mark Rogers

Typeset by TNQ Books and Journals

 Dedicated to

Dr. Paul C. Messina

Leader, colleague, collaborator, mentor, friend
Foreword

High Performance Computing is a needed follow-on to Becker and Sterling’s 1994 creation of the
Beowulf clusters recipe to build scalable high performance computers (also known as a supercomputers)
from commodity hardware. Beowulf enabled groups everywhere to build their own supercomputers.
Now with hundreds of Beowulf clusters operating worldwide, this comprehensive text addresses the crit-
ical missing link of an academic course for training domain scientists and engineersdand especially
computer scientists. Competence involves knowing exactly how to create and run (e.g., controlling,
debugging, monitoring, visualizing, evolving) parallel programs on the congeries of computational
elements (cores) that constitute today’s supercomputers.
Mastery of these ever-increasing, scalable, parallel computing machines gives entry into a compar-
atively small but growing elite, and is the authors’ goal for readers of the book. Lest the reader believes
the name is unimportant: the first conference in 1988 was the ACM/IEEE Supercomputing Confer-
ence, also known as Supercomputing 88; in 2006 the name evolved to the International Conference
on High Performance Computing, Networking, Storage, and Analysis, abbreviated SCXX. About
11,000 people attended SC16.
It is hard to describe a “supercomputer,” but I know one when I see one. Personally, I never pass up
a visit to a supercomputer having seen the first one in 1961dthe UNIVAC LARC (Livermore
Advanced Research Computer) at Lawrence Livermore National Laboratory, specified by Edward
Teller to run hydrodynamic simulations for nuclear weapons design. LARC consisted of a few dozen
cabinets of densely packed circuit board interconnected with a few thousand miles of wires and a few
computational units operating at a 100 kHz rate. In 2016 the largest Sunway Light supercomputer in
China operated a trillion times faster than LARC. It consists of over 10 million processing cores oper-
ating at a 1.5 GHz rate, and consumes 15 MW. The computer is housed in four rows of 40 cabinets,
containing 256 processing nodes. A node has four interconnected 8 MB processors, controlling 64 pro-
cessing elements or cores. Thus the 10.6 million processing elements deliver 125 peak petaflops, i.e.,
160 cabinets
256 physical nodes
4 computers
(1 control þ 8
8) processing elements or cores
with a 1.31 PB memory (160
256
4
8 GB). Several of the Top 500 supercomputers have
O(10,000) computing nodes that connect and control graphic processing units (GPUs) with O(100)
cores. Today’s challenge for computational program developers is designing the architecture and
implementation of programs to utilize these megaprocessor computers.
From a user perspective, the “ideal high performance computer” has an infinitely fast clock,
executes a single instruction stream program operating on data stored in an infinitely large and fast
single-memory, and comes in any size to fit any budget or problem. In 1957 Backus established the
von Neumann programming model with Fortran. The first or “Cray” era of supercomputing from
the 1960s through the early 1990s saw the evolution of hardware to support this simple, easy-to-use
ideal by increasing processor speed, pipelining an instruction stream, processing vectors with a single
instruction, and finally adding processors for a program held in the single-memory computer. By the
early 1990s evolution of a single computer toward the ideal had stopped: clock speeds reached a few
GHz, and the number of processors accessing a single memory through interconnection was limited to
a few dozen. Still, the limited-scale, multiple-processor shared memory is likely to be the most
straightforward to program and use!

xix
xx FOREWORD

Fortunately, in the mid-1980s the “killer microprocessor” arrived, demonstrating cost effectiveness
and unlimited scaling just by interconnecting increasingly powerful computers. Unfortunately, this
multicomputer era has required abandoning both the single memory and the single sequential program
ideal of Fortran. Thus “supercomputing” has evolved from a hardware engineering design challenge of
the single (mono-memory) computer of the Seymour Cray era (1960e95) to a software engineering
design challenge of creating a program to run effectively using multicomputers. Programs first oper-
ated on 64 processing elements (1983), then 1000 elements (1987), and now 10 million (2016) pro-
cessing elements in thousands of fully distributed (mono-memory) computers in today’s
multicomputer era. So in effect, today’s high performance computing (HPC) nodes are like the super-
computers of a decade ago, as processing elements have grown 36% per year from 1000 computers in
1987 to 10 million processing elements (contained in 100,000 computer nodes).
High Performance Computing is the essential guide and reference for mastering supercomputing,
as the authors enumerate the complexity and subtleties of structuring for parallelism, creating, and
running these large parallel and distributed programs. For example, the largest climate models
simulate ocean, ice, atmosphere, and land concurrently created by a team of a dozen or more domain
scientists, computational mathematicians, and computer scientists.
Program creation includes understanding the structure of the collection of processing resources and
their interaction for different computers, from multiprocessors to multicomputers (Chapters 2 and 3),
and the various overall strategies for parallelization (Chapter 9). Other topics include synchronization
and message-passing communication among the parts of parallel programs (Chapters 7 and 8), addi-
tional libraries that form a program (Chapter 10), file systems (Chapter 18), long-term mass storage
(Chapter 17), and components for the visualization of results (Chapter 12). Standard benchmarks for
a system give an indication of how well your parallel program is likely to run (Chapter 4). Chapters 16
and 17 introduce and describe the techniques for controlling accelerators and special hardware cores,
especially GPUs, attached to nodes to provide an extra two orders of magnitude more processing per
node. These attachments are an alternative to the vector processing units of the Cray era, and typified
by the Compute Unified Device Architecture, or CUDA, model and standard to encapsulate
parallelism across different accelerators.
Unlike the creation, debugging, and execution of programs that run interactively on a personal
computer, smartphone, or within a browser, supercomputer programs are submitted via batch process-
ing control. Running a program requires specifying to the computer the resources and conditions for
controlling your program with batch control languages and commands (Chapter 5), getting the pro-
gram into a reliable and dependable state through debugging (Chapter 14), checkpointing, i.e., saving
intermediate results on a timely basis as insurance for the computational investment (Chapter 20), and
evolving and enhancing a program’s efficacy through performance monitoring (Chapter 13).
Chapter 21 concludes with a forward look at the problems and alternatives for moving supercom-
puters and the ability to use them to petascale and beyond. In fact, the only part of HPC not described
in this book is the incredible teamwork and evolution of team sizes for writing and managing HPC
codes. However, the most critical aspect of teamwork resides with the competence of the individual
members. This book is your guide.

Gordon Bell
October 2017
Index
‘Note: Page numbers followed by “f” indicate figures, “t” indicate tables, and “b” indicate boxes.’

Anatomy of supercomputer, 14e16

A computer performance, 16e21
Accelerator architecture, 451e453
peak performance, 17e18
evolution of graphics processing unit functionality, 466e471,
performance, 16
468fe469f, 470t
performance degradation, 19e20
graphics processing units, 464e466, 465f
performance improvement, 20e21
heterogeneous system architecture, 477e478
scaling, 18e19
historic perspective, 454e463
sustained performance, 18
accelerators in processor I/O space, 461
Antikythera mechanism, 53f
accelerators with industry-standard interfaces, 462e463
Application profiling, significance of, 390e391
coprocessors, 456e461
Application programming, 4, 8e9
Intel 8087, 457e459
Application-level checkpointing, 598e601
Motorola MC68881, 459e461
Architecture overview, essential SLURM, 147, 147f
modern graphics processing unit architecture, 471e477,
Arithmetic logic unit (ALU), 63e64, 70
472t
Arithmetic on variables, Linux, 654
compute architecture, 471e474, 474f
Asynchronous multitasking, 613e615
interconnects, 475e476
actor synchronization, 615
memory implementation, 474e475
global address space, 614e615
programming environment, 476e477
message-driven computation, 613e614
outcomes, 480
multithreaded, 613
Accelerators
runtime system software, 615
with industry-standard interfaces, 462e463
Atomics, 504e505
in processor I/O space, 461
Automated calculators through mechanical technologies,
Actor synchronization, 615
22e23
Advection equation using finite difference, 295e297,
296fe298f
Aggregated storage, 534e544 B
network attached storage, 543e544 Backus, John, 317
redundant array of independent disks, 532f, 534e541 Barrier directive, OpenMP programming model, 243
hybrid redundant array of independent disks variants, Basic linear algebra subprograms (BLAS), 317e322
539e541 Level 1 rotation operations, 320t
RAID 0: striping, 534e535 Level 1 vector operations, 320te321t
RAID 1: mirroring, 535 Levels 2 and 3, 321te322t
RAID 2: bit-level striping with hamming code, 536 routines, precision prefixes used by, 319t
RAID 3: byte-level striping with dedicated parity, Basic methods of use, commodity clusters, 104e113
536e537 compilers and compiling, 112e113
RAID 4: block-level striping with dedicated parity, 537 logging on, 104e105
RAID 5: block-level striping with single distributed package configuration and building, 110e111
parity, 538 running applications, 113
RAID 6: block-level striping with dual distributed parity, user space and directory system, 105e110
539 Basic two-input logic gates, 56f
storage area networks, 541e542 Benchmarking
tertiary storage, 544, 544f, 545t benchmarks used in HPC community, 118t
Allgather, 272e274 Graph500, 132e135
Alltoall, 277e278, 277f highly parallel computing linpack, 120e123
Amdahl’s law, 70e73 high performance conjugate gradients, 126e128
Amdahl’s law plus, 196e199, 197fe199f HPC challenge benchmark suite, 123e124

677
678 INDEX

Benchmarking (Continued ) Clearspeed Advance e710 accelerator board, 463f

key properties of, 117e120 Climate change, understanding of, 12e14
miniapplications as benchmarks, 135e138 CMOS, 55
NAS parallel benchmarks, 130e131 Collective data movement, 265e267, 265fe267f
nonproprietary, 119t Command substitution, Linux, 654
outcomes, 138e139 Command-line utilities, Linux, 659e663
overview, 115e117 data compression and archiving, 662e663
standard HPC community benchmarks, 120 process management, 661e662
top-performing supercomputer, 119t text tools, 659e661
Beowulf class of parallel computer architecture, 77f Commands, SLURM scheduling, 151e166
Beowulf cluster project, 91e93 sacct, 164e165
Bit-level striping with hamming code (RAID 2), 536 salloc, 160e161
Block-level striping sbatch, 161e162
with dedicated parity (RAID 4), 537 scancel, 163e164
with dual distributed parity (RAID 6), 539 sinfo, 165e166
with single distributed parity (RAID 5), 538 squeue, 162e163
Branch prediction, symmetric multiprocessor architecture, srun, 151e160
201e202 Commercial parallel debuggers, 431e432
Breadth first search, parallel algorithms, 306e310, 307fe309f Commercial systems summary, commodity clusters, 95
Brief history of supercomputing, 21e38 Commodity clusters
automated calculators through mechanical technologies, basic methods of use, 104e113
22e23 compilers and compiling, 112e113
communicating sequential processors and very large scale logging on, 104e105
integration, 34e37 package configuration and building, 110e111
instruction-level parallelism, 29e30 running applications, 113
multicore petaflops, 37 user space and directory system, 105e110
neodigital age and beyond Moore’s law, 37e38 Beowulf cluster project, 91e93
single-instruction multiple data array, 33 cluster elements, 85e86
vector processing and integration, 30e33 definition of, 84
von Neumann architecture in vacuum tubes, 24e29 hardware architecture, 93e95
Broadcast, 268e269 commercial systems summary, 95
Buffer flush, 560 node, 93e94
Buffered File I/O, 559e562 secondary storage, 95
buffer flush, 560 system area networks, 94e95
conversion between streams and file descriptors, 561e562 history, 88e90
file open and close, 559 HPC architecture, 77e78
offset update and query, 560 impact on top 500 list, 86e87
sequential data access, 560 motivation and justification for clusters,
Bush, Vannevar, 22e23 84e85
Byte-level striping with dedicated parity (RAID 3), 536e537 outcomes, 113e114
overview, 84e90
programming interfaces, 97e98
C
high performance computing programming languages,
C++ AMP, 486
97
Cannon’s algorithm, 301e303, 301fe307f
parallel programming modalities, 97e98
Catalyzing fraud detection, 10
software environment, 98e104
CDC-7600, 318f
operating systems, 98e99
Cellular automata, 618e619
resource management, 99e100
Checkpointing, 591
debugger, 101
application-level, 598e601
performance profiling, 101
outcomes, 602
visualization, 101e104
system-level, 592e597
Communicating sequential processors and, 34e37
Checkpoint/restart, 47e48
 INDEX 679

Communication collectives, 265e278 GDB cheat sheet, 430

allgather, 272e274 setting a variable, 428e430, 430f
alltoall, 277e278, 277f threads, 430, 431fe432f
broadcast, 268e269 watch points and catch points, 425e426, 427f
collective data movement, 265e267, Valgrind, 430e431, 433t
265fe267f unprotected shared variable assessment, 433e434, 436f
gather, 271e272 Digital logic, HPC architecture, 55e58
reduction operations, 274e276, 274t Directives, OpenMP programming model, 230e231
scatter, 269e271 Discovering oil and gas, 10
Communicators, 255e258 Distributed computation, MapReduce, 584
example, 257e258 Distributing oil and gas, 10
rank, 256e257 Divide and conquer, parallel algorithms, 287e291,
size, 256 288fe291f
Compilation, Linux, 656e659 Domain-specific languages for linear algebra, 329, 330fe332f
Compiler flags for debugging, 439 Dongarra, Jack, 116b
Compilers and compiling, commodity clusters, 112e113 Dynamic random access memory (DRAM), 45, 61e62
Compute architecture, 471e474, 474f
Compute unified device architecture, 485 E
Computer performance, 16e21
Earth Simulator (ES), 64b
Concurrent applications, SLURM job scripting, 167e169
Efficiency, HPC architecture, 46
Connection Machine 2 (CM-2), 454e463
Eigenvalue SoLvers for Petaflop-applications, 328
Connection Machine 5 (CM-5), 252f
Elastic computing, SLURM scheduling, 150
Control Data Corporation (CDC) 6600, 51e55
Electronic numerical integrator and computer (ENIAC), 51f
Control flow, Linux, 654e656
Emacs, 645e646
Conversion between streams and file descriptors, 561e562
Embarrassingly parallel, parallel algorithms, 292e293, 293f
Coprocessors, 456e461
Enabling technology, HPC architecture, 51e62
Intel 8087, 457e459
digital logic, 55e58
Motorola MC68881, 459e461
memory technologies, 58e62
Cray, Seymour, 30e33
early memory devices, 59e61
Cray-1 Supercomputer, 32f
modern memory technologies, 61e62
Critical synchronization directive, OpenMP programming
roles of technologies, 55
model, 242
technology epochs, 51e55
Environment variables
D OpenMP programming model, 229e230
Data access with explicit offset, 556 SLURM job scripting, 169e171
Data compression and archiving, Linux, 662e663 Environment variables of interest, 185e186
Data management, 497e501 Essential bash, Linux, 647e656
Data reuse and locality, symmetric multiprocessor arithmetic on variables, 654
architecture, 204e205 command substitution, 654
Dataflow, 617e618 control flow, 654e656
Deadlock, 434e439 input/output, 649e650
Debugger, commodity clusters, 101 path expansion, 647e648
Debugging, 421e422 pipelines, 649e650
compiler flags for debugging, 439 redirection, 649e650
debugging MPI example, deadlock, 434e439 special character handling, 649
outcomes, 445 variables, 650e653
system monitors to aid debugging, 441e445 Essential gperftools, 391e398
tools, 423e432 Essential Linux
commercial parallel debuggers, 431e432 command-line utilities, 659e663
GNU debugger, 423e430 data compression and archiving, 662e663
back trace, 427e428, 428fe429f process management, 661e662
break points, 424e425, 425f, 425t, 426f text tools, 659e661
680 INDEX

Essential Linux (Continued ) overview of collective calls, 262e263

compilation, 656e659 user-defined data types, 281e283
essential bash, 647e656 Essential OpenMP, 225e226
arithmetic on variables, 654 outcomes, 245e246
command substitution, 654 overview of OpenMP programming model, 226e231
control flow, 654e656 runtime library and environment variables, 228e231
input/output, 649e650 directives, 230e231
path expansion, 647e648 environment variables, 229e230
pipelines, 649e650 runtime library routines, 230
redirection, 649e650 thread parallelism, 226e228
special character handling, 649 thread variables, 228
variables, 650e653 parallel threads and loops, 231e240
files editing, 645e647 parallel “for”, 233e238
Emacs, 645e646 parallel threads, 231e232
Gedit, 646 private, 232e233
Kate, 646 sections, 239e240
Nano, 646 reduction, 244
Vi, 645 synchronization, 241e243
logging In, 637e638 barrier directive, 243
navigating the file system, 641e645 critical synchronization directive, 242
remote access, 639e640 master directive, 242e243
Essential MPI, 250 single directive, 243
communication collectives, 265e278 Essential portable batch system, 172e187
allgather, 272e274 PBS cheat sheet, 186e187
alltoall, 277e278, 277f PBS commands, 174e183
broadcast, 268e269 pbsnodes, 183
collective data movement, 265e267, qdel, 180
265fe267f qstat, 180e182
gather, 271e272 qsub, 174e179
reduction operations, 274e276, 274t tracejob, 182e183
scatter, 269e271 PBS job scripting, 184e186
communicators, 255e258 environment variables of interest, 185e186
example, 257e258 MPI jobs, 185
rank, 256e257 OpenMP jobs, 184e185
size, 256 portable batch system architecture, 173e174, 173f
message-passing interface basics, 253e255 portable batch system overview, 172e173
message-passing interface example, 254e255 Essential POSIX file interface, 554e562
MPI_Finalize, 254 buffered File I/O, 559e562
mpi.h, 253 buffer flush, 560
MPI_Init, 253 conversion between streams and file descriptors,
message-passing interface standards, 251e253 561e562
nonblocking point-to-point communication, file open and close, 559
279e281 offset update and query, 560
outcomes, 283 sequential data access, 560
point-to-point messages, 258e262 system calls for file access, 554e558
example, 260e262 data access with explicit offset, 556
message-passing interface data types, 259, 260t file length adjustment, 556e557
MPI Recv, 259e260 file offset manipulation, 556
MPI send, 259 file open and close, 554e555
synchronization collectives, 262e265 file status query, 557e558
barrier synchronization, 263, 263f sequential data access, 555e556
example, 264e265 synchronization with storage device, 557
 INDEX 681

Essential resource management SLURM cheat sheet, 171e172

essential portable batch system, 172e187 SLURM job scripting, 166e171
PBS cheat sheet, 186e187 concurrent applications, 167e169
PBS Commands, 174e183 environment variables, 169e171
pbsnodes, 183 MPI scripts, 167
qdel, 180 OpenMP scripts, 167
qstat, 180e182 script components, 166e167
qsub, 174e179 SLURM scheduling, 149e150
tracejob, 182e183 elastic computing, 150
PBS job scripting, 184e186 gang scheduling, 149
environment variables of interest, 185e186 generic resources, 150
MPI jobs, 185 high-throughput computing, 150
OpenMP jobs, 184e185 preemption, 149e150
portable batch system architecture, 173e174, 173f trackable resources, 150
portable batch system overview, 172e173 workload organization, 148, 148f
essential SLURM, 146e172 Ethernet, 213e214
architecture overview, 147, 147f Exascale computing, 610e612
commands, 151e166 accelerated approach, 612
sacct, 164e165 capacity, 611e612
salloc, 160e161 challenges to, 611
sbatch, 161e162 lightweight cores, 612
scancel, 163e164 Execution pipeline, symmetric multiprocessor architecture,
sinfo, 165e166 200e201
squeue, 162e163 Expanded parallel programming models,
srun, 151e160 606e607
SLURM cheat sheet, 171e172 advance in message-passing interface,
SLURM job scripting, 166e171 606e607
concurrent applications, 167e169 advances in OpenMP, 607
environment variables, 169e171 MPI+X, 607
MPI scripts, 167 Extended high performance computing architecture,
OpenMP scripts, 167 608e609
script components, 166e167 field programmable gate arrays, 609
SLURM scheduling, 149e150 lightweight architectures, 608e609
elastic computing, 150 world’s fastest machine, 608
gang scheduling, 149 External I/O interfaces, symmetric multiprocessor
generic resources, 150 architecture, 213e222
high-throughput computing, 150 JTAG, 218e219
preemption, 149e150 network interface controllers, 213e215
trackable resources, 150 Ethernet, 213e214
workload organization, 148, 148f InfiniBand, 215
managing resources, 142e146 serial advanced technology attachment,
outcomes, 187e189 215e218
Essential SLURM, 146e172 universal serial bus, 220e222
architecture overview, 147, 147f
commands, 151e166 F
sacct, 164e165 Fast Fourier transform (FFT), 628e631
salloc, 160e161 Field programmable gate arrays, 609
sbatch, 161e162 File length adjustment, 556e557
scancel, 163e164 File management, 350
sinfo, 165e166 File offset manipulation, 556
squeue, 162e163 File open and close, 554e555, 559
srun, 151e160 File status query, 557e558
682 INDEX

File systems, 86 GNU scientific library, 326, 327f

essential POSIX file interface, 554e562 Gnuplot, 365e368, 366fe370f
buffered File I/O, 559e562 gperftools, 391e398
buffer flush, 560 Graph algorithms, 329
conversion between streams and file descriptors, Graph500, 132e135, 133fe134f, 134t, 135fe137f
561e562 Graphics processing units, 464e466, 465f
file open and close, 559 functionality, evolution of, 466e471, 468fe469f,
offset update and query, 560 470t
sequential data access, 560 modern architecture, 471e477, 472t
system calls for file access, 554e558 compute architecture, 471e474, 474f
data access with explicit offset, 556 interconnects, 475e476
file length adjustment, 556e557 memory implementation, 474e475
file offset manipulation, 556 programming environment, 476e477
file open and close, 554e555 Gropp, William D., 251f
file status query, 557e558 Gyrokinetic Toroidal code, 9f
sequential data access, 555e556
synchronization with storage device, 557 H
general parallel file system, 565e569, 566f, 566t
Hadoop, 585e588
Lustre file system, 569e575
Halo exchange, parallel algorithms, 294e299, 294f
network file system, 562e565, 563f
advection equation using finite difference, 295e297,
outcomes, 575e576
296fe298f
role and function, 549e553
sparse matrix vector multiplication, 297e299,
Files editing, Linux, 645e647
299fe300f
Emacs, 645e646
Hard-disk drives, 514e520, 515f, 517f, 519t
Gedit, 646
Hardware architecture, commodity clusters, 93e95
Kate, 646
commercial systems summary, 95
Nano, 646
node, 93e94
Vi, 645
secondary storage, 95
Flops, 46
system area networks, 94e95
Flynn’s taxonomy, 48e50, 50f
Hardware fault, 47e48
Forkejoin model
Heterogeneous computer structures, 78
master/worker threads, 228f
Heterogeneous system architecture, 477e478
parallel algorithms, 286e287, 287fe288f
High performance computing
Forwarding, symmetric multiprocessor architecture, 202
computational science research areas, 314f
Foundational visualization concepts, 364
disciplines, 3e9
application programming, 8e9
G application programs, 4
Gang scheduling, SLURM scheduling, 149 definition, 3e4
Gara, Alan, 610e612 high performance computing systems, 5e7
Gather, 271e272 performance and metrics, 4e5
Gedit, 646 supercomputing problems, 7e8
General parallel file system, 565e569, 566f, 566t programming languages, commodity clusters, 97
Generic resources, SLURM scheduling, 150 systems, 5e7, 7f
Gigabit Ethernet High performance linpack, 120e123, 122f
network interface card, 214f High performance conjugate gradients, 126e128
switch, 214f High throughput computing, SLURM scheduling, 150
Global address space, 614e615 Historic perspective, of accelerator architecture, 454e463
GNU debugger, 423e430 accelerators in processor I/O space, 461
back trace, 427e428, 428fe429f accelerators with industry-standard interfaces,
break points, 424e425, 425f, 425t, 426f 462e463
GDB cheat sheet, 430 coprocessors, 456e461
setting a variable, 428e430, 430f Intel 8087, 457e459
threads, 430, 431fe432f Motorola MC68881, 459e461
watch points and catch points, 425e426, 427f Hopper, Grace Brewster Murray, 421f, 423f
 INDEX 683

Host node, 86 Interconnects, 475e476

HPC architecture I/O system management, 351
enabling technology, 51e62 Isosurfaces, 364, 365f
digital logic, 55e58
memory technologies, 58e62 J
early memory devices, 59e61
JTAG, 218e219, 219f
modern memory technologies, 61e62
roles of technologies, 55
technology epochs, 51e55 K
heterogeneous computer structures, 78 Kate, 646
key properties, 44e48 Kernels construct, 495e496
efficiency, 46 Key properties, HPC architecture, 44e48
parallelism, 45e46 efficiency, 46
power, 46e47 parallelism, 45e46
programmability, 48 power, 46e47
reliability, 47e48 programmability, 48
speed, 45 reliability, 47e48
multiprocessors, 73e78 speed, 45
commodity clusters, 77e78 K-means clustering, 582e584
massively parallel processors, 76
shared-memory multiprocessors, 74e76 L
outcomes, 78e79 Lapack, 324, 325f
overview, 44 driver routines, 324t
parallel architecture families, Flynn’s taxonomy, 48e50 Libraries, 313e315, 315t
single-instruction, multiple data array, 69e73 graph algorithms, 329
Amdahl’s law, 70e73 linear algebra, 315e329, 316f
single-instruction, multiple data architecture, 69e70 basic subprograms, 317e322
vector and pipelining, 64e68 domain-specific languages for, 329, 330fe332f
pipeline parallelism, 65e68 eigenvalue SoLvers for Petaflop-applications, 328
vector processing, 68 GNU scientific library, 326
von Neumann sequential processors, 62e64 Hypre, scalable linear solvers and multigrid methods,
HPC challenge benchmark suite, 123e124 328e329
Hybrid redundant array of independent disks variants, package, 324e326
539e541 portable extensible toolkit for scientific computation,
Hypre, scalable linear solvers and multigrid methods, 327e328
328e329 scalable library for eigenvalue problem computations,
328
I scalable linear algebra, 326
IBM Blue Gene supercomputers, 610e612 supernodal LU, 326e327
ILLIAC (Illinois Automatic Computer), 454e463 mesh decomposition, 333e334, 340f
Independent disks variants, hybrid redundant array of, outcomes, 342e343
539e541 parallel input/output, 330e332, 335f
InfiniBand, 215 parallelization, 334
port, 216f partial differential equations, 329
Input/output , Linux, 649e650 performance monitoring, 341e342
Instruction interface, 70 signal processing, 334e341
Instruction set architecture (ISA), 44, 54 visualization, 334
Instruction-level parallelism, 29e30 Lightweight architectures, 608e609
symmetric multiprocessor architecture, 201 Linear algebra, 315e329, 316f
Integrated performance monitoring toolkits, 407e410, 410f basic subprograms, 317e322
Intel 8087, 457e459, 457fe458f domain-specific languages for, 329, 330fe332f
Intel Touchstone Delta, 35e36 eigenvalue SoLvers for Petaflop-applications, 328
684 INDEX

Linear algebra (Continued ) storage area networks, 541e542

GNU scientific library, 326 tertiary storage, 544, 544f, 545t
Hypre, scalable linear solvers and multigrid methods, history, 512, 513f
328e329 outcomes, 545e546
package, 324e326 storage device technology, 514e532
portable extensible toolkit for scientific computation, hard-disk drives, 514e520, 515f, 517f, 519t
327e328 magnetic tape, 524e528, 527fe528f, 529t
scalable library for eigenvalue problem computations, 328 optical storage, 529e532, 530f
scalable linear algebra, 326 solid-state drive storage, 520e524, 521fe522f, 525t,
supernodal LU, 326e327 529t
Linpack, 116b, 120e123 Massively parallel processor (MPP), 55, 76f
Local registers, 70 HPC architecture, 76
Logging in, Linux, 637e638 Master directive, OpenMP programming model, 242e243
Logging on, commodity clusters, 104e105 Matplotlib, 369e372, 371fe372f
Loop scheduling, 501e504 Mellanox IB cards, 215f
Lower/upper decomposition, 624e628 Memory block, 70
Lustre file system, 569e575 Memory hierarchy, symmetric multiprocessor architecture,
204e209, 204f
data reuse and locality, 204e205
M
memory system performance, 207e209
Magnetic tape, 524e528, 527fe528f, 529t
Memory implementation, 474e475
Magnetism, 60e61
Memory management, 350, 358e361
Makefiles, 111
virtual address translation, 359e361
Managereworker, parallel algorithms, 291e292, 292f
virtual memory, 359
Managing oil and gas, 10
virtual page addresses, 359
Managing resources, 142e146
Memory system performance, symmetric multiprocessor
Manufacturing, accelerating innovation in, 10e11
architecture, 207e209
MapReduce, 579
Memory technologies, HPC architecture, 58e62
distributed computation, 584
early memory devices, 59e61
Hadoop, 585e588
modern memory technologies, 61e62
Map and Reduce, 579e584
Mesh decomposition, 333e334, 340f
K-means clustering, 582e584
Mesh tessellation, 364, 366f
shared neighbors, 581e582
Message-driven computation, 613e614
word count, 581
Message passing interface (MPI), 253e255
outcomes, 588e589
advance in, 606e607
Market data analytics, 10
example, 254e255
Mass storage, 509e512
jobs, 185
aggregated storage, 534e544
message passing interface example, 254e255
network attached storage, 543e544
MPI_Finalize, 254
redundant array of independent disks, 532f, 534e541
mpi.h, 253
hybrid redundant array of independent disks variants,
MPI_Init, 253
539e541
scripts, SLURM job scripting, 167
RAID 0: striping, 534e535
standards, 251e253
RAID 1: mirroring, 535
Metrics, 4e5
RAID 2: bit-level striping with hamming code, 536
Miniapplications as benchmarks, 135e138, 137t
RAID 3: byte-level striping with dedicated parity,
Mirroring (RAID 1), 535
536e537
Modern graphics processing unit architecture, 471e477,
RAID 4: block-level striping with dedicated parity,
472t
537
compute architecture, 471e474, 474f
RAID 5: block-level striping with single distributed
interconnects, 475e476
parity, 538
memory implementation, 474e475
RAID 6: block-level striping with dual distributed
programming environment, 476e477
parity, 539
 INDEX 685

Monitoring hardware events, 398e407 parallel construct, 493e495

perf, 398e404 variable scope, 504
performance application programming interface, 404e407 environment variables, 491e492
Moore, Gordon, 17 library calls, 489e491
Moore’s law, 37e38 programming concepts, 487e489
Motivation and justification, for clusters, 84e85 OpenCL, 485e486
Motorola MC68881, 459e461, 459f OpenMP
MPI_Finalize, 254 advances in, 607
mpi.h, 253 jobs, 184e185
MPI_Init, 253 programming model, 226e231
MPI+X, 607 runtime library and environment variables,
Multicore petaflops, 37 228e231
Multigrid methods, 328e329 directives, 230e231
Multiprocessors, HPC architecture, 73e78 environment variables, 229e230
commodity clusters, 77e78 runtime library routines, 230
massively parallel processors, 76 thread parallelism, 226e228
shared-memory multiprocessors, 74e76, 74f thread variables, 228
Multithreading, 613 scripts, SLURM job scripting, 167
symmetric multiprocessor architecture, 203 Operating systems, 347e349, 348f
commodity clusters, 98e99
N memory management, 358e361
virtual address translation, 359e361
Nano, 646
virtual memory, 359
NAS parallel benchmarks, 130e131, 130t
virtual page addresses, 359
Navigating the file system, Linux, 641e645
outcomes, 361
Neodigital age, 37e38, 616e620
process management, 351e357
cellular automata, 618e619
process control block, 353e354
dataflow, 617e618
process management activities,
neuromorphic, 619
354e355
quantum computing, 619e620
process states, 352e353
Nested parallel threads, 229f
scheduling, 355e357, 356f
Network attached storage, 543e544
structures and services, 349e351
Network file system, 562e565, 563f
file management, 350
Network interface controllers, 213e215
I/O system management, 351
Ethernet, 213e214
memory management, 350
InfiniBand, 215
process management, 349e350
Neuromorphic computing, 619
secondary storage management, 351
Node, commodity clusters, 93e94
system components, 349
Nonblocking point-to-point communication, 279e281
threads, 357, 358f
Nonuniform memory access (NUMA), 75e76
Optical storage, 529e532, 530f
architectures, 75f
Numerical integration, 621e624
P
O Package configuration and building, commodity clusters,
Offset update and query, 560 110e111
OpenACC, 483e487, 488f Parallel algorithms
C++ AMP, 486 divide and conquer, 287e291, 288fe291f
compute unified device architecture, 485 embarrassingly parallel, 292e293, 293f
directives, 492e505 forkejoin, 286e287, 287fe288f
atomics, 504e505 generic classes of, 286t
data management, 497e501 halo exchange, 294e299, 294f
Kernels construct, 495e496 advection equation using finite difference, 295e297,
loop scheduling, 501e504 296fe298f
686 INDEX

Parallel algorithms (Continued ) commodity clusters, 101

sparse matrix vector multiplication, 297e299, essential gperftools, 391e398
299fe300f significance of application profiling, 390e391
managereworker, 291e292, 292f Permutation: Cannon’s algorithm, parallel algorithms,
outcomes, 310e311 301e303, 301fe307f
overview, 285e286 Personalized medicine and drug discovery, 11
permutation: Cannon’s algorithm, 301e303, 301fe307f Petaflops
task dataflow: breadth first search, 306e310, 307fe309f applications, 328
Parallel architecture families, 48e50 multicore, 37
Parallel construct, 493e495 Pipeline parallelism, HPC architecture, 65e68, 67f
Parallel “for,” OpenMP programming model, 233e238 Pipelines, Linux, 649e650
Parallel input/output, 330e332, 335f Point-to-point messages, 258e262
Parallel programming modalities, commodity clusters, 97e98 example, 260e262
Parallel threads/loops, OpenMP programming model, message-passing interface data types, 259, 260t
231e240 MPI Recv, 259e260
parallel “for”, 233e238 MPI send, 259
parallel threads, 231e232 Portable batch system (PBS)
private, 232e233 architecture, 173e174, 173f
sections, 239e240 cheat sheet, 186e187
Parallelism, HPC architecture, 45e46 commands, 174e183
Parallelization, 334 pbsnodes, 183
ParaView, 379, 379f qdel, 180
Partial differential equations, 329 qstat, 180e182
Pascal GP100 architecture, 473f job status query, 180e181
Pascaline mechanical calculator, 53f queue status query, 181
Path expansion, Linux, 647e648 server status query, 182
pbsnodes, 183 qsub, 174e179
PCI bus, 209e213, 210f tracejob, 182e183
connectors supporting, 211f job scripting, 184e186
PCI express (PCIe), 210e212, 211f environment variables of interest, 185e186
slots, 212f MPI jobs, 185
Peak performance, 17e18 OpenMP jobs, 184e185
perf, 398e404 overview, 172e173
Performance, 4e5, 16 Portable extensible toolkit for scientific computation,
degradation, 19e20 327e328, 328t
improvement, 20e21 Power, HPC architecture, 46e47
monitoring, 341e342, 383e385 Predicting natural disasters, 12e14
integrated performance monitoring toolkits, 407e410, Preemption, SLURM scheduling, 149e150
410f Private, OpenMP programming model, 232e233
monitoring hardware events, 398e407 Process control block, 353e354
perf, 398e404 Process management, 349e357
performance application programming interface, activities, 354e355
404e407 Linux, 661e662
outcomes, 417e418 process control block, 353e354
performance profiling, 390e398 process management activities, 354e355
essential gperftools, 391e398 process states, 352e353
significance of application profiling, 390e391 scheduling, 355e357, 356f
profiling in distributed environments, 411e417, Process states, 352e353
414fe415f, 417f Processor core architecture, symmetric multiprocessor
time measurement, 385e390 architecture, 199e203, 200t
Performance application programming interface, 404e407 branch prediction, 201e202
Performance profiling, 390e398 execution pipeline, 200e201
 INDEX 687

forwarding, 202 S
instruction-level parallelism, 201 sacct, 164e165
multithreading, 203 salloc, 160e161
reservation stations, 202 sbatch, 161e162
Profiling in distributed environments, 411e417, 414fe415f, Scalable library for eigenvalue problem computations, 328
417f Scalable linear algebra, 326
Programmability, HPC architecture, 48 Scalable linear solvers, 328e329
Programming Scaling, 18e19
concepts, 487e489 scancel, 163e164
environment, 476e477 Scatter, 269e271
interfaces, commodity clusters, 97e98 Scheduling, 355e357, 356f
high performance computing programming languages, Science, supercomputing impact on, 10e14
97 Scott, Steven, 35
parallel programming modalities, 97e98 Script components, SLURM job scripting, 166e167
Secondary storage
Q commodity clusters, 95
qdel, 180 management, 351
qstat, 180e182 Sections, OpenMP programming model, 239e240
job status query, 180e181 Security, supercomputing impact on, 10e14
queue status query, 181 Sequencer controller, 70
server status query, 182 Sequential data access, 555e556, 560
qsub, 174e179 Serial advanced technology attachment (SATA), 215e218
Quantum computing, 619e620 connectors, 216f
interface variants, 218f
Serial quicksort algorithm, 289f
R Shannon, Claude Elwood, 23
Redirection, Linux, 649e650
Shared neighbors, 581e582
Reduction
Shared-memory multiprocessors, 227f
OpenMP programming model, 244
HPC architecture, 74e76, 74f
operations, 274e276, 274t
Signal processing, 334e341
Redundant array of independent disks, 532f, 534e541
SIMD array class of parallel computer architecture, 69f
hybrid redundant array of independent disks variants,
Simple Linux Utility for Resource Management (SLURM)
539e541
cheat sheet, 171e172
RAID 0: striping, 534e535
job scripting, 166e171
RAID 1: mirroring, 535
concurrent applications, 167e169
RAID 2: bit-level striping with hamming code, 536
environment variables, 169e171
RAID 3: byte-level striping with dedicated parity, 536e537
MPI scripts, 167
RAID 4: block-level striping with dedicated parity, 537
OpenMP scripts, 167
RAID 5: block-level striping with single distributed parity,
script components, 166e167
538
scheduling, 149e150
RAID 6: block-level striping with dual distributed parity,
elastic computing, 150
539
gang scheduling, 149
Reliability, HPC architecture, 47e48
generic resources, 150
Remote access, Linux, 639e640
high-throughput computing, 150
Reservation stations, symmetric multiprocessor architecture,
preemption, 149e150
202
trackable resources, 150
Resource management, commodity clusters, 99e100
sinfo, 165e166
debugger, 101
Single directive, OpenMP programming model, 243
performance profiling, 101
Single-bit full adder, 66f
visualization, 101e104
Single-instruction
Running applications, commodity clusters, 113
multiple data architecture, 69e70
Runtime library routines, OpenMP programming model, 230
multiple data array, 33, 69e73
Runtime system software, 615
688 INDEX
Single-instruction (Continued )
Amdahl’s law, 70e73
single-instruction, multiple data architecture, 69e70
Society, supercomputing impact on, 10e14
Software environment, commodity clusters, 98e104
operating systems, 98e99
resource management, 99e100
debugger, 101
performance profiling, 101
visualization, 101e104
Solid-state drive storage, 520e524, 521fe522f, 525t, 529t
SoLvers for Petaflop-applications, 328
Sparse matrix vector multiplication, 297e299, 299fe300f
Special character handling, Linux, 649
Speed, HPC architecture, 45
squeue, 162e163
srun, 151e160
Standard HPC community benchmarks, 120
Static random access memory (SRAM), 45, 61
Storage area networks, 541e542
Storage device technology, 514e532
hard-disk drives, 514e520, 515f, 517f, 519t
magnetic tape, 524e528, 527fe528f, 529t
optical storage, 529e532, 530f
solid-state drive storage, 520e524, 521fe522f, 525t, 529t
Streamlines, 364, 364f
Streams and file descriptors, conversion between, 561e562
Striping (RAID 0), 534e535
Structures and services, 349e351
file management, 350
I/O system management, 351
memory management, 350
process management, 349e350
secondary storage management, 351
system components, 349
Supercomputing, impact on science, society, and security,
10e14
accelerating innovation in manufacturing, 10e11
catalyzing fraud detection and market data analytics, 10
discovering oil and gas, 10
distributing oil and gas, 10
managing oil and gas, 10
personalized medicine and drug discovery, 11
predicting natural disasters, 12e14
understanding climate change, 12e14
Supercomputing problems, 7e8, 8t
Supernodal LU, 326e327
Sustained performance, 18
Symmetric memory processor (SMP), 55, 75, 194f
examples and characteristics, 192t
Symmetric multiprocessor architecture
Amdahl’s law plus, 196e199, 197fe199f
architecture overview, 192e196
external I/O interfaces, 213e222
JTAG, 218e219
network interface controllers, 213e215
Ethernet, 213e214
InfiniBand, 215
serial advanced technology attachment, 215e218
universal serial bus, 220e222
memory hierarchy, 204e209, 204f
data reuse and locality, 204e205
memory hierarchy, 205e207, 206f
memory system performance, 207e209
outcomes, 222e223
overview, 191e192
PCI bus, 209e213, 210f
processor core architecture, 199e203, 200t
branch prediction, 201e202
execution pipeline, 200e201
forwarding, 202
instruction-level parallelism, 201
multithreading, 203
reservation stations, 202
Synchronization, OpenMP programming model, 241e243
barrier directive, 243
critical synchronization directive, 242
master directive, 242e243
single directive, 243
Synchronization collectives, 262e265
barrier synchronization, 263, 263f
example, 264e265
overview of collective calls, 262e263
Synchronization with storage device, 557
System area networks, commodity clusters, 94e95
System calls for file access, 554e558
data access with explicit offset, 556
file length adjustment, 556e557
file offset manipulation, 556
file open and close, 554e555
file status query, 557e558
sequential data access, 555e556
synchronization with storage device, 557
System components, 349
System monitors to aid debugging, 441e445
System-level checkpointing, 592e597
T
Tabulator built for the 1890 census, 54f
Task dataflow, parallel algorithms, 306e310, 307fe309f
Technology epochs, HPC architecture, 51e55
Tertiary storage, 544, 544f, 545t
Text tools, Linux, 659e661
Thermal control, 47
Another random document with
no related content on Scribd:
necessary to cover it with some well-warmed and non-conducting
material and to have the room well warmed. If bed-sores are already
formed, they are to be treated according to ordinary surgical
principles. Antiseptic means should be in the foreground in the case
of the malignant bed-sore. It is to be remembered that the latter is a
gangrenous process, and, in so far as the formation of a line of
demarcation between the necrotic and the conserved tissue is
concerned, the ordinary expectant rules of surgery govern the case:
the water-bath appears to yield the best results. Ordinary bed-sores
yield readily to mechanical protection and stimulating ointments or
the balsam of Peru. Iodoform is recommended, but it produces
granulations of an indolent character as compared with those
obtained through the use of resinous ointments.

The warm bath is probably the most useful measure in acute

myelitis. In cases due to exposure I do not believe it can be applied
too soon. Its temperature should, in the beginning at least, not be
higher than about 88° F., and the duration about seven minutes. The
continued administration will depend on the immediate effect on the
patient, and the sittings can be ultimately prolonged to half an hour
or even longer. It should be administered once daily, and, when the
patient is not disturbed by the manipulation connected with its use,
even oftener. Cold baths are recommended by the Germans, but it
seems rather in the passive period of the disease than during its
active phase.

The management of the bladder trouble is one of the most critical

points in the treatment of most cases. The results of retention of
urine are more frequently the cause of a fatal issue than any other
single complication. Where there is complete retention continuous
catheterization is to be employed, as recommended by Strümpell.106
In other cases the bladder should be emptied thrice in the twenty-
four hours. The catheter is to be thoroughly disinfected, and if the
slightest sign of cystitis is noted the bladder should be washed out
with astringent and disinfectant solutions. The continuous irrigation
with a solution of corrosive sublimate, 1:2000, appears to give the
best results.
 106 A. Nelaton's catheter is introduced into the bladder, and kept in place by strips of
adhesive plaster fixed to the inner aspect of the thighs. A perforated condom fixed to
the catheter, and then fastened to the inguinal region, is in my opinion a better
measure, especially in patients who have some motor power left. The catheter is
connected with a rubber tube, which should run on a decline in order to effect
complete drainage.

If there be a syphilitic taint present—and more particularly is this true

of cases where no other cause can be assigned—large doses of
iodide of potassium107 should be administered, and mercurial
ointment energetically applied, even to the point of salivation.
107 Here, if anywhere, the so-called heroic doses are applicable.

I am unable to say anything with regard to the local use of electricity

in the active period of the disease. Ascending currents are
recommended by Erb, weak currents of about five minutes' duration
being employed over the part supposed to be affected. In order to
secure sufficiently deep deflection of the current large electrodes are
applied. The use of galvanism and faradism on the affected
periphery, the former being preferred where atrophic, the latter
where anæsthetic conditions preponderate, is to be instituted early
and continued through the duration of the disease. The rules stated
in the section devoted to electro-therapeutics are to be followed. In
cases where the bladder or rectum are paralytic, these organs
should have galvanism applied to them by means of electrodes
shaped like catheters or sounds.

After the acute phase of myelitis is passed, the secondary period is

to be managed according to the principles laid down in the section
on the Treatment of the Sclerotic Processes.
 THE CHRONIC INFLAMMATORY AND
DEGENERATIVE AFFECTIONS OF THE SPINAL
CORD.

BY E. C. SPITZKA, M.D.

The intimate dovetailing of parenchymatous and connective

substance in the spinal cord renders the determination of the precise
origin of a morbid process often difficult if not impossible. Whatever
the mode of origin, the typical termination of inflammatory and
degenerative spinal disorders is sclerosis—a condition in which the
essential nervous elements suffer diminution or destruction, while
the connective substance is either absolutely or relatively in
preponderance. The clinical result is nearly the same in all cases,
and with few exceptions depends less upon the histological
character than on the topographical distribution of the lesion. It is for
this reason that we shall follow rather the topographical than the
morbid histological principle in classification.

The chronic disorders of the spinal cord here considered have as a

common character the slow and progressive displacement of the
normal conducting nerve-tubes by a grayish, reddish, or otherwise
discolored tissue, which is firmer in texture than normal nerve-
substance. Minute examination shows that it is above all the myelin
of the nerve-tubes that is wasted or destroyed, while the axis-
cylinders may share in this dissolution, or, as in one form of
sclerosis, they may persist and even continue in their conducting
function. This increased consistency is found to be due to the
increase of connective-tissue trabeculæ and of the neuroglia. In
some cases this increase is active and absolute, like the increase in
certain phases of hepatic cirrhosis; in others it is rather relative and
passive, being merely a condensation of the connective framework
after the intervening elements have wasted away, just as the texture
of a compressed sponge is more compact than that of an expanded
one.

Crudely, the affections of the cord characterized by this change may

be divided into two great groups—the regular and the irregular. The
former are characterized by their limitation to special nerve-tracts.
Thus, in primary lateral sclerosis we find a special tract in the
posterior part of the lateral columns occupied by the lesion. A
number of spinal diseases with well-defined symptoms are due to
active morbid processes which similarly follow the normal distribution
of the great nerve-tracts of the cord, with the same, almost
mathematical, regularity with which the medullary white develops in
the fœtal and infantile cord. In another class of cases we find that the
same tracts are involved in like manner—not in the way of primary
disease, but through the interruption of the physiological continuity of
the nerve-tract by some other disease, to which the sclerosis is
therefore secondary. These are the so-called secondary
degenerations.

The irregular affections of the cord comprise cases in which the

sclerotic foci are either disseminated through the cerebro-spinal axis
in numerous foci, or diffusely involve a greater or lesser length and
thickness of the medullary cylinder, or, finally, extend in the
transverse plane completely or nearly so across the section area of
the cord at a definite level.

As the symptoms of the regular affections of the cord are by far the
most readily recognizable, and a preliminary knowledge of them will
facilitate the better understanding of the irregular forms, we shall
consider the former first. They may be subdivided into two groups.
The largest, longest known, and best studied consists of acquired,
the other, containing less numerous cases and varieties, and
rendered familiar to the profession only within the last decade,
comprises the spinal disorders due to defective development of the
cerebro-spinal and spinal-fibro systems.

Tabes Dorsalis.

SYNONYMS.—Locomotor ataxia and ataxy, Posterior spinal sclerosis;

Graue degeneration der Hinterstränge (Leyden), Rückenmarksdarre
(popular), Rückenmarksschwindsucht; Tabes dorsualis (Romberg);
Ataxie locomotrice progressive (Duchenne).

HISTORY.—Undoubtedly, this disease must have been observed by

the ancient masters of medicine, but their descriptions group
together so many symptoms of other organic as well as functional
diseases of the cord that it is impossible to obtain a clear idea as to
what special form they had in view when they spoke of tabes
dorsualis. This term was adopted by the German writers of the last
and the earlier part of this century as designating all wasting
affections of the cord; and it was not till Romberg in 1853 and
Duchenne in 1858 differentiated the characteristic clinical signs, and
Cruveilhier and Todd (1847) determined the distribution of the lesion
in the posterior columns, that the medical profession began to
recognize the distinctness of this the most common form of organic
spinal ailment.

At this time, when tabes dorsalis became recognized as a clinical

entity depending on sclerotic degeneration of the posterior segment
of the cord, the inco-ordination or ataxia manifested in the peculiar
gait of the patient was regarded as the most characteristic, essential,
and constant symptom of the disease. It was regarded as an axiom
in physiology that the posterior segment of the cord was subservient
to centripetal and sensory conduction, and nothing appeared to
follow more naturally than the conclusion that the patient exhibited
inco-ordination because, his sensory conduction being interrupted or
destroyed, he had lost the necessary gauge for judging of the
required extent and force of a given movement. Since then both
spinal physiology and pathology have undergone a profound
revolution. So far are we to-day from regarding anterior and motor or
posterior and sensory as convertible terms that we locate the most
important centrifugal tract of the cord behind the plane which divides
it into anterior and posterior halves, while the most recently
differentiated centripetal tract is represented by Gowers as lying in
front of it. It has been determined, through Türck, Bouchard, and
Flechsig and his pupils, that the posterior white columns of the cord
contain several systems of fibres, whose functions—where
determinable—are of an exceedingly complex nature. The elaborate
investigations of Charcot, Schultze, and Strümpell, made during the
past decade, have shown that the distribution of the diseased field in
the posterior segment is not uniform, but is most intense in special
areas. These were found to be affected with a remarkable constancy
and regularity. Almost coeval with this discovery came the
recognition of symptoms found in the earliest phases of the disorder
by Westphal, which, attributable as they were found to be to the
involvement of the special reflex mechanisms of the cord, enable us
to demonstrate the existence and extent of the disease at periods
which previously were not within the ken of the clinician. The
reproach which Leyden was justified in making, that most tabic
patients pass through the pre-ataxic phase of their disorder under
the diagnosis of rheumatism or some similar affection, can now no
longer be made. Our ability to recognize the advent of this disease
has reached such a degree of refinement that the question arises
whether we are always justified in alarming a patient who has a
prospect of remaining free from invalidating complications for many
years with the announcement of a disease which, above all other
spinal affections, is looked upon with dread as equally hopeless,
insidious, and distressing.

CLINICAL HISTORY.—The development of tabes dorsalis is typically

slow and its precise commencement usually not determinable. In
some cases this or that one of the characteristic symptoms of the
disease preponderates from the beginning, and continues
throughout the illness as a prominent feature; in others distinct
stages can be recognized, each marked by one or several symptoms
which were absent or slight in the other stages. In some cases the
progress of the affection is marked by episodes which are absent or
rare in other cases. As a rule, however, it may be stated that tabes is
a progressive affection, marked by pronounced temporary changes
for better or worse, which are often developed with astonishing
rapidity, and subside as quickly, terminating fatally unless its
progress be arrested by treatment or interrupted by some other fatal
affection. The latter is by far the more frequent termination in the
well-to-do class of sufferers.

For purposes of convenience we may consider the symptoms of the

earlier period of the illness as pre-ataxic, and those of the later as
the ataxic. It is to be remembered, however, that in the strictest
sense of the term there is usually some disturbance of co-ordination
even in the early period, while the symptoms of the pre-ataxic period
continue over the ataxic period, and may even become aggravated
with it. Often the patient does not himself suspect a spinal, or indeed
any nervous, disorder, and consults the physician either on account
of rheumatic pains, double sciatic neuralgia, bladder disturbance,
diminished sexual power, amaurosis, diplopia, or even gastric
symptoms, which on closer examination are revealed to be
evidences of tabes. The determination of the initial symptoms is
retrospective as a rule.

Of the subjective signs, one of the commonest, if indeed it be ever

entirely absent, is a tired feeling, particularly noted in the knees and
ankles. This sensation is compared to ordinary muscle-tire, but is
provoked by slight exertion, and not as easily remedied by rest.
Often a numb feeling is associated with it, although no objective
diminution of cutaneous sensation be determinable. This combined
feeling of tire and numbness, described as a going-to-sleep feeling in
the ankle, has indeed been claimed by one observer1 to be
pathognomonic of early tabes. Next in frequency, and almost as
universal, are peculiar pains: these are manifold in character and
distribution, but so distinctive as to alone suggest the existence of
the disease from the manner in which the patient describes them.
One variety, the lightning-like, is compared to a sudden twinge of
great intensity shooting through the limb. The sciatic and anterior
crural branches are the lines usually followed by this pain, but there
are cases where perineal and abdominal regions are affected. Often
the pain is so severe that the patient cries out or the limb is violently
contracted under its influence. It differs from rheumatic pain in the
fact that it is distinctly paroxysmal and that the intermissions are
complete; that it is not greatly aggravated by motion nor relieved by
rest, while the rheumatic pain is; that tabic pain is usually relieved,
and rheumatic pain aggravated, by pressure, while hyperæsthesia is
present with the former, and either absent or barely indicated with
the latter. The lightning-like pains are sometimes combined with
another form, which is even more distinctive in character. This form
does not affect the distribution of special nerves, but is found limited
to a small area which the patient is able to localize definitely: it may
not exceed a centimeter or two in diameter, and within this area the
pain is excruciating. It is either of a burning character or compared to
the firm pressure of a vise or heavy weight, or to the tearing, boring,
and jumping of a violent toothache.
1 Canfield, Lancet, 1885, vol. ii. p. 110.

While some patients escape these pains almost entirely,2 others are
tormented with them at intervals for years, their intensity usually
diminishing when the ataxic period is reached. There is little question
among those who have watched patients in this condition that their
pains are probably the most agonizing which the human frame is
ever compelled to endure. That some of the greatest sufferers
survive their martyrdom appears almost miraculous to themselves.
Thus, in one case the patient, who had experienced initial symptoms
for a year, woke up at night with a fulminating pain in the heels which
recurred with the intensity of a hot spear-thrust and the rapidity of a
flash every seven minutes; then it jumped to other spots, none of
which seemed larger than a pin's head, till the patient, driven to the
verge of despair and utterly beside himself with agony, was in one
continued convulsion of pain, and repeatedly—against his conviction
—felt for the heated needles that were piercing him. In another case
the patient, with the pathetic picturesqueness of invalid misery,
compared his fulminating pains to strokes of lightning, “but not,” he
added, “as they used to appear, like lightning out of a clear sky, but
with the background of a general electrical storm flashing and
playing through the limbs.”
2 I have at present under observation two intelligent patients (one of whom had been
hypochondriacally observant of himself for years) who experienced not a single pain,
as far as they could remember, and who have developed none while under
observation. Seguin mentioned a case at a meeting of the Neurological Society with a
record of but a single paroxysm of the fulgurating variety. Bramwell (Brit. Med. Journ.,
Jan. 2, 1886) relates another in which the pains were entirely absent.

Either while the pains are first noticed or somewhat later other signs
of disturbed sensation are noted. Certain parts of the extremities feel
numb or are the site of perverted feelings. The soles of the feet, the
extremities of the toes, the region about the knee-pan, and the
peroneal distribution, and, more rarely, the perineum and gluteal
region, are the localities usually affected.3 In a considerable
percentage of cases the numbness and tingling are noted in the little
finger and the ulnar side of the ring finger; that is, in the digital
distribution of the ulnar nerve. The early appearance of this symptom
indicates an early involvement of the cord at a high level. Some
parallelism is usually observable between the distribution of the
lightning-like pains when present and the anæsthesia and
paræsthesia if they follow them. With these signs there is almost
invariably found a form of illusive sensation known as the belt
sensation. The patient feels as if a tight band were drawn around his
body or as if a pressure were exerted on it at a definite point. This
sensation is found in various situations, according as the level of the
diseased part of the cord be a low or high one. Thus, when the lower
limbs are exclusively affected or nearly so the belt will be in the
hypogastric or umbilical region; if the upper limbs be much involved,
in the thoracic region; and if occipital pain, anæsthesia of the
trigeminus, and laryngeal crises are present, it may even be in the
neck. Correspondingly, it is found in the history of one and the same
patient: if there be a marked ascent—that is, a successive
involvement of higher levels in the cord—the belt will move up with
the progressing disease. This occurrence, however, is less
frequently witnessed than described. In the majority of cases of
tabes disturbances of the bladder function occur very early in the
disease. Hammond indeed claims that in the shape of incontinence it
may be the only prodromal symptom for a long period.4
3 In the exceptional cases where the initial sensory disturbance is marked in the
perineal and scrotal region I have found that the antecedent fulminating pains had
been attributed to the penis, rectum, and anal region; and in one case the subjective
sense of a large body being forcibly pressed through the rectum was a marked early
sign.

4 New England Medical Monthly, 1883.

I have under observation a patient who has been compelled to use

the catheter daily for years, who has gradual disappearance of the
knee-jerk and reflex iridoplegia, but who has presented no other
evidence of tabes during the year and a half he has been watched.
Among the exact signs of tabes, reflex iridoplegia and abolition of the
knee-jerk are probably the earliest to appear. It may be assumed
with safety that in ninety-nine out of a hundred cases both the
inability of the pupil to respond to light and the absence of the knee-
jerk will be found long before ataxia is developed. Cases are
recorded where no other positive signs were found, and no other
signs of the disease developed for a number of years,5 and others
where disappearance of the knee-jerk was the very first indication.
5 Westphal, also Tuczek, Archiv für Psychiatrie, xiii. p. 144.

The opinion of observers as to the frequency of double vision as an

early symptom of tabes is far from being unanimous. The majority of
writers speak of it as rare, but it is probable that this usually transient
symptom is forgotten by the patient, or because of its apparent
triviality escapes notice. The patient while looking at an object sees a
double image of the latter. This may last for a few seconds, minutes,
or hours, and rarely for a day or week.6 A distinct history of this
symptom was given by 58 out of 81 patients in whom I recorded the
oculo-motor signs. By far the most important of the exact prodromal
signs of tabes are two symptoms—one involving a special faculty of
co-ordination, the other the reflex movements of the pupil. One or
both of these must be present to justify the diagnosis of incipient
tabes.7 The disturbance of co-ordination consists in an inability of the
patient to stand steadily when his eyes are closed. The majority of
healthy persons when tested in this way may show a little swaying in
the beginning, but eventually they stand as steadily as they do with
open eyes, and there is no subjective feeling of uncertainty as to
falling. But the tabic patient exhibits oscillations, and makes efforts to
overcome them which, instead of neutralizing, usually aggravate
them and betray the great disturbance of his equilibrium. It is not as
if he swayed merely because he is uncertain of his upright position,
but as if some perverted force were active in throwing him out of it. It
is found to be a pronounced feature even in cases where the patient
with open eyes is able to walk nearly as well as normal persons, and
experiences no trouble in performing intricate evolutions, such as
dancing, walking a line, or even walking in the dark—faculties which
the patient is destined to lose as his disease progresses.8
6 It has been asserted that the severer and more persistent diplopias are found with
tabes dependent on syphilis.

7 Not even the absence of the knee-jerk ranks as high as these two signs. Aside from
the fact that this is a negative symptom, it is not even a constant feature in advanced
tabes.

8 It does not seem as if the disturbance of static equilibrium were due merely to the
removal of the guide afforded by the eyes, for it is noted not alone in patients who are
able to carry out the average amount of locomotion in the dark, but also in those who
have complete amaurosis. Leyden (loc. cit., p. 334) and Westphal (Archiv für
Psychiatrie, xv. p. 733) describe such cases. The act of shutting the eyes alone,
whether through a psychical or some occult automatic influence, seems to be the
main factor.

In most cases of early tabes it is found that the pupil does not
respond to light; it may be contracted or dilated, but it does not
become wider in the dark nor narrower under the influence of light.
At the same time, it does contract under the influence of the
accommodative as well as the converging efforts controlled by the
third pair, and in these respects acts like the normal pupil. It is
paralyzed only in one sense—namely, in regard to the reflex to light;
just as the muscles which extend the leg upon the thigh may be as
powerful as in health, but fail to contract in response to the reflex
stimulus applied when the ligamentum patellæ is struck. For this
reason it is termed reflex iridoplegia.9 It is, when once established,
the most permanent and unvarying evidence of the disease, and is
of great differential diagnostic value, because it is found in
comparatively few other conditions.
9 It is also known as the Argyll-Robertson pupil. Most of the important symptoms of
tabes are known by the names of their discoverers and interpreters. Thus, the
swaying with the eyes closed is the Romberg or Brach-Romberg symptom; the
absence of the knee-phenomenon, Westphal's or the Westphal-Erb symptom; and the
arthropathies are collectively spoken of as Charcot's joint disease.

In a number of cases ptosis of one or both eyelids is noted at an

early stage of the disease. It is usually temporary, and coincides as
to time with the diplopia, if present.

Patients presenting some or all of the subjective and objective

manifestations of tabes mentioned may continue in a condition of
otherwise comparative health, enabling them to attend to their
vocation for from one to twenty years, and it is not improbable that
the pre-ataxic period may extend over nearly a lifetime. In a less
fortunate minority of cases some of the most distressing evidences
usually marking the last stages of the disease are found developed
at the onset. Thus, cases are known where optic nerve-atrophy
preceded the true tabic period by ten or more years; others in which
trophic disturbances, manifested in spontaneous fractures of bones10
or violent gastric crises, or even mental disturbance, inaugurated
tabes dorsalis, instead of closing or accompanying the last chapter
of its history, which is the rule.
10 Berger, Deutsche medizinische Wochenschrift, 1885, 1 and 2.
The disturbance of co-ordination above spoken of as manifested in
the inability of the patient to stand well with his eyes closed is the
first step in the development of the characteristic ataxia which marks
the full-blown affection. The patient finds that he tires more and more
on slight exertion—not because his muscles are weak, but because
he has to make more voluntary effort than a person in health. He
finds that he stumbles easily—is unable to ascend and descend at
the curbstone or to walk over an irregular surface with ease. Going
down stairs is peculiarly irksome. “I would rather,” aptly said one
patient, “troubled as I am in walking, go a mile in the street than walk
up three flights of stairs; but I would rather go up six flights of stairs
than walk down one.” Soon the patient notices that walking in the
dark becomes more a feat of relative skill than the easy, almost
automatic, act it was in health. His vocation, if it was one involving
the use of the feet, becomes irksome, difficult, and finally impossible,
and in a number of cases the upper extremities are also involved.11
Delicate motions, such as those required in needlework, in writing,
and by watchmakers, musicians, opticians, and lapidaries, are
clumsily performed; even coarser movements, such as buttoning the
clothes and carrying a glass filled with water to the mouth, are
performed in an uncertain and clumsy manner. Meanwhile, the
disturbance of motion in the lower limbs progresses. Difficult as it
formerly was for the patient to stand on one foot or with both feet
together while the eyes were closed, he is now unable to do either
with the eyes open. He straddles in his walk, or, in order to overcome
the element of uncertainty involved in moving the knee-joint, keeps
this joint fixed and walks with short, stiff steps. If ordered to halt
suddenly while thus walking, the patient sways violently, and makes
movements with his hands or arms to recover his balance, in some
cases staggering and even falling down. He shows a similar
unsteadiness when told to rise suddenly from a chair or to mount
one, and it becomes impossible for him to walk backward. Later on,
it will be found that his feet interfere in walking. He has lost the
power of gauging the extent and power of his motions to such a
degree that he may actually trip himself up. To neutralize in some
way this element of uncertainty of his steps, the patient is compelled
to exaggerate all his ambulatory movements, and there results that
peculiar gait which was the first symptom directing attention to the
disease to which it is due. The feet are thrown outward, and violently
strike the ground; the heel touches the latter first, and the patient
appears as if he were punishing the ground and stamping along
instead of walking. The reason for his adopting these tactics are
twofold. In the first place, he has a subjective sense of walking in a
yielding substance, as if on a feather bed, air-bladders, cushions, or
innumerable layers of carpet, and he therefore makes efforts to
touch firm bottom. In the second place, his motor inco-ordination, in
so far as it is not the result of anæsthesias, is greatest in those
segments of his limbs which are farthest removed from the trunk,
and which, enjoying the greatest freedom of combined motion, are
also most readily disturbed. His uncertainty is therefore greater in the
toes than in the ankle, greater in the ankle than in the knee, and
greater in the knee than in the thigh: he prefers to touch the ground
with the heel to touching it with the toes, and to move his limb in the
hip than in the knee-joint. As the patient advances in life even this
limited and clumsy form of locomotion becomes impossible: he takes
to his bed, and it is found that he loses all sense of the position of his
lower and occasionally of his upper limbs. He is unable to tell which
limb overlies the other when his leg is crossed—unable to bring one
limb in parallel position with another without the aid of his eyes. If
told to touch one knee or ankle with the toes of the other side, his
limb oscillates around uncertainly, and makes repeated unskilful
dashes at the wrong point, and ultimately all but the very coarsest
muscular co-ordination appears to be lost, even when the
supplementary aid of the eyes is invoked.
11 Cases in which the upper extremities are intensely involved in the beginning are
uncommon, and those in which they are more intensely involved than the lower, or
exclusively involved, may be regarded as pathological curiosities.

Together with this gradual impairment and abolition of co-ordination,

which has given the name of locomotor ataxia to the disease, but not
always in that strict parallelism with it on which Leyden12 based his
theory of inco-ordination, the sensory functions proper become
perverted and impaired. Usually the determinable anæsthesias are
preceded by subjectively perverted sensations, such as the
numbness already referred to, or even by hyperæsthesia. Usually, all
categories of cutaneous sensation, whether special or pathic, are
impaired in advanced tabes; the points of the æsthesiometer are not
readily differentiated; the patient is unable to correctly designate the
locality which is touched or pinched; the pain-sense is occasionally
so much blunted that a needle may be run through the calf of the leg
without producing pain, and in some cases without being
appreciated in any form. Even if the pain-sense be preserved, it will
be found that its appreciation by the patient is delayed as to time.
Not infrequently bizarre misinterpretations are made of the
impressions acting on the skin. One of these, the feeling as if the
patient were standing on carpet, cushions, or furs, whereas he may
be standing on stone flagging, has already been mentioned as a
factor in the disturbed locomotion of the patient. In the later period,
numerous perversions of this kind are noted: to one of these, already
mentioned by Leyden, Obersteiner has called renewed attention. It
consists in a confusion of sides; the patient when pricked or touched
on one foot or leg correctly indicates the spot touched, but attributes
it to the wrong side.13
12 Klinik der Rückenmarkskrankheiten, Band ii.

13 Allochiria is the term applied by Obersteiner. Hammond has offered an explanation,

which, as it is based on the assumption of altogether hypothetical nerve-tracts, and
not in any sense accords with positively established facts, is more properly a subject
for consideration in a theoretical treatise.

While it may be affirmed, as a general proposition, that the tendency

of the tabic process is to abolish sensation below the level of the
disease, there are noteworthy exceptions, not only in individual
cases, but with regard to certain kinds of sensation; nor do the
different kinds of sensation always suffer together. Thus, the pain-
sense may be blunted and the contact-sense preserved, or, more
commonly, the latter blunted and the former exaggerated, so that the
unfortunate patient, in addition to being debarred of the useful
varieties of sensation, those of pressure and space, has the painful
ones exaggerated, as a hyperalgesia. According to Donath,14 the
temperature-sense is usually blunted in tabes, and a greater degree
of heat or cold can be borne without discomfort than in health; but in
about two-sevenths of the cases studied there was increased
sensitiveness to heat, and in one-seventh to cold. One of the
commonest manifestations is delayed conduction. This interesting
phenomenon has been especially observed in that phase of the
disease where tactile perception is beginning to be blunted. If the
patient be pricked with a pin, he feels the contact of the latter at the
proper moment as a tactile perception, and then after a distinct
interval, varying from one to four seconds, his limb is suddenly
drawn up and his face contorted under the influence of an
exaggerated pain. This fact furnishes one of the chief grounds for the
assumption that there are distinct channels for the transmission of
pain and tactile perception in the cord, and that they may be involved
separately or with different intensity in the disease under
consideration. In some advanced cases it is found that not only the
transmission of pain-appreciation is delayed, but that there are after-
sensations recurring at nearly regular intervals of several seconds,
and accurately imitating the first pain-impression.
14 Archiv für Psychiatrie, xv. p. 707.

To what extent the muscular sense is affected in tabes at various

stages of the disorder is somewhat in doubt. Strümpell15 by
implication, and other writers directly, attribute the inco-ordination
observed on closing the eyes to the loss of the muscular sense. As
this symptom is also observed in patients who when they lie on their
backs are able to execute intricate movements notwithstanding the
exclusion of the visual sense, and as their uncertainty in an early
stage is not always with regard to the position of their limbs nor the
innervation of individual or grouped muscles, it seems inadmissible
to refer the Romberg symptom16 to the loss of muscular sense alone.
15 Lehrbuch der Speciellen Pathologie und Therapie, p. 193, vol. ii.

16 It should be designated as static ataxia, in distinction from locomotor or motor

ataxia, which is manifested in unskilled movements.
The important part played by eye symptoms in the early stages of
tabes has been already referred to. Reflex iridoplegia is one of these
early and persisting features; it is sometimes complicated with
mydriasis, and occasionally with paralysis of accommodation of one
eye. In the majority of my cases there was spinal myosis, often of
maximum intensity; in a large number there was in addition
irregularity of the outline of the pupil; and where there was mydriasis
I found it to be quite symmetrical, in this respect differing from the
experience of Müller17 and Schmeichler.18
17 Centralblatt für die Gesammte Therapie.

18 Loc. cit.

Atrophy of the optic nerve is a common and sometimes, as stated,

the initial symptom of tabes. It is rarely found in its incipient phase in
advanced stages. The patient who escapes involvement of the optic
nerve in the pre-ataxic stage is very apt to escape it altogether. It is
more frequently found either to precede the pre-ataxic period by
months and even years, or to develop during this period, leading to
complete atrophy in the ataxic period, and sometimes before. Erb
calculates that 12 per cent. of tabic patients have optic-nerve
atrophy; he probably includes only such cases in which the atrophy
was marked or led to amblyopia. Including the lesser degrees of
atrophy, it is found in a larger number of patients. Schmeichler
claims as high as 40 per cent.19 If we regard those cases in which
there is noted progressing limitation of the color-field of the retina as
beginning optic-nerve atrophy, the majority of tabic patients may be
said to have some grade of this disorder. The ophthalmoscopic
changes are quite distinctive: at first there is noted a discoloration of
the papilla and apparent diminution of the number of arterial vessels;
the veins then become dilated; and finally the papilla becomes
atrophied, the vessels usually undergoing a narrowing after their
previous dilatation. In the first stages of this process visual power is
not gravely impaired, but as soon as shrinkage has set in visual
power sinks rapidly to a minimum, decreasing till only quantitative
light-perception remains. This limited function usually remains
throughout, but in a few cases complete amaurosis ensues. As the
field of vision becomes diminished, the concentric extinction of
quantitative perception is preceded by concentric extinction of color-
perception, the color-field for green being the first to suffer; red
follows, and blue remains last.
19 Archives of Ophthalmology and Otology, 1883.

In a large number of cases the only symptoms attributable to a

disturbance of the cerebral functions are those connected with the
motor relations of the eyeball and pupil, and the function and
appearance of the optic nerve. The special senses other than those
of sight and touch are rarely affected. Sometimes there is obstinate
tinnitus, exceptionaly followed by deafness, attributed to atrophy of
the auditory nerve. The development of symptoms resembling those
named after Ménière is not referable with certainty to a disturbance
of the same nerve.

The sexual functions become involved in all cases of tabes sooner or

later. In the majority of cases there is a slow, gradual extinction of
virile power; in a large minority this extinction is preceded by irritative
phenomena on the part of the genital apparatus. Some patients
display increased sexual desire and corresponding performing
powers, but mostly they suffer from erections of long duration which
may be painful, and loss of the normal sensations attending the
sexual orgasm. There is no constant relationship between the
intensity of the general affection and the diminution of sexual power.
It is retained to a limited degree by patients who are barely able to
walk, and it may be entirely destroyed in those who have but entered
the initial period.

The bladder disturbance,20 which in some forms is usually found

among the initial symptoms, is always a marked feature in the ataxic
period. Usually, there is a frequent desire for micturition, with more or
less after-dribbling; sometimes there is retention, alternating with
involuntary discharge; complete incontinence may close the scene
through the channel of an ensuing cystitis and pyelitis. With the
incontinence of urine there is usually found obstinate constipation,
which may be varied by occasional spells of incontinence of feces.
The crises of tabes often complicate these visceral symptoms.
20 Contrary to what might be anticipated from the topographical nearness of the
vesical and genital centres in the cord, the disturbances of bladder function and virile
power do not go hand in hand.

The reflex disturbances are among the most continuous evidences

of the disease. Of two of these, the disappearance of the patellar jerk
or knee-phenomenon and the inability of the pupil to react to light,
we have already spoken when discussing the initial period.
Practically, it may be claimed that both are always found in typical
tabes. It has been claimed that the patellar jerk may be exaggerated,
or even that its disappearance is preceded by exaggeration. As this
disappearance usually occurs extremely early in the pre-ataxic
period, it is difficult to follow the deductions of those who claim to
have watched an alleged earlier phase of exaggeration. It is more
than probable that cases of combined sclerosis, in which the lateral
columns were affected together with or earlier than the anterior, have
been mistaken for typical tabes. Here, it is true, the jerk is first
exaggerated through the disease of the lateral column, and later
abolished as the lesion in the posterior reflex arch progresses and
becomes absolute.

Other tendinous reflexes21 suffer with the knee-jerk in the peripheries

corresponding to and below the involved level of the cord. The
cutaneous reflexes are usually abolished, but may be retained in
advanced stages of the disease. The same is true of the cremaster
reflex.
21 Whether the tendon phenomena are true reflexes or not is a question still agitating
physiologists. Opinion inclines in favor of their reflex nature, and, pathologically
considered, it is difficult to regard them in any other light.

Opinion is divided as to the electrical reactions in tabes dorsalis.

That qualitative changes never occur in uncomplicated cases all
authorities are agreed, but while Strümpell and other modern writers
claim there is no change of any kind, a number of careful
investigators have found an increased irritability in the initial period,
particularly marked in the peroneal group of muscles (Erb). In my
own experience this is frequently the case, where lightning-like pains
are the only subjectively distressing symptoms complained of.

While the symptoms thus far considered as marking the origin and
progress of tabes dorsalis are more or less constant, and although
some of them show remarkable remissions and exacerbations, yet
may in their entity be regarded as a continuous condition slowly and
surely increasing in severity, there are others which constitute
episodes of the disease, appearing only to disappear after a brief
duration varying from a few hours to a few days: they have been
termed the crises of tabes dorsalis. These crises consist in
disturbances of the functions of one or several viscera, and are
undoubtedly due to an error in innervation provoked by the
progressing affection of the spinal marrow and oblongata. The most
frequent and important are the gastric crises. In the midst of
apparent somatic health, without any assignable cause, the patient is
seized with a terrible distress in the epigastric region, accompanied
by pain which may rival in severity the fulgurating pains of another
phase of the disease, and by uncontrollable vomiting. Usually, these
symptoms are accompanied by disturbances of some other of the
organs under the influence of the pneumogastric and sympathetic
nerves. The heart is agitated by violent palpitations, a cold sweat
breaks out, and a vertigo may accompany it, which, but for the fact
that it is not relieved by the vomiting and from its other associations,
might mislead the physician into regarding it as a reflex symptom. In
other cases the symptoms of disturbed cardiac innervation or those
of respiration are in the foreground, constituting respectively the
cardiac and bronchial crises. Laryngeal crises are marked by a
tickling and strangling sensation in the throat, and in their severer
form, which is associated with spasm of the glottis, a crowing cough
is added.22 Enteric crises, which sometimes coexist with gastric
crises, at others follow them, and occasionally occur independently,
consist in sudden diarrhœal movements, with or without pain, and
may continue for several days. Renal or nephritic crises are
described23 as resembling an attack of renal colic. The sudden