Professional Documents
Culture Documents
Kai Hwang: Advanced Computer Architecture
Kai Hwang: Advanced Computer Architecture
ARCHITECTURE:
Parallelism, Scalability, Programmability
., J
Kai Hwang
Professor of Electrical Engineering
and Computer Science
University of Southern California
fl
* . *
Iriventar-Nr.:
Sachgebiete: ....<
Standort:
McGraw-Hill, Inc.
New York St Louis San Francisco Auckland Bogota
Caracas Lisbon London Madrid Mexico
Milan Montreal New Delhi Paris
San Juan Singapore Sydney Tokyo Toronto
Contents
Foreword
xvii
Preface
xix
PART
THEORY OF PARALLELISM
Chapter 1
1.1
1
3
3
3
6
9
14
19
19
24
27
27
27
30
32
33
38
41
41
43
44
1.6
45
IX
Contents
Chapter 2 Program and Network Properties
2.1 Conditions of Parallelism
2.1.1 Data and Resource Dependences
2.1.2 Hardware and Software Parallelism
2.1.3 The Role of Compilers
2.2 Program Partitioning and Scheduling
2.2.1 Grain Sizes and Latency
2.2.2 Grain Packing and Scheduling
2.2.3 Static Multiprocessor Scheduling
2.3 Program Flow Mechanisms
2.3.1 Control Flow Versus Data Flow
2.3.2 Demand-Driven Mechanisms
2.3.3 Comparison of Flow Mechanisms
2.4 System Interconnect Architectures
2.4.1 Network Properties and Routing
2.4.2 Static Connection Networks
2.4.3 Dynamic Connection Networks
2.5 Bibliographic Notes and Exercises
Chapter 3 Principles of Scalable Performance
3.1 Performance Metrics and Measures
3.1.1 Parallelism Profile in Programs
3.1.2 Harmonic Mean Performance
3.1.3 'Efficiency, Utilization, and Quality
3.1.4 Standard Performance Measures
3.2 Parallel Processing Applications
3.2.1 Massive Parallelism for Grand Challenges
3.2.2 Application Models of Parallel Computers
3.2.3 Scalability of Parallel Algorithms
3.3 Speedup Performance Laws
3.3.1 Amdahl's Law for a Fixed Workload
3.3.2 Gustafson's Law for Scaled Problems
3.3.3 Memory-Bounded Speedup Model
3.4 Scalability Analysis and. Approaches
3.4.1 Scalability Metrics and Goals
3.4.2 Evolution of Scalable Computers
3.4.3 Research Issues and Solutions
3.5 Bibliographic Notes and Exercises
PART II
HARDWARE TECHNOLOGIES
'.
.'
51
51
51
57
60
61
61
64
67
70
71
74
75
76
77
80
89
96
105
105
105
108
112
115
118
118
122
125
129
129
131
134
138
138
143
147
149
155
Contents
xi
157
157
157
162
165
169
177
178
182
.. 184
188
188
190
194
196
196
198
205
208
213
213
213
216
218
221
224
225
228
232
236
238
239
242
244
248
248
252
253
256
Chapter 6
265
xii
Contents
6.1 Linear Pipeline Processors
6.1.1 Asynchronous and Synchronous Models
6.1.2 Clocking and Timing Control
6.1.3 Speedup, Efficiency, and Throughput
6.2 Nonlinear Pipeline Processors
6.2.1 Reservation and Latency Analysis
6.2.2 Collision-Free Scheduling
6.2.3 Pipeline Schedule Optimization
6.3 Instruction Pipeline Design
6.3.1 Instruction Execution Phases
6.3.2 Mechanisms for Instruction Pipelining
6.3.3 Dynamic Instruction Scheduling
6.3.4 Branch Handling Techniques
6.4 Arithmetic Pipeline Design
6.4.1 Computer Arithmetic Principles
6.4.2 Static Arithmetic Pipelines
6.4.3 Multifunctional Arithmetic Pipelines
6.5 Superscalar and Superpipeline Design
6.5.1 Superscalar Pipeline Design
6.5.2 Superpipelined Design
6.5.3 Supersymmetry and Design Tradeoffs
6.6 Bibliographic Notes and Exercises
PART III
265
265
267
268
270
270
274
276
280
280
283
288
291
297
297
299
307
308
310
316
320
322
329
331
331
333
336
341
348
348
351
358
364
368
368
370
372
375
375
Contents
7.5
xiii
7.4.2 Deadlock and Virtual Channels
7.4.3 Flow Control Strategies
7.4.4 Multicast Routing Algorithms
Bibliographic Notes and Exercises
379
383
387
393
'.
".
403
403
403
408
410
415
415
419
425
429
435
436
437
442
447
447
449
453
457
457
460
462
465
468
475
475
476
480
482
483
486
490
490
495
499
504
xiv
Contents
9.3.1 Fine-Grain Parallelism
9.3.2 The MIT J-Machine
9.3.3 The Caltech Mosaic C
9.4 Scalable and Multithreaded Architectures
9.4.1 The Stanford Dash Multiprocessor
9.4.2 The Kendall Square Research KSR-1
9.4.3 The Tera Multiprocessor System
9.5 Dataflow and Hybrid Architectures
9.5.1 The Evolution of Dataflow Computers
9.5.2 The ETL/EM-4 in Japan
9.5.3 The MIT/Motorola *T Prototype
9.6 Bibliographic Notes and Exercises
PART IV
'
:
505
506
514
516
516
521
524
531
531
534
.536
539
545
547
547
547
551
554
556
559
560
560
562
564
567
567
570
573
578
578
581
585
592
596
599
599
602
605
610
Contents
10.6
xv
Bibliographic Notes and Exercises
612
Chapter 12
'.
617
617
617
621
623
625
625
628
629
634
634
637
640
644
644
645
647
648
648
652
656
661
667
667
668
670
671
671
672
672
674
678
683
683
683
685
12.4
686
687
688
xvi
'
12.4.3 Message-Based Communications
,
12.4.4 Virtual Memory Management
12.5 OSF/1 Architecture and Applications
12.5.1 The OSF/1 Architecture
12.5.2 The OSF/1 Programming Environment
12.5.3 Improving Performance with Threads
12.6 Bibliographic Notes and Exercises
Contents
694
697
701
702
707
709
712
Bibliography
717
Index
739
765