You are on page 1of 47

COMPUTER SCIENCE

S E D G E W I C K / W A Y N E
PA R T I : P R O G R A M M I N G I N J AVA
Computer Science

Computer 7. Performance
Science
An Interdisciplinary Approach

ROBERT SEDGEWICK
K E V I N WAY N E
Section 4.1
http://introcs.cs.princeton.edu

COMPUTER SCIENCE
S E D G E W I C K / W A Y N E
PA R T I : P R O G R A M M I N G I N J AVA

7. Performance
• The challenge
• Empirical analysis
• Mathematical models
• Doubling method
• Familiar examples

CS.7.A.Performance.Challenge

The challenge (since the earliest days of computing machines)

“ As soon as an Analytic Engine exists, it will necessarily guide the future course of the
science. Whenever any result is sought by its aid, the question will arise—By what
course of calculation can these results be arrived at by the machine in the shortest time?”

− Charles Babbage

Difference Engine #2
Designed by Charles
Q. How many times do you

Babbage, c. 1848 have to turn the crank?
Built by London Science
Museum, 1991

3

The challenge (modern version)

Q. Will I be able to use my program to solve a large practical problem?

Q. If not, how might I understand its performance characteristics so as to improve it?

Key insight (Knuth 1970s). Use the scientific method to understand performance.
4

To develop a basis for understanding the
 An algorithm is a method for problem and for designing new algorithms solving a problem that is • Enables new technology. • How can I make my program faster? } StdOut. if (Math. int goal = Integer. We study several algorithms later in this course. for (int t = 0. To compare algorithms and implementations.
 Taking more CS courses? You'll learn dozens of algorithms.Three reasons to study program performance public class Gambler { 1. while (cash > 0 && cash < goal) 2.random() < 0. To predict program behavior public static void main(String[] args) { • Will my program finish? int stake = Integer.parseInt(args[0]).print(wins + " wins of " + trials). 5 . • When will my program finish? int trials = Integer. } 3. • Enables new research. int wins = 0.parseInt(args[1]). t < trials.
 else cash--. suitable for implementation as a computer program. t++) { int cash = stake. • Will this change make my program faster? } if (cash == goal) wins++.5) cash++.parseInt(args[2]).

An algorithm design success story N-body simulation • Goal: Simulate gravitational interactions among N bodies. Andrew Appel PU '81 senior thesis N2 time limit on available time NlogN problem size (N ) 6 . • Issue (1970s): Too slow to address scientific problems of interest. • Success story: Barnes-Hut algorithm uses NlogN steps and enables new research. • Brute-force algorithm uses N 2 steps per time unit.

. N2 time limit on available time NlogN problem size (N ) 7 . • Brute-force algorithm uses N 2 steps. spectroscopy.Another algorithm design success story Discrete Fourier transform • Goal: Break down waveform of N samples into periodic components. • Applications: digital signal processing.. • Issue (1950s): Too slow to address commercial applications of interest.. John Tukey 1915–2000 • Success story: FFT algorithm uses NlogN steps and enables new technology.

Quick aside: binary logarithms 4 Def.01 if (N == 1) return "1". 220 1 million 20 6. 1 0 or log2N 1 2 4 8 16 N Q.02 } 230 1 billion 30 9. Fact.03 Prove by induction. A. Binary logarithms arise in the study of algorithms based on recursively solving problems 
 half the size (divide-and-conquer algorithms). 8 . FFT and Barnes-Hut. like convert. Details in "sorting and searching" lecture. The number of bits in the binary representation of N is 1+ lg N . Fact. Largest integer less than or equal to lg N (written lg N ). return convert(N/2) + (N % 2). The binary logarithm of a number N (written lg N )
 3 lg N 2 is the number x satisfying 2x = N. How many recursive calls for convert(N)? Frequently encountered values N approximate value lgN log10N public static String convert(int N) { 210 1 thousand 10 3.

int[] a = StdIn. For simplicity. Given N integers.txt ✗ 30 -30 -20 -10 40 0 % java ThreeSum < 6ints. Can we solve this problem for N = 1 million? ✓ 9 . 
 public static void main(String[] args) • Does one polygon fit inside another? { • Robot motion planning. • [a surprisingly long list] } } % more 6ints.An algorithmic challenge: 3-sum problem Three-sum.readAllInts().println(count(a)). enumerate the triples that sum to 0. */ } • Find collinear points. just count them. public class ThreeSum { Applications in computational geometry public static int count(int[] a) { /* See next slide. StdOut.txt 30 -30 0 3 30 -20 -10 ✗ -30 -10 40 Q.

triples 
 1 3 4 -30 -10 40 return cnt. with i < j < k 1 3 5 -30 -10 0 } 1 4 5 -30 40 0 2 3 4 -20 -10 40 2 3 5 -20 -10 0 Q. 0 1 5 30 -30 0 0 2 3 30 -20 -10 0 2 4 30 -20 40 public static int count(int[] a) 0 2 5 30 -20 0 { Keep i < j < k to 0 3 4 30 -10 40 int N = a. j < N. j++) for (int k = j+1.length.Three-sum implementation i j k a[i] a[j] a[k] "Brute force" algorithm 0 1 2 30 -30 -20 i 0 1 2 3 4 5 • Process all possible triples. a[i] 30 -30 -20 -10 40 0 0 1 3 30 -30 -10 0 1 4 30 -30 40 • Increment counter when sum is 0. How much time will this program take for N = 1 million? 2 4 5 -20 40 0 3 4 5 -10 40 0 10 . i < N. k < N. i++) 1 2 3 -30 -20 -10 for (int j = i+1. 0 4 5 30 40 0 for (int i = 0. k++) 1 2 4 -30 -20 40 if (a[i] + a[j] + a[k] == 0) 1 2 5 -30 -20 0 cnt++. avoid processing
 each triple 6 times 0 3 5 30 -10 0 int cnt = 0.

jpg http://commons.org/wiki/File:Andrew_Apple_(FloC_2006).jpg http://commons.org/wiki/File:John_Tukey.wikimedia.A.wikimedia._1834-1871.jpg CS.Challenge .wikimedia.org/wiki/File:Hubble's_Wide_View_of_'Mystic_Mountain'_in_Infrared.wikimedia.org/wiki/File:Charles_Babbage_1860.org/wiki/File:Babbages_Analytical_Engine.jpg http://commons.7. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA Image sources http://commons.jpg http://commons.Performance._(9660574685).wikimedia.

7.Empirical . Performance • The challenge • Empirical analysis • Mathematical models • Doubling method • Familiar examples CS.Performance.B. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA 7.

M)). 1 -861312 -2 int N = Integer. -690436 -10 for (int i = 0. 360294 1 } 0 } -7 not much chance of a 3-sum good chance of a 3-sum 13 . M) -425582 -2 594752 public static void main(String[] args) 1 600579 { -4 -483784 int M = Integer.println(StdRandom.parseInt(args[1]). Input generator for ThreeSum % java Generator 1000000 10 28773 public class Generator -807569 % java Generator 10 10 { // Generate N integers in [-M. i++) -732636 -4 StdOut. • Option 2: Write a program to generate representative inputs.parseInt(args[0]).uniform(-M. i < N.A first step in analyzing running time Find representative inputs • Option 1: Collect actual input data.

printf("%d (%.currentTimeMillis() / 1000. int cnt = count(a).Empirical analysis Replace println() in ThreeSum Run experiments Measure running time with this code. • Tabulate and plot results. StdOut.0. double now = System.currentTimeMillis() / 1000.start). • Start with a moderate input size N.0.0f seconds)\n". cnt. now . Tabulate and plot results 250 Run experiments N time (seconds) 200 1000 0 % java Generator 1000000 1000 | java ThreeSum 2000 4 time (seconds) 59 (0 seconds) 4000 31 % java Generator 1000000 2000 | java ThreeSum 8000 248 522 (4 seconds) 100 % java Generator 1000000 4000 | java ThreeSum 3992 (31 seconds) % java Generator 1000000 8000 | java ThreeSum 0 31903 (248 seconds) 0 1000 2000 4000 8000 problem size (N ) 14 . • Double input size N. • Repeat. • Measure and record running time. double start = System.

Aside: experimentation in CS is virtually free. one million experiments one experiment Chemistry Biology Physics Computer Science Bottom line. No excuse for not running experiments to understand costs. 15 . particularly by comparison with other sciences.

8000 248 13 8 ✓ log-log plot Do the math x-intercept (use lg in anticipation of next step) 8 lgTN = lga + 3lgN equation for straight line of slope 3 5 log time (lg TN) TN = aN 3 raise 2 to a power of both sides straight line 2 of slope 3 248 = a × 80003 substitute values from experiment a = 4. 248 • Solve for a with the data.5 • If points are on a straight line (often the case).84 × 10 −10 solve for a −1 TN = 4.84 × 10 −10 × N 3 • Plot on log-log scale.84 × 10 −10 × N 3 substitute 10 11 12 13 log problem size (lg N ) a curve that fits the data ? 16 . 1000 0.5 10 −1 0. 4000 31 12 5 31 • The exponent b is the slope of the line. a 2000 4 11 2 4 power law holds—a curve of the form aN b fits.Data analysis Curve fitting N TN lgN lg TN 4.

84 × 10 −10 × N 3. Running time for N = 16. Running time of ThreeSum is 4. about half an hour % java Generator 1000000 16000 | java ThreeSum 31903 (1985 seconds) ✓ Q.000 will be 1982 seconds. Prediction. 17 . How much time will this program take for N = 1 million? A. 484 million seconds (more than 15 years).Prediction and verification Hypothesis.

84 × 10 −10 × N 3 seconds 250 2010s: 10.Another hypothesis 1970s 5. Running times on different computers differ by only a constant factor.000+ times faster N time (seconds) 200 1000 0 2000 4 4000 31 100 8000 248 Macbook Air 0 0 1000 2000 4000 8000 Hypothesis.2 × 10 −6 × N 3 seconds N time (seconds) 2000000 1000 5319 2000 43221 4000 343774 8000 2654384 1000000 (estimated ) 0 0 1000 2000 4000 8000 VAX 11/780 4. 18 .

B.org/wiki/File:FEMA_-_2720_-_Photograph_by_FEMA_News_Photo.org/wikipedia/commons/2/28/Cut_rat_2.jpg http://pixabay.com/en/lab-research-chemistry-test-217041/ http://upload. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA Image sources http://commons.wikimedia.wikimedia.com/en/view-glass-future-crystal-ball-32381/ CS.Performance.7.Empirical .jpg http://pixabay.

7. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA 7.Performance.Math .C. Performance • The challenge • Empirical analysis • Mathematical models • Doubling method • Familiar examples CS.

1968–present) Yes! • Determine the set of operations. Knuth. too complicated. Don Knuth
 1974 Turing Award 21 . (Prevailing wisdom. • Find the frequency of execution of each operation (depends on algorithm and inputs).Mathematical models for running time Q. E. • Total running time: sum of cost × frequency for all operations. 1960s) No. A. • Find the cost of each operation (depends on computer and system software). Can we write down an accurate formula for the running time of a computer program? A. (D.

i++) less than compare 1/2 ns N+1 if (a[i] == 0) cnt++. Q.Warmup: 1-sum operation cost frequency function call/return 20 ns 1 public static int count(int[] a) { variable declaration 2 ns 2 int N = a. for (int i = 0. knowing exact values may require study and experimentation. assignment 1 ns 2 int cnt = 0. cN + 26.length. i < N. array access 1/2 ns N } increment 1/2 ns between N and 2N Note that frequency of increments depends on input. depending on input. Formula for total running time ? A. 22 .5. equal to compare 1/2 ns N return cnt.5 nanoseconds. where c is between 2 and 2. representative estimates (with some poetic license).

i < N.Warmup: 2-sum operation cost frequency public static int count(int[] a) function call/return 20 ns 1 { variable declaration 2 ns N+2 int N = a. assignment 1 ns N+2 for (int i = 0. int cnt = 0... where. 23 . Formula for total running time ? A. [complicated definitions].length. } increment 1/2 ns between N (N + 1)/2 and N 2 exact counts tedious to derive N N (N 1 ) #i < j = = 2 2 Q. j < N. array access 1/2 ns N (N − 1) return cnt. i++) less than compare 1/2 ns (N + 1) (N + 2)/2 for (int j = i+1. j++) if (a[i] + a[j] == 0) equal to compare 1/2 ns N (N − 1)/2 cnt++. c1N 2 + c2N + c3 nanoseconds.

276. 5/4 N 2 + 13/4 N + 53/2 ~ 5/4 N 2 1.250. for N = 1.3% Q.000. • When N is small.000 within . f(N ) ~ g(N ) means f(N )/g(N ) ➛ 1 as N ➛ ∞ Ex. 24 . everything is negligible. ignored terms are negligible. Formula for 2-sum running time when count is not large (typical case)? eliminate dependence on input A.000 1.253. ~ 5/4 N 2 nanoseconds.5 for N = 1. • When N is large. Def. • Ignore the slower-growing terms.Simplifying the calculations Tilde notation Rationale • Use only the fastest-growing term.

✓ matches 4. ~ N 3/2 nanoseconds. i++) for (int j = i+1. j++) less than compare 1/2 ns ~ N 3/6 for (int k = j+1. Formula for total running time when return value is not large (typical case)? A.Mathematical model for 3-sum operation cost frequency public static int count(int[] a) function call/return 20 ns 1 { int N = a. array access 1/2 ns ~ N 3/2 return cnt.84 × 10 −10 × N 3 empirical hypothesis 25 . variable declaration 2 ns ~N int cnt = 0. } increment 1/2 ns ~ N 3/6 N N (N 1)(N 2) N3 assumes count #i < j < k = = is not large 3 6 6 Q. j < N. k < N. k++) if (a[i] + a[j] + a[k] == 0) equal to compare 1/2 ns ~ N 3/6 cnt++.length. i < N. assignment 1 ns ~N for (int i = 0.

• Predict events using the hypothesis. • Might involve advanced mathematics. running time as a function of N. 26 . • Fit a curve to experimental data to get a • Useful for predicting and explaining. • Useful for predicting. formula for running time as a function of N. but not explaining. • Hypothesize a model consistent with observations. • Applies to any computer. René Descartes • Validate by refining until hypothesis and observations agree. Good news.Context Scientific method • Observe some feature of the natural world. Francis Bacon 1561–1626 • Verify the predictions by making further observations. Mathematical models are easier to formulate in CS than in other sciences. 1596–1650 John Stuart Mill 1806–1873 Empirical analysis of programs Mathematical analysis of algorithms • "Feature of natural world" is time taken by a • Analyze algorithm to develop a formula for program on a computer.

jpg CS.jpg http://commons.wikimedia.jpg http://commons.Performance.wikimedia.C.org/wiki/File:KnuthAtOpenContentAlliance.wikimedia.7.org/wiki/File:John_Stuart_Mill_by_London_Stereoscopic_Company.wikimedia.org/wiki/File:Pourbus_Francis_Bacon.Math .org/wiki/File:Frans_Hals_-_Portret_van_René_Descartes.jpg http://commons._c1870. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA Image sources http://commons.

D. Performance • The challenge • Empirical analysis • Mathematical models • Doubling method • Familiar examples CS.Doubling . COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA 7.7.Performance.

How do you know? A. Might also have a (lgN )c factor. many specific algorithms and applications. there's good chance of that. Real-world data is also often simply structured. A. A. Programs are built from simple constructs (examples to follow). Yes.Key questions and answers Q. Q. Is the running time of my program ~ a Nb seconds? A. A. Deep connections exist between such models and a wide 
 variety of discrete structures (including some programs). 29 . Computer scientists have applied such models for decades 
 to many.

2000 4 8 4000 31 7. 1000 0. 23 = 8 30 .5 • Measure and record running time. 1024000 248 × 87 = 520093696 8 math model says running time should be aN 3 Bottom line. 8000 248 8 ✓ • Repeat while you can afford it.75 • Double size.Doubling method no need to calculate a (!) Hypothesis. 16000 248 × 8 = 1984 8 • Verify that ratios of running times approach 2b. 32000 248 × 82 = 15872 8 • Predict by extrapolation: . The running time of my program is TN ~ a Nb . Proof: = 3-sum example Doubling method N TN TN/TN/2 • Start with a moderate size. ( ) Consequence. As N increases.. It is often easy to meet the challenge of predicting performance.. multiply by 2b to estimate T2N and repeat. T2N/TN approaches 2b.

Experimental validation Explanation with mathematical model When we execute a program on a computer that is X times Machine. Hypothesis. Order of growth is a property of the algorithm. If a function f (N ) ~ ag(N ) we say that g(N ) is the order of growth of the function.and system-dependent features of the model faster. 31 .Order of growth Def. are all constants. not the computer or the system. we expect the program to be X times faster.

if (N == 0) return. { public static void f(int N) .. since constant base not relevant Evidence.Order of growth log instead of lg Hypothesis.. i++) .. j < N. for (int i = 0. f(N/2). for (int k = j+1. . Logarithmic (log N) Linearithmic (N log N) public static void f(int N) Exponential (2N) { public static void f(int N) if (N == 0) return. i < N. k++) . i++) for (int i = 0. many programs with simple and similar structure. j++) . f(N-1). j < N. i < N. f(N/2).... The order of growth of the running time of my program is Nb (logN )c. k < N. for (int j = i+1. Known to be true for many. i++) for (int i = 0. { } . j++) for (int j = i+1. f(N/2)..... f(N-1). Linear (N) Quadratic (N2) Cubic (N3) for (int i = 0.... i++) . } } Stay tuned for examples... i < N... if (N == 0) return.... ignore for practical purposes
 (infeasible for large N ) 32 .. ....... .. i < N.

33 . use doubling method and solve for b = lg(TN/TN/2) to estimate order of growth to be N b.Order of growth classifications order of growth factor for slope of line in log-log plot doubling N3 N2 log-log plot (b) N description function method (2b ) N logN constant 1 0 1 logarithmic logN 0 1 linear N 1 2 Time linearithmic N logN 1 2 8T quadratic N2 2 4 4T logN 2T cubic N3 3 8 T 1 if input size doubles 1K 2K 4K 8K running time increases Problem size by this factor math model may have log factor If math model gives order of growth. use doubling method to validate 2b ratio. If not.

$X = 4aN 3 N2 $X $2X $4X . 2 years 4 years 2M years Do the math now from now from now from now TN = aN 3 running time today N $X $X $X ... $2M X = 4TN N3 $X $4X $16X . cryptography. My problem size also doubles every 2 years. Computer power increases by a roughly a factor of 2 every 2 years. $4M X A... transaction processing.. $X T2N = (a / 2 ) (2N ) 3 running time in 2 years N logN $X $X $X ... How much do I need to spend to get my job done? a very common situation: weather prediction... Q. 34 .. You can't afford to use a quadratic algorithm (or worse) to address increasing problem sizes.An important implication Moore's Law.

out to get a pizza. Hmmm. Time for some pizza! Doubling experiments provide good insight on program performance • Best practice to plan realistic experiments for debugging. Still didn't finish. anyway. too. many situations. • Having some idea about performance is better than having no idea. I'm going to run long to finish. I'm going a doubling experiment. • Performance matters in many. The experiment showed my program to have a higher order of growth than I expected.Meeting the challenge My program is taking too Mine. 35 . I found and fixed the bug.

Caveats It is sometimes not so easy to meet the challenge of predicting performance. leading term oscillates? Your machine model is Where’s the log factor? too simple: My computer has parallel processors and a cache. a(2N)b (lg(2N))c 1 c = 2b 1 + aNb (lg N)c (lg N) 2b Good news. 36 . Doubling method is robust in the face of many of these challenges. We need more terms in the There are many other apps math model: N lg N + 100N ? running on my computer ! Your input model is too simple: My real input data is What happens when the completely different.

org/detail/169320/girl-head-by-jza https://openclipart. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA Image sources https://openclipart.Performance.Doubling .org/detail/191873/manga-girl---true-svg--by-j4p4n-191873 CS.7.D.org/detail/25617/astrid-graeber-adult-by-anonymous-25617 https://openclipart.

COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA 7.E.Examples .Performance.7. Performance • The challenge • Empirical analysis • Mathematical models • Doubling hypothesis • Familiar examples CS.

4. % java Gambler 4000 8000 100 } 55 wins of 100 (56 seconds) } % java Gambler 8000 16000 100 53 wins of 100 (286 seconds) A.5) t++.8 million seconds (about 2 months).parseInt(args[1]). 32000 1172 × 4 = 4688 4 int wins = 0. i < trials.0f seconds)\n".0.0. { int t = stake.currentTimeMillis()/1000.parseInt(args[0]).random() < 0. % java Gambler 2000 4000 100 StdOut. now . % java Gambler 16000 32000 100 48 wins of 100 (1172 seconds) 39 .start). order of growth } % java Gambler 1000 2000 100 should be N 2 if (t == goal) wins++.Example: Gambler’s ruin simulation Q. 52 wins of 100 (17 seconds) StdOut..10 int goal = Integer. math model says else t--. 1024000 1172 × 46 = 4800512 4 while (t > 0 && t < goal) { if (Math.25 { public static void main(String[] args) 4000 56 3. for (int i = 0. 16000 1172 4.print(wins + " wins of " + trials). How long to compute chance of doubling 1 million dollars? N TN TN/TN/2 1000 4 public class Gambler 2000 17 4.currentTimeMillis()/1000. i++) .09 ✓ double start = System.parseInt(args[2]).29 { int stake = Integer. 8000 286 5. int trials = Integer..printf(" (%. 53 wins of 100 (4 seconds) } double now = System.

4000 80 4 .parseInt(args[0]). 40 ..Pop quiz on performance Q. Q. } 8000 320 4 } Q.. Predict the running time for N = 64.. Estimate the order of growth.. int N = Integer. Let TN be the running time of program Mystery and consider these experiments: N TN (in seconds) TN/TN/2 public class Mystery { 1000 5 public static void main(String[] args) { 2000 20 4 .000.

Let TN be the running time of program Mystery and consider these experiments. int N = Integer. 64000 5120 × 4 = 20480 4 A.. since lg 4 = 2. N 2. Estimate the order of growth.000. 20480 seconds.Pop quiz on performance Q... N TN (in seconds) TN/TN/2 public class Mystery { 1000 5 public static void main(String[] args) { 2000 20 4 . 41 . Predict the running time for N = 64.parseInt(args[0]). } 8000 320 4 } 16000 320 × 4 = 1280 4 32000 1280 × 4 = 5120 4 Q. A. Q.. 4000 80 4 .

StdOut. N*Math. int trials = Integer. } should be N logN } } double now = System. for (int i = 0. math model says cardcnt++.0.0f seconds)\n".printf(“%d %.0f “. % java Collector 250000 100 } 250000 3251607 3173727 (14 seconds) } % java Collector 500000 100 500000 6849787 6772679 (31 seconds) might run out of memory % java Collector 1000000 100 A. 1000000 31 × 2 = 63 2 while (valcnt < N) { int val = (int) (StdRandom() * N).print(cardcnt/trials). int cardcnt = 0. How long to simulate collecting 1 million coupons? N TN TN/TN/2 125000 7 public class Collector { public static void main(String[] args) { 250000 14 2 int N = Integer. trying for 1 billion 1000000 14392721 14368813 (66 seconds) ✓ 42 . double start = System. % java Collector 125000 100 StdOut. if (!found[val]) order of growth { valcnt++. About 1 minute.21 ✓ { int valcnt = 0. 125000 1539160 1518646 (7 seconds) StdOut. found[val] = true.parseInt(args[1]).57721*N). N.log(N) + .start). i++) 500000 31 2. now .printf(" (%. i < trials.parseInt(args[0]). boolean[] found = new boolean[N].Another example: Coupon collector Q.currentTimeMillis()/1000.0.currentTimeMillis()/1000.

1 gigabyte (GB) is about 1 billion bytes. 1 megabyte (MB) is about 1 million bytes.Analyzing typical memory requirements A bit is 0 or 1 and the basic unit of memory. 43 . A byte is eight bits — the smallest addressable unit. 2000-by-2000 double array uses ~32MB. System-supported data structures (typical) Primitive-type values type bytes type bytes boolean 1 Note: not 1 bit int[N] 4N + 16 char 2 double[N] 8N + 16 int 4 int[N][N] 4N 2 + 20N + 16 ~ 4N 2 float 4 double[N][N] 8N 2 + 20N + 16 ~ 8N 2 long 8 double 8 String 2N + 40 Example.

Summary Use computational experiments. Example: Solve 3-sum efficiently 44 . Q. What if it's not fast enough? A. and the scientific method 
 to learn whether your program might be useful to solve a large problem. does yes no it scale? buy a new computer and learn a better algorithm solve bigger problems found yes no one? invent a better algorithm Plenty of new algorithms awaiting discovery. mathematical analysis.

does Larry Page and Sergey Brin yes no it scale? buy a new computer and solve bigger problems learn a better algorithm maybe changing found yes not yet no the world? one? PageRank
 (see Section 1. 2 CS grad students had a program to index and rank the web (to enable search).6) yes invent a better algorithm Lesson. Performance matters! 45 .Case in point Not so long ago.

org/wiki/File:Google_page_brin. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA Image source http://en.wikipedia.E.7.Performance.jpg CS.Examples .

edu .1 http://introcs. Performance Science An Interdisciplinary Approach ROBERT SEDGEWICK K E V I N WAY N E Section 4. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I : P R O G R A M M I N G I N J AVA Computer Science Computer 7.cs.princeton.