## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

1

Insertion Sort

Insertion-Sort(A)

sort an array A[1..n] containing a sequence of length n (length[A])

input array A contains the sorted output sequence when finished

**input numbers are sorted in place
**

the numbers are arranged within the array A, with at most a constant number of them stored outside the array at any time

2

(i.e., n)

3

**Pseudo Code Notations
**

Liberal use of English

employs whatever expressive method is most clear and concise to specify a given algorithm

algorithms are expressed to humans, not to computers

Use of indentation for block structure Omission of error handling and other such things needed in real programs

not concerned with issues of software engineering in order to convey the essence of the algorithm more concisely

4

Loop Invariants

Technique for proofs of the correctness of algorithms Conditions and relationships that are satisfied by the variables and data structures at the end of each iteration of the loop Established by induction on the number of passes through the loop

similar to mathematical induction

to prove a property holds, prove a base case and an inductive step

5

**Properties of Loop Invariants
**

Initialization

it is true prior to the first iteration of the loop

Maintenance

if it is true before an iteration of the loop, it remains true before the next iteration

Termination

when the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct

6

**Loop Invariants of Insertion Sort
**

Initialization

show that the loop invariants holds before the first loop iteration

j = 2 ⇒ A[1 .. j - 1] = A[1] is in sorted order

Maintenance

show that each iteration maintains the loop invariant

at the start of each iteration of the ”outer” for loop, the subarray A[1 .. j - 1] consists of the elements originally in A[1 .. j - 1] but in sorted order

Termination

examine what happens when the loop terminates

j = n + 1 ⇒ A[1 .. j - 1] = A[1 .. n]

7

Analyzing Algorithms

To predict the resources that the algorithm requires

resource examples

memory, communication bandwidth, computer hardware, computational time (measured most often)

Computational model

random-access machine (RAM) model

instructions are executed one after another

each instruction takes a constant amount of time

**no concurrent operations data types are integer and floating point
**

the size of each word of data does not grow arbitrarily

**no caches or virtual memory
**

no memory-hierarchy effects

8

Primitive Instructions

Basic computations performed by an algorithm

exact definition is not important assumed to take a constant amount of time in the RAM model

Examples

arithmetic: add, subtract, multiply, divide, remainder, floor, ceiling), shift left/shift right data movement: load, store, copy control: conditional/unconditional branch, subroutine call and return

9

Running Time

The number of primitive operations (steps) executed

steps are machine-independent each line of pseudo code requires a constant amount of time.

one line may take a different amount of time than another, but each execution of line i takes the same amount of time ci

10

Running Time

Depends on

input size

the time generally grows with the size of the input e.g., 6 elements vs. 6000 elements

input itself

may take different amounts of time on two inputs of the same size e.g., sorted vs. reversely sorted

11

**Running Time Analysis
**

Worst case

maximum time on any input of size n usually used

Average case

average time over all inputs of size n occasionally used

Best case

minimum time on any input of size n hardly used

12

execute tj times for iteration j

Running Time: depends on the values of tj

• tj : the number of times the while loop test in line 5 is executed for the value of j

13

**Best Case of Insertion Sort
**

Array already sorted

always A[i] ≤ key upon the first time the while loop test is run (when i = j - 1) all tj are 1

∑ t j = ∑1 = n − 1

j =2

n

n

n

j =2

∑t

j =2

j

− 1 = ∑1 − 1 = 0

j =2

n

14

Best Case Running Time

=

an + b for constants a and b (that depend on the statement costs ci ) ⇒ T(n) is a linear function of n

15

**Worst Case of Insertion Sort
**

Array in reverse sorted order (i.e., decreasing order)

always A[i] > key in while loop test compared with j - 1 elements one additional test after the j - 1 tests ⇒ tj = j

∑t j = ∑ j =

j =2 n j =2 n j =2 j =2

n

n

n(n + 1) −1 2 n(n − 1) 2

16

∑t j −1 = ∑ j −1 =

**Worst Case Running Time
**

∑j=

2 n

n( n + 1) −1 2

∑ ( j − 1) =

2

n

n( n − 1) 2

= an2+bn+c for constants a, b and c (that depend on the statement costs ci ) ⇒ T(n) is a quadratic function of n

17

**Worst-Case Analysis
**

Worst-case running time is usually preferred

the longest running time for any input of size n

Reasons

the worst-case running time is an upper bound on the running time for any input for some algorithms, the worst case occurs fairly often

e.g., the worst case of searching often occurs when

the item being searched for is not present searches for absent items may be frequent

**the average case is often roughly as bad as the worst case
**

18

Order of Growth

Use abstraction to ease analysis and focus on the important features

consider only the leading term of formula and drop lower-order terms

e.g., an2 instead of an2 + bn + c

**ignore the constant coefficient in the leading term
**

e.g., n2 instead of an2

**use Θ-notation as the running time
**

e.g., Θ(n2) ⇒ the worst-case running time T (n) grows like n2, but it does not equal n2

19

Algorithm Efficiency

One algorithm is considered to be more efficient than another if its worst case running time has a smaller order of growth

the evaluation may be in error for small inputs

due to constant factors and lower order terms

**the evaluation will hold for large enough inputs
**

e.g., Θ(n2) algorithm will run more quickly in the worst case than a Θ(n3) algorithm

20

Designing Algorithms

Incremental approach

e.g., insertion sort, having sorted the subarray A[1 .. j - 1], inserts the single element A[j] into its proper place, yielding the sorted subarray A[1 .. j]

**Divide and conquer approach
**

e.g., merge sort recursively sorted two sequences and merge the two sorted sequences into a bigger sorted sequence

21

**Divide and Conquer Paradigm
**

Divide the problem into a number of subproblems

the subproblems are similar to the original problem but smaller in size

**Conquer the subproblems by solving them recursively
**

if the subproblem sizes are small enough, however, just solve the problems in a straightforward manner

**Combine the solutions to the subproblems into the solution to the original problem
**

22

**Merge Sort Approach
**

Divide

divide the n-element sequence to be sorted into two subsequences of n/2 elements each

Conquer

sort the two subsequences recursively using merge sort

Combine

merge the two subsequences to produce the sorted answer

23

**Merge Sort Procedure
**

State each subproblem as sorting a subarray A[p . . r]

initially, p = 1 and r = n the values of p and r change as recurring through subproblems

To sort A[p . . r]

divide by splitting into two subarrays A[p .. q] and A[q + 1 .. r], where q is the halfway point of A[p .. r] conquer by recursively sorting the two subarrays A[p .. q] and A[q + 1 .. r]

the recursion bottoms out when the subarray has just 1 element, which is trivially sorted

**combine by merging the two sorted subarrays A[p .. q] and A[q + 1 .. r] to produce a single sorted subarray A[p .. r]
**

procedure MERGE(A, p, q, r)

24

**Merge Sort Pseudocode
**

Initial call: MERGE-SORT(A, 1, n)

A = 〈A[1], A[2], …, A[n]〉

25

**Bottom View of Merge Sort
**

n=8 n = 11

26

Merging

Input: Array A and indices p, q, r such that

p≤q<r subarray A[p .. q] is sorted and subarray A[q + 1 .. r] is sorted

by the restrictions on p, q, r, neither subarray is empty

**Output: the two subarrays are merged into a single sorted subarray in A[p .. r] Running time: Θ(n) time
**

n = r - p + 1 = the number of elements being merged

27

Θ(1) → Θ(1) → Θ(1) →

{ Θ(n ) {

Θ(n1)

2

Θ(n1 + n2) = Θ(n)

Θ(1) → Θ(1) → Θ(1) → Θ(1) → Θ(n)

}

sentinel value

28

Example

A call of MERGE(A, 9, 12, 16)

29

Example(cont.)

30

**Loop Invariant for Merge Sort
**

At the start of each iteration of the for loop of lines 12-17, the subarray A[p .. k-1] contains the k - p smallest elements of L[1 .. n1+1] and R[1 .. n2+1], in sorted order L[i] and R[j] are the smallest elements of their arrays that have not been copied back into A

31

**Loop Invariants of Merge Sort
**

Initialization

show that the loop invariants holds prior to the first iteration of for loop

k = p ⇒ A[p .. k - 1] is empty, k - p = 0 elements i=j=1

Maintenance

show that each iteration maintains the loop invariant

case L[i] ≤ R[i] case R[i] ≤ L[i]

Termination

examine what happens when the loop terminates

k = r + 1 ⇒ A[p .. j - 1] = A[p .. r]

32

**Analyzing Divide-andConquer Algorithms
**

Use a recurrence equation (more commonly, a recurrence)

describes the overall running time on a problem of size n in terms of the running time on smaller inputs use mathematical tools later to solve the recurrence and provide bounds on the performance of the algorithm a recurrence for the running time of a divide-andconquer algorithm is based on the three steps of the basic paradigm

33

**Running Time of Divideand-Conquer Algorithms
**

Let T(n) = running time on a problem of size n

if the problem size is small enough (say, n ≤ c for some constant c), we have a base case

the straightforward solution takes constant time: Θ(1)

**otherwise, suppose that we divide into a subproblems, each 1/b the size of the original (in merge sort, a = b = 2)
**

there are a subproblems to solve, each of size n/b ⇒ each subproblem takes T(n/b) time to solve ⇒ solving a subproblems takes aT(n/b) time

**let the time to divide a size-n problem be D(n) let the time to combine solutions be C(n)
**

34

Recurrence of Divide-andConquer Algorithms

aT(n/b) : the time to solve a subproblems D(n) : the time to divide a size-n problem D(n) C(n) : the time to combine solutions

35

**Analyzing Merge Sort
**

Assume that n is a power of 2 for simplicity

each divide step yields two subproblems, both of size exactly n/2

When n = 1

the base case occurs and takes constant time ⇒ Θ(1)

When n > 1

divide: just compute q as the average of p and r ⇒ D(n) = Θ(1) conquer: recursively solve 2 subproblems, each of size n/2 ⇒ aT(n/b) = 2T(n/2) combine: MERGE on an n-element subarray takes Θ(n) time ⇒ C(n) = Θ(n)

36

**Recurrence of Merge Sort
**

⇐ T(n) ⇐ Θ(1) ⇐ Θ(1) ⇐ D(n) ⇐ T(n/2) ⇐ T(n/2) ⇐ Θ(n) ⇐ C(n)

Recurrence of the running time

D(n) + C(n) = Θ(1) + Θ(n) = Θ(n)

37

**Worst-Case Running Time of Merge Sort
**

By the master theorem in Chapter 4, it can be shown that this recurrence has the solution T(n) = Θ(n lg n) Merge sort is faster than insertion sort (Θ(n2) worst-case time)

on small inputs, insertion sort may be faster for large enough inputs, merge sort will always be faster

38

**Solving the Merge-Sort Recurrence
**

Let c be a constant that describes the running time for the base case as well as the time per array element of the divide and combine steps Rewrite the recurrence as

Draw a recursion tree, which shows successive expansions of the recurrence

39

Recursion Tree

40

**Recursion Tree (cont.)
**

lg n + 1 levels

each level has total cost cn

**total cost of all levels cn (lg n + 1) = cn lg n + cn ⇒ Θ(n lg n)
**

41

- CSCI2100 07 Sortinguploaded byenergy0124
- Node Ranking 1uploaded byV.m. Chitra
- lecture17(Sortring)uploaded byavinash
- Linear Gate Assignment: A Fast Statistical Mechanics Approachuploaded byal9028
- binary tressuploaded byArpita Sengupta
- 2152.pdfuploaded byLivia Marsa
- Lecture-14-CS210-2012 (1).pptxuploaded byMoazzam Hussain
- Made Easy Online Test Series 1uploaded byANKIT
- 02-analyuploaded byLinh Do
- Analysis of Algorithms Iuploaded bySerge
- On the Hardness of Being Truthful Schapira-harndess-cpppuploaded byDexter Uman
- exam2Auploaded byvirpara
- hw1uploaded byasteinb1
- Algoritmos de aproximación para problemas combinatoriosuploaded byJuan Sebastián Poveda Gulfo
- 1-s2.0-S095741741300242X-mainuploaded bySantiago Cardenas
- Programing Assignmentuploaded bychs360
- Report on Sortinguploaded byRajan Padalia
- Sortinguploaded bySandip Das
- Algorithms CSuploaded byalexia_roberta123
- C and Data Structureuploaded byUnmesh Mandal
- csec mayjune2012 informationtechonology sr 1uploaded byapi-247871582
- Devise, Extend and Upgraded Algorithm of Gene Sequence for Local Alignmentuploaded byijecs
- A Novel Solution Approach to Capacitated Lot-Sizing Problems with Setup Times and Setup Carry-Over.uploaded byPaul Sandu
- Fast Joins Using Join Indices1uploaded bymaxitis
- Know Thy Complexitiesuploaded bybhuvangates
- 10 Reconﬁguring Process Plansuploaded byAkatew Haile Mebrahtu
- Big-O Algorithm Complexity Cheat Sheetuploaded bykaushik5295
- CSEC MayJune2012 InformationTechonology.sruploaded bydextonslab7
- pp.163-175uploaded byHANBALCONTRAMUNDUM
- A Beginner’s Guide to Big O Notation « Rob Bell.pdfuploaded byJubril Akinwande

- Lecture03 Fourier Representations of Signalsuploaded bychw2054
- GMap專題口試報告uploaded bychw2054
- Downloaduploaded bychw2054
- Thesis Writ Tinguploaded bychw2054
- 99博士班入學考試排程uploaded bychw2054
- Lecture02 LTI Time-Domain Representationsuploaded bychw2054
- 011uploaded bychw2054
- C Reference Card (ANSI) 2.2uploaded bychw2054
- 這樣記憶最有效uploaded bychw2054
- G2930Tanguploaded bychw2054
- 數位邏輯設計 chap01 2005uploaded bychw2054
- [電子書][教學]魔術教學uploaded bychw2054
- 兩性關係之探討970307uploaded bychw2054
- 這樣記憶最有效uploaded bychw2054

Close Dialog## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

Loading