You are on page 1of 87

Concepts Covered

 Arrays
 Array Lists
 Linked Lists
 Analysis of Algorithms
 Stacks
 Queues
o Priority Queues
 Recursion
 Trees
 Binary Search Trees
 Heaps
 Priority Queues
 Hash Maps
 Skip Lists
 AVL Trees
 B – Trees
 Splay Trees
 Sorting
o Bubble Sort
o Cocktail Shaker Sort
o Insertion Sort
o Selection Sort
o Merge Sort
o Quick Sort
o K – Select (Quick Select)
o LSD Radix Sort
o MSD Radix Sort
 Iterators
 String Searching
o Brute – Force
o Boyer – Moore
o KMP
o Rabin – Karp
 Graphs
o Depth – First Search
o Breadth – First Search
o Dijkstra’s Algorithm
o Prim’s Algorithm (MST)
o Kruskal’s Algorith
 Dynamic Programming
o Least Common Subsequence (LCS)

Arrays and Array Lists
Arrays
Array Definition
 A sequenced collection of variables of the same type (Homogeneous)
 Every cell has an index denoting its location in the array
o The cells are numbered starting at 0, 1, 2, …
 Each value stored in an array is called an element:

Array Length and Capacity:
 The length of an array determines the max number of items it can store, it is it’s capacity
 This length is established when the array is created, it is fixed
 This length can be accessed with a.length where a is the array
o Note the cells are numbered 0, 1, 2, …, a.length – 1
o You can access cell at index k with a[k]

Array Creation
 You can use an array literal or an array declaration to create an array
o Element type for array is any Java base type or class type
 E.g. Capacity of N = 4
o Array literal: int[] myArray = {1, 2, 3, 2}
o Array declaration: int[] myArray = new int[4]

Arrays of Character or Object References
 An array can store primitive elements, like characters or references to objects

Array Lists
Array Lists
 An array can be used as a backing structure for a list
 An array list stores objects, not primitive data types
o Arrays can store both
 When an array list is full, it dynamically resizes to a larger array list
o A new, larger backing array is created and the content is copied from the old to
the new array
 Resizing policy depends on implementation, Java resizes it to 1.5 times the original size

Adding an Entry
 To add an entry into myArray at index I, we have to shift forward the n-i entries
 E.g. Say myArray has capacity 16 and its size is n = 10; we add an element at i = 6, so we
must shift elements 6-9 to positions 7-10

Adding an Entry

Removing an Entry
 To remove an entry e at index t, we need to fill the hole left by e by shifting backward the
n – i – 1 elements: board[i + 1], …, board[n – 1]
 E.g. Say myArray has capacity 16 and it’s size is n = 11; we remove an element at i = 6,
so we must shift elements 7-10 to positions 6-9

Removing an Entry

Array List Summary and Complexity
 Array lists are dynamic and store objects
 Big Os
o Accessing: O(1)
o Inserting, searching or removing: From anywhere other than the back is O(n)
 Array lists are used in tracking characters in online game maps

Notes From CodeFights
 Static Arrays:
o Strength of static arrays:
 Item look up is quick
 Low memory footprint
o Weaknesses:
 Have to be implemented in contiguous memory
 Adding and deleting is O(n)
 Don’t allow for quick rearrangement
 Searching is O(n)
 Dynamic Arrays:
o Logical Size: number of elements used in array
o Physical Size: Size of the actual array (how many elements it can hold total)
o Note: Appending (adding at end) is amortized O(1), deleting from end is O(1),
deleting and adding in general are O(n) as expected
 Multi-Dimensional Arrays:
o Jagged Array: The length of each row isn’t necessarily the same
o Regular Array: rectangular 2D array

Linked Lists Linked List  A data structure consisting of a sequence of nodes  Each node in the linked list stores the data and a pointer to the next/previous nodes  Nodes are created/destroyed as necessary when adding and removing  A head pointer is usually present and points to the first node o Tail pointer is optional  LLs are good for manipulating data o Array lists are better for storing/accessing data Singly Linked List  A version of a linked list where each node only has a pointer to the next node (not previous ones) o It starts from the head pointer  Each node stores: o The element o Pointer to the next node  E. On the right  The next reference of the last node points to null: Inserting at the Head  Allocate a new node with data and next reference  Have the new node’s next reference point to the old head  Update the head reference to point to the new node .g.

Inserting at the Tail  Allocate a new node with data and next reference  Have new node’s next reference point to null  Have the last node’s next reference point to the new node  Update the tail if there is a tail pointer to point to the new node Removing at the Head  Update head to point the next node  Allow garbage collector to reclaim the former first node Removing at the Tail  Removing the tail of a singly linked list is NOT efficient! There is no constant time way to update it .

 Iterate to the node just before the last node  Update the tail pointer if it exists  Set the current node’s next pointer to null Pseudo – Codes .

Doubly Linked List  A linked list where each node has a next and a previous pointer  Can be traversed forward and backward  Each node has: o Element o Link to next node o Link to previous node Generic Code for Doubly Linked List Doubly Linked List .

… … Linked List Summary and Complexity  LLs are dynamic and can store primitives and objects  Big Os: o Accessing/Searching: O(n) o Adding/Removing to front if there is a head pointer: O(1)  Useful when you need to manipulate data  Singly linked lists are used in RFID scanning. round-robin scheduling algorithms for operating systems  E.g. scroll bars or forward/backward buttons Circular Linked List  The last node points to the first node (and first can point to last o Either singly or doubly linked  Big O depends on linking. Singly and doubly circular linked lists Notes from CodeFights  Advantages: o Can be resized dynamically (size is only limited by amount of memory  Weaknesses: o Can require more space in memory than arrays  Use them when you don’t know the number in the list before you start or if the list will need to grow . app wizards or levels in video games  Doubly linked lists are used in browser history. Big O values will be the same as singly or doubly linked equivalent  Used in gifs. music playlists.

and best case runtime is not considered  We focus on the worst case runtime (Big – O) o Easier to analyze and approximate Growth of Functions  Seven functions describe growth of functions that often appear in algorithm analysis o Constant = 1 o Logarithmic = log(n) o Linear = n o N-Log-N = n*log(n) o Quadratic = n2 o Cubic = n3 o Exponential 2n  The Big-O of a function is usually described using one of these functions . Analysis of Algorithms Running Time  The runtime is the period during which the computer is executing. and typically grows with input size  Average case duntime may be difficult to determine.

Primitive Operations  Basic computations performed by an algorithm  Indentifiable in pseudocode  Largely independent from the programming language  These functions are considered to run in O(1)  Examples o Evaluating an expression o Assigning a value to a variable .

lines 6-8 may be executed 0 to n times. there are other factors . the time to do a certain number of operations in seconds with a function of a certain Big-O might be as follows o Assume that only one operation is done for each input Notes on Big-O  The Big-O of a function only gives runtime of a function with respect to input size o A function that is O(n2) may run faster than O(n) for some input sizes  Given the Big-O of a function. o Indexing into an array o Calling a method o Returning from a method o Returning from a method Estimating Running Time  Given the algorithm below. you cannot just calculate the time it will take for the function to run an input of certain size. all other lines will be executed once  So the big O is O(n) Why Big-O Matters  Given a processor that can do 100 operations per second.

or general idea. add to. Stack Stack  A stack is a data structure where insertions and deletions are LIFO (last-in first out) o The last object added is the first object returned  You can only access the object at the top of the stack o You cannot access. without a specific implementation o They are an Abstract Data Type (ADT) o Has multiple implementations  Main operations: o Push(object): inserts an element o Object pop(): removes and returns the last inserted element  Auxillary stack operations o Object top(): returns the last inserted element without removing it o Integer size(): Returns the number of elements stored o Boolean isEmpty(): indicates whether no elements are stored  Example: Array-Backed Stack  A simple way of implementing a stack uses an array  Add elements from left to right and remove from right to left  A variable keeps track of the index of the top element . of how the data structure functions. or remove from any other point in the stack  Stacks are an abstraction.

then 2 will come off: Pseudo – Code: Linked List-Backed Stack  A linked list could also be used to back a stack  Elements are added and removed to the same end of the list o Usually it is most efficient to and and remove from the front of the linked list: O(1)  E. the implementation may throw an exception or resize the backing array o This behavior is not defined or forced by the ADT  E. size is 4. if you pop.g. o Or a variable representing the size of the stack can be used  If the array storing the stack is full and an element is added. the size of the stack is 4. an array implementation for a Stack ADT where the top of the stack is index 3. The top of the stack is at the head.g. if you pop the number 2 will come off: Performance and Limitations  Performance o Let n be the number of elements in the stack o The space used is O(n) .

 Big Os: o Push(obj). if the array is to be resized when adding. size(): O(1)  Limitations o For an array-backed stack. So the push operation is amortized O(n) o For LL backed stack. initial maximum size must be defined before hand o For an array-backed stack. isEmpty(). top(). pop(). it is O(n). these two limitations don’t exist Applications of Stacks  Direct Applications: o Page-visited history in a Web Browser o Undo Sequence in a text editor o Chain of method calls in many programming languages  Indirect applications: o Auxillary data structure for algorithms o Component of other data structures Activation Stack … .

queues are an ADT with more than one implementation  Main operations: o enqueu(object): inserts an element at the end of the queue o object dequeuer(): Removes and returns the element at the front of the queue  Auxillary queue operations: o Object first(): Returns the element at the front without removing it o Integer size(): Returns the number of elements stored o boolean isEmpty(): indicates whether no elements are stored  E. remove or access from any other point in the queue  Like stacks. array location back = (front + size) modN is the first empty slot past the rear of the queue  Example of queue using “wraparound” technique . Queues Queue  A data structure where insertions and deletions are FIFO (first-in first-out) o The first object you add in the queue is the first one removed  Insertions are at the rear of the que and removals at the front o You do not add. Array – Backed Queue  Use an array of size N in a circular fashion  Two variables keep track of the front and size o Front: index of the front element o Size: number of stored elements  When the queue has fewer than N elements.g.

I think you should just null it out Linked List-Backed Queue  A linked list could be used as the backing data structure of a queue  Elements would be added and removed from opposite ends o The most efficient implementation depends on the list Performance  Let n bet he number of elements in the queue o The space used is O(n) o Each operations runs in O(1) time .Pseudo – Codes:  Note that the following does not account for resizing  Node that the following does not null out the element it dequeues (which may be a problem if you attempt to “dequeuer” an empty queue).

this is O(n) o But enqueuer() itself is Amortized O(1) o When resizing. the queue would be unwound.Limitations  For an array-backed queue. if the backing array needs to be resized. so the front of the queue would be in index 0 of the resized array  Linked list-back queues do not have these problems Applications of Queues:  Direct Applications o Waiting lists o Access to shared resources o Multithreading  Indirect Applications o Auxillary data structure algorithms o Component of other data structures Round Robin Schedulers … . the initial maximum size must be decided before hand  For an array-backed queue.

Recursion Recursion definition  Recursion is a programming process where a method calls itself repeatedle until it reaches a defined point of termination Recursive Example:  The factorial function: o N = 1 * 2 * 3 * … * (n – 1) * n o Recursive definition: o As a Java method Construction of a Recursive Method  Recursive method will have statements to check for termination and recursive calls that advance a parameter towards a base case  Base case(s): o Value of the parameter for which no recursive call is made o Every recursive method must have at least one base case o Every possible chain of recursive call must eventually reach a base case  Recursive calls o Explicit calls to the current method o The call must advance towards the base case Visualizing Recursion  Recursion trace: o A box for each recursive call o An arrow from each caller to callee o An arrow from each callee to caller showing the return value  E. Factorial .g.

Searching for 19: .g. For binary search Binary Search Example  We consider three cases o If the target equals data[mid] o If the target < data[mid] then we recursively call the first half o …  E.Binary Search Method  E.g.

avoiding inefficiencies o Tail recursion is when every recursive method call is the last step before returning from the method o There is no need to store anything on the call stack because the result is returned to the original method call o Such methods might receive optimization benefits at runtime  Not in Java though. ..… … … Tail Recursion  A special form of recursion that can be interpreted as iteration.

Trees What is a Tree?  An abstract model of a hierarchal structure  Has nodes with parent-child relationship  Applications: o Organization charts o File systems o Programming environments  Linked Lists are trees!  Trees are limited graphs with no cycles. (THIS SEEMS TO BE IMPLEMENTATION DEPENDENT)  Subtree of a node: Tree consisting of a node and its descendants . depth of the root is 1. etc. root’s children have depth 2. highly recursive Example Trees  We assume that there are no duplicates in our trees: Terminology  Root Node: …  Internal Node: …  Leaf/External node: …  Parents of a node: …  Ancestors of a node: …  Children of a node: …  Descendants of a node: …  Height of a node: Maximum number of nodes traversed to reach a leaf node. height of a leaf node is 0  Depth of a node: Max number of nodes traversed to reach the root.

decision processes. searching. etc. searching Properties of Binary Trees  . Binary Search Tree  BSTs are a special case of a Binary tree where the left child is smaller than the parent and the right child is bigger  Same goes with the children’s children ..A Note on Coding  Trees are a recursive data structures: the left child of the root is a tree of it’s own (so is the right child obviously)  A lot of operations on trees (adding. removing. Binary Tree ADT  The Binary Tree ADT extends the Tree ADT and inherits all it’s methods  …  .) are thus done recursively Binary Trees  Each node in a binary tree has at most two children that are an ordered pair  Applications are arithmetic expressions..

then the right subtree  ONLY FOR BINARY SEARCH TREES o In which case it visits the elements in ascending order . you can perform a preorder. which traverses the data of the tree in ascending order Pre-order Traversal  A node is visited first. then the node itself. you can also do an inorder traversal. or level-order traversal o For BST’s. then right) Post-order Traversal  The children are visited first (left. then right).Tree Traversal  A traversal visits the nodes of a tree in a systematic manner  For all trees. then the parent node itself In-order Traversal  The left subtree is visited first. then the child nodes are visited (left. postorder.

Doing the Traversals on Paper  Draw three spokes and then a line from the top left around and back to the top right  … Traversal Pseudocode .

.

and you need to get the final subtree o You’re basically assuming that the recursive call will always return what the new subtree should be o When you return the node in the method. the class representing a node in the tree may not have a reference to the parent node  It is good practice to use pointer reinforcement so that the new left/right nodes are updated o When making a recursive call to the left/right child. the structure of the subtree might change. Subtree Update Method  When adding or removing from a BST. make sure you return the new left/right child of the parent Searching …straightforward… .g. pretend you have a subtree there o After the call. Binary Search Tree Idea of Binary Search Trees  Arrays or lists can be used to store data items in ascending or descending order  Binary search can then be used to search for a particular item in O(log(n)) time  The idea behind binary search is turned into a data structure: the binary search tree (BST)  E.

add a new node containing the data as either the left or right child of the leaf node  If the data is in the tree. then just remove it and the parent will no longer have a left/right child 2. If the node has no children. you are at a leaf node). then what happens is implementation defined … Removing  Follow the same steps as searching until you find the data in the tree or go off the tree (in which case the data is not in it)  To remove a node: 1.Adding  Follow the same steps as searching until you find the data in the tree or reach a leaf node  If the data is not in the tree (i. then you need to get the predecessor or successor to replace the current node  The predecessor is …  The successor is … … . then that child node takes the place of this node. If the node has one child. If the node to be removed has two children.e. the parent node’s left/right child is now the current nodes child 3.

O(n) CodeFight Notes … . O(log(n))  Worst case: If the BST is degenerate (linked list).Performance  Best case: Search can be O(1). Insertion and deletion are O(log(n))  Average case: BST divides data in half at every level.

we use a binary heap  Heaps are a binary heap where the root is the largest or smallest element o Maxheap and minheap respectively  Can also be represented as an array  E.g. so we can get the largest or smallest item quickly  Can be trivially done with a list but insertion is O(n) o Instead. at any point. There is no relationship between the children o Shape Property: A heap is always complete. the right child would be at index 2i + 1  The parent of an item at index I would be at i/2 (if you do integer division because truncation)  E. each level of the tree is completely filled except for the last level which is filled from left to right  In addition. the parent node is always larger (or smaller) than its children. Properties of a Heap  Heaps have two properties: o Order Property: In a heap. . only the root item of a heap can be accessed o YOU CANNOT SEARCH THROUGH A HEAP Heaps as an Array  The “root” of the heap is at index 1 (we skip index 0)  The left child of an item at index i would be at index 2i.g. Heaps and Priority Queues Heaps  A way of keeping a set of items in ascending or descending order.

then swap the parent and child o This is called heapify. if the order property is brokeN. we’ll fix it  Compare the newly added item to the parent. which may break the order property but it’s okay.Adding  Add the new item into the next open slot in the tree/array. the algorithm is recursive  Repeat the previous step until o You don’t make any swaps OR o You reach the root of the heap … Removing  YOU CAN ONLY REMOVE FROM THE ROOT .

 Remove the root and replace it with the last slot of the heap. returns the items in ascending/descending order (depending on implementation)  … . OK  Compare the item with the left and right children. when given some items. which may break the order property. if either of them is smaller/larger (depending on max/minheap). swap the item with the smaller/larger of the two children  Repeat until you don’t make swaps or reach the bottom of the heap Performance  Best case: Adding to a heap could be O(1) if you add items in ascending order (for minheap) or descending order (for maxheap) because the item you just added will never get promoted  Average/Worst: Adding to a heap will be O(log(n)) because you need to promote the item you just added o Removing would be O(log(n)) because the item that gets moved to the root may have to go down log(n) levels Priority Queue  A priority queue is a data structure that.

where n is the old length  When resizing. each key is unique but there may be duplicate values (some implementations allow duplicate keys but I don’t think we go over that)  Maps can easily be implemented with either an array or a linked list o Both result in O(n) search performance. searches are usually O(1)  Keys are unique but there may be duplicate values  The length of the backing array tends to be a prime number o Somehow. but the calculation on its own may give an index which is really large or negative  So you reduce this hash code to the length of the backing array using the modulo (%) operator o a%b will always return a value between 0 and b – 1 o That way you always have a valid index Load Factor  The load factor of a hash map determines when it needs to be resized  The load factor is calculated by dividing the number of key-value pairs in the array by the array length  If there are 8 items in an array of length 11. but you can bring this down to O(log(n)) depending on your key  The hashing is added to the map to determine the index in which an object goes o So in the ideal case searching becomes O(1) because you know the index it should be at  Hash maps are usually implemented using an array (so you have O(1) access to any index) Index Calculation  The hash code first determines what index the key-value pair should go into. then the LF is 8/11 = 0. an example formula would be 2n+1. Hash Maps  A map is a data structure where each key has a single value associated with it  Usually. this makes it less likely that items that collided in the smaller map will still collide in the bigger one Collision Resolution Methods . the backing array is resized o The new size is implementation defined.75 Properties of Hash Maps  Since Hash Maps use the hash code of the key to find the associated value.727272… (regardless of the collision resolution method  When the LF exceeds a certain threshold. the individual index of each key-value pair needs to be recalculated (since it is based on the length of the array) o DO NOT just copy all the old items back in the same index o A common load factor limit is 0.

then you’ve found it! Awesome o If it’s not. so you check 5. without modding may be the same for two different objects Linear Probing  If index i supposed to be used but is already taken. check i+2.  Because of the modding. calculate the index where the key should be  If the key is at that index. once you find the key. do not just set that index to NULL o This may break searching for a key that had to be placed somewhere besides it’s ideal index o Instead. then i+3. then 6. then searching for 9 . then check the next index (loop back to beginning if you reach the end) o Continue until you’ve found the key.g. you reach an index with NULL. If you’re adding 18 to the following array. o If you reach the end of the array. which is taken. keep probing from the start  Keep going until you find a free spot  E. and to continue searching  E. it is possible that two values would go into the same index o The hashcode itself. when removing a key-value pair. then we check i+1 o If i+1 is already taken. etc. replacing it with DELETED. or you go through the entire array Removing  When you remove a key-value pair. Think about removing the number 72 from the Hash map below and replacing it with NULL vs.g. we set a deleted/defunct marker that indicates that something used to be stored at that index. which works Searching  When searching for a key. where the number is the hash code itself: 18%7 = 4.

Back to Adding  Because of the deleted markers. adding to a linear-probing hashmap is a bit more complicated  When adding. you check i. then (6+1)%7 = 0. it should go to 6. Try to add 27 to the following array.g. and continue searching o If you later find that the key associated with your pair is already in the hash map. update the value at that index. add the new key-value pair at the index you saved earlier  You do this to keep all the key-value pairs as close to the ideal index as possible so the search time is kept closer to O(1) than O(n) Quadratic Probing  Similar to linear probing but instead of checking consecutive indices. which is also taken. which is taken.  This spaces out collisions more  E. not the one you saved earlier o If you don’t find the key and instead reach an index with NULL or go through the entire array. i+22. etc. which is free! . store the index the first time it happens. then (6+4)%7= 3. i+12. i+32. if you come across an index with a deleted marker.

and then 18 to the following array Searching and Removing: . with a backing array of length n. 6. 1. and 6) o In these cases. there’s pretty much a linked list in each index of the array  When a collision occurs. etc. if you have probed n times. 0. 1. if an ideal index is 6. it is possible that a key-value pair will not be added because quadratic probing simply does not check every index o For example. with a backing array of length 7. 3.Forced Resizing  One thing about quadratic probing is that. then you should resize the array External Chaining  Using this method. 3. even if there are empty slots in the array. (only checks 0. If you add 25. the indices that will be checked are 6. 1. 3. the new key-value pair is added to the linked list  E. 0. the backing array needs to be resized regardless of the load factor o In general.g.

Removing 19 from the Hash Map below  becomes  Performance  Given that the keys generate a wide variety of hash codes for different objects and that the array is large enough. and removing in the worst case become: O(n) . search for the key and remove the pair from the linked-list o E.g. searching. few collisions should occur  Adding. and removing should be: O(1)  If the keys have a limited range of hash codes. or that collisions occur frequently for other reasons.  First determine the index. the pair should be in and search the linked list for the key  To remove a key-value pair. then adding searching.

SkipLists .

larger on right. smaller one left. AVL Trees Problems with Binary Search Trees  A BST has O(log(n)) on average for adding. all operations can be O(n) if the tree is unbalanced o Can be done easily if items are added in sorted order  Try to prevent this AVL Trees  Whenever you’re adding or removing a data item. which side is heavier  BF = left tree height – right tree height  BF = 0 means tree is perfectly balanced and no rotations needed  -1 < BF < 1 is still considered balanced but heavier on left if 1 or right if -1 o Rotating this will not make the subtree perfectly balanced  Any other BF means tree is not balanced so needs a rotation or more  Examples: (height in blue. then make rotations  AVLs are just balanced BSTs Properties  AVLs have many properties similar to BSTs (max 2 children. in the worst case. removing and searching  However. check to see if the other nodes are balanced o If they aren’t. BF in red) . height calculation)  Searching is the same in both  Adding and removing is the same with extra steps o You don’t just have heights. if not balanced. you also have a balance factor Balance Factor  Describes how balanced the subtree of that node is and.

Rotations  After adding or removing something  There are two single rotations and two double rotations to choose from o Double rotations are just two single rotations put together  Which one happens depends on the children and load factor Left Rotation  When you have BF = -2 and right child has BF = 0 | -1 Right Rotation  When you have BF = 2 and left child has BF = 0 | 1 Left – Right Rotation  When you have BF = 2 and left child has BF = -1 Right – Left Rotation  When you have BF = -2 and right child has BF = 1 .

rotations prevent it from turning to a linked list . search and deletion are O(log(n))  Worst case is O(log(n)) o Because if adding elements in sequential order.Adding and Removing  Basic steps are the same as for a BST  After a node is added/removed. the heights and balance factors of the ancestors of the added/removed nodes are updated o Done recursively (pointer reinforcement) from the parent nodes all the way up the rest of the tree and do necessary rotations on the way o If rotations are done. update heights and BFs on the way  Adding pseudo-code AVLs Performance  Average-case for insertion.

…  E. Order 5 means 5 children and 4 data items  We focus on 2-4 trees o 2-4 children. B .g. You know it……. first data item < second child < second data item.Trees B – Trees  Similar to BSTs in that smaller items are to the left and larger items are to the right  Different in that nodes can have multiple data items and nodes can have multiple children  Order of B-tree determines how many data items and children o E.g. A<x<B<y<C<z<D:  Example 2-4 trees: Properties:  The depth of all the leaf nodes are the same  If there are n data items in a node. adding to.  Pseudo – code: . there must be 0 or n + 1 children  This makes a B-tree always balanced  Searching. or removing from a B – tree is O(log(n)) Searching:  Start at root  Compare ……………. 0-3 data items Node  Data in each node stored in ascending order  First child < first data item.

e. Psudo . if the parent doesn’t exist.. then it is created o The node itself is split into two o Smaller items than the promoted one go left..code . then the node may have too many data items  In this case. not an internal node  Follow the same steps as searching until you find the data in the tree or a leaf node o If the data is not in the tree (i. the middle data item (2nd or 3rd here. then add the data to the leaf node o If the data is in the tree then implementation defined (may add duplicate. others go right  …. it would have 4 if overflowing) is promoted to the parent node. update it or do nothing)  Examples: If you’re adding 742 to the first tree and 917 to the second one Overflow  If the data is added to a leaf.Adding  The new data item is always added to a leaf node. you are at a leaf node).

search and deletion are O(log(n)) for average AND worst case  Best case. bring down a reasonable data item from the parent into the empty node and merge it with the left or right sibling  If this causes an empty parent node. If any of the child node’s siblings has more than one data item. you’re good 2.Removing  Same steps as for searching until you find the data in the tree or go off the tree (in which case data is not in there  Once you find the node with the data. then rotate the data to fill this empty node (like a rotation in AVL) 4. remove the data and 1. if this leaves an empty node where the prede/successor used to be. If the node has no children nodes and there is at least 1 data item. If the node has children nodes. continue to 3 3. the depth of the nodes being the same) insertion. continue from step 3 …… Examples……. move the successor/predecessor into this node. Performance  Because of how items are added into a B – tree (esp. If all the empty node’s siblings have only one data item in each node. searching is O(1) if data is at root .

Splay Trees Motivation  The previous data structures have a lot of overhead. and complicated implementations  Here we relax the constraint for O(log(n)) performance in exchange for less overhead and better simplicity  We prioritize data that appears more frequently What is a Splay Tree?  Same data as a BST. x moves two nodes closer to the root . this is just a left or a right rotation o y would be the unbalanced node  Notice that after this. y be the parent of x and z be the grandparent of x Zig(x)  Used when x has no grandparent (when it is close to the root)  In AVL terminology. but restructures itself so that more frequently accessed data is moved closer to the root o These operations are called splaying Splaying  Splaying a node “x” means restructuring the tree so that node x is moved to the root  Three types of rotations: o Zig o Zig-Zag o Zig-Zig  Let x be the node you are splaying. this is just a left-right or right-left rotation o Z would be the unbalanced node  Notice that after this. x moves one node closer to the root ZigZag(x)  Used when z exists and there is a “bend” in the path between x and z  In AVL terminology. space inefficiency.

ZigZig(x)  Used when z exists and there is no “bend” in the path to z  There is no AVL equivalent o This is NOT just two left or two right rotations  Notice that after this. x moves two nodes closer to the root Splay(x)  Input: x is the node to splay  Output: Nothing. but the tree will be modified  This is done recursively but this is the idea Example: Splay(19) .

Split – Joint Operations  Split: Given a BST and data x. split returns a BST with all data less than or equal to x and one with all data greater than x .

 Join: Given two BSTs L and R. join will merge them into a single BST Split(x)  Input: x is the data used to split the tree  Output: Two BSTs called L and R with data(L) < x < data(R)  Here we assume that x is in the tree  Process: o Find where x is in the tree o Call splay(x). now x is at the root so split off the right child to get R . R)  Find 15 in tree then splay it:  R is not empty so find min(R) and splay it o Attach L to the left child of the new R . find the minimum element o Call splay(min). If L is empty. everything on the left is L Join(L. Join(L. make R the whole tree Example: Split(15). R)  Input: BSTs L and R  Output: Merged BST of L and R  Process: o Use either L or R to fill in root (here R) o In R. where all the data is less than all the data in R. its left sub-child will be null since you splayed it o Attach L to the left child of min o If R is empty. make L the whole tree. this moves min to the root.

call split(x). there’s nothing to do  WE USE APPROACH 1 IN CS – 1332 (so I will skip approach 2…) Approach 1: Split and Join the Tree  Adding x: o If x is already in the tree. splay the parent of where you should’ve found it  Removing x: o If not found. now join L and R without x Example: Remove(20)  Find it. which involves splaying  Then join L and R Example: Remove(5)  It exists so split it.When to Splay?  Different implementations have different rules about the conditions under which to splay  Approach 1: The approach used by Gnarley Trees (the original approach)  Approach 2: The approach in the textbook.left the left child of x  Accessing x: o If found. which involves splay(5). R Example: add(20) . splay the node x o If not found. R  Join L. a zigzag to get L.right the right child of x o If y > x. to get L and R. make x the new root and make y the right child of x. better amortized performance  Note: if they say to splay the parent and you’re already at the root. splay the duplicate o If x is not in the tree. make x the new root and make y the left child of x. splay the parent of where you should’ve found x o If found. make y. it is in the tree so split(20). make y. splay the parent of where x should go splay(y) o If y < x.

not a duplicate so splay parent  Attach 18 at the root Example: add(19) Example: get(18) . it is not a duplicate so splay the parent: splay(15)  Attach 20 at root Example: add(18)  Find where 18 should be.  Find 20.

the more balanced the tree tends to become  Overall: Basically.Example: get(25) Applications of Splay Trees  Cache Implementations  “Situations where there is a highly biased query distribution (subset of data is accessed more frequently than others)”  Network Routers (alongside HashTables)  Data Compression Performance  Average: O(log(n))  Worst: Amortized O(log(n)) o Amortized because a single operation can still be O(n) but it ensures that following operations will not be O(n) Intuition of Performance  If you are getting something near the root. high costs have the tradeoff of making the tree more balanced for future searches. lower costs have the tradeoff of making the tree less balanced… Why not just use Zig every time? ………… . the tree will tend to become more unbalanced  If you are getting something far from the root. it becomes more balanced on the way up o The more costly a search is.

balance factor. etc.Pros and Cons  Pros: o Achieves best O(log(n)) and worst amortized O(log(n)) o Doesn’t require extra book-keeping like height. o Simple to implement o Less overhead and space constraints than AVLs or 2-4 trees o Can outperform said trees if highly biased distribution (overall is about same though)  Cons: o Worst case is amortized O(log(n)) but individual operations can still take O(n) so not useful if every operation must be efficient o Cannot be used in multithreaded applications since even access operations modify the tree .

keep them as is and check that second item with the next one  Repeat until end of array. in correct order  Repeat until array is sorted o You know it is sorted if you don’t make any swaps during an iteration or if you have iterated through length – 1 Pseudo – code Bubble Sort Performance . repeat one iteration and the two largest are at the end. swap them. Bubble Sort  Compare the first two items. Sorting The problem with computers is that they can’t compare more than two items at a time. In-Place and Stable Sorts  In-Place: Doesn’t copy elements into another array/list (creating variables to store fixed number of items is allowed) o Regardless of the length of the array. which doesn’t compare items. once there. which humans can. you’ve completed one iteration  Now the largest item is at the end of the array. for local variables neded for sorting o An out-of-place algorithm sort allocates variable amounts of additional space  Stable Sorts: The order of duplicate items is preserved Sorting Algorithms: All the sorts we study below are comparison sorts except for radix. if not. it uses a fixed amount of (additional) space. if they are in the wrong order.

that’s one iteration  Repeat with the third item. every item must slide all the way down so max number of comparisons every time n(n – 1)/2  In-place and stable . unlike bubble sort” Pseudo – code: Insertion Sort Performance  Best-case: O(n) o Happens if the array is already sorted because only one comparison is made each iteration for n – 1 iterations  Average-case: O(n2)  Worst-case: O(n2) o Same reason as bubble sort: if in reverse order. placing it correctly along the first two.  Best-case: O(n) o Happens if array is already sorted so only one iteration of bubble sort is done and no swaps are made  Average-case: O(n2) o Because the actual number of comparisons made is somewhere between n-1 and n(n-1)/2  Worst-case: O(n2) o If the array is sorted in reverse order: all n-1 iterations are needed and at least one swap will always be made n(n – 1)/2 comparisons  Bubble sort is in-place and stable Insertion Sort  Assume the first item is sorted. then take the second one and slide it to the left so that it is in the correct position relative to that first item o The first two items are now sorted. repeat until fully sorted  “Runs a fixed number of iterations.

if odd number. repeat until fully divided  To merge. swap it with the second one  Repeat until array is sorted Pseudo – code: Selection Sort Performance  Best/Average/Worst-case: O(n2) o Because it must search the entire array to find/make sure it’s the smallest item everytime  In-place but NOT stable Merge and Quick Sorts  Unlike the previous sorts. swap it with the first one  Search entire array (excluding first item) for the next smallest item. then stick the middle element in either arrays)  Divide each sub array into two. add the smaller of the two into a larger (merged) array and increase that marker . Merge Sort  Divide the array into two parts (split at the halfway point. these two are recursive rather than iterative  They divide the arrays and then sort them  One advantage is that portions of the sorting algorithms can be done in parallel (multiple threads)  Remember that an array of length 1 is always sorted. have a marker on first element of each item.Selection Sort  Search entire array for smallest item.

 Repeat until all items have been added to merged array Pseudo-code Merge Sort Performance  Best/Average/Worst-case: O(nlog(n)) .

that way you only make as many comparisons as the number of items in the smaller array o Worst is when array is sorted in reverse order  Out-of-place but stable Quick Sort  Choose a random item as pivot and swap it with the first item in the array  Have a “left” marker that starts immediately after the pivot and a “right” marker at the last item  If the item the left marker points to is smaller than the pivot. repeat until it points to something smaller than the pivot or passes the left marker o If the right marker and pivot are the same. o Best case is when all items in one of the split arrays are smaller than all the items in the other array. repeat until it points to something bigger than the pivot or passes the right marker o If the left marker and pivot are the same. you can do either  After the markers cross over. move it left one. you can do either  If the item the right marker points to is bigger than the pivot. swap the pivot and right marker (note: right marker is now to left of left marker)  All items left of the pivot are now smaller than it. all items to its right are larger o Perform quicksort on the smaller and larger items Pseudo – code: . move it right one.

create 10 buckets (19 if dealing with negative numbers) o Each bucket is a queue  For each number in array. …)  Two types: Least Significant Digit (LSD) and Most Significant Digit (MSD) Radix sorts LSD Radix Sort  Assuming number is base 10.Quick Sort Performance  Best-case: O(nlog(n)) o The pivot divides the array in half every time  Worst-case: O(n2) o If the pivot chosen every time is the smallest or largest element. in which case quick sort becomes something like selection sort  In-place but NOT stable o Note: Can be made stable but makes it Out-of-Place Radix Sort  Different from the previous ones: does not do any comparisons  Can only be done on numbers of a base (base 10. look at the first digit (LSD) and put number in corresponding bucket  After all numbers have been added. remove all the numbers one at a time starting with bucket 0  Repeat this for as many times as there are digits in the longest number (not the largest one) . base 16. base 8.

for each bucket. remove the first number from each bucket until they are empty Pseudo-code . if there are two or more elements. this will be first digit  For each number look at that digit and add the number to the corresponding bucket  After all numbers have been added.Pseudo-code LSD Radix Sort Performance  Efficiency depends on the length of the longest number (k) and the number of items in the array (n)  Best/Average/Worst-case: O(kn)  Stable but Out-of-place MSD Radix Sort  Assuming number is base 10. create 10 buckets (19 if dealing with negative numbers) o The buckets are NOT queues  Find length of longest number. run MSD Radix sort on the bucket  Starting with the first bucket.

then MSD radix sort will take fewer iterations to finish than LSD (maybe just one if there are few numbers)  Out-of-Place. NOT stable .MSD Radix Sort Performance  Best/Average/Worst-case: O(kn)  When long numbers are used and the most significant digits are different.

tress.com/javase/8/docs/api/java/util/Iterator. etec. NoSuchElementException if called with no next element o remove(): Removes the most-recently returned item  https://docs. array lists. Iterators Iterators  There are many data structures: arrays. hash maps. o We often need to go through all the items in the structure o But because the structures are different. linked lists. they just throw UnsupportedOperationException when implementing it o But in Java 8. go to the next or previous  Java has an Iterator interface in java.html  The Iterable interface has one method: iterator() which returns an instance of the Iterator interface for that data structure  Many data structures do not need to support remove().util. you would need a different code to iterate through most data structures o Instead: Iterator!  The are usually a special class/object to easily traverse a structure  They abstract away the specific operations for a structure and provide generic methods o Get() the current item. the default implementation does just that so you don’t ned to implement it AT ALL  Example: One way you can implement iterator for a singly-linked list: .oracle.package Using an Iterator  So the data structure must implement the iterator (as an inner class or private class) o The inner or private class implements the Iterator<> interface and handles going through each item in the linked list o The parent class itself must also implement the Iterable<> interface which has only one method: iterator() which returns an instance of the iterator for that specific data structure o The user would ask for an instance of the iterator() o Then you call the methods in the Iterator interface to get the items o Apparently changing the data structure (by adding/removing elements) may break the iterator: might through a ConcurrentModificationException when the data structure is changed Iterators in Java  The Iterator interface has three methods: o hasNext(): Tells you whether calling next() will return the next item (Boolean) o next(): Returns the next item in the data structure.

 Note how the next() method gets the data to return and then moves the node forward  Here’s how the user might use an iterator: For-Each Loop in Java  You can use a for-eqch loop (enhanced for-loop) instead of dealing with the Iterator to go through the structure o Less error-prone and less code to write  You need to implement Iterable and return the proper Iterator though  Basically. you can rewrite the last code as follows: .

which is O(1) for iterator) but is O(n2) if using the regular get() (because getting each element is O(n)) .g. you can have pre/in/post-order iterators  But the iterator() method can only return one.g. For tress. one that goes from front to back or one from back to front o E.Fancy Iterators  Note that it is possible for data structures to implement different types of iterators o E. For Array list.g. using get() is O(n) for an Iterator (if you just keep getting the next item. and the for-each loop can only use one o So I guess you’d have your iterator() method return the one you want to use Performance  Same or better time complexity than using other methods  E. using get() and using an iterator have same time complexity  But for a linked list.

we look at the number of comparisons Brute Force  Least efficient string searching algorithm  Align the pattern with the first character in the text. repeat  If a match is found. you can keep searching by shifting the pattern right one and repeating  If any part of the pattern “hangs off” the end of the text: no more matches  Examples: Pattern is acc . etc. until they all match (found!) or mismatch o Mismatch: shift the whole pattern by 1 to the right. compare them o Match: compare the next one. String Searching String Searching  Looking to see if a string is within another string  String: Sequence of characters  Alphabet: The set of possible characters in a family of strings  Length of text: n  Length of pattern: m  When looking at efficiency.

Brute Force Performance:  Best-case: o O(m) IF WE’RE ONLY LOOKING FOR ONE MATCH o O(mn) otherwise  Best/Average/Worst-case: O(mn) o Happens if there is a mismatch on the very last character of the pattern for each alignmend Boyer-Moore  Constructs a last table that determines how much to shift the pattern by on a mismatch  The algorithm starts from the back fo the pattern. not the front Boyer-Moore Last Table  Basically you iterate through the characters in your pattern and make a table of where the index at which they occur last in the pattern o Use * to represent  Examples: Last occurrence tables for “dog” and “Dig-Dug” .

 Start by aligning the pattern with the beginning of the text  Compare the last character in the pattern with the text at that index o Match: compare previous one. instead shift it by 1 right  If looking for multiple matches. shift pattern right by 1 o Stop when part of pattern “hangs off”  Example: Keep in mind that we look for the mismatched character IN THE STRING in the table and align that index of the pattern with the mismatch . align the index (given by table) of the pattern with the mismatched character o Basically. repeat o Mismatch: Go to the table for the mismatched character IN THE TEXT. you just align the last character in the pattern that matches the mismatch in the text  Do not do this if it means the pattern is shifted back. if it mismatches. once one is found.

Pseudo-code: Note that this only returns the first match Performance  Best-case: o O(m) IF YOU’RE LOOKING FOR ONE MATCH ONLY  Worst-case: O(mn) .

don’t move j o Different AND i = 0: Write 0 at entry j and move j forward by 1. there is a shorter one of “ab” but we need to use the longest one  “KMP needs to know the length of the longest suffix for the first 2 to m characters of the pattern”???? Creating the Failure Table  Create two markers i and j for the first and second characters in the pattern o Set first entry in table as 0  Compare the characters pointed to by i and j o Same: Write i + 1 into entry j of the table and move both I and j forward one o Different AND i ≠ 0: Move I back to the value at (I – 1) in the table. For “ababaaababbabab”.g. the longest suffix is “abab”. when shifted. constructs a failure table to determine alignment upon mismatch  Starts at beginning of pattern but. o Happens if the mismatch is always the first character in the pattern  Average-case: O(m + n) KMP  Knuth-Morris-Pratt (KMP). For “revararev”. may or may not restart from beginning KMP Failure Table  Table is of same length as pattern  Each entry contains number representing length of longest suffix (up to that point) that is also a prefix in the pattern  E.g. don’t move i  Repeat this until j goes past the end of the string and all entries in the table have a value  Examples: Pseudo-code: . the longest suffix is “rev”  E.

align that index of the pattern with mismatch in text.KMP Search Procedure  Align pattern with beginning of text  Compare first characters o Match: Go to next character. get the number there o Then align the letter in the pattern represented by this number after the last letter in the text o Continue comparing from there o Continue until something “hangs off” . continue comparing from index j – 1 of table  Once you’ve found a complete match. to look for additional matches: o Look at last index of failure table. repeat o Mismatch:  On first letter of pattern: Shift pattern right by one. look at j – 1 of failure table. repeat from first pattern character  Not on first letter: Say mismatch was on index j of pattern.

 Examples: Pseudo-code: .

compares the hashes of the pattern and substring of text Rabin – Karp Rolling Hash  Calculates the hash of the pattern and a hash of the substring of the text o The substring is the same length as the pattern  If hashes are not equal. j is number of letters in text: o In the examples here they do j – i – 1 for the BASE exponent  Example: For the text “crow” with base 101 (Note: they use the Unicode of the letter as the textp[i] . so compare each character  Calculating initial hash is O(n) but “sliding” is O(1)  Rolling hash function that is commonly used is known as the Rabin fingerprint o BASE is a prime number. then rolling hash slides to right one character  If equal then they may be equal (but no guarantee because hash collision).Performance:  Best-case: o O(m) IF ONLY LOOKING FOR FIRST MATCH  Best/Average/Worst-case: O(m + n) Rabin-Karp  Doesn’t directly compare characters.

you can just compute the hash of the first j characters of a text).  Once you have the initial hash (of length j. compute the hash of the entire pattern and of the first j characters of the tex (j is the length of the pattern) o Match: compare each character in the pattern. you can “slide” the hash using: o a is the index of the character being removed. if not same. b is the one being added Rabin-Karp Searching Procedure:  First. go to Mismatch o Mismatch: Slide the hash of the text by one character to the right  Repeat until end of text if looking for more than one match Pseudo – code: .

matching prefixes and suffixes  Boyer-Moore is better for long alphabets because. then you will probably have charaters repeating at the end of your pattern. thus. meaning you don’t shift it by a lot . if you have a small alphabet.Rabin-Karp Performance:  Best-case: o O(m) IF LOOKING FOR ONE MATCH o O(m + n) Otherwise  Worst-case: O(mn) o If the pattern hash and text hash are always equal (would happen if bad hashing function)  Best/Average-case: O(m + n) Note:  KMP is better suited for short alphabets because you have more chances of getting repeat characters and.

v) o If all edges undirected: undirected graph Graph Terms:  Self – Loop: Edge that connects a vertex to itself  Two edges are parallel if they connect the same pair of vertices  When an edge connects two vertices. we say that they are adjacent to one another and the edge is incident on both vertices  Degree of a vertex: Number of edges incident on it  … Graph ADT … Graph ADT Structures  The two most common implementations of a graph are the adjacency matrix (static implementation) and the adjacency list (dynamic implementation) . Graphs Graphs  A set of nodes/vertices. v) o If all edges directed: directed graph  Undirected edge: Unordered par of vertices (u. E  Order of a graph: The number of vertices  Size of a graph: Number of edges  Degree of a vertex: Number of edges incident to the vertex o In-degree and out-degree are number of edges going into and out of vertex respectively o If all degrees are the same then the graph is said to have that degree  Path: Set of edges connecting two nodes  Can have edges not connected to the graph  Graphs can also be: o Directed: edges flow in one direction o Undirected: edges flow in both directions o Weighted: there are weights associated with each edge Edge Types:  Directed edge: Ordered pair of vertices from origin u. connected through edges/arcs. to desitnation v. (u. V.

but that may not happen if graph is directed. allows one to show if graph is disconnected. o There is also edge list implementation  For the graph on the right: o Adjacency Matrix: It is symmetrical about the main diagonal in this case. More complex to find an edge between two vertices Adjacency Matrix … Adjacency List Structure … Edge List Structure (3rd) … Implementation with Linked Lists … Structure Performance . inflexible for adding or removing vertices: o Adjacency List: Has advantages for finding adjacent nodes. Waste of memory.

determine the path with lowest length Depth – First Search DFS  Find a way to systematically visit every vertex and every edge of a graph o Start from one vertex o Move forward all along one path (do not pass through a vertex already visited) o When stuck.Algorithms:  Depth First Search: Uses a stack  Breadth First Search: Uses a queue  Shortest Path Algorithm: Shortest path in a graph given two vertices. turn back until you can step forward to an unvisited vertex o DFS finds some path from source vertex v to target vertex u  Uses a stack  Pseudo – code: . p and q.

Breadth – First Search  Another systematic way of visiting the vertices of a graph G o Start from a vertex. the path is disregarded  Computes the distance of all vertices from a given start vertex s  Assume the graph is connected. the edges are undirected. shoos stop vertex o Move forward all along shortest path (do not pass through a vertex already visisted) using BFS o Keep track of the distance of the path compared to other paths o If you do not reach stop vertex.youtube. and the edge weights are nonnegative  Store the distance of each vertex v from s in a table and update as you grow the cloud of vertices  Pseudo – Code: . then step forward all vertices adjacent to its children  Uses a queue  Classic method to find path with fewest nodes from source vertex v to target vertex u  Pseudo – Code: IF CONFUSED: https://www.com/watch?v=bIA8HEEUxZI Dijkstra’s Shortest Path  Find the shortest path for weighted directional graphs o Start from one vertex. step forward all vertices adjacent to it.

Fully solved Example: .

weight(f) <= weight(e)  Proof: .g. Minimum Spanning Trees  Spanning subgraph: subgrah of a graph G containing all the vertices of G  Spanning tree: spanning subgraph that is itself a (free) tree  MST: Spanning tree of a weighted graph with minimum total edge weight  Applications: o Communications networks o Transportation networks Cycle Property  Let T be a minimum spanning tree of a weighted graph G  Let e be an edge of G that is not T and C be the cycle formed by e with T  For every edge f of C. then e is part of the MST  E. Minimum Spanning Trees Minimum Spanning Trees  A minimum spanning tree of an edge-weighted graph is a spanning tree that connects all vertices in G and has a minimum total weight  Concept: Let V be any subset of the vertices of G and let edge e be the smallest edge connecting V to G-V.

starting from s . o By contradiction o If weight(f) > weight(e) we can get a spanning tree of smaller weight by replacing e with f Partition Property: … MST Pseudo Code … MST Example Prim .Jarnik’s Algorithm  Similar to Dijkstra’s algorithm  We pick an arbitrary vertex s and we grow the MST as a cloud of vertices.

parent and locator labels of vertex z. where each key change takes O(log(n)) time .  We store with each vertex v label d(v) representing the smallest weight of an edge connecting v to a vertex in the cloud  At each step o Add to the cloud the vertex u outside the cloud with the smallest distance label o We update the labels of the vertices adjacent to u Prim – Jarnik Pseudo Code: Analysis  Graph operations: We cycle through the incident edges once for each vertex (to find minimum one?)  Label operations: o We set/get the distance. where each insertion/removal takes O(lg(n)) time o The key of a vertex w in the priority queue is modified at most deg(w) times. O(deg(z)) times o Setting/getting a label takes O(1) time  Priority queue operations: o Each vertex is inserted once into and removed once from the priority queue.

 Prim – Jarnik’s algorithm runs on O((n + m) log(n)) time provided the graph is represented
by the adjacency list structure
o Recall that …
 The running time is O(mlog(n)) since the graph is connected

Kruskal’s Approach
 Maintain a partition of the vertices into clusters
o Initially, single-vertex clusters
o Keep an MST for each cluster
o Merge “closest” clusters and their MSTs
 A priority queue stores the edges outside clusters
o Key: weight
o Element: edge
 At the end of the algorithm
o One cluster and one MST
 Basically, what you do is add the edge with minimum weight, then the next one with
minimum weight, then repeat until all your vertices are connects

Kruskal’s Algorithm

Example

Data Structure for Kruskal’s Algorithm

List – Based Partition

Partition – Based Implementation

 A partition based version of Kruskal’s algorithm
o Cluster merges as unions
o Cluster locations as finds
 Running time O((n + m) log(n))
o Priority Queue operations: O(mlog(n))
o Union – Find operations: O(nlog(n))
o I think n is the number of vertices and m is the number of edges

Dynamic Programming
Dynamic Programming (DP)
 An algorithm technique for solving a complex problem by breaking it down into a
collection of simpler recurrent subproblems
o Solving each subproblems just once and storing their solutions
 Called a Bottom-Up approach:
o Identify smaller subproblems
o “Order of solving subproblems”
o Make sure subproblems are solved before the larger ones
o “Subproblems overlap or are NOT
interdependent”
 E.g. Matrix Multiplication

DP for Matrix Chain Products
 Dynamic Programming is an algorithm design
paradigm
 Review of Matrix Multiplication:
o C = A*B
o A is a dxe matrix, B is an exf matrix:

 O(def) time

Matrix Chain – Products
 Matrix Chain – Product:
o Compute A = A0*A1*…*An-1
o Ai is di x di+1
o Problem: How to parenthesize
 Example:
o B is 3x100
o C is 100x5
o D is 5x5
o (B*C)*D takes 1500 + 75 = 1575 ops
o B*(C*D) takes 1500 + 2500 = 4000 ops

A Characterizing Equation
 “The global optimal has to be defined in terms of optimal subproblem, depending on
where the final multiply is at”
 Let’s consider all possible places for the final multiply:
o Recall that Ai is a di x di+1 dimensional matrix
o So a characterizing equation for Ni, j is:

A Dynamic Programming Algorithm
 Since subproblems overlap, we don’t use recursion
 Instead, we construct
subproblems “bottom-up”
 Ni,I’s are easy so start
with them
 Then do length 2, 3, …
subproblems and so on
 The running time is O(n3)

DP Algorithm Visualization
 The bottom-up construction fills in the N array by diagonals
 Ni, j gets values from previous entries in the ith row and jth column
 Filling each entry in the N table takes O(n) time
 Total run time: O(n3)
 Getting actual parenthesization can be done by remembering “k” for each N entry

The DP Technique
 Applies to problems that seem to require a lot of time (possibly exponential), provided
we have:
o Simple subproblemts: the subproblems can be defined in terms of a few variables
like j, k, l, m and so on
o Subproblem optimality: The global optimum value can be defined in terms of
optimal subproblems
o Subproblem overlap: The subproblems are not independent, but instead they
overlap (hence, should be constructed bottom-up)

Real DP Example, Subsequences
 A subsequence of a character string x0x1x2…xn-1 is a string of the form xi1xi2…xjk, where
ij < ij+1
 Not the same as a substring!
 Example String: ABSDEFGHIJK
o Subsequence: ACEGIJK

T})  Example: ABSDEFG and XZACKDFWGH have ACDFG as a longest common subsequence DP Approach to the LCS Problem  Define L[i. G. j] = L[i-1. j]. -1] = 0 to indicate that the null part of X or Y has no match with the other  Then we can define L[i. then L[I. j-1]} (we have no match here) An LCS Algorithm Visualizing the LCS Algorithm . so L[-1. then L[i.. C. j] in the general case: o If xi=yk. j] = max{L[i-1. j] to be the length of the longest common subsequence of X[0…i] and Y[0. j-1] + 1 (we can add this match) o If xi =/=yk. L[I. the longest common subsequence (LCS) problem is to find a longest subsequence common to both X and Y  Has applications to DNA similarity testing (alphabet is {A. o Subsecquence: DFGHK o Not subsequence: DAGH The Longest Common Subsequence (LCS) Problem  Given two strings X and Y.j]  Allow for -1 as an index. k] = 0 and L[k.

com/watch?v=P-mMvhfJhu8 . fear not. total running time is O(nm)  Answer is constained in L[n.Analysis of LCS Algorithm  We have two nested loops o The outer one iterates n times o The inner one iterates m times o A constant amount of work is done inside each iteration of the inner loop o Thus. m] (and the subsequence can be recovered from the L table) If all this is confusing af.youtube. just watch this: https://www.