You are on page 1of 2

Proof of optimality of Huffman codes

Optimality of Huffman Codes: Cost function: cost = Σc C (f (c)dc ) where C is the alphabet (character set), f(c) is the frequency and dc is the code length of the character c C . Goal is to optimize the above cost. 1. Proposition 1. The Huffman tree is a full binary tree. Proof: It follows from the construction of the Huffman tree, i.e., we always create a new node by having two subtrees as it’s two children. 2. proposition 2: Two least frequent characters α and β must be the two deepest nodes in the optimal tree and these two nodes are siblings. proof: Assume α is not the deepest node in the optimal tree. Then there exists a node γ which is deeper than α. The cost due to the characters α and γ : f (α)dα + f (γ )dγ . If we swap α with γ in the tree the cost is f (α)dγ + f (γ )dα notation dα and dγ are used to denote the code lengths of the respective characters before the swap. Then f (α)dγ + f (γ )dα < f (α)dα + f (γ )dγ because (dγ − dα ) is positive, f (α) < f (γ ) and therefore, f (α)(dγ − dα ) < f (γ )(dγ − dα ) The above is a contradiction because the tree was optimal before the swap and the cost of the new tree created by the swap is less. 3. Huffman’s algorithm is a greedy algorithm which gives optimal results. Therefore, it must satisfy the optimal substructure property, i.e., optimal solution to the problem contains within it the optimal solutions to the subproblems. Theorem: Let T be a full binary tree representing an optimal prefix code over an alphabet C. Consider two characters x, y C that appear as sibling of a parent node z in the tree T. Now consider the tree T with 1

then this new tree will correspond to the alphabet C. This new tree T3 will have cost B (T3 ) = B (T2 ) + f (x) + f (y )———— 3 From 1. y } ∪ {z }. 2 . Assume that T1 is not optimal. We want to show T1 is optimal by contradiction. except for the cost from z to x and z to y in the tree.———– 2 z appears as a leaf in T2 because we have assumed it to be a character. Then there exists another tree T2 such that B (T2 ) < B (T1 ). 2 and 3 we get B (T3 ) = B (T2 ) + f (x) + f (y ) < B (T1 ) + f (x) + f (y ) = B (T ) This is a contradiction because cost for T3 is less than that for T . Proof: Let B (T ) and B (T1 ) be the costs of the trees T and T1 .x and y removed and z becomes a character in this tree with frequency f(x)+f(y). Then B (T ) = B (T1 ) + f (x) + f (y ) ——— 1 This is because B (T1 ) covers all cost of T including those for x and y. If we add the characters x and y as children of Z and assume z not a character any more but an internal node of this new tree T3 . respectively. Then this new tree T1 represents an optimal prefix code over the alphabet C − {x.