Block 2 Part (2) Self-Assessments

Activity [2.2]
• What message has the following ASCII coding?
• 100111110010110111111

100 1111 100 1011 011 1111

O K ?
The message = OK?

‫ ارقام‬7 ‫هنا نقسم الكود لمجموعات كل مجموعة بيها‬ 

‫ ارقام‬4 ‫ ارقام تدل على الكود الموجود اعلى الجدول واخر‬3 ‫اول‬ 
‫تدل على الكود في يسار الجدول‬

========================================================
Activity [2.6]
What is the lowest achievable number of bits per symbol using fixed-length codes for the

following sources?

a) The 26 letters of the alphabet plus a space symbol (27 different source symbols).

b) The result of a sports match, reporting three possibilities: win, lose, and draw.

c) The result of tossing a coin: heads or tails.

d) The report of a remote weather station that returns one of four different visibility

conditions: very poor, poor, moderate or good.

a) 27 different source symbols = log2 (27) = 4.75 ≈ 5 bit per symbol

b) 3 different possibilities of a sports match = log2 (3) = 1.58 ≈ 2 bit per symbol

c) 2 different of tossing a coin = log2 (2) = 1 bit per symbol

d) 4 different of a remote weather station= log2 (4) = 2 bit per symbol

Activity [2.9]
 Suppose that the weather reporting station is located in a very sunny place, and that most
of the time the visibility is good. Sometimes it is moderate, and only rarely is it poor or
very poor. The visibility probabilities are 0.5, 0.3, 0.1 and 0.1 respectively.
 Table 2.7 contains a code (an instantaneous code) that aims to reduce the average number
of bits per symbol by taking advantage of the different frequencies of the visibility
conditions.

Visibility Code word Code word length , i Visibility probability , p i×p

Very poor 111 3 0.1 3 × 0.1 = 0.3
Poor 110 3 0.1 3 × 0.1 = 0.3
Moderate 10 2 0.3 2 × 0.3 = 0.6
Good 0 1 0.5 1 × 0.5 = 0.5
Sum 1 1.7

 4 different of a remote weather station = log2 (4) = 2 bit per symbol

 1.7 less than 2 bits required for the fixed-length code  this code is more efficient

========================================================
Activity [2.2]
 A space mission to Mars is recording data about the planet’s surface, and sending the
data back to scientists on Earth for analysis. The experimental data consists of
measurements, given to the nearest centimeter, of how far a probe is able to penetrate
the surface. Scientists have predicted that the surface of the region under test will be
similar to a certain region on Earth, for which they already have penetration data. This
data is given in Table 2.10, where the first column gives the penetration depth (to the
nearest centimeter) and the second column gives the relative frequency of this outcome.
The communications link has a low data rate, so a very efficient coding must be used for
the transmission. A Huffman code is therefore used, based on the data in Table 2.10.
a. Using a fixed-length code, how many bits would be needed for each code word?
b. Work out a Huffman code to find a possible set of code words for the different
penetration depths.
c. What is the average code word length using the Huffman code?
d. Use either your own spreadsheet or the online entropy calculator to determine the
entropy of this source.
e. Calculate the efficiency of:
i. The fixed-length code
ii. The Huffman code.

a) 6 different source symbols = log2 (6) = 2.58 ≈ 3 bit per symbol

b)
Symbol ‫لما نرسم نبدأ أوال بترتيب االحتماالت من االكبر لالصغر وليس‬ 
‫ثانيا نجمع اصغر االحتماالت مع بعض ونرتب القيم من االكبر لالصغر الى نهاية الرسمة‬ 
‫ عند جمع احتمالين مع بعض‬1 ‫ او‬0 ‫نكتب‬ 
0 ‫ واالحتمال االكبر ياخد‬1 ‫االحتمال االصغر ياخد‬ 

0
0.39 0.39 0.39 0.61 1
0.39 0

1 0.24 0.37 0.39 1

0.24 0
0.24

2
0.18 0.19 0 0.24 1
0.18

3
0.12 0
0.12 0.18 1

4 1
0.07
0.06 0

5
1
0.01
 ‫ نكتب المعادلة ونعمل جدول‬Average code word ‫لما يطلب‬ 
c) Average code word length
𝑛

𝑙 ҧ = ෍ 𝑙𝑖 𝑝𝑖
𝑖=1

Symbol Code word Code word length (i) Frequency (P) i × P

0 1
1 01
2 001
3 0000
4 00010
5 00011
Sum
1 0.39 1 × 0.39 = 0.39
2 0.24 2 × 0.24 = 0.48
3 0.18 3 × 0.18 = 0.54
4 0.12 4 × 0.12 = 0.48
5 0.06 5 × 0.06 = 0.3
5 0.01 5 × 0.01 = 0.05
1 2.24

d) Entropy of the source  H

𝒏 ‫ نكتب المعادلة ونعمل جدول‬Entropy ‫لما يطلب‬ 
𝑯 = − ෍ 𝒑𝒊 𝒍𝒐𝒈𝟐 ൫𝒑𝒊 ൯ Average code word length ‫الزم يكون مجموع القيم اقل من‬ 
𝒊=𝟏
Symbol Symbol probability , p - log2 (p) p × - log2 (p)
0 0.39 - log2 (0.39) = 1.358 0.39 × 1.358 = 0.530
1 0.24 - log2 (0.24) = 2.059 0.24 × 2.059 = 0.494
2 0.18 - log2 (0.18) = 2.474 0.18 × 2.474 = 0.445
3 0.12 - log2 (0.12) = 3.059 0.12 × 3.059 = 0.367
4 0.06 - log2 (0.06) = 4.059 0.06 × 4.059 = 0.244
5 0.01 - log2 (0.01) = 6.644 0.01 × 6.644 = 0.066
Sum 1 Entropy  2.146

e)
i. Efficiency of fixed-length code = Entropy / fixed-length code = 2.146 / 3 = 0.715 ≈ 72%
ii. Efficiency of Huffman code = Entropy / Huffman code = 2.146 / 2.24 = 0.958 ≈ 96%

‫ من الفقرة‬fixed-length code ‫ ناخد قيمة‬Efficiency of fixed-length code ‫لما طلب‬

‫ نفس طريقة حل الفقرة االولى من السؤال‬Symbol ‫االولى من السؤال او نحسبها من عدد‬

C ‫ النه بالسؤال فقرة‬Average code word length ‫ هي نفس قيمة‬Huffman code ‫قيمة‬
Huffman code ‫ باستخدام‬Average code word ‫طلب‬
Activity [2.13]
A children’s multimedia package contains a simple application for creating music, based
on a keyboard with seven notes: the notes A, B, C, D, E, F, and G
a. How many bits are needed to represent a note of the music using a fixed-length code?
b. The developers of the software carry out some testing with children, and find that the
children click on the keys towards the center of the keyboard much more often than the
keys at each end. In a test of 1000 mouse clicks, the number of times each key was clicked
is as shown in Table 2.11. Calculate the probability of each note being used, based on
these numbers.

c. Derive a Huffman code to represent the notes and calculate the average length of the code
words.
d. Use either your own spreadsheet or the online entropy calculator to determine the
entropy of this source.
e. Calculate the efficiency of:
a. The fixed-length code
b. The Huffman code.

a) 7 different source symbols = log2 (7) = 2.81 ≈ 3 bit per symbol

b)

Note Number of clicks probability

A 30 30 / 1000 = 0.03
B 190 190 / 1000 = 0.19
C 200 200 / 1000 = 0.20
D 210 210 / 1000 = 0.21
E 170 170 / 1000 = 0.17
f 160 160 / 1000 = 0.16
G 40 40 / 1000 = 0.04
c)

D
0.21
0.21 0.23 0.36 0.41 0.59
0 1
C 0.20 0.21
0.20 0.23 0.36 0 0.41
1
B 0.20
0.19 0.21 0.23
0 1
0.19
0.19 0
E
0.17 0.20 1
0.17 0.17 1
F
0.16 0
0.16

G
0.07 1
0.04 0

A
0.03 1

Average code word length

𝑛

𝑙 ҧ = ෍ 𝑙𝑖 𝑝𝑖
𝐢=𝟏

Symbol Code word Code word length (i) Probability (P) i × P

D 10
C 11
B 000
E 001
F 010
G 0110
A 0111
Sum
2 0.21 2 × 0.21 = 0.42
2 0.20 2 × 0.20 = 0.40
3 0.19 3 × 0.19 = 0.57
3 0.17 3 × 0.17 = 0.51
3 0.16 3 × 0.16 = 0.48
4 0.04 4 × 0.04 = 0.16
4 0.03 4 × 0.03 = 0.12
1 2.66
 d) Entropy
𝒏

𝑯 = − ෍ 𝒑𝒊 𝒍𝒐𝒈𝟐 ൫𝒑𝒊 ൯
𝒊=𝟏

Symbol Symbol probability , p - log2 (p) p × - log2 (p)

A 0.03 - log2 (0.03) = 5.059 0.03 × 5.059 = 0.152
B 0.19 - log2 (0.19) = 2.396 0.19 × 2.396 = 0.455
C 0.20 - log2 (0.20) = 2.322 0.20 × 2.322 = 0.464
D 0.21 - log2 (0.21) = 2.252 0.21× 2.252 = 0.473
E 0.17 - log2 (0.17) = 2.556 0.17 × 2.556 = 0.435
F 0.16 - log2 (0.16) = 2.644 0.16 × 2.644 = 0.423
G 0.04 - log2 (0.04) = 4.644 0.04 × 4.644 = 0.186
Sum 1 Entropy  2.588

e)
i. Efficiency of fixed-length code = Entropy / fixed-length code = 2.588 / 3 = 0.863 ≈ 86%
ii. Efficiency of Huffman code = Entropy / Huffman code = 2.588 / 2.66 = 0.973 ≈ 97%
Activity [2.16]
a. Encode the following data using the run-length encoding described above:
 E B B B B B B B B A A A A G G G G G G G C C B B B F G.
b. Decode the following sequence, which has been encoded using the same run-length
encoding:
 E 4 C 1 D 1 A 1 C 6 B 6 A 2.
a. E 1 B 8 A 4 G 7 C 2 B 3 F 1 G 1
b. E E E E C D A C C C C C C B B B B B B A A
Activity [2.20]
Starting with an initialized coding table, follow through the LZW algorithm step by step and
complete Table 2.18 in order to encode the following sequence:
BACCABACBACBAC
 ‫‪Input data‬‬
‫‪P‬‬ ‫‪Q‬‬ ‫‪Code output‬‬ ‫‪New coding table entry‬‬
‫‪code‬‬ ‫‪Data‬‬

‫‪1‬‬ ‫‪B‬‬ ‫‪B‬‬ ‫‪A‬‬ ‫‪66‬‬ ‫‪256‬‬ ‫‪BA‬‬

‫‪2‬‬ ‫‪A‬‬ ‫‪A‬‬ ‫‪C‬‬ ‫‪65‬‬ ‫‪257‬‬ ‫‪AC‬‬
‫‪3‬‬ ‫‪C‬‬ ‫‪C‬‬ ‫‪C‬‬ ‫‪67‬‬ ‫‪258‬‬ ‫‪CC‬‬
‫‪4‬‬ ‫‪C‬‬ ‫‪C‬‬ ‫‪A‬‬ ‫‪67‬‬ ‫‪259‬‬ ‫‪CA‬‬
‫‪5‬‬ ‫‪A‬‬ ‫‪A‬‬ ‫‪B‬‬ ‫‪65‬‬ ‫‪260‬‬ ‫‪AB‬‬
‫‪6‬‬ ‫‪B‬‬ ‫‪B‬‬ ‫‪A‬‬
‫‪7‬‬ ‫‪A‬‬ ‫‪BA‬‬ ‫‪C‬‬ ‫‪256‬‬ ‫‪261‬‬ ‫‪BAC‬‬
‫‪8‬‬ ‫‪C‬‬ ‫‪C‬‬ ‫‪B‬‬ ‫‪67‬‬ ‫‪262‬‬ ‫‪CB‬‬
‫‪9‬‬ ‫‪B‬‬ ‫‪B‬‬ ‫‪A‬‬
‫‪10‬‬ ‫‪A‬‬ ‫‪BA‬‬ ‫‪C‬‬
‫‪11‬‬ ‫‪C‬‬ ‫‪BAC‬‬ ‫‪B‬‬ ‫‪261‬‬ ‫‪263‬‬ ‫‪BACB‬‬
‫‪12‬‬ ‫‪B‬‬ ‫‪B‬‬ ‫‪A‬‬
‫‪13‬‬ ‫‪A‬‬ ‫‪BA‬‬ ‫‪C‬‬
‫‪14‬‬ ‫‪C‬‬ ‫‪BAC‬‬ ‫‪261‬‬

‫بناء على الجدول االول هنجيب الكود للحرف المكتوب في عمود ‪ P‬ونكتبه في ‪Code output‬‬ ‫‪‬‬
‫ناخد الكود الجديد من العمود ‪ P‬و ‪ Q‬ونعطي لهم كود جديد يبدا من ‪256‬‬ ‫‪‬‬
‫لو اتكرر نفس الكود من العمودين ‪ P‬و ‪ Q‬ننزل للصف اللي بعده ونكتبه في عمود ‪ P‬ونشوف الحرف الموجود في ‪ Q‬مع العمود‬ ‫‪‬‬
‫‪ P‬لو م اتكرر نعطي كود متسلسل بناء ع اخر ترقيم لالكواد الجديدة‬