Information theory
APMA 1710
Fall 2011








Week Lectures Videos Readings Homework/Exams
Sep 05 Introduction
Symbol codes
(IC 1) (optional)
(IC 2.1-2.5)
Cover 5.1
Sep 12 Kraft-McMillan inequality
Source coding theorem
(IC 2.6-2.11)
(IC 3)
Cover 5.2-5.5
Sep 19 Huffman coding (IC 4) Cover 5.6-5.8 Cover Ch.5: exercises #1,3,10,30,35,37
Sep 26 Arithmetic coding (IC 5) MacKay 6.1-6.3 Huffman coding
Oct 03 Entropy and friends (IC 6) Cover 2.1-2.5 Arithmetic coding
Oct 10 Entropy-related properties
Cover 2.1-2.5 Cover Ch.2: #1,3,4,5,6,28 (Extra Credit: #19)
Oct 17 Inequalities
Cover 2.6-2.8, 2.11 Cover Ch.2: #9,15,16,25,36 (Extra Credit: #14)
Oct 24 Typicality
Cover 3.1-3.3, 7.7 MIDTERM
(Friday 10/28)
Oct 31 Repetition codes
Hamming codes

MacKay 1.1-1.4
Cover 7.11
Typicality
Nov 07 Channel coding theorem
Cover 7.1-7.10, 7.12 Hamming codes
Nov 14 Channel coding theorem
Differential entropy

Cover 7.1-7.10, 7.12
Cover 8.1-8.6
Cover Ch.7:
Prove Theorem 7.2.1
Exercises #4,5,8,13
(Extra Credit: #9)
Nov 21 Gallager codes
MacKay 47.1-47.4
MacKay paper
(No homework)
Nov 28 Rate-distortion theory
Cover 10.1-10.5 Gallager codes
Dec 05 Review (Reading period is 12/8-12/12)


Dec 12 (Finals) (Finals) (Finals) FINAL






(Go back to main page)