Department of Computer Science
CSU Stanislaus
California State University
CS4450-001: Coding
and Information Theory
Fall 2023
Instructor: Dr. Xuejun Liang
My Office: DBH 282
Office Hours: MW 1:00 p.m.-2:00 p.m. & Th
11:00 a.m.-12:00 a.m.
ZOOM Meeting ID: 4438930033, Phone: (209) 667-3169, Email: xliang@csustan.edu
Class Information
Classroom: Bizzini 111
Class Date & Time: TTh 12:30PM-1:45PM
Class Website: https://www.cs.csustan.edu/~xliang/Courses2/CS4450-23F
Course Materials
Textbook:
· Information and Coding Theory, by Gareth A. Jones and J. Mary Jones, Springer, 2000, ISBN 978-1-85233-622-6
Reference Book
·
Information Theory,
Coding and Cryptography, by Arijit Saha, Nilotpal
Manna, Mandal, Pearson India, 2013, Print ISBN-13: 978-81-317-9749-5, Web
ISBN-13: 978-93-325-1785-1
Lecture Slides
Chapter 1 Source Coding (A, B, C)
Chapter 2 Optimal Codes (A, B)
Chapter 4 Information Channels (A, B, C)
Chapter 5 Using an Unreliable Channel (A, B)
Chapter 6 Error-correcting Codes (A, B, C, D)
Chapter 7 Linear Codes (A, B, C, D, E)
Lecture Slides on Probability and
Mathematical Fundamentals
Overview of Probability: Probability-A and Probability-B
Mathematical Fundamentals (Part-A, Part-B, Part-C)
Course Syllabus and Major Topics
Course
Description
Topics to be selected
from error detecting and correcting codes, encryption and decryption
techniques, RSA and knapsack codes, algebraic coding theory, Hamming
distance, sphere packing and its relation to optimal codes, Hamming, Huffman
and Gray codes, entropy, channel capacity and Shannon's theorem, bandwidth and
the sampling theorem.
Course Outcomes:
Students who successfully complete the course should be able to
1. Determine whether a given code can be decoded uniquely or is instantaneous, construct instantaneous codes, including Huffman code for a source or an extension of a source, and compute the average word length.
2. Describe the concept of Entropy and the meaning of Shannon's First Theorem, compute Entropy for a source, extension, and products, and compute word lengths for Shannon-Fane codes.
3. Describe the concepts and definitions of information channel, system entropies (input entropy, output entropy, equivocation, and join entropy), mutual information, and channel capacity, and apply them to BSC and BEC.
4. Describe Shannon’s fundamental theorem, apply the ideal observer rule, the maximum likelihood rule, and the nearest neighbor decoding to BSC and BEC, and compute PrE and PrC
5. Compute the (extended) Hamming code, Hamming's sphere-packing bound, and the Gilbert-Varshamov Bound, and construct a Hadamard matrix and its corresponding codes.
6. Compute the generator matrix and parity-check matrix (in systematic form) of a linear code and the minimum distance of a linear code. Calculate with the Hamming Codes, the Golay Codes and the Standard Array.
Homework Assignments