Department of Computer Science & Engineering

University of Ioannina

Information and Coding Theory

Course Feature
Class Description

Course ID: A8

Unit: DATA SCIENCE AND ENGINEERING – Unit A: Algorithms and Information Technologies

Weekly Hours: 4


ECTS Credits: 7

Course Homepage: 

Description: Principles of Information Theory. Information Measures. Shannon’s theorem. Entropy, Relative Entropy and Mutual Information. Asymptotic Equipartition Property. Entropy Rates of a Stochastic Process. Probability of Error. Entropy Rate. Data Compression (Variable-length Lossless Compression, Fixed-length (almost lossless compression, Slepian-Wolf problem, universal compression). Differential Entropy. Gaussian Channel. Rate Distortion Theory. Information Theory and Statistics. Maximum Entropy. Universal Source Coding. Channel coding (Channel coding: Achievability bounds. Linear codes. Channel capacity. Channels with input constraints. Gaussian channels. Lattice codes, Lattice codes Channel coding: Energy-per-bit, continuous-time channels, Advanced channel coding. Source-Channel separation, Channel coding with feedback, Capacity-achieving codes via Forney concatenation). Kolmogorov Complexity. Network Information Theory. NP-hard coding theoretic problems. Applications in Complexity theory.