COURSE

IOTA5507

Information Theory and Its Applications in Machine Learning

Information Hub / Internet of Things · 3 Credits

This course explores how much information can be compressed, transmitted, extracted and learned from data and what the fundamental limits of these processes are by introducing the fundamentals of information theory and its modern applications in machine learning and statistics. We begin with classical foundations--entropy, divergences, compression, coding, and data-processing inequalities--and extend them to modern topics in inference, hypothesis testing, and learning. Applications such as generalization bounds and statistical estimation illustrate how information theory characterizes performance limits and guides system design. By the end, students will gain both the tools and intuition to apply these methods to contemporary problems in data and learning. They will also develop a new and unified perspective that connects theory with practice across diverse fields, such as AI, data science, and computational modeling.