Information and Coding Theory (EE 410)

2021 Spring
Faculty of Engineering and Natural Sciences
Electronics Engineering(EE)
3
6
Özgür Erçetin oercetin@sabanciuniv.edu,
Click here to view.
English
Undergraduate
MATH201 MATH203
Formal lecture
Interactive,Communicative
Click here to view.

CONTENT

Mathematical models for communication channels and sources; entropy, information, lossless data compression, Huffman coding, channel capacity, Shannon's theorems rate-distortion theory.

OBJECTIVE

To learn about information, how to measure it and how to use it to better design information systems.

LEARNING OUTCOMES

  • Define the information content of an information source mathematically and define information theoretical measures such as entropy, conditional entropy, joint entropy mutual information, differential entropy etc.
  • Describe the fundamental limit in source coding and learn Shannon?s Source Coding Theorem
  • Design and implement some of the practical source codes
  • Describe the fundamental limit in maximum information rate at which the information is sent reliably and learn Shannon?s Channel Capacity Theorem.
  • Design and implement some of the practical channel codes
  • Describe the capacity of Gaussian Channel and optimal power allocation over Gaussian Channel using Water-Filling algorithm.
  • Describe the application of information theory to some of engineering problems through the course project.

PROGRAMME OUTCOMES


1. Understand the world, their country, their society, as well as themselves and have awareness of ethical problems, social rights, values and responsibility to the self and to others. 2

2. Understand different disciplines from natural and social sciences to mathematics and art, and develop interdisciplinary approaches in thinking and practice. 1

3. Think critically, follow innovations and developments in science and technology, demonstrate personal and organizational entrepreneurship and engage in life-long learning in various subjects; have the ability to continue to educate him/herself. 3

4. Communicate effectively in Turkish and English by oral, written, graphical and technological means. 4

5. Take individual and team responsibility, function effectively and respectively as an individual and a member or a leader of a team; and have the skills to work effectively in multi-disciplinary teams. 4


1. Possess sufficient knowledge of mathematics, science and program-specific engineering topics; use theoretical and applied knowledge of these areas in complex engineering problems. 3

2. Identify, define, formulate and solve complex engineering problems; choose and apply suitable analysis and modeling methods for this purpose. 4

3. Develop, choose and use modern techniques and tools that are needed for analysis and solution of complex problems faced in engineering applications; possess knowledge of standards used in engineering applications; use information technologies effectively. 4

4. Have the ability to design a complex system, process, instrument or a product under realistic constraints and conditions, with the goal of fulfilling specified needs; apply modern design techniques for this purpose. 1

5. Design and conduct experiments, collect data, analyze and interpret the results to investigate complex engineering problems or program-specific research areas. 1

6. Possess knowledge of business practices such as project management, risk management and change management; awareness on innovation; knowledge of sustainable development. 1

7. Possess knowledge of impact of engineering solutions in a global, economic, environmental, health and societal context; knowledge of contemporary issues; awareness on legal outcomes of engineering solutions; knowledge of behavior according to ethical principles, understanding of professional and ethical responsibility. 1

8. Have the ability to write effective reports and comprehend written reports, prepare design and production reports, make effective presentations, and give and receive clear and intelligible instructions. 1


1. Comprehend key concepts in biology and physiology, with emphasis on molecular genetics, biochemistry and molecular and cell biology as well as advanced mathematics and statistics. 1

2. Develop conceptual background for interfacing of biology with engineering for a professional awareness of contemporary biological research questions and the experimental and theoretical methods used to address them. 1


1. Design, implement, test, and evaluate a computer system, component, or algorithm to meet desired needs and to solve a computational problem. 3

2. Demonstrate knowledge of discrete mathematics and data structures. 3

3. Demonstrate knowledge of probability and statistics, including applications appropriate to computer science and engineering. 5


1. Use mathematics (including derivative and integral calculations, probability and statistics, differential equations, linear algebra, complex variables and discrete mathematics), basic sciences, computer and programming, and electronics engineering knowledge to (a) Design and analyze complex electronic circuits, instruments, software and electronics systems with hardware/software or (b) Design and analyze communication networks and systems, signal processing algorithms or software 5


1. Applying fundamental and advanced knowledge of natural sciences as well as engineering principles to develop and design new materials and establish the relation between internal structure and physical properties using experimental, computational and theoretical tools. 1

2. Merging the existing knowledge on physical properties, design limits and fabrication methods in materials selection for a particular application or to resolve material performance related problems. 1

3. Predicting and understanding the behavior of a material under use in a specific environment knowing the internal structure or vice versa. 1

ASSESSMENT METHODS and CRITERIA

  Percentage (%)
Final 40
Midterm 40
Quiz 10
Participation 10

RECOMENDED or REQUIRED READINGS

Textbook

Thomas and Cover, "Elements of Information Theory", 2nd Edition

Readings

Shannon's seminal papers on Information Theory and Secrecy. Additional papers illustrating the applications of Information Theory in real life systems will also be provided.