Moses - Information Theory

Display language

To modulepage Generate PDF

Diese Modulbeschreibung gilt nicht im aktuellen Semester.

Die Modulbeschreibung gilt nur für den Zeitraum SS 2017 - SS 2019.

Module / Version:
#40230 / #2

Validity:
SS 2017 - SS 2019

Default display language:
English

Module title:
Information Theory

Credits:
6

Responsible person:
Caire, Giuseppe

Grading:
benotet

Type of exam:
Portfolioprüfung

Zugehörigkeit

Faculty:
Fakultät IV

Institute:
Institut für Telekommunikationssysteme

Area of expertise:
34331600 FG Theoretische Grundlagen der Kommunikationstechnik

Prüfungsausschuss:
No information

Kontakt

Office:
HFT 6

Contact person:
Caire, Giuseppe

E-mail address:
caire@tu-berlin.de

Website:

http://www.commit.tu-berlin.de/menue/lehre/parameter/en/

Learning Outcomes

Students are presented with fundamental and advanced concepts in Information Theory. The module is formed by two courses in sequence. In the first course, basic information measures and single-source/single-destination problems are presented. In the second course, multiuser network problems are presented. Information theory forms the theoretical foundation of communication networks, data compression, data storage, and source coding, such as video and audio coding.

Content

- Definition and main properties of entropy, cross-entropy (information divergence) and mutual information - Typical sequences and typical sets, fundamental lemmas of typicality. - Data compression, Human codes, arithmetic codes, notion of universal lossless source coding (Lempel-Ziv algorithm). - Channel capacity of discrete memoryless channels - Gaussian channels, spectral efficiency, waterfilling power allocation with relation to bit-loading and OFDM. - Rate-distortion theory and quantization, reverse waterfilling with relation to subband coding, successive refinement coding, with reference to scalable image and video coding. - The Multiple Access Channel - The Broadcast Channel - The Interference Channel - Graphical networks and network coding - Relay networks and noisy network coding

Module Components

Pflichtgruppe:

All Courses are mandatory.

Course Name	Type	Number	Cycle	Language	SWS	VZ
Information Theory	VL	0432 L 654	WiSe	No information	2
Network Information Theory	VL	34331600 L 001	SoSe	No information	2

Workload and Credit Points

Information Theory (VL):

Workload description	Multiplier	Hours	Total
Attendance time	15.0	2.0h	30.0h
Examination preparation	1.0	30.0h	30.0h
Preparation and postprocessing	15.0	2.0h	30.0h
			90.0h(~3 LP)

Network Information Theory (VL):

Workload description	Multiplier	Hours	Total
Attendance time	15.0	2.0h	30.0h
Examination preparation	1.0	30.0h	30.0h
Preparation and postprocessing	15.0	2.0h	30.0h
			90.0h(~3 LP)

The Workload of the module sums up to 180.0 Hours. Therefore the module contains 6 Credits.

Description of Teaching and Learning Methods

The module consists of conventional frontal teaching in class, developing theoretical and mathematical concepts, exercises developed in class, in order to develop problem-solving skills and reinforce comprehension of the theory, and homework exercises in order to develop independent and autonomous thinking skills in the students.

Requirements for participation and examination

Desirable prerequisites for participation in the courses:

Prerequisite for participation to courses are a mathematical background at the level of beginning MS students in Electrical Engineering (multivariate calculus, Fourier and Laplace Transforms, signals and systems, linear algebra and notions of matrix theory). The course is open to students enrolled in any MSc in EE CS, Mathematics and Physics

Mandatory requirements for the module test application:

This module has no requirements.

Module completion

Grading

graded

Type of exam

Portfolio examination

Type of portfolio examination

100 Punkte insgesamt

Language

English

Test elements

Name	Points	Categorie	Duration/Extent
(Deliverable assessment) Project presentation	40	oral	20 min
(Examination) Oral examination/discussion	20	oral	20 min
(Examination) Written test	40	written	90 min

Grading scale

Notenschlüssel »Notenschlüssel 3: Fak IV (3)«

Gesamtpunktzahl	1.0	1.3	1.7	2.0	2.3	2.7	3.0	3.3	3.7	4.0
100.0pt	85.0pt	80.0pt	75.0pt	70.0pt	65.0pt	60.0pt	55.0pt	50.0pt	45.0pt	40.0pt

Test description (Module completion)

The final grade according to § 47 (2) AllgStuPO will be calculated according to Notenschlüssel 3 of Faculty IV.

Duration of the Module

The following number of semesters is estimated for taking and completing the module:
2 Semester.

This module may be commenced in the following semesters:
Winter- und Sommersemester.

Maximum Number of Participants

This module is not limited to a number of students.

Registration Procedures

Course teaching and organization (not module examination enrollment at Examination office/Prüfungsamt) is supported by an ISIS course. Registration details are provided at the beginning of the module.

Recommended literature
El Gamal, Abbas, and Young-Han Kim. Network information theory. Cambridge University Press, 2011.
T. Cover and J. Thomas, Elements of Information Theory, 2nd Ed., Wiley Interscience, 2006.

Assigned Degree Programs

Semester:

Zeige auch ausgelaufene Studiengänge und StuPOs

This module is used in the following Degree Programs (new System):

	Studiengang / StuPO	StuPOs	Verwendungen	Erste Verwendung	Letzte Verwendung
This module is not used in any degree program.

Students of other degrees can participate in this module without capacity testing.

Miscellaneous

The module is formed by two courses, taking place in two consecutive semesters (WS - SS)

Information Theory