Moses - Quality Assessment for Machine Learning

Display language

Diese Modulbeschreibung gilt nicht im aktuellen Semester.

Die Modulbeschreibung gilt nur für den Zeitraum WiSe 2022/23 - WiSe 2022/23. Die Version für das aktuelle Semester (SoSe 2024) finden Sie hier: Modulbeschreibung des SoSe 2024

Module / Version:
#41069 / #1

Validity:
WiSe 2022/23 - WiSe 2022/23

Default display language:
English

Module title:
Quality Assessment for Machine Learning

Credits:
3

Responsible person:
Haufe, Stefan

Grading:
benotet

Type of exam:
Portfolioprüfung

Zugehörigkeit

Faculty:
Fakultät IV

Institute:
Institut für Softwaretechnik und Theoretische Informatik

Area of expertise:
34352700 FG S-Professur Unsicherheit, inverse Modellierung und maschinelles Lernen

Prüfungsausschuss:
No information

Kontakt

Office:
MAR 4-4

Contact person:
Haufe, Stefan

E-mail address:
haufe@tu-berlin.de

Website:

https://www.tu.berlin/uniml/

Learning Outcomes

Knowledge: Students will be familiar with the requirements imposed on AI systems by current and prospective regulatory frameworks. They will know different quality dimensions of AI systems and will have elaborated procedures and metrics for measuring quality along these dimensions and technical means for assuring high quality. Skills: Students will acquire practical skills to debug data science applications by analyzing data and/or code. In addition, students will also acquire or refine skills to independently review and systematically structure the literature of a well circumscribed field in order to address a given set of questions, and will gain experience in presenting the outcome to a critical audience as well as in participating in scientific discussions. Competencies: Students will be able to analyze the the risks and failure modes of a given AI system or product. They will be able to suggest procedures and tests to benchmark whether a given system/product will function as intended in a variety of settings and suggest ways to overcome unwanted system behavior. Students will also be able to discuss ethical, economic and other implications of failure modes of AI systems.

Content

- motivating examples and use cases from the medical, automotive and finance domains - taxonomy of use cases, associated risks and failure modes - legal approaches to regulate AI systems for critical applications - ethical considerations - quality dimensions for AI/ML systems * data quality * model performance, generalization * model robustness * model fairness * transparency * uncertainty calibration * model interpretation and types of explainability * data privacy - quantitative metrics and test to measure quality - practical approaches to ensure/improve quality - simulations and benchmarks - current standardization efforts

Module Components

Pflichtgruppe:

All Courses are mandatory.

Course Name	Type	Number	Cycle	Language	SWS	VZ
Quality Assurance for Machine Learning	SEM		WiSe/SoSe	English	2

Workload and Credit Points

Quality Assurance for Machine Learning (SEM):

Workload description	Multiplier	Hours	Total
Attendance	15.0	2.0h	30.0h
Pre/post processing	15.0	4.0h	60.0h
			90.0h(~3 LP)

The Workload of the module sums up to 90.0 Hours. Therefore the module contains 3 Credits.

Description of Teaching and Learning Methods

The module currently consists of a seminar. Students will prepare a presentation to a specific topic based on a provided collection of published material. Student presentations will be framed by short lecture segments introducing, contextualizing and connecting the presented topics. Each course slot will contain discussion periods, in which active participation is fostered. Students will also conduct a group work to analyze a given AI system, and present their results to the seminar audience. Moreover, students will complete a homework.

Requirements for participation and examination

Desirable prerequisites for participation in the courses:

A BSc degree in Computer Science is recommended. Successful completion of an introductory module on ML such as "Machine Learning I" or "Machine Intelligence I" is recommended. Programming skills in at least one language (e.g., Matlab, Python, R) are required.

Mandatory requirements for the module test application:

This module has no requirements.

Module completion

Grading

graded

Type of exam

Portfolio examination

Type of portfolio examination

100 Punkte insgesamt

Language

English

Test elements

Name	Points	Categorie	Duration/Extent
(Deliverable assessment) Presentation	50	oral	30 min
(Deliverable assessment) Group work	25	oral	60 min
(Deliverable assessment) Homework	25	practical	3 pages

Grading scale

Notenschlüssel »Notenschlüssel 1: Fak IV (1)«

Gesamtpunktzahl	1.0	1.3	1.7	2.0	2.3	2.7	3.0	3.3	3.7	4.0
100.0pt	86.0pt	82.0pt	78.0pt	74.0pt	70.0pt	66.0pt	62.0pt	58.0pt	54.0pt	50.0pt

Test description (Module completion)

The module grade is calculated based on 1. The quality of a paper presentation (50%). 2. The quality of the presentation of a group work (25%). 3. The quality of a homework (25%).

Duration of the Module

The following number of semesters is estimated for taking and completing the module:
1 Semester.

This module may be commenced in the following semesters:
Wintersemester.

Maximum Number of Participants

The maximum capacity of students is 25.

Registration Procedures

Students can sign up for the course in MOSES.

Recommended literature
No recommended literature given

Assigned Degree Programs

Semester:

Zeige auch ausgelaufene Studiengänge und StuPOs

This module is used in the following Degree Programs (new System):

	Studiengang / StuPO	StuPOs	Verwendungen	Erste Verwendung	Letzte Verwendung
This module is not used in any degree program.

Miscellaneous

No information

Quality Assessment for Machine Learning