Moses - Event-based Robot Vision

Display language

To modulepage Generate PDF

Module / Version:
#40993 / #1

Validity:
Seit SoSe 2020

Default display language:
English

Module title:
Event-based Robot Vision

Credits:
6

Responsible person:
Gallego, Guillermo

Grading:
benotet

Type of exam:
Portfolioprüfung

Zugehörigkeit

Faculty:
Fakultät IV

Institute:
Institut für Technische Informatik und Mikroelektronik

Area of expertise:
34342000 FG Robotic Interactive Perception

Prüfungsausschuss:
No information

Kontakt

Office:
MAR 5-5

Contact person:
No information

E-mail address:
guillermo.gallego@tu-berlin.de

Website:

https://www.digital-future.berlin/en/about-us/professors/prof-dr-guillermo-gallego/

Learning Outcomes

Participants will learn basic concepts, theoretical foundations and relevant algorithms developed in the field of event-based (i.e., neuromorphic) vision. Upon completing the module, participants will have an overview of the field, spanning from the principle of operation of event-based sensors (e.g., event-based cameras), their advantages and disadvantages, to the methods used to process their output for a target application. Participants will also be aware of the differences with standard (frame-based) computer vision, in terms of methods, performance criteria and applications.

Content

This course is the first of its kind, worldwide. To the best of the instructor's knowledge, no similar course has been offered anywhere due to the novelty of the topics covered, which have appeared in research conferences and journals over the last ten years. The topics covered include the following: Bio-inspired principle of operation of event-based (i.e., neuromorphic) sensors. Event-based feature detection and tracking. Event-based motion estimation: optical flow estimation, 3D reconstruction, camera localization and ego-motion estimation, simultaneous localization and mapping (E-SLAM). Stereo depth estimation in dynamic scenes. Image intensity reconstruction from events. Event-based pattern recognition, classification and machine learning. Event-based signal processing and filtering. Event-based sensor fusion. Event-based control. Event-based (i.e., spike-based) hardware. Novel applications in event-based vision.

Module Components

Pflichtgruppe:

All Courses are mandatory.

Course Name	Type	Number	Cycle	Language	SWS	VZ
Event-based Robot Vision	UE	34342000L-002	SoSe	English	2
Event-based Robot Vision	VL	34342000L-001	SoSe	English	4

Workload and Credit Points

Event-based Robot Vision (UE):

Workload description	Multiplier	Hours	Total
Attendance	15.0	2.0h	30.0h
Pre/post processing	15.0	4.0h	60.0h
			90.0h(~3 LP)

Event-based Robot Vision (VL):

Workload description	Multiplier	Hours	Total
Attendance	15.0	4.0h	60.0h
Pre/post processing	15.0	2.0h	30.0h
			90.0h(~3 LP)

The Workload of the module sums up to 180.0 Hours. Therefore the module contains 6 Credits.

Description of Teaching and Learning Methods

The lectures will present the topics (sensors and algorithms) from a theoretical point of view, highlighting the underlying principles and mathematical tools used. Participation/interaction is encouraged and expected, including the possibility of reading assignments. Participants are expected to rehearse topics after class in preparation for the exercises. The exercises take place in parallel. They offer the participants the opportunity to get practical insights about the technology of event-based cameras. Simple event-based processing algorithms will be discussed in detail and partially implemented.

Requirements for participation and examination

Desirable prerequisites for participation in the courses:

The module covers many topics at the forefront of research. It requires a basic knowledge of information technology, image processing, computer vision and machine learning.

Mandatory requirements for the module test application:

This module has no requirements.

Module completion

Grading

graded

Type of exam

Portfolio examination

Type of portfolio examination

100 Punkte insgesamt

Language

English

Test elements

Name	Points	Categorie	Duration/Extent
(Examination) Written test	40	written	75 minutes
(Examination) Written test 2	30	written	75 minutes
(Deliverables) Practice exercises	30	practical	2-4 pages, program code

Grading scale

Notenschlüssel »Notenschlüssel 1: Fak IV (1)«

Gesamtpunktzahl	1.0	1.3	1.7	2.0	2.3	2.7	3.0	3.3	3.7	4.0
100.0pt	86.0pt	82.0pt	78.0pt	74.0pt	70.0pt	66.0pt	62.0pt	58.0pt	54.0pt	50.0pt

Test description (Module completion)

No information

Duration of the Module

The following number of semesters is estimated for taking and completing the module:
1 Semester.

This module may be commenced in the following semesters:
Sommersemester.

Maximum Number of Participants

The maximum capacity of students is 30.

Registration Procedures

For any questions about the module, contact Prof. Gallego guillermo.gallego@tu-berlin.de

Recommended literature
Gallego et al., Event-based Vision: A survey, 2019. https://arxiv.org/abs/1904.08405
Posch et al., Retinomorphic Event-Based Vision Sensors: Bioinspired Cameras With Spiking Output. Proc. IEEE (2014), 102(10):1470-1484
List of Event-based Vision Resources: https://github.com/uzh-rpg/event-based_vision_resources
CVPR 2019 Second International Workshop on Event-based Vision and Smart Cameras - Slides and Videos: http://rpg.ifi.uzh.ch/CVPR19_event_vision_workshop.html
ICRA 2017 First International Workshop on Event-based Vision - Slides and Videos: http://rpg.ifi.uzh.ch/ICRA17_event_vision_workshop.html

Assigned Degree Programs

Semester:

Zeige auch ausgelaufene Studiengänge und StuPOs

This module is used in the following Degree Programs (new System):

Studiengang / StuPO	StuPOs	Verwendungen	Erste Verwendung	Letzte Verwendung
Computer Engineering (M. Sc.)	1	27	SoSe 2020	SoSe 2024
Computer Science (Informatik) (M. Sc.)	1	27	SoSe 2020	SoSe 2024
Elektrotechnik (M. Sc.)	1	18	SoSe 2020	SoSe 2024
Information Systems Management (Wirtschaftsinformatik) (M. Sc.)	1	9	SoSe 2020	SoSe 2024

Miscellaneous

Mastermodul: Kognitive Systeme und Informationssysteme

Event-based Robot Vision