Content
Both the sciences and industry are currently undergoing a profound transformation: large-scale, diverse data sets - derived from Internet-of-Things sensors, the web, or via crowd sourcing - present a huge opportunity for data-driven decision making. This data poses new challenges in a variety of dimensions: in its unprecedented volume, in the speed at which it is generated (its velocity) and in the variety of data sources that need to be integrated. The field of Big Data Systems deals with the technological means of processing high-volume of data to gain insights from data. A whole new breed of systems and paradigms has been developed to to cope with that these challenges. However, current systems still fall short in addressing many user needs. Therefore, further system research is necessary to achieve robust query performance in any scenario. As a result, students will conduct projects that deal with topics related to current data management trends, such as Modern Hardware, Stateful Data Stream Processing, Sensor Data Management, Compiler Technology, Query Optimization, Machine Learning for Databases, and many others. This scope of the project will be adjusted to the final group size to reflect the overall workload of the course (i.e., 270h of work per student) For that, students will learn the algorithms, system design, and actual implementation of the so called Distributed Processing Platforms (e.g., Flink, NebulaStream, Spark). These are systems that execute parallel computations on terabytes of data on clusters as well as distributed Internet-of-Things topologies of up to several thousand machines. At the start of the project, a student will receive a topic as well as some information material. The team, with the assistance of the lecturer, will decide on a project environment with the suitable tools for team work, project communication, development and testing. Next, the problem will have to be analyzed, modelled and decomposed into individual components, from which tasks are derived that are subsequently assigned to smaller teams or individuals. At weekly project meetings, the project team presents progress and milestones that have been reached. In consultation with the lecturer, it is decided which further steps to take. The project is concluded with a final presentation which includes a demonstration of the prototype.
