Florian Maurer, M.Sc.

At LIS, we want to use the Duckietown hardware and software ecosystem to experiment with our reinforcement learning-based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/.

More information on Duckietown can be found at https://www.duckietown.org/.

In this student work, we want to extend the bot's current abilities (lane following).

The goal of this work is to enable the bots to follow each other with a constant distance.
At the end, there should be a seamless integration in the Lane Following Pipeline.

• Knowledge about Image Processing
• Python

At LIS, we want to use the Duckietown hardware and software ecosystem to experiment with our reinforcement learning-based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/.

More information on Duckietown can be found at https://www.duckietown.org/.

In this student work, we want to extend the bot's current abilities (lane following).

The goal of this work is to enable the bots to avoid obstacles on the road (e.g. ducks, other bots, ...) and to stop at intersections (red lines) for a predefined time.
At the end, there should be a seamless integration in the Lane Following Pipeline.

• Knowledge about Image Processing
• Python

This thesis compares safety implementations for LCTs and decisively determines the superior one through simulations. The aim is to identify the safety mechanism with the best performance without violating any constraints.
To achieve this, different approaches (shielding, forbidden classifier) have to be implemented in MATLAB and good settings have to be found for each implementation.

The project is divided into two phases.

To accomplish our objective, we will implement different approaches, such as the Forbidden Classifier, Preemptive Shielding, and Post-posed Shield. We will compare these implementations with the archive we already have.   Throughout the implementation process, we will determine several properties, including how to build the forbidden classifier table, how to build the shield (i.e., what actions should be valid), and how to set the reward for the post-posed shield. Optimizing performance may require fine-tuning.

After completing the simulation phase in Matlab, we will make a decision on whether to implement the approach in hardware or test it in our Duckietown environment.

At LIS we want to use the Duckietown hardware and software ecosystem for experimenting with our reinforcement learning based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/

More information on Duckietown can be found on https://www.duckietown.org/.

In this student work, we want to enable the use of the FPGA in the Lane Detection.
Previous work already experimented with the communication between NVIDIA Jetson and the FPGA via a DMA.

Goal of this work is to port the LSD to FPGA to benefit from offloading parts of the Lane Detection Alogithm from the CPU and execute them accelerated on the FPGA.
At the end, there should be a seamless integration in the Lane Following Pipeline.

• Knowledge about Image Processing
• Lots of FPGA experience
• VHDL
• Python

In our IPF project, we optimize application execution during runtime by using self-aware DVFS and task mapping algorithms.

For "real-world" testing, we need a packet forwarding application running on our SparcV8 processors. This application should periodically generate and process packets and provide metrics such as "generation time", "scheduling time" and "deadline".
It should also be possible to discard packets if the deadline is exceeded.

It is planned to implement this application within RTEMS.

• Experience in low-level programming (registers, timers, etc)
• Experience with real time operating systems
• Strong problem-solving skills, attention to detail, and the ability to work both independently and collaboratively in a team environment

This Seminar should investigate, which attempts have been made to implement rule-based RL in HW /FPGAs since a first try in 2006: https://www.sciencedirect.com/science/article/pii/S1383762106000208

RL tries to maximies the reward over time.

Therefore, the way the reward is assigned to the agent plays a central role.

Often, the reward is assumed to be predetermined or certain functions are coming from nowhere.

This seminar should investigate reward assignment strategies to solve RL problems.

At LIS we want to use the Duckietown hardware and software ecosystem for experimenting with our reinforcement learning based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/

More information on Duckietown can be found on https://www.duckietown.org/.

In this student work, we want to enable the use of the FPGA in the Lane Detection.
Therefore, the different stages of the Lande Detection Pipeline should be ported to FPGA.
In order to comunicate with the NVIDIA Jetson Nano Platform, the ported algorithm has to connect to the XILINX PCIE DMA IP-Core.

• Knowledge about VHDL and Xilinx IP-cores

At LIS we want to use the Duckietown hardware and software ecosystem for experimenting with our reinforcement learning based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/

More information on Duckietown can be found on https://www.duckietown.org/.

Towards this goal, we need a (followup) working student who is improving the current infrastructure.

Towards this goal, the following three major tasks are necessary:

1. Developping an infrastructure to track and visualize measurement data of the platform (e.g. CPU utilization) as well as the executed application.
2. During this task also the source and periodicity of already provided data should be analyzed.
3. Setting up all Duckiebots incl. all their features and a pipeline to reflash them in case it's needed.
4. FPGA-Extension: Searching for a concept, as well as implementing it.
5. Final goal: demonstration of data exchange between NVIDIA Jetson and FPGA including protocol to specify the type of transfered data