Duckietown

Duckietown, is an environment for studying autonomous driving in a scaled-down manner. It consists of Duckiebots that can move autonomously in a modular environment. The Duckiebots are robots consisting of sensors and motors controlled by an NVIDIA Jetson Nano.

We use this environment to visualize our research in the field of application specific MPSoCs. As a first use case we want to investigate and demonstrate the capabilities and the behavior of our hardware-optimized learning classifier tables (LCTs). These are rule-based reinforcement learning (RL) engines developed in our IPF project.

Initially, we analyze the image processing pipeline of the Duckiebots and apply our LCTs as additional controllers in the autonomous driving application of the Duckiebots. Based on the status of the Duckiebots, the LCTs should help to learn specific behaviors by influencing relevant parameters of the processing pipeline. In a first step, student works use a software version of the LCTs. In subsequent steps the Duckiebots should be extended by an FPGA board, which enables a hardware-based and independent realization of the LCTs.

How to get involved?

As we use Duckietown as a basis for our research and also to demonstrate results in the currently hot application domain of autonomous driving, there are plenty of opportunities for students to contribute.

You can get involved depending on your level of experience / study progress:

  • Already during the bachelor phase you can practically apply your knowledge from LinAlg, StoSi and Regelungstechnik while you improve your coding skills and aquire knowledge about hardware architectures and autonomous driving. Get in touch with us, to talk about open tasks and to get access to our physical setup.
  • Towards the end of your bachelor, it's the first time that you have to write a thesis. For that, have a look at open BA topics below.
    The same applies for research internships (FPs) and master theses (MAs).
  • Depending on the currently planed steps, there might be tasks, which can't be assigned as BA, FP or MA. In such cases we also offer paied working student jobs. Open positions of that type are also listed below.

Thesis Offers

Interested in an internship or a thesis? Please send us an email.
The given type of work is just a guideline and could be changed if needed.
From time to time, there might be some work, that is not announced yet. Feel free to ask!

Assigned Theses

Duckietown - Computer Vision Based Measurements for Performance Analysis

Beschreibung

At LIS, we use Duckietown's hardware and software ecosystem to experiment with our reinforcement learning-based Learning Classifier Tables (LCTs) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/.

More information about Duckietown can be found at https://www.duckietown.org/.

We are currently developing the infrastructure for our LCT experiments.
While several students are improving the autonomous driving capabilities of the Duckiebots, the goal of this work is to develop a system that provides insight into the driving performance of the bots.
Therefore, we need objective measurements from a third party that allow us to compare the driving performance (average speed, deviation from the optimal line, ...) between bots that might be running different SWs.
To be able to compare specific scenarios, these measurements should not only be a time series, but also be enriched with the position of the bot at that moment. Similar to modern sports applications: https://www.outdooractive.com/de/route/wanderung/tegernsee-schliersee/auf-dem-prinzenweg-vom-schliersee-an-den-tegernsee/1374189/#dm=1.

To achieve this goal, it's necessary to develop a camera-based mechanism that identifies bots in the field, extracts their concrete position and calculates certain metrics like direction, speed, on/off lane, spinning, deviation from ideal path, ... .
This data should then be logged for later analysis and visualization.

Depending on the type of work (BA/FP vs. MA), this work may include enhancing these measurements with internal data from the bots to compare internal and external perception, and developing tools for easy analysis.

Possible steps (always with visualization and logging if applicable)

  1.     Camera calibration
  2.     Image segmentation (for on/off lane detection)
  3.     Duckiebot detection
  4.     Speed estimation
  5.     on/off lane detection
  6.     Wrong way detection
  7.     Optimal path calculation
  8.     Deviation from optimal path calculation

Voraussetzungen

  •     Independent work style
  •     Problem solving skills
  •     Computer vision skills - segmentation, object detection
  •     Programming skills in Matlab
  •     Linux basics

Kontakt

flo.maurer@tum.de

Betreuer:

Duckietown - Lane Following with Platooning

Beschreibung

At LIS, we want to use the Duckietown hardware and software ecosystem to experiment with our reinforcement learning-based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/.

More information on Duckietown can be found at https://www.duckietown.org/.

In this student work, we want to extend the bot's current abilities (lane following).

The goal of this work is to enable the bots to follow each other with a constant distance.
At the end, there should be a seamless integration in the Lane Following Pipeline.

Voraussetzungen

  • Knowledge about Image Processing
  • Python

Kontakt

flo.maurer@tum.de
michael.meidinger@tum.de

Betreuer:

Florian Maurer, Michael Meidinger

Duckietown - Lane Detection with Obstacle Avoidance and Intersection Recognition

Beschreibung

At LIS, we want to use the Duckietown hardware and software ecosystem to experiment with our reinforcement learning-based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/.

More information on Duckietown can be found at https://www.duckietown.org/.

In this student work, we want to extend the bot's current abilities (lane following).

The goal of this work is to enable the bots to avoid obstacles on the road (e.g. ducks, other bots, ...) and to stop at intersections (red lines) for a predefined time.
At the end, there should be a seamless integration in the Lane Following Pipeline.

Voraussetzungen

  • Knowledge about Image Processing
  • Python

Kontakt

flo.maurer@tum.de
michael.meidinger@tum.de

Betreuer:

Florian Maurer, Michael Meidinger

Duckietown - Image Processing on FPGAs

Beschreibung

At LIS we want to use the Duckietown hardware and software ecosystem for experimenting with our reinforcement learning based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/

More information on Duckietown can be found on https://www.duckietown.org/.

In this student work, we want to enable the use of the FPGA in the Lane Detection.
Previous work already experimented with the communication between NVIDIA Jetson and the FPGA via a DMA.

Goal of this work is to port the LSD to FPGA to benefit from offloading parts of the Lane Detection Alogithm from the CPU and execute them accelerated on the FPGA.
At the end, there should be a seamless integration in the Lane Following Pipeline.

Voraussetzungen

  • Knowledge about Image Processing
  • Lots of FPGA experience
  • VHDL
  • Python

Kontakt

flo.maurer@tum.de
michael.meidinger@tum.de

Betreuer:

Florian Maurer, Michael Meidinger

Duckietown Bring-Up

Beschreibung

At LIS we want to use the Duckietown hardware and software ecosystem for experimenting with our reinforcement learning based learning classifier tables (LCT) as part of the control system of the Duckiebots: https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/

More information on Duckietown can be found on https://www.duckietown.org/.

Towards this goal, we need a (followup) working student who is improving the current infrastructure.

Towards this goal, the following three major tasks are necessary:

  1. Developping an infrastructure to track and visualize measurement data of the platform (e.g. CPU utilization) as well as the executed application.
  2. During this task also the source and periodicity of already provided data should be analyzed.
  3. Setting up all Duckiebots incl. all their features and a pipeline to reflash them in case it's needed.
  4. FPGA-Extension: Searching for a concept, as well as implementing it.
  5. Final goal: demonstration of data exchange between NVIDIA Jetson and FPGA including protocol to specify the type of transfered data

Kontakt

flo.maurer@tum.de

Betreuer:

Completed Theses

Kontakt

flo.maurer@tum.de

Betreuer:

Florian Maurer, Thomas Wild

Kontakt

flo.maurer@tum.de

Betreuer:

Florian Maurer, Thomas Wild

Kontakt

flo.maurer@tum.de

Betreuer:

Florian Maurer, Thomas Wild

Kontakt

flo.maurer@tum.de

Betreuer:

Florian Maurer, Thomas Wild

Kontakt

flo.maurer@tum.de

Betreuer:

Florian Maurer

Betreuer:

Florian Maurer, Thomas Wild

Betreuer:

Florian Maurer, Thomas Wild

Betreuer:

Florian Maurer, Thomas Wild

Betreuer:

Florian Maurer, Thomas Wild

Betreuer:

Thomas Wild, Florian Maurer

Betreuer:

Thomas Wild, Florian Maurer