Bachelor Theses

At LIS, we leverage the Duckietown hardware and software ecosystem to experiment with our reinforcement learning (RL) agents, known as learning classifier tables (LCTs), as part of the Duckiebot control system. More information on Duckietown can be found here.
In previous work, the default deterministic controller for the robot's speed was replaced by an RL agent. It uses an LCT with current speed as the system state and actions that cause relative speed changes. To enable platooning via this agent, a second version includes the distance to a Duckiebot driving ahead as a second entry to the state vector. With this limited set of rules and a modified SARSA/Q-learning method for Q-value updates, the agent quickly learns to let the bot drive at a given target speed.
However, the RL-based speed controller does not consider the track's properties. A given target speed might be too high to take turns properly, especially when the agent is eventually combined with the separately developed steering agent. On the other hand, consistently letting the robot drive slowly would be a waste of time on a mostly straight track.
To resolve this problem, this thesis should introduce the curvature of the track into the state observation. To detect curves in the camera images, the student will extend the image processing pipeline beyond how the steering mechanism takes care of turns. The student will expand the rule set, adapt the reward function, and investigate the effects on learning performance. A likely consequence of a more extensive rule set is the need for a more advanced Q-value update method. Additional improvements might concern the speed measurement accuracy of the wheel encoders. The enhanced speed controller will be evaluated for learning speed, driving performance in terms of deviation from target speed in different scenarios, and resource utilization.
With these extensions to the speed controller, those to the steering controller, and the object recognition algorithm developed in further student work, we will set the foundation to deploy our RL-based Duckiebots in more difficult-to-navigate environments, e.g., tracks with intersections and sharp turns.

At LIS, we leverage the Duckietown hardware and software ecosystem to experiment with our reinforcement learning (RL) agents, known as learning classifier tables (LCTs), as part of the Duckiebot control system. More information on Duckietown can be found here.
In previous work, an algorithm to detect obstacles in the path of a Duckiebot was developed. It uses its Time-of-Flight (ToF) sensor for general obstacles in front of the robot. It can also specifically detect duckies in the camera image, mainly used for lane detection, by creating a color-matching mask with bounding rectangles within a certain size range and position on the track. If the robot detects any obstacle in its path, it slows down, then stops and waits for the obstacle to disappear if it is too close.
While this algorithm works well for large obstacles right in front of the ToF sensor and duckies on straight tracks, it struggles to detect obstacles in other scenarios. The ToF sensor only covers a narrow measurement region and misses objects that are off-center or too high/low. Its measurement results are also not very reliable, especially for higher distances. The camera-based recognition sometimes interchanges duckies with the track's equally yellow centerline. It can fail to detect them due to blind spots (caused by a dynamic region of interest, not an actual blind spot of the camera) when driving on a curved track.
We also want to include other objects besides duckies in our recognition algorithm, e.g., stop lines or traffic signs at intersections. Since the camera approach is tuned to the size and color of duckies, manual effort would be needed to extend it to different objects. Therefore, in this Bachelor's thesis, we want to overhaul our object recognition to be more reliable and detect various objects in different scenarios. A good approach could be using the YOLO (You Only Look Once) algorithm, a single-pass real-time object detection algorithm based on a convolutional neural network. There is also some related work around this topic in the Duckietown community.
The student will start with a continued analysis of the problems of the previous method and literature research regarding viable object detection algorithms. Afterward, they will implement and integrate the selected algorithm into the existing framework. Depending on the approach, some training will be necessary. The student will start with a continued analysis of the current problems and literature research regarding viable object detection algorithms. Afterward, they will implement and integrate the selected algorithm into the existing framework. Depending on the approach, some training will be necessary. Furthermore, they will evaluate the detection pipeline for accuracy in different scenarios, latency, and resource utilization. Once a reliable detection system is established, our system can be extended to more complex behavior, such as circumnavigating duckies or responding to intersectional traffic signs.

Prerequisites

Their main advantages are an easy design with only 1 Transistor per Bit and a high memory density make DRAM omnipresend in most computer architectures. However, DRAM accesses are rather slow and require a dedicated DRAM controller
that coordinates the read and write accesses to the DRAM as well as the refresh cycles. In order to reduce the DRAM access latency, memory prefetching is a common technique to access data prior to their actual usage. However, this requires sophisticated prediction algorithms in order to prefetch the right data at the right time.

The Goal of this thesis is to extendan existing DRAM preloading mechanism on an FPGA based prototype platform by a hardware performance counter and statistics module. This requires a profund understanding of AHB communication protocolls as well as the functionalities of the cache and memory hierarchie of an MPSoC system. The new component should be integrated in the whole architecture design and tested and evaluated under different scenarios.

Towards this goal, you'll complete the following tasks:
1. Understanding the existing Memory Access and Preloading mechanism
2. VHDL implementation of the refined preloading functionalities
3. Write and execute small baremetal test programs
4. Analyse and discuss the performance results

Prerequisites

Their main advantages are an easy design with only 1 Transistor per Bit and a high memory density make DRAM omnipresend in most computer architectures. However, DRAM accesses are rather slow and require a dedicated DRAM controller
that coordinates the read and write accesses to the DRAM as well as the refresh cycles. In order to reduce the DRAM access latency, memory prefetching is a common technique to access data prior to their actual usage. However, this requires sophisticated prediction algorithms in order to prefetch the right data at the right time.

The Goal of this thesis is to transfer an existing DRAM preloading mechanism to an FPGA based prototype platform of the RISC-V CVA6 architecture. This requires a profund understanding of AHB and AXI communication protocolls as well as the functionalities of the cache and memory hierarchie of an MPSoC system.

Towards this goal, you'll complete the following tasks:
1. Understanding the existing Memory Access and Preloading mechanism
2. VHDL implementation of the refined preloading functionalities
3. Write and execute small baremetal test programs
4. Analyse and discuss the performance results

Prerequisites

With the advent of research on the next generation of
mobile communications 6G, we are engaged in exploring
architecture extensions for Smart Network Interface Cards
(SmartNICs). To enable adaptive, energy-efficient and
low-latency network interfaces, we are prototyping a
custom packet processing pipeline on FPGA-based NICs,
partially based on the open-nic project
(https://github.com/Xilinx/open-nic).

To test the performance of a SmartNIC-assisted server
under peak loads and achieve precise measurements of
key performance indicators (KPIs) such as throughput and latency, high performance packet trace replay and measurements are required. As software mechanisms for replay of 100Gbps traffic is difficult, an FPGA-based Network Tester for 100 Gbps links shall be implemented and tested. For this, the Alveo U55C FPGA-based SmartNICs shall be used. A previous implementation for 10Gbps links (FlueNT10G) can be used for reference.

The goal of this work is to implement the required logic modules in HDL (Verilog), integrate these modules into the OpenNIC Shell platform and test the design on the Alveo U55C FPGAs. Additionally, a software-interface to control the network tester and feed the packet traces should be adapted from the FlueNT10G. The design should also be evaluated regarding the performance of the packet replay as well as the precision in throughput and latency measurement.

Prerequisites

At LIS, we leverage the Duckietown hardware and software ecosystem to experiment with our reinforcement learning (RL) agents, known as learning classifier tables (LCTs), as part of the Duckiebots' control system (https://www.ce.cit.tum.de/lis/forschung/aktuelle-projekte/duckietown-lab/).
More information on Duckietown can be found at https://www.duckietown.org/.

In previous work, an LCT agent to steer Duckiebots has been developed using only the angular heading error for the system state. In this Bachelor's thesis, the vehicle steering agent should be improved and its functionality extended.
Starting with the existing Python/ROS implementation of the RL agent and our image processing pipeline, multiple system parts should be enhanced. On the environment side, detecting the lateral offset from the center of a lane should be improved for reliability. This will require an analysis of the current problems and some adaptations in the pipeline, possibly some hardware changes.
With more reliable lane offset values, the agent's state observation can also include it, allowing us to move further from the default PID control towards a purely RL-based steering approach. This will involve modifications to the rule population, the reward function, and potentially the learning method. Different configurations are to be implemented and evaluated in terms of their resulting performance and efficiency.
The thesis aims to shift the vehicle steering entirely to the RL agent, ideally reducing the effort for manual parameter tuning while being comparable in driving performance and computation effort.

Prerequisites

Random Linear Network Coding (RLNC) is a coding scheme commonly used in wireless networks. In RLNC, packets are encoded as linear combinations of a set of source packets (called a generation) before being sent over the network. As the transmitted packets are linear combinations, any lost or dropped packet may be recovered by the sender. And in addition to allowing endpoints to recover individual packets in the case of packet loss, RLNC also increases throughput in random networks.

However, the use of RLNC is rare in wired networks and few studies have been done to determine its performance in these circumstances. It would therefore be useful to have an RLNC implementation running on an FPGA-based SmartNIC, so that it can be evaluated in real-world systems.

In this project, the student will have to design and implement a hardware encoder for select RLNC schemes, such as PACE and convolutional coding. Furthermore, the student should incorperate the decoder into our SmartNIC platform and test the system in-network. 

Prerequisites