Student projects and final year projects at the Chair of Media Technology

We constantly offer topics for student projects (engineering experience, research experience, student, IDPs) and final year projects (bachelor thesis or master thesis).

 

Open Thesis

Incorporating AoA information to MPDC fingerprinting for indoor localization

Keywords:
indoor positioning; fingerprinting; ULA

Description

Fuse AoA information with multipath delay componens for robust indoor localization.

Contact

fabian.seguel@tum.de

of 1916

Supervisor:

Fabian Esteban Seguel Gonzalez

Human-Object Interaction Detection in Videos

Keywords:
Deep Learning, Spatio-temporal feature, Object detection, Transformer.

Description

We identify semantic information about human and object in the video by object detection and human detection and later classify the interaction information by a Transformer model. A public dataset will be used and we will collect the dataset ourselves (we provide the sensors and code to collect the dataset).

Prerequisites

  • Good programming skills in Python, Pytorch
  • Knowledge of computer vision
  • Basic knowledge of handling sequential data

Contact

yuankai.wu@tum.de

Supervisor:

Yuankai Wu

Multiband evaluation of passive signal for human activity recognition

Keywords:
CSI; human activity recognition; AI

Description

Obtain signal information to determine different human activities tasks. Use nexmon firmware package.

https://ieeexplore.ieee.org/document/8307394

Prerequisites

Python

AI/ML

Raspberry pi

Contact

fabian.seguel@tum.de

of. 1916

Supervisor:

Fabian Esteban Seguel Gonzalez

Optimizing network deployment for AI-aided AoA estimation

Keywords:
AI; Network deployment; Indoor positioning

Description

The student will optimize through simulations the network atchitecture in order to obtain a robust positioning  estimate in a indoor set-up.

 

 

Prerequisites

matlab

AI/ML

 

Contact

fabian.seguel@tum.de

of. 1916

Supervisor:

Fabian Esteban Seguel Gonzalez

OCC for indoor positioning and identification in harsh environments

Keywords:
optical camera communications; video tracking
Short Description:
Identify and track multiple light sources in a video stream

Description

Identify and track multiple light sources in a  video stream.

The student must record the video multiple objects to the tracked in the video stream.

Prerequisites

Python

Signal/image processing

 

Contact

fabian.seguel@tum.de

of. 1916

Supervisor:

Fabian Esteban Seguel Gonzalez

Multiband analysis of mm-Waves for passive patient monitoring

Keywords:
IQ signals; RADAR; mm-wave
Short Description:
In bed patient monitoring using mm-wave radar systems

Description

The student must use a radar sytem to obtain vital signs of a patient.

The system must be embedded in a hospital bed.

Vital signs such as breathing rate and HR will be targeted; Others applications must be discussed.

 

 

Prerequisites

Python

Radar

IQ signal processing

Contact

 

fabian.seguel@tum.de

of. 1916

 

Supervisor:

Fabian Esteban Seguel Gonzalez

Sub-band analysis for indoor positioning: extracting robust features

Keywords:
OFDMA; CSI; indoor positioning
Short Description:
To obtain samples

Description

The student must set up a transmission scheme based on MIMO technology and OFDMA.

Take samples in different points inside an indoor environment for further processing of the signal characteristics to obtain an estimated position of the mobile node.

Prerequisites

Python

WIreless communications with focus in channel state information and OFDMA MIMO systems

Knowledge in USRP not required but is a plus

Contact

fabian.seguel@tum.de

of. 1916

Supervisor:

Fabian Esteban Seguel Gonzalez

Prior-based 3D Reconstruction

Keywords:
3D Reconstruction, 3D Point Clouds, Digital Twin

Description

The generation of a Digital Twin through 3D reconstruction is and will be a major topic for AR applications, 6G communication, and robotics.

Following [1], this thesis also follows the approach of using prior CAD database information for the 3D reconstruction. This thesis may use self-attention-based networks, confidence-part-based shape retrieval [2], and generative networks to partially reconstruct/ partially retrieve objects, thus achieving higher accuracy.

References

[1] Bokhovkin, Alexey and Angela Dai. “Neural Part Priors: Learning to Optimize Part-Based Object Completion in RGB-D Scans.” ArXiv  abs/2203.09375 (2022): n. pag. https://arxiv.org/abs/2203.09375

[2] Beyer, Tim and Angela Dai. “Weakly-Supervised End-to-End CAD Retrieval to Scan Objects.” ArXiv abs/2203.12873 (2022): n. pag. https://arxiv.org/abs/2203.12873

[3] Tang, Jiaxiang, Xiaokang Chen, Jingbo Wang and Gang Zeng. “Point Scene Understanding via Disentangled Instance Mesh Reconstruction.” ArXiv abs/2203.16832 (2022): n. pag. https://arxiv.org/abs/2203.16832

Prerequisites

  • Experience with Git
  • Python (Pytorch)
  • Knowledge in working with 3D Point Clouds and Meshes (preferable)

Contact

driton.salihu@tum.de

Supervisor:

Driton Salihu

Optimization of 3D Object Detection Procedures for Indoor Environments

Keywords:
3D Object Detection, 3D Point Clouds, Digital Twin, Optimization

Description

3D object detection has been a major task for point cloud-based 3D reconstruction of indoor environments. Current research has focused on having a low inference time for 3D object detection. While this is preferable, a lot of cases do not profit from this. Especially considering the use of a pre-defined static Digital Twin for AR and robotics application, thus this decreases the incentive for low inference time at the cost of accuracy.

As such this thesis will follow the approach of [1] (in this work only based on point cloud data) to generate proposals of layout and objects in a scene through for example [2]/[3] and use some form of optimization algorithm (reinforcement learning, genetic algorithm) to optimize to the correct solution.

Further, for more geometrical-reasonable results the use of a relationship graph neural network, as in [4], would be applied in the pipeline.

References

[1] Hampali, Shreyas, et al. “Monte Carlo Scene Search for 3D Scene Understanding.” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)  (2021): 13799-13808. https://arxiv.org/abs/2103.07969#:~:text=We explore how a general, from noisy RGB-D scans.

[2] Chen, Xiaoxue, Hao Zhao, Guyue Zhou, and Ya-Qin Zhang. “PQ-Transformer: Jointly Parsing 3D Objects and Layouts From Point Clouds.” IEEE Robotics and Automation Letters  7 (2022): 2519-2526. https://arxiv.org/abs/2109.05566

[3] Qi, C., Or Litany, Kaiming He and Leonidas J. Guibas. “Deep Hough Voting for 3D Object Detection in Point Clouds.” 2019 IEEE/CVF International Conference on Computer Vision (ICCV)  (2019): 9276-9285. https://arxiv.org/abs/1904.09664

[4] Avetisyan, Armen, Tatiana Khanova, Christopher Bongsoo Choy, Denver Dash, Angela Dai and Matthias Nießner. “SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans.” ArXiv  abs/2003.12622 (2020): n. pag. https://arxiv.org/abs/2003.12622

 

Prerequisites

  • Python (Pytorch)
  • Experience with Git
  • Knowledge in working with 3D Point Clouds (preferable)
  • Knowledge about optimization methods (preferable)

Contact

driton.salihu@tum.de

Supervisor:

Driton Salihu

Development of a Zoom Chatbot for Virtual Audience Feedback

Description

Virtual conference systems provide an alternative to physical meetings that have significantly grown in importance over the last years. However, larger events require the audience to be muted to avoid an accumulation of background noise and distorted audio. While this is sufficient for unidirectional meetings, many types of meetings strongly rely on the feedback of their audience, such as in performing arts.
In this project, we want to extend Zoom sessions with a simple Chatbot that collects the audience participation of each user using a straightforward button interface. Then, the system renders the overall audience feedback based on the feedback state collected from each user. The project combines signal and audio processing with the chance to gain practical experience with app development and SDKs.

References

Prerequisites

  • Good knowledge in Nodejs/JavaScript

  • Experience with Git

  • Experience with Zoom SDK would be a plus

Supervisor:

High-level Robotic Teleoperation via Scene Editing

Description

Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on user interaction methods for describing a robotic task via modifying the viewed scene.

 

Prerequisites

Useful background:

- 3D Human-computer interfaces

- Game Design

- Digital signal processing

 

Required abilities:

- Experience with Unity and ROS

- Motivation to yield a good work

 

 

Contact

furkan.kaynar@tum.de

 

(Please provide your CV and transcript in your application)

 

 

Supervisor:

Hasan Furkan Kaynar

Robotic grasp learning from human demonstrations

Keywords:

Description

Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on learning from human demonstration for improving the robot autonomy.

Prerequisites

Required background:

- Digital signal processing

- Computer vision

- Neural networks and other ML algorithms

 

Required abilities:

- Experience with Python or C++

- Experience with Tensorflow or PyTorch

- Motivation to yield a good thesis

 

 

Contact

furkan.kaynar@tum.de

 

(Please provide your CV and transcript in your application)

 

 

Supervisor:

Hasan Furkan Kaynar

Reinforcement Learning for Estimating Virtual Fixture Geometry to Improve Robotic Manipulation

Description

Robotic teleoperation is often used to accomplish complex tasks remotely with human-in-the-loop. In cases, where the task requires very precise manipulation, virtual fixtures can be used to restrict and guide the motion of the end effector of the robot while the person teleoperates. In this thesis, we will analyze the geometry of virtual fixtures depending on the scene and task. We will use reinforcement learning to estimate ideal virtual fixture model parameters. At the end of the thesis, the performance can be evaluated with user experiments.

Prerequisites

Useful background:

- Machine learning (Reinforcement Learning)

- Robotic simulation

 

Requirements:

- Experience with Python & Deep learning frameworks (PyTorch / Tensorflow...)

- Experience with a RL framework

- Motivation to yield a good outcome

 

Contact

(Please provide your CV and transcript in your application)

 

furkan.kaynar@tum.de

diego.prado@tum.de

 

Supervisor:

Hasan Furkan Kaynar, Diego Fernandez Prado

Jacobian Null-space Energy Dissipation TDPA for Redundancy Robots in Teleoperation

Keywords:
Teleoperation, Robotics, Control Theory

Description

Teleoperation Systems

 

Bilateral teleoperation with haptic feedback provides its users with a new dimension of immersion in virtual or remote environments. This technology enables a great variety of applications in robotics and virtual reality, such as remote surgery and industrial digital twin [1]. Figure 1 shows a generalized human-in-the-loop teleoperation system with kinesthetic feedback, where the operator commands a remote/virtual robot to explore the environment and experiences the interactive force feedback through the haptic interface. 

Teleoperation systems face many challenges caused by unpredictable environment changes, time-delayed feedback, limited network capacity, etc. [2]. These issues inevitably distort the force feedback signal, degrading the transparency and stability of the system. In the past decades, many control algorithms and hardware architectures were developed to tackle these problems in the past decades [3].

Time Domain Passivity Method (TDPA)

 

TDPA is a passivity-based control scheme that ensures the stability of teleoperation systems in the presence of communication delays [4] (See Figure 2.). It abstracts two-port networks from the haptic system and observes the energy flow between the networks. Passivity condition is maintained by dissipating extra energy generated by non-passive networks. Original TDAP suffers from position drift and feedback force jump [5], and one reason for the position drift is that the energy, which is generated by the delayed communication, is dissipated in the task space of the robots.

 

Jacobian Null-Space for Redundancy Robot

 

Many robot mechanisms have redundant degrees of freedom (rDOFs), which means that they have a larger number of joints than the number of dimensions of their task or configuration space. The null space of the Jacobian null space stands for the redundant dimensions which can be exploited to dissipate extra energy by damping the null space motion without affecting the task space [5].

Your Task and Target

In this work, we target at improving the performance of TDPA by considering dissipating energy generated by time delay and other factors in the Jacobian null-space of the kinesthetically redundant robots. With the help of the Jacobian null-space method, we can avoid dissipating energy in the task space, so as to alleviate position drift and force distortion while keeping the system passive. For more information, previous work can be referred to as [7-9].

In this master's internship, your work will include

1.      1. surveying the related algorithms

2.      2. constructing the simulation environment

3.      3. experimenting with the state-of-the-art Jacobian null-space TDPA method.

4.      4. analyzing system passivity in Cartesian task space, joint space, null space, etc.

Prerequisites

Requirements

All requirements are recommended but not mandatory. However, you will need extra effort to catch up if you are unfamiliar with the following topics:

1.    3. Basic knowledge about robotics and control theory is favorable.

2.    2. Experience with robotics simulation software and platforms is favorable.

3.    1. C++, Matlab, and Python would be the primary working language. Basic knowledge about one or more of them is highly recommended.

Contact

zican.wang@tum.de

xiao.xu@tum.de

Supervisor:

Zican Wang, Xiao Xu

Student work in game-engine based human activity simulation

Description

This is a working student position at the Human Activity Understanding Group at the Chair of Media Technology.

The student will support various initiatives in game-engine based 3D simulation. We work with both Unity3D as well as Unreal Engine. Some videos of our simulator in action can be seen at: https://www.ce.cit.tum.de/lmt/forschung/themen/human-activity-understanding/

In particular, the student will extend and maintain the existing human activity simulators, and also work on interfacing VR hardware such as:

  • Head-mounted displays like Oculus or Vive, Microsoft Hololens
  • Motion capture system (OptiTrack)
  • VR gloves
  • Custom hardware setups

Also see: https://portal.mytum.de/schwarzesbrett/hiwi_stellen/NewsArticle_20220407_071536

Prerequisites

Strong interest and experience in game development and 3D simulation. Experience with one of ideally both Unity3D and Unreal Engine, OR experience with Blender and interest in learning Unity3D/Unreal.

 

Please send in your CV and describe your experience at the time of application.

Contact

https://www.ce.cit.tum.de/lmt/team/mitarbeiter/chaudhari-rahul/

Supervisor:

Rahul Chaudhari

Learning-based human-robot shared autonomy

Keywords:
robot learning, shared control

Description

In shared control teleoperation, the robot intelligence and human input can be blended together to achieve improved task performance and reduce the human workload. In this topic, we would like to investigate how we can combine human input and robot intelligence effectively to achieve at the end full robot autonomy. We will employ robot learning from demonstration approaches, where we provide task demonstrations using teleoperation. 

We aim to test the developed algorithms in simulation and using Franka Emika robot arm.

Requirements:

Basic experience in C/C++

ROS is a plus

High motivation to learn and conduct research

Supervisor:

MATLAB tutor for Digital Signal Processing lecture in summer semester 2022

Description

Tasks:

  • Help students with the basics of MATLAB (e.g. matrix operations, filtering, image processing, runtime errors)
  • Correct some of the homework problems 
  • Understand the DSP coursework material

 

We offer:

  • Payment according to the working hours and academic qualification
  • The workload is approximately 6 hours per week from May 2022 to August 2022
  • Technische Universität München especially welcomes applications from female applicants

 

Application:

  • Please send your application with a CV and transcript per e-mail to basak.guelecyuez@tum.de 
  • Students who have taken DSP course preferred.

Supervisor:

Embodied Object Detection in Real Environments

Description

In this project, we investigate embodied (active) object detection in real-world environments. The goal is to actively adapt the path of an autonomous agent to increase the performance of an object detector.

Prerequisites

  • Object detection
  • Pytorch
  • Knowledge of graph optimization

Supervisor:

Scene graph generation models analysis using Visual Genome benchmark and self-recorded datasets

Description

Scene graphs were first proposed [1] as a data structure that describes the object instances in a scene and the relationships between these objects. The nodes in the scene graph represent the detected target objects, whereas the edges denote the detected pairwise relationships. A complete scene graph can represent the detailed semantics of a dataset of scenes.

Scene graph generation (SGG) is a visual detection task for building structured scene graphs. In this work, we would like to compare the three traditional SGG models: Neural Motifs [2], Graph R-CNN [3], and IMP [4]. We will train and evaluate the models using the Visual Genome dataset and other commonly used datasets. In addition, our dataset will be annotated and then utilized.

[1]: J. Johnson, et al. "Image retrieval using scene graphs."

[2]: R. Zellers, et al. "Neural motifs: Scene graph parsing with global context."

[3]: J. Yang, et al. "Graph r-cnn for scene graph generation."

[4]: D. Xu, et al. "Scene graph generation by iterative message passing."

Prerequisites

  • Good Programming Skills (Python, GUI design)
  • Knowledge about Ubuntu/Linux/Pytorch
  • Knowledge about Computer vision/Neural network
  • Motivation to learn and conduct research

Contact

dong.yang@tum.de

Supervisor:

Dong Yang, Xiao Xu

Generalized Robot Learning from Demonstration

Keywords:
LfD, imitation learning, Franka Arm

Description

In this work, we would like to design a generative model for robot learning from demonstration. We will consider a table cleaning task, determine varying task conditions, and apply a generative model such that the robot can generalize to novel unseen conditions. We aim to use both simulation and Franka arm for testing our algorithms. 

Prerequisites

Requirements:

Experience in C/C++

ROS is a plus

High motivation to learn and conduct research

 

Supervisor:

Network Aware Imitation Learning

Keywords:
Teleoperation, Learning from Demonstration

Description

When teaching robots remotely via teleoperation, the communication (between the human demonstrator and the remote robot learner) imposes challenges. Network delay, packet loss, and data compression might cause completely or partially degraded demonstration qualities. In this thesis, we would like the make the best out of varying quality demonstrations provided via teleoperation with haptic feedback. We will use different teleoperation setups to test the developed robot learning approaches:

Prerequisites

Requirements:

Experience in C/C++

ROS is a plus

High motivation to learn and conduct research

 

Supervisor:

Basak Gülecyüz, Edwin Babaians

Robotic task learning from human demonstration

Description

Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on learning from human demonstration for improving the robot autonomy.

Prerequisites

Required background:

- Digital signal processing

- Computer vision

- Neural networks and other ML algorithms

 

Required abilities:

- Experience with Python or C++

- Experience with Tensorflow or PyTorch

- Motivation to yield a good thesis

 

Contact

furkan.kaynar@tum.de

 

(Please provide your CV and transcript in your application)

 

Supervisor:

Hasan Furkan Kaynar

Embedded TPU-based Inertial-Camera Edge AI

Keywords:
embedded linux, ARM Cortex, PCB Design, IMU

Description

In this project, we aim to capture low-latency RGB data from a CSI camera in sync with an inertial measurement unit(IMU) and use the TPU unit for AI processing, especially for real-time segmentation applications. The first step of the project is to modify the provided embedded Linux kernel and the embedded board with an ARM Cortex processor to communicate with the TPU module. In the next step of the project, the synchronization of the inertial measurement data must be addressed with the frames of the RGB data from the camera. We use Edge TPU ML accelerator ASIC designed by Google that provides high-performance ML inferencing for TensorFlow Lite models.

Prerequisites

Good programming skills in C++.

Strong understanding of embedded Linux programming.

Knowledge about Debian Kernel and UBoot.

Knowledge about ARM Cortex Processors.

Contact

Please send your application to:

mojtaba.karimi@tum.de

edwin.babaians@tum.de

Supervisor:

Mojtaba Karimi, Edwin Babaians

 

You can find important information about writing your thesis, giving a talk at LMT as well as templates for Powerpoint and LaTeX here.