Student projects and final year projects at the Chair of Media Technology

The main task is to creade a GUI that provides online positioning and tracking. 

Python or matlab

to develop a haptic offline coding scheme based on previous studies.

More details coming soon

matlab or python

signal processing background

Deep learning-based depth estimation has been studied extensively for perceiving and understanding the surrounding environment. Due to physical limitations and the sensitivity of the measurement results on the scene characteristics and environmental conditions of individual sensors, the performance of depth estimation is insufficient in many applications where only a single type of sensor data is applied. To tackle this issue, the fusion of multiple sensor modalities has been studied as a promising solution, especially in the fog environment.

In this work, the student needs to investigate the selective sensor fusion strategies (camera, LiDAR, and radar) under different fog concentrations using deep learning-based methods.

• High motivation to learn and conduct research
• Good programming skills in Python, Pytorch, Linux
• Basic experience with deep learning, neural network

In this work you are going to investigate the action msegmentation problem in an unsupervised setting. You are especially getting familiar with object detection networks (e.g.: YOLO) and you are getting familiar with segmenting videos without the availability of labels. This work especially focuses on learning current computer vision and deep learning techniques, while programming in python. Overall, this work will help you to broaden your deep- learning  knowledge, which is particularly helpful for later work  in the area of deep learning and computer vision.

References:

https://arxiv.org/pdf/1506.02640.pdf

https://arxiv.org/pdf/2103.11264.pdf

Deep Learning Knowledge

Python

The problem of registering point clouds can be reduced to estimating a Euclidean transformation between two sets of 3D points [1]. Once the transformation is estimated, it can be used to register two point clouds in a common coordinate system.

Applications of point cloud registration include 3D reconstruction, localization, or change detection. However, these applications rely on a high similarity between point clouds and do not account for disturbances in the form of noise, occlusions, or outliers. Such defects degrade the quality of the point cloud and thus the accuracy of the registration-dependent application. One approach to deal with these effects is to quantify the registration uncertainty. The general idea is to use uncertainty as a guide for point cloud registration quality. If the uncertainty is too high, a new registration iteration or re-scanning is needed.

In this project, we investigate uncertainty quantification for current learning-based approaches to point cloud registration [1, 2, 3]. First, several methods for uncertainty quantification are selected [4]. Of particular interest are approaches based on Bayesian inference. The approaches are then modified to fit current point cloud registration frameworks and evaluated against benchmark datasets such as ModelNet or ShapeNet. In the evaluation, different types of scan perturbations need to be tested.

References

[1] Huang, Xiaoshui, et al. A Comprehensive Survey on Point Cloud Registration. arXiv:2103.02690, arXiv, 5 Mar. 2021. arXiv.org, http://arxiv.org/abs/2103.02690.

[2] Yuan, Wentao, et al. DeepGMR: Learning Latent Gaussian Mixture Models for Registration. arXiv:2008.09088, arXiv, 20 Aug. 2020. arXiv.org, http://arxiv.org/abs/2008.09088.

[3] Huang, Shengyu, et al. “PREDATOR: Registration of 3D Point Clouds with Low Overlap.” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2021, pp. 4265–74. DOI.org (Crossref), https://doi.org/10.1109/CVPR46437.2021.00425.

[4] Abdar, Moloud, et al. “A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges.” Information Fusion, vol. 76, Dec. 2021, pp. 243–97. ScienceDirect, https://doi.org/10.1016/j.inffus.2021.05.008.

• Python and Git
• Experience with a deep learning framework (Pytorch, Tensorflow)
• Interest in Computer Vision and Machine Learning

A Digital Twin is a virtual representation of an asset, to which is connected in a bi-directional way: changes happening in the real asset are shown in the digital asset and vice-versa.

In this project, we will create a prototype for the Digital Twin of a factory in Nvidia Omniverse. We will create a visual representation and update it using sensor data and artificial intelligence.

Required:

• Python knowledge

Recommended (not all of them):

• Experience in ROS
• C++ knowledge
• Robotics knowledge

Imitation Learning helps robots to learn any skill robustly and much faster than simply using reinforcement learning. In this project, we will use human demonstrations to teach a robot manipulator how to solve different industrial tasks.

To achieve out goal, we will use different sensors (cameras, Force/Torque, etc.) and we will employ state of the art Machine learning/Deep Learning techniques, together with image processing.

Required:

- Experience in Python

- Some Machine Learning / Deep Learning / Image Processing experience

Also beneficial (but not a must):

- ROS experience

- Reinforcement Learning experience

Although there is intensive research in the field of robotics since decades, autonomous robotic grasping and manipulation still remain as challenging abilities under real-life conditions. Autonomous algorithms fail more in unstructured environments such as household environments, which limits the practical use of robots in daily human life. In unsructured environments, the perception gains importance and there can often be novel and unseen cases by which the autonomous algorithms tend to fail. By these cases there is need for human correction or demonstration to increase the task performance or teach new abilities to robots. For this aim, we will create a user interface which is intuitive to use by the user on a mobile device via hand poses. At the same time the interface should provide the necessary data to efficiently assist a robot in a daily home environment. The main application will be teleassistance for robotic grasping.

• Basic knowledge of image processing / computer vision. 
• Basic coding experience, especially with C#.
• Experience with Unity game engine.
• Basic experience with ROS.
• Motivation to yield a successful work.

This work is focused on video understanding, with a special focus on human action segmentation. It is going to involve self-attention models (like Transformers), CNN's and other deep learning architectures.

You are going to evaluate State of the Art approaches on public action segmentation datasets with respect to an action segment prediction problem setting. Furthermore, you can incorporate your own ideas into developing the final model pipeline. Additional ideas from your side are welcome.

Related Work:

• https://arxiv.org/pdf/2110.08568v1.pdf
• https://arxiv.org/pdf/2106.02036.pdf

• Python and Pytorch
• Basic knowledge in deep learning

In this work, we want to investigate how scene graphs can help to improve scene completion/point cloud completion. The scene graph will be generated by object, attributes, and relationships detection with the 2D RGB images as input. The first stage of this work is to exploit the state-of-the-art scene reconstruction framework to construct the scene point clouds. In the second stage, we need to utilize the generated scene graphs to fine-tune and improve the constructed scene point clouds.

• High motivation to learn and conduct research
• Good programming skills in Python, Pytorch
• Basic experience with computer vision

The motion planner of a robotic arm requires planning of the necessary motion, under collision avoidance and regarding the joint limitations of the robot. In this project, we will focus on motion planning of the Panda robot arm, using several planners. We will test OpenRave motion planner and compare it to the moveit motion planner.

We will also implement and test cartesian path planning using methods like the Descartes path planner.

At the end of this project, the student will learn about implementation and usage of different motion/path planners.

Useful background:

- Robotic control

- Experience with ROS

Necessary background:

- Experience with C++

Identify and track multiple light sources in a  video stream.

The student must record the video multiple objects to the tracked in the video stream.

Python

Signal/image processing

The student must set up a transmission scheme based on MIMO technology and OFDMA.

Take samples in different points inside an indoor environment for further processing of the signal characteristics to obtain an estimated position of the mobile node.

Python

WIreless communications with focus in channel state information and OFDMA MIMO systems

Knowledge in USRP not required but is a plus

Virtual conference systems provide an alternative to physical meetings that have significantly grown in importance over the last years. However, larger events require the audience to be muted to avoid an accumulation of background noise and distorted audio. While this is sufficient for unidirectional meetings, many types of meetings strongly rely on the feedback of their audience, such as in performing arts.
In this project, we want to extend Zoom sessions with a simple Chatbot that collects the audience participation of each user using a straightforward button interface. Then, the system renders the overall audience feedback based on the feedback state collected from each user. The project combines signal and audio processing with the chance to gain practical experience with app development and SDKs.

References

• Create a Chatbot App
• Build a Chatbot

• Good knowledge in Nodejs/JavaScript
• Experience with Git
• Experience with Zoom SDK would be a plus

Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on user interaction methods for describing a robotic task via modifying the viewed scene.

Useful background:

- 3D Human-computer interfaces

- Game Design

- Digital signal processing

Required abilities:

- Experience with Unity and ROS

- Motivation to yield a good work

Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on learning from human demonstration for improving the robot autonomy.

Required background:

- Digital signal processing

- Computer vision

- Neural networks and other ML algorithms

Required abilities:

- Experience with Python or C++

- Experience with Tensorflow or PyTorch

- Motivation to yield a good thesis

Teleoperation Systems

Bilateral teleoperation with haptic feedback provides its users with a new dimension of immersion in virtual or remote environments. This technology enables a great variety of applications in robotics and virtual reality, such as remote surgery and industrial digital twin [1]. Figure 1 shows a generalized human-in-the-loop teleoperation system with kinesthetic feedback, where the operator commands a remote/virtual robot to explore the environment and experiences the interactive force feedback through the haptic interface. 

Teleoperation systems face many challenges caused by unpredictable environment changes, time-delayed feedback, limited network capacity, etc. [2]. These issues inevitably distort the force feedback signal, degrading the transparency and stability of the system. In the past decades, many control algorithms and hardware architectures were developed to tackle these problems in the past decades [3].

Time Domain Passivity Method (TDPA)

TDPA is a passivity-based control scheme that ensures the stability of teleoperation systems in the presence of communication delays [4] (See Figure 2.). It abstracts two-port networks from the haptic system and observes the energy flow between the networks. Passivity condition is maintained by dissipating extra energy generated by non-passive networks. Original TDAP suffers from position drift and feedback force jump [5], and one reason for the position drift is that the energy, which is generated by the delayed communication, is dissipated in the task space of the robots.

Jacobian Null-Space for Redundancy Robot

Many robot mechanisms have redundant degrees of freedom (rDOFs), which means that they have a larger number of joints than the number of dimensions of their task or configuration space. The null space of the Jacobian null space stands for the redundant dimensions which can be exploited to dissipate extra energy by damping the null space motion without affecting the task space [5].

Your Task and Target

In this work, we target at improving the performance of TDPA by considering dissipating energy generated by time delay and other factors in the Jacobian null-space of the kinesthetically redundant robots. With the help of the Jacobian null-space method, we can avoid dissipating energy in the task space, so as to alleviate position drift and force distortion while keeping the system passive. For more information, previous work can be referred to as [7-9].

In this master's internship, your work will include

1.      1. surveying the related algorithms

2.      2. constructing the simulation environment

3.      3. experimenting with the state-of-the-art Jacobian null-space TDPA method.

4.      4. analyzing system passivity in Cartesian task space, joint space, null space, etc.

Requirements

All requirements are recommended but not mandatory. However, you will need extra effort to catch up if you are unfamiliar with the following topics:

1.    3. Basic knowledge about robotics and control theory is favorable.

2.    2. Experience with robotics simulation software and platforms is favorable.

3.    1. C++, Matlab, and Python would be the primary working language. Basic knowledge about one or more of them is highly recommended.

Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on learning from human demonstration for improving the robot autonomy.

Required background:

- Digital signal processing

- Computer vision

- Neural networks and other ML algorithms

Required abilities:

- Experience with Python or C++

- Experience with Tensorflow or PyTorch

- Motivation to yield a good thesis