Student projects and final year projects at the Chair of Media Technology

Under this topic, the student will extend our existing VR-based synthetic data generation tool for Hand-Object interactions. Furthermore, the student will generate synthetic data using this tool and evaluate state-of-the-art Computer Vision and Deep Learning models for tracking Hand-Object Interactions in 3D.

• Strong familiarity with Python programming
• Interest and first experiences in Computer Graphics, VR, Computer Vision, and Deep Learning.
• Ideally also interest and experience in Blender 3D software

This is a working student position for a variety of tasks in Python programming:

• experimenting with Blender-related python libraries,
• writing interfaces between 3D rendering software (Blender) and middleware (ROS, www.ros.org)

• Strong interest in 3D Computer Graphics and Gaming.
• Comfortable programming in Python
• Ideally: also familiarity with development environments on Linux and windows.

Please send a description of your interest and experience regarding the above points together with your application.

NVIDIA Omniverse is a platform that enables researchers to create custom 3D pipelines and simulate large virtual environments in a fast and convenient manner. It can render the environments very accurately and immersively with the help of GPU acceleration. In this work, we aim to create a digital twin of a teleoperation framework in Omniverse, in which we can use the haptic input device to control the remote robot arm.

• Good Programming Skills (Python, C++)
• Knowledge about Ubuntu/Linux/ROS
• Motivation to learn and conduct research

A Digital Twin is a virtual representation of an asset, to which is connected in a bi-directional way: changes happening in the real asset are shown in the digital asset and vice-versa.

In this project, you will improve force rendering algorithms to make teleoperation more user friendly through means of the Digital Twin of a factory. 

Required:

• Python knowledge
• Chai3D (ideally you have participated in the Computational Haptics Laboratory)

Recommended (not all of them):

• Experience in ROS
• C++ knowledge
• Robotics knowledge
• MuJoCo

Robotic teleoperation is often used to accomplish complex tasks remotely with human-in-the-loop. In cases, where the task requires very precise manipulation, virtual fixtures can be used to restrict and guide the motion of the end effector of the robot while the person teleoperates. In this thesis, we will analyze the geometry of virtual fixtures depending on the scene and task. We will use reinforcement learning to estimate ideal virtual fixture model parameters. At the end of the thesis, the performance can be evaluated with user experiments.

Useful background:

- Machine learning (Reinforcement Learning)

- Robotic simulation

Requirements:

- Experience with Python & Deep learning frameworks (PyTorch / Tensorflow...)

- Experience with a RL framework

- Motivation to yield a good outcome

In this work you are going to refine and combine existing state-of-the-art simulator tools in Unreal Engine. You are going to explore recent state of the art methods for creating synthetic data based on realistic simulation environments and integrate existing hand animations into the simulation in order to enable hand-object interactions. Furthermore you are going to deal with environmental 3D pointclouds. 

Reference:

https://arxiv.org/pdf/1810.06936.pdf

https://github.com/3dperceptionlab/unrealrox

General Programming Knowledge

Interest in Game Engines and Simulation

Knowledge of Unreal Engine is a plus

This work falls within the scope of indoor localization, more precisely the fingerprinting-based indoor localization. 

Your task will be to investigate the potential and outcomes of opting for a multi-layer reference map during the "Offline phase", which corresponds to a potential improvement idea that has never been opted for or tested in the current state-of-the- Art fingerprinting schemes.

The aim here is to achieve a better trade off between both performance and costs yielding a better localization method. 

Required: 

 - Python and/or Matlab

 - Basic knowledge in signal processing and wireless communication

 - Analytical thinking and creativity 

Electrical Impedance Tomography (EIT) is an imaging technique that estimates the impedance of human body tissues by passing an alternating current through pairs of electrodes and measuring the voltage and current among other pairs of electrodes.

The inverse problem aims to reconstruct a cross-section tomographic image of the body part given the measurements.

EIT Wearable devices were applied successfully to the area of hand gesture classification and resulted in high-accuracy machine learning models [1][2].

The goal of the project is to research and test possible calibration approaches for sufficient and reproducible measurement results of EIT (similar to [3]), as well as go into hand gesture classification based on the measured impedance values of a human forearm [1][2].

We provide the wearable EIT band that is being developed in collaboration with Enari GmbH and the necessary computer vision building blocks for dataset collection.

The attached figure shows a pipeline of the image reconstruction

[1] Zhang, Yang & Harrison, Chris. (2015). Tomo: Wearable, Low-Cost Electrical Impedance Tomography for Hand Gesture Recognition. 167-173. 10.1145/2807442.2807480.

[2] D. Jiang, Y. Wu and A. Demosthenous, "Hand Gesture Recognition Using Three-Dimensional Electrical Impedance Tomography," in IEEE Transactions on Circuits and Systems II: Express Briefs, vol. 67, no. 9, pp. 1554-1558, Sept. 2020, doi: 10.1109/TCSII.2020.3006430.

[3] Zhang, Y., Xiao, R., & Harrison, C. (2016). Advancing Hand Gesture Recognition with High Resolution Electrical Impedance Tomography. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (pp. 843–850). Association for Computing Machinery.

• Knowledge with python
• Knowledge in Digital Signal Processing or Deep Learning

Description:

This thesis project will examine the integration of artificial intelligence (AI) and blockchain technology for digital watermarking of audio. Digital watermarking is a technique used to embed hidden information, such as ownership or copyright information, into digital audio files. The goal of this project is to develop new AI-based techniques for digital watermarking that can be secured and protected using blockchain technology.

Prerequisites:

• Strong background in signal processing and digital audio
• Familiarity with machine learning and AI techniques
• Basic understanding of blockchain technology and its applications
• Experience with programming languages such as Python and JavaScript
• Strong analytical and problem-solving skills
• Strong written and verbal communication skills

This project is an exciting opportunity to work at the intersection of AI and blockchain, where you will have the chance to apply your skills and knowledge to the development of new technologies that could have a significant impact on the audio industry. You will be working with an innovative startup in the heart of Silicon Valley, where you will have the opportunity to contribute to the development of cutting-edge technology. If you are passionate about AI, blockchain, and signal processing and are looking for a challenging and rewarding research experience, this thesis project is for you!

Please send your CV and Transcript of Records. Tell me why you are interested in this topic:

Description:

This thesis project will examine the integration of artificial intelligence (AI) and blockchain technology for digital watermarking of images/videos. Digital watermarking is a technique used to embed hidden information, such as ownership or copyright information, into image files. The goal of this project is to develop new AI-based techniques for digital watermarking that can be secured and protected using blockchain technology.

Prerequisites:

• Strong background in signal processing and digital images
• Familiarity with machine learning and AI techniques
• Basic understanding of blockchain technology and its applications
• Experience with programming languages such as Python and JavaScript
• Strong analytical and problem-solving skills
• Strong written and verbal communication skills

This project is an exciting opportunity to work at the intersection of AI and blockchain, where you will have the chance to apply your skills and knowledge to the development of new technologies that could have a significant impact on the media industry. You will be working with an innovative startup in the heart of Silicon Valley, where you will have the opportunity to contribute to the development of cutting-edge technology. If you are passionate about AI, blockchain, and signal processing and are looking for a challenging and rewarding research experience, this thesis project is for you!

Please send your CV and Transcript of Records. Tell me why you are interested in this topic:

Description:

We are excited to offer a unique and innovative thesis project that combines cutting-edge technology with digital multimedia. As an "AI and Blockchain Generative Media Researcher," you will have the opportunity to explore the potential of using AI algorithms to generate one-of-a-kind pieces of media content and using blockchain technology to protect the rights of the original creators via smart licensing mechanisms.

This project is an external thesis with a startup in San Francisco, which will give you the chance to work with real-world industry experts and gain valuable experience in a startup environment. This is not just about writing a thesis, it's about making a real-world impact on the media technology industry. You will have the chance to conduct research, explore new possibilities and create something truly unique.

Prerequisites:

• Strong background in signal processing and digital images
• Familiarity with machine learning and AI techniques
• Basic understanding of blockchain technology and its applications
• Experience with programming languages such as Python and JavaScript
• Strong analytical and problem-solving skills
• Strong written and verbal communication skills

Don't miss this opportunity to be part of a revolutionary project that combines your passion for computer science and art. Apply now and take the first step in unlocking the potential of AI and blockchain technology in generative art, with the added bonus of gaining valuable experience working with a startup in San Francisco.

Please send your CV and Transcript of Records. Tell me why you are interested in this topic:

We are excited to offer a cutting-edge thesis project that explores the potential of using blockchain technology to decentralize Digital Rights Management (DRM) in the music and video streaming industry. As a "Web3 DRM Researcher," you will have the opportunity to investigate the use of encryption and encoding techniques, powered by blockchain technology, to secure and protect digital content in a decentralized way.

This project will involve research on the current state of DRM technology used by companies like Spotify and Netflix, and the challenges they face in ensuring the security and protection of digital content. You will then explore the potential of blockchain technology to address these challenges, and investigate the implementation of encryption and encoding techniques to secure and protect digital content in a decentralized manner.

Prerequisites:

• Strong background in signal processing and audio/video encryption/encoding
• Familiarity with machine learning and AI techniques
• Basic understanding of blockchain technology and its applications
• Experience with programming languages such as Python and JavaScript
• Strong analytical and problem-solving skills
• Strong written and verbal communication skills

This is an exciting opportunity for a student to work on a cutting-edge project that has the potential to make a real-world impact on the music and video streaming industry. Apply now and take the first step in decentralizing web3 DRM using blockchain technology.

Please send your CV and Transcript of Records. Tell me why you are interested in this topic:

to develop a haptic offline coding scheme based on previous studies.

More details coming soon

matlab or python

signal processing background

Deep learning-based depth estimation has been studied extensively for perceiving and understanding the surrounding environment. Due to physical limitations and the sensitivity of the measurement results on the scene characteristics and environmental conditions of individual sensors, the performance of depth estimation is insufficient in many applications where only a single type of sensor data is applied. To tackle this issue, the fusion of multiple sensor modalities has been studied as a promising solution, especially in the fog environment.

In this work, the student needs to investigate the selective sensor fusion strategies (camera, LiDAR, and radar) under different fog concentrations using deep learning-based methods.

• High motivation to learn and conduct research
• Good programming skills in Python, Pytorch, Linux
• Basic experience with deep learning, neural network

In this work you are going to investigate the action msegmentation problem in an unsupervised setting. You are especially getting familiar with object detection networks (e.g.: YOLO) and you are getting familiar with segmenting videos without the availability of labels. This work especially focuses on learning current computer vision and deep learning techniques, while programming in python. Overall, this work will help you to broaden your deep- learning  knowledge, which is particularly helpful for later work  in the area of deep learning and computer vision.

References:

https://arxiv.org/pdf/1506.02640.pdf

https://arxiv.org/pdf/2103.11264.pdf

Deep Learning Knowledge

Python

A Digital Twin is a virtual representation of an asset, to which is connected in a bi-directional way: changes happening in the real asset are shown in the digital asset and vice-versa.

In this project, we will create a prototype for the Digital Twin of a factory in Nvidia Omniverse. We will create a visual representation and update it using sensor data and artificial intelligence.

Required:

• Python knowledge

Recommended (not all of them):

• Experience in ROS
• C++ knowledge
• Robotics knowledge

Imitation Learning helps robots to learn any skill robustly and much faster than simply using reinforcement learning. In this project, we will use human demonstrations to teach a robot manipulator how to solve different industrial tasks.

To achieve out goal, we will use different sensors (cameras, Force/Torque, etc.) and we will employ state of the art Machine learning/Deep Learning techniques, together with image processing.

Required:

- Experience in Python

- Some Machine Learning / Deep Learning / Image Processing experience

Also beneficial (but not a must):

- ROS experience

- Reinforcement Learning experience

Although there is intensive research in the field of robotics since decades, autonomous robotic grasping and manipulation still remain as challenging abilities under real-life conditions. Autonomous algorithms fail more in unstructured environments such as household environments, which limits the practical use of robots in daily human life. In unsructured environments, the perception gains importance and there can often be novel and unseen cases by which the autonomous algorithms tend to fail. By these cases there is need for human correction or demonstration to increase the task performance or teach new abilities to robots. For this aim, we will create a user interface which is intuitive to use by the user on a mobile device via hand poses. At the same time the interface should provide the necessary data to efficiently assist a robot in a daily home environment. The main application will be teleassistance for robotic grasping.

• Basic knowledge of image processing / computer vision. 
• Basic coding experience, especially with C#.
• Experience with Unity game engine.
• Basic experience with ROS.
• Motivation to yield a successful work.

The motion planner of a robotic arm requires planning of the necessary motion, under collision avoidance and regarding the joint limitations of the robot. In this project, we will focus on motion planning of the Panda robot arm, using several planners. We will test OpenRave motion planner and compare it to the moveit motion planner.

We will also implement and test cartesian path planning using methods like the Descartes path planner.

At the end of this project, the student will learn about implementation and usage of different motion/path planners.

Useful background:

- Robotic control

- Experience with ROS

Necessary background:

- Experience with C++

Virtual conference systems provide an alternative to physical meetings that have significantly grown in importance over the last years. However, larger events require the audience to be muted to avoid an accumulation of background noise and distorted audio. While this is sufficient for unidirectional meetings, many types of meetings strongly rely on the feedback of their audience, such as in performing arts.
In this project, we want to extend Zoom sessions with a simple Chatbot that collects the audience participation of each user using a straightforward button interface. Then, the system renders the overall audience feedback based on the feedback state collected from each user. The project combines signal and audio processing with the chance to gain practical experience with app development and SDKs.

References

• Create a Chatbot App
• Build a Chatbot

• Good knowledge in Nodejs/JavaScript
• Experience with Git
• Experience with Zoom SDK would be a plus