Open Thesis

A perceptual-based rate scalable haptic coding scheme

Description

to develop a haptic offline coding scheme based on previous studies.

 

More details coming soon

Prerequisites

matlab or python

signal processing background

Supervisor:

Selective Sensor Fusion Strategies for Depth Estimation in Fog Environment

Keywords:
Sensor Fusion, Depth Estimation, Fog Environment

Description

Deep learning-based depth estimation has been studied extensively for perceiving and understanding the surrounding environment. Due to physical limitations and the sensitivity of the measurement results on the scene characteristics and environmental conditions of individual sensors, the performance of depth estimation is insufficient in many applications where only a single type of sensor data is applied. To tackle this issue, the fusion of multiple sensor modalities has been studied as a promising solution, especially in the fog environment.

In this work, the student needs to investigate the selective sensor fusion strategies (camera, LiDAR, and radar) under different fog concentrations using deep learning-based methods.

Prerequisites

  • High motivation to learn and conduct research
  • Good programming skills in Python, Pytorch, Linux
  • Basic experience with deep learning, neural network

Contact

mengchen.xiong@tum.de

(Please attach your CV and transcript to your application)

Supervisor:

A Scene Graph based Refinement for 3D Scene Point Clouds Completion

Keywords:
Scene Completion, Point Clouds, Scene Graph, Object/Relationship detection, Deep Learning

Description

In this work, we want to investigate how scene graphs can help to improve scene completion/point cloud completion. The scene graph will be generated by object, attributes, and relationships detection with the 2D RGB images as input. The first stage of this work is to exploit the state-of-the-art scene reconstruction framework to construct the scene point clouds. In the second stage, we need to utilize the generated scene graphs to fine-tune and improve the constructed scene point clouds.

Prerequisites

  • High motivation to learn and conduct research
  • Good programming skills in Python, Pytorch
  • Basic experience with computer vision

Contact

dong.yang@tum.de

(Please attach your CV and transcript)

Supervisor:

Dong Yang, Xiao Xu

Jacobian Null-space Energy Dissipation TDPA for Redundancy Robots in Teleoperation

Keywords:
Teleoperation, Robotics, Control Theory

Description

Teleoperation Systems

 

Bilateral teleoperation with haptic feedback provides its users with a new dimension of immersion in virtual or remote environments. This technology enables a great variety of applications in robotics and virtual reality, such as remote surgery and industrial digital twin [1]. Figure 1 shows a generalized human-in-the-loop teleoperation system with kinesthetic feedback, where the operator commands a remote/virtual robot to explore the environment and experiences the interactive force feedback through the haptic interface. 

Teleoperation systems face many challenges caused by unpredictable environment changes, time-delayed feedback, limited network capacity, etc. [2]. These issues inevitably distort the force feedback signal, degrading the transparency and stability of the system. In the past decades, many control algorithms and hardware architectures were developed to tackle these problems in the past decades [3].

Time Domain Passivity Method (TDPA)

 

TDPA is a passivity-based control scheme that ensures the stability of teleoperation systems in the presence of communication delays [4] (See Figure 2.). It abstracts two-port networks from the haptic system and observes the energy flow between the networks. Passivity condition is maintained by dissipating extra energy generated by non-passive networks. Original TDAP suffers from position drift and feedback force jump [5], and one reason for the position drift is that the energy, which is generated by the delayed communication, is dissipated in the task space of the robots.

 

Jacobian Null-Space for Redundancy Robot

 

Many robot mechanisms have redundant degrees of freedom (rDOFs), which means that they have a larger number of joints than the number of dimensions of their task or configuration space. The null space of the Jacobian null space stands for the redundant dimensions which can be exploited to dissipate extra energy by damping the null space motion without affecting the task space [5].

Your Task and Target

In this work, we target at improving the performance of TDPA by considering dissipating energy generated by time delay and other factors in the Jacobian null-space of the kinesthetically redundant robots. With the help of the Jacobian null-space method, we can avoid dissipating energy in the task space, so as to alleviate position drift and force distortion while keeping the system passive. For more information, previous work can be referred to as [7-9].

In this master's internship, your work will include

1.      1. surveying the related algorithms

2.      2. constructing the simulation environment

3.      3. experimenting with the state-of-the-art Jacobian null-space TDPA method.

4.      4. analyzing system passivity in Cartesian task space, joint space, null space, etc.

Prerequisites

Requirements

All requirements are recommended but not mandatory. However, you will need extra effort to catch up if you are unfamiliar with the following topics:

1.    3. Basic knowledge about robotics and control theory is favorable.

2.    2. Experience with robotics simulation software and platforms is favorable.

3.    1. C++, Matlab, and Python would be the primary working language. Basic knowledge about one or more of them is highly recommended.

Contact

zican.wang@tum.de

xiao.xu@tum.de

Supervisor:

Zican Wang, Xiao Xu

Ongoing Thesis

Bachelor's Theses

haptic data redution for position-position teleoperation control architecture

Keywords:
teleoperation control, haptics

Description

Using a teleoperation system with haptic feedback, the users can thus truly immerse themselves into a distant environment, i.e., modify it, and execute tasks without physically being present but with the feeling of being there. A typical teleoperation system with haptic feedback (referred to as a teleoperation system) comprises three main parts: the human operator OP)/master system, the teleoperator (TOP)/slave system, and the communication link/network in between. During teleoperation, the slave and master devices exchange multimodal sensor information over the communication link. This work aims to develop a haptic data reduction scheme based on a position-position teleoperation architecture and compare the performance with the classical position-force control architecture.

 

Your work:

(1) build up a teleoperation system that can switch between position-position and position-force architectures.

(2) integrate the existing haptic data reduction scheme with the PP architecture.

(3)  introduce delays, and implement existing passivity based control scheme to ensure system stability

(4) compare the performance difference between the PF and PP architectures.

Prerequisites

C++, matlab simulink

Supervisor:

Research Internships (Forschungspraxis)

Handshake for Plug-and-Play Haptic Interaction system

Keywords:
Teleoperation, GUI, Qt, JavaScript
Short Description:
Our project aims to implement handshake communication protocol for Plug-and-Play Haptic Interaction system according to the IEEE standard.

Description

Our project aims to implement handshake communication protocol for Plug-and-Play Haptic Interaction system according to the IEEE standard. For the system, the main achievements are:

 

1.     Plug and play on the Leader side: When the Leader device disconnects from the system, the Follower device will turn to the waiting state and will remain in its initial position same as when it’s activated in the system until the next re-insertion of the Leader device.

2.     Automatic adjustment of device parameters according to the specific type of Leader device to guarantee the performance of human perception: First of all, when connecting, the Leader device will transmit its media and interface information to the Follower side, so-called Metadata, and at the same time it will inform the Follower device of the specific model type it is using. The Follower device will adjust its parameters according to the received information to adapt to the Leader if the type of Leader device has different precision from the Follower device and transmits its metadata to the Leader.

   For the adjustments to the message transmission process:

1).   Achieve the PnP adjustment on the follower side.

 

2).   The message sending order, the format of the interface, the mode of pushing data packets into stacks, and the decoding function should obey the regulations of the IEEE standard.

 

Prerequisites

C/C++

socket programming

visual studio IDE

Contact

Email Adress: siwen.liu@tum.de, xiao.xu@tum.de

Supervisor:

Siwen Liu, Xiao Xu

GUI for Plug-and-Play Haptic Interaction system

Keywords:
Teleoperation, GUI, Qt, JavaScript
Short Description:
Our project aims to build a GUI for Plug-and-Play Haptic Interaction system according to the IEEE standard.

Description

Our project aims to build a GUI for Plug-and-Play Haptic Interaction system according to the IEEE standard. For the system, the main achievements are:

 

1.     Plug and play on the Leader side: When the Leader device disconnects from the system, the Follower device will turn to the waiting state and will remain in its initial position same as when it’s activated in the system until the next re-insertion of the Leader device.

2.     Automatic adjustment of device parameters according to the specific type of Leader device to guarantee the performance of human perception: First of all, when connecting, the Leader device will transmit its media and interface information to the Follower side, so-called Metadata, and at the same time it will inform the Follower device of the specific model type it is using. The Follower device will adjust its parameters according to the received information to adapt to the Leader if the type of Leader device has different precision from the Follower device and transmits its metadata to the Leader.

 

 

Prerequisites

The requirements of our project are as follows. For the GUI part:

1.   The GUI should be implemented under either Qt or JavaScript (first considering Qt) on both the Leader and Follower sides.

2.     For the Leader side, the GUI should be proposed including these functions:

1). Chooses the device on the Leader side.

2). Shows whether the handshaking is successful or not.

3). Shows the device type used on the Follower side after the handshake.

4). When the Leader device is disconnected from the system, show as well.

3.    For the Follower side, the GUI should be proposed including these functions:

1). Chooses the device on the Follower side.

2). Shows whether the handshaking is successful or not.

3). Shows the device type used on the Leader side, adjusts the parameters on the Follower side, and then shows the adjusted device type if the handshake is successful.

4). When the Leader device is disconnected from the system, show as well. And then shows the initial position of the Follower device in the waiting state.

4.    For the adjustments to the message transmission process:

1).   Achieve the PnP adjustment on the follower side.

 

2).   The message sending order, the format of the interface, the mode of pushing data packets into stacks, and the decoding function should obey the regulations of the IEEE standard.

 

Contact

Email Adress: siwen.liu@tum.de, xiao.xu@tum.de

Supervisor:

Siwen Liu, Xiao Xu

Perceptual-oriented objective quality assessment for time-delayed teleoperation

Description

Recent advances in haptic communication cast light onto the promise of full immersion into remote real or virtual environments.

The quality of compressed haptic signals is crucial to fulfill this promise. Traditionally, the quality of haptic signals is evaluated through a series of subjective experiments. So far, only very limited attention was directed toward developing objective quality measures for haptic communication. In this work, we focus on the compression distortion and the delay compensation distortion that contaminate the force/velocity haptic signals generated by physical interaction with objects in the remote real or virtual environment.

 

Prerequisites

 

Contact

 

Supervisor:

Student Assistant Jobs

implementation of teleoperation systems using Raspberry PI

Description

We have already a framework of teleoperation system running in Windows, where two haptic devices are connected through UDP protocol, one as the leader device and the other is the follower.

Your tasks are:

1. move the framework to Linux system.

2. setup a ROS-based virtual teleoperation environment.

3. shift the framework to a raspberry PI.

Prerequisites

Linux, socket programming (e.g. UDP protocol), C and C++, ROS

Supervisor:

Student Assistant for distributed haptic training system

Keywords:
server-client, UDP, GUI programming

Description

Your tasks:

1.      build a serve-client telehaptic training system based on current code.

2.     develop GUI for client side.

required skills:

- c++

- knowledge about socket programming, e.g. UDP

- GUI programming, e.g. QT

- working environment: windows+visual studio



This work has a tight connection with the project Teleoperation over 5G networks and the IEEE standardization P1918.1.1.

https://www.ei.tum.de/lmt/forschung/ausgewaehlte-projekte/dfg-teleoperation-over-5g/

https://www.ei.tum.de/lmt/forschung/ausgewaehlte-projekte/ieee-p191811-haptic-codecs

Supervisor:

Student Assistant for reference software of time-delayed teleoperation with control schemes and haptic codec

Short Description:
In this work, you need to extend the current teleoperation reference software for enabling different control schemes and haptic data reduction approaches.

Description

Your tasks:

1.      Current code refactoring and optimization

2.      New algorithm implementation

 

This work has a tight connection with the project Teleoperation over 5G networks and the IEEE standardization P1918.1.1.

https://www.ei.tum.de/lmt/forschung/ausgewaehlte-projekte/dfg-teleoperation-over-5g/

https://www.ei.tum.de/lmt/forschung/ausgewaehlte-projekte/ieee-p191811-haptic-codecs

Prerequisites

 

C++, knowledge for control engineering and communication protocol

Supervisor: