Open Thesis

intuitive teleoperation and behavior understanding

Description

You need to follow our previous development and implement a demo for real-time human behavior understanding/prediction.

job 1: dataset generation

job 2: pipeline and demo implementation

job 3: algo development

 requirements: knowledge of yolo, opencv, mediapipe, programming with python or C++

Supervisor:

Ongoing Thesis

Bachelor's Theses

haptic data redution for position-position teleoperation control architecture

Keywords:
teleoperation control, haptics

Description

Using a teleoperation system with haptic feedback, the users can thus truly immerse themselves into a distant environment, i.e., modify it, and execute tasks without physically being present but with the feeling of being there. A typical teleoperation system with haptic feedback (referred to as a teleoperation system) comprises three main parts: the human operator OP)/master system, the teleoperator (TOP)/slave system, and the communication link/network in between. During teleoperation, the slave and master devices exchange multimodal sensor information over the communication link. This work aims to develop a haptic data reduction scheme based on a position-position teleoperation architecture and compare the performance with the classical position-force control architecture.

 

Your work:

(1) build up a teleoperation system that can switch between position-position and position-force architectures.

(2) integrate the existing haptic data reduction scheme with the PP architecture.

(3)  introduce delays, and implement existing passivity based control scheme to ensure system stability

(4) compare the performance difference between the PF and PP architectures.

Prerequisites

C++, matlab simulink

Supervisor:

Master's Theses

A perceptual-based rate scalable haptic offline coding scheme

Description

A perceptual-based rate scalable haptic coding for offline kinesthetic compression

to develop a haptic offline coding scheme based on previous studies.

Supervisor:

Digital twin of a position-force teleoperation framework in Nvidia Omniverse

Description

NVIDIA Omniverse is a platform that enables researchers to create custom 3D pipelines and simulate large virtual environments in a fast and convenient manner. It can render the environments very accurately and immersively with the help of GPU acceleration. In this work, we aim to create a digital twin of a teleoperation framework in Omniverse, in which we can use the haptic input device to control the remote robot arm.

Prerequisites

  • Good Programming Skills (Python, C++)
  • Knowledge about Ubuntu/Linux/ROS
  • Motivation to learn and conduct research

Contact

dong.yang@tum.de

Please attach your CV and transcript

Supervisor:

Dong Yang, Xiao Xu

Research Internships (Forschungspraxis)

Handshake for Plug-and-Play Haptic Interaction system

Keywords:
Teleoperation, GUI, Qt, JavaScript
Short Description:
Our project aims to implement handshake communication protocol for Plug-and-Play Haptic Interaction system according to the IEEE standard.

Description

Our project aims to implement handshake communication protocol for Plug-and-Play Haptic Interaction system according to the IEEE standard. For the system, the main achievements are:

 

1.     Plug and play on the Leader side: When the Leader device disconnects from the system, the Follower device will turn to the waiting state and will remain in its initial position same as when it’s activated in the system until the next re-insertion of the Leader device.

2.     Automatic adjustment of device parameters according to the specific type of Leader device to guarantee the performance of human perception: First of all, when connecting, the Leader device will transmit its media and interface information to the Follower side, so-called Metadata, and at the same time it will inform the Follower device of the specific model type it is using. The Follower device will adjust its parameters according to the received information to adapt to the Leader if the type of Leader device has different precision from the Follower device and transmits its metadata to the Leader.

   For the adjustments to the message transmission process:

1).   Achieve the PnP adjustment on the follower side.

 

2).   The message sending order, the format of the interface, the mode of pushing data packets into stacks, and the decoding function should obey the regulations of the IEEE standard.

 

Prerequisites

C/C++

socket programming

visual studio IDE

Contact

Email Adress: siwen.liu@tum.de, xiao.xu@tum.de

Supervisor:

Siwen Liu, Xiao Xu

GUI for Plug-and-Play Haptic Interaction system

Keywords:
Teleoperation, GUI, Qt, JavaScript
Short Description:
Our project aims to build a GUI for Plug-and-Play Haptic Interaction system according to the IEEE standard.

Description

Our project aims to build a GUI for Plug-and-Play Haptic Interaction system according to the IEEE standard. For the system, the main achievements are:

 

1.     Plug and play on the Leader side: When the Leader device disconnects from the system, the Follower device will turn to the waiting state and will remain in its initial position same as when it’s activated in the system until the next re-insertion of the Leader device.

2.     Automatic adjustment of device parameters according to the specific type of Leader device to guarantee the performance of human perception: First of all, when connecting, the Leader device will transmit its media and interface information to the Follower side, so-called Metadata, and at the same time it will inform the Follower device of the specific model type it is using. The Follower device will adjust its parameters according to the received information to adapt to the Leader if the type of Leader device has different precision from the Follower device and transmits its metadata to the Leader.

 

 

Prerequisites

The requirements of our project are as follows. For the GUI part:

1.   The GUI should be implemented under either Qt or JavaScript (first considering Qt) on both the Leader and Follower sides.

2.     For the Leader side, the GUI should be proposed including these functions:

1). Chooses the device on the Leader side.

2). Shows whether the handshaking is successful or not.

3). Shows the device type used on the Follower side after the handshake.

4). When the Leader device is disconnected from the system, show as well.

3.    For the Follower side, the GUI should be proposed including these functions:

1). Chooses the device on the Follower side.

2). Shows whether the handshaking is successful or not.

3). Shows the device type used on the Leader side, adjusts the parameters on the Follower side, and then shows the adjusted device type if the handshake is successful.

4). When the Leader device is disconnected from the system, show as well. And then shows the initial position of the Follower device in the waiting state.

4.    For the adjustments to the message transmission process:

1).   Achieve the PnP adjustment on the follower side.

 

2).   The message sending order, the format of the interface, the mode of pushing data packets into stacks, and the decoding function should obey the regulations of the IEEE standard.

 

Contact

Email Adress: siwen.liu@tum.de, xiao.xu@tum.de

Supervisor:

Siwen Liu, Xiao Xu

Perceptual-oriented objective quality assessment for time-delayed teleoperation

Description

Recent advances in haptic communication cast light onto the promise of full immersion into remote real or virtual environments.

The quality of compressed haptic signals is crucial to fulfill this promise. Traditionally, the quality of haptic signals is evaluated through a series of subjective experiments. So far, only very limited attention was directed toward developing objective quality measures for haptic communication. In this work, we focus on the compression distortion and the delay compensation distortion that contaminate the force/velocity haptic signals generated by physical interaction with objects in the remote real or virtual environment.

 

Prerequisites

 

Contact

 

Supervisor:

Student Assistant Jobs

implementation of teleoperation systems using Raspberry PI

Description

We have already a framework of teleoperation system running in Windows, where two haptic devices are connected through UDP protocol, one as the leader device and the other is the follower.

Your tasks are:

1. move the framework to Linux system.

2. setup a ROS-based virtual teleoperation environment.

3. shift the framework to a raspberry PI.

Prerequisites

Linux, socket programming (e.g. UDP protocol), C and C++, ROS

Supervisor:

Student Assistant for distributed haptic training system

Keywords:
server-client, UDP, GUI programming

Description

Your tasks:

1.      build a serve-client telehaptic training system based on current code.

2.     develop GUI for client side.

required skills:

- c++

- knowledge about socket programming, e.g. UDP

- GUI programming, e.g. QT

- working environment: windows+visual studio



This work has a tight connection with the project Teleoperation over 5G networks and the IEEE standardization P1918.1.1.

https://www.ei.tum.de/lmt/forschung/ausgewaehlte-projekte/dfg-teleoperation-over-5g/

https://www.ei.tum.de/lmt/forschung/ausgewaehlte-projekte/ieee-p191811-haptic-codecs

Supervisor:

Student Assistant for reference software of time-delayed teleoperation with control schemes and haptic codec

Short Description:
In this work, you need to extend the current teleoperation reference software for enabling different control schemes and haptic data reduction approaches.

Description

Your tasks:

1.      Current code refactoring and optimization

2.      New algorithm implementation

 

This work has a tight connection with the project Teleoperation over 5G networks and the IEEE standardization P1918.1.1.

https://www.ei.tum.de/lmt/forschung/ausgewaehlte-projekte/dfg-teleoperation-over-5g/

https://www.ei.tum.de/lmt/forschung/ausgewaehlte-projekte/ieee-p191811-haptic-codecs

Prerequisites

 

C++, knowledge for control engineering and communication protocol

Supervisor: