Incorporating AoA information to MPDC fingerprinting for indoor localization
indoor positioning; fingerprinting; ULA
Description
Contact
fabian.seguel@tum.de
of 1916
Supervisor:
Human-Object Interaction Detection in Videos
Deep Learning, Spatio-temporal feature, Object detection, Transformer.
Description
We identify semantic information about human and object in the video by object detection and human detection and later classify the interaction information by a Transformer model. A public dataset will be used and we will collect the dataset ourselves (we provide the sensors and code to collect the dataset).
Prerequisites
- Good programming skills in Python, Pytorch
- Knowledge of computer vision
- Basic knowledge of handling sequential data
Contact
yuankai.wu@tum.de
Supervisor:
Multiband evaluation of passive signal for human activity recognition
CSI; human activity recognition; AI
Description
Obtain signal information to determine different human activities tasks. Use nexmon firmware package.
https://ieeexplore.ieee.org/document/8307394
Prerequisites
Python
AI/ML
Raspberry pi
Contact
fabian.seguel@tum.de
of. 1916
Supervisor:
Optimizing network deployment for AI-aided AoA estimation
AI; Network deployment; Indoor positioning
Description
The student will optimize through simulations the network atchitecture in order to obtain a robust positioning estimate in a indoor set-up.
Prerequisites
matlab
AI/ML
Contact
fabian.seguel@tum.de
of. 1916
Supervisor:
OCC for indoor positioning and identification in harsh environments
optical camera communications; video tracking
Identify and track multiple light sources in a video stream
Description
Identify and track multiple light sources in a video stream.
The student must record the video multiple objects to the tracked in the video stream.
Prerequisites
Python
Signal/image processing
Contact
fabian.seguel@tum.de
of. 1916
Supervisor:
Multiband analysis of mm-Waves for passive patient monitoring
IQ signals; RADAR; mm-wave
In bed patient monitoring using mm-wave radar systems
Description
The student must use a radar sytem to obtain vital signs of a patient.
The system must be embedded in a hospital bed.
Vital signs such as breathing rate and HR will be targeted; Others applications must be discussed.
Prerequisites
Python
Radar
IQ signal processing
Contact
fabian.seguel@tum.de
of. 1916
Supervisor:
Sub-band analysis for indoor positioning: extracting robust features
OFDMA; CSI; indoor positioning
To obtain samples
Description
The student must set up a transmission scheme based on MIMO technology and OFDMA.
Take samples in different points inside an indoor environment for further processing of the signal characteristics to obtain an estimated position of the mobile node.
Prerequisites
Python
WIreless communications with focus in channel state information and OFDMA MIMO systems
Knowledge in USRP not required but is a plus
Contact
fabian.seguel@tum.de
of. 1916
Supervisor:
Prior-based 3D Reconstruction
3D Reconstruction, 3D Point Clouds, Digital Twin
Description
The generation of a Digital Twin through 3D reconstruction is and will be a major topic for AR applications, 6G communication, and robotics.
Following [1], this thesis also follows the approach of using prior CAD database information for the 3D reconstruction. This thesis may use self-attention-based networks, confidence-part-based shape retrieval [2], and generative networks to partially reconstruct/ partially retrieve objects, thus achieving higher accuracy.
References
[1] Bokhovkin, Alexey and Angela Dai. “Neural Part Priors: Learning to Optimize Part-Based Object Completion in RGB-D Scans.” ArXiv abs/2203.09375 (2022): n. pag. https://arxiv.org/abs/2203.09375
[2] Beyer, Tim and Angela Dai. “Weakly-Supervised End-to-End CAD Retrieval to Scan Objects.” ArXiv abs/2203.12873 (2022): n. pag. https://arxiv.org/abs/2203.12873
[3] Tang, Jiaxiang, Xiaokang Chen, Jingbo Wang and Gang Zeng. “Point Scene Understanding via Disentangled Instance Mesh Reconstruction.” ArXiv abs/2203.16832 (2022): n. pag. https://arxiv.org/abs/2203.16832
Prerequisites
- Experience with Git
- Python (Pytorch)
- Knowledge in working with 3D Point Clouds and Meshes (preferable)
Contact
driton.salihu@tum.de
Supervisor:
Optimization of 3D Object Detection Procedures for Indoor Environments
3D Object Detection, 3D Point Clouds, Digital Twin, Optimization
Description
3D object detection has been a major task for point cloud-based 3D reconstruction of indoor environments. Current research has focused on having a low inference time for 3D object detection. While this is preferable, a lot of cases do not profit from this. Especially considering the use of a pre-defined static Digital Twin for AR and robotics application, thus this decreases the incentive for low inference time at the cost of accuracy.
As such this thesis will follow the approach of [1] (in this work only based on point cloud data) to generate proposals of layout and objects in a scene through for example [2]/[3] and use some form of optimization algorithm (reinforcement learning, genetic algorithm) to optimize to the correct solution.
Further, for more geometrical-reasonable results the use of a relationship graph neural network, as in [4], would be applied in the pipeline.
References
[1] Hampali, Shreyas, et al. “Monte Carlo Scene Search for 3D Scene Understanding.” 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021): 13799-13808. https://arxiv.org/abs/2103.07969#:~:text=We explore how a general, from noisy RGB-D scans.
[2] Chen, Xiaoxue, Hao Zhao, Guyue Zhou, and Ya-Qin Zhang. “PQ-Transformer: Jointly Parsing 3D Objects and Layouts From Point Clouds.” IEEE Robotics and Automation Letters 7 (2022): 2519-2526. https://arxiv.org/abs/2109.05566
[3] Qi, C., Or Litany, Kaiming He and Leonidas J. Guibas. “Deep Hough Voting for 3D Object Detection in Point Clouds.” 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019): 9276-9285. https://arxiv.org/abs/1904.09664
[4] Avetisyan, Armen, Tatiana Khanova, Christopher Bongsoo Choy, Denver Dash, Angela Dai and Matthias Nießner. “SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans.” ArXiv abs/2003.12622 (2020): n. pag. https://arxiv.org/abs/2003.12622
Prerequisites
- Python (Pytorch)
- Experience with Git
- Knowledge in working with 3D Point Clouds (preferable)
- Knowledge about optimization methods (preferable)
Contact
driton.salihu@tum.de
Supervisor:
Development of a Zoom Chatbot for Virtual Audience Feedback
Description
Virtual conference systems provide an alternative to physical meetings that have significantly grown in importance over the last years. However, larger events require the audience to be muted to avoid an accumulation of background noise and distorted audio. While this is sufficient for unidirectional meetings, many types of meetings strongly rely on the feedback of their audience, such as in performing arts.
In this project, we want to extend Zoom sessions with a simple Chatbot that collects the audience participation of each user using a straightforward button interface. Then, the system renders the overall audience feedback based on the feedback state collected from each user. The project combines signal and audio processing with the chance to gain practical experience with app development and SDKs.
References
Prerequisites
-
Good knowledge in Nodejs/JavaScript
-
Experience with Git
-
Experience with Zoom SDK would be a plus
Supervisor:
High-level Robotic Teleoperation via Scene Editing
Description
Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on user interaction methods for describing a robotic task via modifying the viewed scene.
Prerequisites
Useful background:
- 3D Human-computer interfaces
- Game Design
- Digital signal processing
Required abilities:
- Experience with Unity and ROS
- Motivation to yield a good work
Contact
furkan.kaynar@tum.de
(Please provide your CV and transcript in your application)
Supervisor:
Robotic grasp learning from human demonstrations
Description
Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on learning from human demonstration for improving the robot autonomy.
Prerequisites
Required background:
- Digital signal processing
- Computer vision
- Neural networks and other ML algorithms
Required abilities:
- Experience with Python or C++
- Experience with Tensorflow or PyTorch
- Motivation to yield a good thesis
Contact
furkan.kaynar@tum.de
(Please provide your CV and transcript in your application)
Supervisor:
Reinforcement Learning for Estimating Virtual Fixture Geometry to Improve Robotic Manipulation
Description
Robotic teleoperation is often used to accomplish complex tasks remotely with human-in-the-loop. In cases, where the task requires very precise manipulation, virtual fixtures can be used to restrict and guide the motion of the end effector of the robot while the person teleoperates. In this thesis, we will analyze the geometry of virtual fixtures depending on the scene and task. We will use reinforcement learning to estimate ideal virtual fixture model parameters. At the end of the thesis, the performance can be evaluated with user experiments.
Prerequisites
Useful background:
- Machine learning (Reinforcement Learning)
- Robotic simulation
Requirements:
- Experience with Python & Deep learning frameworks (PyTorch / Tensorflow...)
- Experience with a RL framework
- Motivation to yield a good outcome
Contact
(Please provide your CV and transcript in your application)
furkan.kaynar@tum.de
diego.prado@tum.de
Supervisor:
Jacobian Null-space Energy Dissipation TDPA for Redundancy Robots in Teleoperation
Teleoperation, Robotics, Control Theory
Description
Teleoperation Systems
Bilateral teleoperation with haptic feedback provides its users with a new dimension of immersion in virtual or remote environments. This technology enables a great variety of applications in robotics and virtual reality, such as remote surgery and industrial digital twin [1]. Figure 1 shows a generalized human-in-the-loop teleoperation system with kinesthetic feedback, where the operator commands a remote/virtual robot to explore the environment and experiences the interactive force feedback through the haptic interface.
Teleoperation systems face many challenges caused by unpredictable environment changes, time-delayed feedback, limited network capacity, etc. [2]. These issues inevitably distort the force feedback signal, degrading the transparency and stability of the system. In the past decades, many control algorithms and hardware architectures were developed to tackle these problems in the past decades [3].
Time Domain Passivity Method (TDPA)
TDPA is a passivity-based control scheme that ensures the stability of teleoperation systems in the presence of communication delays [4] (See Figure 2.). It abstracts two-port networks from the haptic system and observes the energy flow between the networks. Passivity condition is maintained by dissipating extra energy generated by non-passive networks. Original TDAP suffers from position drift and feedback force jump [5], and one reason for the position drift is that the energy, which is generated by the delayed communication, is dissipated in the task space of the robots.
Jacobian Null-Space for Redundancy Robot
Many robot mechanisms have redundant degrees of freedom (rDOFs), which means that they have a larger number of joints than the number of dimensions of their task or configuration space. The null space of the Jacobian null space stands for the redundant dimensions which can be exploited to dissipate extra energy by damping the null space motion without affecting the task space [5].
Your Task and Target
In this work, we target at improving the performance of TDPA by considering dissipating energy generated by time delay and other factors in the Jacobian null-space of the kinesthetically redundant robots. With the help of the Jacobian null-space method, we can avoid dissipating energy in the task space, so as to alleviate position drift and force distortion while keeping the system passive. For more information, previous work can be referred to as [7-9].
In this master's internship, your work will include
1. 1. surveying the related algorithms
2. 2. constructing the simulation environment
3. 3. experimenting with the state-of-the-art Jacobian null-space TDPA method.
4. 4. analyzing system passivity in Cartesian task space, joint space, null space, etc.
Prerequisites
Requirements
All requirements are recommended but not mandatory. However, you will need extra effort to catch up if you are unfamiliar with the following topics:
1. 3. Basic knowledge about robotics and control theory is favorable.
2. 2. Experience with robotics simulation software and platforms is favorable.
3. 1. C++, Matlab, and Python would be the primary working language. Basic knowledge about one or more of them is highly recommended.
Contact
zican.wang@tum.de
xiao.xu@tum.de
Supervisor:
Student work in game-engine based human activity simulation
Description
This is a working student position at the Human Activity Understanding Group at the Chair of Media Technology.
The student will support various initiatives in game-engine based 3D simulation. We work with both Unity3D as well as Unreal Engine. Some videos of our simulator in action can be seen at: https://www.ce.cit.tum.de/lmt/forschung/themen/human-activity-understanding/
In particular, the student will extend and maintain the existing human activity simulators, and also work on interfacing VR hardware such as:
- Head-mounted displays like Oculus or Vive, Microsoft Hololens
- Motion capture system (OptiTrack)
- VR gloves
- Custom hardware setups
Also see: https://portal.mytum.de/schwarzesbrett/hiwi_stellen/NewsArticle_20220407_071536
Prerequisites
Strong interest and experience in game development and 3D simulation. Experience with one of ideally both Unity3D and Unreal Engine, OR experience with Blender and interest in learning Unity3D/Unreal.
Please send in your CV and describe your experience at the time of application.
Contact
https://www.ce.cit.tum.de/lmt/team/mitarbeiter/chaudhari-rahul/
Supervisor:
Learning-based human-robot shared autonomy
robot learning, shared control
Description
In shared control teleoperation, the robot intelligence and human input can be blended together to achieve improved task performance and reduce the human workload. In this topic, we would like to investigate how we can combine human input and robot intelligence effectively to achieve at the end full robot autonomy. We will employ robot learning from demonstration approaches, where we provide task demonstrations using teleoperation.
We aim to test the developed algorithms in simulation and using Franka Emika robot arm.
Requirements:
Basic experience in C/C++
ROS is a plus
High motivation to learn and conduct research
Supervisor:
MATLAB tutor for Digital Signal Processing lecture in summer semester 2022
Description
Tasks:
- Help students with the basics of MATLAB (e.g. matrix operations, filtering, image processing, runtime errors)
- Correct some of the homework problems
- Understand the DSP coursework material
We offer:
- Payment according to the working hours and academic qualification
- The workload is approximately 6 hours per week from May 2022 to August 2022
- Technische Universität München especially welcomes applications from female applicants
Application:
- Please send your application with a CV and transcript per e-mail to basak.guelecyuez@tum.de
- Students who have taken DSP course preferred.
Supervisor:
Embodied Object Detection in Real Environments
Description
In this project, we investigate embodied (active) object detection in real-world environments. The goal is to actively adapt the path of an autonomous agent to increase the performance of an object detector.
Prerequisites
- Object detection
- Pytorch
- Knowledge of graph optimization
Supervisor:
Scene graph generation models analysis using Visual Genome benchmark and self-recorded datasets
Description
Scene graphs were first proposed [1] as a data structure that describes the object instances in a scene and the relationships between these objects. The nodes in the scene graph represent the detected target objects, whereas the edges denote the detected pairwise relationships. A complete scene graph can represent the detailed semantics of a dataset of scenes.
Scene graph generation (SGG) is a visual detection task for building structured scene graphs. In this work, we would like to compare the three traditional SGG models: Neural Motifs [2], Graph R-CNN [3], and IMP [4]. We will train and evaluate the models using the Visual Genome dataset and other commonly used datasets. In addition, our dataset will be annotated and then utilized.
[1]: J. Johnson, et al. "Image retrieval using scene graphs."
[2]: R. Zellers, et al. "Neural motifs: Scene graph parsing with global context."
[3]: J. Yang, et al. "Graph r-cnn for scene graph generation."
[4]: D. Xu, et al. "Scene graph generation by iterative message passing."
Prerequisites
- Good Programming Skills (Python, GUI design)
- Knowledge about Ubuntu/Linux/Pytorch
- Knowledge about Computer vision/Neural network
- Motivation to learn and conduct research
Contact
dong.yang@tum.de
Supervisor:
Generalized Robot Learning from Demonstration
LfD, imitation learning, Franka Arm
Description
In this work, we would like to design a generative model for robot learning from demonstration. We will consider a table cleaning task, determine varying task conditions, and apply a generative model such that the robot can generalize to novel unseen conditions. We aim to use both simulation and Franka arm for testing our algorithms.
Prerequisites
Requirements:
Experience in C/C++
ROS is a plus
High motivation to learn and conduct research
Supervisor:
Network Aware Imitation Learning
Teleoperation, Learning from Demonstration
Description
When teaching robots remotely via teleoperation, the communication (between the human demonstrator and the remote robot learner) imposes challenges. Network delay, packet loss, and data compression might cause completely or partially degraded demonstration qualities. In this thesis, we would like the make the best out of varying quality demonstrations provided via teleoperation with haptic feedback. We will use different teleoperation setups to test the developed robot learning approaches:
Prerequisites
Requirements:
Experience in C/C++
ROS is a plus
High motivation to learn and conduct research
Supervisor:
Robotic task learning from human demonstration
Description
Autonomous grasping and manipulation are complicated tasks which require precise planning and a high level of scene understanding. Although robot autonomy is evolving since decades, there is still need for improvement, especially for operating in unstructured environments like households. Human demonstration can improve the autonomous robot abilities further to increase the task success in different scenarios. In this thesis we will work on learning from human demonstration for improving the robot autonomy.
Prerequisites
Required background:
- Digital signal processing
- Computer vision
- Neural networks and other ML algorithms
Required abilities:
- Experience with Python or C++
- Experience with Tensorflow or PyTorch
- Motivation to yield a good thesis
Contact
furkan.kaynar@tum.de
(Please provide your CV and transcript in your application)
Supervisor:
Embedded TPU-based Inertial-Camera Edge AI
embedded linux, ARM Cortex, PCB Design, IMU
Description
In this project, we aim to capture low-latency RGB data from a CSI camera in sync with an inertial measurement unit(IMU) and use the TPU unit for AI processing, especially for real-time segmentation applications. The first step of the project is to modify the provided embedded Linux kernel and the embedded board with an ARM Cortex processor to communicate with the TPU module. In the next step of the project, the synchronization of the inertial measurement data must be addressed with the frames of the RGB data from the camera. We use Edge TPU ML accelerator ASIC designed by Google that provides high-performance ML inferencing for TensorFlow Lite models.
Prerequisites
Good programming skills in C++.
Strong understanding of embedded Linux programming.
Knowledge about Debian Kernel and UBoot.
Knowledge about ARM Cortex Processors.
Contact
Please send your application to:
mojtaba.karimi@tum.de
edwin.babaians@tum.de