Real time 3D tracking of objects using one or more cameras is crucial to build a Digital Twin. In this project, you will improve an algorithm for 3D tracking and pose estimation, and use it to update a Digital Twin of a factory environment that is used in robotic manipulation tasks.

We will pay special attention to the tracking of textureless objects and the speed of the algorithm. We will also try to compare the results using one and more cameras.

For this work, good knowledge of C++ is required.

Some knowledge of Python and ROS will be useful, but it is not required.

A Digital Twin is a virtual representation of an asset, to which is connected in a bi-directional way: changes happening in the real asset are shown in the digital asset and vice-versa.

In this project, you will improve force rendering algorithms to make teleoperation more user friendly through means of the Digital Twin of a factory. 

Required:

• Python knowledge
• Chai3D (ideally you have participated in the Computational Haptics Laboratory)

Recommended (not all of them):

• Experience in ROS
• C++ knowledge
• Robotics knowledge
• MuJoCo

Robotic teleoperation is often used to accomplish complex tasks remotely with human-in-the-loop. In cases, where the task requires very precise manipulation, virtual fixtures can be used to restrict and guide the motion of the end effector of the robot while the person teleoperates. In this thesis, we will analyze the geometry of virtual fixtures depending on the scene and task. We will use reinforcement learning to estimate ideal virtual fixture model parameters. At the end of the thesis, the performance can be evaluated with user experiments.

Useful background:

- Machine learning (Reinforcement Learning)

- Robotic simulation

Requirements:

- Experience with Python & Deep learning frameworks (PyTorch / Tensorflow...)

- Experience with a RL framework

- Motivation to yield a good outcome