Motivation

Chemical laboratories rely on a range of automated machines for tasks like liquid handling or centrifugation. Yet many repetitive procedures are still performed by human chemists, particularly if dexterous manipulation or adaptation to changing conditions are needed. Fully automating these tasks through traditional programming is impractical, since lab setups or execution protocols can vary significantly over time or between sites.

Learning from demonstration offers a compelling alternative. Instead of explicitly programming every motion and decision, an expert demonstrates a task and the robot learns a generalizable policy from a set of demonstrations. Combined with multimodal sensing, where data from cameras, tactile sensors and the robot's own proprioception are fused to understand the scene and guide execution, this enables a system that can adapt to variations and recover from disturbances.

The approach is not limited to laboratory tasks and can generalize to applications in manufacturing, assembly, etc.

System

The setup consists of a 4-DOF SCARA robot arm equipped with a parallel gripper, camera(s), tactile sensors and laboratory equipment. A demonstration setup already exists and is capable of showcasing the robot's manipulation capabilities in a laboratory environment.

Research Project

In this project, you will build on the existing demonstration setup and advance it towards robust learning from demonstration. This includes improving the physical setup, defining meaningful laboratory tasks to be learned and developing a multimodal sensing and learning pipeline. You will work with state-of-the-art models and algorithms in robot learning, adapt/tune them to the SCARA platform and evaluate how reliably new tasks can be acquired from a limited number of expert demonstrations. The work combines elements of robotics, machine learning, sensor fusion, and mechatronics.

Goals

Improve the existing demonstration setup and define a set of representative laboratory tasks for learning.

Improve and further develop a multimodal sensing pipeline that fuses data from cameras, robot proprioception and (possibly) tactile sensors to inform task execution.

Implement and tune learning-from-demonstration algorithms that allow the robot to acquire new tasks from expert demonstrations.

Evaluate robustness: the learned policies should handle (small) variations in conditions and recover from disturbances during execution.

Voraussetzungen

Motivation

Legged robotic platforms (robot dogs) enable remote inspections in environments that may be unstructured or hazardous to humans. Their ability to navigate rough terrain makes them more versatile than wheeled alternatives. Equipping a highly mobile quadruped with a stereoscopic vision system and linking it to an operator's head-mounted display (HMD) creates a telepresence system that offers a natural first-person view of the remote site, while keeping the human operator safe and preserving their situational awareness.

One central challenge of this approach is latency. Any delay between the operator's head movement and the corresponding visual update on the HMD breaks the sense of presence and can lead to motion sickness. In practice, this can heavily limit uninterrupted teleoperation time. Addressing this requires intelligent compensation strategies that anticipate and mask transmission delays.

System

The platform consists of a quadruped robot carrying a mechanically actuated stereoscopic camera system, which is wirelessly linked to the operator's HMD. The camera system mirrors the operator's head orientation in real time, providing a natural first-person perspective of the remote environment.

Research Project

In this project, you will develop and evaluate latency compensation techniques for the stereoscopic vision system. By exploiting optical and geometric properties of the camera system, combined with learning-based networks that predict the operator's gaze and steering intent from sensor and control data, the system is supposed to proactively prepare visual outputs before they are needed. You will integrate your approach into the existing teleoperation platform and evaluate it through user studies. The work combines elements of computer vision, machine learning, robotics, communication engineering and human-robot interaction.

Goals

Design and implement a latency compensation approach that exploits the vision system's design and develop learning-based intention prediction algorithms to maintain immersion under realistic network conditions. Ideally, you will be able to demonstrate a measurable reduction of motion sickness and visual discomfort compared to an uncompensated baseline. A central goal is to validate that operators can more comfortably perform continuous remote inspections for extended periods of time.

Voraussetzungen