Diffusion models are powerful generative models that enable many successful applications, such as image, video, and 3D generation from texts. It's inspired by non-equilibrium thermodynamics, which defines a Markov chain of diffusion steps to slowly add random noise to data and then learn to reverse the diffusion process to construct desired data samples from the noise. 

In this work, we aim to explore the application of the diffusion model or its variants in imitation learning and evaluate it on the real-world Franka robot arm.

• Good Programming Skills (Python, C++)
• Knowledge about Ubuntu/Linux/ROS
• Motivation to learn and conduct research

NVIDIA Omniverse is a platform that enables researchers to create custom 3D pipelines and simulate large virtual environments in a fast and convenient manner. It can render the environments very accurately and immersively with the help of GPU acceleration. In this work, we aim to create a digital twin of a teleoperation framework in Omniverse, in which we can use the haptic input device to control the remote robot arm.

• Good Programming Skills (Python, C++)
• Knowledge about Ubuntu/Linux/ROS
• Motivation to learn and conduct research

Multi-sensor data can provide rich environmental information for robots. In practical applications, it is necessary to ensure real-time and synchronous processing of sensor data. In this work, the student needs to design a ROS-based sensor data acquisition and processing framework and carry it on an existing robot platform. Specifically, the sensors involved in this project include RGBD camera, millimeter-wave radar, LiDAR, and IMU. There exist clock deviations between different sensors. The student needs to calibrate the clocks uniformly to make the timestamps of the data collected by the sensors consistent, transmit the collected data to the robot platform in real-time, and process them into the required data, such as point clouds, RGB pictures, etc.

• Strong familiarity with ROS, C++, and Python programming
• Experience with hardware and sensors
• Basic knowledge of robotics