This topic focuses on improving robotic grasping networks and advancing embodied intelligence. Grasping is a fundamental capability in robotic manipulation and often plays a decisive role in the overall success of a task. Despite significant progress in learning-based grasping, current models still struggle with generalization and robustness in unstructured environments. Our goal is to enhance the success rate of existing grasping models and deploy them in real-world scenarios, where they can provide intelligent assistance during teleoperation tasks. By leveraging pre-trained grasping networks, we aim to reduce the human operator's workload, increase autonomy, and improve manipulation efficiency in complex and dynamic settings. This work offers a unique opportunity to work at the intersection of perception, control, and learning—pushing the boundaries of what robots can achieve through smarter, more adaptive grasping.

Prerequisites

Diffusion models are powerful generative models that enable many successful applications, such as image, video, and 3D generation from texts. It's inspired by non-equilibrium thermodynamics, which defines a Markov chain of diffusion steps to slowly add random noise to data and then learn to reverse the diffusion process to construct desired data samples from the noise. 

In this work, we aim to explore the application of the diffusion model or its variants in imitation learning and evaluate it on the real-world Franka robot arm.

Prerequisites

With the rapid development of embodied intelligent robots, real-time and accurate scene understanding is crucial for robots to complete tasks efficiently and effectively. Scene graphs represent objects and their relations in a scene via a graph structure. Previous studies have generated scene graphs from images or 3D scenes, also with the assistance of large language models (LLMs). 

In this work, we investigate the application of scene graphs in assisting the human operator during the teleoperated manipulation task. Leveraging real-time generated scene graphs, the robot system can obtain a comprehensive understanding of the scene and also reason the best solution to complete the manipulation task based on the current robot state.

Prerequisites