See TUMonline

After successful completion of the course, students have understood the working principles of teleoperation systems with haptic feedback. They are ready to write code to connect haptic devices with remote robots, manipulate the robots and interact with remote objects while perceiving stable force feedback. Moreover, the students are also able to program and remotely transfer necessary knowledge to the robot for executing complex tasks at full autonomy. Besides, the students will be able to explain the main components of autonomous drones, i.e. state estimation, mapping, motion planning and control. In addition, they will know in detail the working principles of 3D occupancy mapping and planning, and be able to implement them as a real-time ROS/C++ application. They will furthermore be given awareness for what intuitive user interaction requires regarding ease of use as well as safety.

● Overview of robotics and control o Introduction to robotics and control of robotic systems: a general overview of kinematics, dynamics and control of the robotic systems are given. o Motion control: Decentralized control, PD plus gravity and inverse dynamics controller and their closed-loop dynamics are illustrated. o Force/impedance control: The impedance control in task space and joint space is shown is reviewed. ● Telerobotic systems and applications o Overview of telerobotics: the importance of telerobotics and different examples are reviewed. o Control architectures: different control architecture used in teleoperation is introduced. ▪ Direct control ▪ Shared control ▪ Supervisory control o Stability and transparency in tele-manipulations: the importance of stability and transparency is shown in illustrative example scenario. o Telepresence reference platform at MIRMI: our telepresence setup with different protocols is introduced, and the performance of the system in contact with different environments is presented. o Clear intensive guidance for programming with haptic devices o Designing a teleoperation system for remote interaction and skill transfer ● Teleoperation with haptic devices o Intensive guidance for programming with haptic devices and designing teleoperation system for remote interaction and skill transfer. o Making students user-ready to haptic technology and challenges for teleoperation systems over real communication networks (e.g. system stability, communication load, etc.). o Working with haptic devices and remote robot in virtual reality, introduction to ROS, Chai3D, and then a quick overview about available examples. o Overview of teleoperation design in real environment and corresponding stability issues, and remote learning from demonstration technologies. o Writing codes and programming the teleoperation system either in virtual or real environments to perform a certain number of tasks that are ranging from simple to complex. ● Drone and its application o Simulation (Gazebo) and a real one that has basic semi-automatic control already implemented, including on-board localization. o Students will implement is on-board occupancy mapping using the mounted depth camera, as well as (local) planning to avoid collisions with the environment. o Develop a final demo, in which the drone can be steered intuitively with a keyboard, while the drone is inherently safe and will only ever move as far as safe to avoid colliding with the environment.

- Fundamentals of control theory - Fundamentals of robotics - Fundamentals of mobile robotics - Fundamentals of 3D computer vision - C, C++ - Python

During the lectures, students are instructed in a teacher-centered style. In the lab students will perform several experiments and solve various assignments. In particular: ● Lectures (for direct transfer of theoretical knowledge) ● Lab assignments (for testing the learned approaches) ● Final task (to evaluate whether students can transfer the methods they have learned during the course and applied to real-life complex task)

The evaluation is based on lab assignments where students write code to perform robotic tasks. After completing the assignments, students demonstrated their ability to solve given issues. A primary task weighs 40% of the grade will be assigned, followed by a final presentation. It enables them to apply their learning from the assignments, and then to overcome a comprehensive problem related to the topics from the lectures. -4 Lab assignments, each weighing 15% of the final course grade, assessed as -Write code 5%: to demonstrate algorithms simulation and real embedded system. -Implement the real systems 5%: dealing with real robots -Solve the given problem 5%: applying the learning to overcome the specific issues. -A final task weighs 40% of the final course grade, assessed as -Programming 10%: implementing the complete a robotic program to a specific task. -Solving the task 25%: Apply the knowledge from lectures sections to overcome a complex issue. -Final presentation 5%

- Siciliano, Bruno, Oussama Khatib, and Torsten Kröger, eds. Springer handbook of robotics. Vol. 200. Berlin: springer, 2008. - Peter F. Hokayem∗, MarkW. Spong. Bilateral teleoperation: A historical survey, Automatica 2006. - Jae-young Lee, Shahram Payandeh. Haptic Teleoperation Systems - Signal Processing Perspective. Springer, 2015 - S. Sonia Chernova, Andrea L. Thomaz. Robot Learning from Human Demonstration. Springer, 2014 - D. Fox, S. Thrun, W. Burgard: Probabilistic Robotics - Siliciano, Bruno, et al. "Robotics: modelling, planning and control." New YorN, NY, USA: Springer (2010): 415-418. - Peter F. Hokayem∗, MarkW. Spong. Bilateral teleoperation:An historical survey, Automatica 2006. - E. Steinbach, et, al. Haptic Communications, Proceedings of IEEE, 2012 - Lawrence, Dale A. "Stability and transparency in bilateral teleoperation." IEEE transactions on robotics and automation 9.5 (1993): 624-637. - Nuño, Emmanuel, Luis Basañez, and Romeo Ortega. "Passivity-based control for bilateral teleoperation: A tutorial." Automatica 47.3 (2011): 485-495. - Chen, Xiao, et al. "On the Communication Channel in Bilateral Teleoperation: An Experimental Study for Ethernet, WiFi, LTE and 5G." 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022.