Human-centered Control

The human-inspired technological developments enable innovative control applications which are difficult to achieve with traditional control methods. In human-in-the-loop systems, for instance, employing classical system identification techniques for model derivation is challenging because of the complexity and probabilistic nature of human behavior. Thus, our research focuses on developing novel concepts of automatic control for stochastic systems, based on data-driven human models with uncertainties. In addition to advancements in control theory and machine learning techniques, we employ neuroscientific methodologies to understand how the human behavior is controlled by the CNS which are then explicitly interfaced with various control applications such as pHRI.

Current topics:

Related projects:

Current topics:

Machine learning for monitoring of Parkinson's disease

Researcher: Satoshi Endo, Neha Das

Motivation

Our research efforts centre on a continuous, non-invasive monitoring system for patients with movement deficits, and patient-specific adaptive rehabilitation/healthcare technologies. Neurological disorders such as Parkinson’s disease (PD) has complex phenomenology not only across patients but also within themselves across time. Thus, continuous and non-invasive monitoring of clinical states of the patients during daily activities could provide valuable support for healthcare management such as a medication plan or physiotherapy training. Nevertheless, the current healthcare models typically involve clinical evaluations performed very sparsely (e.g. every 4 to 6 months) because of the physicians’ time and economic constraints. Thus, we are developing robust control-oriented modelling and estimation methods for closely monitoring changes in the patient's state and neurological deficits. These include employing various forms of treatment together with a low-cost wearable sensor (e.g., a smart watch).

Research question 

  • Quantitative characterisation of clinical symptoms and interaction effects of various factors such as medication and fitness level in unconstrained (i.e. daily) environments.
  • Robust modelling and predicting of clinical state of patients against noisy measurements of movements in low-cost sensors.
  • Transferring the clinical insights into control applications for optimising treatment efficacy.

Approach

We are currently developing a machine learning technique for reliably estimating progression and fluctuation of motor deficits in patients with PD using inertia measurements available from a low-cost wearable sensor (e.g., a smart watch). In particular, we employ Gaussian Processes (GP) in order to ensure that the errors from low-cost devices are transparently and robustly processed by the model for estimating a current PD state. 

Key results and achievements 

  • Extraction of tremor information from inertia measurements during daily activities using continuous wavelet transform and machine learning.
  • Parameterisation of PD-specific motions and application of GP for severity estimation

  • Pfister, Franz M. J.; Um, Terry Taewoong; Pichler, Daniel C.; Goschenhofer, Jann; Abedinpour, Kian; Lang, Muriel; Endo, Satoshi; Ceballos-Baumann, Andres O.; Hirche, Sandra; Bischl, Bernd; Kulić, Dana; Fietzek, Urban M.: High-Resolution Motor State Detection in Parkinson's Disease Using Convolutional Neural Networks. Scientific Reports 10 (1), 2020, 5860 mehr… BibTeX
  • M. Lang; F.J. Pfister; J. Fröhner; K. Abedinpour; D. Pichler; U. Fietzek; T.T. Um; D. Kulić; S. Endo; S. Hirche: A Multi-layer Gaussian Process for Motor Symptom Estimation in People with Parkinson’s Disease. IEEE Transactions on Biomedical Engineering 66 (11), 2019, 3038-3049 mehr… BibTeX
  • S. Endo; F.J. Pfister; J. Fröhner; U. Fietzek; D. Pichler; K. Abedinpour; T.T. Um; D. Kulić; M. Lang; S. Hirche: Dynamics-based estimation of Parkinson's disease severity using Gaussian Processes. Second IFAC Conference on Cyber-Physical & Human Systems, 2018 mehr… BibTeX
  • F. Pfister; D. Kulić; T. Um; D. Pichler; A. Ahmadi; M. Lang; G. König; F. Achilles; S. Endo; K. Abedinpour; K. Ziegler; K. Bötzel; S. Hirche; A. Ceballos-Baumann; U. Fietzek: Deep Learning in Objective Classification of Spontaneous Movement of Patients with Parkinson’s Disease Using Large-Scale Free-Living Sensor Data. International Parkinson and Movement Disorder Society 201721st International Congress, Vancouver, BC mehr… BibTeX
  • P. Beckerle; G. Salvietti; R. Unal; D. Prattichizzo; S. Rossi; G. Castellini; S. Hirche; S. Endo; H.B. Amor; M. Ciocarlie; F. Mastrogiovanni; A.D.Brenna; M. Bianchi: A Human–Robot Interaction Perspective on Assistive and Rehabilitation Robotics. Frontiers in Neurorobotics (11), 2017 mehr… BibTeX
  • T. Um; F. Pfister; D. Pichler; S. Endo; M. Lang; S. Hirche; U. Fietzek ; D. Kulić: Data augmentation of wearable sensor data for parkinson's disease monitoring using convolutional neural networks. Proceedings of the 19th ACM International Conference on Multimodal Interaction - ICMI 2017, 2017 mehr… BibTeX

Neuroscientific models of Human-Human and Human-Robot Interaction

Researcher: Satoshi Endo

Motivation

Our research interest centres around neuroscientific understandings of human movement control in dynamic environments, with a special emphasis on motor coordination with another agent including a robotic system. We conduct scientific research about how neuroscientific variables such as perception, cognition, and motor control influence the interaction dynamics of multi-agent coordination. The results are used to derive techniques for effectively controlling mult-agent interactions in, for example, assistive technologies.  

Research questions

  • Perceptual and cognitive factors attributing observation of external input as intentional or erroneous.
  • Evolution of cooperative strategies between agents
  • Models of intention recognition based on interaction dynamics and perceptual cues

Approach

  • Our method consists of modelling human behavior as a stochastic dynamical system and studying how the coordination is moderated by human agents. Human-driven models are then implemented for adaptive control of a robotic system in HRI tasks.

Key results and achievements

  • Modelling multi-agent coordination as a dynamical system to statistically characterise the coordination strategy.
  • Application of bio-inspired motions and human models for object manipulation in pHRI. 

  • Kostavelis, Ioannis; Kargakos, Andreas; Skartados, Evangelos; Peleka, Georgia; Giakoumis, Dimitrios; Sarantopoulos, Iason; Agriomallos, Ioannis; Doulgeri, Zoe; Endo, Satoshi; Stüber, Heiko; Janjoš, Faris; Hirche, Sandra; Tzovaras, Dimitrios: Robotic Assistance in Medication Intake: A Complete Pipeline. Applied Sciences 12 (3), 2022 mehr… BibTeX
  • Johannsen, Leif; Potwar, Karna; Saveriano, Matteo; Endo, Satoshi; Lee, Dongheui: Robotic Light Touch Assists Human Balance Control During Maximum Forward Reaching. Human Factors: The Journal of the Human Factors and Ergonomics Society, 2020, 001872082095053 mehr… BibTeX
  • Endo, Satoshi; Fröhner, Jakob; Music, Selma; Hirche, Sandra; Beckerle, Philipp: Effect of External Force on Agency in physical Human-Machine Interaction. Frontiers in Human Neuroscience 14, 2020 mehr… BibTeX
  • H. Börner; S. Endo; S. Hirche: Estimation of Involuntary Components of Human Arm Impedance in Multi-Joint Movements via Feedback Jerk Isolation. Frontiers in Neuroscience 14, 2020, 459 mehr… BibTeX
  • S. Kager; A. Hussain; A. Cherpin; A. Melendez-Calderon; A. Takagi; S. Endo; E. Burdet; S. Hirche; M. H. Ang; D. Campolo: The Effect of Skill Level Matching in Dyadic Interaction on Learning of a Tracing Task. 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR), 2019 mehr… BibTeX
  • J. R. Medina; H. Börner; S. Endo; S. Hirche: Impedance-based Gaussian Processes for Modeling Human Motor Behavior in Physical and Non-physical Interaction. IEEE Transactions on Biomedical Engineering 66 (9), 2019, 2499-2511 mehr… BibTeX
  • P. Donner; S. Endo; M. Buss: Physically Plausible Wrench Decomposition for Multieffector Object Manipulation. Transaction on Robotics (34(4)), 2018, 1053-1067 mehr… BibTeX
  • H. Börner; S. Endo; S. Hirche: Estimation of Involuntary Impedance in Multi-joint Arm Movements. IFAC Conference on Cyber-Physical & Human Systems (CPHS), 2018 mehr… BibTeX
  • J. Fröhner; P. Beckerle; S. Endo; S. Hirche: An embodiment paradigm in evaluation of human-in-the-loop control. IFAC Conference on Cyber-Physical & Human Systems (CPHS), 2018 mehr… BibTeX
  • D. Cehajic; S. Endo; M. Aggravi; D. Prattichizzo; S. Hirche: Spatial guidance of the human arm motion using vibrotactile feedback. IEEE World Haptics 2017, 2017Work-In-Progress paper mehr… BibTeX
  • P. Beckerle; G. Salvietti; R. Unal; D. Prattichizzo; S. Rossi; G. Castellini; S. Hirche; S. Endo; H.B. Amor; M. Ciocarlie; F. Mastrogiovanni; A.D.Brenna; M. Bianchi: A Human–Robot Interaction Perspective on Assistive and Rehabilitation Robotics. Frontiers in Neurorobotics (11), 2017 mehr… BibTeX
  • J. R. Medina; S. Endo; S. Hirche: Impedance-based Gaussian Processes for Predicting Human Behavior during Physical Interaction. International Conference on Robotics and Automation 2016, 2016 mehr… BibTeX
  • M. Lang, S. Endo, O. Dunkley and S. Hirche: Object Handover Prediction using Gaussian Processes clustered with Trajectory Classification. arXiv preprint: arXiv:1707.02745, 2014 mehr… BibTeX
  • A.M. Wing; S. Endo; A. Bradbury; D. Vorberg: Optimal feedback correction in string quartet synchronization. Journal of The Royal Society Interface 11 (93), 2014 mehr… BibTeX
  • A. Bartolo; Y. Coello; M.G. Edwards; S. Delepoulle; S. Endo; A.M. Wing.: Contribution of the motor system to the perception of reachable space: an fMRI study. European Journal of Neuroscience, 2014, 1-11 mehr… BibTeX
  • M. Prada; A. Remazeilles; A. Koene; S. Endo: Implementation and experimental validation of Dynamic Movement Primitives for object handover. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014 mehr… BibTeX
  • M. Prada; A. Remazeilles; A. Koene; S. Endo: Dynamic Movement Primitives for Human-Robot interaction: Comparison with human behavioral observation. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2013 mehr… BibTeX
  • A. Koene; J. Honisch; S. Endo; A.M. Wing: Relative information content of gestural features of non-verbal communication related to object-transfer interactions. Tilburg Gestures Research Meeting 2013, 2013 mehr… BibTeX

Inferring intent during physical Human-Robot-Interaction

Researcher: Samuel Tesfazgi

Motivation

With recent advances in robotic technologies, lightweight robots are becoming more accessible and are increasingly being deployed in close proximity to humans. In order for humans and robots to cooperate effectively in previously unspecified contexts, the robotic partner needs the capacity to infer the intent of the human during tasks and adapt its behavior accordingly. One common approach to perform this intention estimation is the framework of inverse reinforcement learning (IRL), also referred to as inverse optimal control. However, methods based on this approach suffer from intrinsic optimality assumptions towards the observed agent, therefore, not generalizing well to suboptimal and learning agents. Our research efforts center around generalizing IRL frameworks for suboptimal agents, such as human with injuries or limited movement capabilities, by combining insights from control theory and data-driven learning methods.

Research questions

  • Design of an inverse reinforcement learning algorithm based on suboptimal demonstrations
  • Provide guaranteed bound for predicted intent and behavior
  • Adaptation of inferred cost function for agents with time-varying control policies

Approach

We are currently designing a machine learning technique based on neural networks to model the agent’s behavior as a dynamical system which is analyzed using tools of nonlinear control.

Modeling motion health and healthy motion generation

Researcher: Neha Das

Motivation

Patients with motor impairments (incurred, for example, as a result of stroke) in one or more joints often overuse other redundant joints in order to accomplish activities of daily living or perform rehabilitation exercises. However, in the long term, this can hinder regaining full use of the impaired joints. Identification and correction of such compensatory behavior is therefore imperative for successful patient recovery. However, compensatory strategies adopted by patients are often task and context dependent as well dependent on the patient's habits and the severity of the impairment and manifest in a very wide variety of observable atypicalities in the performed motion, which makes the task of identification very challenging. Our research is focused on devising robust methods that overcome these challenges to identify compensatory behavior in patients accurately and incorporate this information in a feedback cycle to the patients via various means, such as direct communication and force feedback through rehabilitation robots in order to facilitate faster patient recovery.

Research questions

  • Accurate identification of compensating joints across a wide variety of motion classes or primitives as well as for a wide variety of patients.
  • Designing a method of feedback that can encourage healthy motion in patients personalized based on the patient severity and a flexible rehabilitation goal.

Approach

We propose leveraging white-box classification models such as linear classification and tree-based models for compensating joint identification, and using these models as a cost function for facilitating healthy motions or for providing force feedback via Inverse Optimal Control.

Key results and achievements

  • Design of a framework for online identification of a compensating joint.

Estimation of human arm impedance for motor behavior models

Researcher: Samuel Tesfazgi, Satoshi Endo

Motivation

Whenever humans execute a desired motor task, the produced joint torques consist of a feedforward and a feedback component. According to a theory of sensorimotor control, the feedforward component is determined a priori using internal models. During execution, deviations from the planned behavior are compensated by the feedback component, which consists of effects of muscle intrinsic viscoelastic properties, short- and long-latency reflexes, and voluntary feedback at cognitive level. As voluntary feedback possesses the longest delays, it may not be sufficient for the maintenance of stability. Thus, the central nervous system must rely on the intrinsic and reflexive feedback components, which can be modeled by a mechanical impedance consisting of inertia, damping, and stiffness. While the inertia is solely determined by limb kinematics, the latter two can be influenced by muscle activation, which enables the a priori cognitive modulation of the respective characteristics. In addition to fundamental insights into motor control, the analysis of such modulation strategies may yield models that are transferable to robotics domains such as human robot collaboration and variable impedance control.

Research questions

  • Estimation of impedance characteristics in multi-joint arm movements
  • Analysis and data-driven modeling of impedance modulation strategies
  • Transfer of models to robotics domains, e.g., human robot interaction

Approach

We have designed an experimental framework specifically tailored to meet the requirements of impedance estimation during multi-joint arm movements. The experimental apparatus is capable of inducing short, precise disturbances during otherwise free motion of the participants. As we intend to analyze transient impedance modulation strategies, we have developed a single trial feedback component isolation method. The obtained impedance characteristics are included in stochastic dynamical systems that model human motor behavior.

Key results and achievements

  • Design of a framework for multi-joint arm impedance estimation
  • Development of a single trial feedback component isolation method
     

  • H. Börner; S. Endo; S. Hirche: Estimation of Involuntary Components of Human Arm Impedance in Multi-Joint Movements via Feedback Jerk Isolation. Frontiers in Neuroscience 14, 2020, 459 mehr… BibTeX
  • J. R. Medina; H. Börner; S. Endo; S. Hirche: Impedance-based Gaussian Processes for Modeling Human Motor Behavior in Physical and Non-physical Interaction. IEEE Transactions on Biomedical Engineering 66 (9), 2019, 2499-2511 mehr… BibTeX
  • H. Börner; S. Endo; S. Hirche: Estimation of Involuntary Impedance in Multi-joint Arm Movements. IFAC Conference on Cyber-Physical & Human Systems (CPHS), 2018 mehr… BibTeX

Embodiment under autonomous control

Researcher: Satoshi Endo

Motivation 

Various forms of assistive control have been introduced to help users in operating machines as found in smart vehicles and prosthetics for example. Commonly, semi-autonomous controllers are designed to maximise the task performance to compensate for suboptimal inputs by humans, for example, due to a lack of expertise or task knowledge. However, from a user perspective, introducing semi-autonomous control directly reduces controllability of the system, and a controller which maximises the performance may not necessarily bring positive user experience. This can lead to an undesirable outcome in long-term interaction. Therefore, we research and evaluate control designs from user perspectives by studying how well the autonomous system is embodied in the "control system" of the user by adopting psychological tools and theories.

Research questions

  • Evaluating the quality of human-in-the-loop control using the psychological construct of embodiment in a haptic human-machine interaction tasks
  • Human behaviour modelling and characterisation
  • Assessment of haptic feedbacks

Approach

We have designed a haptic human-machine interaction task to evaluate the quality of human-in-the-loop control using a psychological construct of embodiment. For this task, we composed an experimental framework consisting of a haptic assistance and a virtual environment broadcasted through virtual reality goggles. With this framework, we tested the perceived embodiment under different assistive control algorithms in a reaching task with human participants.

Key results and achievements

  • Design of an experimental setup consisting of a 2-DoF haptic device and a virtual environment for human studies.
  • Investigation of subjective embodiment using psychometric scales.

  • Endo, Satoshi; Fröhner, Jakob; Music, Selma; Hirche, Sandra; Beckerle, Philipp: Effect of External Force on Agency in physical Human-Machine Interaction. Frontiers in Human Neuroscience 14, 2020 mehr… BibTeX
  • J. Fröhner; P. Beckerle; S. Endo; S. Hirche: An embodiment paradigm in evaluation of human-in-the-loop control. IFAC Conference on Cyber-Physical & Human Systems (CPHS), 2018 mehr… BibTeX