Doctoral Research Seminar on "Robust Human Motion Forecasting using Transformer-based Model"

The Doctoral Research Seminar on November 28th will deal with the topic of "Robust Human Motion Forecasting using Transformer-based Model". It will be an online talk by Esteve Valls Mascaro, TU Wien.


Comprehending human motion is a fundamental challenge for developing Human-Robot Collaborative applica- tions. Computer vision researchers have addressed this field by only focusing on reducing error in predictions, but not taking into account the requirements to facilitate its implementation in robots. In this paper, we propose a new model based on Transformer that simultaneously deals with the real time 3D human motion forecasting in the short and long term. Our 2- Channel Transformer (2CH-TR) is able to efficiently exploit the spatio-temporal information of a shortly observed sequence (400ms) and generates a competitive accuracy against the current state-of-the-art. 2CH-TR stands out for the efficient performance of the Transformer, being lighter and faster than its competitors. In addition, our model is tested in conditions where human motion is severely occluded, demonstrating its robustness in reconstructing and predicting 3D human motion in a highly noisy environment. Our experiment results show that the proposed 2CH-TR outperforms the ST-Transformer, which is another state-of-the-art model based on the Transformer, in terms of reconstruction and prediction under the same condi- tions of input prefix. Our model reduces in 8.89% the mean squared error of ST-Transformer in short-term prediction, and 2.57% in long-term prediction in Human3.6M dataset with 400ms input prefix.

The talk will be online, starting at 11h. In case you wish to attend, please get int touch with Katrin Schulleri.