Research Interests

I am interested in developing resilient autonomy for robotic systems whose dynamics are uncertain, learned, or imperfectly modeled. My current research focuses on how learned models should be trained and evaluated for downstream decision-making, how policy optimization can exploit imperfect differentiable models, and how planning and control can remain reliable under model uncertainty.

• Learning-Based Control – Developing data-driven and learning-enabled control methods for complex robotic systems, with an emphasis on adaptation, robustness, and the interaction between learning and feedback control.
• Model Learning for Decision-Making – Studying how dynamical models should be learned, represented, and evaluated when their ultimate purpose is planning or control rather than prediction alone.
• Planning and Decision-Making under Uncertainty – Developing methods that explicitly reason about uncertain dynamics, models, and observations to enable reliable robotic behavior in partially known and changing environments.

 

Student Projects and Thesis

I am looking for motivated students interested in learning-based control, robot learning, and data-driven control. I particularly welcome inquiries for Forschungspraxis (FP) and Master’s theses (MA). Bachelor’s thesis (BA) or Ingenieurpraxis (IP) topics may occasionally be available when there is a suitable well-defined project.

Please include your preferred starting date as well as your CV and transcript of records in your e-mail. This helps me to select a topic matching your background.

Potential Topics

The following topics are possible starting points. The exact scope can be adjusted depending on the project type and the student's background.

• Decision-Aware Dynamics Learning for Control (FP/MA)
  Many learned dynamics models are trained to minimize prediction error, although prediction accuracy does not necessarily reflect how useful a model is for downstream control. This topic studies how models can instead be evaluated and trained according to their decision fidelity.
  Possible directions include decision-sensitive model evaluation, value-aware or sensitivity-weighted learning objectives, task-relevant model error metrics, and benchmarks illustrating when prediction-optimal models lead to poor control decisions. Depending on the project stage, the work may range from systematic benchmarking to developing new learning objectives.
  • Reference:
    • Farahmand, Barreto, and Nikovski, “Value-Aware Loss Function for Model-Based Reinforcement Learning,” AISTATS 2017.
    • Grimm et al., “The Value Equivalence Principle for Model-Based Reinforcement Learning,” NeurIPS 2020.
    • Claas A Voelcker, Victor Liao, Animesh Garg and Amir-massoud Farahmand, "Value Gradient weighted Model-Based Reinforcement Learning", ICLR, 2022
• Robust Differentiable Policy Optimization under Model Mismatch (FP/MA)
  Differentiable dynamics models and simulators can provide policy gradients that enable highly sample-efficient optimization, but these gradients may become unreliable when the model differs from the real system. This topic investigates when model-derived gradients remain useful and how policy optimization can be made robust to model mismatch.
  Possible directions include first-order policy gradients, residual policy learning, gradient-fidelity diagnostics, controlled model-mismatch benchmarks, robust or uncertainty-weighted gradient updates, hybrid model-based/model-free optimization, and comparisons with methods such as SHAC and PPO.
  • Reference:
    • Pan, Jiahe, et al. "Learning on the fly: Rapid policy adaptation via differentiable simulation." IEEE Robotics and Automation Letters, 2026
    • Hyung Ju Suh, Max Simchowitz, Kaiqing Zhang, Russ Tedrake, “Do Differentiable Simulators Give Better Policy Gradients?,” Proceedings of the 39th International Conference on Machine Learning, 2022.
    • Luo et al., “Residual Policy Learning for Perceptive Quadruped Control Using Differentiable Simulation,” ICRA 2025.
• Risk-Aware Planning and Control under Learned Dynamics Uncertainty (FP/MA)
  Robots operating with learned dynamics models must often make decisions before model uncertainty has been fully resolved. This topic studies how epistemic model uncertainty can be incorporated into planning and control to avoid unsafe or unreliable decisions.
  Possible directions include uncertainty propagation through predicted trajectories, risk-sensitive or chance-constrained planning, uncertainty-aware obstacle avoidance, safe fallback strategies, uncertainty-aware MPC or sampling-based control, and comparisons between different probabilistic dynamics models and planners.
  • Reference:
    • Cai, Xiaoyi, et al. "Evora: Deep evidential traversability learning for risk-aware off-road autonomy." IEEE Transactions on Robotics, 2024
    • Hewing, Kabzan, and Zeilinger, “Cautious Model Predictive Control Using Gaussian Process Regression,” IEEE TCST 2020.
    • Kabzan et al., “Learning-Based Model Predictive Control for Autonomous Racing,” IEEE RA-L 2019.
    • Hewing et al., “Learning-Based Model Predictive Control: Toward Safe Learning in Control,” Annual Review of Control, Robotics, and Autonomous Systems, 2020.

• Risk-Aware Navigation with Uncertain Terrain Perception and Learned Dynamics (FP/MA)

  Autonomous robots often rely on terrain maps inferred from imperfect visual observations while also operating with uncertain dynamics, such as unknown traction, friction, or wheel slip. Even when a terrain type is correctly identified, its physical properties may remain unknown. This topic investigates how uncertainty in terrain perception interacts with uncertainty in learned dynamics, and how both should influence planning and control decisions.
  A central question is which perception or model uncertainties actually matter for safe task execution, rather than merely for prediction accuracy.
  Possible directions include uncertainty-aware terrain perception and mapping, calibration and benchmarking of probabilistic terrain maps, joint propagation of perception and dynamics uncertainty, and risk-sensitive planning under ambiguous terrain conditions. More advanced directions may involve task-aware information acquisition, where a robot decides whether to obtain additional visual observations, physically probe uncertain terrain, or proceed with its task.
  Depending on the project scope, the work may range from integrating existing perception methods and evaluating their impact on closed-loop navigation to developing new uncertainty-aware planning or information-acquisition methods.

  References:

  • Cai et al., “EVORA: Deep Evidential Traversability Learning for Risk-Aware Off-Road Autonomy,” IEEE Transactions on Robotics, 2024.
  • “Perception-Aware Model Predictive Control for Constrained Control in Unknown Environments,” Automatica, 2024.
  • “Safe Planning Using Conformalized Semantic Maps,” IEEE RA-L, 2026.

Desired Background

Useful background depending on the topic: control theory, numerical optimization, reinforcement learning, probabilistic machine learning, deep learning, Python/JAX/PyTorch, or robotics. You do not need to have experience in all of these areas.