ICRA 2019 Workshop

Bringing perception-based manipulation to the real world: standardizing robot manipulation learning

Manipulation is the key skill to a ffect, interact with and learn from the environment. In recent years we have seen many successful learning approaches in a variety of manipulation tasks, relying on di fferent robotic arms, ende ffectors, perception systems and computational resources. However, the heterogeneity of these setups is hindering researchers from focusing their e fforts on the key open issues and is preventing them from performing objective comparisons. This workshop aims to specify a reference robot manipulation platform that is capable of addressing the most relevant open problems in robot manipulation learning. This field consists of countless subproblems and defining a common platform for them is a significant challenge in its own right. However, it is a first step towards having a basis for comparable research in the field of perception-based manipulation learning. It will involve participants from academia and industry in order to jointly defi ne these problems and the necessary specications that a prototypical robotic setup requires in order to solve them.

This workshop aims for the definition of a reference platform to evaluate perception-based robot manipulation learning in a standardized manner. As a community-driven effort, we will send an call for platforms open to any hardware provider or robotics developers who would like to contribute. The call will specify the requirements of the platform as defined by the organizers of the workshop. The tentative timeline until the workshop is the following:

  • 1st of March: Internal feedback and suggestions for general framework for benchmark platform
  • 10th of March: Internal feedback and suggestions for concrete experimental setups for benchmark tasks
  • 15th of March: Open call for platforms
  • 3rd of May: Deadline for platform submissions
  • 10th of May: Submission reviews and notifications
  • 23rd of May: Workshop at ICRA


Robot manipulation learning has become one of the key topics in robotics research. Researchers mostly focus on algorithmic aspects of robot learning, which are then evaluated in very specific, maybe even unique and nowhere else accessible robotic platforms. Although the dependency on specific robot hardware has been neglected, its influence on results should not be understated. At the same time, this hardware dependency prevents researchers from performing objective, unbiased and effortless comparisons between algorithms. This workshop aims for the definition of an \emph{accessible and high-performance robotic setup} capable of addressing the most relevant open problems in robot manipulation learning and that, at the same time, serves as a benchmarking platform. To this end, renowned experts in the field will share their experience with different hardware platforms, including manipulators, end-effectors, tactile/force sensors, vision system and computational resources, and their interplay and dependencies with robot learning algorithms. Instrumental for the workshop's goal is the final discussion, which will first jointly define the most relevant set of problems and their respective performance measures followed by a second session targeting specific hardware specifications. This session will be complemented with the opinion of invited hardware system manufacturers, that will provide their own view on the problem and the feasibility of potential platforms based on currently available products.


Time Talk


Introduction by Tamim Asfoour

Session 1 Chair: Tamim Asfour

Sami Haddadin - Platform Requirements and Reference Skills for
High-Performance Robot Control

9:30-9:50 Gregory Hager - Machine Learning and Perception-Guided Manipulation:
Datasets Necessary, but not Sufficient
9:50-10:10 Dieter Fox - Benchmarking Object Manipulation in Articulated Environments
Platform Spotlights I  
10:10-10:25 Shadow
10:25-10:40 Roboception
10:40-10:55 Franka Emika
Coffee break  
Session 2 Chair: Dieter Fox
11:10-11:30 TBD
11:30-11:50 Aude Billard - Platform Needs for Teaching by Demonstration and Fine Manipulation
11:50-12:10 Oliver Brock - A Plea for Diversity: Balancing Problem-Solving and Standardization
12:10-12:30 Tamim Asfour - Reference Robotic Platforms for Manipulation Learning
Lunch break  
Session 3 Chair: Tamim Asfour
Platform Spotlights II  
13:15-13:30 Kinova
13:30-13:45 Schunk
13:45-14:00 PAL Robotics
14:00-14:20 Patrick van der Smagt - A Machine-Learner's View on Tactile Manipulation
14:20-14:40 Wolfram Burgard - TBD

Matei Ciocarlie - Case Studies in Hand Sensors: A Tactile Finger,
a Proprioceptive Controller, and a Highly Sensorized Hand

15:00-15:20 Oussama Khatib - TBD
Coffee break  
16:00-17:00 Discussion Panel: Problems and Benchmark Systems (Chair: Patrick van der Smagt)
17:00-18:00 Discussion Panel: Platform Specifications (Chair: Aude Billard)


We will advertise the workshop in the robotics mailing lists the mailing lists of the mentioned technical committees. In addition, we will also contact key industrial players to join the workshop, participate in the discussion panel and expose their solutions in the industrial spotlight session and the poster session during the coffee breaks. Participants will be able to complement and support the discussion session with the presented results.

The workshop's main goal is the final discussion panel to figure out how a prototypical setup for robot manipulation learning should be, what should it do and how robot manipulation learning should be benchmarked. The impact of the workshop's outcome will only be significant with the input and support of the whole robot learning community. This includes not only the knowledge and experience of world-renowned experts but also the hands-on experience of early career researchers who spend days directly interacting with the hardware and hardware manufacturers. The planned poster sessions and poster preview session will be the first step to encourage the interaction between these two groups. The final discussion session will encourage an open and fruitful discussion among all participants. The two one hour sessions will specifically provide slots for experts, early career researchers and industrial speakers in order to consider all necessary opinions for a real community-driven outcome.

This workshop will publish, if speakers agree, all presented slides on the workshop's website. The outcome of the discussion session of the workshop will be summarized in a document and published on the workshop's website. Additionally, the workshop organizers will aim for a joint publication with all invited speakers where the resulting chosen problems and the platform specifications and benchmarks are formally described. This will serve as a reference sheet to deeply understand and also fully exploit the capabilities of the platform. After the workshop, the organizers will host a community-maintained website to keep track of all experiments, publicly available code, benchmarks and publications done in the chosen prototypical setup.

The organizers of the workshop invite all interested partners, companies and institutions to participate in this workshop by submitting a proposal for a standard manipulation learning platform.

Submission instructions

Please refer to the following steps when submitting a platform proposal.

  1. Download the full call here and the platform specifications and task descriptions here.
  2. Prepare your proposal as follows:
    1. Maximum one page of description including figures.
    2. Attach as many appendixes as you like. Appendices can be datasheets, videos or lists of references.
    3. There is no template required.
  3. Submit your proposal to icra19workshop@msrm.tum.de. Please prepare your documents as single PDF file. All other appendixes such as videos may be either sent in the same mail (if the size allows this) or provided as an external download (e.g. via Dropbox).

The deadline for submissions is the 3rd of May. The acceptance notifications will be send out approximately on the 10th of May. The selected platforms will be invited to present their systems at ICRA.

If you have questions feel free to send an email icra19workshop@msrm.tum.de.