APEX-MR: Multi-Robot Asynchronous Planning
and Execution for Cooperative Assembly

Compared to a single-robot workstation, a multi-robot system offers several advantages: 1) it expands the system’s workspace, 2) improves task efficiency, and more importantly, 3) enables robots to achieve significantly more complex and dexterous tasks, such as cooperative assembly. However, coordinating the tasks and motions of multiple robots is challenging due to issues, e.g., system uncertainty, task efficiency, algorithm scalability, and safety concerns. To address these challenges, this paper studies multi-robot coordination and proposes APEX-MR, an asynchronous planning and execution framework designed to safely and efficiently coordinate multiple robots to achieve cooperative assembly, e.g., LEGO assembly. In particular, APEX-MR provides a systematic approach to post-process multi-robot tasks and motion plans to enable robust asynchronous execution under uncertainty. Experimental results demonstrate that APEX-MR can significantly speed up the execution time of many long-horizon LEGO assembly tasks by 48% compared to sequential planning and 36% compared to synchronous planning on average. To further demonstrate the performance, we deploy APEX-MR to a dual-arm system to perform physical LEGO assembly. To our knowledge, this is the first robotic system capable of performing customized LEGO assembly using commercial LEGO bricks. The experiment results demonstrate that the dual-arm system, with APEX-MR, can safely coordinate robot motions, efficiently collaborate, and construct complex LEGO structures.

Given an assembly plan and the robot skill library (detailed below), APEX-MR distributes the tasks to different robots and generates a sequential task plan. In particular, it formulates the task distribution as an integer-linear programming (ILP) to maximize efficiency. To enable seamless collaboration, APEX-MR extends the Temporal Plan Graph (TPG) to multiple robots. It post-processes the sequential task plan and generates an asynchronous execution plan. More importantly, the asynchronous execution plan accommodates execution delays and contingencies to ensure robust and safe bimanual manipulation under uncertainty. APEX-MR significantly improves task efficiency, scales to large-scale/long-horizon tasks, is robust against uncertainty, and ensures collaboration safety.

Robots need different skills to perform different tasks, and we assume the required skills are pre-learned. A skill can be a learned policy, a parameterized motion, or even a simple trajectory. APEX-MR selects appropriate skills from the library and coordinates the robots to accomplish the task as shown above. Specifically in Lego assembly, we learn the following robot skills.

The TPG post-processing and shortcutting in APEX-MR significantly reduces makespan by 48% and wait time by 85% on average compared to the initial sequential motion plan at the horizontal dashed line. Compared to the synchronized motion plan, our asynchronous plans from APEX-MR are still consistently shorter and have less wait time. We show a comparison of the sequential, synchronous, and APEX-MR motion (with shortcut) plan in simulation below.

We deploy APEX-MR to a dual-industrial-arm system and demonstrate the first robotic system that can accomplish customized, delicate assemblies using commercial LEGO in REAL. All demo videos are 16x.