Dual-Arm Manipulation with Optimization-Based Whole-Body Control and Online Priority Adaptation

Reactive dual-arm manipulation in dynamic and constrained environments is a challenging control problem due to tight geometric constraints, kinematic redundancy, and frequent physical interaction with the environment. Modern approaches increasingly rely on whole-body control formulations that must simultaneously address motion generation, collision avoidance, and multiple task-space objectives.

Recent work [1] proposed a predictive multi-agent planning and landing controller that tightly couples vector-field-based planning with a closed-form cooperative set-based task-priority (CoSTP) controller, achieving robust real-time performance in complex dual-arm manipulation scenarios. While this framework demonstrates strong empirical results, its control layer is based on analytical task-priority constructions involving null-space projections and switching logic, which introduces several limitations:

• Inequality constraints (e.g., joint limits and workspace boundaries) are handled implicitly through set-based switching rather than being enforced explicitly.
• Task prioritization is hard-coded, making it difficult to generalize, tune, or adapt online.
• Incorporating additional whole-body constraints, such as explicit joint-space obstacle avoidance, requires nontrivial analytical derivations.

In contrast, quadratic-programming–based whole-body controllers offer several advantages:

• Explicit and unified handling of equality and inequality constraints.
• Natural integration of whole-body constraints, including collision avoidance.
• Online and continuous adaptation of task and constraint priorities through weighting or hierarchical formulations.

This project investigates whether an optimization-based whole-body controller can reproduce the reactive behaviors of the original CoSTP-based framework in simulation, while enabling explicit constraint handling and online priority adaptation. Furthermore, the project explores the practical trade-offs between analytical task-priority control and QP-based formulations in reactive dual-arm manipulation.

Project Goals

Specifically, the student will:

1. Study and understand the predictive multi-agent planner and the cooperative set-based task-priority (CoSTP) controller proposed in [1].
2. Reproduce the original control pipeline in a simulation environment using the provided code.
3. Replace the original task-priority controller with a QP-based whole-body controller while preserving the same planning and task definitions.
4. Evaluate and compare the performance of the two control approaches in simulation.

The goal is to develop a deep understanding of modern reactive manipulation controllers and to investigate how quadratic-programming-based formulations handle multi-task and whole-body constraints compared to closed-form task-priority methods.

Expectations

• Strong motivation to read and understand a technically demanding robotics paper.
• Solid programming skills and the ability to implement and debug simulation-based control systems.
• Clean, well-documented code and a clear, well-structured technical report.

Reference

[1] R. Laha, et al., “Predictive Multi-Agent-Based Planning and Landing Controller for Reactive Dual-Arm Manipulation,” IEEE Transactions on Robotics, vol. 40, pp. 864–885, 2024, doi: 10.1109/TRO.2023.3341689.