Motivation
Learning a control policy for dexterous manipulation is hard: a 20-DoF robotic hand together with a 7-DoF robotic arm form a 27-dimensional action space, and exploring it directly with reinforcement or imitation learning can be intractable [1]. The human motor system does not control each finger joint independently, and hand joint positions are well described by a small number of coordinated movements, or synergies [2]. This suggests that coupling joint motions into a low dimensional basis, and not actuating every joint independently, could make learning substantially easier. We want to analyze how much an action space reduction helps, and what are its trade-offs.
Outline
This project aims to study the impact of action space reduction on learning manipulation with a fully actuated hand. The goal is to collect grasping data, compute a low dimensional hand synergy basis (eigengrasps) via principal component analysis [3], and use it to reduce the hand’s action space: instead of commanding all joints, the policy outputs a few coefficients from which joint positions are reconstructed. Policies (reinforcement learning / imitation learning) are then trained on the reduced action space and compared against a baseline. Can eigengrasps maintain (or even improve) performance, while improving convergence and sample efficiency? We evaluate task success rate and reward scores as the main metric. If time permits, the project can be performed on real hardware.
Milestones
- M1 (Weeks 1-3): Setup and collect a grasping dataset. Extract the eigengrasp basis via PCA.
- M2 (Weeks 4-7): Implement the reduced action space and integrate it with the learning framework.
- M3 (Weeks 8-11): Train policies for varying numbers of components, and compare against the baselines.
- M4 (Weeks 12-14): Evaluate the performance trade-offs, deploy on real hardware.
Requirements
- Highly motivated for the topic
- Programming experience (Python)
- Knowledge of Machine Learning / Reinforcement Learning
- Strong mathematical background
- Knowledge of robotics is a plus
- Familiarity with ROS is a plus
Contact
Send an e-mail to Gregorio Valenti ([email protected]). To apply, please include a short motivation for this specific project, your CV, and your BSc/MSc transcripts.
Preferences: Between September and December 2026
References
[1] Rajeswaran, Aravind, et al. “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations.” Robotics: Science and Systems (RSS), 2018.
[2] Santello, Marco, Martha Flanders, and John F. Soechting. “Postural hand synergies for tool use.” Journal of Neuroscience 18.23 (1998): 10105-10115.
[3] Ciocarlie, Matei, Corey Goldfeder, and Peter Allen. “Dexterous grasping via eigengrasps: A low-dimensional approach to a high-complexity problem.” Robotics: Science and Systems Manipulation Workshop, 2007.