Outline
This project aims to develop and evaluate reinforcement learning (RL) policies that are robustly transferable from simulation to the real world for the NVIDIA JetBot, using techniques such as domain randomization, domain adaptation, and hybrid sim-real fine-tuning.
Background
Deep RL has demonstrated remarkable success in robotic control tasks. However, policies trained in simulation often degrade significantly in real-world deployment due to the *sim-to-real gap* — mismatches in sensor noise, dynamics, lighting, and environmental textures. For low-cost mobile robots like the JetBot, collecting extensive real-world training data is expensive and time-consuming, making simulation-based training attractive.Domain randomization [1,3] and adaptation methods [2] have emerged as promising strategies to mitigate the sim-to-real gap by training on varied simulated conditions or aligning feature distributions between sim and real domains. This project will implement and compare these techniques for JetBot navigation in obstacle-rich environments, with the goal of maximizing zero-shot transfer performance and minimizing post-transfer fine-tuning requirements.
Milestones
- M1 (Weeks 1–2): Literature review; set up JetBot simulation in Isaac Sim or Gazebo with ROS integration.
- M2 (Weeks 3–6): Implement and train navigation policies in simulation using domain randomization.
- M3 (Weeks 7–10): Apply domain adaptation techniques (e.g., adversarial feature alignment) and evaluate in sim.
- M4 (Weeks 11–13): Deploy policies to real JetBot, analyze performance gaps, and apply minimal real-world fine-tuning.
- M5 (Weeks 14–16): Final evaluation, results analysis, and report writing.
Requirements
We look for motivated students with a strong background in mathematics, dynamical systems and control, and machine learning. Familiarity with reinforcement learning, PyTorch, and ROS is highly recommended. If interested, please send a short paragraph on your background and fit for the project along with your BS and MS transcripts to: [email protected], [email protected].
References
1. Tobin, Josh, et al. Domain randomization for transferring deep neural networks from simulation to the real world. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017.
2. Bousmalis, Konstantinos, et al. Using simulation and domain adaptation to improve efficiency of deep robotic grasping. 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2018.
3. Chen, Xiaoyu, et al. Understanding domain randomization for sim-to-real transfer. arXiv preprint arXiv:2110.03239 (2021).