Ongoing projects
Multimodal Human Pose Forecasting
Introducing MotionMap, a heatmap-based approach for multimodal human pose forecasting that provides uncertainty estimation, controllability, and confidence measures while effectively capturing diverse and rare motion predictions.
HEADS-UP: Head-Mounted Egocentric Dataset for Trajectory Prediction in Blind Assistance Systems
HEADS-UP is the first egocentric dataset collected from head-mounted cameras, designed specifically for trajectory prediction in blind assistance systems.
HHI-Assist: A Dataset and Benchmark of Human-Human Interaction in Physical Assistance Scenario
The increasing labor shortage and aging population underscore the need for assistive robots to support human care recipients. Developing such robots requires accurate human motion prediction to ensure their responsiveness and safety. This task is challenging due to the variability in scenarios and the necessity of modeling interactions between agents. To address these challenges, we present HHI-Assist, a collection of motion capture clips of human-human interaction (HHI) for physical assistance.
Multi-Transmotion
We propose a pre-trained transformer-based model that can predict multimodal multitask human motion under different frame settings. We also create a unified human motion data framework by merging seven datasets with standardized settings.
Human Pose Forecasting
First, we develop an open-source library for human pose forecasting. Next, we model the uncertainty in the task, and then propose a denoising diffusion model to handle noisy inputs.
Social-Transmotion
We translate the idea of a prompt from Natural Language Processing (NLP) to the task of human trajectory prediction, where a prompt can be a sequence of x-y coordinates on the ground, bounding boxes or body poses.
Vehicle Trajectory Prediction
Predicting how the future will unroll is essential for a self-driving car system. We propose generalizable prediction models for this regard. The predictions are then employed in the planning pillar or for risk assessment.
Robust Trajectory Prediction
How robust are the trajectory prediction models? We study this question from different perspectives on vehicle and pedestrian trajectory prediction models.
Social-NCE
Learning socially-aware motion representations is at the core of recent advances in multi-agent problems, such as human motion forecasting and robot navigation in crowds. In this work, we introduce a contrastive learning approach built with negative data augment to promote robust motion representations.
TrajNet++: Socially-Aware Human Trajectory Forecasting
We present an in-depth analysis of existing deep learning based methods for modelling social interactions in crowds. To objectively compare the performance of these interaction-based forecasting models, we develop a large scale interaction-centric benchmark TrajNet++, a significant yet missing component in the field of human trajectory forecasting.
Pedestrian Stop and Go
Predicting whether pedestrians will stop walking (Stop) or start to walk (Go) in the near future, for better trajectory prediction around road traffic.
Pedestrian Bounding Box Prediction
A libary for predicting 2D and 3D bounding boxes of humans in autonomous driving scenarios