Injecting Knowledge in Data-driven Vehicle Trajectory Predictors

Mohammadhossein Bahari, Ismail Nejjar, Alexandre Alahi

Transportation research part C: emerging technologies, 2021

Illustration of our Realistic Residual Block (RRB) model. The knowledge-based model generates a scene-compliant trajectory (yellow dots) which could not effectively account for other agents, hence, is too conservative. Our data-driven RRB improves the prediction (purple dots) by adding confined residuals conditioned on other agents in the scene (illustrated by the arrows). The blue region shows the physically-constrained output space for our RRB predictions.

Paper      Webpage     Code

Abstract: Vehicle trajectory prediction tasks have been commonly tackled from two distinct perspectives: either with knowledge-driven methods or more recently with data-driven ones. On the one hand, we can explicitly implement domain-knowledge or physical priors such as anticipating that vehicles will follow the middle of the roads. While this perspective leads to feasible outputs, it has limited performance due to the difficulty to hand-craft complex interactions in urban environments. On the other hand, recent works use data-driven approaches which can learn complex interactions from the data leading to superior performance. However, generalization, i.e., having accurate predictions on unseen data, is an issue leading to unrealistic outputs. In this paper, we propose to learn a “Realistic Residual Block” (RRB), which effectively connects these two perspectives. Our RRB takes any off-the-shelf knowledge-driven model and finds the required residuals to add to the knowledge-aware trajectory. Our proposed method outputs realistic predictions by confining the residual range and taking into account its uncertainty. We also constrain our output with Model Predictive Control (MPC) to satisfy kinematic constraints. Using a publicly available dataset, we show that our method outperforms previous works in terms of accuracy and generalization to new scenes.


title = {Injecting knowledge in data-driven vehicle trajectory predictors},
journal = {Transportation Research Part C: Emerging Technologies},
volume = {128},
pages = {103010},
year = {2021},
issn = {0968-090X},
doi = {},
url = {},
author = {Mohammadhossein Bahari and Ismail Nejjar and Alexandre Alahi},