Motion Patterns for Multiple Object Tracking

Non-Markovian Globally Consistent Multi-Object Tracking

Many state-of-the-art approaches to multi-object tracking rely on detecting them in each frame independently, grouping detections into short but reliable trajectory segments, and then further grouping them into full trajectories. This grouping typically relies on imposing local smoothness constraints but almost never on enforcing more global ones on the trajectories.

We propose a non-Markovian approach to imposing global consistency by using behavioral patterns to guide the tracking algorithm. When used in conjunction with state-of- the-art tracking algorithms, this further increases their already good performance on multiple challenging datasets. We show significant improvements both in supervised settings where ground truth is available and behavioral patterns can be learned from it, and in completely unsupervised settings.


First video presents examples of the learned patterns.

Second video presents an example, in which tracking with motion patterns improves the quality of tracking.

Third video presents examples of the patterns learned in the absence of ground truth, by iterating between learning the patterns and tracking people.


On github


Non-Markovian Globally Consistent Multi-Object Tracking

A. Maksai; X. Wang; F. Fleuret; P. Fua 

2017. Proceedings of the IEEE International Conference on Computer Vision.