3D Pose Based Motion Correction for Physical Exercises

With the rise of self-management for treatment of musculoskeletal disorders, people tend to exercise alone and without supervision. However it is dangerous to attempt these exercises without feedback, as it can be difficult to realize when one is performing the exercise incorrectly. This could lead to further injury. Therefore our goal is to design a self-feedback system with the help of deep learning methods.

Making use of recent progresses in fields of pose estimation, action recognition and motion prediction, we have begun to analyze movements in detail and provide feedback as a corrected version of the performed exercise[1]. We have also gathered a small dataset containing videos, 2D and 3D poses of correct and incorrect executions of different movements that are SQUATS, lunges, and planks and labels identifying the mistake in each practice of that exercise. This dataset is used to demonstrate our motion correction model. We have also designed a graph convolutional network architecture based on the work of [2] in order to output the corrected version of the exercise.

However, there are many short comings that still need to be addressed:

  • Due to the dataset being small and containing few subjects, our model tends to overfit to the training subjects’ mistakes and correct performances. In reality, there may be more than one correct method of performing the exercises. Therefore, an expansion of the dataset is necessary.
  • The framework is not adequattely studied and many improvements remain to be made. We would like to focus on architecture design and training strategies that give more accurate motion corrections.

The goal of this project therefore is to first expand the physical exercise dataset to include more subjects and actions, and to improve upon the existing framework for more accurate exercise corrections. The project will be jointly supervised by Sena Kiciroglu and Isinsu Katircioglu.

References:
[1] Vinzant et al., “3D Pose Based Motion Correction for Physical Exercises”, EPFL Masters Thesis 2021.
[2] Mao et al., “Learning Trajectory Dependencies for Human Motion Prediction”, ICCV 2019.

Misc

The candidate should have Python programming experience. Previous experience with deep learning, particularly using PyTorch, is a plus.

30% Theory and Report
30% Dataset Acquisition
40% Research and Experiments

Contact

For further information, send an e-mail. Contacts: