Motion Models for Monocular 3–D Human Body Tracking
Click on the images to jump to some results. |
We investigated the use of motion models based on either Principal Component Analysis or Gaussian Processes to formulate the monocular tracking problem as one of minimizing differentiable objective functions. As shown in the results below, we can handle cyclic motions, such as walking and running, as well as acyclical ones, such as a golf swing.
Furthermore, by detecting key postures, we can achieve robust and fully automated 3D motion recovery of people seen from arbitrary viewpoints by a single and potentially moving camera.
Results
Golf Swing #1 [top]
By tracking the feet, knees, hands, and head of the golfer in the video sequence and fitting a parameterized model of a golf swing to the resulting trajectories, we can recover the 3D motion of the skeleton.
On the left we see the 3D skeleton reprojected onto the original video sequence and on the right a 3D avatar that performs the same motion as the real golfer. The hands were tracked using a method we developed in earlier work.
Golf Swing #2 [top]
The same procedure is applied to a long swing, as opposed to the short swing shown above.
Pedestrian Seen from Arbitrary Viewpoints [top]
The algorithm automatically detects the pedestrian at the moment of the walking cycle when his legs are furthest apart and fits a walking motion model in between detections. As a result the subject can be tracked from an arbitrary viewpoint, his full 3D motion recovered, and his instantaneous speed estimated.
Code
No code available.
Materials
References
Field Guide to Northern Tree-related Microhabitats: Descriptions and size limits for their inventory in boreal and hemiboreal forests of Europe and North America
Swiss Federal Institute for Forest, Snow and Landscape Research WSL, Switzerland, 2024.Data Champions Lunch Talks – Green Bytes: Data-Driven Approaches to EPFL Sustainability
Data Champions Lunch Talks – Green Bytes: Data-Driven Approaches to EPFL Sustainability, EPFL, CM 1 221, April 18, 2024.Comparison of Three Viral Nucleic Acid Preamplification Pipelines for Sewage Viral Metagenomics
Food and Environmental Virology. 2024. DOI : 10.1007/s12560-024-09594-3.How to Support Students to Develop Skills that Promote Sustainability
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.How to Support Students Giving Each Other Constructive Feedback, Especially When It Is Difficult to Hear
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.How teachers can use the 3T PLAY trident framework to design an activity that develops transversal skills
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.The conceptual foundations of innate immunity: Taking stock 30 years later
Immunity. 2024-04-09. Vol. 57, num. 4, p. 613-631. DOI : 10.1016/j.immuni.2024.03.007.Radio-Activities: Architecture and Broadcasting in Cold War Berlin
Cambridge, MA; London: MIT Press, 2024.No Last One
Revue Matières. 2024. num. 18.All That is Solid
Transcalar Prospects in Climate Crisis; Zurich: Lars Müller, 2024.Field Guide to Northern Tree-related Microhabitats: Descriptions and size limits for their inventory in boreal and hemiboreal forests of Europe and North America
Swiss Federal Institute for Forest, Snow and Landscape Research WSL, Switzerland, 2024.Data Champions Lunch Talks – Green Bytes: Data-Driven Approaches to EPFL Sustainability
Data Champions Lunch Talks – Green Bytes: Data-Driven Approaches to EPFL Sustainability, EPFL, CM 1 221, April 18, 2024.Comparison of Three Viral Nucleic Acid Preamplification Pipelines for Sewage Viral Metagenomics
Food and Environmental Virology. 2024. DOI : 10.1007/s12560-024-09594-3.How to Support Students to Develop Skills that Promote Sustainability
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.How to Support Students Giving Each Other Constructive Feedback, Especially When It Is Difficult to Hear
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.How teachers can use the 3T PLAY trident framework to design an activity that develops transversal skills
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.The conceptual foundations of innate immunity: Taking stock 30 years later
Immunity. 2024-04-09. Vol. 57, num. 4, p. 613-631. DOI : 10.1016/j.immuni.2024.03.007.Radio-Activities: Architecture and Broadcasting in Cold War Berlin
Cambridge, MA; London: MIT Press, 2024.No Last One
Revue Matières. 2024. num. 18.All That is Solid
Transcalar Prospects in Climate Crisis; Zurich: Lars Müller, 2024.Field Guide to Northern Tree-related Microhabitats: Descriptions and size limits for their inventory in boreal and hemiboreal forests of Europe and North America
Swiss Federal Institute for Forest, Snow and Landscape Research WSL, Switzerland, 2024.Data Champions Lunch Talks – Green Bytes: Data-Driven Approaches to EPFL Sustainability
Data Champions Lunch Talks – Green Bytes: Data-Driven Approaches to EPFL Sustainability, EPFL, CM 1 221, April 18, 2024.Comparison of Three Viral Nucleic Acid Preamplification Pipelines for Sewage Viral Metagenomics
Food and Environmental Virology. 2024. DOI : 10.1007/s12560-024-09594-3.How to Support Students to Develop Skills that Promote Sustainability
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.How to Support Students Giving Each Other Constructive Feedback, Especially When It Is Difficult to Hear
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.How teachers can use the 3T PLAY trident framework to design an activity that develops transversal skills
Teaching Transversal Skills for Engineering Studens: A Practical Handbook of Activities with Tangibles; EPFL, 2024.The conceptual foundations of innate immunity: Taking stock 30 years later
Immunity. 2024-04-09. Vol. 57, num. 4, p. 613-631. DOI : 10.1016/j.immuni.2024.03.007.Radio-Activities: Architecture and Broadcasting in Cold War Berlin
Cambridge, MA; London: MIT Press, 2024.No Last One
Revue Matières. 2024. num. 18.All That is Solid
Transcalar Prospects in Climate Crisis; Zurich: Lars Müller, 2024.Contacts
Pascal Fua | [e-mail] |
Raquel Urtasun | [e-mail] |
Andrea Fossati | [e-mail] |