Learning Monocular Reconstruction from Multi-view Images

Accurate 3D human pose estimation from single images is possible with sophisticated deep-net architectures that have been trained on very large datasets. However, this still leaves open the problem of capturing motions for which no such database exists. Manual annotation is tedious, slow, and error-prone.

In this project, we propose to replace most of the annotations by the use of multiple views, at training time only. We propose the following two alternative methods.

Learning Monocular 3D Human Pose Estimation from Multi-view Images

In this pose-centered approach, we train the system to predict the same pose articulated pose in all available views. Such a consistency constraint is necessary but not sufficient to predict accurate poses. We therefore complement it with a supervised loss aiming to predict the correct pose in a small set of labeled images, and with a regularization term that penalizes drift from initial predictions. Furthermore, we propose a method to estimate camera pose jointly with human pose, which lets us utilize multi-view footage where calibration is difficult, e.g., for pan-tilt or moving handheld cameras. We demonstrate the effectiveness of our approach on established benchmarks, as well as on a new Ski dataset with rotating cameras and expert ski motion, for which annotations are truly hard to obtain. Learning an unsupervised representation

The new Ski-pose PTZ-camera dataset is available here: Ski-PosePTZ-Dataset.
Pre-print: https://arxiv.org/abs/1803.04775

Unsupervised Geometry-Aware Representation for 3D Human Pose Estimation

Learning an unsupervised representation

In this approach, we propose to overcome remaining problems by learning a geometry-aware body representation from multi-view images without any 3D annotations. To this end, we use an encoder-decoder that predicts an image from one viewpoint given an image from another viewpoint. Because this representation encodes 3D geometry, using it in a semi-supervised setting makes it easier to learn a mapping from it to 3D human pose. As evidenced by our experiments, our approach significantly outperforms fully-supervised methods given the same amount of labeled data, and improves over the first semi-supervised method while using as little as 1% of the labeled data.

Pre-print: https://arxiv.org/abs/1804.01110
Pytorch network definition and training code: github.com/hrhodin/

References

Église castrale d’Akerentia. Étude archéologique d’un ensemble monumental au cœur de la Calabre

A. A-L. Terrier 

Lausanne: EPFL Presses, 2025.

Ukrainian Literary Imaginaries of Past after 1991: From Substitution to Restoration?

A. Dmitriev 

The Routledge Companion to Literature and Crisis; London: Routledge, 2024-12-08. p. 470.

Советская городская память в юбилейных форматах [Soviet urban memory in anniversary formats (late 1930s – late 1980s of the XX century)]

A. Dmitriev 

HistoriCity: городские исследования и история современности [HistoriCity: urban research and contemporary history ; Moscow: Novoe Literaturnoe Obozrenie, 2024-09-17.

«Novye idei» i disciplinarnye tradicii: perevodcheskie i publikatorskie strategii izdatel’stva «Obrazovanie» v 1910–1920-e gody

A. Dmitriev 

KOYNE Almanach.. 2024-07-16. Vol. 2024, num. N 2. .

Experimental and ab initio derivation of interface stress in nanomultilayered coatings: Application to immiscible Cu/W system with variable in-plane stress

G. Lorenzin; J. F. Troncoso; M. Liyanage; A. Druzhinin; L. P. H. Jeurgens et al. 

Applied Surface Science. 2024-07-15. Vol. 661, p. 159994. DOI : 10.1016/j.apsusc.2024.159994.

Energy-Efficient Frequency Selection Method for Bio-Signal Acquisition in AI/ML Wearables

H. Taji; J. A. Miranda Calero; M. Peon Quiros; D. Atienza Alonso 

2024-07-04. ACM/IEEE International Symposium on Low Power Electronics and Design (ISLPED ’24), Newport Beach, CA, USA, August 5–7, 2024. DOI : 10.1145/3665314.3670815.

Wearable Activity Trackers: A Survey on Utility, Privacy, and Security

K. S. Niksirat; L. Velykoivanenko; N. Zufferey; M. Cherubini; K. Huguenin et al. 

Acm Computing Surveys. 2024-07-01. Vol. 56, num. 7, p. 183. DOI : 10.1145/3645091.

Quantitative convergence rates for scaling limit of SPDEs with transport noise

F. Flandoli; L. Galeati; D. Luo 

Journal Of Differential Equations. 2024-06-15. Vol. 394, p. 237-277. DOI : 10.1016/j.jde.2024.02.053.

Separation of kinetic rate orders in extreme ultraviolet transient grating spectroscopy

H. J. B. Marroux; S. Polishchuk; O. Cannelli; R. A. Ingle; G. F. Mancini et al. 

Journal Of Physics B-Atomic Molecular And Optical Physics. 2024-06-14. Vol. 57, num. 11, p. 115401. DOI : 10.1088/1361-6455/ad421f.

Coronal jets identification using Deep Learning as Image and Video Object Detection

A. Joliat 

2024-06-12.