Publications by Helge Jochen Rhodin

Temporal Representation Learning on Monocular Videos for 3D Human Pose Estimation

Pascal Fua, Mathieu Salzmann, Victor Constantin, Helge Jochen Rhodin, Sina Honari

In this paper we propose an unsupervised feature extraction method to capture temporal information on monocular videos, where we detect and encode subject of interest in each frame and leverage contrastive self-supervised (CSS) learning to extract rich lat ...

2022

Self-supervised Segmentation via Background Inpainting

Pascal Fua, Mathieu Salzmann, Victor Constantin, Isinsu Katircioglu, Helge Jochen Rhodin

While supervised object detection and segmentation methods achieve impressive accuracy, they generalize poorly to images whose appearance significantly differs from the data they have been trained on. To address this when annotating data is prohibitively e ...

2022

LiftPose3D, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals

Pascal Fua, Pavan P Ramdya, Helge Jochen Rhodin, Semih Günel, Victor Lobato Rios, Daniel Eduardo Morales Garza, Marco Pietro Abrate, Adám Gosztolai

LiftPose3D infers three-dimensional poses from two-dimensional data or from limited three-dimensional data. The approach is illustrated for videos of behaving Drosophila, mice, rats and macaques. ...

NATURE PORTFOLIO2021

Human Detection and Segmentation via Multi-view Consensus

Pascal Fua, Mathieu Salzmann, Isinsu Katircioglu, Helge Jochen Rhodin

Self-supervised detection and segmentation of foreground objects aims for accuracy without annotated training data. However, existing approaches predominantly rely on restrictive assumptions on appearance and motion. For scenes with dynamic activities and ...

IEEE2021

PCLs: Geometry-aware Neural Reconstruction of 3D Pose with Perspective Crop Layers

Mathieu Salzmann, Helge Jochen Rhodin

Local processing is an essential feature of CNNs and other neural network architectures - it is one of the reasons why they work so well on images where relevant information is, to a large extent, local. However, perspective effects stemming from the proje ...

2021

Deformation-aware Unpaired Image Translation for Pose Estimation on Laboratory Animals

Pascal Fua, Pavan P Ramdya, Helge Jochen Rhodin, Semih Günel, Mirela Ostrek, Siyuan Li

Our goal is to capture the pose of neuroscience model organisms, without using any manual supervision, to be able to study how neural circuits orchestrate behaviour. Human pose estimation attains remarkable accuracy when trained on real or simulated datase ...

2020

XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera

Pascal Fua, Gerard Pons Moll, Helge Jochen Rhodin

We present a real-time approach for multi-person 3D motion capture at over 30 fps using a single RGB camera. It operates successfully in generic scenes which may contain occlusions by objects and by other people. Our method operates in subsequent stages. T ...

2020

ActiveMoCap: Optimized Viewpoint Selection for Active Human Motion Capture

Pascal Fua, Mathieu Salzmann, Sudipta Sinha, Sena Kiciroglu, Helge Jochen Rhodin

The accuracy of monocular 3D human pose estimation depends on the viewpoint from which the image is captured. While freely moving cameras, such as on drones, provide control over this viewpoint, automatically positioning them at the location which will yie ...

IEEE2020

Mo(2)Cap(2): Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera

Pascal Fua, Helge Jochen Rhodin

We propose the first real-time system for the egocentric estimation of 3D human body pose in a wide range of unconstrained everyday activities. This setting has a unique set of challenges, such as mobility of the hardware setup, and robustness to long capt ...

2019

Neural Scene Decomposition for Multi-Person Motion Capture

Pascal Fua, Mathieu Salzmann, Victor Constantin, Isinsu Katircioglu, Helge Jochen Rhodin

Learning general image representations has proven key to the success of many computer vision tasks. For example, many approaches to image understanding problems rely on deep networks that were initially trained on ImageNet, mostly because the learned featu ...

IEEE2019

Gravity as a Reference for Estimating a Person's Height from Video

Pascal Fua, Didier Bieler, Helge Jochen Rhodin, Semih Günel

Estimating the metric height of a person from monocular imagery without additional assumptions is ill-posed. Existing solutions either require manual calibration of ground plane and camera geometry, special cameras, or reference objects of known size. We f ...

IEEE2019

What Face and Body Shapes Can Tell Us About Height

Pascal Fua, Helge Jochen Rhodin, Semih Günel

Recovering a person's height from a single image is important for virtual garment fitting, autonomous driving and surveillance. However, it is also very challenging without absolute scale information. Here, we examine the rarely addressed case, where camer ...

IEEE COMPUTER SOC2019

DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila

Pascal Fua, Pavan P Ramdya, Helge Jochen Rhodin, Semih Günel, Daniel Eduardo Morales Garza, João Henrique Campagnolo

Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in 3-dimensional (3D) space. Deep neural networks can estimate 2-dimensional (2D) pose in freely behaving and tethered animals. Ho ...

2019

Are Existing Monocular Computer Vision-Based 3D Motion Capture Approaches Ready for Deployment? A Methodological Study on the Example of Alpine Skiing

Pascal Fua, Helge Jochen Rhodin, Mirela Ostrek

In this study, we compared a monocular computer vision (MCV)-based approach with the golden standard for collecting kinematic data on ski tracks (i.e., video-based stereophotogrammetry) and assessed its deployment readiness for answering applied research q ...

2019

MonoPerfCap: Human Performance Capture From Monocular Video

Helge Jochen Rhodin

2018

Unsupervised Geometry-Aware Representation Learning for 3D Human Pose Estimation

Pascal Fua, Mathieu Salzmann, Helge Jochen Rhodin

Modern 3D human pose estimation techniques rely on deep networks, which require large amounts of training data. While weakly-supervised methods require less supervision, by utilizing 2D poses or multi-view imagery without annotations, they still need a suf ...

2018

Learning Monocular 3D Human Pose Estimation from Multi-view Images

Pascal Fua, Mathieu Salzmann, Frédéric Meyer, Victor Constantin, Isinsu Katircioglu, Helge Jochen Rhodin

Accurate 3D human pose estimation from single images is possible with sophisticated deep-net architectures that have been trained on very large datasets. However, this still leaves open the problem of capturing motions for which no such database exists. Ma ...

IEEE2018