Publications related to Integrated Analytic and Linearized Inverse Kinematics for Precise Full Body Interactions

From finger animation to full-body embodiment of avatars with different morphologies and proportions

VR (Virtual Reality) is a real-time simulation that creates the subjective illusion of being in a virtual world. This thesis explores how integrating the user's body and fingers can be achieved and beneficial for the user to experience VR. At the advent of ...

EPFL2023

Real-time Self-contact Retargeting of Avatars down to Finger Level

Ronan Boulic, Bruno Herbelin, Mathias Guy Delahaye

We interact with the world through a body that includes hands and fingers. Likewise, providing an avatar allowing the control of a virtual body with fingers is an important step to improve the user experience in VR. When the user and their avatar skeleton ...

The Eurographics Association2023

Automated post-earthquake damage assessment of stone masonry buildings integrating machine learning, computer vision, and physics-based modeling

Bryan German Pantoja Rosero

Current post-earthquake damage assessment methodologies are not only time-consuming but also subjective in nature and difficult to document. Recent advancements in artificial intelligence and technological devices make it possible to accomplish this task a ...

EPFL2023

Memory of Motion for Initializing Optimization in Robotics

Teguh Santoso Lembono

Many robotics problems are formulated as optimization problems. However, most optimization solvers in robotics are locally optimal and the performance depends a lot on the initial guess. For challenging problems, the solver will often get stuck at poor loc ...

EPFL2022

Encoder-Decoder Models for Human Segmentation and Motion Analysis

Isinsu Katircioglu

Detecting people from 2D images and analyzing their motion in 3D have been long standing computer vision problems central to numerous applications such as autonomous driving and athletic training. Recently, with the availability of large amounts of trainin ...

EPFL2022

A Wearable Smart Glove and Its Application of Pose and Gesture Detection to Sign Language Classification

Josephine Anna Eleanor Hughes

Advances in soft sensors coupled with machine learning are enabling increasingly capable wearable systems. Since hand motion in particular can convey useful information for developing intuitive interfaces, glove-based systems can have a significant impact ...

Institute of Electrical and Electronics Engineers Inc.2022

Preventing Breaks in Embodiment in Immersive Virtual Reality

Thibault Serge Mario Porssut

Virtual reality (VR) is immersive not only because of visual integration, but because we can act and perform in a virtual environment (VE). Beyond the mere fact that a VR system provides images computed directly from the users' viewpoint in stereoscopy, it ...

EPFL2021

Model-based 3D Face Reconstruction

Christophe René Joseph Ecabert

The human face plays an essential role in social interactions as it brings information about someone's identity, state of mind, or mood. People are, by nature, very good at catching this non-spoken information. Therefore, scientists have been interested in ...

EPFL2021

Drift-Free Foot Orientation Estimation in Running Using Wearable IMU

Kamiar Aminian, Frédéric Meyer, Benoît Mariani, Grégoire Millet, Mathieu Pascal Falbriard

This study aimed to introduce and validate a new method to estimate and correct the orientation drift measured from foot-worn inertial sensors. A modified strap-down integration (MSDI) was proposed to decrease the orientation drift, which, in turn, was fur ...

2020

Accurate Nod and 3D Gaze Estimation for Social Interaction Analysis

Yu Yu

Non-verbal behaviours play an important role in human communication since it can indicate human attention, serve as communication cue in interactions, or even reveal higher level personal constructs. For instance, head nod, a common non-verbal behaviour, c ...

EPFL2020

Intent Prediction Based on Biomechanical Coordination of EMG and Vision-Filtered Gaze for End-Point Control of an Arm Prosthesis

Aude Billard, Silvestro Micera, Iason Batzianoulis, Nili Eliana Krausz

We propose a novel controller for powered prosthetic arms, where fused EMG and gaze data predict the desired end-point for a full arm prosthesis, which could drive the forward motion of individual joints. We recorded EMG, gaze, and motion-tracking during p ...

2020

Hough Forest With Optimized Leaves for Global Hand Pose Estimation With Arbitrary Postures

Daniel Thalmann, Hui Liang, Jun Lee

Vision-based hand pose estimation is important in human-computer interaction. While many recent works focus on full degree-of-freedom hand pose estimation, robust estimation of global hand pose remains a challenging problem. This paper presents a novel alg ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2019

Real-Time 3D Hand Pose Estimation with 3D Convolutional Neural Networks

Daniel Thalmann, Hui Liang

In this paper, we present a novel method for real-time 3D hand pose estimation from single depth images using 3D Convolutional Neural Networks (CNNs). Image-based features extracted by 2D CNNs are not directly suitable for 3D hand pose estimation due to th ...

IEEE COMPUTER SOC2019

Neural Scene Decomposition for Multi-Person Motion Capture

Pascal Fua, Mathieu Salzmann, Victor Constantin, Isinsu Katircioglu, Helge Jochen Rhodin

Learning general image representations has proven key to the success of many computer vision tasks. For example, many approaches to image understanding problems rely on deep networks that were initially trained on ImageNet, mostly because the learned featu ...

IEEE2019

Mo(2)Cap(2): Real-time Mobile 3D Motion Capture with a Cap-mounted Fisheye Camera

Pascal Fua, Helge Jochen Rhodin

We propose the first real-time system for the egocentric estimation of 3D human body pose in a wide range of unconstrained everyday activities. This setting has a unique set of challenges, such as mobility of the hardware setup, and robustness to long capt ...

2019

FlyJacket: An Upper Body Soft Exoskeleton for Immersive Drone Control

Dario Floreano, David Atienza Alonso, Carine Rognon, Alexandre Thierry Cherpillod, Fabio Isidoro Tiberio Dell'Agnola, Stefano Mintchev

Most human–drone interfaces, such as joysticks and remote controllers, require attention and developed skills during teleoperation. Wearable interfaces could enable a more natural and intuitive control of drones, which would make this technology accessible ...

2018

Real-Time Generative Hand Modeling and Tracking

Anastasia Tkach

In our everyday life we interact with the surrounding environment using our hands. A main focus of recent research has been to bring such interaction to virtual objects, such as the ones projected in virtual reality devices, or super-imposed as holograms i ...

EPFL2018

Structure of a dense release produced by varying initial conditions

Anton Schleiss, Mário Jorge Rodrigues Pereira Da Franca, Jessica Zordan

Buoyancy driven flows such as gravity currents, present in nature and human made applications, are conveyors of particles or dissolved substances for long distances with clear implications for the environment. This transport depends on the triggering condi ...

2018

Learning Robust Features and Latent Representations for Single View 3D Pose Estimation of Humans and Objects

Bugra Tekin

Estimating the 3D poses of rigid and articulated bodies is one of the fundamental problems of Computer Vision. It has a broad range of applications including augmented reality, surveillance, animation and human-computer interaction. Despite the ever-growin ...

EPFL2018

Range-based underwater target localization using an autonomous surface vehicle: Observability analysis

In the field of marine robotics, the problem of range based underwater target localization can be defined as that of localizing an unknown - fixed or moving - target from a surface vehicle called the tracker, using range information available about the tar ...

2018