Publications related to Jointly Optimizing Sensing Pipelines for Multimodal Mixed Reality Interaction

TTool: A Supervised Artificial Intelligence-Assisted Visual Pose Detector for Tool Heads in Augmented Reality Woodworking

Yves Weinand, Julien Gamerro, Andrea Settimi, Florian Aymanns, Naravich Chutisilp

We present TimberTool (TTool v2.1.1), a software designed for woodworking tasks assisted by augmented reality (AR), emphasizing its essential function of the real-time localization of a tool head’s poses within camera frames. The localization process, a fu ...

2024

Generalization and Personalization of Machine Learning for Multimodal Mobile Sensing in Everyday Life

Lakmal Buddika Meegahapola

A range of behavioral and contextual factors, including eating and drinking behavior, mood, social context, and other daily activities, can significantly impact an individual's quality of life and overall well-being. Therefore, inferring everyday life aspe ...

EPFL2024

Customizing the human-avatar mapping based on EEG error related potentials during avatar-based interaction

Ronan Boulic, Ricardo Andres Chavarriaga Lozano, Bruno Herbelin, José del Rocio Millán Ruiz, Olaf Blanke, Fumiaki Iwane, Thibault Serge Mario Porssut

Objective. A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside th ...

2024

Immersive VR for upper-extremity rehabilitation in patients with neurological disorders: a scoping review

Silvestro Micera, Elena Losanno

Background Neurological disorders, such as stroke and chronic pain syndromes, profoundly impact independence and quality of life, especially when affecting upper extremity (UE) function. While conventional physical therapy has shown effectiveness in provid ...

BioMed Central2024

Additively manufactured stretchable zipping electrostatic actuators

Giulio Grasso

The use of soft and stretchable materials allows the development of adaptive robotic systems and human-machine interfaces that are more natural and comfortable to interact with. One of the application fields that benefits the most from these compliant mate ...

EPFL2024

Investigating neural resource allocation in the sensorimotor control of extra limbs

Giulia Dominijanni

The rise of robotic body augmentation brings forth new developments that will transform robotics, human-machine interaction, and wearable electronics. Extra robotic limbs, although building upon restorative technologies, bring their own set of challenges i ...

EPFL2024

SORI: A softness-rendering interface to unravel the nature of softness perception

Jamie Paik, Mustafa Mete, Hwayeong Jeong

Tactile perception of softness serves a critical role in the survival, well-being, and social interaction among various species, including humans. This perception informs activities from food selection in animals to medical palpation for disease detection ...

National Academy of Sciences2024

The Metaverse as a Virtual Model of Platform Urbanism: Its Converging AIoT, XReality, Neurotech, and Nanobiotech and Their Applications, Challenges, and Risks

Simon Elias Bibri

With their exponentially rising computational power, digital platforms are heralding a new era of hybrid intelligence. There has recently been much enthusiasm and hype that the Metaverse has the potential to unlock hybrid intelligence. This is premised on ...

MDPI2023

Projector-based augmented stacking framework for irregularly shaped objects

Yves Weinand, Katrin Beyer, Julien Gamerro, Andrea Settimi, Qianqing Wang, Edward Carlo Giorgio Andò

Augmented reality in additive fabrication is predominantly applied to the manufacturing of structures from regularly shaped materials. Our work however embraces natural heterogeneity, by focusing on the assembly of irregularly shaped elements such as miner ...

2023

Affordance segmentation of hand-occluded containers from exocentric images

Andrea Cavallaro

Visual affordance segmentation identifies the surfaces of an object an agent can interact with. Common challenges for the identification of affordances are the variety of the geometry and physical properties of these surfaces as well as occlusions. In this ...

Ieee Computer Soc2023

A Stand-Alone Augmented Reality Intervention for Chronic Pain using Embodied Systolic Stimulation

Skander Mensi, Sylvain Cardin, Oliver Alan Kannape, Robert Leeb, Fabien Bourban, Alexis Dorier, Andrea Serino

Chronic pain presents a tremendous personal, societal, and financial burden. Treatment options are often limited to managing symptoms as opposed to treating the causes, and the consequences of pharmacological treatments can be immensely harmful as evidence ...

IEEE2023

Updates on moleculARweb, the Swiss Portal for Chemistry and Structural Biology Education Using Augmented and now also Virtual Reality

Luciano Andres Abriata, Fabio Jose Cortes Rodriguez

We present a new activity on the moleculARweb plat-form on molecular symmetry elements, two more languages, a new marker-less augmented reality mode, and a new virtual reality tool for VR headsets. ...

SWISS CHEMICAL SOC2023

'connEcted caesarean section': creating a virtual link between MOthers and their infanTs to ImprOve maternal childbirth experieNce - study protocol for a PILOT trial (e-motion-pilot)

Sandra Maria Marcadent, Karine Lepigeon, Fiona Corbaz, David Desseauve

IntroductionOne-third of mothers rate their childbirth as traumatic. The prevalence of childbirth-related post-traumatic stress disorder (CB-PTSD) is 4.7%. Skin-to-skin contact is a protective factor against CB-PTSD. However, during a caesarean section (CS ...

BMJ PUBLISHING GROUP2023

Human Interactive Robotic Surfaces for Physical Immersion

Fabio Zuliani

Computers have long been augmenting human capabilities and communication, and with emerging technologies, physical immersion in virtual worlds is now attainable. These advancements push the boundaries of human potential, allowing designers, architects, eng ...

EPFL2023

Out-of-body illusion induced by visual-vestibular stimulation

Bruno Herbelin, Olaf Blanke, Florian Lance, Sophie Jacqueline Andrée Betka, Hsin-Ping Wu, Estelle Nakul

Out-of-body experiences (OBEs) are characterized by the subjective feeling of being located outside one's physical body and perceiving one's own body from an elevated perspective looking downwards. OBEs have been correlated with abnormal integration of bod ...

Cell Press2023

Chaotic microcomb inertia-free parallel ranging

Tobias Kippenberg, Anton Stroganov, Anton Lukashchuk

Ever growing pixel acquisition rates in the fields of augmented reality, autonomous driving and robotics have increased interest in solid state beam scanning without moving parts. Modern photonic integrated laser ranging advances towards passive beam steer ...

AIP Publishing2023

Practical considerations of diffusion-weighted MRS with ultra-strong diffusion gradients

André Döring

IntroductionDiffusion-weighted magnetic resonance spectroscopy (DW-MRS) offers improved cellular specificity to microstructure-compared to water-based methods alone-but spatial resolution and SNR is severely reduced and slow-diffusing metabolites necessita ...

Frontiers Media Sa2023

Physical Body Orientation Impacts Virtual Navigation Experience and Performance

Olaf Blanke, Hyukjun Moon, Emanuela De Falco, Hsin-Ping Wu

Most human navigation studies in MRI rely on virtual navigation. However, the necessary supine position in MRI makes it fundamentally different from daily ecological navigation. Nonetheless, until now, no study has assessed whether differences in physical ...

Soc Neuroscience2023

An immersive virtual reality system for ecological assessment of peripersonal and extrapersonal unilateral spatial neglect

Ravi Tej Sai Giridhar Tadi, Olaf Blanke, Andrea Serino, Roberta Ronchi

BackgroundUnilateral spatial neglect (USN) is a debilitating neuropsychological syndrome that often follows brain injury, in particular a stroke affecting the right hemisphere. In current clinical practice, the assessment of neglect is based on old-fashion ...

BioMed Central2023

Effects of Head-Tracking Artefacts on Externalization and Localization in Azimuth With Binaural Wearable Devices

Hervé Lissek, Gilles André Courtois, Vincent Pierre Olivier Grimaldi

Head tracking combined with head movements have been shown to improve auditory externalization of a virtual sound source and contribute to the performance in localization. With certain technically constrained head-tracking algorithms, as can be found in we ...

Audio Engineering Soc2023