Publications associées à Interaction multimodale

SVGC-AVA: 360-Degree Video Saliency Prediction With Spherical Vector-Based Graph Convolution and Audio-Visual Attention

Pascal Frossard, Chenglin Li, Li Wei, Qin Yang, Yuelei Li

Viewers of 360-degree videos are provided with both visual modality to characterize their surrounding views and audio modality to indicate the sound direction. Though both modalities are important for saliency prediction, little work has been done by joint ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Generalization and Personalization of Machine Learning for Multimodal Mobile Sensing in Everyday Life

Lakmal Buddika Meegahapola

A range of behavioral and contextual factors, including eating and drinking behavior, mood, social context, and other daily activities, can significantly impact an individual's quality of life and overall well-being. Therefore, inferring everyday life aspe ...

EPFL2024

Keep Sensors in Check: Disentangling Country-Level Generalization Issues in Mobile Sensor-Based Models with Diversity Scores

Daniel Gatica-Perez, Lakmal Buddika Meegahapola

Machine learning models trained with passive sensor data from mobile devices can be used to perform various inferences pertaining to activity recognition, context awareness, and health and well-being. Prior work has improved inference performance through t ...

Assoc Computing Machinery2023

A Conceptual Framework for Integrating Conversational Agents in Digital Education

Juan Carlos Farah

The presence of conversational agents (or chatbots) in educational contexts has been steadily increasing over the past few years. Recent surveys have shown widespread interest in the use of chatbots in education, both for research and practice. Although th ...

EPFL2023

Haptics based collaborative steering control in automated driving environments

Tomohiro Nakade

A vehicle's steering is a particular system in that it is exposed to individual subjective reviews based on criteria that are hard to assess quantitatively. Haptic design of such systems is a prime concern that has been at the center of industrial developm ...

EPFL2023

Towards Novel Evaluation Methods for Social Dialog Systems

Ekaterina Svikhnushina

Language has shaped human evolution and led to the desire to endow machines with language abilities. Recent advancements in natural language processing enable us to achieve this breakthrough in human-machine interaction. However, introducing conversational ...

EPFL2023

A Multimodal Dataset for Automatic Edge-AI Cough Detection

David Atienza Alonso, Tomas Teijeiro Campo, Lara Orlandic, Jérôme Paul Rémy Thevenot

Counting the number of times a patient coughs per day is an essential biomarker in determining treatment efficacy for novel antitussive therapies and personalizing patient care. There is a need for wearable devices that employ multimodal sensors to perform ...

Zenodo2023

Focus on People: Five Qestions from Human-Centered Computing

Daniel Gatica-Perez

A substantial body of research in multimodal interaction has studied how people naturally interact -face-to-face and through machinesand developed technology to analyze, support, and extend such forms of interaction. The talk will share personal experience ...

Assoc Computing Machinery2022

Simultaneous Prediction of Wrist and Hand Motions via Wearable Ultrasound Sensing for Natural Control of Hand Prostheses

Xingchen Yang

Simultaneous prediction of wrist and hand motions is essential for the natural interaction with hand prostheses. In this paper, we propose a novel multi-out Gaussian process (MOGP) model and a multi-task deep learning (MTDL) algorithm to achieve simultaneo ...

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC2022

Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning

Barbara Bruno, Jauwairia Nasir

In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of prod ...

2022

Meta Transfer Learning for Early Success Prediction in MOOCs

Vinitra Swamy, Tanja Christina Käser Jacober, Mirko Marras

Despite the increasing popularity of massive open online courses (MOOCs), many suffer from high dropout and low success rates. Early prediction of student success for targeted intervention is therefore essential to ensure no student is left behind in a cou ...

ACM2022

Textual Explanations and Critiques in Recommendation Systems

Diego Matteo Antognini

Artificial intelligence and machine learning algorithms have become ubiquitous. Although they offer a wide range of benefits, their adoption in decision-critical fields is limited by their lack of interpretability, particularly with textual data. Moreover, ...

EPFL2022

Empirical observations of multi-modal network-level models: Insights from the pNEUMA experiment

Nikolaos Geroliminis, Emmanouil Barmpounakis

Multi-modal interactions at the network-level remain unexplored due to the lack of highresolution data for all transportation modes involved. The current work investigates the effect of multi-modal interactions at space-mean network speed for each mode usi ...

2021

Cyclic thermomechanical behaviour of pile-soil interface and design methods for energy pile foundations

Elena Ravera

Energy piles are shallow geothermal systems integrated into structures already required for structural functions of civil works that provide both geothermal energy supply and structural support to the built environments. This research work is driven by the ...

EPFL2021

Many Are The Ways to Learn: Identifying multi-modal behavioral profiles of collaborative learning in constructivist activities

Pierre Dillenbourg, Barbara Bruno, Jauwairia Nasir, Aditi Kothiyal

Understanding how learners engage with learning technologies, and its relation with their learning, is crucial to the design of effective learning interventions. Assessing the learners’ state however, is non-trivial. Research suggests that performance is n ...

2021

Emergent nonlinear phenomena in a driven dissipative photonic dimer

Tobias Kippenberg, Rui Ning Wang, Hairun Guo, Mikhail Churaev, Junqiu Liu, Alexey Tikan, Kenichi Komagata

Collective effects leading to spatial, temporal or spatiotemporal pattern formation in complex nonlinear systems driven out of equilibrium cannot be described at the single-particle level and are therefore often called emergent phenomena. They are characte ...

NATURE RESEARCH2021

Hybrid Human-Machine Interface for Gait Decoding Through Bayesian Fusion of EEG and EMG Classifiers

Silvestro Micera, Fiorenzo Artoni, Luca Tonin, Stefano Tortora

Despite the advances in the field of brain computer interfaces (BCI), the use of the sole electroencephalography (EEG) signal to control walking rehabilitation devices is currently not viable in clinical settings, due to its unreliability. Hybrid interface ...

2020

Visual Attention and Perceptual Quality in Omnidirectional Imaging

Evgeniy Upenik

Omnidirectional imaging has reached a level of widespread availability driven by recent advances in integrated circuit technology, image sensors, and computer graphics which allow now capturing, rendering and displaying of such type of immersive content in ...

EPFL2020

Being Part of the Swarm: Experiencing Human-Swarm Interaction with VR and Tangible Robots

Pierre Dillenbourg, Barbara Bruno, Arzu Güneysu Özgür, Kevin Gonyop Kim, Hala Khodr

A swarm is the coherent behavior that emerges ubiquitously fromsimple interaction rules between self-organized agents. Understand-ing swarms is of utmost importance in many disciplines and jobs,but hard to teach due to the elusive nature of the phenomenon, ...

2020

Psychometrics of Disembodiment and Its Differential Modulation by Visuomotor and Visuotactile Mismatches

Oliver Alan Kannape, Bigna Lenggenhager

Altered states of embodiment are fundamental to the scientific understanding of bodily self consciousness. The feeling of disembodiment during everyday activities is common to clinical conditions; however, the direct study of disembodiment in experimental ...

2020