Publications related to Multimodal sentiment analysis

Functionalization of harmonic nanoparticles for drug release and multimodal imaging applications

Cancer is among the leading causes of death worldwide, and as knowledge of the disease continues to grow there is an increasing interest towards precision medicine: more specifically towards the theranostics field, i.e the development of targeted molecular ...

EPFL2024

Understanding the sentiment associated with cultural ecosystem services using images and text from social media

Devis Tuia

Social media is increasingly being employed to develop Cultural Ecosystem Services (CES) indicators. The image-sharing platform Flickr has been one of the most popular sources of data. Most large-scale studies, however, tend to only use the number of image ...

2024

Examining European Press Coverage of the Covid-19 No-Vax Movement: An NLP Framework

Daniel Gatica-Perez

This paper examines how the European press dealt with the no-vax reactions against the Covid-19 vaccine and the dis- and misinformation associated with this movement. Using a curated dataset of 1786 articles from 19 European newspapers on the anti-vaccine ...

ASSOC COMPUTING MACHINERY2023

A single-beat algorithm to discriminate farfield from nearfield bipolar voltage electrograms from the pulmonary veins

Jean-Marc Vesin, Adrian Luca, Vincent Schlageter

Background Superimposition of farfield (FF) and nearfield (NF) bipolar voltage electrograms (BVE) complicates the confirmation of pulmonary vein (PV) isolation after catheter ablation of atrial fibrillation. Our aim was to develop an automatic algorithm ba ...

SPRINGER2023

Personalized Productive Engagement Recognition in Robot-Mediated Collaborative Learning

Barbara Bruno, Jauwairia Nasir

In this paper, we propose and compare personalized models for Productive Engagement (PE) recognition. PE is defined as the level of engagement that maximizes learning. Previously, in the context of robot-mediated collaborative learning, a framework of prod ...

2022

Text as signal. A tutorial with case studies focusing on social media (Twitter)

Erick Jorge Canales Rodriguez

Sentiment analysis is the automated coding of emotions expressed in text. Sentiment analysis and other types of analyses focusing on the automatic coding of textual documents are increasingly popular in psychology and computer science. However, the potenti ...

SPRINGER2022

How Did Europe's Press Cover Covid-19 Vaccination News? A Five-Country Analysis

Daniel Gatica-Perez

Understanding how high-quality newspapers present and discuss major news plays a role towards tackling disinformation, as it contributes to the characterization of the full ecosystem in which information circulates. In this paper, we present an analysis of ...

Assoc Computing Machinery2022

CAFS: Cost-Aware Features Selection Method for Multimodal Stress Monitoring on Wearable Devices

David Atienza Alonso, Maria del Carmen Sandi Perez, Adriana Arza Valdes, João Pedro de Matos Rodrigues, Niloofar Momeni

Objective: Today, stress monitoring on wearable devices is challenged by the tension between high-detection accuracy and battery lifetime driven by multimodal data acquisition and processing. Limited research has addressed the classification cost on multim ...

2022

Model-based 3D Face Reconstruction

Christophe René Joseph Ecabert

The human face plays an essential role in social interactions as it brings information about someone's identity, state of mind, or mood. People are, by nature, very good at catching this non-spoken information. Therefore, scientists have been interested in ...

EPFL2021

Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers

Frédéric Kaplan, Maud Ehrmann, Sofia Ares Oliveira, Raphaël Barman

The massive amounts of digitized historical documents acquired over the last decades naturally lend themselves to automatic processing and exploration. Research work seeking to automatically process facsimiles and extract information thereby are multiplyin ...

2021

Temporal analysis of multimodal data to predict collaborative learning outcomes

Kshitij Sharma, Jennifer Kaitlyn Olsen

The analysis of multiple data streams is a long-standing practice within educational research. Both multimodal data analysis and temporal analysis have been applied successfully, but in the area of collaborative learning, very few studies have investigated ...

WILEY2020

Predicting learners' effortful behaviour in adaptive assessment using multimodal data

Kshitij Sharma, Jennifer Kaitlyn Olsen

Many factors influence learners' performance on an activity beyond the knowledge required. Learners' on-task effort has been acknowledged for strongly relating to their educational outcomes, reflecting how actively they are engaged in that activity. Howeve ...

ASSOC COMPUTING MACHINERY2020

Multimodal person recognition in audio-visual streams

Do Hoang Nam Le

Multimedia databases are growing rapidly in size in the digital age. To increase the value of these data and to enhance the user experience, there is a need to make these videos searchable through automatic indexing. Because people appearing and talking in ...

EPFL2019

BookTubing Across Regions: Examining Differences based on Nonverbal and Verbal Cues

Daniel Gatica-Perez, Dinesh Babu Jayagopi

BookTubers are a rapidly growing community in YouTube who shares content related to books. Previous literature has addressed problems related to automatically analyzing opinions and mood of video logs (vlogs) as a generic category in YouTube. Unfortunately ...

ASSOC COMPUTING MACHINERY2019

Modeling the Impact of Modifiers on Emotional Statements

Pearl Pu Faltings, Valentina Sintsova, Margarita Bolívar Jiménez

Humans use a variety of modifiers to enrich communications with one another. While this is a deliberate subtlety in our language, the presence of modifiers can cause problems for emotion analysis by machines. Our research objective is to understand and com ...

SPRINGER NATURE SWITZERLAND AG2018

Modeling Dyadic and Group Impressions with Inter-Modal and Inter-Person Features

Daniel Gatica-Perez, Laurent Son Nguyen, Oya Aran

This article proposes a novel feature-extraction framework for inferring impression personality traits, emergent leadership skills, communicative competence, and hiring decisions. The proposed framework extracts multimodal features, describing each partici ...

2018

Acquiring Broad Commonsense Knowledge for Sentiment Analysis Using Human Computation

Marina Boia

While artificial intelligence is successful in many applications that cover specific domains, for many commonsense problems there is still a large gap with human performance. Automated sentiment analysis is a typical example: while there are techniques tha ...

EPFL2016

Adaptive sentiment-aware one-class collaborative filtering

Andrei Popescu-Belis, Nikolaos Pappas

This paper presents a novel application of sentiment analysis to recommender systems relying on explicit one-class user feedback (favorites or likes), namely joint models of unary feedback and sentiment of free-form user comments. This combination is achie ...

Elsevier2016

Teaching Analytics: Towards Automatic Extraction of Orchestration Graphs Using Wearable Sensors

Pierre Dillenbourg, Maria Jesus Rodriguez Triana, Kshitij Sharma, Luis Pablo Prieto Santos

"Teaching analytics" is the application of learning analytics techniques to understand teaching and learning processes, and eventually enable supportive interventions. However, in the case of (often, half-improvised) teaching in face-to-face classrooms, su ...

Assoc Computing Machinery2016

Dystemo: Distant Supervision Method for Multi-Category Emotion Recognition in Tweets

Pearl Pu Faltings, Valentina Sintsova

Emotion recognition in text has become an important research objective. It involves building classifiers capable of detecting human emotions for a specific application, for example, analyzing reactions to product launches, monitoring emotions at sports eve ...

Assoc Computing Machinery2016