Publications related to Natural-language user interface

Social-Transmotion: Promptable Human Trajectory Prediction

Alexandre Massoud Alahi, Saeed Saadatnejad, Yang Gao, Kaouther Messaoud Ben Amor

Accurate human trajectory prediction is crucial for applications such as autonomous vehicles, robotics, and surveillance systems. Yet, existing models often fail to fully leverage the non-verbal social cues human subconsciously communicate when navigating ...

2024

Multi-task prompt-RSVQA to explicitly count objects on aerial images

Devis Tuia, Christel Marie Tartini-Chappuis, Sylvain Lobry, Javiera Francisca Castillo Navarro, Nicola Antonio Santacroce

Introduced to enable a wider use of Earth Observation images using natural language, Remote Sensing Visual Question Answering (RSVQA) remains a challenging task, in particular for questions related to counting. To address this specific challenge, we propos ...

2023

Prompting Large Language Models to Power Educational Chatbots

Denis Gillet, Juan Carlos Farah, Sandy Ingram, Fanny Kim-Lan Lasne

The recent rise in both popularity and performance of large language models has garnered considerable interest regarding their applicability to education. Technologies like ChatGPT, which can engage in human-like dialog, have already disrupted educational ...

Springer2023

Framing the News: From Human Perception to Large Language Model Inferences

Daniel Gatica-Perez

Identifying the frames of news is important to understand the articles' vision, intention, message to be conveyed, and which aspects of the news are emphasized. Framing is a widely studied concept in journalism, and has emerged as a new topic in computing, ...

Assoc Computing Machinery2023

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Karl Aberer, Rémi Philippe Lebret, Mohammadreza Banaei

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller net ...

Assoc Computational Linguistics-Acl2023

Chat2Code: A Chatbot for Model Specification and Code Generation, The Case of Smart Contracts

Shailesh Mishra

The potential of automatic code generation through Model-Driven Engineering (MDE) frameworks has yet to be realized. Beyond their ability to help software professionals write more accurate, reusable code, MDE frameworks could make programming accessible fo ...

Ieee Computer Soc2023

XTab: Cross-table Pretraining for Tabular Transformers

Mahsa Shoaran, Bingzhao Zhu

The success of self-supervised learning in computer vision and natural language processing has motivated pretraining methods on tabular data. However, most existing tabular self-supervised learning models fail to leverage information across multiple data t ...

JMLR.org2023

Latency Interfaces for Systems Code

Rishabh Ramesh Iyer

This thesis demonstrates that it is feasible for systems code to expose a latency interface that describes its latency and related side effects for all inputs, just like the code's semantic interface describes its functionality and related side effects. S ...

EPFL2023

Modeling Structured Data in Attention-based Models

Alireza Mohammadshahi

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

EPFL2023

On matching data and model in LF-MMI-based dysarthric speech recognition

Enno Hermann

In light of steady progress in machine learning, automatic speech recognition (ASR) is entering more and more areas of our daily life, but people with dysarthria and other speech pathologies are left behind. Their voices are underrepresented in the trainin ...

EPFL2023

TiC-SAT: Tightly-coupled Systolic Accelerator for Transformers

David Atienza Alonso, Giovanni Ansaloni, Alireza Amirshahi, Joshua Alexander Harrison Klein

Transformer models have achieved impressive results in various AI scenarios, ranging from vision to natural language processing. However, their computational complexity and their vast number of parameters hinder their implementations on resource-constraine ...

2023

Backpropagation-free training of deep physical neural networks

Romain Christophe Rémy Fleury, Ali Momeni, Babak Rahmani, Matthieu Francis Malléjac, Marc Philipp Del Hougne

Recent successes in deep learning for vision and natural language processing are attributed to larger models but come with energy consumption and scalability issues. Current training of digital deep-learning models primarily relies on backpropagation that ...

American Association for the Advancement of Science2023

Empathetic Conversational Agents for Distress Support

Kalpani Anuradha Welivita

Due to the increasing demands of today's fast-paced world, mental health concerns are on the rise, which necessitates innovative approaches to provide support to those in need. Open-domain conversational agents known as chatbots, offer a unique opportunit ...

EPFL2023

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

Prompt–RSVQA: Prompting visual context to a language model for Remote Sensing Visual Question Answering

Devis Tuia, Christel Marie Tartini-Chappuis, Sylvain Lobry, Valérie Zermatten

Remote sensing visual question answering (RQA) was recently proposed with the aim of interfacing natural language and vision to ease the access of information contained in Earth Observation data for a wide audience, which is granted by simple questions in ...

2022

SKILL: Structured Knowledge Infusion for Large Language Models

Martin Jaggi, Fedor Moiseev

Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. However, it is largely unexplored whether they can better internalize knowledge from a structured data, such as a knowledge graph, or from ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2022

Textual Explanations and Critiques in Recommendation Systems

Diego Matteo Antognini

Artificial intelligence and machine learning algorithms have become ubiquitous. Although they offer a wide range of benefits, their adoption in decision-critical fields is limited by their lack of interpretability, particularly with textual data. Moreover, ...

EPFL2022

On Breathing Pattern Information in Synthetic Speech

Mathew Magimai Doss, Zohreh Mostaani

The respiratory system is an integral part of human speech production. As a consequence, there is a close relation between respiration and speech signal, and the produced speech signal carries breathing pattern related information. Speech can also be gener ...

ISCA-INT SPEECH COMMUNICATION ASSOC2022

A Two-Step Approach To Leverage Contextual Data: Speech Recognition In Air-Traffic Communications

Petr Motlicek, Juan Pablo Zuluaga Gomez, Amrutha Prasad

Automatic Speech Recognition (ASR), as the assistance of speech communication between pilots and air-traffic controllers, can significantly reduce the complexity of the task and increase the reliability of transmitted information. ASR application can lead ...

IEEE2022

Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer-Review Data Modeling

Vinitra Swamy, Tanja Christina Käser Jacober, Thiemo Wambsganss

Natural Language Processing (NLP) has become increasingly utilized to provide adaptivity in educational applications. However, recent research has highlighted a variety of biases in pre-trained language models. While existing studies investigate bias in di ...

2022