Publications related to Introducing the HIPE 2022 Shared Task: Named Entity Recognition and Linking in Multilingual Historical Documents

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

The ability to reason, plan and solve highly abstract problems is a hallmark of human intelligence. Recent advancements in artificial intelligence, propelled by deep neural networks, have revolutionized disciplines like computer vision and natural language ...

EPFL2024

Social-Transmotion: Promptable Human Trajectory Prediction

Alexandre Massoud Alahi, Saeed Saadatnejad, Yang Gao, Kaouther Messaoud Ben Amor

Accurate human trajectory prediction is crucial for applications such as autonomous vehicles, robotics, and surveillance systems. Yet, existing models often fail to fully leverage the non-verbal social cues human subconsciously communicate when navigating ...

2024

Transformer Models for Vision

Jean-Baptiste Francis Marie Juliette Cordonnier

The recent developments of deep learning cover a wide variety of tasks such as image classification, text translation, playing go, and folding proteins. All these successful methods depend on a gradient-based learning algorithm to train a model on massive ...

EPFL2023

Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models

Karl Aberer, Rémi Philippe Lebret, Mohammadreza Banaei

Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks. However, their enormous size often makes them impractical on memory-constrained devices, requiring practitioners to compress them to smaller net ...

Assoc Computational Linguistics-Acl2023

Modeling Structured Data in Attention-based Models

Alireza Mohammadshahi

Natural language processing has experienced significant improvements with the development of Transformer-based models, which employ self-attention mechanism and pre-training strategies. However, these models still present several obstacles. A notable issue ...

EPFL2023

XTab: Cross-table Pretraining for Tabular Transformers

Mahsa Shoaran, Bingzhao Zhu

The success of self-supervised learning in computer vision and natural language processing has motivated pretraining methods on tabular data. However, most existing tabular self-supervised learning models fail to leverage information across multiple data t ...

JMLR.org2023

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

Framing the News: From Human Perception to Large Language Model Inferences

Daniel Gatica-Perez

Identifying the frames of news is important to understand the articles' vision, intention, message to be conveyed, and which aspects of the news are emphasized. Framing is a widely studied concept in journalism, and has emerged as a new topic in computing, ...

Assoc Computing Machinery2023

Backpropagation-free training of deep physical neural networks

Romain Christophe Rémy Fleury, Babak Rahmani, Ali Momeni, Marc Philipp Del Hougne, Matthieu Francis Malléjac

Recent successes in deep learning for vision and natural language processing are attributed to larger models but come with energy consumption and scalability issues. Current training of digital deep-learning models primarily relies on backpropagation that ...

American Association for the Advancement of Science2023

Multi-task prompt-RSVQA to explicitly count objects on aerial images

Devis Tuia, Christel Marie Tartini-Chappuis, Nicola Antonio Santacroce, Sylvain Lobry, Javiera Francisca Castillo Navarro

Introduced to enable a wider use of Earth Observation images using natural language, Remote Sensing Visual Question Answering (RSVQA) remains a challenging task, in particular for questions related to counting. To address this specific challenge, we propos ...

2023

Improving Generalization of Pretrained Language Models

Rabeeh Karimi Mahabadi

In this dissertation, we propose multiple methods to improve transfer learning for pretrained language models (PLMs). Broadly, transfer learning is a powerful technique in natural language processing, where a language model is first pre-trained on a data-r ...

EPFL2023

Text as a Richer Source of Supervision in Semantic Segmentation Tasks

Devis Tuia, Valérie Zermatten, Lloyd Haydn Hughes, Javiera Francisca Castillo Navarro

This paper introduces TACOSS a text-image alignment approach that allows explainable land cover semantic segmentation by directly integrating semantic concepts encoded from texts. TACOSS combines convolutional neural networks for visual feature extraction ...

The Institute of Electrical and Electronics Engineers, Inc2023

TiC-SAT: Tightly-coupled Systolic Accelerator for Transformers

David Atienza Alonso, Giovanni Ansaloni, Alireza Amirshahi, Joshua Alexander Harrison Klein

Transformer models have achieved impressive results in various AI scenarios, ranging from vision to natural language processing. However, their computational complexity and their vast number of parameters hinder their implementations on resource-constraine ...

2023

Empathetic Conversational Agents for Distress Support

Kalpani Anuradha Welivita

Due to the increasing demands of today's fast-paced world, mental health concerns are on the rise, which necessitates innovative approaches to provide support to those in need. Open-domain conversational agents known as chatbots, offer a unique opportunit ...

EPFL2023

Deep Generative Models for Autonomous Driving: from Motion Forecasting to Realistic Image Synthesis

Saeed Saadatnejad

Forecasting is a capability inherent in humans when navigating. Humans routinely plan their paths, considering the potential future movements of those around them. Similarly, to achieve comparable sophistication and safety, autonomous systems must embrace ...

EPFL2023

Breaking the Curse of Dimensionality in Deep Neural Networks by Learning Invariant Representations

Leonardo Petrini

Artificial intelligence, particularly the subfield of machine learning, has seen a paradigm shift towards data-driven models that learn from and adapt to data. This has resulted in unprecedented advancements in various domains such as natural language proc ...

EPFL2023

Interpretable Representation Learning and Evaluation for Abstractive Summarization

Andreas Thomas Marfurt

Abstractive summarization has seen big improvements in recent years, mostly due to advances in neural language modeling, language model pretraining, and scaling models and datasets. While large language models generate summaries that are fluent, coherent, ...

EPFL2023

MuseReduce: A Generic Framework for Hierarchical Music Analysis

Martin Alois Rohrmeier, Petter Harald Ericson

In comparison to computational linguistics, with its abundance of natural-language datasets, corpora of music analyses are rather fewer and generally smaller. This is partly due to difficulties inherent to the encoding of music analyses, whose multimodal r ...

Humanities Commons2023

impresso Text Reuse at Scale. An interface for the exploration of text reuse data in semantically enriched historical newspapers

Maud Ehrmann, Matteo Romanello

Text Reuse reveals meaningful reiterations of text in large corpora. Humanities researchers use text reuse to study, e.g., the posterior reception of influential texts or to reveal evolving publication practices of historical media. This research is often ...

2023

A Two-Step Approach To Leverage Contextual Data: Speech Recognition In Air-Traffic Communications

Petr Motlicek, Amrutha Prasad, Juan Pablo Zuluaga Gomez

Automatic Speech Recognition (ASR), as the assistance of speech communication between pilots and air-traffic controllers, can significantly reduce the complexity of the task and increase the reliability of transmitted information. ASR application can lead ...

IEEE2022