Publications related to Language model

Infusing structured knowledge priors in neural models for sample-efficient symbolic reasoning

The ability to reason, plan and solve highly abstract problems is a hallmark of human intelligence. Recent advancements in artificial intelligence, propelled by deep neural networks, have revolutionized disciplines like computer vision and natural language ...

EPFL2024

Driving and suppressing the human language network using large language models

Martin Schrimpf

Transformer models such as GPT generate human-like language and are predictive of human brain responses to language. Here, using functional-MRI-measured brain responses to 1,000 diverse sentences, we first show that a GPT-based encoding model can predict t ...

Nature Portfolio2024

Revisiting Character-level Adversarial Attacks for Language Models

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu, Elias Abad Rocamora

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adv ...

2024

Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling

Robert West, Maxime Jean Julien Peyrard, Marija Sakota

Generative language models (LMs) have become omnipresent across data science. For a wide variety of tasks, inputs can be phrased as natural language prompts for an LM, from whose output the solution can then be extracted. LM performance has consistently be ...

Assoc Computing Machinery2024

Graph generative deep learning models with an application to circuit topologies

Igor Krawczuk

Modern integrated circuits are tiny yet incredibly complex technological artifacts, composed of millions and billions of individual structures working in unison. Managing their complexity and facilitating their design drove part of the co-evolution of mode ...

EPFL2024

Student Answer Forecasting: Transformer-Driven Answer Choice Prediction for Language Learning

Paola Mejia Domenzain, Vinitra Swamy, Tanja Christina Käser Jacober, Elena Grazia Gado, Tommaso Martorella, Jibril Albachir Frej, Luca Zunino

Intelligent Tutoring Systems (ITS) enhance personalized learning by predicting student answers to provide immediate and customized instruction. However, recent research has primarily focused on the correctness of the answer rather than the student's perfor ...

2024

Text-to-Microstructure Generation Using Generative Deep Learning

Jamie Paik

Designing novel materials is greatly dependent on understanding the design principles, physical mechanisms, and modeling methods of material microstructures, requiring experienced designers with expertise and several rounds of trial and error. Although rec ...

Wiley-V C H Verlag Gmbh2024

The multimodality cell segmentation challenge: toward universal solutions

Sahand Jamal Rahi, Vojislav Gligorovski, Marco Labagnara, Jun Ma, Maxime Emmanuel Scheder, Xin Yang, Yao Zhang, Bo Wang, Yixin Wang, Lin Han

Cell segmentation is a critical step for quantitative single-cell analysis in microscopy images. Existing cell segmentation methods are often tailored to specific modalities or require manual interventions to specify hyper-parameters in different experimen ...

Nature Portfolio2024

Towards improving full-length ribosome density prediction by bridging sequence and graph-based representations

Pierre Vandergheynst, Felix Naef, Cédric Gobet, Francesco Craighero, Mohan Vamsi Nallapareddy

Translation elongation plays an important role in regulating protein concentrations in the cell, and dysregulation of this process has been linked to several human diseases. In this study, we use data from ribo-seq experiments to model ribosome densities, ...

PMLR2024

Fast adaptation to rule switching using neuronal surprise

Wulfram Gerstner, Martin Louis Lucien Rémy Barry

In humans and animals, surprise is a physiological reaction to an unexpected event, but how surprise can be linked to plausible models of neuronal activity is an open problem. We propose a self-supervised spiking neural network model where a surprise signa ...

2024

Dynamic Voxels Based on Ego-Conditioned Prediction: An Integrated Spatio-Temporal Framework for Motion Planning

Alexandre Massoud Alahi, Ting Zhang

Prediction is a vital component of motion planning for autonomous vehicles (AVs). By reasoning about the possible behavior of other target agents, the ego vehicle (EV) can navigate safely, efficiently, and politely. However, most of the existing work overl ...

IEEE Institute of Electrical and Electronics Engineers2024

Social-Transmotion: Promptable Human Trajectory Prediction

Alexandre Massoud Alahi, Saeed Saadatnejad, Yang Gao, Kaouther Messaoud Ben Amor

Accurate human trajectory prediction is crucial for applications such as autonomous vehicles, robotics, and surveillance systems. Yet, existing models often fail to fully leverage the non-verbal social cues human subconsciously communicate when navigating ...

2024

Random matrix methods for high-dimensional machine learning models

Antoine Philippe Michel Bodin

In the rapidly evolving landscape of machine learning research, neural networks stand out with their ever-expanding number of parameters and reliance on increasingly large datasets. The financial cost and computational resources required for the training p ...

EPFL2024

Land Cover Mapping From Multiple Complementary Experts Under Heavy Class Imbalance

Devis Tuia, Valérie Zermatten, Javiera Francisca Castillo Navarro, Xiaolong Lu

Deep learning has emerged as a promising avenue for automatic mapping, demonstrating high efficacy in land cover categorization through various semantic segmentation models. Nonetheless, the practical deployment of these models encounters important challen ...

Ieee-Inst Electrical Electronics Engineers Inc2024

Post-correction of Historical Text Transcripts with Large Language Models: An Exploratory Study

Frédéric Kaplan, Maud Ehrmann, Matteo Romanello, Emanuela Boros, Sven-Nicolas Yoann Najem

The quality of automatic transcription of heritage documents, whether from printed, manuscripts or audio sources, has a decisive impact on the ability to search and process historical texts. Although significant progress has been made in text recognition ( ...

Association for Computational Linguistics2024

GELEX: Generative AI-Hybrid System for Example-Based Learning

Paola Mejia Domenzain, Aybars Yazici, Tanja Christina Käser Jacober, Jibril Albachir Frej

Traditional example-based learning methods are often limited by static, expert-created content. Hence, they face challenges in scalability, engagement, and effectiveness, as some learners might struggle to relate to the examples or find them relevant. To a ...

Association for Computing Machinery2024

Toward universal cell embeddings: integrating single-cell RNA-seq datasets across species with SATURN

Maria Brbic, Ziang Li

Analysis of single-cell datasets generated from diverse organisms offers unprecedented opportunities to unravel fundamental evolutionary processes of conservation and diversification of cell types. However, interspecies genomic differences limit the joint ...

Nature Portfolio2024

Performing and Detecting Backdoor Attacks on Face Recognition Algorithms

Alexander Carl Unnervik

The field of biometrics, and especially face recognition, has seen a wide-spread adoption the last few years, from access control on personal devices such as phones and laptops, to automated border controls such as in airports. The stakes are increasingly ...

EPFL2024

Residual-based attention in physics-informed neural networks

Nikolaos Stergiopoulos, Sokratis Anagnostopoulos

Driven by the need for more efficient and seamless integration of physical models and data, physics -informed neural networks (PINNs) have seen a surge of interest in recent years. However, ensuring the reliability of their convergence and accuracy remains ...

Elsevier Science Sa2024

Seeking the new, learning from the unexpected: Computational models of surprise and novelty in the brain

Alireza Modirshanechi

Human babies have a natural desire to interact with new toys and objects, through which they learn how the world around them works, e.g., that glass shatters when dropped, but a rubber ball does not. When their predictions are proven incorrect, such as whe ...

EPFL2024