Publications de Maxime Jean Julien Peyrard

Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling

Maxime Jean Julien Peyrard, Robert West, Marija Sakota

Generative language models (LMs) have become omnipresent across data science. For a wide variety of tasks, inputs can be phrased as natural language prompts for an LM, from whose output the solution can then be extracted. LM performance has consistently be ...

Assoc Computing Machinery2024

Language Model Decoding as Likelihood–Utility Alignment

Boi Faltings, Maxime Jean Julien Peyrard, Robert West, Martin Josifoski, Valentin Hartmann, Jiheng Wei, Frano Rajic, Debjit Paul

A critical component of a successful language generation pipeline is the decoding algorithm. However, the general principles that should guide the choice of a decoding algorithm re- main unclear. Previous works only compare decoding algorithms in narrow sc ...

Assoc Computational Linguistics-Acl2023

Distribution Inference Risks: Identifying and Mitigating Sources of Leakage

Léo Nicolas René Meynent, Maxime Jean Julien Peyrard, Robert West, Valentin Hartmann

A large body of work shows that machine learning (ML) models can leak sensitive or confidential information about their training data. Recently, leakage due to distribution inference (or property inference) attacks is gaining attention. In this attack, the ...

IEEE COMPUTER SOC2023

REFINER: Reasoning Feedback on Intermediate Representations

Boi Faltings, Maxime Jean Julien Peyrard, Robert West, Beatriz Maria Borges Ribeiro, Antoine Bosselut, Mahammad Ismayilzada, Debjit Paul

Language models (LMs) have recently shown remarkable performance on reasoning tasks by explicitly generating intermediate inferences, e.g., chain-of-thought prompting. However, these intermediate inference steps may be in- appropriate deductions from the i ...

2023

GenIE: Generative Information Extraction

Maxime Jean Julien Peyrard, Robert West, Martin Josifoski

Structured and grounded representation of text is typically formalized by closed information extraction, the problem of extracting an exhaustive set of (subject, relation, object) triplets that are consistent with a predefined set of entities and relations ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2022

Laughing Heads: Can Transformers Detect What Makes a Sentence Funny?

Maxime Jean Julien Peyrard, Robert West, Kristina Gligoric, Beatriz Maria Borges Ribeiro

The automatic detection of humor poses a grand challenge for natural language processing. Transformer-based systems have recently achieved remarkable results on this task, but they usually (1) were evaluated in setups where serious vs humorous texts came f ...

International Joint Conferences on Artificial Intelligence Organization2021

Better than Average: Paired Evaluation of NLP Systems

Maxime Jean Julien Peyrard, Robert West

Evaluation in NLP is usually done by comparing the scores of competing systems independently averaged over a common set of test instances. In this work, we question the use of averages for aggregating evaluation scores into a final number used to decide wh ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2021

On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation

Maxime Jean Julien Peyrard, Robert West, Wei Zhao, Yang Gao

Evaluation of cross-lingual encoders is usually performed either via zero-shot cross-lingual transfer in supervised downstream tasks or via unsupervised cross-lingual textual similarity. In this paper, we concern ourselves with reference-free machine trans ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2020

A Simple Theoretical Model of Importance for Summarization

Maxime Jean Julien Peyrard

Research on summarization has mainly been driven by empirical approaches, crafting systems to perform well on standard datasets with the notion of information Importance remaining latent. We argue that establishing theoretical models of Importance will adv ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2019

Studying Summarization Evaluation Metrics in the Appropriate Scoring Range

Maxime Jean Julien Peyrard

In summarization, automatic evaluation metrics are usually compared based on their ability to correlate with human judgments. Unfortunately, the few existing human judgment datasets have been created as by-products of the manual evaluations performed durin ...

ASSOC COMPUTATIONAL LINGUISTICS-ACL2019