Publications de Volkan Cevher | EPFL Graph Search

Single-pass Detection of Jailbreaking Input in Large Language Models

Volkan Cevher, Grigorios Chrysos, Yongtao Wu, Elias Abad Rocamora

Defending aligned Large Language Models (LLMs) against jailbreaking attacks is a challenging problem, with existing approaches requiring multiple requests or even queries to auxiliary LLMs, making them computationally heavy. Instead, we focus on detecting ...

2025

How Gradient Descent Balances Features: A Dynamical Analysis For Two-Layer Neural Networks

Volkan Cevher, Zhenyu Zhu, Fanghui Liu

This paper investigates the fundamental regression task of learning k neurons (a.k.a. teachers) from Gaussian input, using two-layer ReLU neural networks with width m (a.k.a. students) and m, k = O(1), trained via gradient descent under proper initializati ...

2025

Accelerating Spectral Clustering under Fairness Constraints

Volkan Cevher

Fairness of decision-making algorithms is an increasingly important issue. In this paper, we focus on spectral clustering with group fairness constraints, where every demographic group is represented in each cluster proportionally as in the general populat ...

2025

Generalization of Noisy SGD in Unbounded Non-convex Settings

Volkan Cevher, Leello Tadesse Dadi

We study the generalization of iterative noisy gradient schemes on smooth non-convex losses. Formally, we establish time-independent information theoretic generalization bounds for Stochastic Gradient Langevin Dynamics (SGLD) that do not diverge as the ite ...

2025

IL-SOAR : Imitation Learning with Soft Optimistic Actor cRitic

Volkan Cevher, Luca Viano

This paper introduces the SOAR framework for imitation learning. SOAR is an algorithmic template that learns a policy from expert demonstrations with a primal dual style algorithm that alternates cost and policy updates. Within the policy updates, the SOAR ...

2025

CHAMELEON: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning

Volkan Cevher, Wanyun Xie

Training data mixtures greatly impact the generalization performance of large language models. Existing domain reweighting methods often rely on costly weight computations and require retraining when new data is introduced. To this end, we introduce a flex ...

2025

Faster Inference Of Flow-Based Generative Models Via Improved Data-Noise Coupling

Volkan Cevher, Leello Tadesse Dadi

Conditional Flow Matching (CFM), a simulation-free method for training continuous normalizing flows, provides an efficient alternative to diffusion models for key tasks like image and video generation. The performance of CFM in solving these tasks depends ...

2025

Best of Both Worlds: Regret Minimization versus Minimax Play

Volkan Cevher, Luca Viano

In this paper, we investigate the existence of online learning algorithms with bandit feedback that simultaneously guarantee O(1) regret compared to a given comparator strategy, and Õ(√ T) regret compared to any fixed strategy, where T is the number of rou ...

2025

Adversarial Training For Defense Against Label Poisoning Attacks

Volkan Cevher

As machine learning models grow in complexity and increasingly rely on publicly sourced data, such as the human-annotated labels used in training large language models, they become more vulnerable to label poisoning attacks. These attacks, in which adversa ...

2025

Continuous-Time Analysis of Heavy Ball Momentum in Min-Max Games

Volkan Cevher

Since Polyak's pioneering work, heavy ball (HB) momentum has been widely studied in minimization. However, its role in min-max games remains largely unexplored. As a key component of practical min-max algorithms like Adam, this gap limits their effectivene ...

2025

Training Deep Learning Models with Norm-Constrained LMOs

Volkan Cevher, Thomas Michaelsen Pethick, Zhenyu Zhu, Kimon Antonakopoulos, Wanyun Xie

In this work, we study optimization methods that leverage the linear minimization oracle (lmo) over a norm-ball. We propose a new stochastic family of algorithms that uses the lmo to adapt to the geometry of the problem and, perhaps surprisingly, show that ...

2025

Hadamard product in deep learning: Introduction, Advances and Challenges

Volkan Cevher, Yongtao Wu

While convolution and self-attention mechanisms have dominated architectural design in deep learning, this survey examines a fundamental yet understudied primitive: the Hadamard product. Despite its widespread implementation across various applications, th ...

2025

Adaptive Bilevel Optimization

Volkan Cevher, Luca Viano, Kimon Antonakopoulos

We propose a new adaptive optimization algorithm based on mirror descent for a class of possibly non-convex smooth bilevel optimization problems. The bilevel optimization template is broadly applicable in machine learning as it features two coupled problem ...

Association for Computing Machinery (ACM)2025

Layer-wise Quantization for Quantized Optimistic Dual Averaging

Dan Alistarh, Volkan Cevher, Kimon Antonakopoulos

Modern deep neural networks exhibit heterogeneity across numerous layers of various types such as residuals, multi-head attention, etc., due to varying structures (dimensions, activation functions, etc.), distinct representation characteristics, which impa ...

2025

On the Generalization of Stochastic Gradient Descent with Momentum

Volkan Cevher, Kimon Antonakopoulos

While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods. In this work, we first show that th ...

Microtome Publishing2024

Universal Gradient Methods for Stochastic Convex Optimization

Volkan Cevher, Ali Kavis, Yongtao Wu, Kimon Antonakopoulos

We develop universal gradient methods for Stochastic Convex Optimization (SCO). Our algorithms automatically adapt not only to the oracle's noise but also to the Hölder smoothness of the objective function without a priori knowledge of the particular setti ...

2024

Improving SAM Requires Rethinking its Optimization Formulation

Volkan Cevher, Fabian Ricardo Latorre Gomez, Thomas Michaelsen Pethick, Kimon Antonakopoulos, Wanyun Xie

This paper rethinks Sharpness-Aware Minimization (SAM), which is originally formulated as a zero-sum game where the weights of a network and a bounded perturbation try to minimize/maximize, respectively, the same differentiable loss. We argue that SAM shou ...

2024

Revisiting Character-level Adversarial Attacks for Language Models

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu, Elias Abad Rocamora

Adversarial attacks in Natural Language Processing apply perturbations in the character or token levels. Token-level attacks, gaining prominence for their use of gradient-based methods, are susceptible to altering sentence semantics, leading to invalid adv ...

2024

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

Volkan Cevher, Fanghui Liu

This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate shifts. By a bias-variance decomposi ...

2024

Robust NAS under adversarial training: benchmark, theory, and beyond

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu

Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robus ...

2024