Publications by Maksym Andriushchenko

Understanding generalization and robustness in modern deep learning

In this thesis, we study two closely related directions: robustness and generalization in modern deep learning. Deep learning models based on empirical risk minimization are known to be often non-robust to small, worst-case perturbations known as adversari ...

EPFL2024

Sparse-RS: A Versatile Framework for Query-Efficient Sparse Black-Box Adversarial Attacks

Maksym Andriushchenko, Nicolas Henri Bernard Flammarion, Francesco Croce

We propose a versatile framework based on random search, Sparse-RS, for score-based sparse targeted and untargeted attacks in the black-box setting. Sparse-RS does not rely on substitute models and achieves state-of-the-art success rate and query efficienc ...

ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE2022

Towards Understanding Sharpness-Aware Minimization

Maksym Andriushchenko, Nicolas Henri Bernard Flammarion

Sharpness-Aware Minimization (SAM) is a recent training method that relies on worst-case weight perturbations which significantly improves generalization in various settings. We argue that the existing justifications for the success of SAM which are based ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2022

On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

Maksym Andriushchenko

Fine-tuning pre-trained transformer-based language models such as BERT has become a common practice dominating leaderboards across various NLP benchmarks. Despite the strong empirical performance of fine-tuned models, fine-tuning is an unstable process: tr ...

2021

Understanding and Improving Fast Adversarial Training

Maksym Andriushchenko, Nicolas Henri Bernard Flammarion

A recent line of work focused on making adversarial training computationally efficient for deep learning models. In particular, Wong et al. (2020) showed that ℓ∞-adversarial training with fast gradient sign method (FGSM) can fail due to a phenomenon called ...

2020

Square Attack: a query-efficient black-box adversarial attack via random search

Maksym Andriushchenko, Nicolas Henri Bernard Flammarion, Francesco Croce

We propose the Square Attack, a new score-based black-box

l_2

and

l_\infty

adversarial attack that does not rely on local gradient information and thus is not affected by gradient masking. The Square Attack is based on a randomized search scheme where ...

2020

Sparse-RS: a versatile framework for query-efficient sparse black-box adversarial attacks

Maksym Andriushchenko, Nicolas Henri Bernard Flammarion, Francesco Croce

A large body of research has focused on adversarial attacks which require to modify all input features with small l2- or l∞-norms. In this paper we instead focus on query-efficient sparse attacks in the black-box setting. Our versatile framework, Sparse-RS ...

2020