Publications by Leello Tadesse Dadi

Generalization of Noisy SGD in Unbounded Non-convex Settings

We study the generalization of iterative noisy gradient schemes on smooth non-convex losses. Formally, we establish time-independent information theoretic generalization bounds for Stochastic Gradient Langevin Dynamics (SGLD) that do not diverge as the ite ...

2025

Faster Inference Of Flow-Based Generative Models Via Improved Data-Noise Coupling

Volkan Cevher, Leello Tadesse Dadi

Conditional Flow Matching (CFM), a simulation-free method for training continuous normalizing flows, provides an efficient alternative to diffusion models for key tasks like image and video generation. The performance of CFM in solving these tasks depends ...

2025

Efficient Continual Finite-Sum Minimization

Volkan Cevher, Leello Tadesse Dadi, Efstratios Panteleimon Skoulakis

Given a sequence of functions

f_1,\ldots,f_n

with

f_i:\mathcal{D}\mapsto \mathbb{R}

, finite-sum minimization seeks a point

{x}^\star \in \mathcal{D}

minimizing

\sum_{j=1}^nf_j(x)/n

. In this work, we propose a key twist into the finite-sum minimizat ...

2024

Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks

Volkan Cevher, Leello Tadesse Dadi, Fanghui Liu

Recent studies show that a reproducing kernel Hilbert space (RKHS) is not a suitable space to model functions by neural networks as the curse of dimensionality (CoD) cannot be evaded when trying to approximate even a single ReLU neuron (Bach, 2017). In thi ...

2024

Finding Actual Descent Directions For Adversarial Training

Volkan Cevher, Igor Krawczuk, Fabian Ricardo Latorre Gomez, Thomas Michaelsen Pethick, Leello Tadesse Dadi

Adversarial Training using a strong first-order adversary (PGD) is the gold standard for training Deep Neural Networks that are robust to adversarial examples. We show that, contrary to the general understanding of the method, the gradient at an optimal ad ...

2023

The spectral bias of polynomial neural networks

Volkan Cevher, Moulik Choraria, Leello Tadesse Dadi, Grigorios Chrysos

Polynomial neural networks (PNNs) have been recently shown to be particularly effective at image generation and face recognition, where high-frequency information is critical. Previous studies have revealed that neural networks demonstrate a spectral bias ...

2022

Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization

Volkan Cevher, Ali Kavis, Leello Tadesse Dadi, Efstratios Panteleimon Skoulakis, Kimon Antonakopoulos

We propose an adaptive variance-reduction method, called AdaSpider, for minimization of L-smooth, non-convex functions with a finite-sum structure. In essence, AdaSpider combines an AdaGrad-inspired [Duchi et al., 2011, McMahan & Streeter, 2010], but a fai ...

2022

The Effect of the Intrinsic Dimension on the Generalization of Quadratic Classifiers

Paul Thierry Yves Rolland, Fabian Ricardo Latorre Gomez, Leello Tadesse Dadi

It has been recently observed that neural networks, unlike kernel methods, enjoy a reduced sample complexity when the distribution is isotropic (i.e., when the covariance matrix is the identity). We find that this sensitivity to the data distribution is no ...

2021

Generating Sparse Stochastic Processes Using Matched Splines

Michaël Unser, Shayan Aziznejad, Leello Tadesse Dadi

We provide an algorithm to generate trajectories of sparse stochastic processes that are solutions of linear ordinary differential equations driven by Levy white noises. A recent paper showed that these processes are limits in law of generalized compound-P ...

2020