Publications related to Positive-definite kernel

High-Dimensional Kernel Methods under Covariate Shift: Data-Dependent Implicit Regularization

This paper studies kernel ridge regression in high dimensions under covariate shifts and analyzes the role of importance re-weighting. We first derive the asymptotic expansion of high dimensional kernels under covariate shifts. By a bias-variance decomposi ...

2024

Density estimation in RKHS with application to Korobov spaces in high dimensions

Fabio Nobile, Yoshihito Kazashi

A kernel method for estimating a probability density function from an independent and identically distributed sample drawn from such density is presented. Our estimator is a linear combination of kernel functions, the coefficients of which are determined b ...

2023

Machine Learning Methods for Robust Uncertainty Quantification and Controller Approximation

Emilio Maddalena

This thesis is situated at the crossroads between machine learning and control engineering. Our contributions are both theoretical, through proposing a new uncertainty quantification methodology in a kernelized context; and experimental, through investigat ...

EPFL2023

Probabilistic methods for neural combinatorial optimization

Nikolaos Karalias

The monumental progress in the development of machine learning models has led to a plethora of applications with transformative effects in engineering and science. This has also turned the attention of the research community towards the pursuit of construc ...

EPFL2023

The Completion Of Covariance Kernels

Victor Panaretos, Kartik Waghmare

We consider the problem of positive-semidefinite continuation: extending a partially specified covariance kernel from a subdomain Omega of a rectangular domain I x I to a covariance kernel on the entire domain I x I. For a broad class of domains Omega call ...

INST MATHEMATICAL STATISTICS-IMS2022

Sketches, metrics and fast algorithms

Navid Nouri

As it has become easier and cheaper to collect big datasets in the last few decades, designing efficient and low-cost algorithms for these datasets has attracted unprecedented attention. However, in most applications, even storing datasets as acquired has ...

EPFL2022

SPAHM: the spectrum of approximated Hamiltonian matrices representations

Anne-Clémence Corminboeuf, Ksenia Briling, Alberto Fabrizio

Physics-inspired molecular representations are the cornerstone of similarity-based learning applied to solve chemical problems. Despite their conceptual and mathematical diversity, this class of descriptors shares a common underlying philosophy: they all r ...

2022

Multikernel Regression with Sparsity Constraint

Michaël Unser, Shayan Aziznejad

In this paper, we provide a Banach-space formulation of supervised learning with generalized total-variation (gTV) regularization. We identify the class of kernel functions that are admissible in this framework. Then, we propose a variation of supervised l ...

2021

Deterministic error bounds for kernel-based learning techniques under bounded noise

Colin Neil Jones, Paul Scharnhorst, Emilio Maddalena

We consider the problem of reconstructing a function from a finite set of noise-corrupted samples. Two kernel algorithms are analyzed, namely kernel ridge regression and epsilon-support vector regression. By assuming the ground-truth function belongs to th ...

2021

Impact of quantum-chemical metrics on the machine learning prediction of electron density

Anne-Clémence Corminboeuf, Ksenia Briling, Alberto Fabrizio

Machine learning (ML) algorithms have undergone an explosive development impacting every aspect of computational chemistry. To obtain reliable predictions, one needs to maintain a proper balance between the black-box nature of ML frameworks and the physics ...

AMER INST PHYSICS2021

Oblivious Sketching of High-Degree Polynomial Kernels

Mikhail Kapralov, Amir Zandieh

Kernel methods are fundamental tools in machine learning that allow detection of non-linear dependencies between data without explicitly constructing feature vectors in high dimensional spaces. A major disadvantage of kernel methods is their poor scalabili ...

ASSOC COMPUTING MACHINERY2020

Kernel Density Estimation through Density Constrained Near Neighbor Search

Mikhail Kapralov, Navid Nouri

In this paper we revisit the kernel density estimation problem: given a kernel K(x, y) and a dataset of n points in high dimensional Euclidean space, prepare a data structure that can quickly output, given a query q, a (1 + epsilon)-approximation to mu := ...

IEEE2020

Scaling up Kernel Ridge Regression via Locality Sensitive Hashing

Mikhail Kapralov, Amir Zandieh, Navid Nouri

Random binning features, introduced in the seminal paper of Rahimi and Recht '07, are an efficient method for approximating a kernel matrix using locality sensitive hashing. Random binning features provide a very simple and efficient way to approximate the ...

ADDISON-WESLEY PUBL CO2020

Fourier Sampling in Signal Processing and Numerical Linear Algebra

Amir Zandieh

This thesis focuses on developing efficient algorithmic tools for processing large datasets. In many modern data analysis tasks, the sheer volume of available datasets far outstrips our abilities to process them. This scenario commonly arises in tasks incl ...

EPFL2020

Optimal Convergence for Distributed Learning with Stochastic Gradient Methods and Spectral Algorithms

Volkan Cevher, Junhong Lin

We study generalization properties of distributed algorithms in the setting of nonparametric regression over a reproducing kernel Hilbert space (RKHS). We first investigate distributed stochastic gradient methods (SGM), with mini-batches and multi-passes o ...

2020

Learning Positive Functions with Pseudo Mirror Descent

Negar Kiyavash

The nonparametric learning of positive-valued functions appears widely in machine learning, especially in the context of estimating intensity functions of point processes. Yet, existing approaches either require computing expensive projections or semidefin ...

Curran Associates, Inc.2019

A Generalized Representer Theorem for Hilbert Space - Valued Functions

Colin Neil Jones, Sanket Sanjay Diwale

The necessary and sufficient conditions for existence of a generalized representer theorem are presented for learning Hilbert space - valued functions. Representer theorems involving explicit basis functions and Reproducing Kernels are a common occurrence ...

2019

Decompositions Of Log-Correlated Fields With Applications

Janne Matias Junnila

In this article, we establish novel decompositions of Gaussian fields taking values in suitable spaces of generalized functions, and then use these decompositions to prove results about Gaussian multiplicative chaos. We prove two decomposition theorems. Th ...

2019

Neural Tanget Kernel: Convergence and Generalization in Neural Networks

Clément Hongler, Franck Raymond Gabriel, Arthur Ulysse Jacot-Guillarmod

At initialization, artificial neural networks (ANNs) are equivalent to Gaussian processes in the infinite-width limit [12, 9], thus connecting them to kernel methods. We prove that the evolution of an ANN during training can also be described by a kernel: ...

NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)2018

Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization

Sebastian Urban Stich

We present the first accelerated randomized algorithm for solving linear systems in Euclidean spaces. One essential problem of this type is the matrix inversion problem. In particular, our algorithm can be specialized to invert positive definite matrices i ...

2018