Generalization error

Formal sciences
Statistics
Data analysis
Cross-validation

About
Privacy
Disclaimer

Graph Chatbot

Related publications (29)

Consensus Control for Decentralized Deep Learning

Martin Jaggi, Tao Lin, Sebastian Urban Stich, Anastasiia Koloskova, Lingjing Kong

Decentralized training of deep learning models enables on-device learning over networks, as well as efficient scaling to large compute clusters. Experiments in earlier works reveal that, even in a data-center setup, decentralized training often suffers fro ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data

Martin Jaggi, Tao Lin, Sebastian Urban Stich, Sai Praneeth Reddy Karimireddy

Decentralized training of deep learning models is a key element for enabling data privacy and on-device learning over networks. In realistic learning scenarios, the presence of heterogeneity across different clients' local datasets poses an optimization ch ...

JMLR-JOURNAL MACHINE LEARNING RESEARCH2021

Partitioning soil organic carbon into its centennially stable and active fractions with machine-learning models based on Rock-Eval (R) thermal analysis (PARTY(SOC)v2.0 and PARTY(SOC)v2.0 (EU))

Juan Carlos Quezada Rivera

Partitioning soil organic carbon (SOC) into two kinetically different fractions that are stable or active on a century scale is key for an improved monitoring of soil health and for more accurate models of the carbon cycle. However, all existing SOC fracti ...

COPERNICUS GESELLSCHAFT MBH2021

CAFS: Cost-Aware Features Selection Method for Multimodal Stress Monitoring on Wearable Devices

David Atienza Alonso, Maria del Carmen Sandi Perez, Adriana Arza Valdes, João Pedro de Matos Rodrigues, Niloofar Momeni

Objective: Today, stress monitoring on wearable devices is challenged by the tension between high-detection accuracy and battery lifetime driven by multimodal data acquisition and processing. Limited research has addressed the classification cost on multim ...

2022

Robust Generalization via $\alpha$-Mutual Information

Michael Christoph Gastpar, Amedeo Roberto Esposito, Ibrahim Issa

The aim of this work is to provide bounds connecting two probability measures of the same event using Rényi

\alpha

-Divergences and Sibson’s

\alpha

-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon’s Mutual ...

ETHZ2020

On Vacuous and Non-Vacuous Generalization Bounds for Deep Neural Networks

Konstantinos Pitas

Deep neural networks have been empirically successful in a variety of tasks, however their theoretical understanding is still poor. In particular, modern deep neural networks have many more parameters than training data. Thus, in principle they should over ...

EPFL2020

Generalization error in high-dimensional perceptrons: Approaching Bayes error with convex optimization

Florent Gérard Krzakala, Lenka Zdeborová

We consider a commonly studied supervised classification of a synthetic dataset whose labels are generated by feeding a one-layer non-linear neural network with random iid inputs. We study the generalization performances of standard classifiers in the high ...

Curran Associates, Inc.2020

Scaling description of generalization with number of parameters in deep learning

Matthieu Wyart, Clément Hongler, Mario Geiger, Stefano Spigler, Levent Dogus Sagun, Franck Raymond Gabriel, Arthur Ulysse Jacot-Guillarmod, Arthur Jacot

Supervised deep learning involves the training of neural networks with a large number N of parameters. For large enough N, in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsitybased arguments would suggest th ...

IOP PUBLISHING LTD2020

AL2: Progressive Activation Loss for Learning General Representations in Classification Neural Networks

Sabine Süsstrunk, Majed El Helou, Frederike Dümbgen

The large capacity of neural networks enables them to learn complex functions. To avoid overfitting, networks however require a lot of training data that can be expensive and time-consuming to collect. A common practical approach to attenuate overfitting i ...

IEEE2020

Page 2 of 2