Publications related to Sigmoid function

Machine Learning for Modeling Stock Returns

Throughout history, the pace of knowledge and information sharing has evolved into an unthinkable speed and media. At the end of the XVII century, in Europe, the ideas that would shape the "Age of Enlightenment" were slowly being developed in coffeehouses, ...

EPFL2024

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

What can linearized neural networks actually say about generalization?

Pascal Frossard, Seyed Mohsen Moosavi Dezfooli, Guillermo Ortiz Jimenez

For certain infinitely-wide neural networks, the neural tangent kernel (NTK) theory fully characterizes generalization, but for the networks used in practice, the empirical NTK only provides a rough first-order approximation. Still, a growing body of work ...

2021

Disentangling feature and lazy training in deep neural networks

Matthieu Wyart, Mario Geiger, Stefano Spigler, Arthur Ulysse Jacot-Guillarmod, Arthur Jacot

Two distinct limits for deep learning have been derived as the network width h -> infinity, depending on how the weights of the last layer scale with h. In the neural tangent Kernel (NTK) limit, the dynamics becomes linear in the weights and is described b ...

IOP PUBLISHING LTD2020

ESSOP: Efficient and Scalable Stochastic Outer Product Architecture for Deep Learning

Irem Boybat Kara, Evangelos Eleftheriou, Abu Sebastian

Deep neural networks (DNNs) have surpassed human-level accuracy in a variety of cognitive tasks but at the cost of significant memory/time requirements in DNN training. This limits their deployment in energy and memory limited applications that require rea ...

IEEE2020

Learning Activation Functions in Deep (Spline) Neural Networks

Michaël Unser, Pakshal Narendra Bohra, Shayan Aziznejad, Harshit Gupta, Joaquim Gonçalves Garcia Barreto Campos

We develop an efficient computational solution to train deep neural networks (DNN) with free-form activation functions. To make the problem well-posed, we augment the cost functional of the DNN by adding an appropriate shape regularization: the sum of the ...

2020

Artificial neural networks can be used for Ambrosia pollen emission parameterization in COSMO-ART

Charlotte Marie Bürki

The emission parameterization is a crucial part of numerical pollen dispersion models. This paper shows that Artificial Neural Networks (ANNs) can substantially improve the performance of the Ambrosia pollen emission in numerical pollen dispersion models s ...

Elsevier2019

Logistic regression over encrypted data from fully homomorphic encryption

Zhicong Huang

Background: One of the tasks in the 2017 iDASH secure genome analysis competition was to enable training of logistic regression models over encrypted genomic data. More precisely, given a list of approximately 1500 patient records, each with 18 binary feat ...

2018

Optimization of high order perceptrons

Neural networks are widely applied in research and industry. However, their broader application is hampered by various technical details. Among these details are several training parameters and the choice of the topology of the network. The subject of this ...

EPFL1997