Publications associées à Perceptron multicouche

MaskCLR: Attention-Guided Contrastive Learning for Robust Action Representation Learning

Alexandre Massoud Alahi, Mohamed Ossama Ahmed Abdelfattah, Mariam Ahmed Mahmoud Hegazy Hassan

Current transformer-based skeletal action recognition models tend to focus on a limited set of joints and low-level motion patterns to predict action classes. This results in significant performance degradation under small skeleton perturbations or changin ...

2024

Deep Learning Theory Through the Lens of Diagonal Linear Networks

Scott William Pesme

In this PhD manuscript, we explore optimisation phenomena which occur in complex neural networks through the lens of

2

-layer diagonal linear networks. This rudimentary architecture, which consists of a two layer feedforward linear network with a diagonal ...

EPFL2024

Robust NAS under adversarial training: benchmark, theory, and beyond

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Yongtao Wu

Recent developments in neural architecture search (NAS) emphasize the significance of considering robust architectures against malicious data. However, there is a notable absence of benchmark evaluations and theoretical guarantees for searching these robus ...

2024

Machine Learning for Modeling Stock Returns

Teng Andrea Xu

Throughout history, the pace of knowledge and information sharing has evolved into an unthinkable speed and media. At the end of the XVII century, in Europe, the ideas that would shape the "Age of Enlightenment" were slowly being developed in coffeehouses, ...

EPFL2024

Student Answer Forecasting: Transformer-Driven Answer Choice Prediction for Language Learning

Paola Mejia Domenzain, Vinitra Swamy, Tanja Christina Käser Jacober, Elena Grazia Gado, Tommaso Martorella, Jibril Albachir Frej, Luca Zunino

Intelligent Tutoring Systems (ITS) enhance personalized learning by predicting student answers to provide immediate and customized instruction. However, recent research has primarily focused on the correctness of the answer rather than the student's perfor ...

2024

ReLU Neural Network Galerkin BEM

Fernando José Henriquez Barraza

We introduce Neural Network (NN for short) approximation architectures for the numerical solution of Boundary Integral Equations (BIEs for short). We exemplify the proposed NN approach for the boundary reduction of the potential problem in two spatial dime ...

SPRINGER/PLENUM PUBLISHERS2023

A new variable shape parameter strategy for RBF approximation using neural networks

Jan Sickmann Hesthaven

The choice of the shape parameter highly effects the behaviour of radial basis function (RBF) approximations, as it needs to be selected to balance between the ill-conditioning of the interpolation matrix and high accuracy. In this paper, we demonstrate ho ...

PERGAMON-ELSEVIER SCIENCE LTD2023

The inductive bias of deep learning: Connecting weights and functions

Guillermo Ortiz Jimenez

Years of a fierce competition have naturally selected the fittest deep learning algorithms. Yet, although these models work well in practice, we still lack a proper characterization of why they do so. This poses serious questions about the robustness, trus ...

EPFL2023

Benign Overfitting in Deep Neural Networks under Lazy Training

Volkan Cevher, Grigorios Chrysos, Fanghui Liu, Zhenyu Zhu

This paper focuses on over-parameterized deep neural networks (DNNs) with ReLU activation functions and proves that when the data distribution is well-separated, DNNs can achieve Bayesoptimal test error for classification while obtaining (nearly) zero-trai ...

2023

A Neural-Network-Based Convex Regularizer for Inverse Problems

Michaël Unser, Pakshal Narendra Bohra, Alexis Marie Frederic Goujon, Sebastian Jonas Neumayer, Stanislas Ducotterd

The emergence of deep-learning-based methods to solve image-reconstruction problems has enabled a significant increase in quality. Unfortunately, these new methods often lack reliability and explainability, and there is a growing interest to address these ...

2023

Privacy-preserving federated neural network training and inference

Sinem Sav

Training accurate and robust machine learning models requires a large amount of data that is usually scattered across data silos. Sharing, transferring, and centralizing the data from silos, however, is difficult due to current privacy regulations (e.g., H ...

EPFL2023

Stable parameterization of continuous and piecewise-linear functions

Michaël Unser, Alexis Marie Frederic Goujon, Joaquim Gonçalves Garcia Barreto Campos

Rectified-linear-unit (ReLU) neural networks, which play a prominent role in deep learning, generate continuous and piecewise-linear (CPWL) functions. While they provide a powerful parametric representation, the mapping between the parameter and function s ...

2023

Text Representation Learning for Low Cost Natural Language Understanding

Jan Frederik Jonas Florian Mai

Natural language processing and other artificial intelligence fields have witnessed impressive progress over the past decade. Although some of this progress is due to algorithmic advances in deep learning, the majority has arguably been enabled by scaling ...

EPFL2023

RMAML: Riemannian meta-learning with orthogonality constraints

Soumava Kumar Roy

Meta-learning is the core capability that enables intelligent systems to rapidly generalize their prior ex-perience to learn new tasks. In general, the optimization-based methods formalize the meta-learning as a bi-level optimization problem, that is a nes ...

ELSEVIER SCI LTD2023

An exact mapping from ReLU networks to spiking neural networks

Wulfram Gerstner, Stanislaw Andrzej Wozniak, Ana Stanojevic, Giovanni Cherubini, Angeliki Pantazi

Deep spiking neural networks (SNNs) offer the promise of low-power artificial intelligence. However, training deep SNNs from scratch or converting deep artificial neural networks to SNNs without loss of performance has been a challenge. Here we propose an ...

2023

2D Nanosystems: Applications of 2D Semiconductors for In-Memory Computing

Guilherme Migliato Marega

Machine learning and data processing algorithms have been thriving in finding ways of processing and classifying information by exploiting the hidden trends of large datasets. Although these emerging computational methods have become successful in today's ...

EPFL2023

Deep Learning Detection of GPS Spoofing

Mirjana Stojilovic, Olivia Jullian Parra

Unmanned aerial vehicles (UAVs) are widely deployed in air navigation, where numerous applications use them for safety-of-life and positioning, navigation, and timing tasks. Consequently, GPS spoofing attacks are more and more frequent. The aim of this wor ...

Springer, Cham2022

A Deep-Learning Approach to Side-Channel Based CPU Disassembly at Design Time

Mirjana Stojilovic

Side-channel CPU disassembly is a side-channel attack that allows an adversary to recover instructions executed by a processor. Not only does such an attack compromise code confidentiality, it can also reveal critical information on the system’s internals. ...

IEEE2022

Exploring quantum perceptron and quantum neural network structures with a teacher-student scheme

Near-term quantum devices can be used to build quantum machine learning models, such as quantum kernel methods and quantum neural networks (QNN), to perform classification tasks. There have been many proposals on how to use variational quantum circuits as ...

2022

Sharp asymptotics on the compression of two-layer neural networks

Marco Mondelli

In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M < N nodes. More precisely, we consider the setting in which the weights of the target network are i.i.d. sub-Gaussian, and we minimiz ...

IEEE2022