Publication
Neural Networks (NNs), renowned for their universal approximation capabilities, have proven well-suited for parametrizing nonlinear control policies and modeling complex processes from available data. However, several NN architectures are physics-agnostic, susceptible to small input changes, and cumbersome to endow with desired properties, which limits their application, particularly in modeling and safety-critical control frameworks.
In this thesis, we propose leveraging advanced notions from dynamical systems theory, such as contractivity and dissipativity, to make NNs dependable for control, system identification, and image classification tasks.
Here, "dependable" refers to NNs' ability to perform consistently as intended, backed by formal guarantees and certificates.
For instance, in control, this means ensuring closed-loop stability and robustness, while in system identification, it involves learning models that are stable or physically consistent.
Recent works have explored connections between NNs and dynamical systems, leading to the development of Neural Ordinary Differential Equations (NODEs), which transform input data through a continuous-time ODE over a finite time interval, embedding optimization parameters.
This makes NODEs capable of coping with irregularly sampled data, efficient in the number of parameters, and allows the use of dynamical systems theory to analyze and modify their properties.
Particularly, we aim at deriving NNs/NODEs architectures enjoying unconstrained parametrizations, i.e. possessing desired properties for all possible parameter values.
This approach enables the use of standard optimization routines to tune the NNs/NODEs parameters, eliminating the need for a posteriori property verification, parameter projections, and constrained optimization. Consequently, unconstrained parametrization facilitates computationally efficient and scalable training.
The key results presented in this dissertation are the following ones.
First, we introduce a class of NODE-based distributed controllers that are dissipative by design and, moreover, we harness well-established results from dissipation theory to ensure closed-loop stability. Second, considering the fragility of image classification pipelines, we leverage contraction theory to enhance the robustness of NODEs against input noise.
Third, we propose NODEs-based System Identification Methods by leveraging Backpropagation (SIMBa) to identify linear or nonlinear models from data, where the identified model has desired properties such as stability, sparsity, or adherence to physical laws such as those of thermodynamics. We showcase that by incorporating prior knowledge, one can outperform state-of-the-art system identification methods on both simulated and real-world data.
Altogether, this thesis paves the path to explore the synergy between NNs and systems theory, by identifying machine learning models that facilitate more dependable applications across various do