Lectures related to Gradient Descent

Optimization on Manifolds

Covers optimization on manifolds, focusing on smooth manifolds and functions, and the process of gradient descent.

Differential Forms on Manifolds

Introduces differential forms on manifolds, covering tangent bundles and intersection pairings.

Riemannian connections

Explores Riemannian connections on manifolds, emphasizing smoothness and compatibility with the metric.

Connections: motivation and definition

Explores the definition of connections for smooth vector fields on manifolds.

Retractions vector fields and tangent bundles: Tangent bundles

Covers retractions, tangent bundles, and embedded submanifolds on manifolds with proofs and examples.

Differentiating vector fields: Why do it?

Explores the importance of differentiating vector fields and the correct methodology to achieve it, emphasizing the significance of going beyond the first order.

General Manifolds and Topology

Covers manifolds, topology, smooth maps, and tangent vectors in detail.

Riemannian metrics and gradients: Why and definition of Riemannian manifolds

Covers Riemannian metrics, gradients, vector fields, and inner products on manifolds.

Differential Forms Integration

Covers the integration of differential forms on smooth manifolds, including the concepts of closed and exact forms.

Topology of Riemann Surfaces

Covers the topology of Riemann surfaces, focusing on orientation and orientability.

Taylor expansions: second order

Explores Taylor expansions and retractions on Riemannian manifolds, emphasizing second-order approximations and covariant derivatives.

Optimization on Manifolds: Context and Applications

Introduces optimization on manifolds, covering classical and modern techniques in the field.

Manopt: Optimization Toolbox for Manifolds

Introduces Manopt, a toolbox for optimization on manifolds, focusing on solving optimization problems on smooth manifolds using the Matlab version.

Riemannian metrics and gradients: Riemannian gradients

Explains Riemannian submanifolds, metrics, and gradients computation on manifolds.

Non-analytic Smooth Functions

Explores non-analytic smooth functions, their properties, and applications in differential geometry and partitioning unity.

Taylor Expansions: First Order

Explores Taylor expansions of first order in optimization on manifolds.

Momentum methods and nonlinear CG

Explores gradient descent with memory, momentum methods, conjugate gradients, and nonlinear CG on manifolds.

Newton's method: Optimization on manifolds

Explores Newton's method for optimizing functions on manifolds using second-order information and discusses its drawbacks and fixes.

Optimality conditions: second order

Explores necessary and sufficient optimality conditions for local minima on manifolds, focusing on second-order critical points.

Riemannian metrics and gradients: Examples and Riemannian submanifolds

Explores Riemannian metrics on manifolds and the concept of Riemannian submanifolds in Euclidean spaces.