FAU DCN-AvH Seminar: Generalization bounds for neural ODEs and Clustering in discrete-time self-attention

Next Thursday July 18, 2024:

Organized by: FAU DCN-AvH, Chair for Dynamics, Control, Machine Learning and Numerics – Alexander von Humboldt Professorship at FAU, Friedrich-Alexander-Universität Erlangen-Nürnberg (Germany)

Title: Generalization bounds for neural ODEs and Clustering in discrete-time self-attention
Speaker: Lucas Versini
Affiliation: Visiting student at FAU DCN-AvH from École Polytechnique

Abstract. This presentation details the findings from two machine learning research projects.

The first project focuses on neural Ordinary Differential Equations (neural ODEs), which serve as continuous-time analogs to traditional neural networks. By establishing generalization bounds, the study aims to elucidate the predictive performance of neural ODEs on unseen data. Theoretical results were corroborated through numerical experiments conducted on synthetic datasets.

The second project examines the long-time behavior of discrete-time self-attention mechanisms commonly employed in language and vision tasks. We demonstrate that iterative application of self-attention mechanisms can lead to convergence towards a single cluster. Numerical experiments support these findings, leading to conjectures that suggest possible directions for future research, despite not being fully proven during the course of this internship.

WHEN

Thu. July 18, 2024 at 14:00H

WHERE

Room H13
Friedrich-Alexander-Universität Erlangen-Nürnberg

_
See all Seminars at FAU DCN-AvH

Don’t miss out our last news and connect with us!

LinkedIn | Twitter | Instagram