Next Thursday July 18, 2024:
Organized by: FAU DCN-AvH, Chair for Dynamics, Control, Machine Learning and Numerics – Alexander von Humboldt Professorship at FAU, Friedrich-Alexander-Universität Erlangen-Nürnberg (Germany)
Title: Generalization bounds for neural ODEs and Clustering in discrete-time self-attention
Speaker: Lucas Versini
Affiliation: Visiting student at FAU DCN-AvH from École Polytechnique
Abstract. This presentation details the findings from two machine learning research projects.
The first project focuses on neural Ordinary Differential Equations (neural ODEs), which serve as continuous-time analogs to traditional neural networks. By establishing generalization bounds, the study aims to elucidate the predictive performance of neural ODEs on unseen data. Theoretical results were corroborated through numerical experiments conducted on synthetic datasets.
The second project examines the long-time behavior of discrete-time self-attention mechanisms commonly employed in language and vision tasks. We demonstrate that iterative application of self-attention mechanisms can lead to convergence towards a single cluster. Numerical experiments support these findings, leading to conjectures that suggest possible directions for future research, despite not being fully proven during the course of this internship.
WHEN
Thu. July 18, 2024 at 14:00H
WHERE
Room H13
Friedrich-Alexander-Universität Erlangen-Nürnberg
_
See all Seminars at FAU DCN-AvH
Don’t miss out our last news and connect with us!
