BEGIN:VCALENDAR
VERSION:2.0
METHOD:PUBLISH
CALSCALE:GREGORIAN
PRODID:-//WordPress - MECv7.36.4//EN
X-ORIGINAL-URL:https://dcn.nat.fau.eu/
X-WR-CALNAME:FAU DCN-AvH
X-WR-CALDESC:Chair for Dynamics, Control, Machine Learning and Numerics -Alexander von Humboldt Professorship
X-WR-TIMEZONE:Europe/Berlin
BEGIN:VTIMEZONE
TZID:Europe/Berlin
X-LIC-LOCATION:Europe/Berlin
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:20260329T030000
RRULE:FREQ=YEARLY;BYMONTH=03;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:20261025T020000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=4SU
END:STANDARD
END:VTIMEZONE
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-PUBLISHED-TTL:PT1H
X-MS-OLK-FORCEINSPECTOROPEN:TRUE
BEGIN:VEVENT
CLASS:PUBLIC
UID:MEC-74349b2e19170fe0bbb4feb6f5720c5a@dcn.nat.fau.eu
DTSTART;TZID=Europe/Berlin:20261026T130000
DTEND;TZID=Europe/Berlin:20261026T150000
DTSTAMP:20261009T193101Z
CREATED:20261009
LAST-MODIFIED:20261009
PRIORITY:5
SEQUENCE:14
TRANSP:OPAQUE
SUMMARY:PhD Thesis defense by A. Alcalde
DESCRIPTION:Next Monday October 26, 2026 our team member Albert Alcalde will present his PhD Thesis on:\n“A Dynamical Perspective on Transformers, with Applications to Data-Driven Modeling”\nAdvisors:\n• Prof. Giovanni Fantuzzi, FAU – Friedrich Alexander-Universität Erlangen-Nürnberg (Germany)\n• Prof. Enrique Zuazua ( https://dcn.nat.fau.eu/zuazua/ ), FAU – Friedrich Alexander-Universität Erlangen-Nürnberg (Germany)\nAbstract. Transformers form a central class of architectures in modern deep learning, where interpolating maps are built primarily through the alternation of simpler parametric functions known as self-attention and feed-forward layers. This thesis advances the study of these models from a dynamical perspective: inputs are interpreted as collections of particles, layers as time steps, self-attention as a nonlinear interaction rule, and feed-forward layers as pointwise forcing terms. The goal is to characterize rigorous asymptotic regimes of the forward dynamics of Transformers in terms of their parameters and to connect them with interpretable methods for data-driven modeling.\nThe first part of the thesis analyzes the key components of Transformer models and characterizes their dynamics in a range of parameter regimes. In a pure-attention model where softmax self-attention layers are replaced by the hardmax rule, the dynamics admit a clear geometric interpretation: particles or tokens converge to cluster points determined by special tokens called leaders. The same hardmax limit connects self-attention with the well-known Frank–Wolfe optimization scheme, in which a functional is minimized over a convex set, given here by the convex hull of the tokens. This connection yields convergence rates and relates the clustering results to the concentration phenomena observed empirically in softmax Transformers.\nThe thesis uses the resulting insight into the geometry of attention to study the expressive power of Transformers through exact sequence interpolation, where finite collections of input tokens must be driven exactly to prescribed target collections of output tokens. Specifically, it proves that both hardmax and softmax Transformers can exactly interpolate such datasets, with explicit bounds on the number of layers and the total parameter count. These results quantitatively characterize when sufficiently deep models can perfectly memorize a training dataset.\nMotivated by the long input sequences processed by modern large language models, the second part of the thesis then studies the infinite-token limit of Transformers through a mean-field formulation. By considering the resulting continuity equation, which describes the evolution of token distributions across layers, it obtains quantitative concentration rates as a function of the model parameters. For Gaussian input distributions and affine feed-forward layers, the continuity equation reduces to a system of differential equations for the mean and covariance. The reachability and asymptotic dynamics of this reduced model are studied through their connection with Riccati-type dynamics. This analysis identifies parameter regimes in which the forward dynamics are well-posed in terms of the signs of the parameter matrices, a property relevant to the stability of deep models.\nThe third part of the thesis is devoted to applications in data-driven modeling, where the goal is to build evolution models from trajectory observations. Leveraging the theoretical study of Transformers, the thesis first proposes a minimal attention-based architecture for learning dynamics. Its minimality allows it to be interpreted as a nonlinear extension of the linear baseline of time-delayed dynamic mode decomposition. The proposed method shows improved performance on chaotic and weakly chaotic datasets, where its nonlinear structure provides additional predictive power compared with the linear baseline.\nFinally, the thesis considers the more challenging problem of learning a model with an additional certificate of boundedness of the trajectories. This task is addressed by jointly learning polynomial models and a Lyapunov function that determines a trapping region. This results in a system identification method that provably discovers bounded polynomial ODEs and performs robustly on benchmark problems in ODE discovery.\nWHEN\nMon. October 26, 2026 at 13:00H (Berlin time)\nWHERE\nOn-site. FAU\nBoard of examiners\n|| You might like: Upcoming events\n\n.mec-single-event .mec-events-event-image{\n    margin-top: 20px;\n  }\n\n
URL:https://dcn.nat.fau.eu/events/phd-thesis-defense-by-a-alcalde/
ORGANIZER;CN=FAU - Faculty of Sciences:MAILTO:
CATEGORIES:EZuazua,PhD,Seminar/Talk
ATTACH;FMTTYPE=image/png:https://dcn.nat.fau.eu/wp-content/uploads/FAUDCNAvH_phDthesisDefense_aAlcalde_26oct2026_img.png
END:VEVENT
END:VCALENDAR