FAU MoD Lecture Series. Special December 2024

Date: Tue. December 03, 2024
Event: FAU MoD Lecture Series (Special double edition. December 2024)
Organized by: FAU MoD, the Research Center for Mathematics of Data at Friedrich-Alexander-Universität Erlangen-Nürnberg (Germany)

Session 01: 14:30H
FAU MoD Lecture: The implicit bias phenomenon in deep learning
Speaker: Prof. Dr. Holger Rauhut
Affiliation: Mathematisches Institut der Universität München (Germany)

Abstract. Deep neural networks are usually trained by minimizing a non-convex loss functional via (stochastic) gradient descent methods. Unfortunately, the convergence properties are not very well-understood. Moreover, a puzzling empirical observation is that learning neural networks with a number of parameters exceeding the number of training examples often leads to zero loss, i.e., the network exactly interpolates the data. Nevertheless, it generalizes very well to unseen data, which is in stark contrast to intuition from classical statistics which would predict a scenario of overfitting.

A current working hypothesis is that the chosen optimization algorithm has a significant influence on the selection of the learned network. In fact, in this overparameterized context there are many global minimizers so that the optimization method induces an implicit bias on the computed solution. It seems that gradient descent methods and their stochastic variants favor networks of low complexity (in a suitable sense to be understood), and, hence, appear to be very well suited for large classes of real data.

Initial attempts in understanding the implicit bias phenomen considers the simplified setting of linear networks, i.e., (deep) factorizations of matrices. This has revealed a surprising relation to the field of low rank matrix recovery (a variant of compressive sensing) in the sense that gradient descent favors low rank matrices in certain situations. Moreover, restricting further to diagonal matrices, or equivalently factorizing the entries of a vector to be recovered, shows connection to compressive sensing and l1-minimization.

-Coffee/tea break-

Session 02: 16:30H
FAU MoD Lecture: Counterintuitive approximations
Speaker: Prof. Dr. Christian Bär
Affiliation: Institut für Mathematik. Universität Potsdam (Germany)

Abstract. One of the oldest questions in differential geometry is whether every curved space arises as a subspace of Euclidean space ℝⁿ. More precisely, does every Riemannian manifold admit an isometric (i.e. length preserving) smooth embedding into ℝⁿ? Nash showed that the answer is yes but the dimension n of the Euclidean space may be much larger than that of the manifold. Often there are non-isometric embeddings of much lower codimension.

The Nash-Kuiper embedding theorem is a prototypical example of a counterintuitive approximation result in geometry: any short (but highly non-isometric) embedding of a Riemannian manifold into Euclidean space can be approximated by isometric embeddings. They are generally not smooth but of regularity C¹. This implies that any surface with a given geometry can be isometrically C¹-embedded into an arbitrarily small ball in ℝ³. For C²-embeddings this completely false due to curvature restrictions.

After explaining this historical and conceptual context, I will present a general result which ensures approximations by maps satisfying strongly overdetermined equations (such as being isometric) on open dense subsets. This will be illustrated by three examples: Lipschitz functions with surprising derivative, surfaces in 3-space with unexpected curvature properties, and a similar statement for abstract Riemannian metrics on manifolds. Our method is based on “cut-oﬀ homotopy”, a concept introduced by Gromov in 1986.

This is based on joint work with Bernhard Hanke.

OUR SPEAKERS

Holger Rauhut
Prof. Dr. Holger Rauhut studied Mathematics at TU Munich, where he obtained also his doctorate degree in 2004. After a period as postdoc, he received the habilitation from the University of Vienna in 2008. He held professor positions at the University of Bonn and RWTH Aachen University. In 2023, he joined LMU Munich where he is the head of the Chair of Mathematics of Information Processing. He received an ERC Starting Grant in 2010. He was spokesperson of the SFB 1481 Sparsity and Singular Structures at RWTH Aachen University 2022-2023. His research focuses on the mathematical foundations of machine learning and of signal processing. This includes aspects of high-dimensional probability, of optimization, of harmonic analysis and of the analysis of algorithms in these fields.

Christian Bär
Prof. Dr. Christian Bär holds a PhD degree (1990) and a habilitation (1993) from the University of Bonn. He held professor positions in Freiburg and Hamburg. Since 2003 he is professor for geometry at the University of Potsdam. He was president of the Deutsche Mathematiker-Vereinigung and an elected member of the mathematics panel of the Deutsche Forschungsgemeinschaft. Currently, he is editor-in-chief of zbmath Open. His research interests are centered around differential geometry, global analysis and applications to mathematical physics.

See poster

AUDIENCE

This is a hybrid event (On-site/online) open to: Public, Students, Postdocs, Professors, Faculty, Alumni and the scientific community all around the world.

WHEN

Tue. December 03, 2024 at 14:30H (Berlin time)

WHERE

On-site / Online

[On-site] Friedrich-Alexander-Universität Erlangen-Nürnberg
Felix Klein building. Department Mathematik
Room H13 Johann-Radon-Hörsaal
Cauerstraße 11, 91058 Erlangen
GPS-Koord. Raum: 49.579737N, 11.029862E

[Online] FAU Zoom link: https://go.fau.de/1bcfg
Meeting ID: 667 9081 1368 | PIN code: 716845

* Photo credits (Prof. Bär): Tobias Hopfgarten