Introduction to Control and Machine Learning
Master Course – FAU Erlangen–Nürnberg
This course explores the deep connections between control theory, dynamical systems, and modern machine learning, highlighting how mathematical tools developed for the analysis of differential equations can help understand and design modern AI systems.
Course Overview
The course introduces the mathematical foundations of dynamical systems governed by ordinary and partial differential equations (ODEs and PDEs) and explains how classical ideas from control theory—such as controllability, observability, and stability—provide valuable insight into modern machine learning architectures.
Special attention is given to understanding the dynamics of:
• deep neural networks
• federated learning systems
• large language models (LLMs)
Through this perspective, the course highlights how concepts developed in control theory and applied mathematics help interpret and analyze modern learning systems.
Instructor
Prof. Enrique Zuazua
FAU Erlangen–Nürnberg
The course has been offered since the Summer Semester 2021–2022.
Teaching Assistants and Contributors
The course has benefited from the contributions of:
• Dr. Martín Hernández, UCLA
• Dr. Zhengping Ji – FAU Erlangen–Nürnberg
• Dr. Young Song – Nanyang Technological University, Singapore
• Ziqi Wang – PhD Candidate, FAU Erlangen–Nürnberg
Their contributions significantly enriched the course through computational demonstrations and supplementary material.
Learning Objectives
By the end of the course, students will be able to:
• understand the mathematical foundations of dynamical systems used in machine learning
• analyze the optimization dynamics of deep learning algorithms
• interpret neural network architectures using control-theoretic concepts
• understand how large-scale learning systems can be analyzed using dynamical systems tools
• present advanced technical material clearly and rigorously.
Course Prerequisites
Students are expected to have a basic background in:
• calculus
• linear algebra
• ordinary differential equations
• partial differential equations
Additional knowledge of
• functional analysis
• numerical analysis
• calculus of variations
is helpful but not required.
Course Materials
Lecture Slides
The main lecture materials include:
• General course introduction
• Fundamentals of Optimal Control (pdf)
• Optimization Methods in Machine Learning (pdf)
• Wiener’s Gaussian Ansatz (pdf)
• MATLABpytorch (.zip)
• Tutorial
• PyTorch Tutorial (.zip)
Student Projects
Students complete final projects exploring topics at the intersection of control theory and machine learning.
Example topics include:
• Nature-inspired optimization algorithms
• Federated learning systems
• Reinforcement learning for control problems
• Physics-informed neural networks
• Deep learning architectures for time-series prediction
• Transformers and large language models
• Graph neural networks and path planning
• Interpretability methods in machine learning
• Neural ODE models
These projects allow students to explore advanced topics and develop presentation skills.
References
The course material draws on textbooks, research articles, and online resources from both control theory and machine learning.
Control Theory
Coron, J.-M. (2009). Control and Nonlinearity. AMS.
Nocedal, J., & Wright, S. (2006). Numerical Optimization. Springer.
Zuazua, E. (2005). Propagation, observation, and control of waves approximated by finite difference methods. SIAM Review.
Zuazua, E. (2006). Controllability and observability of PDEs. Handbook of Differential Equations.
Machine Learning
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
Bottou, L., Curtis, F., & Nocedal, J. (2018). Optimization methods for large-scale machine learning. SIAM Review.
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning. Springer.
Higham, C., & Higham, D. (2019). Deep learning: An introduction for applied mathematicians. SIAM Review.
Ruiz-Balet, D., & Zuazua, E. (2023). Neural ODE control for classification, approximation and transport. SIAM Review.
Online Educational Resources
Sanderson, G.
Neural Networks – 3Blue1Brown
https://www.3blue1brown.com/topics/neural-networks
Computational Resources
Stevens, E., Antiga, L., & Viehmann, T. (2020). Deep Learning with PyTorch. Manning.
PyTorch Tutorials
https://docs.pytorch.org/tutorials
Historical Perspective
Fernández-Cara, E., & Zuazua, E. (2003). Control theory: history and perspectives.
Fradkov, A. L. (2020). Early history of machine learning.
Expository and Outreach Articles
Zuazua, E. (2022). Control and Machine Learning. SIAM News.
