From Optimal Control to Reinforcement Learning: A Motivating
Example from the Perspective of Linear Quadratic Regulators
By Fredson Silva De Souza Aguiar, Visiting Master Student. University of Sorbonne Paris NordRepository with code and trained models available at FAU DCN-AvH GitHub: https://github.com/DCN-FAU-AvH/oc_rl_lqr
We motivate the study of interconnections and relevant results at the interface between Control Theory and Reinforcement Learning, with a special focus on the class of problems known as Linear Quadratic Regulators. We seek to motivate the existence of continuous paths joining techniques and characteristics from the two communities that initially seem to lie in separate universes. By doing so, we highlight how the two fields are not only complementary but, in fact, can benefit greatly from each other.
We start our discussion by walking the reader between two extremes: fully model-based optimal control and fully model-free reinforcement learning.
We highlight the advantages and disadvantages of each method, and motivate the existence of intermediate methodologies.
Trough a final mixed methodology, we show the power and importance of data-driven techniques, but also how theoretical results are fundamental to justify and guide efficient solutions. This example also justifies how classical optimal control and reinforcement learning are, in fact, part of a single continuum; even further: their frontier is blurry.
Finally, the reproducible codes and trained models used in these notes are made available in the following repository: https://github.com/DCN-FAU-AvH/oc_rl_lqr
|| Go to the Academy main page
