The Foundations of AI Seminar Series is dedicated to topics of interest in artificial intelligence, machine learning, both empirically and theoretically, as well as related areas. Our goal is for these meetings to serve as a forum for discussions and quick dissemination of results. We invite anyone interested in the latest advancements in AI/ML to join us!

Next Seminar


Conservation Laws for Gradient Flows

Gabriel Peyre

Speaker: Gabriel Peyre (CNRS Researcher, DMA/ENS, France) Date: 2pm, June 14th, 2024 Location: MB0.07, University of Warwick, Coventry, UK

Download iCalendar File

Abstract

Understanding the geometric properties of gradient descent dynamics is a key ingredient in deciphering the recent success of very large machine learning models. A striking observation is that trained over-parameterized models retain some properties of the optimization initialization. This “implicit bias” is believed to be responsible for some favorable properties of the trained models and could explain their good generalization properties. In this talk I will first rigorously expose the definition and basic properties of “conservation laws”, which are maximal sets of independent quantities conserved during gradient flows of a given model (e.g. of a ReLU network with a given architecture) with any training data and any loss. Then I will explain how to find the exact number of these quantities by performing finite-dimensional algebraic manipulations on the Lie algebra generated by the Jacobian of the model. In the specific case of linear and ReLu networks, this procedure recovers the conservation laws known in the literature, and prove that there are no other laws. This is a joint work with Sibylle Marcotte and Rémi Gribonval. The associated paper can be found here: https://arxiv.org/pdf/2307.00144


About Gabriel Peyre

Gabriel Peyré is CNRS senior researcher and professor at the Ecole Normale Supérieure, Paris. He works at the interface between applied mathematics, imaging and machine learning. He obtained 2 ERC grants (starting in 2010 and consolidator in 2017), the Blaise Pascal prize from the French academy of sciences in 2017, the Magenes Prize from the Italian Mathematical Union in 2019 and the silver medal from CNRS in 2021. He is invited speaker at the European Congress for Mathematics in 2020. He is the deputy director of the Prairie Institute for artificial intelligence, the director of the ENS center for data science and the former director of the GdR CNRS MIA. He is the head of the ELLIS (European Lab for Learning & Intelligent Systems) Paris Unit. He is engaged in reproducible research and code education, in particular through the platform .

Upcoming Events


Speaker Image

Conservation Laws for Gradient Flows

Gabriel Peyré - CNRS researcher| DMA/ENS, France
Calendar Icon Jun 14, 2024 at 2:00pm
Location Icon Mathematical Sciences Building, MB0.07

More Info
Speaker Image

Physics-informed machine learning as a kernel method

Francis Bach - Researcher at Inria
Calendar Icon Jun 18, 2024 at 11:00am
Location Icon Mathematical Sciences Building, MB0.07

More Info
Speaker Image

TBD

René Vidal - Rachleff University Professor, University of Pennsylvania, USA
Calendar Icon Jun 19, 2024 at 3:00PM
Location Icon MB0.07

More Info
Speaker Image

TBD

Ilija Bogunovic - Assistant Professor, UCL, UK
Calendar Icon Oct 11, 2024 at 2:00PM
Location Icon TBD

More Info
Speaker Image

Causal Effect Estimation with Context and Confounders

Arthur Gretton - Professor, UCL, UK
Calendar Icon TBD
Location Icon TBD
*** This talk will be rescheduled. ***

More Info

Organising Team

Fanghui Liu

Fanghui Liu

Assistant Professor, CS Department, University of Warwick

Paris Giampouras

Paris Giampouras

Assistant Professor, CS Department, University of Warwick

Long Tran-Thanh

Long Tran-Thanh

Associate Professor, CS Department, University of Warwick