Lorenzo Rosasco
University of Genoa, MIT
Beyond the curse of dimensionality with hyper-kernel ridge regression
Speaker: Lorenzo Rosasco (Professor, University of Genoa) Date: 2pm, March 3rd, 2025 Location: Zeeman A1.01, University of Warwick, Coventry, UK
Abstract
Deep neural networks excel in high-dimensional problems, often surpassing models such as kernel methods, which suffer from the curse of dimensionality. However, the theoretical foundations of their success remain poorly understood. We propose that the compositional sparsity of the target function is the key factor determining when deep networks outperform other approaches. Taking a step toward formalizing this idea, we consider a simple sparse compositional model, namely the multi-index model. In this context, we introduce and study hyper-kernel ridge regression (HKRR), an approach blending neural networks and kernel methods. Our main contribution is sample complexity results demonstrating that HKRR can efficiently learn in settings where classical kernel methods fail, thus overcoming the curse of dimensionality. These theoretical findings are supported by numerical experiments.
About Lorenzo Rosasco
Lorenzo Rosasco is a professor at the University of Genova. He is a research affiliate at the Massachusetts Institute of Technology (MIT) and a visiting scientist at the Italian Technological Institute (IIT). He is a founder and serves as a coordinator of the Machine Learning Genova center (MaLGa) and the Laboratory for Computational and Statistical Learning, focusing on the theory, algorithms, and applications of machine learning. He obtained his PhD in 2006 from the University of Genova and was a visiting student at the Center for Biological and Computational Learning at MIT, the Toyota Technological Institute at Chicago (TTI-Chicago), and the Johann Radon Institute for Computational and Applied Mathematics. From 2006 to 2013, he worked as a postdoc and research scientist at the Brain and Cognitive Sciences Department at MIT. He is a fellow at Ellis and serves as the co-director of the “Theory, Algorithms and Computations of Modern Learning Systems” program as well as the Ellis Genoa unit. Lorenzo has received several awards, including an ERC consolidator grant.