Dr. Cuong V. Nguyen

Assistant Professor, Durham University

Speaker: Dr. Cuong V. Nguyen Date: 25-02-2025 2pm-3pm (BST) Location: Department of Computer Science, CS1.01, University of Warwick, Coventry, UK

TBD

Abstract

Despite rapid advancement in deep learning research, performing full Bayesian analysis for deep learning still remains a challenge due to various reasons. Understanding these reasons from a theoretical viewpoint is thus important for applying Bayesian inference to modern deep learning models. In this talk, we will investigate theoretical properties of Hamiltonian Monte Carlo (HMC) on neural network models. We will show that despite its correctness, HMC sampling on ReLU-based networks is less sample efficient than sampling on networks with smooth activation functions. This new theoretical result allows us to obtain a guideline for tuning HMC that suggests a scaling for the step size and an optimal acceptance probability of 0.45 on ReLU-based neural networks. We also perform several simulations to demonstrate our theoretical results and suggest some future research directions into this problem.

---

About Dr. Cuong V. Nguyen

Cuong Nguyen is an assistant professor in statistics at Durham University. Before joining Durham, he was a faculty member at the Florida International University, an applied scientist at Amazon Web Services, and a postdoctoral researcher at Cambridge University. He received Bachelor and PhD degrees in computer science from the National University of Singapore. His research interests include both theoretical and practical aspects of machine learning, with applications to large-scale problems.