Speaker: Molei Tao, Georgia Tech
Title: "Where do all the scores come from? - an end-to-end accuracy analysis of diffusion model, and multimodal sampling via diffusion annealing"
Abstract:
Diffusion model is a prevailing Generative AI technology. It uses a score function to characterize how a complicated data distribution morphs into an easy distribution. This talk will mainly report progress in two topics based on diffusion model, both closely related to how the score function is approximated.
The first topic will focus on a quantifying the generation accuracy of diffusion model. The importance of this problem already led to a rich and substantial literature; however, most existing theoretical investigations assumed that an epsilon-accurate score function has already been oracle-given, and focused on just the inference process of diffusion model. I will instead describe a first quantitative understanding of the end-to-end generative modeling protocol, including both score training (optimization) and inference (sampling). The resulting error analysis will elucidate, theoretically, how to design the training and inference processes for effective generation.
Part 2 of the talk will discuss some recent progress toward a holy-grail in computational statistics, namely the sampling problem. The goal is leverage the fact that diffusion model is very good at handling multimodal distributions, and extrapolate this capability to the challenging problem of efficient sampling from a multimodal density. There, one needs to rethink about how to obtain the score function, as no more data samples are available and one instead has an unnormalized density. A new sampler that is insensitive to metastability, with performance guarantee, and only requiring zeroth-order oracle, will be presented.
If time permits, progress beyond the aforementioned results will also be mentioned, including quantifying the generalizability of diffusion model, and high-dim multimodal sampling powered by diffusion model, stochastic optimal control, and deeping learning.
Bio:
Molei Tao is a Professor of mathematics and machine learning and Richard Duke Fellow at Georgia Tech. He is also one of the three founding directors of GT AI4Science Center. Molei received B.S. from Tsinghua Univ. and Ph.D. from Caltech. He then worked as a Courant Instructor and then an assistant, associate, and full professor at Georgia Tech. He is a recipient of W.P. Carey Ph.D. Prize in Applied Mathematics (2011), American Control Conference Best Student Paper Finalist (2013), NSF CAREER Award (2019), AISTATS best paper award (2020), IEEE EFTF-IFCS Best Student Paper Finalist (2021), Cullen-Peck Scholar Award (2022), GT-Emory AI.Humanity Award (2023), SONY Faculty Innovation Award (2024), Best Poster Award at the Recent Advances and Future Directions for Sampling conference (2024), and Richard Duke Fellowship (2025).
In person attendance at this seminar is only open to Columbia University affiliates. External guests are welcome to attend remotely. Please contact [email protected] if you need the Zoom link for this seminar.
