From classical statistics to modern AI -- the puzzle of dimensionality
Date : 2026/07/13 (Mon.) 11:00~12:00
Location : Room 639, Institute of Mathematics (NTU Campus)
Speaker : Mikkhail Belkin
Abstract : Remarkable progress in AI has far surpassed expectations of just a few years ago.
At their core, modern models, such as transformers, implement traditional statistical models -- high order Markov chains. Nevertheless, it is not generally possible to estimate Markov models of that order given any possible amount of data. Therefore these methods must implicitly exploit low-dimensional structures present in data. Furthermore, these structures must be reflected in high-dimensional internal parameter spaces of the models. Thus, to build fundamental understanding of modern AI, it is necessary to identify and analyze these latent low-dimensional structures. Despite a long history of research, our understanding of dimensionality is still lacking.
In my talk I will consider the puzzle of high dimensionality both in the data and in parameter spaces, including some of my work in that direction. In particular, I will discuss some ideas of manifold learning, sparse inference, over-parameterized models and our recent work on feature learning in neural networks, kernel machines and connections to sparse inference. I will aim to identify connections and the missing pieces. I argue that an actionable statistical theory of AI models is an urgent societal need and is also within reach.
