Some Mathematical Questions in Deep Learning


Jian-Guo Liu, Duke University


2017.12.26 10:00-11:00


516 Pao Yue-Kong Library


In this talk I will present some mathematical questions in deep learning including:

(1) Rigorous justification of the small jump approximation of the stochastic gradient descent (SGD) and online principal component analysis (PCA).

SGD and its variants are the most common tools in the supervised learning and it is widely believed that the behavior of SGDs shall be described by stochastic differential equations (SDE). I will present a simple and rigorous justification of this claim by using small jump approximation theory in Markov process. This is a joint work with Lei Li (Duke) and Yuanyuan Feng (CMU)

(2) Shape estimates on the escape time for SGD to escape from unstable stationary points including both saddle points and local maximums. This is a central question in deep learning in very high dimensional and non-convex statistical optimization. I will present a result on shape rate escape time using the theory of large deviation of random dynamical system. This is a joint work with Lei Li (Duke) and Junchi Li (Princeton)

(3) Online learning in optical tomography by stochastic gradient descent (SGD)

Many of inverse problems can be formatted as statistics optimization problems and online deep learning methods such as SGD can be used to effectively solve the prohibitive memory and computation problem in very high dimensions. I will present a successful example of online learning in optical tomography which has many applications in medial image. This is a joint work with Ke Chen and Qin Li (U Wisconsin-Madison).

(4) A modified Levy jump-diffusion model based on market sentiment memory for online jump prediction.

Data assimilation contribute greatly to the success of wealth predication and it is realized by the observation data for many local stations. The idea can be extended to jump diffusion model for option pricing. I will present a method using the market sentiment data collected from Internet to model the Levy jump diffusion model for option pricing. This is a joint work with Lei Li (Duke) and Bill Zhu (Stanford)