NewsAccepted by NeurIPS 2024:[49] Zhongwang Zhang, Pengxiao Lin, Zhiwei Wang, Yaoyu Zhang, Zhi-Qin John Xu*, Initialization is Critical to Whether Transformers Fit Composite Functions by Inference or Memorizing, NeurIPS 2024, arxiv 2405.05409 (2024), and in pdf, and in arxiv Accepted by Communications in Computational Physics (CiCP):[52] Zhiwei Wang, Lulu Zhang, Zhongwang Zhang, Zhi-Qin John Xu*, Loss Jump During Loss Switch in Solving PDEs with Neural Networks. Communications in Computational Physics, Arxiv 2405.03095 pdf, and in arxiv. Overview paper of frequency principle (dedicated to the memory of Professor Zhong-Ci Shi.)[27] Zhi-Qin John Xu*, Yaoyu Zhang, Tao Luo, Overview frequency principle/spectral bias in deep learning. Communications on Applied Mathematics and Computation 2024, arxiv 2201.07395 (2022) pdf, and in arxiv. Accepted by TPAMI:[33] Zhongwang Zhang, Zhi-Qin John Xu*, Implicit regularization of dropout. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024. arxiv 2207.05952 (2022) pdf, and in arxiv. Accepted by ICLR 2024:[41] Zhongwang Zhang, Yuqing Li*, Tao Luo*, Zhi-Qin John Xu*, Stochastic Modified Equations and Dynamics of Dropout Algorithm. ICLR 2024. arxiv 2305.15850 (2023) pdf, and in arxiv. Accepted by CSIAM Transactions on Applied Mathematics:[30] Zhiwei Bai, Tao Luo, Zhi-Qin John Xu*, Yaoyu Zhang*, Embedding Principle in Depth for the Loss Landscape Analysis of Deep Neural Networks. CSIAM Transactions on Applied Mathematics, 2024. arxiv 2205.13283 (2022) pdf, and in arxiv. Accepted by Combustion and Flame:[46] Zhiwei Wang, Yaoyu Zhang, Pengxiao Lin, Enhan Zhao, Weinan E, Tianhan Zhang*, Zhi-Qin John Xu*, Deep mechanism reduction (DeePMR) method for fuel chemical kinetics, Combustion and Flame, 2024. 上线 2023理解深度学习 系列课程Theory and Application of Deep Learning Summer SchoolThe summer school was held from July 3 to 7. 中文总结及学生反馈 Students got “Best Bachelor Thesis”Junjie Yao (2023, A DEEPLEARNING-BASED MODEL FOR SOLVINGCHEMICAL KINETICS) Zhiwei Bai (2022, THE STRUCTURAL STUDY OF LOSS LANDSCAPEOF DEEP LEARNING) Got supported from National Key R&D Program of China (PI)Young scholar project, Grant No. 2022YFA1008200. Joint with Tao Luo, Lei Wu (PKU), Yaoyu Zhang. (重点研发青年科学家项目) Journal of Machine LearningWe are launching a new journal: Journal of Machine Learning (JML, Editor-in-Chief: Weinan E). Welcome to submit papers to JML. Accepted by NeurIPS22:[29] Hanxu Zhou, Qixuan Zhou, Zhenyuan Jin, Tao Luo, Yaoyu Zhang, Zhi-Qin John Xu*, Empirical Phase Diagram for Three-layer Neural Networks with Infinite Width. arxiv 2205.12101 (2022) pdf, and in arxiv, NeurIPS2022. [18] Hanxu Zhou, Tao Luo, Yaoyu Zhang*, Zhi-Qin John Xu*, Towards Understanding the Condensation of Neural Networks at Initial Training. arxiv 2105.11686 (2021) pdf, and in arxiv, see slides and video talk in Chinese, NeurIPS2022. Accepted by MSML22:[14] (Alphabetic order) Tao Luo*, Zheng Ma, Zhiwei Wang, Zhi-Qin John Xu, Yaoyu Zhang, An Upper Limit of Decaying Rate with Respect to Frequency in Deep Neural Network, To appear in Mathematical and Scientific Machine Learning 2022 (MSML22), arxiv 2105.11675 (previous version: 2012.03238) (2020). pdf, and in arxiv Accepted by Combustion and Flame:[26] Tianhan Zhang*, Yuxiao Yi, Yifan Xu, Zhi X. Chen, Yaoyu Zhang, Weinan E, Zhi-Qin John Xu*, A multi-scale sampling method for accurate and robust deep neural network to predict combustion chemical kinetics. (Accepted by Combustion and Flame) arxiv 2201.03549 (2022) pdf, and in arxiv. Accepted by SIAM Journal on Mathematics of Data Science (SIMODS):[13] (Alphabetic order) Tao Luo*, Zheng Ma, Zhi-Qin John Xu, Yaoyu Zhang, On the exact computation of linear frequency principle dynamics and its generalization, SIAM Journal on Mathematics of Data Science (SIMODS) to appear, arxiv 2010.08153 (2020). pdf, and in arxiv, some code is in github. Accepted by Communications in Computational Physics (CiCP) :[20] Lulu Zhang, Tao Luo, Yaoyu Zhang, Weinan E, Zhi-Qin John Xu*, Zheng Ma*, MOD-Net: A Machine Learning Approach via Model-Operator-Data Network for Solving PDEs. Communications in Computational Physics (CiCP) (2022) to appear, arxiv 2107.03673 (2021) pdf, and in arxiv. Accepted by NeurIPS 2021 Spotlight:Yaoyu Zhang*, Zhongwang Zhang, Tao Luo, Zhi-Qin John Xu*, Embedding Principle of Loss Landscape of Deep Neural Networks. NeurIPS 2021 spotlight, arxiv 2105.14573 (2021) pdf, Talk on Bilibili Accepted by CSIAM Trans. Appl. Math.:1) (Alphabetic order) Tao Luo, Zheng Ma, Zhi-Qin John Xu, Yaoyu Zhang, Theory of the frequency principle for general deep neural networks, CSIAM Trans. Appl. Math., in web 2) (Alphabetic order) Jihong Wang, Zhi-Qin John Xu*, Jiwei Zhang*, Yaoyu Zhang, Implicit bias in understanding deep learning for solving PDEs beyond Ritz-Galerkin method, CSIAM Trans. Appl. Math., arxiv 2002.07989 (2020). pdf 2021, 人工智能青年科学家俱乐部（青源会）会员。Member of Qingyuan Club for outstanding contribution to the mathematical foundations of AI. Certificate 2021世界人工智能大会青年优秀论文提名奖Zhi-Qin John Xu* , Yaoyu Zhang, Tao Luo, Yanyang Xiao, Zheng Ma, Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks, pdf, and in web. Accepted by Journal of Machine Learning Research (2021)Phase diagram for two-layer ReLU neural networks at infinite-width limit, arxiv 2007.07497 (2020), pdf, and in arxiv MscaleDNN works are selected in the cover of a special issue on Machine Learning for Scientific Computing.See the Cover and the paper MscaleDNN for non-linear elliptic equations. The MscaleDNN is original proposed in MscaleDNN.pdf. The third work on the Cover is also an application of MscaleDNN. Accepted by AAAI-2021Deep frequency principle towards understanding why deeper learning is faster, pdf, and in arxiv |
