Logo

Butterfly Net: Optimal Function Representation Based on Convolutional Neural Networks

18d22385f6cac743ff48e5fd48e7d97cc97da60e

Speaker

Yingzhou Li, Fudan University

Time

2021.01.19 14:00-15:00

Venue

Room 306, No.5 Science Building

Abstract

Deep networks, especially convolutional neural networks (CNNs), have been successfully applied in various areas of machine learning as well as to challenging problems in other scientific and engineering fields. This talk introduces Butterfly-net, a low-complexity CNN with structured and sparse crosschannel connections, together with a Butterfly initialization strategy for a family of networks. Theoretical analysis of the approximation power of Butterfly-net to the Fourier representation of input data shows that the error decays exponentially as the depth increases. Combining Butterfly-net with a fully connected neural network, a large class of problems are proved to be well approximated with network complexity depending on the effective frequency bandwidth instead of the input dimension. Regular CNN is covered as a special case in our analysis. Numerical experiments validate the analytical results on the approximation of Fourier kernels and energy functionals of Poisson’s equations. Moreover, all experiments support that training from Butterfly initialization outperforms training from random initialization. Also, adding the remaining cross-channel connections, although significantly increase the parameter number, does not much improve the post-training accuracy and is more sensitive to data distribution.

Sponsors