In this talk, I will show that several conventional mathematical models in imaging sciences, such as sparsity models, random field models, diffusion models, etc., can inspire novel deep learning architectures. This idea opens a door for bridging the gap between mathematical modeling and deep learning, and benefits both domains. I will present our several recent advances along this line of research, and illustrate their applications in natural image analysis and medical imaging.