Convolution has played a prominent role in various applications in science and engineering for many years. It is also the most important operation in convolutional neural networks (CNNs). There has been a recent growth of interests of research in generalizing CNNs on 3D objects, often represented as compact manifolds. However, existing approaches cannot preserve all the desirable properties of Euclidean convolutions, namely compactly supported filters, directionality, transferability across different manifolds. In this talk, I will discuss our recent work on a new way of defining convolution on manifolds via parallel transport. This geometric way of defining parallel transport convolution (PTC) provides a natural combination of modeling and learning on manifolds. PTC allows for the construction of compactly supported filters and is also robust to manifold deformations. I will demonstrate its applications to shape analysis using deep neural networks based on parallel transportation convolutional networks (PTC-net).