Anomaly detection in videos is a challenging problem in computer vision because only normal events are available in training set. Most previous work handles the problem within a sparse representation framework: a dictionary is learnt to minimize the reconstruction error for normal events and abnormal events would lead to large reconstruction error. However such sparse representation is computationally expensive in the testing phase. Inspired the optimization of sparse representation, we propose to build a special type of deep neural network, which is a counterpart of sparse coding. Then we simplify the network which not only improves the speed but also accuracy. Further, it is worth noting that anomaly detection refers to the identification of events that do not conform to expected behavior, so we propose to solve anomaly detection within future video frame prediction framework. By simultaneously enforcing the spatial and temporal consistency of videos frames of normal videos, we can predict high quality video frames for normal videos. Extensive experiments validates the effectiveness of such video frame prediction framework over feature reconstruction framework for anomaly detection.