Recurrent neural network (RNN) based models are widely adopted to capture temporal dependencies in the state-of-the-art approaches for cellular traffic prediction. However, RNN is inefficient and incapable of capturing long-range temporal dependencies of traffic data. Besides, its inherent sequential nature makes it time consuming in capture the temporal dependencies. To better capture the long-term temporal dependency and reduce the consumed time in traffic data prediction, we propose a time-wise attention aided convolutional neural network (TWACNet) structure for cellular traffic prediction. In the proposed TWACNet, the time-wise attention mechanism is adopted to capture long-range temporal dependencies of the cellular traffic data and the convolutional neural network (CNN) is adopted to capture the spatial correlation. The performance of TWACNet in traffic prediction is tested in real-world cellular traffic datasets. Experimental results demonstrate that our proposed approach can considerably outperform those existing prediction methods in terms of root mean square errors (RMSE) and training time.