Traffic flow forecasting is closely related to people’s lives, and it also brings a lot of convenience to our travel planning. Since the spatio-temporal data has the dual complexity of time and space, the existing model can not process this problem well. This paper proposes a Residuial Analysis Model (RAM-TF) for traffic flow prediction. The RAM-TF model divides spatio-temporal data into three blocks to deal with the time complexity of spatio-temporal data, and uses convolution to capture the correlation of flow changes between regions. As the number of convolutional layers increases, the accuracy of the training will be improved, and then decreased with convolutional layers continuing to increase. We introduce a residual network to solve the problem. The model was tested on the Beijing Taxi (TaxiBJ) data and the New York Shared Bike (BikeNYC) data, the results of our proposed method are compared with the results of the five existing models, to demonistrating the suprtiority of the proposed mode.