Keywords:
Bayesian Model Averaging
iTransformer
Traffic Speed
Temporal Fusion Transformer
Multistep Prediction

  • Abstract

    The Bayesian Model Averaging (BMA) approach is proposed to combine the traffic speed prediction results of three Transformer models to enhance intelligent transportation management and facilitate route planning. This approach addresses a key limitation in existing literature, which often overlooks the model uncertainty between different traffic speed prediction models. By utilizing a multi-head attention mechanism, the Transformer models can identify stable long-term speed trends. To assess the prediction performance of BMA approach, the model was tested using traffic speed data from an interstate freeway in Minnesota for time interval from 5 minutes to 60 minutes. When compared to the three single Transformer models, the BMA approach demonstrated superior accuracy in predicting short-term freeway traffic speeds.

  • References

    1. Chandra, S. R., & Al-Deek, H. (2009). Predictions of Freeway Traffic Speeds and Volumes Using Vector Autoregressive Models. Journal of Intelligent Transportation Systems, 13(2), 53–72. https://doi.org/10.1080/15472450902858368
    2. Chen, Y., Zou, Y., Xie, Y., Zhang, Y., & Tang, J. (2024). Multimodal vehicle trajectory prediction based on intention inference with lane graph representation. Expert Systems with Applications, 125708. https://doi.org/10.1016/j.eswa.2024.125708.
    3. Feng, H., & Jiang, X. (2022). Multi-step ahead traffic speed prediction based on gated temporal graph convolution network. Physica A: Statistical Mechanics and its Applications, 606, 128075. https://doi.org/https://doi.org/10.1016/j.physa.2022.128075
    4. Lim, B., Arık, S. Ö., Loeff, N., & Pfister, T. (2021). Temporal fusion transformers for interpretable multi-horizon time series forecasting. International Journal of Forecasting, 37(4), 1748-1764. https://doi.org/10.1016/j.ijforecast.2021.03.012
    5. Lu, Z., Lv, W., Xie, Z., Du, B., Xiong, G., Sun, L., & Wang, H. (2022). Graph sequence neural network with an attention mechanism for traffic speed prediction. ACM Transactions on Intelligent Systems and Technology (TIST), 13(2), 1-24. https://doi.org/10.1145/3470889
    6. Ma, C., & Zhao, M. (2023). Spatio-temporal multi-graph convolutional network based on wavelet analysis for vehicle speed prediction. Physica A: Statistical Mechanics and its Applications, 630, 129233. https://doi.org/https://doi.org/10.1016/j.physa.2023.129233
    7. Newton, M. A., & Raftery, A. E. (1994). Approximate Bayesian inference with the weighted likelihood bootstrap. Journal of the Royal Statistical Society Series B: Statistical Methodology, 56(1), 3-26. https://doi.org/10.1111/j.2517-6161.1994.tb01956.x
    8. Montgomery, J. M., & Nyhan, B. (2010). Bayesian model averaging: Theoretical developments and practical applications. Political Analysis, 18(2), 245-270. doi:10.1093/pan/mpq001
    9. Qu, L., Lyu, J., Li, W., Ma, D., & Fan, H. (2021). Features injected recurrent neural networks for short-term traffic speed prediction. Neurocomputing, 451, 290-304. https://doi.org/10.1016/j.neucom.2021.03.054
    10. Shen, L., Wei, Y., & Wang, Y. (2023). GBT: Two-stage transformer framework for non-stationary time series forecasting. Neural Networks, 165, 953-970. https://doi.org/10.1016/j.neunet.2023.06.044
    11. Wang, H., Liu, L., Dong, S., Qian, Z., & Wei, H. (2016). A novel work zone short-term vehicle-type specific traffic speed prediction model through the hybrid EMD–ARIMA framework. Transportmetrica B: Transport Dynamics, 4(3), 159-186. https://doi.org/10.1080/21680566.2015.1060582
    12. Wu, S., Zou, Y., Wu, L., & Zhang, Y. (2023). Application of Bayesian model averaging for modeling time headway distribution. Physica A: Statistical Mechanics and its Applications, 620, 128747. https://doi.org/10.1016/j.physa.2023.128747
    13. Yao, Y., Vehtari, A., Simpson, D., & Gelman, A. (2018). Using stacking to average Bayesian predictive distributions (with discussion). Bayesian Analysis, 13(3), 917-1003. DOI: 10.1214/17-BA1091
    14. Zafar, N., Haq, I. U., Chughtai, J. U. R., & Shafiq, O. (2022). Applying hybrid LSTM-GRU model based on heterogeneous data sources for traffic speed prediction in urban areas. Sensors, 22(9), 3348. https://doi.org/10.3390/s22093348
    15. Zhang, H., Zou, Y., Yang, X., & Yang, H. (2022). A temporal fusion transformer for short-term freeway traffic speed multistep prediction. Neurocomputing, 500, 329-340. https://doi.org/10.1016/j.neucom.2022.05.083
    16. Zou, Y., Chen, Y., Xu, Y., Zhang, H., & Zhang, S. (2024). Short-term Freeway Traffic Speed Multistep Prediction using an iTransformer Model. Physica A: Statistical Mechanics and its Applications, 130185. https://doi.org/10.1016/j.physa.2024.130185
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Copyright (c) 2024 Yajie Zou

How to cite

Zou, Y. (2025). Combination of multiple transformer models for short-term freeway traffic speed prediction based on Bayesian model averaging . Multidisciplinary Science Journal, (| Accepted Articles). https://doi.org/10.31893/multiscience.2025468
Crossref
Scopus
Google Scholar
Europe PMC
  • Article viewed - 109