Journal of Analytical and Applied Computer Sciences
Peer Reviewed Open Access Journal
ISSN: 3139-3349
Peer Reviewed Open Access Journal
ISSN: 3139-3349
The field of time series analysis is rapidly developing as a result of the significant expansion of data production and the diversity of its applications in the fields of economics, finance, energy, healthcare, Meteorology, transport and other areas that rely on accurate forecasting to support decision-making. Traditional studies have relied for many years on statistical models, such as autoregressive models, moving averages (ARIMA) and exponential bootstrapping models, but these models face challenges when dealing with data with nonlinear relationships and complex time patterns. In contrast, machine learning techniques have emerged as advanced tools capable of detecting underlying patterns and extracting complex relationships from data, which has contributed to improving prediction accuracy in many applications. This study aims to provide a comprehensive scientific review of machine learning applications in time series analysis and prediction, by reviewing the theoretical foundations of this field, and discussing the most prominent machine learning algorithms used, including decision trees, random forests, supporting vector machines, artificial neural networks, deep learning models such as long-memory neural networks (LSTM), repeating gate modules (GRU), bypass neural networks (CNN), and transformer models (Transformers). The study also deals with data processing methods, feature extraction, performance evaluation metrics, and the most important challenges associated with data instability, missing values, interpretability of models, and high computer requirements. The study also reviews the results of recent research that compared machine learning models with traditional statistical models, showing the advantages of each and the limits of its use according to the nature of the data and forecasting objectives. Recent trends in the development of hybrid models combining statistical methods and machine learning technologies with the aim of improving the accuracy of forecasting and increasing the reliability of results are also discussed. The study concludes by identifying the most prominent research gaps and proposing a number of future directions, such as explainable artificial intelligence, automated machine learning (AutoML), transfer learning, probabilistic forecasting, and foundational models of time series, which contributes to guiding future research and developing more efficient and effective models in time prediction applications.
machine learning, time series, Time Series Prediction, deep learning, artificial neural networks, long-memory neural networks, transformer models, hybrid models, temporal data analysis, scientific review
Box, G. E. P., Jenkins, G. M., Reinsel, G. C., & Ljung, G. M. (2015). Time series analysis: Forecasting and control (5th ed.). Wiley.
Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324
Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785–794). ACM. https://doi.org/10.1145/2939672.2939785
Cho, K., Van Merriënboer, B., Gulcehre, Ç., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1724–1734). Association for Computational Linguistics.
Dietterich, T. G. (2000). Ensemble methods in machine learning. In Multiple classifier systems (pp. 1–15). Springer.
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019). A guide to deep learning in healthcare. Nature Medicine, 25(1), 24–29. https://doi.org/10.1038/s41591-018-0316-z
Garza, A., & Mergenthaler-Canseco, M. (2023). TimeGPT-1 [Technical report]. Nixtla.
Géron, A. (2022). Hands-on machine learning with Scikit-Learn, Keras & TensorFlow (3rd ed.). O'Reilly Media.
Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735
Hyndman, R. J., & Athanasopoulos, G. (2021). Forecasting: Principles and practice (3rd ed.). OTexts.
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems, 30.
LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
Lim, B., & Zohren, S. (2021). Time-series forecasting with deep learning: A survey. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 379(2194), 20200209. https://doi.org/10.1098/rsta.2020.0209
Makridakis, S., Spiliotis, E., & Assimakopoulos, V. (2022). The M5 accuracy competition: Results, findings and conclusions. International Journal of Forecasting, 38(4), 1346–1364. https://doi.org/10.1016/j.ijforecast.2021.11.013
Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2023). A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations (ICLR).
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., Luccioni, A. S., Maharaj, T., Sherwin, E. D., Mukkavilli, S. K., Kording, K. P., Gomes, C. P., Ng, A. Y., Hassabis, D., Platt, J. C., Creutzig, F., & Bengio, Y. (2022). Tackling climate change with machine learning. ACM Computing Surveys, 55(2), 1–96. https://doi.org/10.1145/3485128
Samek, W., Montavon, G., Vedaldi, A., Hansen, L. K., & Müller, K.-R. (Eds.). (2021). Explainable AI: Interpreting, explaining and visualizing deep learning (2nd ed.). Springer.
Sezer, O. B., Gudelek, M. U., & Ozbayoglu, A. M. (2020). Financial time series forecasting with deep learning: A systematic literature review (2005–2019). Applied Soft Computing, 90, 106181. https://doi.org/10.1016/j.asoc.2020.106181
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, 30 (pp. 5998–6008).
Wang, H., Lei, Z., Zhang, X., Zhou, B., & Peng, J. (2019). A review of deep learning for renewable energy forecasting. Energy Conversion and Management, 198, 111799. https://doi.org/10.1016/j.enconman.2019.111799
Wu, H., Xu, J., Wang, J., & Long, M. (2021). Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. In Advances in Neural Information Processing Systems, 34 (pp. 22419–22430).
Zhang, G. P. (2003). Time series forecasting using a hybrid ARIMA and neural network model. Neurocomputing, 50, 159–175. https://doi.org/10.1016/S0925-2312(01)00702-0
Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., & Zhang, W. (2021). Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, 35(12), 11106–11115. https://doi.org/10.1609/aaai.v35i12.17325
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., & Jin, R. (2022). FEDformer: Frequency enhanced decomposed transformer for long-term series forecasting. In Proceedings of the 39th International Conference on Machine Learning (ICML).
Zonta, T., da Costa, C. A., da Rosa Righi, R., de Lima, M. J., da Trindade, E. S., & Li, G. P. (2020). Predictive maintenance in the Industry 4.0: A systematic literature review. Computers & Industrial Engineering, 150, 106889. https://doi.org/10.1016/j.cie.2020.106889
