Comparison Between LSTM, GRU and VARIMA in Forecasting of Air Quality Time Series Data
DOI:
https://doi.org/10.11113/mjfas.v20n6.3411Keywords:
Air quality, long short-term memory (LSTM), gated recurrent unit (GRU), vector autoregressive integrated moving average (VARIMA), forecasting.Abstract
Air quality forecast is essential in alerting the public, especially those who have respiratory diseases, to take necessary precautions beforehand. The public can be forewarned of any worsening of air quality and be aware of the importance of reducing air pollution. In recent years, forecasting techniques based on deep learning algorithms such as recurrent neural network (RNN) have seen improvements in both accuracy and execution speed. Long short-term memory (LSTM) network and gated recurrent unit (GRU) are among the most popular variants of RNN. In this study, the hourly PM2.5 concentrations at five selected air quality monitoring stations, provided by the Department of Environment Malaysia, are forecasted using LSTM, GRU and vector autoregressive integrated moving average (VARIMA) models respectively. Data containing missing, negative and zero values are pre-processed using an interpolation technique before being split into training and test sets on an 80:20 ratio basis. Optimal combinations of hyperparameter values are selected via manual tuning based on the 10-fold growing window cross-validation results. The model performance is evaluated based on RMSE, MAE and MAPE. The results demonstrate that neural network models significantly outperform the multivariate time series model in which the LSTM and GRU models have comparable performance in forecasting the hourly PM2.5 concentration, with a slightly better prediction in the west coast region for LSTM and the east coast region for GRU. However, due to the complex architecture of neural networks, the computational time to train both LSTM and GRU models is three times longer than that for VARIMA. Additionally, it is observed that a higher percentage of interpolated values leads to lower prediction errors.
References
Mabahwi, N. A. B., Ling, O. H. L., & Omar, D. (2014). Human health and wellbeing: Human health effect of air pollution. Procedia – Social and Behavioral Sciences, 153, 221–229.
WHO. (2018). 9 out of 10 people worldwide breathe polluted air, but more countries are taking action. WHO News Release.
Global Environmental Forum. (2000). Overseas Environmental Measures of Japanese Companies (Malaysia): Research Report on Trends in Environmental Considerations Related to Overseas Activities of Japanese Companies FY 1999. Tokyo: Ministry of the Environment, Government of Japan.
Nur-Nabilah, M. N., Nor-Amani-Filzah, M. K., Norzila, O., Azra-Munirah, M. D., Nurul-Bahiyah, A. W., & Khairuddin, M. K. (2021). Discovering source of residents’ complaint on air quality: Preliminary studies on particulate matter (PM2.5) and sulphur dioxide (SO2). IOP Conference Series: Materials Science and Engineering, 1144, 012045.
Su, B. D., Zhan, M. J., Zhai, J. Q., Wang, Y. J., & Fischer, T. (2015). Spatio-temporal variation of haze days and atmospheric circulation pattern in China (1961–2013). Quaternary International, 380-381, 14–21.
Xing, Y. F., Xu, Y. H., Shi, M. H., & Lian, Y. X. (2016). The impact of PM2.5 on the human respiratory system. Journal of Thoracic Disease, 8(1), E69–E74.
Ao, D., Cui, Z., & Gu, D. (2019). Hybrid model of air quality prediction using k-means clustering and deep neural network. Proceedings of the 38th Chinese Control Conference, 8416–8421.
Caraka, R. E., Chen, R. C., Toharudin, T., Pardamean, B., Yasin, H., & Wu, S. H. (2019). Prediction of status particulate matter 2.5 using state Markov chain stochastic process and HYBRID VAR-NN-PSO. IEEE Access, 7, 161654–161665.
Zhou, X., Xu, J., Zeng, P., & Meng, X. (2019). Air pollutant concentration prediction based on GRU method. Journal of Physics: Conference Series, 1168, 032058.
Bakar, M. A. A., Ariff, N. M., Nadzir, M. S. M., Ong, L. W., & Suris, F. N. A. (2022). Prediction of multivariate air quality time series data using long short-term memory network. Malaysian Journal of Fundamental and Applied Sciences, 18, 52–59.
Uh, B. H., & Majid, N. (2021). Comparison of ARIMA model and artificial neural network in forecasting gold price. Journal of Quality Measurement and Analysis, 17(2), 31–39.
Tsan, Y.-T., Chen, D.-Y., Liu, P.-Y., Kristiani, E., Nguyen, K. L. P., & Yang, C.-T. (2022). The prediction of influenza-like illness and respiratory disease using LSTM and ARIMA. International Journal of Environmental Research and Public Health, 19(3), 1858.
Tan, W. M., & Othman, Z. (2021). Ramalan jumlah kandungan elektron GPS menggunakan ingatan jangka pendek yang panjang dan unit berulang berpagar. Undergraduate Dissertation, Universiti Kebangsaan Malaysia.
Mateus, B. C., Mendes, M., Farinhaa, J. T., Assis, R., & Cardoso, A. M. (2021). Comparing LSTM and GRU models to predict the condition of a pulp paper press. Energies, 14(21), 6958.
ArunKumar, K. E., Kalaga, D. V., Kumar, C. M. S., Kawaji, M., & Brenza, T. M. (2022). Comparative analysis of gated recurrent units (GRU), long short-term memory (LSTM) cells, autoregressive integrated moving average (ARIMA), seasonal autoregressive integrated moving average (SARIMA) for forecasting COVID-19 trends. Alexandria Engineering Journal, 61(10), 7585–7603.
Rusyana, A., Tatsara, N., Balqis, R., & Rahmi, S. (2020). Application of clustering and VARIMA for rainfall prediction. IOP Conference Series: Materials Science and Engineering, 769(1), 012063.
Setiawan, A., Aidi, M. N., & Sumertajaya, I. M. (2015). Modelling of forecasting monthly inflation by using VARIMA and GSTARIMA models. Forum Statistika dan Komputasi: Indonesian Journal of Statistics, 20(2), 60–63.
Zainuri, N. A., Jemain, A. A., & Muda, N. (2015). A comparison of various imputation methods for missing values in air quality data. Sains Malaysiana, 44(3), 449–456.
Jiang, N., Akter, R., Ross, G., White, S., Kirkwood, J., Gunashanhar, G., Thompson, S., Riley, M., & Azzi, M. (2023). On thresholds for controlling negative particle (PM2.5) readings in air quality reporting. Environmental Monitoring and Assessment, 195, 1187.
Gariazzo, S., Giunti, C., & Laveder, M. (2015). Light sterile neutrinos and inflationary freedom. Journal of Cosmology and Astroparticle Physics, 04, 023.
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18, 1527–1554.
Sarker, I. H. (2021). Deep learning: A comprehensive overview on techniques, taxonomy, applications and research directions. SN Computer Science, 2, 420.
Dubey, S. R., Singh, S. K., & Chaudhuri, B. B. (2022). Activation functions in deep learning: A comprehensive survey and benchmark. Neurocomputing, 503, 92–108.
Wechsler, H. (1992). Neural networks for perception. Michigan: Academic Press.
Zhu, N., Liu, X., Liu, Z., Hu, K., Wang, Y., Tan, J., Huang, M., Zhu, Q., Ji, X., Jiang, Y., & Guo, Y. (2018). Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. International Journal of Agricultural and Biological Engineering, 11(4), 32–44.
Taweh Beysolow II. (2017). Introduction to deep learning using R. California: Apress Berkeley.
Ghatak, A. (2019). Deep learning with R. Singapore: Springer.
Jason B. (2017). Long short-term memory networks with Python. Australia: Jason Brownlee.
Cho, K., Merrienboer, B., Gulcehre, C., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods, 1724–1734.
Zhang, A., Lipton, Z. C., Li, M., & Smola, A. J. (2022). Dive into deep learning. Amazon Science.
LazyProgrammer. (2016). Deep learning: Recurrent neural networks in Python: LSTM, GRU and more RNN machine learning architectures in Python and Theano (Kindle ed.). Kindle Scribe.
Schnaubelt, M. (2019). A comparison of machine learning model validation schemes for non-stationary time series data. FAU Discussion Papers in Economics. Friedrich-Alexander University Erlangen-Nuremberg, Institute for Economics.
Cerqueira, V., Torgo, L., & Mozetič, I. (2020). Evaluating time series forecasting models: An empirical study on performance estimation methods. Machine Learning, 109, 1997–2028.
Ab. Rahman, E., Hamzah, F. M., Latif, M. T., & Dominick, D. (2022). Assessment of PM2.5 patterns in Malaysia using the clustering method. Aerosol and Air Quality Research, 22, 210161.
Leh, O. L. H., Ahmad, S., Aiyub, K., Jani, Y. M., & Hwa, T. K. (2012). Urban air environmental health indicators for Kuala Lumpur City. Sains Malaysiana, 41(2), 179–191.
Ramli, N., Abdul Hamid, H., Yahaya, A. S., UI-Saufie, A. Z., Mohamed Noor, N., Abu Seman, N. A., Kamarudzaman, A. N., & Deák, G. (2023). Performance of Bayesian model averaging (BMA) for short-term prediction of PM10 concentration in the Peninsular Malaysia. Atmosphere, 14(2), 311.
Ariff, N. M., Bakar, M. A. A., & Lim, H. Y. (2023). Prediction of PM10 concentration in Malaysia using k-means clustering and LSTM hybrid model. Atmosphere, 14(5), 853.
Sugiyarto, A. W., & Abadi, A. M. (2019). Prediction of Indonesian palm oil production using long short-term memory recurrent neural network (LSTM-RNN). Proceedings of the 2019 1st International Conference on Artificial Intelligence and Data Sciences (AiDAS), 53–57.
Mitrea, C. A., Lee, C. K. M., & Wu, Z. (2009). A comparison between neural networks and traditional forecasting methods: A case study. International Journal of Engineering Business Management, 1(2), 19–24.
Krishan, M., Jha, S., Das, J., Singh, A., Goyal, M. K., & Sekar, C. (2019). Air quality modelling using long short-term memory (LSTM) over NCT-Delhi, India. Air Quality, Atmosphere & Health, 12, 899–908.
Kontopoulou, V. I., Panagopoulos, A. D., Kakkos, I., & Matsopoulos, G. K. (2023). A review of ARIMA vs. machine learning approaches for time series forecasting in data driven networks. Future Internet, 15(8), 255.
Sobolewski, Ł., & Miczulski, W. (2021). Methods of constructing time series for predicting local time scales by means of a GMDH-type neural network. Applied Sciences, 11(12), 5615.
Jaffar, A., Thamrin, N. M., Ali, M. S. A. M., Misnan, M. F., Yassin, A. I. M., & Zan, N. M. (2022). Spatial interpolation method comparison for physico-chemical parameters of river water in Klang River using MATLAB. Bulletin of Electrical Engineering and Informatics, 11(4), 2368–2377.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Yu Nie Ng, Han Ying Lim, Ying Chyi Cham, Mohd Aftar Abu Bakar, Noratiqah Mohd Ariff
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
