A systematic comparison of different machine learning models for the spatial estimation of air pollution

A systematic comparison of different machine learning models for the spatial estimation of air pollution Elena Cerezuela-Escudero1·Juan Manuel Montes-Sanchez1·Juan Pedro Dominguez-Morales1· Lourdes Duran-Lopez1·Gabriel Jimenez-Moreno1 Accepted: 12 October 2023 / Published online: 31 October 2023 © The Author(s) 2023 Abstract Air pollutants harm human health and the environment. Nowadays, deploying an air pollution monitoring network in many urban areas could provide real-time air quality assessment. However, these networks are usually sparsely distributed and the sensor calibration problems that may appear over time lead to missing and wrong measurements. There is an increasing interest in developing air quality modelling methods to minimize measurement errors, predict spatial and temporal air quality, and support more spatially-resolved health effect analysis. This research aims to evaluate the ability of three feed-forward neural network architectures for the spatial prediction of air pollutant concentrations using the measures of an air quality monitoring network. In addition to these architectures, Support Vector Machines and geostatistical methods (Inverse Distance Weighting and Ordinary Kriging) were also implemented to compare the performance of neural network models. The evaluation of the methods was performed using the historical values of seven air pollutants (Nitrogen monoxide, Nitrogen dioxide, Sulphur dioxide, Carbon monoxide, Ozone, and particulate matters with size less than or equal to 2.5 µm and to 10 µm) from an urban air quality monitoring network located at the metropolitan area of Madrid (Spain). To assess and compare the predictive ability of the models, three estimation accuracy indicators were calculated: the Root Mean Squared Error, the Mean Absolute Error, and the coefficient of determination. FFNN-based models are superior to geostatistical methods and slightly better than Support Vector Machines for fitting the spatial correlation of air pollutant measurements. Keywords Air pollution ·Spatial estimation ·Air quality monitoring ·Feed-forward neural networks ·Support vector machines ·Kriging 1 Introduction Exposure to air pollution is a risk factor for diseases such as stroke, asthma, cancer and chronic obstructive pulmonary BElena Cerezuela-Escudero [email protected] Juan Manuel Montes-Sanchez [email protected] Juan Pedro Dominguez-Morales [email protected] Lourdes Duran-Lopez [email protected] Gabriel Jimenez-Moreno [email protected] 1Robotics and Technology of Computers Lab (RTC), ETSI Informática, Universidad de Sevilla, Avd. Reina Mercedes s/n, Sevilla 41012, Spain disease. It is associated with noxious effects on human health and it is especially harmful to vulnerable groups such as children, the elderly and patients with respiratory and cardiovascular diseases [1–4]. Air pollutants not only severely impact public health, but also the climate and ecosystems because several of them are greenhouse gases [1,5]. Many of the air pollutants are also sources of greenhouse gas emissions. Considering the significance of air quality on health and the environment, the World Health Organization (WHO) has developed guidelines to improve air quality by setting limits on the concentrations of various air pollutants: ozone (O3), nitrogen dioxide (NO2), sulphur dioxide (SO2), carbon monoxide (CO), particulate matter with size less than or equal to 2.5 µm (PM2.5) and particulate matter with size less than or equal to 10 µm (PM10) [5]. Air quality monitoring is an important task for governments to provide information on potential health risks and determine appropriate environmental management policies. A systematic comparison of different machine learning models... The development of air pollution sensing technology in the last few decades and the support of government agencies have contributed to build an air quality monitoring network in many urban areas with the aim of analysing and publishing theconcentrationsofseveralairpollutantsthatarepotentially harmful to health 12 . However, these networks are usually sparsely distributed and the sensor calibration problems that may appear lead to missing and wrong measurements [6– 9]. There is an increasing interest in developing air quality modelling methods to minimize measurement errors, predict spatial and temporal air quality, and support more spatiallyresolved health effect analyses [8–13]. Air pollution modelling follows two different approaches. The first approach consists in using the deterministic mathematical modelling of atmospheric pollutant dispersion. The second approach, on the other hand, consists in employing statistical models based on historical air quality data, and, in some cases, meteorological and geographic information too. The deterministic mathematical modelling involves the simulation of pollutant dispersion and transport mechanisms using emission values in industrial and urban areas, physical and chemical processes in the atmosphere, meteorological data, and geographic and topological information. The deterministic methods that are most present in the scientific literature are the “Community Multiscale Air Quality (CMAQ) model” [14,15], “Weather Research and Forecasting model with Chemistry (WRF-Chem)” [16], and “Nested Air Quality Prediction Modelling System (NAQPMS)” [17, 18]. The deterministic modelling has limitations due to the enormous number of pollution sources and the fact that air distribution is influenced by several complex physical/chemical processes that require many variables. The statistical approach takes advantage of the spatial and temporal correlations that are present in the air pollution concentration time series and formulates models that simulate these dependencies with a high degree of accuracy. Severalmethodologies havebeen developedalongthisapproach, including classical statistics [19], artificial intelligence [8,9, 11–13,20,21] and geostatistical techniques [6,7,22]. The application of Artificial Neural Networks (ANNs) has been frequently used to forecast air quality. Some recent articles [7,13,20,21,23–26] use the historical values of various pollutants to predict the air quality index and/or air pollutant concentrations. Several of them use meteorological data too. Machine and deep learning methods show a more remarkable ability to simulate non-linear systems because of their self-learning, self-organizing, and self-adaptation features. Instead, fewer studies have used the ANN technique for the spatial estimation of air pollution [6,8,11,12,27]. The study [8] evaluates the use of a Back-Propagation Neural Networks (BPNN) for modelling the spatial atmospheric pollution of five air pollutants (NO2, O3, SO2, PM2.5 and PM10). The authors of [6] proposed machine learning and geostatistical methods to predict PM2.5 pollution levels. Some of these studies applied Deep learning methods to extract complex and non-linear spatiotemporal correlations [11,12]. The next section further describes the references about the use of machine learning and deep learning methods for predicting air pollution. The main objective of this research was to develop an ANN-based system for modelling the spatial characteristics of air pollutant concentrations measured at an urban air quality monitoring network. To estimate the air pollutant value at the target site based on the measurements collected at nearbylocations,weappliedthreeFeed-Forward NeuralNetwork (FFNN) architectures for regression (with one, two, and three fully connected hidden layers), a Support Vector Machine (SVM) and geostatistical methods. The evaluation of the methods was performed by using the historical values of seven air pollutants (Nitrogen monoxide (NO), NO2, O3, SO2, CO, PM2.5, and PM10) collected at the urban air quality monitoring network located at the greater metropolitan area of Madrid (Spain). For assessing and comparing the predictive ability of the models, three estimation accuracy indicators were calculated: the Root Mean Squared Error (RMSE), the Mean Absolute Error (MAE) and the coefficient of determination (R2). Themain contributions ofthis work includethefollowing: •A broad analysis of the most common pollutants evaluated on a real dataset consisting of samples obtained from the air quality monitoring network deployed in the city of Madrid (Spain). •A FFNN-based air pollution spatial estimation system able to accurately predict NO2, NO, SO2, O3, PM2.5 and PM10 concentrations. •A systematic comparison between two geostatistical models,SVM,andthreeFFNNarchitectures(one-hidden FFNN, bi-layer FFNN, and tri-layer FFNN) for evaluating the prediction of the concentration of seven air pollutants (NO2, NO, CO, SO2, O3, PM2.5 and PM10). The present work is structured as follows. The following section reviews literature related to machine learning and deep learning methods applied to air quality prediction. Section 3presents the study area and the air pollution dataset used in the experiments. In Section 4, the methods applied to estimate air pollutant concentrations are described: Inverse distance weighted (IDW), Ordinary Kriging (OK), SVM, FFNN. In Section 5, every phase of the experiments is pre- 29605 1European Air Quality Index: https://www.eea.europa.eu/themes/air/ air-quality-index. Retrieved October 28, 2023 2AIRNOW: https://www.airnow.gov/aqi/aqi-basics. Retrieved October 28, 2023 E. Cerezuela-Escudero et al. sented. Section 6 describes the results of the experiments performed. In Section 7 we discuss our results. Finally, the conclusions are presented in Section 8. 2 Related work Machine learning methods are widely used to extract nonlinear correlations in air pollutant concentration data. Deploying air pollution monitoring stations in urban areas generates a massive amount of collected data, creating databases suitable for statistical analysis. In addition, machine learning methods do not require a deep understanding of the dynamic and chemical processes between air pollutants and other relative atmospheric variables. ANNs have been improved through years of research and applications, bringing more evolved versions to air pollution prediction. For example, [27] proposed a BPNN to estimate the hourly concentrations of NO2 in unsampled locations at Algeciras Bay (Spain) using the historical values of NO2 concentrations at fourteen monitoring stations and the distances to each monitoring station. The prediction system has a first stage that applies the IDW interpolation and a multiple linear regression method to produce air pollution maps that are used as input of the the BPNN. The highest result achieved was an R2 value of 0.76. They also evaluated the methods separately, showing the BPNN as the best prediction method in most monitoring stations. Instead, [6] used several methods to obtain daily estimates of PM2.5 concentration across the contiguous US, and the results showed a better predictive performance of spatial statistical models over machine learning methods. The 829 monitor stations take measurements every 1, 3, or 6 days, with only approximately 15% of monitors sampling daily, implying an irregular and sparse dataset. In other recent research, [8] applied a BPNN for spatially estimating pollutant concentrations in the metropolitan city of Athens in Greece. Five pollutants were estimated, NO2, O3, PM10, PM2.5 and SO2, and the R2 values for O3 and PM10 were above 0.87,and for NO2, PM2.5, and SO2 were 0.76, 0.69 and 0.55, respectively. Deep learning has become increasingly widely used in air quality prediction because of its ability to extract complex and non-linear spatiotemporal correlations on large datasets. Many researchers employed a Long Short Term Memory (LSTM) network for modelling the complex and non-linear temporal correlation of the historical values of the pollutants [9, 20, 25, 28, 29]. LSTM is an enhanced version of the Recurrent Neural Network for handling long-time sequence data. The recent studies [11, 24, 30] have proposed combined models for air quality prediction based on Convolutional Neural Networks (CNNs) to extract the spatial characteristics, and LSTMs to predict future air pollution concentrations. [24] developed a CNN-LSTM method for predicting the next day’sdaily averagePM2.5concentration inBeijingCity.[30] proposed an attention-based CNN-LSTM multilayer structure to predict the PM2.5 concentration in the next 72 hours at Beijing-Tianjin-Hebei region. This research analysed the historical air quality and meteorological data of 100 monitoring stations for spatiotemporal correlation. [11] proposed spatiotemporal forecasting models of Beijing’s Air Quality Index. Four methods (CNN, LSTM, CNN-LSTM, BPNN) were evaluated to extract the spatiotemporal characteristics of air quality concentration data (hourly PM2.5, PM10, SO2, NO2, O3, and CO) and the relations with meteorological and spatiotemporal data. The method that showed the best performanceinnext-hourforecastingwas theCNN-LSTMmethod. [12] developed a spatiotemporal air quality prediction model based on LSTMs. The input data were Beijing’s historical concentrations of PM2.5, SO2, NO2, O3, and CO and meteorological data. The output was the concentration sequence of PM2.5, CO, NO2, O3, and SO2 at four monitoring sites. The model’s prediction accuracy is high, as shown by the best R2value of the four analysed sites: 0.939 for PM2.5, 0.847 for CO, 0.875 for NO2, 0.935 for O3, and 0.809 for SO2. Some researchers took advantage of the CNN’s ability to process sequence-structure data for air pollutant concentration prediction. For example, [31] used a five-layer CNN to extract the temporal correlation from historical observation data and predict the ozone concentration in the next 24 hours in an urban area. [7] applied the IDW method to interpolate air quality and weather data collected in South Korea and then used the interpolation as input of the CNN to predict PM2.5 and PM10 concentrations. The results show an effective prediction performance with an R2higher than 0.97. In our research, we have deployed and compared six air quality prediction methods present in the recent literature: spatial-based interpolation (IDW), geostatistical model (Kriging), machine learning (SVM and FFNN), and deep learning (FFNN with 2 - 3 hidden layers). Each method is evaluated for extracting spatial dependencies of seven air pollutants: NO2, NO, CO, SO2, O3, PM2.5, and PM10. Therefore, we have broadly compared different methods to predict the concentrations of several air pollutants that are generated by various sources and that have different behaviours. The analysis of so many air pollutants is rare in the literature. The urban area selected has an air quality monitoring network with a spatial density and a sample time higher or similar to recent studies. 3 Study area and dataset The area of study is the city of Madrid, which is the capital of Spain and the largest and most populated metropolitan area of the country. Madrid’s province population has grown 29606 A systematic comparison of different machine learning models... from 6,446 million in 2016 to 6,751 million in 2021 3.The elevation at its centre (40◦25’ N, 3◦41’ W) is 657 m. Madrid area’s expected mean temperature changes from 9.8 ◦Cin January to 32.1 ◦C in July, experiencing cold winters and hot summers. Spring and autumn are the seasons with more expected rainy days, while the summer months are usually dry and sunny 4. Madrid’s air pollution levels are high, although, since the activation of air quality policies in 2011, those levels were effectively reduced [32]. This study analysed hourly time series of four air pollutants (NO2, O3, PM10, and SO2) monitoring in Madrid urban area during the period from 2001 to 2017 by a two-stage method: first, a Hidden Markov Model was used to characterize the air pollution at temporal scales. Then, the spatial distribution was analysed by combining the interpolation results of Ordinary Kriging and Inverse Distance Weighting. [32] concludes that the air pollution spatial analysis is challenging to assess due to meteorological and physical factors and the regional contributions originated in adjacent municipals. Not only is human activity responsible for bad air quality, but also other climate events like Saharan dust intrusions have an impact by rising PM levels [32, 33].The research [34] examined the effects of local road traffic, meteorological conditions, and temporal variables on air pollution in Madrid. Its result showed that air pollutant levels were weakly linked to local vehicular emissions because various elements affect the pollutant concentration, mainly meteorological agents, topography, tree and shrub presence, building distribution, and water streams like rivers. This study uses hourly air quality data measured between January2016 andAugust 2018 bythe airmonitoringnetwork of Madrid. Madrid’s city council operates an air quality monitoring network from 2001 and publishes both real-time and historical air quality data5. We used the Dataset “Air Quality in Madrid”6that contains the processed data from the files offered by the Madrid’s city council with a structured organization based on timestamp and standard format data. It consists of a file for each year where each row is timestamped and the columns are the different measures performed at that point in time in a certain station. In addition, the information 3Instituto Nacional de Estadística (National Statistics Institute): https://www.ine.es/dynt3/inebase/index.htm?padre=1689& capsel=9041. Retrieved October 13, 2023. 4Madrid, Retiro: Madrid, Retiro - Agencia Estatal de Meteorología - AEMET. Gobierno de España: https://www.aemet.es/ es/serviciosclimaticos/datosclimatologicos/valoresclimatologicos? l=3195&k=28. Retrieved October 13, 2023. 5Open data portal of the Madrid City Council: https://datos.madrid.es/ portal/site/egob. Retrieved October 13, 2023. 6KaggleAirQualityinMadrid(2001-2018):https://www.kaggle.com/ datasets/decide-soluciones/air-quality-madrid. Retrieved October 13, 2023. regarding each station (identifier, name, address, coordinates and elevation) is available in another file. The measurements of many pollutants are available (NO, NO2, O3, SO2, CO, PM2.5, PM10, toluene, benzene, methane), but not every station is equipped with all air pollution sensors, which have been increasing over the years. The study area and the distribution of 24 air quality monitoring stations deployed in Madrid are shown in Fig. 1.The maximum distance between two stations is 7 kilometres, and the average distance between stations at the urban centre is 3. Table 1contains the coordinates, elevation and the air pollutant measured in each air quality monitoring station between 2016 and 2018. We selected the most harmful air pollutants according to WHO and several government agencies like the U.S. Environmental Protection Agency7, European Environment Agency8and Health Canada and Environment Canada9. The criteria for the selection of the monitoring sites are the hourly data availability along with homogeneous spatial data coverage for each air pollutant. Table 2presents the measuring units and the range of measured concentrations for each pollutant between 2016 and 2018. 4 Methods In this work, we present a system based on FFNN architecture for regression to predict the air pollutant concentration in a specific location based on the measurements obtained from nearby monitored locations. To compare our proposal, we applied the following geostatistical and machine learning methods: Inverse Distance Weighting (IDW), Ordinary Kriging (OK), and Support Vector Machine (SVM). 4.1 Inverse distance weighting and kriging methods Nearly all spatial interpolation methods share the same general estimation formula, which is as follows: Z(x0)= n  i=1 (wiz(xi)) (1) where Zis the estimated value at the point of interest x0,zis the observed value at the sampled point xi,wiis the weight 7AIRNOW: https://www.airnow.gov/aqi/aqi-basics. Retrieved October 13, 2023 8European Air Quality Index: https://www.eea.europa.eu/themes/air/ air-quality-index. Retrieved October 13, 2023 9Canada Air Quality Health Index: https://www.canada.ca/en/ environment-climate-change/services/air-quality-health-index/about. html. Retrieved October 13, 2023 29607 E. Cerezuela-Escudero et al. Fig. 1 Area of study and position of the different air monitoring stations. Map image is obtained from Google Maps assigned to the sampled point, and nrepresents the number of sampled points used for the estimation. The difference between the methods depends on the formula to calculate the weights. The two most commonly used interpolation methods in the literature are IDW and OK [35,36]. The IDW method uses the following expression for the weight: wi= 1p di n i=11p di (2) where di is the distance between x0 and xi , and p is an exponent that determines the influence of values closest to the interpolated point, while the weight for OK is estimated by minimizing the variance of the prediction errors. It is assumed that the data are part of an intrinsic function z(x) with the sample variogram [37]. The sample variogram is fitted with specific known positive defined functions. The most common functions are linear, spherical, exponential, and Gaussian. In our experiments, we tested the IDW model for p=1 and p=2 and, for the implementation of the OK method, we applied four function models for fitting the sample variogram: spherical, exponential, Gaussian and bounded linear. 4.2 Support vector machine SVM is a popular machine learning tool for classification, but it can also be used for regression analysis [38,39]. SVMs aim to provide a nonlinear function to map a given training data set D: (x1,y1), (x2,y2), ..., (xi,yi)toahighdimensional feature space. In this space, a hyperplane is optimized to be within a certain threshold of the selected data, called the support vectors, and the hyperplane is used for predicting regression. A linear epsilon-insensitive () SVM was used for regression, which is also known as L1 loss. The set of training data included predictor variables and observed response values. The goal was to find a function z(x)that deviates from observed values no greater than for each training point x, and that is as flat as possible at the same time. The training of the SVM with epsilon-insensitive loss function was performed by using quadratic programming for minimising the objective-function. 4.3 Feed-forward neural network for regression ANNsaremassivelyparallelinterconnectednetworksofsimple, hierarchically organized elements (artificial neurons) 29608 A systematic comparison of different machine learning models... Table 1 Air quality monitoring stations, coordinates, and measured pollutants (from 2016 to 2018) Station ID Longitude Latitude Height (m) Measured pollutant 1 -3.7122472 40.4238528 635 NO2, NO, CO, SO2 2 -3.6823194 40.4215638 670 NO2, NO, CO, SO2, O3, PM2.5, PM10 3 -3.6773555 40.4514750 708 NO2, NO 4 -3.63923333 40.4400472 693 NO2, NO, CO, O3 5 -3.7133222 40.3471389 604 NO2, NO, SO2, O3, 6 -3.7318527 40.3947805 630 NO2, NO, CO, SO2, O3, PM10 7 -3.7473472 40.4193555 642 NO2, NO, CO, SO2, O3, PM2.5, PM10 8 -3.5800305 40.4769278 621 NO2, NO, O3 9 -3.7031722 40.4192083 659 NO2, NO, CO, O3 10 -3.6453056 40.4079472 685 NO2, NO, CO, SO2, PM10 11 -3.7071278 40.4455444 698 NO2, NO, SO2, PM2.5, PM10 12 -3.7115417 40.4782278 674 NO2, NO, CO, O3 13 -3.6515222 40.3881527 677 NO2, NO, SO2, PM10 14 -3.686825 40.3981138 599 NO2, NO, PM2.5, PM10 15 -3.6903667 40.4398972 676 NO2, NO, PM2.5, PM10 16 -3.6825833 40.4144444 662 NO2, NO, O3 17 -3.68876945 40.4655722 728 NO2, NO, PM2.5, PM10 18 -3.6121167 40.3729333 627 NO2, NO, O3, 19 -3.5807472 40.4625305 618 NO2, NO, PM10 20 -3.7187278 40.3849639 604 NO2, NO, CO, O3 21 -3.6605028 40.4942083 700 NO2, NO, CO, SO2, PM10 22 -3.7746111 40.5180583 615 NO2, NO, O3 23 -3.60907222 40.4652500 660 NO2, NO, O3 24 -3.6897611 40.5005889 715 NO2, NO, O3, PM10 that attempt to interact with the environment in the same way as the biological nervous system [40]. The output of such an artificial neuron can be calculated using the (3). y=f( n  i=1 (wixi)) +b)(3) where xiare the inputs, nthe number of inputs, withe synaptic weights, bthe threshold and fthe activation function. The mostcommonlyusedactivationfunctionsarelinear,sigmoid, and hyperbolic tangent. Artificial neurons are arranged in several layers and connected by synaptic weights. In this work, three feed-forward, fully connected neural networks were used for regression. The structure includes an input layer, one or more hidden layers, and an output layer. The input layer takes information (predictor data) from the domain and passes it to all the neurons from the first hidden layer. As the first hidden layer is fully connected to the input layer, each subsequent layer is connected to all the neurons from the previous layer. Each neuron of a fully connected layer multiplies the input by the synaptic weight and then adds the multiplication results with the bias. The sum is passed through an activation function. The final fully connected layer produces the network’s output (predicted response values). The proposed architecture for the fully connected layered neural network with two hidden layers isshowninFig.2. An enlarged diagram of a single artificial neuron is presented separately to show its five components -inputs, synaptic weights, sum, bias, and activation function. We chose two activation functions for hidden layers: a Rectified Linear Unit (ReLU) and a sigmoid function. These functions are described in (4) and (5), respectively. Accord- Table 2 Measuring units and range of measured concentrations for each pollutant (years 2016 to 2018) NO2 NO CO SO2 O3 PM2.5 PM10 (μg/m3)(μg/m3)(mg/m 3)(μg/m3)(μg/m3)(μg/m3)(μg/m3) [1,349][1,973][0.1,4.9][1,98][1,193][0,96][0,318] 29609 E. Cerezuela-Escudero et al. Fig. 2 Fully interconnected bi-layer FFNN with 5 input nodes, two hidden layers with 10 neurons and the output layer with one neuron. An enlarged diagram of a single artificial neuron is presented separately to show its five components -inputs, synaptic weights, sum, bias, and activation function ing to the regression problem, the activation function of the output layer is the linear function f(x) = x. f(x)=x,x≥0 0,x<0(4) f(x)=1 1+e−x(5) The training is based on the limited-memory Broyden- Fletcher-Goldfarb-Shanno quasi-Newton algorithm (LBFGS) [41], where the mean squared error (MSE) is minimized. We proposed three FFNN architectures for extracting the spatial characteristics of the air pollution concentration: one hidden layer neural network, a bi-layer neural network, and a tri-layer neural network. Regarding the current application, the number of monitoring sites for predicting each pollutant defines the number of input nodes, and the output layer consists of a fully connected neuron to return the prediction. The number of neurons in each hidden layer is configurable. In order to compare the same structure for different air pollutants, we determined 10 neurons per hidden layer for each architecture. 5 Experiments Figure 3 shows the diagram of the experimental workflow. It was followed for the evaluation of several prediction models to estimate each air pollutant. Firstly, we selected the target location, which should be a point with historical concentrations of the air pollutant to be estimated. We chose target stations based on their position close to the centre of Madrid and the five stations closest to the target site. However, our decision was constrained by the availability of air pollution sensors in each station. Table 3 shows the monitoring station selected as the target site and five nearby monitoring stations used to estimate the air pollutant concentrations. The average distance between the target and the selected stations is 3.26 kilometres. The coordinates and distribution of the air quality monitoring stations are shown in Table 1and in Fig. 1, respectively. Secondly, we processed the dataset to get the air pollutant concentrations from the selected stations. Each sample included the air pollutant concentrations for each station taken simultaneously. The samples with some missing values were removed. The input values for IDW and OK models are the latitudes, longitudes, and pollutant concentration at the five nearby stations. However, the input data for SVM and FFNN are the historical values of the pollutants from six stations, for training, and the pollutant concentration at the five nearby stations, for predicting. Then, the predictive models were designed. We developed two IDW models (p=1 and p=2) for evaluating the influence of the distance in the prediction. When p=2, the method is known as the inverse distance squared weighted interpolation. To implement the OK method, four function models were applied to fit the sample variogram: spherical, exponential, Gaussian, and bounded linear. We proposed a linear -SVM for regression and three different FFNN architectures: one hidden layer neural network, a bi-layer neural network, and a tri-layer neural network. The number of monitoring sites for predicting each pollutant defines the number of input nodes, a parameter set to five in our system. The output layer consists of a fully connected neuron to return the prediction. The number of neurons in each hidden layer is a configurable parameter, and we set it to ten to compare the same structure for different air pollutants. We tested two activationfunctionsfor hiddenlayers: aRectifiedLinear Unit (ReLU) and a sigmoid function. In the case of the SVM and FFNN models, the next phase is training. In both cases, we used 80% of the samples to train the neuronal network model and the remaining 20% to test the performance of the trained model with new data. Figure 4shows the data and processes of training stage. The results are the trained FFNN (or SVM) and the difference and correlation measures. The IDW and OK models do not require training since the estimated value is calculated from the simultaneous measurements taken at the nearby stations and the distances to the target site. We used the test set to validate the SVM and FFNN models and the whole dataset to evaluate the IDW and OK models. The accuracy of each model is based on the comparison of the observed 29610 A systematic comparison of different machine learning models... Fig. 3 Experimental workflow followed for the prediction of each air pollutant and the evaluation of the proposed models and predicted concentrations and the statistical analysis of the residual values. Finally, we determined the mean performance of each model by using a set of difference and correlation measures:Mean AbsoluteError (MAE),Root MeanSquare Error (RMSE), and the coefficient of determination (R2). R2is the proportion of variation in the dependent variable that is predicted by the statistical model (range from 0 to 1). Therefore, R2provides information about the model’s goodness of fit. The following equations determine these error measures Table 3 The selected monitoring stations and distances to the target site for each air pollutant under evaluation Pollutant Target site Monitoring stations and distance to target site NO2andNO15 123911 2.57 km 2.15 km 1.69 km 2.54 km 1.55 km CO 2 1 6 9 10 20 2.55 km 5.15 km 1.79 km 3.49 km 5.11 km SO2 2 1 6 10 11 13 2.55 km 5.15 km 3.49 km 3.39 km 4.54 km O3 9 2 4 6 16 20 1.79 km 5.90 km 3.64 km 1.82 km 4.02 km PM2.5 15 2 7 11 14 17 2.15 km 5.35 km 1.55 km 4.65 km 2.85 km PM10 15 2 10 11 14 17 2.15 km 5.22 km 1.55 km 4.65 km 2.85 km 29611 E. Cerezuela-Escudero et al. Fig. 4 The training phase flowchart of SVM and FFNN methods betweenobserved(xi) andpredicted (yi)values, wheren represents the number of sampled points used for the estimation and μxis the mean of observed values. MAE =1 n n  i=1 |xi−yi|(6) RMSE =    1 n n  i=1 (xi−yi)2(7) R2=1−n i=1(xi−yi)2 n i=1(xi−μx)2(8) This process was followed for the seven selected pollutants: NO, NO2, O3, SO2, CO, PM2.5 and PM10. 6 Results Six methods for modelling the spatial characteristics of the air pollutant concentrations were implemented and evaluated for each of the seven air pollutants (NO2, NO, CO, SO2, O3, PM2.5, and PM10). The experiments used historical pollutant data from monitoring stations of Madrid city’s air quality monitoring network collected from January 1, 2016, to December 31, 2018. Among the different model variants that were tested for the IDW and OK methods, only the best ones are shown in the results. We proposed three FFNN architectures: a FFNN with one fully connected layer (FCL), two FCLs, and three FCLs. Except in the prediction of SO2 concentration, we chose to apply the ReLU function in the hidden layers because it has a greater accuracy than the sigmoid one. Tables 4,5,6,7,8,9and 10 present the assessment of the statistical (IDW and OK) and machine learning (SVM and FFNN) methods by RMSE, MAE and R2. RMSE and MAE are in the same units of pollutant concentrations, and R2ranges from 0 to 1. The results for predicting NO2 and NO showed a better predictive performance of machine learning methods over spatial statistical models. The bi-layer FFNN method presents maximum R2and minimum RMSE for estimating NO2 and NO air pollutants. The proposed models for estimating the CO concentrations result in low accuracy. Neither method is appropriate to extract the spatial characteristics of the CO concentrations. A possible reason for this result may be that CO is a primary pollutant whose concentration is closely linked to local combustion phenomena, aggravated in some cases by low dispersion due to thermal inversion or lackofwind.Heavytrafficortrafficjamsinaspecificareacan cause high local measurements of CO without hardly affecting neighbouring areas. A higher density of measurement stations could be needed to get better results for CO because its concentration relative to the distance from the pollution source can decrease quickly. In the case of SO2, the accuracyofthe machinelearningmethodsisvastlysuperiortothat of the spatial statistical models. FNN methods experiment an exceptionalimprovementwhenincreasingthenumberofhidden layers. The prediction models of O3, PM2.5, and PM10 present similar performances, with slightly higher accuracy for bi-layer FFNN methods. The bi-layer FFNN method exhibits the best result for most of the air pollutants under evaluation, except for the prediction of SO2, where the best performance is led by the tri-layer NN. The highest coefficient of determination (0.9) is reached by the bi-layer FFNN method for the prediction of NO2 and PM10. In most cases, we could rank the methods according to their performance (from best to worst) as follows: bi-layer FFNN, tri-layer FFNN, FFNN, SVM, IDW, and OK. The IDW and OK methods present similar results and have the lowest accuracy for all the evaluated air pollutants. The SVM models exhibit lower predictive ability than the FFNN methods for fitting air pollution concentration. 29612 123 A systematic comparison of different machine learning models... Lourdes Duran-Lopez received the B.S. degree in Biomedical Engineering, the M.S. degree in Biomedical Research, and the Ph.D. degree in industrial computerized, robotic and neuromorphic systems from the University of Seville (Sevilla, Spain), in 2016, 2017 and 2021, respectively. Since 2023, she has been working as Assistant Professor in the Robotics and Technology of Computers Lab. from the University of Seville. Her research interests include image processing, medical image analysis, computer-aided diagnosis systems and deep learning, particularly, convolutional neural networks. Gabriel Jimenez-Moreno received the M.S. degree in physics (electronics) and the Ph.D. degree from the University of Seville, Seville, Spain, in 1987 and 1992, respectively. He was granted a Fellowship from the Spanish Science and Technology Commission (CICYT). Currently, he is an Associate Professor of computer architecture at the University of Seville. He participated in the creation of the Department of Computer Architecture (also at University of Seville). He is the author of various papers and research reports on robotics, rehabilitation technology, and computer architecture. His research interests include neural networks, vision processing systems, embedded systems, computer interfaces, and computer architectures. 29619