Neural Models for Imputation of Missing Ozone Data in Air-Quality Datasets

Resea ch A icle Neu al Models o Impu a ion o Missing Ozone Da a in Ai -Quali y Da ase s Ángel A oyo ,1Ál a o He e o,1Ve ónica T icio,2 Emilio Co chado,3and MichaBWo niak4 1Depa men o Ci il Enginee ing, Uni e si y o Bu gos, Bu gos, Spain 2Depa men o Physics, Uni e si y o Bu gos, Bu gos, Spain 3Depa amen o de In o m´ a ica y Au om´ a ica, Uni e si y o Salamanca, Salamanca, Spain 4Depa men o Sys ems and Compu e Ne wo ks, W ocław Uni e si y o Science and Technology, W ocław, Poland Co espondence should be add essed o ´ Angel A oyo; aa oyo[email p o ec ed] Recei ed 5 Decembe 2017; Accep ed 31 Janua y 2018; Published 8 Ma ch 2018 Academic Edi o : Eloy I igoyen Copy igh ©2018´ Angel A oyo e al. This is an open access a icle dis ibu ed unde he C ea i e Commons A ibu ion License, which pe mi s un es ic ed use, dis ibu ion, and ep oduc ion in any medium, p o ided he o iginal wo k is p ope ly ci ed. Ozone is one o he pollu an s wi h mos nega i e e ec s on human heal h and in gene al on he biosphe e. Many da a-acquisi ion ne wo ks collec da a abou ozone alues in bo h u ban and backg ound a eas. Usually, hese da a a e incomple e o co up and he impu a ion o he missing alues is a p io i y in o de o ob ain comple e da ase s, sol ing he unce ain y and agueness o exis ing p oblems o manage complexi y. In he p esen pape , mul iple- eg ession echniques and A i icial Neu al Ne wo k models a e applied o app oxima e he absen ozone alues om i e explana o y a iables con aining ai -quali y in o ma ion. To compa e he di e en impu a ion me hods, eal-li e da a om six da a-acquisi ion s a ions om he egion o Cas illa y Le´ on (Spain) a e ga he ed in di e en ways and hen analyzed. The esul s ob ained in he es ima ion o he missing alues by applying hese echniques and models a e compa ed, analyzing he possible causes o he gi en esponse. 1. In oduc ion and Rela ed Wo k The ozone (O3) is an odo less, colo less, and highly eac i e gas composed o h ee oxygen a oms. I is o med bo h in he Ea h’s uppe a mosphe e (s a osphe ic ozone) and a g ound le el ( oposphe ic ozone). I can be “good” o “bad” o people’s heal h and o he en i onmen , depending on i s concen a ion le els and loca ion in he a mosphe e [1]. S a osphe ic O3is o med na u ally h ough he in e - ac ion o sola Ul aViole (UV) adia ion wi h molecula oxygen (O2). G ound-le el o “bad” ozone is no emi ed di ec ly in o he ai . In he 1950s, hyd oca bons and ni o- gen oxides (NO𝑥) we e iden i ied as he wo key chemi- cal p ecu so s o pho ochemical smog and i s concomi an high concen a ions o O3and o he pho ochemical oxidan [2]. The majo i y o g ound-le el O3is o med om he pho ochemical oxida ion o Vola ile O ganic Compounds (VOCs) in he p esence o NO and o he NO𝑥. Signi ican sou ces o VOCs a e chemical plan s, gasoline pumps, oil- based pain s, au obody shops, and p in shops. NO𝑥 esul p ima ily om high empe a u e combus ion, and i s mos signi ican sou ces a e powe plan s, indus ial u naces and boile s, and mo o ehicles [3]. 1.1. Impo ance o Ozone. The O3exposi ion can cause dam- age in di e en ways. In he s a osphe e, educed O3le els asa esul o O 3laye deple ion mean less p o ec ion om he sun’s ays and mo e exposu e o Ul aViole B (sho wa e) ays (UVB) adia ion a he Ea h’s su ace [4]. The e ec s on human heal h o he O3laye deple ion ha e been much analyzed, inc easing he amoun o UVB ha eaches he Ea h’s su ace. UVB causes nonmelanoma skin cance and plays a majo ole in malignan melanoma de elopmen . In addi ion, UVB has been linked o he de elopmen o ce ain ca a ac s, nega i e e ec s in pa ien s wi h as hma, and Hindawi Complexi y Volume 2018, A icle ID 7238015, 14 pages h ps://doi.o g/10.1155/2018/7238015 2Complexi y o he ch onic espi a o y disease. Wi h espec o g ound- le el O3, and i s e ec s on human heal h, b ea hing O3can igge a a ie yo heal hp oblems.Peoplewi has hmaand o he ch onic espi a o y disease a e a la ge and g owing segmen o he popula ion and a e also known o be especially suscep ible o he e ec s o O3exposu e. On days wi h high le els o O3, people wi h as hma end o expe ience inc eased espi a o y symp oms [3]. The laye O3deple ion has also nega i e e ec s on he p ocess o he de elopmen o plan s, e ec s on he ma ine ecosys ems like a di ec educ ion in phy oplank on p oduc ion, nega i e e ec s on ma e ials like biopolyme s, and so o h. T oposphe ic O3does no p o ide hep o ec i e unc ion ha i ul illsin hes a osphe e,being high eac i i y. I s s ong oxidizing capaci y, when i s le els ise abo e he na u al backg ound, can cause ad e se e ec s in ma e ials (de i ed om i s co osi e e ec s), on ege a ion and ecosys ems. The p esen wo k ocuses on oposphe ic O3,whichis a isk o heai quali y[3].Gi en heinc easeinO 3le els in he oposphe e, i is cu en ly conside ed one o he mos impo an a mosphe ic pollu an s. 1.2. Ozone Le el Moni o ing. A ound he wo ld he e a e nu- me ous da a-acquisi ion ne wo ks o he measu emen o O3 le els and o he pollu an s, which consis o many s a ions in di e en loca ions whe e di e en senso s measu e co e- sponding magni udes. These ne wo k s a ions acqui e da a a pe iodic in e als o ime (pe iods be ween en and i een minu es a e he mos equen ones) bu equen ly appea missing o co up ed da a. In Eu ope, da a a e conside ed as co up ed when no mee ing he Council Decision 97/101/EC o Janua y 27, 1997 [5], which es ablish a ecip ocal exchange o in o ma ion and da a om ne wo ks and indi idual s a ions measu ing ambien ai pollu ion wi hin he Membe S a es. Some o hese ne wo ks p o ide in o ma ion abou he alidi y o he da a, indica ing h ough codes i he da a is co ec , i has no been possible o acqui e, o i is co up , bu in o he occasions his ype o in o ma ion is no p o ided while he da a a e s ill missing. Some easons o such ailu es ha ebeenpinpoin ed[6],namely,adamagedcable, helosso p ope elec ical g ounding, hal -mel ed os o snow on he dome, communica ions ailu e, and so o h. Some o hese causes a e empo a y and may disappea spon aneously, bu o he ones equi e he in e en ion o a main enance ask o ce, and he e o e e o s pe sis o di e en pe iods o ime.Theabsenceo alidda amayalsobedue o easons such as he ollowing: mishandling o samples, low signal- o-noise a io, measu emen e o , non esponse, o dele ed abe an alue [7]. This is a p oblem o he analysis o he in o ma ion coming om he measu emen ne wo ks, and he impu a ion o hese missing da a [8] is necessa y. Any o he a iables acqui ed in ne wo k s a ions may su e om he p oblem o he absence o da a. I many da a a iables a e omi ed o co up ed in he same eco d, he whole sample mus be wi hd awn, when some models a e applied [9], o subsequen asks such as con ol, classi ica ion, o ecas . Al e na i ely, i da a o he same pollu an a e missing in se e al adjacen ows, emo ing ha a iable may also be an al e na i e solu ion. In conclusion, ha ing a comple e se o da a is necessa y o pe o m a eliable s udy and o apply some models ha canno deal wi h missing da a. 1.3. Missing Values and Rela ed Wo k. The s anda d classi i- ca ion o missing da a phenomenon [10] includes di e en si ua ions: (i) Missing Comple ely A Random (MCAR), when he p obabili y o an ins ance (case) ha ing a missing alue o a a iable does no depend on ei he he known alues o he missing da a. (ii) Missing A Random (MAR), when he p obabili y o an ins ance ha ing a missing alue o a a iable may depend on he known alues bu no on he alue o he missing da a i sel . (iii) No Missing A Random (NMAR), when he p ob- abili y o an ins ance ha ing a missing alue o a a iable could depend on he alue o ha a iable. As p e ious au ho s ha e poin ed ou , he complexi y a ies be ween hese pa e ns o missing da a [11]. Usually, in hecaseo ai -quali yda a,missing aluesa eassocia edwi h MAR o MCAR. The ci cums ances ha may in e e e wi h he acquisi ion o he da a a e many and no easily p edic able [12]. To sol e he missing da a p oblem, a wide a ie y o di e en me hodsha ebeenappliedup onow[8,10,13]. These impu a ion me hods (IMs) a e usually classi ied as ollows: (i) Single impu a ion (SI): he me hod ills in one alue o each missing one [12]. (ii) Mul iple impu a ion (MI): mul iple simula ed alues a e gene a ed a he same ime [14]. The uni a ia e and mul i a ia e impu a ion me hods di e in which he app oxima ion o he missing alues o he a iable unde s udy a e calcula ed om he es o he alues o he e y same a iable (uni a ia e) o using alues o he es o he a iables (mul i a ia e) [12]. Wi h he aim o educing he complexi y o o he MI applied me hods [11], he p esen pape ocuses on single and mul i a ia e impu a ion o he O3magni ude in ai pollu ion da ase s. To do so, mul iple- eg ession (linea and nonlinea ) echniques oge he wi h A i icial Neu al Ne wo ks (ANN) a e applied o eal-li e da ase s ob ained om public ai - quali y ne wo ks. Up o now, di e en A i icial-In elligence (AI) ech- niques ha e been applied o impu a ion o missing da a. In [7] impu a ion me hods based on six di e en echniques a e compa ed: K-Nea es Neighbo s (KNN), Fuzzy K-Means (FKM), Singula Value Decomposi ion, Bayesian P incipal Componen Analysis (bPCA) and Mul iple Impu a ions by Chained Equa ions. These me hods a e applied o ou da ase s spli in o wo g oups o a ious sizes: small da ase s (I is and E. coli) and la ge da ase s (b eas cance s 1 and 2). bPCA and FKM appea ed o be he mos obus impu a ion me hods in he es ed condi ions. In [15] he accu acy o di e en impu a ion me hods is e alua ed: MissFo es (MF) and Mul iple Impu a ion based Complexi y 3 on Expec a ion-Maximiza ion (MIEM), along wi h wo o he impu a ion me hods: Sequen ial Ho -Deck and Mul iple Impu a ion based on Logis ic Reg ession (MILR). The mod- els a e applied o e ou een bina y da ase s, wi h a ange o missing da a a es be ween 5% and 50%. The esul s om 10- old C oss-Valida ion (CV) show ha he pe o mance o he impu a ion me hods a ies subs an ially be ween di e en classi ie s and a di e en a es o missing alues. Al hough many impu a ion me hods ha e been p oposed up o now, scan a en ion has been paid o alida e ANN o such a ask, aking ad an age o hei eg ession capabili y [16]. Among hese p e ious s udies, ANN ha e been applied o he es ima ion o los alues in [17], whe e he main goal is iden i ying Lea ning Disabili ies (LD) in child en a ea ly s ages. In [18], au ho s p oposed a SI app oach elying on a Mul ilaye Pe cep on (MLP) whose aining is conduc ed wi h di e en lea ning ules, and a MI app oach based on he combina ion o MLP and KNN. 24 eal and simula ed da ase s om he UCI eposi o y, he P omise eposi o y, and mlda a.o g we e exposed o a pe u ba ion expe imen wi h andom gene a ion o mono one missing da a pa e n. In [19] six di e en ypes o ANN a e p oposed as IM: MLP and i s a ia ions ( he Time-Lagged Feed o wa d Ne wo k (TLFN)), he Gene alized Radial-Basis-Func ion (GRBF) ne wo k, he Recu en Neu al Ne wo k (RNN), and i s a ia ions ( he Time Delay Recu en Neu al Ne wo k (TDRNN)). Addi ionally, he Coun e p opaga ion Fuzzy- Neu al Ne wo k (CFNN) along wi h di e en op imiza ion me hods is applied o in illing missing daily o al p ecipi a- ion and ex eme empe a u e se ies om 15 wea he s a ions. The s anda d MLP and TLFN appea o p o ide he mos accu a e econs uc ion o missing p ecipi a ion and daily ex eme empe a u es eco ds wi h esul s o he Rco ela- ion coe icien be ween he obse ed and he econs uc ed daily se ies close o 1. In [20] a no el nonpa ame ic algo i hm named Gen- e alized eg ession neu al ne wo k Ensemble o Mul iple Impu a ion (GEMI) is p oposed. Addi ionally, a SI e sion o his app oach (GESI) is p oposed. The algo i hms we e es ed on 98 syn he ic and eal-wo ld da ase s. All simula ion esul s show head an ageso GEMIascompa edwi hcon en ional algo i hms. GEMI has hea y memo y s o age equi emen s bu ou pe o med o he SI algo i hms. In [21] i een eal and simula ed da ase s a e exposed o a pe u ba ion expe imen , based on he andom gene a ion o missing alues. Se e al a chi ec u es and lea ning algo i hms o he MLP a e es ed and compa ed wi h h ee classic impu a ion p ocedu es: mean/mode impu a ion, eg ession, and ho -deck [22]. In [23] a me hodology based on Gaussian Mix u e Model (GMM) and Ex eme Lea ning Machine (ELM) is de eloped and es edonsomeda ase s om heUCIMachineLea ning Reposi o y and he LIACC eg ession eposi o y. GMM is used o model he da a dis ibu ion which is adap ed o handle missing alues, while ELM enables de ising a Mul iple Impu a ion s a egy o inal es ima ion. The combina ion o GMMandELMisshown obesupe io inalmos all es ed cases o e he me hod based on condi ional mean impu a ion. In[24]aSIapp oach elyingonaMLPandaMIapp oach based on he combina ion o MLP and K-NN is p oposed. The models a e applied o 18 eal and simula ed da ase s like domains such as biology, medicine, chemis y, elec on- ics, social su eys, census, and business. Fo da ase s wi h only quan i a i e a iables MIMLP model p o ided he bes esul s, wi h IMLP being he bes me hod o da ase s wi h ca ego ical a iables. In [25] a wo-s age hyb id model o illing he missing aluesusing uzzyc-meansclus e ingandMLPisp oposed. I is applied o a Wine da ase wi h a 1% o 5% o gene a ed missing alues and he accu acy o he model is checked using heMeanAbsolu ePe cen ageE o (MAPE).TheMAPE ob ained o s age 2 (MLP eg ession o he ob ained da ase as a esul o applying uzzy c-means in s age 1) is 4.95% o 1% missing- alue eco ds and 8.36% o 5% missing- alue eco ds. In he case o ai -quali y da a, ew impu a ion me hods ha e been p oposed up o now. In [13], an impo an se o SI: Lis wise, Uncondi ional mean, Modi ied Median, P in- cipal Componen -based, Expec a ion-Maximiza ion (EM) (Regula ized-EM), and MI me hods a e applied o h ee da ase s wi h he mos impo an pollu an a iables (NO, NO2,NO 𝑥,CO,O 3, PM10, and PM2.5) and a pe cen age o missing da a among he 3.85% and he 23.52% depending on he yea . Missing da a o he eigh a iables a e impu ed in o de o assess he e ec i eness o he me hods applied. In gene al, MI ends o yield mo e sca e ed alues han i s coun e pa s, mainly when he a iables ha e many oids and hey co ela e poo ly o he o he a iables like CO wi h 43.5% o missing da a in 2006 and hey co ela e poo ly o he o he a iables. In [11] some me hods o he impu a ion o missing ai - quali y da a a e compa ed: in he con ex o SI (linea , spline, and nea es neighbo in e pola ions), MI ( eg ession-based impu a ion, mul i a ia e nea es neighbo , Sel -O ganizing Maps (SOM), and Mul ilaye Backp opaga ion (MLBP) ne s) and hyb id me hods o he a o emen ioned. The da ase uses he mos common pollu an s: NO𝑥,NO 2,O 3, PM10, SO2, and CO concen a ions, all on a ime-scale o one pe hou (hou ly a e aged), oge he wi h ou me eo ological pa am- e e s. The pe o mance o he p oposed uni a ia e missing da a in e pola ion was limi ed, and in gene al hey we e able o illonly e ysho gapso con iguousmissingda a.The gene al pe o mance o he applied impu a ion me hods was ai good when conside ing he pollu an s (NO𝑥,NO 2,O 3, PM10, SO2, and CO) which a e he mos impo an ones in e ms o ai -quali y modelling, bu no so good ega ding me eo ological a iables. The esul s sugges ed ha SOM andMLBPa e heme hodso choice o ai -quali yda a impu a ion and e en be e esul s can be achie ed by using he MI. 1.4. Main Con ibu ions. The main con ibu ions o his wo k a e as ollows: (i) Deep s udy o he eal-li e human heal h p o ec ion ask in Spanish egion o Cas illa y Le´ on. (ii) Mul isenso o O3da a analysis. 4Complexi y (iii) Expe imen al e alua ion o he p oposed app oach based on mul iple- eg ession echniques oge he wi h ANN models. To he bes o au ho s knowledge, his is he i s app oach o impu a ion me hods o O3based on bo h MLP and Radial- Basis-Func ion Ne wo ks. The es o his pape is o ganized as ollows. Sec ion 2 p esen s he echniques and models applied. Sec ion 3 de ails he eal-li e case s udy ha is add essed in p esen wo k, while Sec ion 4 desc ibes he expe imen s and esul s. Finally, Sec ion 5 se s ou he main conclusions and u u e wo k. 2. Reg ession Techniques and ANN Models In o de o ill missing o co up ed alues o O3in high dimensional da ase s wi h ai -quali y in o ma ion, wo e- g ession echniques and wo ANN models ha e been applied in p esen s udy. This se o echniques applied as impu a ion me hods is desc ibed in his sec ion. 2.1. Reg ession Techniques. Linea eg ession a emp s o model he ela ionship be ween wo a iables by i ing a linea equa ion o obse ed da a. One a iable is conside ed o be an explana o y a iable, and he o he is conside ed o be a dependen a iable [26]. The gene al pu pose o mul iple eg essions [27] is o lea n mo e abou he ela ionship be ween se e al indepen- den o p edic o a iables and a dependen o c i e ion a iable. 2.1.1. Mul iple Linea Reg ession. Mul iple linea eg ession (MLR) a emp s o model he ela ionship be ween wo o mo e explana o y a iables and a esponse a iable by i ing a linea equa ion o obse ed da a [28]. E e y alue o he independen a iable (𝑥) is associa ed wi h a alue o he dependen a iable (𝑦). The popula ion eg ession line o 𝑝 explana o y a iables 𝑥1,𝑥2,...,𝑥𝑝(1) isde ined obe 𝑢𝑦=𝛽0+𝛽1𝑥1+𝛽2𝑥2+⋅⋅⋅+𝛽𝑝𝑥𝑝.(2) This line desc ibes how he mean esponse 𝑢𝑦changes wi h he explana o y a iables. The obse ed alues o y a y abou hei means 𝑢𝑦anda eassumed oha e hesame s anda d de ia ion 𝜎.The i ed alues𝑏0,𝑏1,...,𝑏𝑝es ima e he pa ame e s 𝛽0,𝛽1,...,𝛽𝑝o he popula ion eg ession line. Since he obse ed alues o y a y abou hei means u𝑦, he mul iple- eg ession models include a e m o his a ia ion. The model is exp essed as DATA = FIT + RESID- UAL, whe e he “FIT” e m ep esen s he exp ession 𝛽0+ 𝛽1𝑥1+𝛽2𝑥2+⋅⋅⋅+𝛽𝑝𝑥𝑝. The “RESIDUAL” e m ep esen s he de ia ions o he obse ed alues 𝑦 om hei means 𝑢𝑦, which a e no mally dis ibu ed wi h mean 0 and a iance 𝜎. The no a ion o he model de ia ions is 𝜀. Fo mally, he model o mul iple linea eg ession, gi en nobse a ions, is [28] 𝑌𝑖=𝛽0+𝛽1𝑥𝑖1 +𝛽2𝑥𝑖2 +⋅⋅⋅+𝛽𝑝𝑥𝑖𝑝 +𝜀𝑖 o 𝑖=1,2,...,𝑛. (3) 2.1.2. Mul iple Nonlinea Reg ession. AMul ipleNonlinea Reg ession (MN-LR) is a o m o eg ession analysis in which obse a ional da a a e modelled by a unc ion which is a nonlinea combina ion o he model pa ame e s and depends on one o mo e independen a iables [29]. The da a a e i ed by a me hod o successi e app oxima ions. The pa ame e s can ake he o m o an exponen ial, igonome ic, powe , o any o he nonlinea unc ion. To de e mine he nonlinea pa ame e es ima es, an i e a i e algo i hm is ypically used. 𝑦=𝑓(𝑋,𝐵)+𝜀, (4) whe e 𝐵 ep esen s nonlinea pa ame e es ima es o be compu ed, 𝑋is he dependen o c i e ion a iables, and 𝜀 ep esen s he e o e ms. 2.2. A i icial Neu al Ne wo ks. A i icial Neu al Ne wo ks (ANN), also known as A i icial Neu al Sys ems (ANS), connec ionis sys ems, adap i e ne wo ks, and dis ibu ed and pa allel p ocessing a e simpli ied models o na u al neu al sys ems. The ollowing de ini ion, gi en by Hech - Nielsenin1989[30], o malizes heconcep o ANN: An ANN is a pa allel p ocessing compu e sys em dis ibu ed, consis ing o a se o elemen a y p o- cessing uni s equipped wi h a small local memo y and in e connec ed in a ne wo k h ough connec- ions wi h associa ed weigh s. Each p ocessing uni has one o mo e inpu connec ions and a single ou pu connec ion ha links o many colla e al connec ions as desi ed. All p ocessing associa ed wi h an elemen a y uni is a local, i.e. depends only on he alues ha ake inpu signals om he uni and he in e nal s a e o he same. 2.2.1. Mul ilaye Pe cep on (MLP). TheMLPconsis so a sys em o simple in e connec ed neu ons o nodes. The nodes a e connec ed by weigh s and ou pu signals which a e a unc ion o he sum o he inpu s o he node modi ied by a simple nonlinea ans e , o ac i a ion, unc ion. The a chi ec u e consis s o se e al laye s o neu ons; he inpu laye se es opass heinpu ec o o hene wo k.The e ms “inpu ec o s” and “ou pu ec o s” e e o he inpu s and ou pu so heMLPandcanbe ep esen edassingle ec o s [31]. A MLP may ha e one o mo e hidden laye s and inally anou pu laye .MLPa e ullyconnec ed,wi heachnode connec ed o e e y node in he nex and p e ious laye . To pe o m a comp ehensi e compa ison, he MLP is ained wi h he ollowing algo i hms: (1) Le enbe g-Ma qua d backp opaga ion (LM) Complexi y 5 (2) G adien Descen wi h momen um and adap i e lea ning a e backp opaga ion (GDX) [32] (3) Ba ch T aining wi h weigh and bias lea ning ules (TB) (4) Scaled Conjuga e G adien backp opaga ion (SCG) (5) Bayesian Regula iza ion backp opaga ion (BR). 2.2.2. Radial-Basis-Func ion Ne wo ks (RBFN). In a RBFN [33] each uni in he hidden laye o his ne wo k has i s own cen oid, and, o each inpu ec o 𝑥=(𝑥𝑙,𝑥2,...,𝑥𝑛), i compu es he dis ance be ween 𝑥and i s cen oid. I s ou pu o heuni iscalcula edasanonlinea unc iono his dis ance. Assuming ha he e a e inpu nodes and mou pu nodes, he o e all esponse unc ion wi hou conside ing nonlinea i y in an ou pu node has he ollowing o m [34]: 𝑀 ∑ 𝑖=1𝑊𝑖∗𝐾(𝑥−𝑧𝑖 𝜎𝑖)=𝑀 ∑ 𝑖=1𝑊𝑖∗𝑔(󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨𝑥−𝑧𝑖󵄨󵄨󵄨󵄨󵄨󵄨󵄨󵄨 𝜎𝑖), (5) whe e 𝑀∈Nis he numbe o uni s in he hidden laye , 𝑊𝑖∈R𝑚is he ec o o weigh s linking he 𝑖 h hidden-laye uni o he ou pu nodes, xis an inpu ec o , Kis a adially symme ic ke nel unc ion o a uni in he hidden laye , z𝑖 and 𝜎𝑖a e he cen oid and smoo hing ac o o he 𝑖 h ke nel node, espec i ely, and 𝑔:[0,∞)→Ris a unc ion called he ac i a ion unc ion, which cha ac e izes he ke nel shape. 3. Case S udy In p esen s udy, da a om ai -quali y s a ions in Cas illa y Le´ on (CyL) a e analyzed. CyL is a Spanish egion loca ed a he no h-cen e o he Ibe ian Peninsula. I is composed o nine p o inces and i is he mos ex ensi e egion o Spain wi ha o alsu aceo 94,226squa ekilome e sand hesix h wi h mo e popula ion: 2,435,797 habi an s. G oss Domes ic P oduc (GDP) in CyL ep esen s he 5.3% o coun y’s GDP [35]. Clima e in CyL app oaches wha is known as he con inen al ocean, cha ac e ized by cold win e s and ho summe s wi h sho sp ing and au umn pe iods. CyL egion p o ides a wide ne wo k o s a ions [36] o he acquisi ion o ai -quali y da a. These da a a e public a ailable acco ding o he Open Da a Ini ia i e om he Spanish Go e nmen [37]. S a ions om his ne wo k ha e some in e es ing cha ac- e is ics: (1) S a ions a e classi ied in ypes: u ban, backg ound, ando ien ed o he ege a ionp o ec ion[36]. (2) These s a ions collec he undamen al ai -quali y pollu an s, and among hem is he O3,whichis he objec i e pollu an o his s udy. Daily a e ages da a [38] o each pollu an a e p o ided in each loca ion. (3) This da a p esen s emp y o co up ed da a in all o i s a iablesinsome owsandina easonablepe cen age o be es ima ed. Figu e 1: Loca ion o he six selec ed s a ions in CyL, by Google Maps. In hep esen s udy,pollu an da a eco dedinsix di e en s a ions om he CyL ne wo k a e analyzed. Daily da a a e ages om yea s 2000 o 2008 ha e been selec ed. Fo some pe iods o ime wi hin he selec ed ime window, da a a e no a ailable o all he a iables and, hus, he whole example is ejec ed o he s udy. Th ee o he s a ions a e loca ed in he cen e o he ci ies and labeled as u ban s a ions; hese s a ions a e o ien ed o he p o ec ion o he human heal h. The o he h ee s a ions a e backg ound s a ions and a e also o ien ed o he p o ec ion o he human heal h. These s a ions measu e a g ea e numbe o pollu an s han he o he ype o s a ions and a e he mos impo an ones in e ms o ai quali y, and many o hema eno collec eda hes a ions o he ege a ion p o ec ion. This ac is impo an o he de e mina ion o he O3missing alues,as hisgasisespeciallyha m ul o human heal h. The h ee u ban s a ions conside ed in p esen s udy a e as ollows: (1) ´ A ila. “Bus S a ion” s a ion. Geog aphical coo di- na es: 40.65914, −4.68237; 1150 me e s abo e sea le el (masl). (2) A anda de Due o. “Ja dines de Don Diego” s a ion. Geog aphical coo dina es: 41.67111, −3.68388; 801 masl. (3) Le´ on.“A da.SanIgnaciodeLoyola”s a ion.Geo- g aphical coo dina es: 42.60388, −5.58722; 838 masl. The h ee backg ound s a ions a e as ollows: (1) Bu gos. “Fuen es Blancas” s a ion. Geog aphical co- o dina es: 42.33611, −3.63611; 929 masl. (2) Sego ia. “Acueduc o” s a ion. Geog aphical coo di- na es: 40.95555, −4.11055; 951 masl. (3) Medina del Campo (Valladolid). “Bus S a ion” s a ion. Geog aphical coo dina es: 41.31638, −4.90916; 721 masl. Figu e1shows heloca iono hesixselec eds a ions ha ha e been s udied in he p esen pape . The pollu an s ga he ed in he abo e-men ioned s a ions andanalyzedin hep esen s udya eas ollows: 6Complexi y Table 1: Co ela ion ma ix o he six a iables in he da ase . O3CO NO NO2PM10 SO2 O31.000 −0.123 −0.161 −0.202 0.072 −0.013 CO −0.123 1.000 0.360 0.412 0.358 0.299 NO −0.161 0.360 1.000 0.540 0.233 0.330 NO2−0.202 0.412 0.540 1.000 0.330 0.257 PM10 0.072 0.358 0.233 0.330 1.000 0.251 SO2−0.013 0.299 0.330 0.257 0.251 1.000 Table 2: Pe cen age o missing and co up ed da a o each one o he analyzed a iables. O3NO NO2CO PM10 SO2 Missing 8.104% 8.020% 8.034% 8.554% 9.131% 8.196% Co up ed 1.857% 2.047% 1.815% 2.926% 2.413% 1.801% To al 9.961% 10.067% 9.849% 11.480% 11.544% 9.997% (1) Ozone (O3), 𝜇g/m3, seconda y pollu an . See Sec- ion 1. (2) Ca bon monoxide (CO), mg/m3,p ima ypollu an . I is an odo less, colo less gas o med by he incom- ple e combus ion o uels. When people a e exposed o CO gas, he CO molecules will displace he oxygen in hei bodies and lead o poisoning [39]. (3) Ni ic oxide (NO), 𝜇g/m3,p ima ypollu an .NOis a colo less gas which eac s wi h ozone unde going apid oxida ion o NO2, p edominan in he a mo- sphe e [39]. (4) Ni ogen dioxide (NO2), 𝜇g/m3,p ima ypollu an . F om he s andpoin o heal h p o ec ion, ni ogen dioxide has se exposu e limi s o long and sho du a ion [39]. (5) Pa icula e ma e (PM10), 𝜇g/m3,p ima ypollu an . These pa icles emain s able in he ai o long pe i- ods o ime wi hou alling o he g ound and can be mo ed signi ican dis ances by he wind. I is de ined by he ISO as ollows: “pa icles which pass h ough a size-selec i e inle wi h a 50% e iciency cu -o a 10 𝜇m ae odynamic diame e . PM10 co esponds o he ‘ ho acic con en ion’ as de ined in ISO 7708:1995, Clause 6” [40]. (6) Sulphu dioxide (SO2), 𝜇g/m3,p ima ypollu an .I is a gas. I smells like bu n ma ches. I s smell is also su oca ing. SO2is p oduced by olcanoes and in a ious indus ial p ocesses. In he ood indus y, i is also used o p o ec wine om oxygen and bac e ia [39]. P ima y pollu an s a e injec ed in o he a mosphe e di- ec ly. Seconda y pollu an s a e o med in he a mosphe e h ough chemical and pho ochemical eac ions om he p ima y pollu an s [36]. All da a om hese six a iables we e no malized o he s udy. On he o he hand, all o hem a e highly deco ela ed. Table 1 shows he co ela ion ma ix o he six pollu an s o he case s udy. I is wo h men ioning ha O3is he mos independen pollu an , as i s co ela ion coe icien s wi h he es o he a iables a e close o ze o. The e a e a o al o 13,526 samples, as one sample pe day (daily a e age) was collec ed o he wel e mon hs o e e y yea , be ween yea s 2000 and 2008, in he six s a ions analyzed in his s udy. Missing o co up ed da a appea in all he a iables in some ows, which a e omi ed o he s udy. Table 2 shows he pe cen age o missing o co up ed da a p esen edineach a iablein hewholeda ase . All hesampleswi ha leas onemissingo co up ed alue we e emo ed om he da ase . 4. Expe imen s, Resul s, and Discussion The main a ge o his pape is o ill missing O3 alues in ai pollu ion da ase s. To do so, se e al impu a ion me hods a e comp ehensi ely compa ed as desc ibed below. 4.1. Expe imen al Se ings. The impu a ion me hods desc ib- ed in Sec ion 2 a e applied o di e en da ase s, all o hem wi h he six a iables desc ibed in Sec ion 3: (1) The Whole Da ase (WD), comp ising he 13,526 sam- ples: esul s o his da ase s a e shown in Sec ion 4.2. (2) The Season Da ase (SD): samples in WD a e spli in ou subse s acco ding o he ou seasons o he yea : sp ing (3,453 samples), summe (3,349 samples), au umn (3,295 samples), and win e (3,429 samples). Resul s o his da ase a e shown in Sec ion 4.3. (3) The Type s a ion Da ase (TD): samples in WD a e spli in o wo subse s acco ding o he ype o he s a ion whe e he da a come om; “u ban” (6,763 samples) o “backg ound” (6,763 samples). Resul s o his da ase s a e shown in Sec ion 4.4. Fo he h ee da ase s, bo h s a is ical and neu al impu a- ion me hods we e applied and he pe o mance is calcula ed h ough n- old C oss-Valida ion (CV). The main idea behind CV is o spli da a, no mally many imes, o es ima ing he Complexi y 7 Table 3: Linea eg ession and nonlinea eg ession esul s o he WD. Me hod MSE Time (s) Mean STD Mean STD MLR 5.490𝐸−06 2.311𝐸−08 0.089 0.216 MN-LR 5.415E −06 2.437𝐸−08 2.143 0.254 Table 4: Radial-basis unc ion ne wo k esul s o he WD. #o neu ons MSE Time (s) Mean STD Mean STD 10 5.104E −06 2.723𝐸−08 0.050 1.091 30 5.108𝐸−06 1.273𝐸−08 0.050 1.091 50 5.105𝐸−06 2.513𝐸−08 0.047 0.098 isk, e o , o pe o mance o each algo i hm. Pa o da a ( he aining samples) is used o aining each algo i hm, and he emaining pa ( he alida ion samples) is used o alida ing he algo i hm(s). Then, CV selec s he algo i hm wi h he smalles es ima ed isk [41]. CV p e en s om o e i ing because he aining sample is independen o he alida ion sample. The numbe o he 𝑘pa ame e s (da a pa i ions) was 10 o all he expe imen s in he p esen s udy. I means ha 90% o he da a a e used o aining and 10% o alida ion. In he case o neu al models, he aining p ocess is epea ed en imes (one o each old). In he case o MLP, aining is also epea ed o each aining algo i hm (see Sec ion 2.2). Fo all he expe imen s he Mean and he S anda d De ia ion (STD) o he Mean Squa e E o (MSE) o he en olds a ep esen edinTables3–11.TheMeanand heSTDo he execu ion ime (in seconds) a e also p esen ed in Tables 3–11 o he 10 olds. Fo MLP and RBFN di e en ne wo k opologies ha e been applied: combina ions o 10, 20, and 30 neu ons in he hidden laye . Addi ionally, in he case o MLP, he model is ained 10 imes wi h he same combina ion o pa ame e s o educe he e ec o andomness and ge mo e s a is ically signi ican esul s. 4.2. Resul s om he Whole Da ase . In his sec ion, esul s in e ms o MSE and execu ion ime when applying MLR, MN- LR, RBFN, and MLP o he WD a e p esen ed. In Tables 3 and 4, i can be obse ed ha he MSE Mean alues o he de e mina ion o he O3a e e y simila o he h ee applied me hods (MLR, MN-LR, and RBFN). In he case o RBFN, sligh ly lowe alues o MSE a e ob ained, wi h he lowes one being ob ained wi h 10 neu ons in he hidden laye . Rega ding execu ion imes, he MN-LR me hod u ns ou o be he slowes and RBFN he quicke . The high alues o STD o he un imein hecaseo RBFNa edue o he ac ha i g ea ly a ies om one old o he o he s. As i can be seen in Table 5, he LM, SCG, and BR aining algo i hms p esen he lowes alues o MSE Mean in all cases (10, 30, and 50 neu ons) and e y close o hose shown in Tables 3 and 4. The lowes alue o MSE was ob ained wi h heLMlea ningalgo i hmand50neu ons.Thelea ning algo i hm ha a ained he wo s esul s (in e ms o MSE) is GDX. Wi h espec o execu ion ime, he SCG algo i hm a ained he bes esul s, while LM and BR a e he second bes ones, while TB was he slowes o he i e algo i hms. Ob iously, he aining algo i hms ake mo e ime when 50 neu ons a e de ined in he hidden laye , he TB algo i hm being he one wi h g ea es e ec . 4.3. Resul s om he Season Da ase . In Tables 6–8 esul s o applying MLR, MN-LR, RBFN, and MLP o subse s wi h da a om he ou seasons o he yea (sp ing, summe , au umn, and win e ) a e p esen ed. In Tables 6 and 7 he 3 me hods p esen simila alues in MSE Mean, and he lowes MSE Mean is achie ed by he RBFN wi h 50 neu ons in he hidden laye o he summe season. The MSE Mean alues a e highe han ha obse ed o heWD.Theseasono heyea wi h helowes alues o MSE Mean is he summe . One eason may be ha he e a e ew a ia ions in pollu ion condi ions du ing summe ime. This is due o he small a ia ion in wea he condi ions du ing summe as well as low indus ial ac i i y and a ic in u ban a eas due o aca ion ime. Fu he mo e, co ela ion coe icien s in mo e han 20 pollu an s analyzed in [42] a e highe o measu emen s in he summe compa ed wi h co ela ions o measu emen s o e all days combined. The season o he yea wi h he wo s esul s in he calcula ion o he MSE has been he au umn in he case o he wo eg ession echniques and RBFN, al hough he di e ences be ween he h ee seasons (sp ing, summe , and au umn) is no signi ican . In e ms o execu ion ime, i is p obed once again ha MN-LR is he slowes me hod, while RBFN is he quickes one, e u ning e y simila esul s o he ou seasons o he yea . In Table 8, simila ly o Table 5, he aining algo i hms ha achie e he bes esul s in e ms o MSE Mean a e LM, SCG, and BR. LM achie ed he bes alue o MSE Mean in 10 o he 12 cases shown in Table 8, being exceeded by BR by a minimum alue o he win e and sp ing seasons wi h a con igu a ion o 10 neu ons. GDX eco ds he wo s MSE aluesin he12casesshowninTable8.Again, hebes MSE Mean is ob ained o he summe season, educing he MSE Mean in compa ison wi h hose egis e ed by RBFN. The season o he yea wi h he wo s esul s in he calcula ion o 8Complexi y Table 5: Mul ilaye pe cep on esul s o he WD. # o neu ons T aining algo i hm MSE Time (s) Mean STD Mean STD 10 LM 4.731𝐸−06 5.143𝐸−08 0.070 0.287 GDX 1.129𝐸−04 6.825𝐸−05 0.317 0.287 TB 5.889𝐸−05 4.973𝐸−05 0.642 0.022 SCG 5.216𝐸−06 1.514𝐸−07 0.060 0.001 BR 4.775𝐸−06 1.092𝐸−07 0.074 0.003 30 LM 4.599𝐸−06 1.015𝐸−07 0.102 0.442 GDX 4.045𝐸−04 3.523𝐸−04 0.481 0.442 TB 4.223𝐸−05 2.087𝐸−05 1.420 0.025 SCG 5.162𝐸−06 1.19𝐸−07 0.063 0.001 BR 4.727𝐸−06 5.667𝐸−08 0.102 0.005 50 LM 4.512E −06 8.952𝐸−08 0.160 1.080 GDX 1.541𝐸−04 3.722𝐸−04 0.648 1.080 TB 4.812𝐸−05 3.032𝐸−05 2.156 0.051 SCG 5.014𝐸−06 8.322𝐸−08 0.068 0.001 BR 4.731𝐸−06 1.099𝐸−07 0.161 0.010 Table 6: Linea eg ession and nonlinea eg ession esul s o he Season Da ase . Subse Me hod MSE Time (s) Mean STD Mean STD Sp ing MLR 1.895𝐸−05 1.406𝐸−07 0.085 0.208 MN-LR 1.895𝐸−05 1.242𝐸−07 0.169 0.298 Summe MLR 2.101𝐸−05 1.447𝐸−07 0.085 0.215 MN-LR 1.343E −05 1.365𝐸−07 0.665 0.321 Au umn MLR 2.106𝐸−05 2.079𝐸−07 0.085 0.208 MN-LR 2.101𝐸−05 1.447𝐸−07 0.677 0.259 Win e MLR 1.895𝐸−05 1.406𝐸−07 0.088 0.214 MN-LR 1.895𝐸−05 1.242𝐸−07 0.168 0.274 Table 7: Radial-basis unc ion ne wo k esul s o he Season Da ase . Subse # o neu ons MSE Time (s) Mean STD Mean STD Sp ing 10 1.845𝐸−05 1.687𝐸−07 0.046 0.096 30 1.847𝐸−05 1.06𝐸−07 0.050 0.098 50 1.846𝐸−05 1.297𝐸−07 0.047 0.096 Summe 10 1.308𝐸−05 1.549𝐸−07 0.045 0.098 30 1.310𝐸−05 1.389𝐸−07 0.045 0.096 50 1.306E −05 1.344𝐸−07 0.047 0.096 Au umn 10 1.986𝐸−05 1.176𝐸−07 0.046 0.096 30 1.987𝐸−05 2.323𝐸−07 0.046 0.097 50 1.987𝐸−05 2.199𝐸−07 0.045 0.095 Win e 10 1.845𝐸−05 1.687𝐸−07 0.046 0.100 30 1.847𝐸−05 1.060𝐸−07 0.045 0.093 50 1.846𝐸−05 1.297𝐸−07 0.046 0.096 Complexi y 9 Table 8: Mul ilaye pe cep on esul s o he Season Da ase . Subse # o neu ons T aining algo i hm MSE Time (s) Mean STD Mean STD Sp ing 10 LM 1.696𝐸−05 2.550𝐸−07 0.063 0.086 GDX 3.298𝐸−04 8.39𝐸−04 0.183 0.086 TB 5.870𝐸−05 2.590𝐸−05 0.383 0.049 SCG 1.959𝐸−05 3.439𝐸−07 0.056 0.001 BR 1.695𝐸−05 1.988𝐸−07 0.076 0.015 30 LM 1.531𝐸−05 4.107𝐸−07 0.068 0.217 GDX 6.652𝐸−04 1.051𝐸−03 0.210 0.217 TB 1.069𝐸−04 3.909𝐸−05 0.473 0.019 SCG 1.870𝐸−05 2.857𝐸−07 0.055 0.005 BR 1.562𝐸−05 3.915𝐸−07 0.071 0.225 50 LM 1.473𝐸−05 4.138𝐸−07 0.080 0.181 GDX 6.286𝐸−04 1.200𝐸−03 0.253 0.181 TB 1.564𝐸−04 1.402𝐸−04 0.770 0.075 SCG 1.809𝐸−05 3.768𝐸−07 0.056 0.001 BR 1.580𝐸−05 3.697𝐸−07 0.089 0.490 Summe 10 LM 1.009𝐸−05 1.203𝐸−07 0.059 0.0663 GDX 4.718𝐸−04 4.791𝐸−04 0.166 0.0663 TB 7.65𝐸−05 6.231𝐸−05 0.355 0.0396 SCG 1.217𝐸−05 2.749𝐸−07 0.053 0.0007 BR 1.010𝐸−05 1.436𝐸−07 0.064 0.0059 30 LM 9.171𝐸−06 2.713𝐸−07 0.065 0.440 GDX 8.070𝐸−04 1.446𝐸−03 0.202 0.440 TB 9.117𝐸−05 5.051𝐸−05 0.500 0.047 SCG 1.118𝐸−05 4.118𝐸−07 0.056 0.001 BR 9.822𝐸−06 3.107𝐸−07 0.073 0.004 50 LM 8.673E −06 3.284𝐸−07 0.083 0.225 GDX 4.572𝐸−04 9.864𝐸−04 0.253 0.225 TB 1.269𝐸−04 6.38𝐸−05 0.743 0.019 SCG 1.089𝐸−05 1.561𝐸−07 0.057 0.001 BR 9.851𝐸−06 2.165𝐸−07 0.085 0.003 Au umn 10 LM 1.622𝐸−05 2.146𝐸−07 0.061 0.101 GDX 1.598𝐸−04 3.096𝐸−04 0.168 0.101 TB 7.248𝐸−05 3.589𝐸−05 0.351 0.058 SCG 1.904𝐸−05 2.617𝐸−07 0.055 0.001 BR 1.628𝐸−05 7.680𝐸−07 0.071 0.009 30 LM 1.495𝐸−05 3.564𝐸−07 0.067 0.196 GDX 1.045𝐸−03 1.520𝐸−03 0.204 0.196 TB 1.09𝐸−04 6.737𝐸−05 0.506 0.048 SCG 1.808𝐸−05 2.756𝐸−07 0.054 0.001 BR 1.522𝐸−05 3.456𝐸−07 0.069 0.001 50 LM 1.401𝐸−05 1.926𝐸−06 0.079 0.307 GDX 5.676𝐸−04 1.700𝐸−03 0.240 0.307 TB 1.005𝐸−04 6.447𝐸−05 0.734 0.029 SCG 1.758𝐸−05 5.509𝐸−07 0.054 0.001 BR 1.559𝐸−05 6.591𝐸−07 0.083 0.103