Predictive modelling : flight delays and associated factors hartsfield–Jackson Atlanta international airport
Abstract
Atualmente, um ponto negativo nas viagens de avião são os atrasos que, constantemente, são anunciados aos passageiros resultando numa diminuição da sua satisfação enquanto clientes. Este e outros fatores fazem com que elevados custos, tanto quantitativos como qualitativos sejam imputados às companhias. Consequentemente, existe a necessidade de prever e mitigar a existência de atrasos aéreos que pode ajudar as companhias aéreas bem como aeroportos a melhorar a sua performance e a aplicar algumas medidas, dirigidas ao consumidor, que permitiam atenuar ou até anular o efeito que estes atrasos provoca nos seus passageiros. Deste modo, este estudo tem como principal objetivo prever a ocorrência de atrasos nas chegadas ao aeroporto internacional de Hartsfield-Jackson. Esta estimativa será possível através da elaboração de um modelo preditivo, recorrendo a diversas técnicas de Data Mining. Com a aplicação destas técnicas, foi possível identificar as variáveis que mais contribuíram para a existência do atraso. No desenvolvimento deste trabalho, foi seguida a metodologia da descoberta de conhecimento em base de dados (conhecida em inglês por Knowledge Discovery Database, KDD). Fases como a recolha dos dados, a aplicação de técnicas de amostragem (SMOTE e Undersampling), a partição dos dados em treino e teste, o pré-processamento (dados omissos e outliers) e transformação dos dados (normalização dos dados e seleção de atributos), a definição de modelos a treinar (Decision Trees, Random Forest e Multilayer Perceptron) bem como a avaliação da performance dos modelos através de métricas variadas foram aplicadas. Depois de testar diferentes abordagens, concluiu-se que o melhor modelo é alcançado com as variáveis relacionadas com a partida, usando o algoritmo Multilayer Perceptron e aplicando a técnica de SMOTE para lidar com dados não balanceados, removendo outliers e selecionando dez variáveis usando GainRatio. Por outro lado, quando as variáveis com informação da partida são excluídas, o algoritmo que melhor se destaca é o Multilayer Perceptron usando a técnica SMOTE, mas desta vez, incluindo os outliers e com quinze variáveis selecionadas novamente pelo GainRatio. Em ambas as hipóteses, as variáveis explicativas que mais contribuem para a existência do atraso na chegada são relacionadas com o clima, com as características do avião e com a propagação do atraso. Os resultados do algoritmo de Random Forests mostraram melhor desempenho, em relação à precisão, em comparação com outros autores (Belcastro, Marozzo, Talia, & Trunfio, 2016; Choi, Kim, Briceno, & Mavris, 2016). Contrariamente, o algoritmo Multilayer Perceptron, apresentou menor precisão em comparação com outro estudo equivalente (Y. J. Kim, Choi, Briceno, & Mavris, 2016).
Full text
i
P edic i e Modelling: Fligh Delays and
Associa ed Fac o s
Inês Viana Fei ei a
Ha s ield–Jackson A lan a In e na ional Ai po
P ojec Wo k epo p esen ed as pa ial equi emen o
ob aining he Mas e ’s deg ee in In o ma ion Managemen
i
P edic i e Modelling: Fligh Delays and Associa ed Fac o s
Ha s ield–Jackson A lan a In e na ional Ai po
Inês Viana Fei ei a
MGI
2018
MGI
i
NOVA In o ma ion Managemen School
Ins i u o Supe io de Es a ís ica e Ges ão de In o mação
Uni e sidade No a de Lisboa
PREDICTIVE MODELLING: FLIGHT DELAYS AND ASSOCIATED
FACTORS
Ha s ield–Jackson A lan a In e na ional Ai po
By
Inês Viana Fei ei a
P ojec Wo k epo p esen ed as pa ial equi emen o ob aining he Mas e ’s deg ee in
In o ma ion Managemen , wi h a specializa ion in Knowledge Managemen and Business
In elligence.
Ad iso : P o esso Doc o Robe o Hen iques, NOVA IMS
Feb ua y 2018
ii
ACKNOWLEDGEMENTS
Success is a science; i you ha e he condi ions, you ge he esul .
– Osca Wilde (1854-1900)
1
The success he e implici was only possible wi h he help o abulous people who, du ing hese
mon hs, suppo ed and helped me. I would no make any sense o submi his wo k wi hou i s
acknowledging hese people and exp essing how g a e ul I am o hem.
I would like o hank my supe iso , P o esso Robe o Hen iques, whom I ha e known since
g adua ion, and whom I ha e always admi ed o his as knowledge, compe ence, and
p o essionalism. Thank you o he eachings you ga e me om he beginning, ei he om his hesis
o my jou ney h ough his uni e si y and o always been a ailable o answe my doub s guiding me
and sugges ing ways o lead he de elopmen o his wo k.
To my pa en s, Ana Lúcia and Vi o , and sis e , Joana, o among all people, be he ones who, a he
end o he day, ha e o lis en o all my dilemmas and conce ns. Fo being always a ailable,
encou aging me o go u he and gi e my all wi hou ea . Fo gi ing me com o in he ha des
hou s. Fo ha , o p o iding me wi h e e y hing I e e needed, and o helping me in inding my
own way in li e, I mus be g a e ul.
To my lo e one, Diogo Fe ei a, o being my pa ne , my bes iend, and my boy iend. Who always
helped me o see h ough he di icul ies. Who always pu he highes us in me, ne e le ing me
down. Fo his e o o help me ind solu ions ha some imes seemed no o exis . Fo he endless
hou s he has hea d me speaking o hings ha , o him, may make no sense. Fo always being by my
side and belie ing me. Fo e e y hing, and mo e g a e ul han wo ds can demons a e, hank you.
A las , bu no leas , o my dea es iends - Ma a Gal ão, Ca a ina And ade and Tiago Cos a -, ha
e en in he wo s momen s made me smile and p o ided me he bes momen s making i ole able.
Fo always being on my side in he longes hou s o his jou ney sha ing expe iences and knowledge,
bu also o dis ac ing me when I needed o.
1
Osca Fingal O'Flahe ie Wills Wilde was a amous I ish d ama is , poe and no elis ( o mo e
in o ma ion go o h p://www.woopidoo.com/biog aphy/osca -wilde/)
iii
RESUMO
A ualmen e, um pon o nega i o nas iagens de a ião são os a asos que, cons an emen e, são
anunciados aos passagei os esul ando numa diminuição da sua sa is ação enquan o clien es. Es e e
ou os a o es azem com que ele ados cus os, an o quan i a i os como quali a i os sejam
impu ados às companhias. Consequen emen e, exis e a necessidade de p e e e mi iga a exis ência
de a asos aé eos que pode ajuda as companhias aé eas bem como ae opo os a melho a a sua
pe o mance e a aplica algumas medidas, di igidas ao consumido , que pe mi iam a enua ou a é
anula o e ei o que es es a asos p o oca nos seus passagei os.
Des e modo, es e es udo em como p incipal obje i o p e e a oco ência de a asos nas chegadas
ao ae opo o in e nacional de Ha s ield-Jackson. Es a es ima i a se á possí el a a és da elabo ação
de um modelo p edi i o, eco endo a di e sas écnicas de Da a Mining. Com a aplicação des as
écnicas, oi possí el iden i ica as a iá eis que mais con ibuí am pa a a exis ência do a aso.
No desen ol imen o des e abalho, oi seguida a me odologia da descobe a de conhecimen o em
base de dados (conhecida em inglês po Knowledge Disco e y Da abase, KDD). Fases como a ecolha
dos dados, a aplicação de écnicas de amos agem (SMOTE e Unde sampling), a pa ição dos dados
em eino e es e, o p é-p ocessamen o (dados omissos e ou lie s) e ans o mação dos dados
(no malização dos dados e seleção de a ibu os), a de inição de modelos a eina (Decision T ees,
Random Fo es e Mul ilaye Pe cep on) bem como a a aliação da pe o mance dos modelos a a és
de mé icas a iadas o am aplicadas.
Depois de es a di e en es abo dagens, concluiu-se que o melho modelo é alcançado com as
a iá eis elacionadas com a pa ida, usando o algo i mo Mul ilaye Pe cep on e aplicando a écnica
de SMOTE pa a lida com dados não balanceados, emo endo ou lie s e selecionando dez a iá eis
usando GainRa io. Po ou o lado, quando as a iá eis com in o mação da pa ida são excluídas, o
algo i mo que melho se des aca é o Mul ilaye Pe cep on usando a écnica SMOTE, mas des a ez,
incluindo os ou lie s e com quinze a iá eis selecionadas no amen e pelo GainRa io.
Em ambas as hipó eses, as a iá eis explica i as que mais con ibuem pa a a exis ência do a aso na
chegada são elacionadas com o clima, com as ca ac e ís icas do a ião e com a p opagação do
a aso.
Os esul ados do algo i mo de Random Fo es s mos a am melho desempenho, em elação à
p ecisão, em compa ação com ou os au o es (Belcas o, Ma ozzo, Talia, & T un io, 2016; Choi, Kim,
B iceno, & Ma is, 2016). Con a iamen e, o algo i mo Mul ilaye Pe cep on, ap esen ou meno
p ecisão em compa ação com ou o es udo equi alen e (Y. J. Kim, Choi, B iceno, & Ma is, 2016).
PALAVRAS-CHAVE
Da a Mining; Análise P edi i a; A aso Aé eo; Ae opo o In e nacional de Ha s ield–Jackson;
Ae opo o In e nacional de A lan a.
i
ABSTRACT
Nowadays, a downside o a eling is he delays ha a e cons an ly ad e ised o passenge s
esul ing in a dec ease in cus ome sa is ac ion. These delays associa ed wi h o he ac o s can cause
cos s, bo h quan i a i e and quali a i e. Consequen ly, he e is a need o an icipa e and mi iga e he
exis ence o ai bo ne delays ha can help ai lines and ai po s imp o ing hei pe o mance o e en
ake some consume -o ien ed measu es ha can undo o a enua e he e ec ha hese delays ha e
on hei passenge s.
This s udy has as p ima y objec i e o p edic he occu ence o a i al delays o he in e na ional
ai po o Ha s ield-Jackson. I was possible by building a p edic i e model, applying se e al Da a
Mining echniques. Wi h hese applica ions, i was possible o show he a iables, among he
p oposals, ha mos con ibu ed o he exis ence o he delay.
In his wo k, he Knowledge Disco e y Da abase (KDD) me hodology was ollowed. Phases such as
da a collec ion; sampling echniques (SMOTE and Unde sampling); Da a pa i ioning in aining and
es ing; P e-p ocessing (missing da a and ou lie s) and da a ans o ma ion (da a no maliza ion and
a ibu e selec ion); And, inally he de ini ion o models o be ained (Decision T ees, Random
Fo es s, and Mul ilaye Pe cep on), as well as he e alua ion o he pe o mance o he models
h ough a ied me ics, we e used.
A e es ing di e en app oaches, i was concluded ha he bes model is achie ed wi h he
a iables ela ed o depa u e, using he Mul ilaye Pe cep on algo i hm and applying SMOTE o
deal wi h unbalanced da a, emo ing ou lie s and selec ing en a iables using GainRa io.
On he o he hand, when he a iables wi h in o ma ion o he depa u e a e excluded, he algo i hm
ha pe o ms bes is also he Mul ilaye Pe cep on using he SMOTE echnique bu , his ime,
including he ou lie s and wi h i een a iables selec ed again by he GainRa io.
On bo h hypo heses, he explana o y a iables ha mos con ibu ed o he exis ence o he delay in
a i als we e ela ed o he wea he , he ai plane cha ac e is ics and he p opaga ion o he delay.
Ou esul s o he Random Fo es s algo i hm shown be e pe o mance, ega ding accu acy,
compa ed o o he au ho s (Belcas o e al., 2016; Choi e al., 2016). Con a y, o he Mul ilaye
Pe cep on algo i hm, was p esen ed a lowe accu acy compa ed o ano he equi alen s udy (Y. J.
Kim e al., 2016).
KEYWORDS
Da a Mining; P edic i e Analysis; Fligh Delays; Ha s ield–Jackson In e na ional Ai po ; A lan a
In e na ional Ai po .
INDEX
1. In oduc ion .................................................................................................................. 1
1.1. Con ex and Rele ance .......................................................................................... 1
1.2. Objec i e ................................................................................................................ 3
1.3. S udy Ou line ......................................................................................................... 4
2. Li e a u e Re iew ......................................................................................................... 5
3. Me hodology .............................................................................................................. 13
3.1. Selec ion .............................................................................................................. 14
3.1.1. S udy Scope .................................................................................................. 14
3.1.1.1. Geog aphical ........................................................................................ 14
3.1.1.2. Tempo al .............................................................................................. 14
3.1.2. Da a .............................................................................................................. 15
3.1.2.1. Sou ces ................................................................................................. 15
3.1.2.2. Da a Valida ion and Limi a ions........................................................... 16
3.1.2.3. Da ase Cons uc ion and Desc ip ion ................................................. 17
3.1.3. Sampling Techniques .................................................................................... 23
3.1.4. Da a Pa i ion ............................................................................................... 25
3.2. Da a P e-P ocessing ............................................................................................. 26
3.2.1. Explo a o y Da a Analysis ............................................................................. 27
3.2.2. Missing Values .............................................................................................. 35
3.2.3. Ou lie s ......................................................................................................... 37
3.3. Da a T ans o ma ion ........................................................................................... 39
3.3.1. No maliza ion ............................................................................................... 39
3.3.2. Va iable Selec ion ......................................................................................... 39
3.4. Da a Mining ......................................................................................................... 42
3.4.1. Decision ees ............................................................................................... 43
3.4.2. Random Fo es s ............................................................................................ 44
3.4.3. Mul ilaye Neu al Ne wo ks ......................................................................... 44
3.5. E alua ion ............................................................................................................ 46
4. Resul s and Discussion ................................................................................................ 48
5. Conclusions ................................................................................................................. 58
6. Limi a ions and Recommenda ions o Fu u e Wo ks ............................................... 60
7. Re e ences .................................................................................................................. 62
8. Annexes ...................................................................................................................... 71
i
Annex 1: P oceedings o Da a Valida ion by Type o Sou ce Da a In o ma ion ....... 71
Annex 2: En i y- ela ionship Physical Model wi h C ow’s Foo No a ion .................. 73
Annex 3: Dis inc ai po s and Ci ies wi h espec o ime-zone ............................... 73
Annex 4: Ai line Companies........................................................................................ 77
Annex 5: Fede al Holidays .......................................................................................... 78
Annex 6: Acquisi ion o he 3 Phases o he Wea he Va iables ................................ 79
Annex 7: In luence o A i al delay on Tempe a u e a iables .................................. 80
Annex 8: In luence o A i al delay on P ecipi a ion a iables .................................. 81
Annex 9: In luence o A i al delay on Wind a iables .............................................. 82
Annex 10: In luence o A i al delay on Visibili y a iables ....................................... 83
Annex 11: Tes Resul s Table o SMOTE Technique ................................................... 84
Annex 12: Tes Resul s Table o Unde sampling Technique ...................................... 85
2
Indi ec ly, he ine iciency in he ai line indus y inc eases he cos o doing business o o he sec o s,
making he associa ed business less p oduc i e, e lec ing a educ ion o 4 billion dolla s in he
coun y’s G oss Domes ic P oduc (GDP). Di ec ly, delays ep esen a cos o 28.9 billion dolla s
whe e:
▪ 16.7 billion dolla s ep esen s he passenge componen (los ime, delayed ligh s,
missed connec ions, among o he s);
▪ 8.3 billion dolla s e e s o he ai line componen ( echnical eam, uel, main enance,
among o he s);
▪ 3.9 billion dolla s ela es o cus ome s who a oid a eling because o delays.
This phenomenon, in addi ion o quan i a i e cos s, also has quali a i e cos s ha in luence he
o me one. Fo he passenge , a ec s his plans, which can cause displeasu e ega ding he company.
Acco ding o he A ia ion Consume P o ec ion Di ision (ACPD) om he O ice o A ia ion
En o cemen and P oceedings (OAEP) (2016) be ween Janua y and Decembe o 2015, six housand,
ou hund ed and hi y- h ee (6433) consume complain s we e epo ed ega ding ligh p oblems
such as cancella ions, delays, and missed connec ions. As a esul , ep esen ing a quali a i e cos o
he ai line, demand, and epu a ion may be ad e sely a ec ed when he e is compe i ion on he
same ou e because passenge ’s choice o ai line can be based on pas e en s (Mazzeo, 2003).
The concep o a eling has been shaping o e ime. In he pas , i was seen as a p i ilege. O e he
yea s he ci cums ances ha e changed, and oday, a el o en ep esen s a necessa y e il. A esul
o ai delays, inc eased secu i y measu es, and deg ada ion o se ices p o ided (Ball e al., 2010).
The analysis o ai delays becomes i al since a be e knowledge o hei exis ence, and
co esponding igge s, can imp o e he pe o mance o ai lines and, consequen ly ai po s, in hei
ope a ions by he possibili y o an icipa ion (Yablonsky e al., 2014) and cons uc ion o schedules o
example.
I should be conside ed he analysis o he delays wi h ocus on he a i als since hese a e mo e
ela ed wi h he passenge 's sa is ac ion and because one a i al delay may igge a delay in a
depa u e (Tu, Ball, & Jank, 2008a).
Based on epo s om se e al ai po s a ound he wo ld, passenge a ic esul s o he busies
ai po s in 2015 pu on op he Ha s ield-Jackson In e na ional Ai po in A lan a. Yea a e yea , i
showed a g ow h o 5.5% o passenge a ic eaching a eco d o mo e han 100 million passenge s
in ha same yea . This ai po bene i s om i s s a egic loca ion being a majo "ga eway" o en y
in o No h Ame ica, and also dis ance i sel o a wo hou ligh om 80% o he popula ion o he
USA. In he 2015 In e na ional Ai po s Council epo i occupies he i s place bo h in he anking o
o al passenge a ic – 101 491 106 passenge s (Table 1) – and in he anking o ai c a mo emen s
(Table 2) (Ai po s Council In e na ional, 2016).
Rank
Ai po ci y
Passenge s
1
A lan a
101 491 106
2
Beijing
89 938 628
3
Dubai
78 010 265
3
4
Chicago
76 949 504
5
Tokyo
75 316 718
Table 1: To al passenge s a ic in 2015
Sou ce: Made by he au ho , adap ed om (Ai po s Council In e na ional, 2016)
Rank
Ai po ci y
Ai c a Mo emen s
1
A lan a
882 497
2
Chicago
875 136
3
Dallas
681 244
4
Los Angeles
655 564
5
Beijing
590 169
Table 2: Ai c a mo emen s in 2015
Sou ce: Made by he au ho , adap ed om (Ai po s Council In e na ional, 2016)
1.2. OBJECTIVE
To cha ac e ize a ligh , da a as he in o ma ion o ai plane numbe , he ai line company, he o igin
and des ina ion, he schedule and ac ual depa u e and a i al ime, he wea he condi ions, among
o he s a e ypically used (Abdel-A y, Lee, Bai, Li, & Michalak, 2007; AhmadBeygi e al., 2008;
Belcas o e al., 2016; Choi e al., 2016; Ionescu, Gwiggne , & Kliewe , 2016; Khanmohammadi,
Tu un, & Kucuk, 2016; M. S. Kim, 2016; Y. J. Kim e al., 2016; Klein, C aun, & Lee, 2010; Muelle &
Cha e ji, 2002; Py gio is, Malone, & Odoni, 2013; Qianya, Lei, Rong, Bin, & Xinhong, 2015; Rebollo &
Balak ishnan, 2014; Xu, Donohue, Laskey, & Chen, 2005; Yao, Jiandong, & Tao, 2010; Zonglei,
Jiandong, & Tao, 2009). This da a can become aluable when applied in a model o o ecas ing
delays in u u e ligh s.
Thus, wi h his s udy, i is in ended o p edic he occu ence o a delay in he a i als o Ha s ield-
Jackson In e na ional Ai po based on he e e ed a iables, u he ones conside ed in he
Me hodology chap e , and hei espec i e con ibu ion o he delay.
The s eps equi ed o each he inal objec i e unde go:
▪ Cons uc a da abase wi h in o ma ion conce ning he ligh s and addi ional
in o ma ion;
▪ Explo e he delays acco dingly o di e en a iables;
▪ Cons uc a p edic i e model using Da a Mining and Machine Lea ning echniques o
p edic he delay o a ligh in he a i al;
▪ Apply he model de eloped in new da a o make p edic ions and see which i s be e
o he p oblem acco ding o he desi ed ad ance o he p edic ion;
▪ Iden i y he a iables ha con ibu e mos o he exis ence o delay.
A he end o his p ojec is expec ed o each an algo i hm ha pe o ms be e in he da a
acco ding o he desi ed ad ance o he p edic ion. Subsequen ly, hese esul s can be compa ed o
esul s p esen ed in ea lie s udies wi h he same con ex o his wo k.
4
1.3. STUDY OUTLINE
This p ojec p esen s an in oduc o y chap e explaining he con ex in which he heme is inse ed
and nowadays ele ance, as well as he main objec i es adjacen o his wo k.
Chap e wo p esen s a li e a u e e iew abou he need o KDD, Da a Mining, and Machine
Lea ning when dealing wi h issues whe e la ge olumes o da a a e inhe en , such as ai bo ne delays.
I also p esen s s a e o he a , abou he s udy o ai delays, bo h a an indus ial communi y le el
as well as a he scien i ic communi y le el.
The hi d chap e exhibi he a ious s ages de ined o he de elopmen o his wo k. He e, he
scope o he s udy is de ined. The en i e p ocess o making he da a a ailable, p ocessing and
cons uc ion o he da ase is explained. Also, he a ious decisions made ega ding he da a
p ep ocessing and ans o ma ion as well as he selec ion o he algo i hm a e desc ibed.
In he ou h chap e , he esul s o he applica ion o he me hodology chap e a e illus a ed and
analyzed. The bes app oaches a e selec ed o each o he algo i hms chosen, and o each o he
hypo hesis o ad ance o he p edic ion. Fu he mo e, ou esul s a e compa ed o s udies in he
same a ea o esea ch and wi h he same a ge ype a iables, bu also o websi es ha p edic s
delays.
Following is he i h chap e , whe e is p esen he conclusions unde lying his wo k, as well as he
chosen o he bes model among all algo i hms and app oaches. Limi a ions and u u e wo ks a e
p oposed in he six h chap e .
5
2. LITERATURE REVIEW
We a e d owning in in o ma ion bu s a ed o knowledge.
– John Naisbi
3
Acco ding o Yablonsky e al. (2014) in June 2010 an inqui y was made abou he wel e mos
oublesome aspec s o a elling. The ligh delays held he se en h place wi h a sco e o 6.8
acco ding o he scale o 1 (less annoying) o 10 (mo e annoying).
Since 2010, he amoun o delays has been luc ua ing, wi h he yea o 2014 ha ing he highes
pe cen age o delay. In 2015, he pe cen age o delayed ligh s was 19.53%, he hi d highes alue
since 2010 (Figu e 1).
O e ime i has become a phenomenon o g ea impo ance. As in o he a eas, he e is a g owing
numbe o uns uc u ed eco ds ha ha e been s o ed and do no add alue o hei pu e s a e.
Wi en, F ank, & Hall (2011a) men ioned ha we a e o e loaded wi h da a and canno con ol he
exponen ial g ow h a ound us. Mo e p ecisely, hey men ioned ha he amoun o da a in he
wo ld's da abases doubles e e y wen y (20) mon hs. In a business con ex , he la ge olume o da a
p e en s he e ec i e use o he same o c ea e business alue, compe i i eness and e iciency. This
made he lack o unde s anding in how o each ad an ageous in o ma ion a colossal obs acle
(La alle, Lesse , Shockley, Hopkins, & K uschwi z, 2011).
As a esul , a gap is c ea ed be ween he p oduc ion o da a and ou unde s anding o i . I is c ucial
o o e come his de ici because in da a he e is in o ma ion bene icial o decision making, and
consequen ly knowledge. To acqui e knowledge om da a he use o ools is equi ed o allow he
disco e y o hidden in o ma ion in hese da abases. Acco ding o Fayyad, Pia e sky-Shapi o, & Smy h
(1996), he ield o Knowledge Disco e y in Da abases, known as KDD (Knowledge Disco e y
Da abase) is he p o ide o he necessa y ools and heo ies.
KDD consis s o he p ocess o disco e ing new, alid, use ul and pe cep ible pa e ns in he da a
(Fayyad, n.d.; Ma iscal, Ma bán, & Fe nández, 2010). Co e s phases such as (1) he selec ion o
3
John Naisbi is an Ame ican au ho and public speake in he a ea o u u es s udies ( o mo e
in o ma ion go o h ps://en.wikipedia.o g/wiki/John_Naisbi )
Figu e 1: Pe cen age o o al delayed ligh s pe yea in he USA, 2010-2017
Sou ce: Made by he au ho , adap ed om (Bu eau o T anspo a ion S a is ics, 2017)
6
c ea ion o a da ase used as he basis o he disco e y o s anda ds. (2) The p ep ocessing o he
same ones whe e unnecessa y in o ma ion is elimina ed and co ec ed o assu e he cohe ence o
he same. (3) The ans o ma ion o da a by educing dimensionali y o ans o ming exis ing ones.
(4) The applica ion o Da a Mining (DM) whe e one looks o pa e ns in he da a depending on he
objec i e. And, inally, (5) he in e p e a ion and e alua ion o he esul s (Aze edo & San os, 2008).
Figu e 2: KDD P ocess
Sou ce: Made by he au ho , adap ed om (Kononenko & Kuka , 2007b)
Thus, DM is conside ed one o he s eps o KDD as p esen ed in (Fayyad, n.d.; Fayyad e al., 1996;
Han & Kambe , 2011; Kononenko & Kuka , 2007b; X. W. X. Wang, 2009). I ep esen s he applica ion
o algo i hms o ex ac pa e ns om he da a, ansla ing in in o ma ion (Fayyad e al., 1996). I has
wo main objec i es. The i s , o ecas ing, whe e u u e da a a e p edic ed abou a pa icula
a iable o in e es based on o he a iables p esen in he da abase (pas in o ma ion). And he
second, desc ip ion, whe e he ocus goes h ough he disco e y o pa e ns hidden in he da a (X.
W. X. Wang, 2009).
As a esul , eso o DM and Machine Lea ning (ML) echniques ha e
gained impo ance in sol ing hese p oblems. The use o DM in his ype
o s udy is because i is a discipline ha ocuses on he disco e y o
knowledge in he da a ha ing as one o he objec i es, he p edic ion o
unknown da a o u u e e en s as al eady men ioned (Kan a dzic, 2011).
On he o he hand, ML is used o DM since i is a ype o app oach o
he disco e y o knowledge in he da a. Focus on he cons uc ion o
compu a ional algo i hms ha can lea n h ough da a ( om he pas ) o
make p edic ions (Wi en e al., 2011a). These disciplines, ep esen ed
in Figu e 3, enable be e decisions and ac ions in eal ime wi hou
human in e en ion because o hei high- alue p edic ions (SAS, 2017).
Belcas o, Ma ozzo, Talia, & T un io (2016) claim ha he use o ML
echniques associa ed wi h DM ools can help o pe cei e complex
phenomena as well as sol e a ious p oblems om many a eas.
The impo ance o ligh delays made hem he cen e o many in es iga ions by bo h he indus ial
and scien i ic communi y.
An example o he i s s and is Kaggle. Kaggle is a pla o m o analysis and p edic i e modeling
compe i ions. The e, companies and esea che s publish da a o people ha ha e in e es and
knowledge in he ield o compe e in he p oduc ion o models whe e can be ewa ded wi h
mone a y awa ds.
Figu e 3: Rela ion be ween
Knowledge Disco e y Da abase
(KDD), Da a mining (DM) and
Machine Lea ning (ML)
Sou ce: Made By he au ho ,
adap ed om (Kononenko &
Kuka , 2007b)
7
This ype o mechanism p o ided GE A ia ion, in collabo a ion wi h Alaska Ai lines, o launch a
con es . The con es was he GE Fligh Ques , whe e 173 eams compe ed wi h hei algo i hms o
ge a model ha p edic ed delays wi h good pe o mance (GE A ia ion, 2012). DOT, as well as o he
companies, ha e p o ided da a on ai delays and cancella ions so ha use s could p edic which
ai lines we e he bes o a el (U.S. Depa men o T anspo a ion, 2017).
On he o he hand - scien i ic a ea -, se e al app oaches we e p oposed by esea che s o o ecas
and model delays. These app oaches a y acco ding o he di e en objec i es o he o ecas
ega ding he ollowing h ee aspec s:
▪ Ne wo k – when looking a he delay in se e al ai po s and he impac o i on he
le el o all ai po s, o a g oup o hem;
▪ Ai po – when he ocus is on he s udy o he s a e o he delay, o example, abou
an ai po - and, las ly;
▪ Fligh – when is wan ed o p edic he delay o each ligh .
I is also possible o dis inguish he ype o app oach used in he de elopmen o hese s udies
(S e nbe g, Soa es, Ca alho, & Ogasawa a, 2017), such as:
▪ S a is ical Analysis (SA) – co e s he use o eg ession models, co ela ion analysis,
econome ic models, pa ame ic and non-pa ame ic es s, mul i a ia e analysis,
among o he s;
▪ Machine Lea ning (ML) – consis s o a esea ch/in es iga ion ha explo es he
de elopmen o compu a ional algo i hms ha can lea n om da a and p o ide
p edic ions om hem;
▪ Ope a ional Resea ch (OR) – includes he de elopmen o ad anced analysis models
(e.g., op imiza ion, simula ion, queue heo y, among o he s) o help s akeholde s
make be e decisions;
▪ P obabilis ic Models (PM) – co e s analysis ools ha es ima e he p obabili y o an
e en based on his o ical da a.
Fo an unde s anding o wha each au ho s udied, and wha ype o app oach implemen ed (i can
be seen a summa y in Figu e 4) i is impo an o con ex ualize i .
In his way, a a Ne wo k le el, se e al in es iga o s used Machine Lea ning me hods o unde s and
he delays. Rebollo & Balak ishnan (2014) p edic ed delays in wo ways. Th ough classi ica ion whe e
hey classi ied a depa u e delay as being mo e o less han a p ede ined h eshold. And, h ough
eg ession, whe e hey es ima ed a u u e delay in depa u e on a speci ic ou e. Resul s o he s udy
showed an a e age p ecision o 81% in he pe o mance o Random Fo es s, algo i hm selec ed o
he classi ica ion, and a mean p edic ion e o o he eg ession o 21 minu es. I should be no ed
ha , o bo h means o o ecas ing, he e o inc eased as he o ecas ad ance also inc eased.
Xu, Donohue, Laskey, & Chen (2005) ha e c ea ed a me hodology o ep esen ing and analyzing how
sys em-wide e ec s a isen om subsys em-le el causes h ough a Bayesian Ne wo k o es ima e he
delay p opaga ion. The model c ea ed used an equa ion o linea eg ession as a p io i p obabili y
(e o a e o 19.1%) ha ing supe io pe o mance abou he o he me hods s udied. The o he
me hods comp ise models whe e he Bayesian Ne wo k was es ima ed wi h he pa ame e s o he
8
aining da a using a uni o m p io dis ibu ion (38.1% e o a e), and whe e he model was based
on a linea eg ession (e o a e o 61.9%). Thus, he au ho s concluded ha he me hod could be
ex ended by including mo e ai po s.
Qianya, Lei, Rong, Bin, & Xinhong (2015), and Rong, Qianya, Bo, Jing, & Dongdong (2015) ha e
p esen ed a me hod o analysis o ligh delays also based on Bayesian Ne wo ks ha could analyze
and p edic he delay du ing a ligh . They analyzed he pe o mance o he p io p obabili ies,
c ea ed om (1) his o ical ligh da a s a is ics, (2) pos e io p obabili ies using he Expec a ion-
Maximiza ion (EM) algo i hm based on he ini e Gaussian mix u e model and (3) eal da a. They
concluded ha p io p obabili ies we e mo e accu a e (81.95%) p o ing ha Bayesian Ne wo ks a e
an e ec i e me hod o analyzing ligh s delays and ha , as in he p e ious s udy (Xu e al., 2005),
hey ha e a g ea alue o analyze sys em-le el e ec s ha esul om lowe -le el causes.
AhmadBeygi, Cohn, Guan, & Belobaba (2008) analyzed he po en ial o delays o sp ead due o
ai lines. Fo ha , hey used p opaga ion ees o compa e wo ai lines. They concluded ha he key
poin s ha had an impac on slowing he sp ead o delay we e when cabin c ews end hei shi , and
when cabin c ew and ai planes we e always oge he on all planned ligh s.
Zonglei, Jiandong, & Tao (2009) buil a ecommenda ion sys em o ale ai po s by moni o ing
ela ed ai po s and in o ming he s a us o he delay.
Howe e , o he esea che s ha e eso ed o Ope a ional Resea ch me hods o ealize he delay.
Py gio is, Malone, & Odoni (2013) conside ed he p opaga ion o delays in he ai po ne wo k
h ough a delay p opaga ion algo i hm (DPA). This acked he sp ead o ai po -calcula ed delays
and hei impac on he ai po ne wo k. The au ho s men ioned ha he goal was no o ep oduce
he exac delays bu a he he ends and beha io s ha a e obse ed in he NAS sys em - ne wo k.
In Ionescu, Gwiggne , & Kliewe (2016) he main objec i e was o unde s and he po en ial o delay
modeled h ough da a used in he obus p og amming o ai line esou ces. They p o ided a
eg ession modeling app oach o daily delay pa e ns based on he de ec ion o spa io empo al
pa e ns in his o ical da a. As a esul , ules eme ged, whe e p ecision was assessed h ough
s a is ical modeling and compa ed o non-pa ame ic andom o es s. They p esen ed, gi en all
decision ules as a whole, a 62.9% accu acy ha could be compa ed o he andom o es s. Howe e ,
he la e p esen an indi idual ca ego iza ion and selec ion o ules as well as a lack o in e p e a ion
acco dingly o he au ho s, con a y o he ules p esen ed by hem. They added ha a possible delay
migh al eady ha e been aken in o accoun h ough ai line scheduling decisions, which is no mal,
ques ioning he gene aliza ion o hei esul s o o he da a om o he ai lines.
Tu e al. (2008a) and Muelle & Cha e ji (2002) used P obabilis ic Models o model hei p oblems.
The o me de eloped a s a egic model o es ima e delays in depa u es. They used non-pa ame ic
me hods o daily and seasonal ends o delay p opaga ion o p edic depa u e delays, and mix u e
dis ibu ion o es ima e esidual e o s, used o calcula e he p obabili y o he delay. Acco ding o
he au ho s, i p esen ed a easonable adjus men quali y, obus ness in he choice o pa ame e s
and good o ecas ing capabili ies and could be easily adap ed o o he ai po /ai line combina ions.
The la e se ou o imp o e he accu acy o o ecas ing delays by calcula ing he p obabili y o
depa u e, en- ou e and a i al delay using Poisson and No mal dis ibu ions. This s udy esul ed in
9
se e al delay me ics o he analysis o an ai po ne wo k (all en ai po s expe iencing signi ican
delays), based on indi idual ai po s h ough i s 21-day e iew pe iod. They concluded ha he
depa u e delay is be e modeled h ough a Poisson dis ibu ion whe eas, he en- ou e and he
a i al delay i he No mal dis ibu ion be e . In addi ion, he au ho s also used he P obabilis ic
Models o analysing he delay a a Ai po Le el.
As p e iously men ioned, Py gio is, Malone, & Odoni (2013) used Ope a ional Resea ch o analyze
he sp ead o ai po delay and ne wo k impac . They used he App oxima e Ne wo k Delays (AND)
model composed o queueing heo y (QE) o calcula e he indi idual ai po mode delay and o be
able o analyze he ne wo k.
Yablonsky e al. (2014) and Klein, C aun, & Lee (2010) eso ed o S a is ical Analysis. The i s ones
ocused on he a e age delay ime a Ha s ield-Jackson A lan a In e na ional Ai po based on da a
o se e al yea s. The main objec i e was o de e mine he annual cyclical delays. They conside ed
ha he in o ma ion esul ing om hei s udy was ele an when o ecas ing pe iods o ai delays.
Concluded ha he e is an annual pa e n o delays being caused by he olume o ligh s a he
ai po and he amoun , and equency, o p ecipi a ion occu ing in A lan a.
The second ones, de eloped a model h ough mul iple linea eg ession ocusing in clima e- ela ed
delays. They we e based on a me ic ha helps o es ima e he impac o clima e on ligh schedules,
and an impac me ic ha helps p edic expec ed wea he in he ligh ime, WITI (Wea he Impac ed
T a ic Index) and WITI-FA (Wea he Impac ed T a ic Index - Fo ecas Accu acy), espec i ely. They
claimed ha 70% o he delay could be explained by he WITI ac o s used as explana o y a iables
and ha he model p edic ed he ime and magni ude o he clima e impac on delay success ully.
On he o he hand, he e we e also hose who used Machine Lea ning o p edic he delay a he
ai po le el. Zonglei, Jiandong, & Tao (2009) ocus on p edic ing he se iousness o he delay a
speci ic ai po s o moni o ai po delays and ale ing speci ic ai po s h ough a ecommenda ion
sys em. In his case a Chinese ai po was analysed o unde s and how his delay could in luence he
delay a a ne wo k le el and o he ai po s ela ed o i . To p edic he se iousness o he delay a
China's ai po , hey based on he K-Nea es Neighbo algo i hm using his o ical da a o compa e and
ecognize simila si ua ions in he pas . The au ho s concluded ha , since he o ecas is based on
he compa ison and analysis o his o ical da a, he p oposed model e lec ed a p ecise and apid
o ecas o e ing logical explana ions. P e iously, Zonglei, Jiandong, & Guansheng (2008) al eady had
p esen ed a me hod, whe e ins ead o based on a ecommenda ion sys em hey based on machine
lea ning, howe e , i also se ed as a la ge ale o ligh delays. They used non-supe ised lea ning
me hods, mo e speci ically clus e ing wi h he K-means algo i hm, o ex ac classes o ai po delay,
and hen used i o p edic he class o delay o he ai po in each day by applying a supe ised
lea ning me hod o he da a. They compa ed se e al supe ised lea ning me hods o p edic he
delay class, and decision ees we e he ones ha s ood ou wi h 80% o con idence.
Smi h & She y (2008) de eloped a p ocess using he Suppo Vec o Machine (SVM). Wi h i a
clima e o ecas o a pa icula a ea could be used as inpu in es ima ing an ai po 's delay. Ha ing
also he abili y o p edic he impac o wea he on u u e ligh s, i.e. how long could expec a ligh
o be delayed due o he wea he . This model, acco ding o he au ho s, showed o be co ec 83% o
he ime.
10
Yao, Jiandong, & Tao (2010) ha e ocused on p edic ing he delay p opaga ion - h ough a p oposed
algo i hm - caused by ai c a , cockpi and cabin c ew. The p edici ion was o si ua ions whe e i is
necessa y o wai o hese elemen s. Fo example, i a ligh is delayed and, a i s des ina ion, he
ai c a o cabin c ew a e needed o ano he ligh , hey will delay he la e as a domino e ec .
Used he esul s o c ea e and p o ide an ala m ank o ligh delays o ai po s. They s a ed ha
hei model and algo i hm could be used o e icien ly calcula e he p opaga ion o delays caused by
equi ed ligh esou ces on he same ligh .
Balak ishna, Ganesan, She y, & Le y (2008) applied a Rein o cemen Lea ning (RL) algo i hm o
es ima e he axi-ou ime ( ime be ween he depa u e o a plane om he boa ding ga e and he
ime i akes ligh ). The Ma ko decision p ocess was used o model he p oblem being sol ed
h ough Machine Lea ning Rein o cemen Lea ning algo i hm. The au ho s we e able o achie e 60%
accu acy o he model.
Y. J. Kim, Choi, B iceno, & Ma is (2016) ha e p oposed ecu en neu al ne wo ks as a me hod o
p edic ing he s a us o day- o-day ai po -le el delay due o cap u ing sequen ial and empo al
ela ionships in he da a. S a ed ha he applica ion o Long Sho -Te m Memo y Recu en Neu al
Ne wo ks (LSTM RNN) a chi ec u es in he o ecas model allowed a mo e eliable acqui e on one-
day delay s a us o an aipo .
Howe e , he a ea o in e es in his s udy is he p edic ion o delay in an Indi idual Fligh le el and
no he ones men ioned abo e. As such, some s udies ha e been p oposed in which he delay o an
indi idual ligh was he objec i e o he o ecas .
Some au ho s used S a is ical Analysis o unde s and and de ec pa e ns on a ligh . Abdel-A y, Lee,
Bai, Li, & Michalak (2007) ha e e alua ed he pe o mance on he a i al o a ligh , h ough a wo-
s age app oach using ma hema ical equency analysis. They we e able o iden i y pa e ns o delay
whe e i was possible o de e mine which we e he mos impo an a iables ha a ec ed he delay.
Subsequen ly, he ela ionship be ween he a iables and he delay was in es iga ed h ough
s a is ical echniques in which he pe iodic pa e ns o delays we e examined. Th ough hei esul s,
hey obse ed ha he delay o a ligh was associa ed wi h he p ecipi a ion, ligh dis ance, ime o
he yea , he day o he week, ime o a i al and space o ime be ween he a i al o wo successi e
ligh s. Acco dingly o he au ho s, hei model could be adjus ed i sel o any ype o da a.
M. S. Kim (2016), concluded ha he Spline Smoo hing eg ession model su passes he linea
eg ession and median eg ession models in p edic ion pe o mance. I adjus ed be e o he da a
and was he one ha ep esen ed he delays bo h in he long e m and in he sho e m. He
conside ed ha he a iables o delay depa u e, ligh ime, ai line, wea he condi ions and ime o
he yea we e ele an in he accu acy o he o ecas , and he use o he delay a iable a depa u e
signi ican ly imp o ed he accu acy. La e , included in his s udy he a ea o P obabilis ic Models, and
sugges ed a me hod o calcula e he p obabili y o he ime o a i al o a ligh adjus ing he
esiduals o he model o a dis ibu ion Skew .
O he au ho s eso ed o Machine Lea ning o he pu pose being he ocus o ou s udy.
Y. J. Kim, Choi, B iceno, & Ma is (2016) as al eady men ioned, in addi ion o p edic ing he s a us o
he day- o-day delay a an ai po le el used neu al ne wo ks o be able o p edic he class o he
11
delay o an indi idual ligh . Ga he ed da a abou he ligh da a (day, season, mon h, da e, o igin,
des ina ion, schedule imes) and wea he da a (wind di ec ion, wind speed, cloud heigh , isibili y,
p ecipi a ion, snow accumula ion, in ensi y, desc ip o and obse a ion code). Fu he mo e, allied
he s a us o he day delay compu ed in he i s s age o hei s udy. They showed ha hei model
achie ed 87.42% accu acy, be e han he bes p edic ions un il hen demons a ed, 83.4% and 81%
(Choi, Kim, B iceno, & Ma is (2016) and Rebollo & Balak ishnan (2014), espec i ely). Due o he
acquisi ion o a mo e eliable day-delay s a us using Long Sho -Te m Memo y Recu en Neu al
Ne wo ks (LSTM RNN) a chi ec u es, he o ecas model o he delay s a e o an indi idual ligh also
became mo e accu a e.
Choi, Kim, B iceno, & Ma is (2016) also p esen ed a classi ica ion model in which he main objec i e
was o exclusi ely p edic ligh delays o indi idual ligh s caused by clima ic condi ions. They used
da a collec ed om he BTS ai line On- ime Pe o mance da ase o he yea s o 2005 o 2015 using
ea u es like ligh schedules and day. Also included wea he condi ions a iables ob ained by he
In eg a ed Su ace da abase o he Na ional Oceanic and A mosphe ic Adminis a ion (NOAA), a he
o igin (Den e In e na ional Ai po ) and he des ina ion (Cha lo e Douglas In e na ional Ai po ).
Fo he pu pose, eso ed o he use o algo i hms o da a mining and machine lea ning. Conside ed
ha he bes classi ie was he Random Fo es based on hei esul s. They also s udied he accu acy
o he esul s o di e en ho izons (5, 1 and 0 days) and concluded ha esul s wi h eal clima ic
condi ions ha e be e accu acy (26.79%, 30.36%, and 80.36% espec i ely). They men ioned ha
accu acy is highe when sampling echnique is no applied and unde s and ha his happened
because classi ie s a e biased owa ds he on- ime class, he majo i y.
In he same line o hough , Belcas o, Ma ozzo, Talia, & T un io (2016) applied a pa allel e sion o
he andom o es algo i hm o p edic he delay in he a i al o a ligh due o he wea he . The
main objec i e was o be able o p edic , wi h a ew days in ad ance, he delay in he a i al o an
indi idual ligh due o he wea he . They used da a abou he ligh in o ma ion such as schedule
imes bu also o igin and des ina ion om he ai line on- ime pe o mance (AOTP) da ase om
RITA-BTS comp ising he yea s o 2009 ill 2013. Joined a iables o wea he condi ions ( empe a u e,
humidi y, wind di ec ion and speed, ba ome ic p essu e, sky condi ions, isibili y and wea he
phenomena desc ip o ) a he o igin and des ina ion acco dingly o he ligh ime able acqui ed
om he Quali y Con olled Local Clima ological Da a (QCLCD) da ase om he Na ional Clima ic
Da a Cen e . Finally, hey s a ed ha he model had an accu acy o 85.8% o a h eshold o 60
minu es and ha e en i hey did no conside he wea he he model would achie e an accu acy o
69.1%. None heless, when conside ing a h eshold o 15 minu es, i achie ed 74.20% o accu acy.
The wo k p esen ed he e, in con as o o he s al eady men ioned, in ends o de elop an algo i hm
o p edic he delay in an indi idual ligh aking in o accoun (1) ligh in o ma ion, simila o Y. J.
Kim, Choi, B iceno, & Ma is (2016) and Zonglei, Jiandong, & Guansheng (2008); (2) clima e a o igin
and des ina ion, simila o Belcas o, Ma ozzo, Talia, & T un io (2016) and Choi, Kim, B iceno, &
Ma is (2016), bu also; (3) in o ma ion o he ai c a as well as (4) possible conges ion o he
sys em. Ha ing he pu pose o unde s anding i i is possible, o imp o e he pe o mance o he
models al eady p esen ed in his ype o app oach (machine lea ning) and wi h his objec i e
(indi idual ligh p edic ion).
18
adap ed om hou s and minu es o o al minu es as i is applied by Kim (2016) o ha e a s anda d
measu e o all a iables o ime. Wi h hose goals, nine a iables we e es ablished such as mon h,
day o he mon h, weekday, scheduled ligh du a ion, dis ance, scheduled depa u e ime, eal
depa u e ime, depa u e delay and scheduled a i al ime.
B. Va iables o ai po in o ma ion
These a iables we e used o gi e geog aphical in o ma ion o each ligh . Fo ha pu pose, only one
a iable was needed, he o igin ai po .
C. Va iables o ligh and plane in o ma ion
The a iables men ioned he e we e used as a way o iden i ying he ai plane, he possibili y o
mechanical p oblems and in o ma ion o possible olume o passenge s as a jus i ica ion o p obable
delays. In ha o de , six a iables we e p o ided such as he ai line company, he ligh numbe , he
an iqui y, he manu ac u e , he model, and he maximum sea s o an ai plane.
D. Va iables o ligh delay p opaga ion in o ma ion
These a iables we e used as a means o pe cei ing whe he he delay has al eady happen o could
ha e happen a a ime di e en om he depa u e ime a he ai po o o igin which gene a es
la ge olumes o passenge s consequen ly. Wi h his in en , wo a iables we e c ea ed: he p e ious
day delay occu ence in he o igin ai po , ha acco dingly o Rebollo & Balak ishnan (2014) a ec s
he abili y o eco e om i when he e is a la ge olume o delays and migh esul in highe ai
a ic alues eusl ing in depa u e delays ha ansla e in o a i al delays by consequence; And, he
holiday occu ence ha can cause a mo e signi ican in lux o passenge s causing a possible delays.
E. Va iables o wea he in o ma ion
The a iables c ea ed o his ca ego y we e used as a means o in o ming he wea he condi ions in
h ee si ua ions ( o a mo e de ailed explana ion consul Annex 6 o he Annexes chap e ):
▪ A he o igin, a he scheduled ime o depa u e;
▪ A he des ina ion, on he scheduled ime o depa u e in o igin, wi h he ime zone o
he des ina ion;
▪ A he des ina ion, a he scheduled ime o a i al.
Va iables wi h En-Rou e clima e in o ma ion we e no conside ed as i is challenging o ob ain
clima e in o ma ion o all posi ions o he ai plane due o he combina ion o he measu emen s o
he epo s o di e en s a ions. They we e also no conside ed due o he need o ake in o accoun
he al i ude a which he plane is a (Takacs, 2014). In ag eemen wi h Muelle & Cha e ji (2002), he
wea he is he la ges con ibu o o delays in he ai a ic con ol sys em.
These h ee phases a e di e en ia ed because he ligh can be delayed and/o canceled by he
wea he . As such, be o e a ligh depa s, an assessmen is made as o whe he he condi ions o
aking o a e me o no .
Th ee possibili ies can make he ligh no o ake o o , a e aking o , no land a he des ina ion
a he scheduled ime, causing a delay in depa u e and a i al. Such possibili ies a e (1) he wea he
is no a o able a he scheduled ime o depa u e a he o igin. (2) The wea he condi ions a e no
a o able a he des ina ion a he same ime ha he ai plane is scheduled o depa a he o igin
19
(and he e o e is in luenced by he ime zone). And, a las , (3) The wea he is no a o able a he
des ina ion a he ime he plane is scheduled o land (K ozel, Capozzi, And e, & Smi h, 2003).
Addi ionally, and acco ding o he websi e o he Bu eau o T anspo a ion S a is ics (2016c), he
o al wea he has a la ge sha e in he pe cen age o delayed ligh s (32.8% in 2015). The o al sha e
o he pe cen age o delays due o he wea he o he BTS is he pe cen age o delayed o canceled
ligh s due o wo ac o s. Fi s , he ex eme wea he ca ego y, which e e s o cases ha en i ely
p e en a plane om aking o (5% in 2015). And, secondly, he ca ego y o delays and cancella ions
assigned o he NAS, which include a ime subca ego y, in which case only hose cases whe e he
sys em can be delayed, bu does no p e en ake-o , a e aking in o accoun (mo e han hal o he
22.9% in he yea o 2015).
Takacs (2014), in his s udy, men ioned ha p ecipi a ion, isibili y, wind speed, and empe a u e
we e he mos c ucial wea he ea u es. Fo such easons, he e we e c ea ed i een a iables o
po ay he wea he cha ac e is ics a he h ee phases o each wea he condi ion: empe a u e,
p ecipi a ion, wind, isibili y, and e en .
Table 5 summa izes all he opics, abo e discussed, whe e all a iables de ined a e illus a ed and
hei impo ance co obo a ed by o he esea che s.
I should be no ed ha no all in o ma ion gi en abou a iables is comple e and some a icles do no
explici ly s a e all he a iables used. Thus, he pe cen age o a iables used in his a icle, in ela ion
o each o he a icles below, is no always p ecise. Fo ha eason, in some si ua ions he
pe cen age is p eceded by an in e io signal (“<”) ep esen ing ha , in ha a icle, he numbe
men ioned is he pe cen age o he a iables explici ly men ioned al hough, i is e e ed ha o he
a iables exis . In ha way, as he numbe o a iables inc eases, he pe cen age o a iables used in
his s udy dec eases.
Va iables
A icles
(Khanmohammadi e al., 2016)
(Muelle & Cha e ji, 2002)
(Py gio is e al., 2013)
(Rebollo & Balak ishnan, 2014)
(Xu e al., 2005)
(Yao e al., 2010)
(Klein e al., 2010)
(AhmadBeygi e al., 2008)
(Kim e al., 2016)
(Belcas o e al., 2016)
(Abdel-A y e al., 2007)
(Ionescu e al., 2016)
(M. S. Kim, 2016)
(Qianya e al., 2015)
(Zonglei e al., 2009)
(Choi e al., 2016)
Mon h
x
x
x
x
x
x
x
Day
x
x
x
x
x
x
x
Weekday
x
x
x
x
x
x
x
Schedule Fligh Du a ion
x
x
x
Dis ance
x
Schedule Depa u e Time
x
x
x
x
x
x
x
x
x
x
x
x
Real Depa u e Time
x
x
x
x
x
x
x
x
Depa u e Delay
x
x
x
x
x
Schedule A i al Time
x
x
x
x
x
x
x
x
x
x
x
x
O igin Ai po
x
x
x
x
x
x
x
x
x
x
x
x
20
Fo DOT and, consequen ly, o BTS, as i o his s udy, a ligh is conside ed o be delayed i he
ac ual ime o a i al is g ea e han 15 minu es in ela ion o he scheduled ime o a i al. Howe e ,
he e is no uni e sal de ini ion o how i is measu ed.
In ha sense, in he model o his s udy, as a dependen a iable, he disc e e bina y a iable o
a i al delay is conside ed as ha ing been used in o he s udies as seen in Table 6. I ep esen s he
exis ence, o no , o he a i al delay. I he ac ual ime o a i al is g ea e han 15 minu es om he
scheduled ime hen he a iable assumes he alue 1, o he wise i assumes he alue o 0, acco ding
o he DOT de ini ion.
Ai line Company
x
x
x
Fligh Numbe
x
x
x
x
x
An iqui y
Manu ac u e
Model
x
Maximum sea s
P e ious Day Delay Occu ence
x
Holiday Occu ence
Tempe a u e a O igin
x
x
Tempe a u e on Des ina ion a
Schedule Depa u e Time
Tempe a u e a Des ina ion
x
x
P ecipi a ion a o igin
x
x
x
P ecipi a ion on des ina ion a Schedule
depa u e ime
P ecipi a ion a Des ina ion
x
x
Wind a O igin
x
x
x
x
Wind on Des ina ion a Schedule
Depa u e Time
Wind a Des ina ion
x
x
x
Visibili y a O igin
x
x
x
x
Visibili y on Des ina ion a Schedule
Depa u e Time
Visibili y a Des ina ion
x
E en a O igin
x
x
x
E en on Des ina ion a Schedule
Depa u e Time
E en a Des ina ion
x
Used Va iable Pe cen age (%)
46
56
67
<26
<17
17
<56
50
53
75
63
<86
67
56
<100
<78
Table 5: Syn hesis o chosen a iables and hei use in o he a icles
Sou ce: Made by he au ho
21
Since his wo k s udies he ai delays, canceled o di e ed ligh s a e no conside ed since hey do
no ep esen in o ma ion abou he delay (Abdel-A y e al., 2007; Belcas o e al., 2016).
Thus, he inal inpu da ase is hen composed o hi y- ou (34) dependen a iables o explain he
independen a iable. I has a o al o 248 956 eco ds o eigh mon hs in he yea o 2015 (Janua y,
Feb ua y, Ap il, May, July, Augus , Oc obe , No embe ) as illus a ed in Table 7.
Va iable
Type
Obse a ion Values
Role
Desc ip ion
Min-Max
N.
le els
Mean
Mode
Missing
Values
MONTH
Nominal
-
8
-
8
0
independen
Mon h ela i e o
he ligh
DAY
Nominal
-
31
-
13
0
independen
Day ela i e o he
ligh
WEEKDAY
Nominal
-
7
-
5
0
independen
Weekday ela i e o
he ligh (1-
Monday; 7-Sunday)
SCHED_FLIGHT
_DUR
Nume ic
24 - 489
-
91,257
-
0
independen
Schedule ligh
du a ion abou he
o igin-des ina ion
ou e (in minu es)
DIST
Nume ic
83 - 4502
-
639,589
-
0
independen
Dis ance be ween
o igin and
des ina ion ai po s
(in miles)
SCHED_DEP
_TIME
Nume ic
15 - 1439
-
738,353
-
0
independen
Schedule depa u e
ime (in minu es
and in local hou )
REAL_DEP
_TIME 5
Nume ic
0 - 1439
-
743,027
-
0
independen
Real depa u e ime
(in minu es and in
local hou )
Type o
Dependen
Va iable
A icles
(Khanmohammadi e al., 2016)
(Py gio is e al., 2013)
(Rebollo & Balak ishnan, 2014)
(Smi h & She y, 2008)
(Xu e al., 2005)
(Yao e al., 2010)
(Klein e al., 2010)
(AhmadBeygi e al., 2008)
(Kim e al., 2016)
(Balak ishna e al., 2008)
(Belcas o e al., 2016)
(Abdel-A y e al., 2007)
(Ionescu e al., 2016)
(Qianya e al., 2015)
(Zonglei e al., 2008)
(Zonglei e al., 2009)
(Choi e al., 2016)
Con inuous
x
x
x
x
x
x
x
x
Disc e e
B
C
C
C
C
B
C
C
C
B
Table 6: Syn hesis o he ype o dependen a iables used in o he a icles
whe e B s ands o Bina y Disc e e ype o dependen a iable and, C o Ca ego ical Disc e e ype o
dependen a iable.
Sou ce: Made by he au ho
22
DEP_DELAY 5
Nume ic
0 - 1289
-
10
-
0
independen
Di e ence in
minu es be ween
Schedule and eal
depa u e ime.
Depa u es ea lie
o he schedule a e
ep esen ed as 0.
SCHED_ARR
_TIME
Nume ic
1 - 1439
-
869,689
-
0
independen
Schedule a i al
ime (in minu es
and in local hou )
ARR_DELAY
Nominal
– Bina y
(0,1)
-
2
-
0
0
dependen
Indica o o a i al
delay. I he delay is
supe io o 15
minu es is
ep esen ed as 1
o he wise is
ep esen ed as 0.
ORIGIN
Nominal
-
168
-
MCO
0
independen
3 le e s code o he
o igin ai po ,
acco dingly o IATA.
AIRLINE_COMP
Nominal
-
11
-
DL
0
independen
Ai line Company
unique code
FLIGHT_NUM
Nominal
-
3265
-
N844AS
0
independen
Fligh N-numbe
a ibu ed by
Na ional A ia ion
Au ho i y (NAA)
ANT
Nume ic
0 - 56
-
16,244
-
8873
(4%)
independen
Ai plane an iqui y
MAN
Nominal
-
26
-
BOEING
1405
(1%)
independen
Ai plane
Manu ac u e name
MOD
Nominal
-
106
-
MD-88
1405
(1%)
independen
Ai plane model
name
MAX_SEATS
Nume ic
2 - 451
-
139,42
-
1464
(1%)
independen
Ai plane maximum
sea numbe
PREV_DAY
_DELAY_OCURR
Nume ic
0 - 810
-
34,9
-
1021
(0%)
independen
Numbe o ligh s
canceled and
delayed (depa u e)
on he p e ious day
a he ai po o
o igin
HOLIDAY
_OCURR
Nominal
-Bina y
(0,1)
-
2
-
0
0
independen
Indica o o holiday
occu ence.
Rep esen ed as 1 i
he same day o in a
bu e o 3 days
occu s some ede al
holiday, o he wise
0
TEMP_ORIGIN
Nume ic
-49,4 –
45,6
-
16,217
-
940
(0%)
independen
Measu ed in Celsius
deg ees
TEMP_DEST
_SCHED_DEP
Nume ic
-11,7 - 35
-
17,165
-
237
(0%)
independen
5
Va iable Dep_Delay and Real_Dep_Time used as a basis o he cons uc ion o wo ypes o da ase
one con empla ing i and ano he wi hou i because o i s known impo ance on de e mining a delay in a i al
as p o ed in (M. S. Kim, 2016) and o no allowing ad ance in he p edic ion.
23
_TIME
TEMP_DEST
Nume ic
-11,7 - 35
-
17,906
-
204
(0%)
independen
PRECIP_ORIGIN
Nume ic
0 – 71,37
-
0,053
-
852
(0%)
independen
Measu ed in
millime e s pe
hou
PRECIP_DEST
_SCHED_DEP
_TIME
Nume ic
0 – 24,89
-
0,08
-
237
(0%)
independen
PRECIP_DEST
Nume ic
0 – 24,89
-
0,08
-
200
(0%)
independen
WIND_ORIGIN
Nume ic
0 – 166,9
-
7,733
-
1335
(1%)
independen
Measu ed in miles
pe hou
WIND_DEST
_SCHED_DEP
_TIME
Nume ic
0 – 32,2
-
7,85
-
433
(0%)
independen
WIND_DEST
Nume ic
0 – 32,2
-
8,13
-
386
(0%)
independen
VISIB_ORIGIN
Nume ic
0 - 99
-
9,239
-
1005
(0%)
independen
Measu ed in miles
VISIB_DEST
_SCHED_DEP
_TIME
Nume ic
0,12 - 10
-
8,921
-
237
(0%)
independen
VISIB_DEST
Nume ic
0,12 - 10
-
8,957
-
200
(0%)
independen
EVENT_ORIGIN
Nominal
-
162
-
BR
217477
(87%)
independen
Si ua ion desc ip o
(METAR codes)
EVENT_DEST
_SCHED_DEP
_TIME
Nominal
-
33
-
BR
210125
(84%)
independen
EVENT_DEST
Nominal
-
33
-
BR
210150
(84%)
independen
Table 7: O e iew o used a iables
Sou ce: Made by he au ho
3.1.3. Sampling Techniques
Mos da ase s a e no made up o a simila numbe o
obse a ions om each class. When, in a classi ica ion p oblem,
he classes a e no ep esen ed equally we a e acing an
unbalanced da ase . This cha ac e is ic may e lec a e y high
accu acy o he model. Howe e , ha ing many obse a ions o a
class will cause he model o lea n om he da a, and always
decide by he majo i y class esul ing in a alse classi ica ion
accu acy (Figu e 7) (Chawla, Bowye , Hall, & Kegelmeye , 2002;
Hoens & Chawla, 2013; Lachhe a & Bawa, 2016; Weiss, 2004).
One way o sol e his is h ough sampling echniques whe e he
goal is o c ea e a da ase ha has a simila dis ibu ion o
classes so ha he classi ie s can co ec ly cap u e he di ision
be ween he majo i y and mino i y classes (Hoens & Chawla,
2013). The e a e wo ypes o me hods o his, (1) copying
Figu e 7: P oblem o unbalanced classes
Sou ce: (Lachhe a & Bawa, 2016)
24
mino i y class obse a ions - o e sampling -, o (2) elimina ing obse a ions om he majo i y class –
unde sampling.
In his s udy, he da ase in ques ion is unbalanced. O he o al obse a ions (248956),
app oxima ely 86% a e ligh s ha did no a i e la e (=0) and app oxima ely 14% we e epo ed as
la e (=1) as shown in Figu e 8. I can be said ha he e is, app oxima ely, a a io o 6:1 whe e o
each obse a ion wi h delay he e a e six wi h no delay, making he exis ence o delay a e, i.e., a
a e class o , mo e gene ally, an imbalanced class (Weiss, 2004). This is wha happens mos o he
ime in da ase s wi h eal da a whe e he "no mal" class is p edominan and, only a small pe cen age
e e s o obse a ions o he class o in e es (Chawla, 2009; Chawla e al., 2002).
Figu e 8: Dis ibu ion o he da ase acco ding o he dependen a iable (ARR_DELAY)
whe e 1 e e s o an obse a ion o a ligh a i es la e and 0 o he wise.
Sou ce: Weka So wa e
An o e sampling echnique, SMOTE (Syn he ic Mino i y O e -sampling Technique) was in oduced
by Chawla, Bowye , Hall, & Kegelmeye (2002) as a solu ion o his si ua ion. I is one o he mos
used app oaches due o i s simplici y and e ec i eness as well as o being able o imp o e he
accu acy o he classi ie s, and o ha eason i was applied o his da ase h ough he SMOTE
supe ised ins ance il e p o ided in Weka.
I is used o c ea e syn he ic samples om he mino i y class ins ead o copies, simila ly o a
adi ional andom o e sampling echnique which can lead o o e i ing p oblems whe e he model
can adjus oo much o aining da a by memo izing i and ailing o co ec ly p edic unknown da a
(Chawla, 2009; Hoens & Chawla, 2013). The algo i hm, acco ding o Hoens & Chawla (2013), “ i s
selec s a mino i y class a ins ance a andom and inds i s k nea es mino i y class neighbo s. The
syn he ic ins ance is hen c ea ed by choosing one o he k nea es neighbo s b a andom and
connec ing a and b o o m a line segmen in he ea u e space. The syn he ic ins ances a e
gene a ed as a con ex combina ion o he wo chosen ins ances a and b”.
Wi h he in en o see which is he bes app oach and check i he e is a be e app oach besides he
applica ion o he o e sampling echnique – SMOTE –, i was also conside ed o implemen a
echnique o unde sampling, which could p e en common classes hiding a e classes (Weiss, 2004),
h ough he Sp eadSubsample supe ised ins ance il e . This il e p oduces a andom subsample o
he da ase by speci ying he maximum “sp ead” be ween he a es and mos common class, i.e., a
a io, assuming o be 1:1, e lec s a da ase whe e o each obse a ion wi h delay he e is one wi h
no delay, being a uni o m dis ibu ion (Uni e si y o Waika o, 2018). Obse a ions om he majo i y
class a e igno ed, and consequen ly, he aining se becomes mo e balanced and he aining
p ocess as e . In con as , as a disad an age, wi h his educ ion o obse a ions, use ul in o ma ion
wi hin his obse a ions is neglec ed (Liu, Wu, & Zhou, 2009). Fo his eason, and in o de o y o
smoo h he unbalanced p oblem in he da ase , a a io o 2:1 is applied ying no o elimina e he
p esence o he majo i y class, ha is, no o ha e a delay – mos common case –, bu also
elimina ing he massi e di e ence o ins ances be ween he classes in he o iginal da ase .
25
This me hod gene a es wo di e en app oaches (one o he applica ion o SMOTE, and ano he o
he applica ion o unde sampling) o apply o he decisions aken in he cou se o his wo k.
3.1.4. Da a Pa i ion
Being a p edic ion p oblem, i is necessa y o unde s and how a p edic i e model can succeed in
unknown da a, i.e., o pe cei e i s gene aliza ion capaci y. I s cen al pu pose is o di ide he da a
in o unique subse s and hen use a se (s) o es ima e he model pa ame e s - aining da a - and he
emaining alida ion o es da a - o alida e he accu acy o he model. Always ying o manage he
ade-o be ween a p edic ion e o , and possible o e i ing o he model wi h espec o he da a,
and he complexi y o he model.
The e a e se e al app oaches and me hods o pa i ioning he da a. The mos s aigh o wa d,
called Holdou me hod (Figu e 10), is one ha andomly di ides he a ailable da a in o wo se s o
aining and es ing o , h ee se s o aining, alida ion, and es ing, being adequa e o la ge
da ase s.
In he i s case, aining da a is used o ain and model he algo i hm, and he es da a is used o
access he pe o mance o he aining model. In he second case, he model is applied o he
aining se in o de o lea n om i , alida ed h ough he alida ion se whe e he p edic ed model
e o is es ima ed and whe e adjus men s a e made o model pa ame e s and es ed h ough he
es se in o de o e alua e he pe o mance o he inal classi ie (Has ie, Tibshi ani, & F iedman,
2009; Pennsyl ania S a e Uni e si y, 2017).
Acco ding o Koha i (1995), he holdou me hod is a pessimis ic es ima o in he sense ha only a
po ion o he da a is p esen ed o he algo i hm o aining which could ep esen a disad an age
when dealing wi h small da ase s. I also p esen s a dilemma in choosing he size o he es da a
because i many obse a ions a e p esen ed, he es ima o is mo e skewed and, i ew obse a ions
a e p esen ed, he con idence in e al o he p ecision o he model will be highe . Howe e , he
au ho men ions ha in his ype o me hod i is usual o designa e 2/3 o he da a o aining and
only 1/3 o es ing.
Figu e 9: Sampling Techniques
Sou ce: (Lachhe a & Bawa, 2016)
26
Figu e 10: Holdou Me hod T aining-Tes and T aining-Valida ion-Tes
Sou ce: Made by he au ho , adap ed om (Wu & Da aLab, 2016)
In Weka, only ou op ions a e supplied as shown in Figu e
11E o! A o igem da e e ência não oi encon ada.. In his
s udy, he e is only one da ase o examples labeled, namely
he Fligh Da a da ase . Th ough Weka and i s ou op ions, i
is possible o (1) build a model on he Fligh Da a da ase and
apply i o he same da ase meaning ha he es ing would
be done on he aining se . I is also possible o (2) build a
model upon he Fligh Da a da ase and apply i o a second
da ase i we had ano he sepa a e ile wi h examples no p esen ed in aining phase. I is also
possible no ha ing o decide which is he bes o ain and which is he bes o es and o ha e he
oppo uni y o do he wo hings, i.e., (3) di ide he Fligh Da a da ase in o, o example, 5 equal
olds, A, B, C, D, E. And hen ain in A, B, C, D and es on E, subsequen ly i will ain on A, B, C, E and
es on D and so on, es ing each old one ime a e aging he accu acy esul s o he i e imes ha
was done. And a las , (4) jus di ide he Fligh Da a da ase by pe cen age a ibu ing X o alida ion
and he emaining o X o es ing.
Applying he i s op ion is no he bes because he model will always be seeing he same da a and
will be o e i ed, ha ing a alse p ecision. The second is no easible because in his s udy he e is
only one da ase . The hi d op ion is ecommended o small da ase s, and in his s udy, we ha e a
e y la ge one. Fo ha eason, he las op ions will be applied because o as e pe o mance and
o being he mos sui able o he ype o da ase in hand.
In his speci ic case, he o iginal da ase will be di ided in o wo da ase s: T aining and Tes . Fo ha
eason, a aining se is p esen o ain and model he algo i hm, and a e he en i e p ocess, he
es se is p o ided, no being used o aining he model o es ima e he accu acy o wha was
ained in unseen da a.
The e o e, he hold-ou me hod (equi alen o he op ion (4), pe cen age spli ) was chosen, and i
ollows he logic o Koha i (1995) wi h a di ision o 70/30 o he es - aining Holdou me hod.
Howe e , he e is no ule on how much da a is equi ed o aining and es ing, always depending
on he da a noise as well as he complexi y o he da a o i he model (Has ie e al., 2009).
3.2. DATA PRE-PROCESSING
Da a p e-p ocessing is one o he mos impo an and a he same ime di icul s eps in he p ocess.
Pyle (1999), in his book Da a P epa a ion o Da a Mining, es ima es ha he ask o da a
Figu e 11: Da a Pa i ion op ions in Weka
Sou ce: WEKA So wa e
27
p epa a ion is equi alen o 60% o he ime spen on a p ojec . The need o a la ge amoun o ime
and i s impo ance in da a quali y due o noise, inconsis ency o absence in e y cases is wha gi es i
g ea impo ance (De Ville, 2001; Wi en e al., 2011a). Pyle (1999) u he a gues ha he need o
p epa e and p ocess he da a helps p epa e he esea che in he con ex o his/he knowledge o
he da a p ocessed and, consequen ly, he design will be be e and as e .
The main objec i e o his s ep is he GIGO concep (Ga bage in, Ga bage Ou ). Consis s in minimizing
he "ga bage" ha en e s he model o minimize he amoun o "ga bage" ha esul s om he
model (La ose, 2005) showing ha wo ked and meaning ul da a a e a p e equisi e in he p oduc ion
o e ec i e models (Pyle, 1999).
Much o his wo k was done when cons uc ing he da abase o c ea ing he inpu da ase .
Howe e , his ini ial ea men was only made due o he need ha a ose in he in eg a ion o he
da a ha was coming om di e en sou ces. A ha s age, i was no sol ed any ype o missing da a
and he e o e, a e co ec ion o alues and o ma s o in eg a ion in he da abase, some ields
we e s ill emp y, and he e was a need o p e-p ocess he inal da ase om he da abase. This is
wha is desc ibed along his sub-chap e because “ he inpu o he da a mining algo i hms is
assumed o be nicely dis ibu ed, con aining no missing and inco ec alues howe e , eali y is much
mo e “di y” han he ideal and he da a usually equi es much p ep ocessing be o e any da a mining
algo i hms can be applied” (S. Zhang, Yang, & Zhang, 2002).
3.2.1. Explo a o y Da a Analysis
Explo a o y Da a Analysis (EDA) is an app oach o analyze he da a and see i s main cha ac e is ics as
well as i s beha io , ei he alone as oge he wi h he dependen a iable, wi h no clea ideas o wha
o look o (Hand e al., 2001).
I has gained a posi ion as he gold s anda d me hodology o analyze da a and was in oduced by
John W. Tukey (1961) whe e he de ined da a analysis as “p ocedu es o analyzing da a, echniques
o in e p e ing he esul s o such p ocedu es, ways o planning he ga he ing o da a o make i s
analysis easie , mo e p ecise o mo e accu a e, and all he machine y and esul s o (ma hema ical)
s a is ics which apply o analyzing da a”. In ag eemen o La ose (2005), i allows he analys o
maximize insigh o a da a se , examine he a ibu es and hei beha io , iden i y in e es ing subse s
o he obse a ions and de elop an ini ial idea o possible associa ions be ween he a ibu es and
he a ge a iable.
Fo being a way o in es iga e a iables h ough he look o his og ams and he explo a ion o he
ela ionships among se s o a iables (La ose, 2005) mo e speci ically, he independen a iables by
he dependen a iable, a b ie iew o hem was ca ied ou , in he beginning, o see how he
o iginal da a beha ed.
I is impo an o no e ha each g aph con ains wo ypes o in o ma ion. One showing he
equency o each class by he dependen a iable on he e ical axis. And he o he , showing he
ele ance, in pe cen age, o each class o he dependen a iable, by each class o he independen
a iable in analysis on he ho izon al axis, bene icial o compa e class impo ance. Some nume ical
a iables we e g ouped in o classes o ensu e i s simplici y and cla i y in his cu en analysis.
34
Figu e 28: In luence o A i al delay on Maximum numbe o sea a iable
Sou ce: Made by he au ho
Rega ding passenge s’ olume and
exis en a ic p oblema ic, i is seen,
wi h he assis ance o Figu e 29, ha
he g ea e he numbe o delays/
cancella ions in he o igin o he ligh ,
on he day be o e, he g ea e he
pe cen age o delayed ligh s.
In he in e al om 800 minu es o 899
minu es, he e a e 60% o ligh s ha
expe ience delay on a i al compa ing
o he in e al ha ep esen s 79% o
he o al delayed ligh s, om 1 minu e
o 99 minu es, whe e only 13.6% o ligh s a e delayed. I is also seen in Figu e 30 ha a ound 14.9%
o ligh s ha p esence a holiday in he same day o he ligh o in a bu e o h ee days su e s a
delay in a i al, agains he 14% ha do no p esence a holiday bu also ha e a delay in a i al.
Figu e 30: In luence o A i al delay on Holiday occu ence a iable
Sou ce: Made by he au ho
In wha wea he a iables a e conce ned, i is impo an o men ion ha he e is an inc ease in he
pe cen age o delayed ligh s in compa ison o he ones whe e he empe a u e in he momen o
obse a ion is o ex eme alues, ei he posi i e and nega i e (see Annex 7 om Annexes chap e ).
When obse ing he p ecipi a ion, i can be no iced ha when he p ecipi a ion is in he ange o 10
o 50 millime e s pe hou , he e is a highe pe cen age o delayed ligh s al hough ew ligh s a e
occu ing du ing his ype o condi ions. None heless, he ones ha occu in hese e ms ha e a
highe pe cen age o delay compa ing o he non-delayed, han he ones occu ing in he o he
in e als (consul Annex 8 om Annexes chap e ).
Figu e 29: In luence o A i al delay on P e ious day delay occu ence
a iable
Sou ce: Made by he au ho
35
By analyzing he wind a iables, i is de ec able ha he e is a highe pe cen age o delays when he
speed o he wind is highe (Annex 9 om Annexes chap e ). The in e al o 32 o 39 miles pe hou
is he one wi h he mos eco ds o delays in h ee phases o he a iable (a he o igin a he
scheduled depa u e ime; a he des ina ion a he scheduled ime o depa u e; and, a he
des ina ion a he scheduled a i al ime). The e is no eco d o highe alues egis e ed being an
excep ion a eco d ha egis e s a wind speed o mo e han 73 miles pe hou ( ha is a speci ic case
on he 14 h o Augus in he ai po o igin o My le Beach, Sou h Ca olina). In his speci ic case a
wind speed o 166.9 mph he equi alen o 268.5 km/h was epo ed which leads o u he esea ch
on he NAS websi e o unde s and wha happened. No egis y o men ion o any majo e en was
ound, and, because o ha , his ype o obse a ion will be conside ed an ou lie .
The isibili y a iables show ha he ligh s ha con ibu es he mos o he delay a e he ones wi h
isibili y be ween 10 o 11 miles, being also he ones ha cons i u e he non-delayed majo i y. As he
isibili y dec eases, he pe cen age o delayed ligh s inc eases, in con as o he ones ha a e no
delayed as expec ed, none heless hey a e always in e io o he non-delayed ones (Annex 10 om
Annexes chap e ).
3.2.2. Missing Values
Missing alues a e known as da a ha a e “missing o some (bu no all) a iables and some (bu no
all) cases” (Allison, 2001) in he da abase. They a e c i ical o he managemen o he da a because
hey can ep esen an issue in da a quali y and comp omise he in e p e a ion o da a. I mus be
emphasized ha eplacing missing alues is a gamble, and he bene i s mus be weighed agains he
possible weakness o he esul s (La ose, 2005).
Di e en easons can cause hem, such as equipmen mal unc ions o ailu es, inabili y o collec an
obse a ion, and so on (Ba is a & Mona d, 2002). The lack o some obse a ions in some a iables
can educe he sample size, and as a esul , he p ecision may be nega i ely a ec ed, he s a is ical
powe weakened, and he pa ame e es ima es could be biased (Soley-bo i, 2013). Fo ha eason,
he e is a need o a good unde s anding o his phenomenon and o app op ia e measu es o deal
wi h i .
Soley-bo i (2013) s a es ha o deal wi h missing da a equi es a me iculous in es iga ion o he da a
o be able o iden i y and unde s and he missing da a o decide on how o ea da a applying he
echnique ha sui s he mos o why da a is missing.
The e a e a ious ways o ea missing da a. A common one is o igno e he obse a ions o
a iables whe e missing da a is p esen . This kind o app oach can be isky because he pa e ns o
missing da a may be sys ema ic.
Ano he way could be dele ing hem which could lead o bias, bu also because dele ing he
obse a ion o a iables wi h missing da a would omi he es o he in o ma ion o he emaining
a iables o he o he obse a ions, espec i ely, only because a ew alues a e missing.
To escape om he d as ic solu ion p e iously p esen ed and when, i is possible o sol e i by o he
means, measu es less d as ic a e aken. This ype o me hods s a e ha he missing alues could be
subs i u ed acco ding o some c i e ia such as (1) eplacing he missing alue by a cons an , speci ied
by he analys ; (2) eplacing he missing alue wi h he mean (in case o nume ical a iables) o he
36
mode (in he case o ca ego ical a iables) al hough i is a gued ha he mean may no always be he
bes choice o wha cons i u es he bes alue; his is because subs i u ing missing alues by he
mean will u n he s a is ical in e ence o e op imis ic since measu es o sp ead will be educed
a i icially; (3) eplacing he missing alues wi h a alue gene a ed andomly om he a iable
dis ibu ion, being a be e me hod han he p e ious one (La ose, 2005); and, a las (4) eplacing
he missing alues wi h impu ed alues based on o he cha ac e is ics o he obse a ion (Han &
Kambe , 2011).
In his pa icula s udy, only some a iables ha e missing alues, and how hey will be ea ed a y
acco ding o he eason why hey ha e missing alues. In he able below (Table 8) he a iables
which ha e missing alues and he ep esen a i e pe cen age conce ning he o al o ins ances a e
ep esen ed.
Va iable
# o Missing Values
% o Missing Values
ANT
8873
4
MAN
1405
1
MOD
1405
1
MAX_SEATS
1464
1
PREV_DAY_DELAY_OCURR
1021
0
TEMP_ORIGIN
940
0
TEMP_DEST_SCHED_DEP_TIME
237
0
TEMP_DEST
204
0
PRECIP_ORIGIN
852
0
PRECIP_DEST_SCHED_DEP_TIME
237
0
PRECIP_DEST
200
0
WIND_ORIGIN
1335
1
WIND_DEST_SCHED_DEP_TIME
433
0
WIND_DEST
386
0
VISIB_ORIGIN
1005
0
VISIB_DEST_SCHED_DEP_TIME
237
0
VISIB_DEST
200
0
EVENT_ORIGIN
217477
87
EVENT_DEST_SCHED_DEP_TIME
210125
84
EVENT_DEST
210150
84
Table 8: Va iables wi h missing alues in he Fligh Da a da ase
Sou ce: Made by he au ho
The e is a eason o e e y missing alue and mainly is he una ailabili y o he da a in he sou ces.
The cases o missing alues can be jus i ied by he missing alues in o he a iables o o being
missing jus o some eason no explained by ano he a iable wi h obse a ions missing. Fo
example, a iables o wea he a e missing because equipmen do no epo in o ma ion o ce ain
hou s in ce ain ai po s. In o he cases, he wea he a iables a e no missing, bu he a iable o
p esen wea he condi ion (e en ) is missing, and his is because many imes occu ences o his ype
o in o ma ion a e no epo ed on METAR in he sense ha no hing wo hwhile epo ing happened.
Tha being said, in his esea ch, he ea men ca ied ou in he a iables men ioned abo e was as
ollows:
37
▪ Fo he obse a ions wi h no alues abou he ai plane in o ma ion such as an iqui y,
manu ac u e , model, and maximum numbe o sea a iables he app oach ha was
aken was o emo e hem om he s udy h ough he Remo eWi hValues il e . I
does no make sense o eplace hem wi h alues ha do no ma ch he eal
cha ac e is ics o he ai plane and, as i ep esen ed a la ge scope han he wea he
in o ma ion (pe cen age o missing da a), i will no ha e he same ea men . This is
because i could impac he da ase and de ia e i om he ue alues (a consequence
o eplacing obse a ions wi h a mean o mode, depending on he a iable ype);
▪ Fo he obse a ions whe e he a iable o he p esence o delay in he p e ious day
was missing, only he case o Janua y 1s (because o he lack o in o ma ion o he
p e ious day in he da ase downloaded since he p e ious day was no pa o he
yea o 2015), he Remo eWi hValues il e was applied;
▪ Fo all obse a ions whe e no wea he a iable was a ailable, because o a lack o
eco ds close o he sou ce, impu a ion was applied h ough he
ReplaceMissingValues il e . This decision was made so as no o lose obse a ions
wi h in o ma ion ha could be impo an o o he a iables and, since impu a ion did
no in ol e many cases, he impac was conside ed o be low;
▪ In he case o he p esen wea he in o ma ion (e en a iable) he e was a
conside able amoun o missing alues. Tha p obably happened because o
equipmen ailu e in cap u ing he in o ma ion, and o ha eason, he h ee
a iables conce ning his opic we e elimina ed h ough he Remo e il e .
3.2.3. Ou lie s
Hawkins (1980) de ines an ou lie as “an
obse a ion which de ia es so much om o he
obse a ions as o a ouse suspicious ha i was
gene a ed by a di e en mechanism”. Fo G ubbs
(1974) “an ou lying obse a ion, o “ou lie ” is one
ha appea s o de ia e ma kedly om o he
membe s o he sample in which occu s”. Is also
de ined as a single, o e y low equency,
occu ence o he alue o a a iable ha is a
away om he ex en o he alues o he a iable
(Pyle, 1999). In o he wo ds, ou lie s a e ex eme
alues ha lie nea he limi s o he da a ange o go
agains he end o he emaining da a (La ose, 2005). They can be ep esen ed as indi idual
occu ences and, some imes, in clus e s o consecu i e alues o he same o de o magni ude bu
also as a g oup si ua ed beyond he ange o he o he alues (Pyle, 1999) as ep esen ed in Figu e
31.
Figu e 31: Examples o Ou lie s: as an indi idual
alue (a) and as clus e s o alues (b)
Sou ce: Wi hd awn om (Pyle, 1999)
38
When p esen ed in he inpu da a hey could skew and mislead he aining p ocess o machine
lea ning algo i hms esul ing in la ge aining imes, less accu a e models and poo and uns able
esul s because ce ain me hods a e sensi i e o he p esence o ou lie s (La ose, 2005).
Fo hose easons, i was necessa y o iden i y which obse a ions we e ou lie s. Ou lie s could be
caused by de ices mal unc ion, audulen beha io , human e o , na u al de ia ions, among o he s.
Ne e heless, one should be ca e ul when d opping ou lie s. I is essen ial o in es iga e he na u e
o he ou lie be o e deciding on wha o do because each case is di e en and he e is no igh way
o do so.
Wi h he help o Weka and i s il e o In e qua ileRange, i was possible o iden i y wha
obse a ions we e ou lie s and ex eme alues based on in e qua ile anges. This il e adds wo
new a ibu es whe he he alues o ins ances can be conside ed ou lie s o ex eme alues. As can
be seen in Figu e 32 he il e conside s an obse a ion o be an ex eme alue i hey exceed he
uppe qua ile (Q3) o all below he lowe qua ile (Q1) by he p oduc be ween he ex eme alue
ac o (use -speci ied) and he in e qua ile ange (IQR). And, o be an ou lie , he ones ha a e no
ex eme alues bu exceed he uppe qua ile (Q3) o all below he lowe qua ile (Q1) by he
p oduc o he ou lie ac o (use -speci ied) and he in e qua ile ange (IQR) (Wi en e al., 2011b).
Co espondingly o he de aul se ings o Weka, and he ones aking in conside a ion when applying
he il e , he alue o he ou lie ac o is h ee (OF=3), and o ex eme alues, he ac o is wo
imes he ou lie ac o (EVF=OF*2).
Figu e 32: De ini ion o Ex eme Value and Ou lie using In e qua ileRange il e
Sou ce: Made by he au ho , adap ed om Weka In e qua ileRange Objec Edi o In o ma ion
Whe e Q1 = 25% qua ile; Q2 = 50% qua ile (median); Q3 = 75% qua ile; IQR = In e qua ile Range; OF = Ou lie Fac o ; EVF
= Ex eme Value Fac o
The applica ion o he il e esul ed in wo new a ibu es: one wi h ex eme alue in o ma ion and
ano he wi h ou lie in o ma ion, bo h o bina y ype. The nex s ep consis ed o elimina ing he
eco ds whe e he obse a ions we e conside ed o be ex eme alues and hen epea i o he
obse a ions conside ed o be ou lie s, h ough he il e Remo eWi hValues.
Fo a compa ison o pe o mance be ween he decision o elimina ing he ou lie s and ex eme
alues, and he decision no o elimina e hem, i was conside ed, o each c ea ed da ase , ega ding
he use and non-use o he a iable Dep_Delay and Real_Dep_Time, and he applica ion o SMOTE
39
and Unde sampling, he c ea ion o wo o he da ase s: one con empla ing he emo al o ou lie s
and ano he no conside ing hem.
3.3. DATA TRANSFORMATION
Besides all he s eps men ioned abo e, o he s eps could be aken o imp o e he pe o mance and
success o he model. Wi en, F ank, and Hall (2011b) s a e ha hese o he s eps could be
conside ed “a kind o da a enginee ing—enginee ing he inpu da a in o a o m sui able o he
lea ning scheme chosen and enginee ing he ou pu o make i mo e e ec i e”. I is also men ioned
in hei book ha , in he s a e o he a , he e is no gua an ee ha his kind o s eps will wo k bu i
is ein o ced ha in his a ea, a ial and e o app oach is p obably he bes .
Fo ou pu poses, and ha ing in mind he aiming o he s udy, he e a e wo ways, among o he s, o
adap he inpu and make hem mo e suscep ible o he lea ning me hods ha will be de eloped
inside his sub-chap e : no maliza ion and a ibu e selec ion.
3.3.1. No maliza ion
The e is a need o da a mine s o no malize nume ical a iables wi h he in en o s anda dizing he
scale o he e ec ha each a iable has on he esul s. This ansla es in o an inc ease o he alue
o he model by equaling he weigh s o di e en scales and meanings o a iables. The e a e se e al
echniques o do his s ep, and he mos popula me hods a e z-sco e and min-max, being he las
he one used in his s udy, on all in e al a iables h ough he No malize il e in Weka.
The no maliza ion o da a h ough he min-max me hod is a p ocedu e ha escales a iables in a
ange be ween 0 and 1 whe e he la ges alue o each a iable is 1, and he smalles alue is 0. The
no malized ield alue (X*) is ob ained by sub ac ing he minimum alue om he o iginal ield alue
(X) and di iding i by he di e ence be ween he maximum and minimum alues (La ose, 2005).
Equa ion 1: Min-Max No maliza ion Technique
Sou ce: Made by he au ho , adap ed om (La ose, 2005)
I is conside ed o be a sui able echnique when he da a has a ying scales, which is he case iin his
s udy. I s impo ance is ela ed o he ac ha some algo i hms a e mo e sensi i e o hese
disc epancies in da a anges han o he s, which can esul in a a iable wi h highe alues o ha e an
inadequa e in luence on he esul s (La ose, 2005).
3.3.2. Va iable Selec ion
A models’ complexi y can be a de imen al ac o o a clea and easy unde s anding o i . This
complexi y esul s om he as amoun s o a iables o algo i hms o handle which mos o he
imes a e i ele an o edundan o he p oblem ha ing no impo ance in he dependen a iable
a i al delay (Han & Kambe , 2011). Fo such eason, hese ype o a ibu es we e dele ed in behal
o a be e unde s anding, a as e execu ion, and a highe possibili y o a be e pe o mance o he
model, despi e ha ing a subs an ial compu a ional cos (Saeys, Inza, & La anaga, 2007).
40
Fo Wi en, F ank, & Hall (2011b), i is clea ha algo i hms can al eady seek o he bes a ibu es
and igno e he i ele an and edundan ones; none heless, hei pe o mance can o en be
enhanced by p eselec ion.
A ibu e selec ion, also known as a iable selec ion, ea u e selec ion o a iable subse selec ion “is
he p ocess o iden i ying and emo ing as much o he i ele an and edundan in o ma ion as
possible” (M. A. Hall & Holmes, 2002). I can ollow a se o s eps in i s implemen a ion such as (1)
he gene a ion p ocedu e, whe e subse s o ea u es a e gene a ed o e alua ion based on a gi en
sea ch me hod; (2) he e alua ion unc ion, o e alua e he subse ha is being examined acco ding
o a ce ain a ibu e e alua o ; (3) he s opping c i e ia, o help o decide when o s op he sea ch,
and; (4) he alida ion p ocedu e, whe e a subse is checked o being alid o no (Dash & Liu, 1997).
Figu e 33: A ibu e Selec ion S eps
Sou ce: Made by he au ho , e ie ed om (Dash & Liu, 1997)
The p ocess o a ibu e selec ion is sepa a ed in o wo pa s, he a ibu e e alua o ha pe o ms
he e alua ion unc ion, and he sea ch me hod ha does he gene a ion p ocedu e.
In he i s pa , a ibu e e alua o , he e a e wo ypes o e alua o s as explained in Da a Mining:
P ac ical Machine Lea ning Tools and Techniques (2011b): he a ibu e subse e alua o and he
single-a ibu e e alua o (Figu e 34).
The o me akes a subse o a ibu es and e u ns a
nume ical measu e ha pilo s he sea ch. I has, as an
asse , he ac ha elimina es edundan and i ele an
a ibu es, and consequen ly, i s sea ch slow. This ype
o e alua o s could be scheme-independen when hey
do no in ol e a classi ie using only he sea ch me hod
and e alua ing h ough a heu is ic, i.e., selec ing
a iables o a subse ega dless he algo i hm, based
only in gene al measu es as co ela ion, o example,
and supp essing he a iables wi h leas in e es o
modeling. None heless, hey end o selec edundan
a iables because hey do no accoun o ela ionships
be ween a iables; hey a e also known as Fil e me hods (Hamon, 2013)(Figu e 35).
Con a ly o shceme-independen ype exis s he Scheme-dependen a ibu e subse e alua o
ype. I can be o W appe me hod ype (Figu e 36) (Kuma , Konga a, & Ramachand a, 2013) when
he ea u e subse selec ion algo i hm guides a sea ch o a good subse using he induc ion
algo i hm i sel as pa o he unc ion ha is e alua ing ea u e subse s (Koha i & John, 1997). O
Figu e 34: A ibu e Selec ion Types
Sou ce: Made by he au ho
41
Embedded me hod ype (Figu e 37) (Kuma e al., 2013), when ying o combine bo h ad an ages o
he p e ious ypes p esen ed, pe o ming he a iable selec ion by inco po a ing i wi h he
algo i hm di ec ly, selec ing he a ibu es ha bes con ibu e o he accu acy o he model when i
is being ained (Wi en e al., 2011b), and ca ying ou ea u e selec ion and classi ica ion a he
same ime (Hamon, 2013).
The ollowing a ibu e e alua o ype, he single-a ibu e e alua o (Figu e 38), ep esen s an
al e na i e o he slow pe o mance o he o me , and ou pu s an o de ed lis , composed by he
numbe o a ibu es o keep and a anged by he impo ance o he a ibu es, in espec o a
me ic, h ough anking. In i s a o has i s as e pe o mance o e weighed wi h he powe o
elimina e i ele an a ibu es ins ead o i ele an and edundan .
The second pa , he sea ch me hod, goes h ough he a ibu e space o ind a good subse o he
e alua o o measu e i s quali y. I can be pe o med h ough he Bes Fi s and G eedyS epwise
sea ch me hods, among o he s, o he a ibu e subse e alua o s, o a Ranke me hod o he
single-a ibu e e alua o (Wi en e al., 2011b).
To see he pe o mance o each one o he ypes o e alua o s, i was selec ed one om each o
hem wi h he excep ion o w appe and embedded me hods. This is because acqui ing a be e
Figu e 38: Single-A ibu e Selec ion
Sou ce: Made by he au ho
Figu e 35:A ibu e Subse E alua o , Fil e Me hod Selec ion
Sou ce: Made by he au ho , adap ed om (Hamon, 2013)
Figu e 36: A ibu e Subse E alua o , W appe Me hod Selec ion
Sou ce: Made by he au ho , adap ed om (Hamon, 2013)
Figu e 37: A ibu e Subse E alua o , Embedded Me hod Selec ion
Sou ce: Made by he au ho , adap ed om (Hamon, 2013)
42
p edic i e accu acy has high needs o compu a ional e o s and can selec a subse o ea u es ha
is biased o he p ede ined classi ie (Tang, Alelyani, & Liu, 2014).
Fo he single-a ibu e e alua o , among o he s, he GainRa ioA ibu eE al, wi h he Ranke
sea ch me hod, main aining he bes a iables acco ding o he numbe ha be e imp o es he
esul s, was chosen due o i s abili y o e alua e a ibu es by measu ing hei gain a io wi h espec
o he class and hen o de ing hem acco ding o hei e alua ion.
Conce ning he a ibu e subse e alua o ca ego y he C sSubse E al (Co ela ion-based Fea u e
Selec ion) was used, wi h he Bes Fi s sea ch me hod being a g eedy hill-climbing sea ch augmen ed
wi h a back acking ap i ude. I assesses he p edic i e capabili y o each a ibu e indi idually, along
wi h he deg ee o edundancy be ween hem p omo ing se s o a iables highly co ela ed wi h he
class a iable bu wi h he lowes in e co ela ion (Wi en e al., 2011b). I is a good e alua o
because i akes in o conside a ion he a iables co ela ion ha when used could cause ins abili y
and inaccu a e esul s o he model (La ose, 2005).
3.4. DATA MINING
Being Da a Mining, ea lie men ioned in chap e 2 (Li e a u e Re iew), he s ep o KDD whe e
algo i hms a e used o ex ac pa e ns om he da a ansla ing i in o in o ma ion (Fayyad e al.,
1996), in his subchap e , he main objec i e is o de ine and p esen he lea ning algo i hms used.
DM has wo main asks: P edic i e modeling (supe ised
lea ning) and Desc ip i e modeling (unsupe ised
lea ning). The o me aims o lea n decision c i e ia,
making i possible o classi y a new and unknown alue
o in e es gi en known obse a ion o o he a iables.
The second aims o ind hidden pa e ns in he da a,
summa izing da a in ways ha will guide o a highe
unde s anding o he da a (Hand e al., 2001; X. W. X.
Wang, 2009).
As he in e es o his wo k is o p edic i ligh s a e
likely o be delayed o no , i is clea ha he ask inhe en o his s udy is p edic ion. Fu he mo e,
he p edic i e modeling ask can be dis inguished in wo ypes o p oblems. The i s , eg ession
p oblem, i he alue o in e es o p edic is con inuous. And, he second, classi ica ion p oblem, i
he ield o p edic is ca ego ical, being wha happens in his s udy due o he ac ha he a iable o
in e es “A _Delay” is a bina y one, hence conside ed a bina y classi ica ion.
Wi h he in e es o helping he p edic i e modeling ask, he e a e se e al ools also known as
lea ning algo i hms. This lea ning algo i hms, in he con ex o his s udy, we e chosen ega ding he
h ee main a icles in which his wo k elies on (Belcas o e al., 2016; Choi e al., 2016; Y. J. Kim e
al., 2016), and also by p o iding a simple algo i hm ha could be easy o unde s and in con as wi h
he o he s. Fo ha eason, he algo i hms selec ed o applied we e he ones wi h he bes
pe o mance wi hin each a icle, Random Fo es s (Belcas o e al., 2016; Choi e al., 2016) and Mul i-
Laye Neu al Ne wo ks (Y. J. Kim e al., 2016). Howe e , o also unde s and i , wi h mo e simplici y,
Figu e 39: Da a Mining Tasks
Sou ce: Made by he au ho
43
unde s andabili y, and abili y o handle all ypes o a iables, highe esul s could be achie ed, he
implemen a ion o Decision T ees was also ca ied ou .
Each lea ning algo i hm has hei pa ame e s ha , o a be e esul and unde s andabili y, needs o
be wisely pa ame ized. This ask is ime-consuming, and i equi es an excellen knowledge o each
algo i hm and he domain (Kobla , 2012). Fo his eason, and o he numbe o e o s needed o
pe o m his ask success ully, he de aul se ings p esen ed by Weka o each algo i hm we e used
as a e e ence o his wo k.
3.4.1. Decision ees
Decision T ees (DT) a e known as a “collec ion o decision nodes, connec ed by b anches, ex ending
downwa d om he oo node un il e mina ing in lea nodes” (La ose, 2005) ha could also be
ep esen ed as se s o i - hen ules imp o ing he unde s andabili y. I is a me hod widely used o
induc i e in e ence (Mi chell, 1997) wi h a di ide and conque philosophy and ad an ages like he
easy in e p e a ion, he abili y o p ocessing a so o da a ypes, among o he s.
I wo ks by classi ying ins ances by so ing hem down om he oo o a lea node ha p o ides he
classi ica ion o an obse a ion. This is done by i s selec ing an a ibu e - based on a s a is ical es
o de e mine how well i classi ies he aining examples- o place a he oo node making one
b anch o each possible alue. The en i e p ocess is hen epea ed ecu si ely o each b anch, using
only he ins ances ha each he b anch, whe e i is selec ed he bes a ibu e again o es a ha
poin o he ee. I a any ime all ins ances o a node ha e he same classi ica ion, he de elopmen
o ha pa o he ee mus be s opped (La ose, 2005; Mi chell, 1997; Wi en e al., 2011b). In a
gene al de ini ion Mi chell (1997) e e s ha decision ees “ ep esen a disjunc ion o conjunc ions
o cons ain s on he a ibu e alues o ins ances” whe e each pa h om he oo o he lea is a
conjunc ion o a ibu e es s and he en i e ee a disjunc ion o hese conjunc ions. Decision ees
seek o ob ain lea nodes ha a e as pu e as possible – each lea only ep esen s eco ds wi hin he
same class – h ough he disc imina ion be ween classes (La ose, 2005).
Va ious algo i hms ha e been de eloped o lea ning decision ees being a ia ions o he co e
algo i hm ha applies a op-down, g eedy sea ch h ough he space o possible decision ees such
as he ID3 algo i hm – a basic algo i hm o decision ee lea ning -, and i s successo C4.5, an
ex ension om he o me (Mi chell, 1997). C4.5 came o handle sho comings in ID3 algo i hm. This
is done by accep ing bo h con inuous and disc e e a iables; by using “p uning” o sol e o e i ing
and imp o ing he p edic i e accu acy emo ing sec ions o he ee ha ha e no powe o classi y
obse a ions; by using an al e na i e measu e o selec ing a ibu es, he gain a io, ha akes in o
accoun he spli in o ma ion ( e m sensi i e o how b oadly and uni o mly an a ibu e can spli he
da a ha p e en o selec a ibu es wi h many uni o mly dis ibu ed obse a ions) and con a ily o
he in o ma ion gain, does no a o a ibu es wi h many alues, o e hose wi h ew i hey a e no
a use ul p edic o o e unseen ins ances; by handling missing da a; and, by ea ing a ibu e wi h
di e en weigh s when, in ce ain lea ning asks, i is necessa y (Mi chell, 1997; Quinlan, 1993).
In Weka, i is possible o apply his algo i hm wi h he use o he ee classi ie J48, he equi alen o
he C4.5 decision ee algo i hm, adop ing he de aul se ings.
50
B. Da ase 2: Fligh Da ase + Dep_Delay & Real_Dep_Time - Ou lie s
Conce ning SMOTE echnique he models chosen o J48 and MLP we e he ones wi h he highe ROC
Cu e, F-Measu e and Accu acy alues. Al ough , in he case o he RF algo i hm, he model chosen
was no he one wi h he highe ROC Cu e nei he he one wi h he highe accu acy. Ins ead, i was
a esul ha ep esen ed a comp omise be ween he wo models. Thus, he model chosen has a
highe accu acy hen he one wi h a highe ROC Cu e (79.77 o accu acy and 0.83 o ROC Cu e
compa ing o one selec ed wi h a 80.00 accu acy and 0.83 ROC Cu e) and a highe ROC Cu e han
he one wi h he highe accu acy (80.12 o accu acy and 0.80 o ROC Cu e compa ing o he chosen
one).
Abou he Unde sampling echnique only he models selec ed o MLP and RF we e he ones wi h he
highe ROC Cu e, F-Measu e and Accu acy. Rega ding he J48 algo i hm, all di e en a ibu e
selec ion echniques had he same alues in he me ics, and o ha eason, we eso o he ime
me ic. This me ic made choose he app oach ha selec ed wen y a ibu es ha ing a minimal
di e ence be ween selec ing 10 o 20 ea u es (1.16 seconds o build he model and 0.28 seconds o
es ing i compa ing o he alues o he chosen app oach wi h 1.18 o building and 0.31 o es ing).
This choice was also because i s p e e able o ha e mo e a iables han less when ha ing he same
esul s and li le ime di e ence be ween build and es ing he model.
C. Da ase 3: Fligh Da ase - Dep_Delay & Real_Dep_Time + Ou lie s
Abou he SMOTE echnique esul s is impo an o see he decisions made o each o one o he
algo i hms. In he case o he J48 algo i hm, he app oach selec ed was he one wi h he highe
accu acy and F-Measu e bu no he highe ROC Cu e alue (83.06 accu acy, 0.80 o F-Measu e and
0.53 ROC Cu e alue compa ing o he one wi h he highe ROC Cu e wi h 80.49 o accu acy, 0.79
o F-Measu e and 0.54 o ROC Cu e alue). This decision was due o he one wi h he highe ROC
Cu e had h ee mo e a iables han he one chosen, no jus i ying he high dec ease o he accu acy
and negligible inc ease o he ROC Cu e alue.
Fo he RF algo i hm, he model chosen was he one wi h highe accu acy bu no he highe ROC
Cu e alue (76.33 o accu acy, 0.77 o F-Measu e and he highes ROC Cu e alue o 0.55,
compa ing o he one chosen wi h an accu acy alue o 80.89, 0.78 o F-Measu e and 0.53 o ROC
Cu e, wi h an inc ease o , app oxima ely, 5 pe cen age poin s o accu acy and a minimal dec ease o
0.02 be ween ROC Cu es).
In e e ence o he MLP algo i hm , he p e e ed model was he one ha had he highe accu acy.
The eason was because, om he chosen one o he one wi h he highes ROC Cu e, i is no iced
ha he e is a highe gain o accu acy (app oxima ely, mo e 2 pe cen age poin ) han a loss o ROC
Cu e (dec ease o 0.02) and also wi h a sligh ly highe F-Measu e han he one in compa ison.
Fo he Unde sample ecnhique, he models chosen o RF and MLP we e he ones wi h he highes
ROC Cu e and Accu acy alues. In he case o J48, wo a ibu e selec ion app oach had he highes
ROC Cu e and Accu acy alues bu p e e ing he app oach ha had mo e a iables wi h a minimal
inc ease o un ime (13.42 seconds o building he model and 0.54 o es ing, compa ing o he
app oach chosen wi h 14.47 seconds o building he model and 0.25 o es ing).
51
D. Da ase 4: Fligh Da ase - Dep_Delay & Real_Dep_Time - Ou lie s
Rega ding he SMOTE o e sampling echnique, he models dis inc ed o he J48 and RF we e he
ones wi h he highe ROC Cu e alues. In ela ion o he model chosen o he MLP was he one wi h
he highes accu acy. Due o a high dec ease o he accu acy and F-Measu e and o a smalle one in
he alue o he ROC Cu e (85.67 o accu acy, 0.79 o F-Measu e and 0.51 o ROC Cu e compa ing
o 56.54 o accu acy wi h a dec ease o app oxima ely 29 pe cen age poin s, 0.63 o F-Measu e wi h
a dec ease o 0.16 and 0.53 o ROC Cu e ha only inc eases in 0.02) when compa ing o he one
wi h he highes ROC Cu e alue.
Fo he Unde sampling echnique, he model selec ed o he RF algo i hm was he one wi h he
highe ROC Cu e and Accu acy alues. In espec o he J48 algo i hm, he model p e e ed was he
one whe e is p esence a highe Accu acy and F-Measu e ins ead o he one wi h he highes ROC
Cu e (85.67 o accu acy, 0.79 o F-Measu e and 0.50 o ROC Cu e agains he alues o 71.77 o
accu acy, 0.74 o F-Measu e and 0.56 o ROC Cu e alue, espec i ely) and also a much as e
pe o mance.
In he case o he MLP he esul s o he app oaches we e impossible o achie ed due o he he
issue o “ un ou o memo y” e o , caused by he inclusion o a iables wi h high numbe o
ca ego ical classes which is he case o he a iables “Fligh _Num” and “Mod”.
This happened because, wi h a smalle da ase
caused by he applica ion o he
unde sampling echnique, he ca ego ical
a iables we e mo e ecu en used han
wi hin he o he echnique, whe e nume ic
a iables we e used mo e imes as i can be
seen in Figu e 42.
I can be seen, o example, ha in SMOTE he
a iables ha a e selec ed mo e equen ly
h ough he a ious app oaches o a ibu e
selec ion (C sSubse E al, GainRa io(10),
GainRa io(15) and GainRa io(20)) a e he ones
o nume ical ype. When i comes o
unde sampling, despi e ha ing nume ical
a iables equen ly in use, he e is a highe
p esence o ca ego ical a iables chosen mo e
imes. Fo example, i can be seen ha he
a iable “Ai line_Comp”, always selec ed
h ough unde sampling (six een imes ou o
he six een imes, ou a ibu e selec ion
app oach o each ou da ase s), i is only
selec ed ou imes in he SMOTE echnique.
The same can be seen h ough he a iable
“Fligh _Num”, he one ha comp omises he
implemen a ion o MLP in unde sampling
Figu e 42: Va iables Usage F equency by Sampling Technique
Sou ce: Made by he au ho
52
because o being selec ed in wel e imes o he six een, and in SMOTE only being selec ed in ou
imes and impai ing ew cases.
A e he selec ion desc ibed abo e, and om he compa ison be ween he use o SMOTE o
unde sampling, he bes app oaches we e selec ed om he ones al eady selec ed. This was based
on he me ics ha had highe esul s, as i can be seen in he able below and whe e i can be seen
ha mos o he da ase s had highe esul s in each algo i hm by he applica ion o he SMOTE
echnique.
1: Fligh Da ase +
Dep_Delay &
Real_Dep_Time +
Ou lie s
2: Fligh Da ase +
Dep_Delay &
Real_Dep_Time - Ou lie s
3: Fligh Da ase -
Dep_Delay &
Real_Dep_Time +
Ou lie s
4: Fligh Da ase -
Dep_Delay &
Real_Dep_Time - Ou lie s
ROC
F
CCI
ROC
F
CCI
ROC
F
CCI
ROC
F
CCI
J48
0.84
0.82
79.77
0.83
0.82
79.77
0.53
0.80
83.06
0.50
0.79
85.67
SMOTE
SMOTE
SMOTE
Unde sampling
RF
0.85
0.82
79.62
0.83
0.82
80.00
0.53
0.78
80.89
0.52
0.79
84.26
Unde sampling
SMOTE
SMOTE
SMOTE
MLP
0.89
0.82
79.72
0.89
0.86
84.06
0.56
0.79
85.63
0.51
0.79
85.67
SMOTE
SMOTE
SMOTE
SMOTE
Table 10: Resume o he bes esul s o each algo i hm and da ase
Sou ce: Made by he au ho
Consequen ly, i is needed o see which app oach pe o ms be e , ei he by including he ou lie s o
emo ing hem. Fo ha eason, a selec ion was made o he able abo e by selec ing he bes
app oaches o each algo i hm when he inclusion o he a iable o Dep_Delay and Real_Dep_Time
is conside ed and when i is no . As a esul o his s ep, he able below was cons uc ed.
Including Dep_Delay and Real_Dep_Time
Excluding Dep_Delay and Real_Dep_Time
T aining Se
Tes Se
T aining Se
Tes Se
ROC
F
CCI
ROC
F
CCI
ROC
F
CCI
ROC
F
CCI
J48
0.95
0.88
94.62
0.84
0.82
79.77
0.81
0.86
87.48
0.53
0.80
83.06
Wi h Ou lie s
Wi h Ou lie s
SMOTE
SMOTE
GainRa io (10)
GainRa io (10)
RF
1
0.99
99.14
0.85
0.82
79.62
1
1
100
0.52
0.79
84.26
Wi h Ou lie s
Wi hou Ou lie s
Unde sampling
SMOTE
GainRa io (10)
GainRa io (10)
MLP
0.88
0.93
94.01
0.89
0.86
84.06
0.73
0.75
78.06
0.56
0.79
85.63
Wi hou Ou lie s
Wi h Ou lie s
SMOTE
SMOTE
GainRa io (10)
GainRa io (15)
Table 11: Resume o he bes esul s o each algo i hm and co esponding aining esul s by wo iews: including
Dep_Delay and Real_Dep_Time and o he wise
Sou ce: Made by he au ho
53
I is possible o obse e ha mos o he imes all classi ie s pe o med be e on he aining da a
hen when p edic ing wi h unseen da a. This happens wi h he excep ion o he MLP algo i hm when
excluding he wo a iables. This is p obably because hey could o e i modeling in e e y mino
a ia ion o he inpu da a despi e he a emp o elimina e his p oblem.
None heless, only in he classi ie s whe e he a iables o “Dep_Delay” and “Real_Dep_Delay” a e
excluded he alues o ROC Cu e mos di e en ia e. Despi e he ac ha his di e ence also exis s
in he models whe e he wo a iables a e conside ed, i can be seen ha is no so d as ic compa ing
o he p e ious si ua ion.
I is no iceable, wi h he help o Table 11, a dec ease o app oxima ely be ween 0.2 and 0.3 in he
ROC Cu e when no using he a iables.
Howe e , i we wan o p edic i a ligh will be la e a he a i al be o e he a el, i is ob ious ha
his ype o a iables should no be included. This will gi e o he ai lines and ai po s a ma gin o
aking ac ions o imp o e hei se ices a he a i al ai po . Fo ha pu pose, he algo i hm ha is
mos sui able, al hough wi h a lowe ROC Cu e alue, is he MLP using he SMOTE echnique,
including he ou lie s and selec ing he a ibu e h ough he GainRa io app oach e aining en
a iables.
I he in e es o he s udy is o p edic i a ligh is going o su e om delay a he a i al a he
des ina ion a e aking o , hen he a iables can be aking in o accoun . Fo his pu pose, he mos
sui able algo i hm is he MLP using he SMOTE echnique bu excluding he ou lie s and wi h he
selec ion o he a iables h ough he GainRa io selec ing en a iables.
Fu he mo e, i is isible by he obse a ion o he able abo e ha he decision ees ha e a smalle
pe o mance in he ROC Cu e alue han he RF and MLP when conside ing he wo a iables. And a
highe ROC Cu e alue han RF bu lowe han MLP when no conside ing he wo a iables. We
conclude ha by i s simplici y, unde s andabili y and capaci y o handle ca ego ical a iables
con a ily o he o he s (as i can be seen in Annex 11 and 12 o he Annexes chap e ) hey s ill do no
ha e highe pe o mances.
When looking o he a iables ha mos con ibu e o he delay in he bes algo i hms selec ed
(p esen ed in Table 11) we can see wi h he help o he images below ha he mos impo an
a iables a e i s ly he ones ela ed o he wea he being selec ed in all he models.
Secondly, he ones ela ed o he in o ma ion o he cha ac e is ics o he ai plane such as he
an iqui y and he numbe o maximum sea , used in i e o he models. Then, a iables o
p opaga ion o he delay such as he occu ence o delay in he p e ious day a he o igin is used in
h ee models selec ed. A las and used in one o he models, he a iables o ligh in o ma ion like
ai line company and ligh numbe .
54
Figu e 43: Va iables Selec ed by GainRa io o he J48 bes algo i hm including Dep_Delay & Real_Dep_Time a iables
Sou ce: Made by he au ho
Figu e 44: Va iables Selec ed by GainRa io o he J48 bes algo i hm excluding Dep_Delay & Real_Dep_Time a iables
Sou ce: Made by he au ho
Figu e 45: Va iables Selec ed by GainRa io o he RF bes algo i hm including Dep_Delay & Real_Dep_Time a iables
Sou ce: Made by he au ho
55
Figu e 46: Va iables Selec ed by GainRa io o he RF bes algo i hm excluding Dep_Delay & Real_Dep_Time a iables
Sou ce: Made by he au ho
Figu e 47: Va iables Selec ed by GainRa io o he MLP bes algo i hm including Dep_Delay & Real_Dep_Time a iables
Sou ce: Made by he au ho
Figu e 48: Va iables Selec ed by GainRa io o he MLP bes algo i hm excluding Dep_Delay & Real_Dep_Time a iables
Sou ce: Made by he au ho
56
I is impo an o no e ha when he a iable o Dep_Delay is included in he model is always he one
wi h he highes ank alue. This is because o wha was al eady men ioned and also concluded by M.
S. Kim (2016), ha he inclusion o his ype o a iables gi es o he model a huge in o ma ion when
modeling he delay in he a i al, being a a iable o g ea impo ance when included.
When compa ing he esul s o he algo i hms o his s udy wi h he o he s udies, ha also s udied
he delay a indi idual ai po s using machine lea ning algo i hms (Belcas o e al., 2016; Choi e al.,
2016; Y. J. Kim e al., 2016), i is possible o dis inguish he app oaches and esul s ob ained by hem,
and by his wo k as p esen in Table 12.
I is impo an o no e ha hese h ee s udies di e om his by he use o di e en a iables and
sou ces. By comp ising di e en pe iods o ime. And also, by applying di e en app oaches such as
in a icle o Choi e al. (2016) whe e hey employed 10-Fold C oss Valida ion, ans o med all
ca ego ical a iables o nume ical and p e-p ocessed he da a ga he ed o es he model in he same
way ha p e-p ocessed he aining se .
In hese cases, he a iables o Dep_Delay and Real_Dep_Time we e no used in he model. Fo ha
eason, he esul s used o compa e his wo k wi h he o he s a e also e e ing o he esul s whe e
hese wo a iables we e no conside ed.
Wi hou knowing in o ma ion abou he a ea unde he ROC Cu e o he o he s udies es esul s,
bu knowing ha all he da ase s a e balanced, including his wo k, each o he abo e a icles can be
analyzed wi h his s udy.
I is in e es ing o no e ha , in e e ence o he MLP algo i hm, he o he au ho s ha e accomplished
a highe accu acy han he one a ained in his s udy (87.42% compa ed o 85.63%). Rega ding he RF
algo i hm he esul s he e ob ained a e much be e han he ones om he o he s udies.
Fu he mo e, al hough he e is no in o ma ion o ROC Cu e in he es esul s o he o he a icles
we know ha he alue accessed in his wo k is low. Ne e heless, when compa ing he ROC Cu e o
he aining da a o he a icle o Choi e al. (2016) we can deno e ha he one compu ed by he
model in his s udy is highe (0.68 compa ing o 1 (p esen ed in Table 11), espec i ely).
A icle
In o ma ion used
Objec i e
Type o
Ca ego ical
P edic ion
Algo i hms Accu acy
Random Fo es
Mul ilaye
Pe cep on
(Y. J. Kim e al.,
2016)
Delay s a us o he
day; His o ical ligh
da a; Wea he da a
P edic Class o Delay o
an indi idual ligh
Mul iclass
-
87.42%
(Belcas o e al.,
2016)
His o ical Fligh
da a; Wea he da a
P edic a i al delays o
indi idual ligh due o
wea he condi ions
Bina y
74.20%
-
(Choi e al.,
2016)
His o ical Fligh
da a; Wea he da a
P edic a i al delays o
indi idual ligh
Bina y
80.36%
-
Ou wo k
His o ical Fligh
da a; Wea he da a;
Ai plane in o; Delay
P opaga ion
in o ma ion
P edic a i al delays o
indi idual ligh
Bina y
84.26%
85.63%
Table 12: Rela ed Wo k Compa ison
Sou ce: Made by he au ho , based on (Belcas o e al., 2016)
57
Addi ionally, when one compa es he p esen wo k esul s o he si es ha p edic i a ligh is going
o be delayed o no one can see ha , he accu acy pe o mance o hei models is highe han 80%
which could be compa ed o his wo k’s accu acy pe o mance. This compa ison could no be done
di ec ly because o he di e en in o ma ion a iables and because o he highe o ecas ho izon.
Howe e , by he same condi ions o no conside ing he a iables o Dep_Delay and Real_Dep_Time
o ha e a p edic ion be o e o he ime o he ligh , his wo k’s bes app oach ha is he MLP
algo i hm has as accu acy a alue o 85.63%.
I s ands in he same le el o he DelayCas pe o mance. Highe han FlighCas e , whe e da a is no
balanced and ha ing a ecall o 60% compa ed o he esul o his s udy wi h a ecall o 85.6%.
None heless, lowe han he KnowDelay ha can p edic a ligh o be delayed due o wea he o e
h ee days in ad ance bu also ha ing high esul s by o e p edic ing bad wea he .
Pla o m
In o ma ion used
Pe o mance
Fligh Cas e
Fligh ; Wea he
Accu acy: 85%
Recall: 60%
(Sma e a el, 2009)
KnowDelay
Ai po Pe o mance; Wea he
Accu acy: 90%
(Knowdelay, 2018)
DelayCas
Fligh ; Wea he ; Ai line Company His o y; Numbe o Passenge s
Accu acy: 80-90%
(Tou ism Re iew, 2008)
Table 13: Rela ed Pla o ms Compa ison
Sou ce: Made by he au ho , based on (Belcas o e al., 2016)
58
5. CONCLUSIONS
Knowledge is powe .
– F ancis Bacon (1561-1626)
6
This p ojec has as i s main objec i e he p edic ion o he occu ence o a delay, in he a i als o
Ha s ield-Jackson In e na ional Ai po , being able o know which a iables con ibu e he mos o
he exis ence o delay.
Fo eaching his objec i e he ollowing s eps had o be achie ed:
▪ Cons uc a da abase wi h in o ma ion conce ning he ligh s and addi ional
in o ma ion;
▪ Explo e he delays acco dingly o di e en a iables;
▪ Cons uc a p edic i e model using Da a Mining and Machine Lea ning echniques o
p edic he delay o a ligh in he a i al;
▪ Apply he model de eloped in new da a o make p edic ions and see which i s be e
o he p oblem acco ding o he desi ed ad ance o he p edic ion;
▪ Iden i y he a iables ha con ibu e mos o he exis ence o delay.
The i s s ep was accomplished by he cons uc ion o a da ase o p esen o he algo i hm ia he
use o he Excel and SQL Managemen S udio. I con empla ed in o ma ion o ligh s and ai planes
bu also wea he , ime zone, ligh du a ions and holidays.
The second s ep was possible h ough he use o he da ase cons uc ed, and acco dingly o he
obse a ions ga he ed. I was possible o no e ha he mon hs wi h a highe pe cen age o delays
we e he ones occu ing in he Win e and Summe seasons. Highe dis ances ha e a highe
pe cen age o delayed ligh s being seen ha mos o ligh s a e o sho dis ances. The schedules
depa u es and a i al imes ha e a highe pe cen age o delays in he a e noon and e ening.
Ai po s o o igin loca ed in he sou h o he USA a e he ones wi h he highe pe cen age o delay. Is
in e es ing ha , when he ai plane ope a ing he ligh has a highe numbe o sea s (>300), he
pe cen age o delayed ligh s inc eases. When looking o he ole o he ligh s ha a e delayed o
canceled in he p e ious day o a ligh a i s o igin, i is seen ha when he numbe inc eases he
pe cen age o delayed ligh s also inc eases. As a con i ma ion o he al eady known by in ui ion,
ex eme alues in empe a u e inc eases he pe cen age o delayed ligh s. Highe le els o
p ecipi a ion ha e ewe numbe s o ligh s, bu he ones ha happen has a high pe cen age o
delayed ligh s. Highe eloci y o he wind causes an inc ease o he delayed ligh pe cen age. And,
a loss o isibili y inc eases he pe cen age o delayed ligh s.
Wi h he insigh s ob ained by he isualiza ion o he da a, he p ep ocessing and ans o ma ion was
ca ied ou o he building o a model capable o p edic ing he delay in o ma ion concluding he
hi d s ep o he speci ic objec i es.
6
F ancis Bacon, 1s Viscoun S Alban, was an English philosophe , s a esman, scien is , ju is , o a o ,
and au ho ( o mo e in o ma ion consul h ps://en.wikipedia.o g/wiki/F ancis_Bacon)
59
Fo pe o ming he ou h s ep, he model ob ained was applied o new da a o see how well i could
pe o m when p edic ing in he unknown da a.
Conce ning he di e en ad ance o he p edic ion ha we could ha e, by including o excluding he
a iables o Dep_Delay and Real_Dep_Time, i was possible o achie e be e pe o mance wi h he
MLP algo i hm among he o he s.
When conside ing he wo a iables, he MLP pe o med be e by he use o SMOTE echnique,
wi hou he inclusion o he ou lie s and including he en a iables mos impo an acco dingly o
he GainRa io. I no conside ing he wo a iables hen he MLP algo i hm pe o ms be e by also
using he SMOTE echnique, wi h he inclusion o he ou lie s and he i een a iables mos
impo an also acco dingly o he GainRa io.
By he conclusions made in he Resul s and Discussion chap e , and esponding o he i h s ep, i
can be seen ha a iables o wea he , in gene al, we e he ones wi h highe impo ance in he bes
model o each ad ance o he p edic ion. When he a iable “Dep_Delay” is included is he one ha
had highe gain a io because o he access o his in o ma ion imp o es he p edic ions as seen in
he Resul s and Discussion chap e .
Also, he a iables “An ” and “Max_sea s” a e conside ed in bo h models bu wi h less impo ance,
happening he same wi h he a iable c ea ed abou he amoun o delayed and canceled ligh s in
he p e ious day a he o igin ai po (“P e _Day_Delay_Occu ”).
Con a ily o wha was expec ed, a iables such as he day o mon h, scheduled depa u e and a i al
ime, dis ances, and o he a iables u n ou o no be such good han he ones chosen in hese wo
models.
66
K ozel, J., Capozzi, B., And e, A. D., & Smi h, P. (2003). The u u e Na ional Ai space Sys em: Design
equi emen s imposed by wea he cons ain s. In AIAA Guidance, Na iga ion, and Con ol
Con e ence (pp. 1–14).
Kuma , G. R., Konga a, V. S., & Ramachand a, D. G. . (2013). An E icien Ensemble Based
Classi ica ion Techniques o Medical Diagnosis. In e na ional Jou nal o La es Technology in
Enginee ing, Managemen & Applied Science, II(VIII), 5–9.
Lachhe a, P., & Bawa, S. (2016). Combining Syn he ic Mino i y O e sampling Technique and Subse
Fea u e Selec ion Technique Fo Class Imbalance P oblem. P oceedings o he In e na ional
Con e ence on Ad ances in In o ma ion Communica ion Technology & Compu ing - AICTC ’16,
(Oc obe ), 1–6. h ps://doi.o g/10.1145/2979779.2979804
La ose, D. T. (2005). Disco e ing Knowledge in Da a: An In oduc ion o Da a Mining. John Wiley &
Sons, Inc. Re ie ed om
h ps://books.google.p /books?id=JbPMdPWQIOwC&pg=PA28&lpg=PA28&dq=Da a+p epa a io
n+is+60%25+o +e o + o +da a+mining+p ocess+(Pyle)&sou ce=bl&o s=jKmcc9mx5Z&sig=cN-
SJ9O5 aX3zGi-amS guxQmwQ&hl=p -PT&sa=X& ed=0ahUKEwj3msG29I7XAhWGshQKHa-
TB MQ6AEIQzAE# =one
La alle, S., Lesse , E., Shockley, R., Hopkins, M. S., & K uschwi z, N. (2011). Big Da a, Analy ics and he
Pa h F om Insigh s o Value. MIT Sloan Managemen Re iew, 52(2), 21–32.
h ps://doi.o g/10.0000/PMID57750728
Liaw, A., & Wiene , M. (2002). Classi ica ion and Reg ession by andomFo es . R News, 2(3), 18–22.
Re ie ed om h p://www.cs.colo ado.edu/depa men /publica ions/ epo s/docs/CU-CS-
954-03.pd %5Cnpape s2://publica ion/uuid/F28DC17B-D961-4E66-9DEF-
B940467A5068%5Cnh p://www.pubmedcen al.nih.go /a icle ende . cgi?a id=3223189& oo
l=pmcen ez& ende ype=abs ac %5Cn
Lippmann, R. P. (1987). An In oduc ion o Compu ing: wi h Neu al Ne s. IEEE ASSP Magazine, (Ap il).
h ps://doi.o g/10.1109/MASSP.1987.1165576
Liu, X.-Y., Wu, J., & Zhou, Z.-H. (2009). Explo a o y Unde sampling o Class Imbalance Lea ning. IEEE
T ansac ions on Sys ems, Man and Cybe ne ics, 39(2), 539–550.
h ps://doi.o g/10.1109/TSMCB.2008.2007853
Ma iscal, G., Ma bán, Ó., & Fe nández, C. (2010). A su ey o da a mining and knowledge disco e y
p ocess models and me hodologies. Knowledge Enginee ing Re iew, 25(2), 137–166.
h ps://doi.o g/10.1017/S0269888910000032
Mazzeo, M. J. (2003). Compe i ion and se ice quali y in he U.S. ai line indus y. Re iew o Indus ial
O ganiza ion, 22(4), 275–296. h ps://doi.o g/10.1023/A:1025565122721
Mi chell, T. M. (1997). Machine lea ning. McG aw-Hill Science/Enginee ing/Ma h.
Moisen, G. G. (2008). Classi ica ion and Reg ession T ees. Encyclopedia o Ecology, 582–588.
h ps://doi.o g/h p://www. s. ed.us/ m/pubs_o he / m s_2008_moisen_g001.pd
Muelle , E., & Cha e ji, G. (2002). Analysis o ai c a a i al and depa u e delay cha ac e is ics.
AIAA Ai c a Technology, In eg a ion and Ope a ions. h ps://doi.o g/10.2514/6.2002-5866
Muenchen, R. A. (2017). 4s a s.com. Re ie ed No embe 14, 2017, om
h p:// 4s a s.com/a icles/popula i y/
67
Nadolski, V. L. (1998). Au oma ed Su ace Obse ing Sys em (ASOS) Use ’s Guide. Re ie ed om
h p://www.nws.noaa.go /asos/pd s/aum- oc.pd
O ice o A ia ion En o cemen and P oceedings. (2016). Ai T a el Consume Repo s. Re ie ed om
h ps://www. anspo a ion.go /ai consume /ai - a el-consume - epo s
Pali , A. K., & Popo ic, D. (2005). Neu al Ne wo ks App oach. In Compu a ional In elligence in Time
Se ies Fo ecas ing: Theo y and Enginee ing Applica ions (Ad ances in Indus ial Con ol) (1s ed.,
p. 372). Sp inge -Ve lag London. h ps://doi.o g/0.1007/1-84628-184-9
Pennsyl ania S a e Uni e si y. (2017). PennS a e Ebe ly College o Science Applied Da a Mining and
S a is ical Lea ning. Re ie ed Oc obe 24, 2017, om
h ps://onlinecou ses.science.psu.edu/s a 857/node/161
Pinkus, A. (1999). App oxima ion heo y o he MLP model in neu al ne wo ks. Ac a Nume ica, 8,
143. h ps://doi.o g/10.1017/S0962492900002919
Plane Spo e s.ne . (2017). Ai line Flee s. Re ie ed Augus 12, 2017, om
h ps://www.planespo e s.ne /ai lines
Pyle, D. (1999). Da a P epa a ion o Da a Mining. O de A Jou nal On The Theo y O O de ed Se s
And I s Applica ions (Vol. 17). Mo gan Kau mann Publishe s, Inc.
h ps://doi.o g/10.1080/713827180
Py gio is, N., Malone, K. M., & Odoni, A. (2013). Modelling delay p opaga ion wi hin an ai po
ne wo k. T anspo a ion Resea ch Pa C: Eme ging Technologies, 27, 60–75.
h ps://doi.o g/10.1016/j. c.2011.05.017
Qianya, L., Lei, W., Rong, F., Bin, W., & Xinhong, H. (2015). An Analysis Me hod o Fligh Delays
based on Bayesian Ne wo k. P oceedings o he 2015 27 h Chinese Con ol and Decision
Con e ence, CCDC 2015, 2561–2565.
Quinlan, J. R. (1993). C4.5: P og ams o Machine Lea ning. San Ma eo, Cali o nia: Mo gan Kau mann
Publishe s, Inc. h ps://doi.o g/10.1007/BF00993309
Rebollo, J. J., & Balak ishnan, H. (2014). Cha ac e iza ion and p edic ion o ai a ic delays.
T anspo a ion Resea ch Pa C: Eme ging Technologies, 44, 231–241.
h ps://doi.o g/10.1016/j. c.2014.04.007
Rong, F., Qianya, L., Bo, H., Jing, Z., & Dongdong, Y. (2015). The P edic ion o Fligh Delays based he
Analysis o Random Fligh Poin s. 34 h Chinese Con ol Con e ence (CCC), 3992–3997.
h ps://doi.o g/10.1109/ChiCC.2015.7260255
Saeys, Y., Inza, I., & La anaga, P. (2007). A e iew o ea u e selec ion echniques in bioin o ma ics.
Bioin o ma ics, 23(19), 2507–2517. h ps://doi.o g/10.1093/bioin o ma ics/b m344
SAS. (2017). Machine Lea ning: O que é e po que é impo an e? Re ie ed July 17, 2017, om
h ps://www.sas.com/p _b /insigh s/analy ics/machine-lea ning.h ml
Shale -Shwa z, S., & Ben-Da id, S. (2014). Unde s anding Machine Lea ning: F om Theo y o
Algo i hms. Camb idge Uni e si y P ess. Re ie ed om h ps://books.google.p /books?hl=p -
PT&l =&id=H 6QAwAAQBAJ&oi= nd&pg=PR15&dq=Unde s anding+machine+lea ning:+ om+
heo y+ o+algo i hms.+2014&o s=2HqkSihNM7&sig=Y76uKAFNYuWYLT4RmsTx 20SJck& edi _e
sc=y# =onepage&q=neu al ne wo ks& = alse
68
Smallen, D. (2016). 2015 U. S. Based Ai line T a ic Da a. Re ie ed om
h p://www. i a.do .go /b s/si es/ i a.do .go .b s/ iles/b s18_16.pd
Sma e a el. (2009). SMARTERTRAVEL. Re ie ed Janua y 30, 2018, om
h ps://www.sma e a el.com/2009/08/19/new- ool-p edic s- ligh -delays/
Smi h, D. a, & She y, L. (2008). Decision Suppo Tool o P edic ing Ai c a A i al Ra es , G ound
Delay P og ams , and Ai po Delays om Wea he Fo ecas s. Sys ems Resea ch.
h ps://doi.o g/10.1109/ICNSURV.2008.4559186
Soley-bo i, M. (2013). Dealing wi h missing da a: Key assump ions and me hods o applied analysis.
Technical Repo No. 4. Re ie ed om h ps://www.bu.edu/sph/ iles/2014/05/Ma ina- ech-
epo .pd
S e nbe g, A., Soa es, J., Ca alho, D., & Ogasawa a, E. (2017). A Re iew on Fligh Delay P edic ion,
1–15. Re ie ed om h p://a xi .o g/abs/1703.06118
Takacs, G. (2014). P edic ing ligh a i al imes wi h a mul is age model. 2014 IEEE In e na ional
Con e ence on Big Da a, IEEE Big Da a 2014, 2, 78–84.
h ps://doi.o g/10.1109/BigDa a.2014.7004435
Tang, J., Alelyani, S., & Liu, H. (2014). Fea u e Selec ion o Classi ica ion: A Re iew. In C. Agga wal
(Ed.), Da a Classi ica ion: Algo i hms and Applica ions (pp. 37–64). CRC P ess.
h ps://doi.o g/10.1.1.409.5195
Teo ey, T. J., Yang, D., & F y, J. P. (1986). A Logical Design Me hodology o Rela ional Da abases
Using he Ex ended En i y-Rela ionship Model. ACM Compu ing Su eys, 18(2), 197–222.
h ps://doi.o g/10.1145/7474.7475
Tou ism Re iew. (2008). Tou ism Re iew. Re ie ed Janua y 30, 2018, om h ps://www. ou ism-
e iew.com/delaycas -p edic ing- ligh -delays-news856
T a elMa h. (2017a). T a elMa h. Re ie ed No embe 17, 2017, om
h ps://www. a elma h.com/ ime-change/
T a elMa h. (2017b). T a elMa h. Re ie ed No embe 17, 2017, om
h ps://www. a elma h.com/ lying- ime/
T y ona, N., Busbo g, F., & Ch is iansen, J. G. B. (1999). s a ER: A Concep ual Model o Da a
Wa ehouse Design. ACM 2nd In e na ional Wo kshop Da a Wa ehousing and OLAP, 3–8.
h ps://doi.o g/10.1145/319757.319776
Tu, Y., Ball, M. O., & Jank, W. S. (2008a). Es ima ing Fligh Depa u e Delay Dis ibu ions—A S a is ical
App oach Wi h Long-Te m T end and Sho -Te m Pa e n. Jou nal o he Ame ican S a is ical
Associa ion, 103(481), 112–125. h ps://doi.o g/10.1198/016214507000000257
Tu, Y., Ball, M. O., & Jank, W. S. (2008b). Es ima ing Fligh Depa u e Delay Dis ibu ions—A S a is ical
App oach Wi h Long-Te m T end and Sho -Te m Pa e n. Jou nal o he Ame ican S a is ical
Associa ion, 103(481), 112–125. h ps://doi.o g/10.1198/016214507000000257
Tukey, J. W. (1961). The Fu u e o Da a Analysis. Annals o he Ins i u e o S a is ical Ma hema ics.
h ps://doi.o g/10.1214/aoms/1177728422
U.S. Depa men o T anspo a ion. (2017). 2015 Fligh Delays and Cancella ions: Which ai line
should you ly on o a oid signi ican delays? Re ie ed Augus 12, 2017, om
69
h ps://www.kaggle.com/usdo / ligh -delays
U.S. Depa men o T anspo a ion, Fede al A ia ion Adminis a ion, & Co po a ion, M. (2004).
Ai po Capaci y Benchma k Repo 2004. Washing on, D.C. Re ie ed om
p:// p.agl. aa.go /ORD DEIS/Re e ance Documen s/Appendix A/App A - Re Doc 10.pd
U.S. O ice o Pe sonnel Managemen . (2017a). OPM.GOV. Re ie ed No embe 17, 2017, om
h ps://www.opm.go /abou -us/ou -mission- ole-his o y/wha -we-do/
U.S. O ice o Pe sonnel Managemen . (2017b). OPM.GOV. Re ie ed Sep embe 8, 2017, om
h ps://www.opm.go /policy-da a-o e sigh /snow-dismissal-p ocedu es/ ede al-holidays/
Uni e si y o Waika o. (2018). Weka Wikispace. Re ie ed Janua y 9, 2018, om
h ps://weka.wikispaces.com/
Wang, C. W. (2006). New Ensemble Machine Lea ning Me hod o Classi ica ion and P edic ion on
Gene Exp ession Da a. Annual In e na ional Con e ence o he IEEE Enginee ing in Medicine and
Biology Socie y., 1(Aug-Sep ), 3478–81. h ps://doi.o g/10.1109/IEMBS.2006.259893
Wang, X. W. X. (2009). In elligen Quali y Managemen Using Knowledge Disco e y in Da abases.
2009 In e na ional Con e ence on Compu a ional In elligence and So wa e Enginee ing, 1–4.
h ps://doi.o g/10.1109/CISE.2009.5364999
Weiss, G. M. (2004). Mining wi h Ra i y: A Uni ying F amewo k. SIGKDD Explo a ions, 6(1), 7–19.
h ps://doi.o g/10.1145/1007730.1007734
Wi en, I. H., F ank, E., & Hall, M. a. (2011a). Da a Mining: P ac ical Machine Lea ning Tools and
Techniques (2nd ed.). Mo gan Kau manne. h ps://doi.o g/0120884070, 9780120884070
Wi en, I. H., F ank, E., & Hall, M. A. (2011b). Da a mining: P ac ical Machine Lea ning Tools and
Techniques, Thi d Edi ion. Mo gan Kau mann se ies in da a managemen sys ems (3 d ed.).
Else ie Inc. h ps://doi.o g/10.1002/1521-3773(20010316)40:6<9823::AID-ANIE9823>3.3.CO;2-
C
Wu, S.-H., & Da aLab. (2016). C oss Valida ion & Ensembling. Re ie ed Oc obe 25, 2017, om
h p://www.cs.n hu.edu. w/~shwu/cou ses/ml/labs/08_CV_Ensembling/08_CV_Ensembling.h
ml
Xu, N., Donohue, G., Laskey, K. B., & Chen, C. (2005). Es ima ion o Delay P opaga ion in he Na ional
A ia ion Sys em Using Bayesian Ne wo ks. 6 h USA/Eu ope Ai T a ic Managemen Resea ch
and De elopmen Semina , 478–489. Re ie ed om
h ps://pd s.seman icschola .o g/9c5a/7c726315387acb5 5c 8e849524046cd207.pd
Yablonsky, G., S eckel, R., Cons ales, D., Fa nan, J., Le cel, D., & Pa anka , M. (2014). Fligh delay
pe o mance a Ha s ield-Jackson A lan a In e na ional Ai po . Jou nal o Ai line and Ai po
Managemen , 4(1), 78–95. h ps://doi.o g/10.3926/jai m.22
Yao, R., Jiandong, W., & Tao, X. (2010). A ligh delay p edic ion model wi h conside a ion o c oss-
ligh plan awai ing esou ces. In P oceedings - 2nd IEEE In e na ional Con e ence on Ad anced
Compu e Con ol, ICACC (Vol. 4, pp. 1–5). h ps://doi.o g/10.1109/ICACC.2010.5487088
Zhang, G., Pa uwo, B. E., & Hu, M. Y. (1997). Fo ecas ing wi h a i icial neu al ne wo ks: The s a e o
he a . In e na ional Jou nal o Fo ecas ing, 14, 35–62. h ps://doi.o g/10.1016/S0169-
2070(97)00044-7
70
Zhang, S., Yang, Q., & Zhang, C. (2002). P oceedings o he Fi s In e na ional Wo kshop on Da a
Cleaning and P ep ocessing (p. iii). Maebashi, Japan. Re ie ed om
h ps://www. esea chga e.ne /p o ile/S e ano_Ce i/publica ion/232866130_GAsRULE_ o _Kn
owledge_Disco e y/links/09e4150c6 1905948 000000/GAsRULE- o -Knowledge-Disco e y.pd
Zonglei, L., Jiandong, W., & Guansheng, Z. (2008). A New Me hod o Ala m La ge Scale o Fligh s
Delay Based on Machine Lea ning. In P oceedings o he In e na ional Symposium on Knowledge
Acquisi ion and Modeling (KAM 08) (pp. 589–592). Wuhan, China: IEEE CS P ess.
h ps://doi.o g/10.1109/KAM.2008.18
Zonglei, L., Jiandong, W., & Tao, X. (2009). A New Me hod o Fligh Delays Fo ecas Based on he
Recommenda ion Sys em. ISECS In e na ional Colloquium on Compu ing, Communica ion,
Con ol and Managemen , 1, 46–49. h ps://doi.o g/10.1109/CCCM.2009.5268153
Zupan, B., & Demsa , J. (2008). Open-Sou ce Tools o Da a Mining. Clinics in Labo a o y Medicine,
28(1), 37–54. h ps://doi.o g/10.1016/j.cll.2007.10.002
71
8. ANNEXES
ANNEX 1: PROCEEDINGS FOR DATA VALIDATION BY TYPE OF SOURCE DATA INFORMATION
Fo he in eg a ion, each sou ce in o ma ion had o be ea ed o p o ide homogenei y as is
explained in his annex.
A. Fligh In o ma ion
The da a di ec ly ex ac ed om he BTS wi h in o ma ion abou he ligh s was no in he igh
o ma . Some o he ope a ions made we e: (1) ep esen ing he minu es as a uni , ins ead o i e
minu es he alues we e ep esen ed as 5.00 minu es, o ha eason all a iables wi h minu e
alues we e ans o med om ex o numbe o a be e unde s anding; (2) o be able o
unde s and he eal ime, because he a iables ha ep esen ed hou s we e in o al o minu es (90)
and no in hou o ma (01:30), he second o ma was applied o all a ibales o which his decision
could be applied. This decision also helped o make possible he calcula ions o hou s o he
cons uc ion o u he a iables and o consul ing he da abase.
Due o a la ge amoun o da a ex ac ed pe mon h, app oxima ely mo e han wen y housand
ligh s, i would no be possible o make hese changes manually in each ex ac ed excel ile.
The e o e, he solu ion o hese p oblems was possible h ough VBA code in he mon hly iles.
B. Ai plane In o ma ion
To ob ain in o ma ion abou he ai plane o each ligh , he FAA websi e was used (Fede al A ia ion
Adminis a ion, 2016a). As an ai plane can be used on mo e han one ligh in a mon h, i s all he
ai c a used in he eigh mon hs in ques ion we e iden i ied, ep esen ing a o al o 3279 di e en
ai c a .
Then, i was necessa y o con i m he alidi y o he unique numbe o he ai c a , he N-Numbe .
Following speci ic ules o o ma , s ipula ed by he FAA, an N-Numbe can con ain:
▪ One o i e numbe s (e.g., N12345);
▪ One o ou numbe s ollowed by one le e (e.g., N1234Z);
▪ One o h ee numbe ollowed by wo le e s (e.g., N123AZ).
On he o he hand, i canno con ain a ze o as he i s numbe (e.g., N01234) o he le e s "i" o "o"
(e.g., N1234I o N123AO) because hey can be mis aken o numbe s one and ze o (Fede al A ia ion
Adminis a ion, 2015).
A e con i ma ion, only wo housand and nine hund ed six y- i e (2965) ai planes had a co ec N-
Numbe . The emaining h ee housand and ou een (314) ai c a w ongly p esen ed ano he ype
o indica o numbe , he so-called Flee Numbe - co esponding o he numbe o he ai plane
wi hin he ai line -, o en supplied o he BTS as he N-Numbe . The e o e, o ob ain he N-Numbe ,
egis e ed by he NAA e e ing o he Flee Numbe p esen in he ligh da a, i was necessa y o
eso o he da abase o hese numbe s o each ai line h ough he use o he Plane Spo e s
websi e, a ci il da abase wi h ai planes in o ma ion (Plane Spo e s.ne , 2017). A e accessing he
da abase wi h in o ma ion abou each ai line's Flee Numbe , we concluded ha om he h ee
housand and ou een (314) eco ds o be alida ed was no possible o iden i y he N-Numbe o
72
se en (7), hus assuming he numbe p esen ed in he ligh da a as in alid and consequen ly, ligh
showing hese N-Numbe s we e no included in he inal da ase .
Finally, i was necessa y o manually sea ch and ill in o ma ion o each ai plane by consul ing he N-
numbe (Fede al A ia ion Adminis a ion, 2016c) and model ai plane (Fede al A ia ion
Adminis a ion, 2016b) on he FAA websi e.
Fo au hen ica e he ul illmen o he FAA egis a ion ules o he N-Numbe , and due o he la ge
amoun o da a, we used Excel unc ions.
C. Wea he In o ma ion
The acquisi ion o wea he in o ma ion was he one which causes mo e di icul ega ding a ailabili y
because i was ha d o ind a eposi o y wi h his o ical in o ma ion o all he ai po 's loca ions. In
ha sense, was ound he IEM, whe e he da a was s o ed in he a chi e and was necessa y o
download hem. The download was done o he yea o 2015 and o all he i y s a es, because he
websi e allowed o download he in o ma ion o mul iple ai po s in he same s a e a once.
I was ob ained he hou ly obse a ions abou empe a u e (in Celsius deg ees), p ecipi a ion (in
millime e s pe hou ), wind speed (in miles pe hou ), isibili y (in miles) and wea he phenomena
e en a he ime o obse a ion (acco ding o METAR wea he phenomena ha a e epo ed in
e ms o ypes and cha ac e is ics, and quali ied wi h espec o in ensi y o p oximi y o he
ae od ome (In e na ional Ci il A ia ion O ganiza ion, 2011)).
Du ing his phase some ai po s senso s did no con ain his o ical da a o ce ain hou s o he day
and a g ea pa o he p esen wea he in o ma ion was ep esen ed wi h a alue “M”, being
men ioned by he websi e as an obse a ion ha was epo ed as missing o ha we e se o missing
a e quali y con ol check, o a alue ha was ne e epo ed by he ASOS senso . This kind o
p oblems may a ise, o example, o some ligh in a ce ain hou ( ha was no a ailable) a missing
alue in a iables o wea he .
A e ex ac ion o he his o ical hou ly da a o each day o he yea o 2015 o all one hund ed
six y-nine (169) ai po s i was necessa y he ea men o each excel o each s a e eso ing o excel
unc ions. Duplica e eco ds elimina ion was necessa y as well as he co ec ion o in o ma ion o
p esen wea he codes assumed by he excel as unc ions and no ex in o ma ion.
73
ANNEX 2: ENTITY-RELATIONSHIP PHYSICAL MODEL WITH CROW’S FOOT NOTATION
ANNEX 3: DISTINCT AIRPORTS AND CITIES WITH RESPECT TO TIME-ZONE
Ai po Name
Ci y and S a e
Time-Zone in
ela ion o A lan a
ATL
A lan a, Geo gia
-------
DFW
Dallas/Fo Wo h, Texas
-01:00
MIA
Miami, Flo ida
00:00
SEA
Sea le, Washing on
-03:00
GSO
G eensbo o/High Poin , No h Ca olina
00:00
LAX
Los Angeles, Cali o nia
-03:00
MDW
Chicago, Illinois
-01:00
ORD
JFK
New Yo k, New Yo k
00:00
LGA
SJU
San Juan, Pue o Rico
00:00
74
SAT
San An onio, Texas
-01:00
RSW
Fo Mye s, Flo ida
00:00
DAY
Day on, Ohio
00:00
AVL
Ashe ille, No h Ca olina
00:00
SDF
Louis ille, Ken ucky
00:00
EWR
Newa k, New Je sey
00:00
STT
Cha lo e Amalie, U.S. Vi gin Islands
00:00
OMA
Omaha, Neb aska
-01:00
JAX
Jackson ille, Flo ida
00:00
PIT
Pi sbu gh, Pennsyl ania
00:00
CMH
Columbus, Ohio
00:00
MEM
Memphis, Tennessee
-01:00
PBI
Wes Palm Beach/Palm Beach, Flo ida
00:00
MKE
Milwaukee, Wisconsin
-01:00
MSY
New O leans, Louisiana
-01:00
MSP
Minneapolis, Minneso a
-01:00
MCO
O lando, Flo ida
00:00
TPA
Tampa, Flo ida
00:00
RDU
Raleigh/Du ham, No h Ca olina
00:00
SNA
San a Ana, Cali o nia
-03:00
PDX
Po land, O egon
-03:00
PHL
Philadelphia, Pennsyl ania
00:00
HOU
Hous on, Texas
-01:00
IAH
CLT
Cha lo e, No h Ca olina
00:00
FLL
Fo Laude dale, Flo ida
00:00
MOB
Mobile, Alabama
-01:00
HNL
Honolulu, Hawaii
-05:00
DCA
Washing on DC, Vi ginia
00:00
IAD
TLH
Tallahassee, Flo ida
00:00
JAN
Jackson/Vicksbu g, Mississippi
-01:00
GRR
G and Rapids, Michigan
00:00
DTW
De oi , Michigan
00:00
EYW
Key Wes , Flo ida
00:00
BDL
Ha o d, Connec icu
00:00
MCI
Kansas Ci y, Missou i
-01:00
BOS
Bos on, Massachuse s
00:00
LAS
Las Vegas, Ne ada
-03:00
PNS
Pensacola, Flo ida
-01:00
STL
S . Louis, Missou i
-01:00
75
DAB
Day ona Beach, Flo ida
00:00
IND
Indianapolis, Indiana
00:00
ATW
Apple on, Wisconsin
-01:00
PVD
P o idence, Rhode Island
00:00
AUS
Aus in, Texas
-01:00
BHM
Bi mingham, Alabama
-01:00
BUF
Bu alo, New Yo k
00:00
SAV
Sa annah, Geo gia
00:00
EGE
Eagle, Colo ado
-02:00
RIC
Richmond, Vi ginia
00:00
ORF
No olk, Vi ginia
00:00
DEN
Den e , Colo ado
-02:00
PHX
Phoenix, A izona
-02:00
SFO
San F ancisco, Cali o nia
-03:00
SYR
Sy acuse, New Yo k
00:00
TUS
Tucson, A izona
-02:00
GPT
Gul po /Biloxi, Mississippi
-01:00
SLC
Sal Lake Ci y, U ah
-02:00
BNA
Nash ille, Tennessee
-01:00
GSP
G ee , Sou h Ca olina
00:00
JAC
Jackson, Wyoming
-02:00
ROC
Roches e , New Yo k
00:00
SAN
San Diego, Cali o nia
-03:00
TRI
B is ol/Johnson Ci y/Kingspo , Tennessee
00:00
AVP
Sc an on/Wilkes-Ba e, Pennsyl ania
00:00
BWI
Bal imo e, Ma yland
00:00
CLE
Cle eland, Ohio
00:00
LIT
Li le Rock, A kansas
-01:00
CVG
Cincinna i, Ken ucky
00:00
MHT
Manches e , New Hampshi e
00:00
ECP
Panama Ci y, Flo ida
-01:00
ABQ
Albuque que, New Mexico
-02:00
MLB
Melbou ne, Flo ida
00:00
CHS
Cha les on, Sou h Ca olina
00:00
ALB
Albany, New Yo k
00:00
FNT
Flin , Michigan
00:00
VPS
Valpa aiso, Flo ida
-01:00
TYS
Knox ille, Tennessee
00:00
COS
Colo ado Sp ings, Colo ado
-02:00
ELP
El Paso, Texas
-02:00
SRQ
Sa aso a/B aden on, Flo ida
00:00
82
ANNEX 9: INFLUENCE OF ARRIVAL DELAY ON WIND VARIABLES
In each g aph is con ained wo ypes o in o ma ion:
▪ he equency o each class by he dependen a iable on he e ical axis
▪ he ele ance in pe cen age o each class o he dependen a iable by each class o
he independen a iable in analysis on he ho izon al axis
The nex igu es show he h ee a iables abou he wind whe e is illus a ed:
A. In luence o A i al Delay on Wind a he o igin a he schedule depa u e ime
B. In luence o A i al Delay on Wind a he des ina ion a he schedule depa u e ime
in he o igin
C. In luence o A i al Delay on Wind a he des ina ion a he schedule a i al ime
83
ANNEX 10: INFLUENCE OF ARRIVAL DELAY ON VISIBILITY VARIABLES
In each g aph is con ained wo ypes o in o ma ion:
▪ he equency o each class by he dependen a iable on he e ical axis
▪ he ele ance in pe cen age o each class o he dependen a iable by each class o
he independen a iable in analysis on he ho izon al axis
The nex igu es show he h ee a iables abou he isibili y whe e is illus a ed:
A. In luence o A i al Delay on Visibili y a he o igin a he schedule depa u e ime
B. In luence o A i al Delay on Visibili y a he des ina ion a he schedule depa u e
ime in he o igin
C. In luence o A i al Delay on Visibili y a he des ina ion a he schedule a i al ime
84
ANNEX 11: TEST RESULTS TABLE OF SMOTE TECHNIQUE
Da ase
Sampling
Technique
Va iable
Dep_Delay +
Real_Dep_Time
P esence
Ou lie s
P esence
A ibu e
Selec ion
Algo i hm
CCI PR RE F ROC
T o build
model
T o es
model
Fligh Da a SMOTE Wi h Wi h C sSubse E al 6 J48 79,7689 0,892 0,798 0,824 0,833 10,29 0,15
RF 79,5948 0,892 0,796 0,823 0,828 132,6 3,04
MLP 79,7006 0,892 0,797 0,823 0,838 121,76 0,19
GainRa io 10 J48 79,7716 0,892 0,798 0,824 0,835 26,76 0,5
RF 79,7716 0,892 0,798 0,824 0,826 172,35 3,63
MLP 54,1045 0,863 0,541 0,601 0,878 197,16 0,41
15 J48 79,5975 0,892 0,796 0,823 0,834 31,7 0,21
RF 79,7649 0,892 0,798 0,824 0,834 217,14 3,58
MLP 79,7247 0,892 0,797 0,824 0,89 374,49 0,57
20 J48 72,2549 0,882 0,723 0,763 0,762 45,88 0,32
RF 79,7609 0,892 0,798 0,824 0,832 236,38 2,65
MLP 14,3653 0,877 0,144 0,037 0,86 464,76 0,49
Wi hou C sSubse E al 4 J48 79,7716 0,892 0,798 0,824 0,831 1,63 0,43
RF 79,7716 0,892 0,798 0,824 0,829 46,91 2,7
MLP 78,4233 0,89 0,784 0,813 0,861 50,22 0,22
GainRa io 10 J48 79,7716 0,892 0,798 0,824 0,829 5,64 0,24
RF 80,0032 0,877 0,8 0,824 0,827 75,7 2,29
MLP 84,0601 0,9 0,841 0,858 0,891 115,27 0,34
15 J48 79,7716 0,892 0,798 0,824 0,829 6,93 0,21
RF 80,1224 0,841 0,801 0,817 0,801 69,5 3,68
MLP - - - - - - -
20 J48 79,7716 0,892 0,798 0,824 0,829 5,23 0,32
RF - - - - - - -
MLP - - - - - - -
Wi hou Wi h C sSubse E al 13 J48 80,4919 0,78 0,805 0,791 0,539 48,73 0,33
RF 76,3319 0,777 0,763 0,77 0,554 228,64 4,81
MLP 84,0387 0,763 0,84 0,791 0,584 290,49 0,32
GainRa io 10 J48 83,0640 0,775 0,831 0,796 0,534 35,26 0,29
RF 80,8936 0,759 0,809 0,781 0,534 203,61 7,15
MLP 84,1124 0,771 0,841 0,794 0,526 191,88 0,26
15 J48 75,6022 0,772 0,756 0,764 0,524 57,66 0,58
RF 79,6096 0,777 0,796 0,786 0,527 227,01 4,47
MLP 85,6267 0,793 0,856 0,792 0,564 324,77 0,48
20 J48 75,9075 0,767 0,759 0,763 0,508 74,95 0,33
RF 78,4956 0,774 0,785 0,779 0,541 268,22 3,21
MLP 62,5343 0,752 0,625 0,675 0,511 569,87 0,57
Wi hou C sSubse E al 7 J48 75,1162 0,745 0,751 0,748 0,485 9,02 0,18
RF 85,4165 0,757 0,854 0,79 0,507 90,79 3,13
MLP 56,5413 0,767 0,565 0,63 0,533 79,96 0,23
GainRa io 10 J48 74,8805 0,747 0,749 0,748 0,502 15,13 0,53
RF 84,2556 0,756 0,843 0,789 0,52 96,99 3,53
MLP 85,6695 0,734 0,857 0,791 0,505 102,47 0,34
15 J48 59,5244 0,757 0,595 0,654 0,509 17,18 0,6
RF 59,9127 0,76 0,599 0,657 0,518 146,94 18,85
MLP - - - - - - -
20 J48 71,5600 0,759 0,716 0,735 0,521 10,54 0,35
RF - - - - - - -
MLP - - - - - - -
85
ANNEX 12: TEST RESULTS TABLE OF UNDERSAMPLING TECHNIQUE
Da ase
Sampling
Technique
Va iable
Dep_Delay +
Real_Dep_Time
P esence
Ou lie s
P esence
A ibu e
Selec ion
CCI PR RE F ROC
T o build
model
T o es
model
Fligh Da a Unde sampling Wi h Wi h C sSubse E al 3 J48 79,7716 0,892 0,798 0,824 0,829 0,35 0,07
RF 79,7555 0,892 0,798 0,824 0,839 7,63 1,47
MLP 79,7622 0,892 0,798 0,824 0,888 89,88 0,2
GainRa io 10 J48 79,7569 0,892 0,798 0,824 0,831 3,93 0,21
RF 79,6203 0,892 0,796 0,823 0,848 39,88 2,33
MLP 28,1736 0,86 0,282 0,283 0,892 162,72 0,35
15 J48 79,7716 0,892 0,798 0,824 0,829 6,79 0,54
RF 75,2045 0,885 0,752 0,787 0,835 63,68 3,2
MLP - - - - - - -
20 J48 79,7716 0,892 0,798 0,824 0,829 751 0,36
RF - - - - - - -
MLP - - - - - - -
Wi hou C sSubse E al 3 J48 79,7716 0,892 0,798 0,824 0,829 0,26 0,18
RF 73,8174 0,883 0,738 0,776 0,829 5,97 2,19
MLP 79,7716 0,892 0,798 0,824 0,839 45,32 0,3
GainRa io 10 J48 79,7716 0,892 0,798 0,824 0,829 1,16 0,28
RF 74,8002 0,881 0,748 0,784 0,831 27,07 2,77
MLP - - - - - - -
15 J48 79,7716 0,892 0,789 0,824 0,829 1,46 0,32
RF 74,8818 0,878 0,749 0,784 0,827 29,88 3,37
MLP - - - - - - -
20 J48 79,7716 0,892 0,798 0,824 0,829 1,18 0,31
RF - - - - - - -
MLP - - - - - - -
Wi hou Wi h C sSubse E al 12 J48 69,016 0,784 0,69 0,727 0,565 13,42 0,54
RF 48,6725 0,764 0,487 0,559 0,526 103,73 3,18
MLP - - - - - - -
GainRa io 10 J48 29,5433 0,724 0,295 0,336 0,458 3,86 0,2
RF 20,9407 0,692 0,209 0,193 0,473 44,13 2,56
MLP 85,6695 0,734 0,857 0,791 0,501 160,89 0,45
15 J48 69,016 0,784 0,69 0,727 0,565 14,47 0,25
RF - - - - - - -
MLP - - - - - - -
20 J48 47,6227 0,779 0,476 0,547 0,52 18,13 0,82
RF - - - - - - -
MLP - - - - - - -
Wi hou C sSubse E al 6 J48 85,6695 0,734 0,857 0,791 0,5 1,52 0,17
RF 54,5061 0,759 0,545 0,613 0,518 63,31 2,85
MLP - - - - - - -
GainRa io 10 J48 71,7648 0,779 0,718 0,744 0,561 3 0,32
RF 53,6974 0,76 0,537 0,606 0,519 91,07 8,63
MLP - - - - - - -
15 J48 71,7662 0,779 0,718 0,744 0,563 4,55 0,38
RF - - - - - - -
MLP - - - - - - -
20 J48 71,9978 0,775 0,72 0,744 0,553 2,75 0,42
RF - - - - - - -
MLP - - - - - - -
Algo i hm