Computer Vision system for rock classification using Artificial Neural Network

UNIVERSIDAD DE VALLADOLID ESCUELA DE INGENIERIAS INDUSTRIALES G ado en Ingenie ía Elec ónica Indus ial y Au omá ica Compu e Vision sys em o ock classi ica ion using A i icial Neu al Ne wo k Au o : Be nal Ma ínez, An onio Eusebio de la Fuen e UC Leu en - Limbu g Valladolid, Julio de 2021. TFG REALIZADO EN PROGRAMA DE INTERCAMBIO TÍTULO: Compu e Vision sys em o ock classi ica ion using A i icial ….……. Neu al Ne wo k ALUMNO: An onio Be nal Ma ínez FECHA: 6 de julio de 2021 CENTRO: Facul y o Enginee ing Technology UNIVERSIDAD: UC Leu en - Limbu g TUTOR: Wim Claes UC LEUVEN - LIMBURG BACHELOR THESIS Compu e Vision sys em o ock classi ica ion using A i icial Neu al Ne wo k Au ho : An onio BERNAL Supe iso s: Roel CONINGS Céd ic HAUBEN Wim CLAES A hesis submi ed in ul illmen o he equi emen s o he deg ee o Elec onics - ICT in he ACRO Resea ch G oup Facul y o Enginee ing Technology July 6, 2021 i Decla a ion o Au ho ship I, An onio BERNAL, decla e ha his hesis i led, “Compu e Vision sys em o ock classi ica ion using A i icial Neu al Ne wo k” and he wo k p e- sen ed in i a e my own. I con i m ha : • This wo k was done wholly o mainly while in candida u e o a e- sea ch deg ee a his Uni e si y. • Whe e any pa o his hesis has p e iously been submi ed o a de- g ee o any o he quali ica ion a his Uni e si y o any o he ins i u- ion, his has been clea ly s a ed. • Whe e I ha e consul ed he published wo k o o he s, his is always clea ly a ibu ed. • Whe e I ha e quo ed om he wo k o o he s, he sou ce is always gi en. Wi h he excep ion o such quo a ions, his hesis is en i ely my own wo k. • I ha e acknowledged all main sou ces o help. • Whe e he hesis is based on wo k done by mysel join ly wi h o he s, I ha e made clea exac ly wha was done by o he s and wha I ha e con ibu ed mysel . Signed: Da e: 06/07/2021 ii UC LEUVEN - LIMBURG Abs ac Facul y o Enginee ing Technology Elec onics - ICT Compu e Vision sys em o ock classi ica ion using A i icial Neu al Ne wo k by An onio BERNAL This p ojec has been p oposed by he Nijs company, loca ed in Belgium. This company wo ks wi h na u al s one iles and cobbles ones. One o i s ac i i ies is he pu chase o la ge quan i ies o old oad s ones a e y low p ices, howe e he p oblem is ha hese s ones, come in ba ches wi h di - e en ypes mixed. A he momen , he e a e wo ope a o s in cha ge o he manual so ing, and he objec i es a e o educe so ing ime and cos s. This p ojec explo es he design o a compu e ision sys em ha is able o classi y be ween he di e en ypes o cobbles ones on a con eyo bel and sepa a e hem in o di e en boxes. To ca y ou he ecogni ion pa , a compu e ision sys em wi h a came a and ligh ing inside a black box, was designed. Addi ionally, a py hon sc ip based on he OpenCV lib a y was w i en, o de ec he cobbles one and ack i . A e ha , a A i icial Neu- al Ne wo k (ANN) was ained wi h his og am ea u es such as, weigh ed mean, skewness, and ku osis o classi y he di e en ypes o cobbles ones. Finally, o he sepa a ion pa , a sc ip was made wi h PLC p og amming so ha when he ock passes nex o i s co esponding box, an ac ua o would push i inside. O e all, i is concluded ha due o he high classi ica ion accu acy, he co ec pe o mance o he sys em, and he ex ao dina y p o i abili y, his p ojec ha has been ca ied ou has been a success, he alding he p omising u u e o a i icial neu al ne wo ks o classi ica ion asks. Keywo ds – Neu al Ne wo k, Compu e Vision, Classi ica ion, Cobbles one iii Acknowledgemen s I would like o hank my supe iso s Roel CONINGS, Céd ic HAUBEN and Wim CLAES, o hei guidance h oughou his p ojec . Also hank my amily and iends, o he suppo hey ha e gi en me. i Con en s Decla a ion o Au ho ship i Abs ac ii Acknowledgemen s iii Lis o Figu es i Lis o Tables iii Lis o Abb e ia ions ix 1 In oduc ion 1 1.1 Resea ch G oup and Company . . . . . . . . . . . . . . . . . . 1 1.2 Mo i a ion.............................. 1 1.3 Objec i es .............................. 3 1.4 ThesisOu line............................ 3 2 Backg ound 4 2.1 Compu e Vision .......................... 4 2.1.1 Image Th esholding . . . . . . . . . . . . . . . . . . . . 5 2.1.2 Sobel il e .......................... 7 2.1.3 Canny Edge De ec o . . . . . . . . . . . . . . . . . . . . 7 2.1.4 Mo phological T ans o ma ions . . . . . . . . . . . . . 8 2.2 A i icial Neu al Ne wo k . . . . . . . . . . . . . . . . . . . . . 10 3 Me hodology 12 3.1 O e iew............................... 12 3.2 Rocks................................. 12 3.3 Tools ................................. 14 3.3.1 Ha dwa e .......................... 14 3.3.2 So wa e........................... 17 3.4 Compu e Vision .......................... 17 3.4.1 Objec De ec ion . . . . . . . . . . . . . . . . . . . . . . 18 3.4.2 Objec T acking . . . . . . . . . . . . . . . . . . . . . . . 22 3.5 Classi ica ion............................. 25 3.5.1 Image Acquisi ion . . . . . . . . . . . . . . . . . . . . . 26 3.5.2 Fea u e Ex ac ion . . . . . . . . . . . . . . . . . . . . . 28 3.5.3 Da ase C ea ion . . . . . . . . . . . . . . . . . . . . . . 32 3.5.4 A i icial Neu al Ne wo k . . . . . . . . . . . . . . . . . 34 4 Resul s and Implemen a ion 37 4.1 Classi ica ion Accu acy . . . . . . . . . . . . . . . . . . . . . . . 37 4.2 Implemen a ion o he Sys em . . . . . . . . . . . . . . . . . . . 38 5 Conclusion 41 5.1 Budge ................................ 41 5.2 P o i abili yS udy.......................... 43 5.3 Summa y............................... 44 5.4 Fu u eWo k............................. 45 A Addi ional Da a 46 Bibliog aphy 52 i Lis o Figu es 1.1 Nijs Na uu s een company . . . . . . . . . . . . . . . . . . . . 1 1.2 Ou line o he 2020 p ojec . . . . . . . . . . . . . . . . . . . . . 2 1.3 Typeso ock aces ......................... 2 2.1 Digi iza ion p ocess . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.2 RGB colo model channels . . . . . . . . . . . . . . . . . . . . . 5 2.3 O su’s h esholding......................... 6 2.4 Global and adap i e h esholding . . . . . . . . . . . . . . . . . 6 2.5 Sobel il e .............................. 7 2.6 Cannyedgede ec o ........................ 8 2.7 E osion................................ 8 2.8 Dila ion................................ 9 2.9 Opening ............................... 9 2.10Closing................................ 9 2.11 Neu on o a neu al ne wo k . . . . . . . . . . . . . . . . . . . . 10 2.12 Fully connec ed neu al ne wo k . . . . . . . . . . . . . . . . . . 11 3.1 Cobbles one ypes.......................... 13 3.2 Cobbles oneshapes......................... 13 3.3 Opaque box and con eyo bel se up . . . . . . . . . . . . . . . 14 3.4 Con eyo bel ............................ 15 3.5 Ligh ing ............................... 16 3.6 Came a................................ 16 3.7 Colo image ............................. 18 3.8 G ayscaleImage .......................... 19 3.9 Edgesde ec ed............................ 19 3.10Imageclosed............................. 20 3.11Imageopened............................ 20 3.12De ec ed ock ............................ 21 3.13Ro a ed ock............................. 21 3.14Reduced ock ............................ 22 3.15 Objec acking example . . . . . . . . . . . . . . . . . . . . . . 24 Chap e 1. In oduc ion 3 1.3 Objec i es The main objec i e o his p ojec is o design a sys em ha au oma ically classi ies be ween 3 ypes o ock, om he isible ace o he ocks, he e o e, an ope a o would ha e o manually place he ock so ha a came a cap u e he ock isible ace. In addi ion, some seconda y objec i es we e es ablished, wi hou which he p ojec would be a success, bu i achie ed, would p o ide g ea added alue. These objec i es a e: • Classi y ega dless o he ace o he ock ha he came a cap u es. In his way, cos s would be u he educed, since he ope a o , in cha ge o placing he ock wi h i s isible ace, would be sa ed. • Ob ain he measu emen s o he ocks, o po en ially eplace he p ojec o classi ica ion by size, and ha e bo h classi ica ions (by size and ype) oge he in he same sys em. 1.4 Thesis Ou line Chap e 1 p o ides an o e iew o he mo i a ion and goals o he p ojec . Chap e 2 p o ides he heo e ical backg ound on which he p ojec is based: compu e ision and a i icial neu al ne wo ks. Chap e 3 desc ibes he me h- ods ha ha e been used o ca y ou he p ojec . Chap e 4 e alua es he e- sul s ob ained and p esen s he global implemen a ion o he sys em. Chap- e 5 summa izes he indings o he hesis and p oposes pa hs o u u e esea ch. 4 Chap e 2 Backg ound This chap e 2, aims o p o ide a be e unde s anding o he undamen al concep s necessa y o unde s and he hesis. This p ojec is based on wo pilla s: compu e ision (2.1) and a i icial neu al ne wo ks (ANN) (2.2). 2.1 Compu e Vision Compu e ision is he se o echniques ha deal wi h ex ac ing ele an in- o ma ion om he physical wo ld om images and ideo, using a compu e sys em. An image is a isual ep esen a ion o a scene. And o i o be p ocessed by a compu e , i needs o be ans o med in o a digi al image. This ans o - ma ion is called digi iza ion (FIGURE 2.1), and oughly speaking, i consis s o wo pa s: sampling and quan i ica ion. Sampling (FIGURE 2.1b) consis s o di iding he image in o small samples o segmen s, and quan i ica ion (FIGURE 2.1c) consis s o assigning a nume ical alue o each o hese sam- ples. (A) Analog image. (B) Sampling. (C) Quan i ica ion. FIGURE 2.1: Digi iza ion p ocess. A digi al image is ep esen ed by a ma ix. Each elemen o he ma ix is called a pic u e elemen (pixel) and has a disc e e alue, called a g ay le el, Chap e 2. Backg ound 5 which ep esen s i s b igh ness o ligh ness. The e a e 256 g ay le els, ang- ing om 0 (black) o 255 (whi e). Real-wo ld images a e in colo , and in hem each pixel has 3 di e en g ay le els (FIGURE 2.2), co esponding o he 3 channels o he RGB colo model ( ed, g een and blue). Tha is, he alue o each pixel can ha e 256 di e en le els o ed, 256 o g een and 256 o blue, and combining hem app oxi- ma ely 16 million o di e en colo s a e ob ained. (A) Red. (B) G een. (C) Blue. (D) Colo . FIGURE 2.2: RGB colo model channels. Sou ce: [1]. In o de o ex ac in o ma ion om he images, i is necessa y o apply image p ocessing echniques, explained in he ollowing sec ions (2.1.1, 2.1.2, 2.1.3 and 2.1.4). 2.1.1 Image Th esholding The simples me hod o image segmen a ion1is h esholding. F om a g ay scale image, bina y images a e c ea ed by applying his segmen a ion me hod [2]. The e a e se e al ypes o h esholding, including global h esholding, O su’s h esholding and adap i e h esholding. 1Image segmen a ion is he p ocess o di iding a digi al image in o se e al segmen s o se s o pixels. The goal is o simpli y he image, so ha i is easie o analyze [2]. Chap e 2. Backg ound 6 In global h esholding (FIGURE 2.4b), o each pixel, he same h eshold alue is applied. I he pixel alue is less han he h eshold, his alue is se o 0; whe eas, i i is g ea e han he h eshold, i is se o 1. O su’s h esholding (FIGURE 2.3), is used o au oma ic h esholding o an image. The algo i hm e u ns a single h eshold ha sepa a es he image pix- els in o wo classes. This h eshold is calcula ed by minimizing he wi hin- class in ensi y a ia ion o , in o he wo ds, by maximizing he be ween-class a ia ion [3]. (A) G ay scale image. (B) G ay scale his og am. (C) Algo i hm ou pu . FIGURE 2.3: O su’s h esholding. The h eshold is he ed line in FIGURE 2.3b. Finally, he adap i e h esholding me hod (FIGURE 2.4c), is used when he image has di e en illumina ion le els. The algo i hm de e mines he h esh- old o a pixel based on a small egion a ound i . The e o e, di e en h esh- olds a e ob ained o di e en egions o he same image [4]. (A) G ay scale image. (B) Global h esholding. (C) Adap i e h esholding. FIGURE 2.4: Compa ison be ween global and adap i e h esh- olding. Sou ce: [4]. Chap e 2. Backg ound 7 2.1.2 Sobel il e The Sobel il e o Sobel ope a o (FIGURE 2.5), used in compu e ision o edge de ec ion, is a disc e e di e en ial ope a o ha calcula es he g adien 2 o he in ensi y o an image, o each pixel. Tha is, o each pixel, i will gi e us in o ma ion abou he la ges possible change (co esponding o he edges), and i s di ec ion and sense. In addi ion, i can speci y he di ec ion o he de i a i es, e ical (FIGURE 2.5b) o ho izon al (FIGURE 2.5c) [6]. (A) G ay scale image. (B) Ve ical Sobel il e . (C) Ho izon al Sobel il e . FIGURE 2.5: Sobel il e . Sou ce: [7]. 2.1.3 Canny Edge De ec o The Canny edge de ec o (FIGURE 2.6) is, a mul i-s age edge de ec ion algo- i hm, capable o de ec ing a wide ange o edges [8]. I is di ided in o 5 s ages: • Fi s , a Gaussian il e 3is applied o emo e noise and smoo h he im- age. • Second, he in ensi y g adien s o he image a e calcula ed. • Thi d, apply lowe bound cu -o supp ession o g adien magni ude h esholding, o ge id o spu ious esponse o edge de ec ion. • Fou h, apply double h esholding o de ec possible edges. • Fi h, apply hys e esis o h esholding. Supp essing he emaining weak edges and hose no connec ed o s ong edges. 2The g adien is a ec o , which indica es he di ec ion in which a scala a ies mo e apidly [5]. 3This il e uses a Gaussian unc ion o blu an image. Chap e 2. Backg ound 8 (A) G ay scale image. (B) Low h esholds. (C) High h esholds. FIGURE 2.6: Canny edge de ec o . 2.1.4 Mo phological T ans o ma ions Mo phological ans o ma ions a e simple ope a ions based on he shape o he image. A s uc u ing elemen o ke nel4decides he na u e o he op- e a ion. The wo basic mo phological ope a ions a e: e osion and dila ion [9]. The e osion (FIGURE 2.7) aims o educe he size o an objec . The ope a ion is as ollows: he ke nel will go h ough all he pixels in he image, when all he pixels on he ke nel ha e a alue o 1 (whi e pixel), a 1 will be assigned o ha pixel. O he wise, i any o he pixels on he ke nel is 0 (black pixel), he pixel will be assigned a 0 [9]. (A) O iginal image. (B) E oded image. FIGURE 2.7: E osion. Sou ce: [9]. The dila ion (FIGURE 2.8), con a y o e osion, se es o inc ease he size o an objec . As in e osion, he ke nel will go h ough all he pixels in he image, when all he pixels on he ke nel ha e a alue o 0, a 0 will be assigned. O he wise, i any o he pixels on he ke nel is a 1, will assign a 1 [9]. Once he wo basic mo phological ope a ions (e osion and dila ion) ha e been de ined, i is now possible o de ine wo mo e use ul and complex ope - a ions: opening and closing. Opening (e osion + dila ion) is used o emo e 4The s uc u ing elemen o ke nel can ha e di e en shapes ( ec angle, ci cle...). Chap e 2. Backg ound 9 (A) O iginal image. (B) Dila ed image. FIGURE 2.8: Dila ion. Sou ce: [9]. noise om he image (FIGURE 2.9). Closing (dila ion + e osion) is use ul o illing small holes ha an objec may ha e (FIGURE 2.10) [9]. (A) O iginal image. (B) Opened image. FIGURE 2.9: Opening. Sou ce: [9]. (A) O iginal image. (B) Closed image. FIGURE 2.10: Closing. Sou ce: [9]. Chap e 2. Backg ound 10 2.2 A i icial Neu al Ne wo k In ecen yea s, he esu gence o he neu al ne wo k has e olu ionized ields such as speech ecogni ion, image ecogni ion, na u al language p ocessing... A neu al ne wo k, called a i icial neu al ne wo k (ANN) o mul i-laye pe - cep on (MLP) , is a supe ised5machine lea ning model ha has he abili y o ep esen complex nonlinea ela ionships in he inpu da a. The model was o iginally de eloped o copy he biological b ain, bu has since di e ged o op imize classi ica ion and eg ession asks6. The basic uni o he neu al ne wo k is he neu on (FIGURE 2.11), which akes a weigh ed a e age o he inpu alues and hen applies an nonlinea ac i a- ion unc ion7which e u ns a scala alue. X1 X2 X3 W1 W2 W3 ∑σY Inpu s Weigh s Sum Ac i a ion Func ion Ou pu FIGURE 2.11: Neu on o a neu al ne wo k. Neu al ne wo ks a e o en made up o laye s, which in u n a e made up o neu ons. These laye s, called hidden laye s, a e hose be ween he inpu and ou pu laye s. When all neu ons in a laye a e connec ed o all neu ons in he p e ious laye , he laye is called dense o ully connec ed (FIGURE 2.12). Neu al ne wo ks wi h a leas one hidden laye and wi h a non-linea ac i a ion unc ion a e uni e sal app oxima o s, which means ha hey can, a p io i, app oxima e any con inuous unc ion o a bi a y complexi y. In 5Supe ised lea ning (SL) is he machine lea ning ask o lea ning a unc ion ha maps an inpu o an ou pu based on example inpu -ou pu pai s [10]. 6Analysis used o p edic he ou come o a ca ego ical a iable, based on he p edic o a iables [11]. 7The ac i a ion unc ion o a neu on, de ines he ou pu o ha node gi en an inpu o se o inpu s. The mos common ac i a ion unc ion is he sigmoid, de ined as: (x) = 1 1+ex, which e u ns an ou pu alue be ween 0 and 1 [12]. Chap e 2. Backg ound 11 p ac ice, mo e laye s o en p o ide mo e lexibili y and powe o ep esen a- ion, allowing he ne wo k o ep esen inc easingly abs ac ela ionships in he da a. Inpu laye Hidden 1 laye Hidden 2 laye Oupu laye FIGURE 2.12: Fully connec ed neu al ne wo k wi h wo hidden laye s. Fo a neu al ne wo k o lea n, he de elope mus de ine a loss unc ion8 o e alua e he pe o mance o he p edic ions o he ne wo k. Du ing aining, he neu al ne wo k i e a i ely sea ches o a se o weigh s ha minimize he loss unc ion h ough a g adien -based op imiza ion algo i hm, such as g adien descen 9. A p ocess called back-p opaga ion10 is used o de e mine he con ibu ion o he e o o each weigh and he e o e how much you mus change each weigh in each i e a ion o he op imiza ion. 8Loss unc ions a e used o de e mine he e o be ween he ou pu o he model and he a ge alue [13]. 9G adien descen is an i e a i e op imiza ion algo i hm o inding a local minimum o a unc ion [14]. 10Back-p opaga ion (backwa d p opaga ion o e o s) is he mos used me hod o calcu- la ing de i a i es in a ANN [15]. 12 Chap e 3 Me hodology 3.1 O e iew The ocks a e going o pass h ough a con eyo bel ha is loca ed ou doo s, so he mos iable and cheapes possible op ion is o ins all a came a a he beginning o he con eyo bel o ob ain an image o he ock. Wi h his image, he ype o ock will be de e mined and a signal will be sen o a PLC o ac i a e an ac ua o , in cha ge o pushing he ock o i s co esponding box. The con eyo bel is made o black ubbe , which acili a es he wo k o de- ec ing he ock. I a me al bel was used, he e would be a lo o e lec ions due o ligh ing o ambien ligh , and i would be mo e di icul o classi y he ocks. The came a and ligh ing will be pu inside an opaque box o always ha e he same illumina ion. I his was no done, as he ambien ligh a ies h ough- ou he day, so ing p oblems would be p esen ed. In he opaque box, he e will be wo openings a he base, one o en y and one o exi , h ough which he ocks will pass. 3.2 Rocks The ypes o cobbles ones which will be classi ied a e 3: Belgian g es, Belgian po phy y and Belgian blue s one. F om now on, o simplici y, hey will be e e ed o as: g es, po phy y and blue, espec i ely. • G es cobbles ones (FIGURE 3.1a), ha e a smoo h and unspeckled ace, and can ha e any plain colo wi hin he whole ange. Chap e 3. Me hodology 19 FIGURE 3.8: G ay scale Image. de ec s ains on he con eyo bel as ocks. The e o e, hese h ee me hods we e disca ded. Finally, he Canny edge de ec o (FIGURE 3.9) was chosen, as i was no ably mo e obus han he Sobel il e , o his pa icula case. FIGURE 3.9: Edges de ec ed by he Canny edge de ec o . Once we ha e he edges o he ock as whi e pixels, we need he in e io o he ock o be whi e as well. As ocks ha e a elie and a oughness, he Canny algo i hm in he chosen con igu a ion also de ec s inne pa s o he ock. This is no a p oblem, on he con a y, because i will help us o ill he in e io o he ock. In o de o ca y ou his ask, a closing and a subsequen opening o he image (explained in sec ion 2.1.4) is pe o med, wi h ke nels wi h geome ic shapes and sizes chosen by ial and e o . The pu pose o he closing is Chap e 3. Me hodology 20 o ill he gaps inside he ock (FIGURE 3.10). And he pu pose o opening, is o emo e he s ains on he con eyo bel ha may ha e been de ec ed, howe e , la ge s ains a e s ill going o be de ec ed (FIGURE 3.11). FIGURE 3.10: Image closed. FIGURE 3.11: Image opened. Once we ha e he image wi h he pa o he ock in whi e and he con eyo bel in black, excep o some spo s, i is ime o apply an algo i hm o ind he con ou s12. This algo i hm will p o ide us wi h a lis o he objec s i de ec s in he image, as well as a se ies o e y use ul cha ac e is ics o hese (size, posi ion, cen oid13...). 12The cu e joining all he con inuous poin s (along he objec bounda y), ha ing same in ensi y. The con ou s a e a use ul ool o objec de ec ion and shape analysis [24]. 13The cen oid o geome ic cen e o a plane igu e, is he a i hme ic mean posi ion o all he poin s in he igu e [25]. Chap e 3. Me hodology 21 To comple ely elimina e he p oblem o s ains on he ape, de ec ed as ocks, all objec s whose a ea is less han 10,000 pixels a e elimina ed, since he e will be no ocks wi h an a ea smalle han his. Now wi h only he ocks in he image, you ge he bounding boxes o hese. The e is a p oblem, and i is ha he ocks may no be aligned wi h he ame ha he came a cap u es, and i needs o be aligned o ob ain he image o he ock and o be able o classi y i . So, o sol e i , he minimum a ea bounding box is ob ained (FIGURE 3.12) and wi h his, applying a pe spec i e ans o - ma ion echnique [26], he aligned ock is ob ained (FIGURE 3.13). FIGURE 3.12: De ec ed ock. FIGURE 3.13: Ro a ed ock. Finally, he image is educed by 20% in bo h wid h and heigh , o make su e ha only wha appea s in he image belongs o he ock (FIGURE 3.14). The main eason o his s ep is ha , al hough mos ocks a e squa e o ec angu- la , he e a e some ocks ha a e no (FIGURE 3.2c) and i his was no done, he cap u ed image (which is necessa ily squa e o ec angula ) would show Chap e 3. Me hodology 22 a pa o he con eyo bel , which would al e he cha ac e is ics o he ock a he ime o classi ica ion. FIGURE 3.14: Rock educed in size. 3.4.2 Objec T acking So a , he ideo ames ha e been ea ed independen ly. Bu i is necessa y o gi e a global app oach o he se o ames ha he came a will cap u e, because he same ock will appea in a la ge numbe o consecu i e ames and only equi es classi ica ion once. Classi ying each ock only once, in one o he ames, will help g ea ly educe he execu ion ime o he algo i hm. I should also be aken in o accoun ha wo o mo e ocks can appea in he same ame a he same ime, i hey ha e been loaded on he con eyo bel e y close oge he . The e o e, he p og am has o be able o keep ack o mul iple objec s a he same ime. Objec acking is a e y use ul echnique in ideo p ocessing, which is used o iden i y he ajec o ies o objec s in a ideo. The e a e many objec ack- ing echniques, bu we will use one based on he Euclidean dis ance [27], which is he dis ance be ween wo poin s in a Euclidean space, which is deduced by applying he Py hago ean heo em14. Fo he wo-dimensional space in which wo k is done, he dis ance be ween wo poin s P1and P2, o Ca esian coo dina es (x1,y1) and (x2,y2) is calcula ed wi h he EQUATION 3.1. dE(P1,P2) = q(x2−x1)2+ (y2−y1)2(3.1) The inpu o his objec acking algo i hm will be he coo dina es o he cen oids o he de ec ed objec s in each ame. Then, om one ame o he nex , he dis ances be ween he cen oids o he i s ame wi h espec o he 14I s a es ha he a ea o he squa e whose side is he hypo enuse, is equal o he sum o he a eas o he squa es on he o he wo sides ( x2+y2=h2) [28]. Chap e 3. Me hodology 23 second will be calcula ed. I his dis ance is less han 25 pixels, i will be he same objec , o he wise i will be a new objec and i will be assigned a new iden i ie numbe [29]. An example o how objec acking wo ks is shown in he FIGURE 3.15. Each ock will en e he opaque box h ough he opening and exi a he op- posi e side. This means ha when he ock is en e ing he came a will only cap u e a pa o he whole ock, and he same happens when i lea es he box. The e o e he ins an , a which he image o each ock will be cap u ed o classi ica ion, will be he one a which he cen oid o he ock is in he cen e o he ame. Bu i canno assume ha all he cen oids o he ocks will be ound in one o he ames in which hey appea , in he middle pixel15. The e o e, i is necessa y o de e mine a ange o middle pixels in which i is ce ain ha he cen oid will appea . To calcula e his middle ange, i is necessa y o know he numbe o pixels ha a cen oid mo es om one ame o he nex . To do his, s a ing om da a such as he ame a e (30 ames second ), he con eyo bel speed (0.4 me e s second) and he esolu ion (0.64 millime e s pixel = 0.00064 me e s pixel ), i s he ame a e and eso- lu ion a e in e ed (EQUATION 3.2), o acili a e uni con e sion, and inally we ope a e (EQUATION 3.3). 1 F ameRa e ∗Speed ∗1 Resolu ion (3.2) 1  second 30 ames ∗0.4  me e s 1  second ∗1pixel 0.00064  me e s =20.83 pixels ame ≈21 pixels ame (3.3) The cen oid o a ock will a el 21 pixels, om one ame o he one im- media ely ollowing i . So wi h his in mind, an in e media e s ipe (FIGURE 3.16) is se in he ame o 64 pixels wide, so ha a leas each cen oid is de ec ed 3 imes. I he ock was no cap u ed when i is in he middle s ipe o he ame, he e would be a high p obabili y ha i would no appea in i s o ali y (FIGURE 3.17), which would gi e p oblems o classi ica ion since p obably, all he cha ac e is ics o he ocks would no be aken in o accoun . 15The middle pixel, o he wid h o he ame (752 pixels) would be hal , i.e., pixel 376. Chap e 3. Me hodology 24 (A) Fi s ame. (B) Second ame. (C) Thi d ame. FIGURE 3.15: Objec acking example, in 3 ames sepa a ed by 2 seconds. Chap e 3. Me hodology 25 FIGURE 3.16: In e media e ame s ipe (space be ween he g een lines), in which he ock classi ica ion will be pe o med. (A) To al cap u e o he ock. (B) Pa ial cap u e o he ock. FIGURE 3.17: Compa ison be ween cap u ing he en i e ock in he cen e o he ame, and pa ially cap u ing he ock on he igh o he ame . 3.5 Classi ica ion Once he ock is loca ed on he con eyo bel , i has o be classi ied. The in- o ma ion a ailable o classi ica ion is he image ob ained a e applying he me hods desc ibed in sec ion 3.4. This image is in colo , and i s dimensions a e a iable depending on he size o he ock, being he smalles ones in he o de o 50x50 pixels, up o 260x260 he la ges ones. Chap e 3. Me hodology 26 Va ious classi ica ion me hods we e explo ed, s a ing om he simples , o he mos complex and no el. The o me we e simple classi ica ion me hods based on a e age colo , bo h in he RGB and hue, sa u a ion, alue (HSV) colo models. Then, an unsupe ised lea ning me hod16 was es ed, wi h he K-Means algo i hm17, bu i was e y slow. The p edic i e powe o hese wo me hods was e y weak, being below 50 %, he e o e, i was decided o use he ANN o classi ica ion. 3.5.1 Image Acquisi ion The i s s ep o aining an ANN is o ha e a su icien ly la ge da abase, so ha i can lea n o classi y ocks. Since no da abase exis ed, i had o be c ea ed. To c ea e i , images o he ocks had o be aken in he same ligh ing con- di ions ha would be used when he sys em was pu in o p oduc ion, o h- e wise he model would no wo k co ec ly. The opaque box, came a and ligh ing men ioned in he sec ion 3.3.1 we e used. I is wo h men ioning ha he ligh ing was kep cons an while all images we e cap u ed. To speed up and acili a e he p ocess o cap u ing images o he ocks, a p og am was c ea ed, adding a small modi ica ion o he one al eady c ea ed o he ock de ec ion pa (3.4.1). The inpu o his p og am is a ideo, which is shown on he sc een when i is execu ed. The ope a ion is as ollows, he use has o p ess he skey, so ha an image is au oma ically sa ed when a ock appea s in he ideo. These images a e sa ed wi h numbe names (1, 2, 3...) and wi h po able ne wo k g aphics (PNG) ex ension. In addi ion, ano he modi ica ion was also added, which is ha i he use p essed he lkey, images a e sa ed wi h numbe names, bu also including he p e ix S (S1, S2, S3...). This las modi ica ion, was in oduced o be able o dis inguish be ween he isible ace o he ocks (sa ed by p essing s) and he es o he aces (sa ed by p essing l), in o de o s udy he possibili y o ul illing he seconda y objec i e (sec ion 1.3), which consis s in classi ying ega dless o he ock ace cap u ed by he came a. 16Unsupe ised lea ning (UL) is a ype o algo i hm ha lea ns pa e ns om un agged da a [30]. 17K-means clus e ing is a me hod o classi ica ion, ha aims o pa i ion nobse a ions in o kclus e s in which each obse a ion belongs o he clus e wi h he nea es mean (clus e cen e s o clus e cen oid) [31]. Chap e 3. Me hodology 27 In o de o make he ideos ha he p e ious p og am wo ks wi h, he whole equipmen had o be mo ed o he Nijs company o one day (FIGURE 3.18), since he e was a la ge amoun o ocks he e. Wi h he help o an expe wo ke , ep esen a i e samples o he 3 ypes o ocks a ying in size, colo , shape and ex u e we e chosen. Videos o app oxima ely 135 ocks we e cap- u ed. A o al o 3 ideos we e eco ded, one o each ock ype. FIGURE 3.18: Nijs equipmen se up o da abase c ea ion, in- cluding opaque box, ligh ing, came a, powe supply and com- pu e . Be o e being eco ded, he ocks we e washed wi h p essu ized wa e and d ied (FIGURE 3.19), as his same p ocedu e will be ca ied ou wi h he sys- em in p oduc ion. Washing he ocks helps o elimina e dus and di ha hey may ha e, in o de o cap u e hei eal cha ac e is ics wi h he came a, elimina ing he noise gene a ed by he di ; and d ying hem helps o a oid e lec ions c ea ed by he wa e ha may ha e emained on hei su ace. The ideos consis o a pe son who pu s a ock in he opaque box, akes his hands ou (so ha hese do no cause in e e ence wi h he ock de ec ion algo i hm (3.4.1)), wai s 1 second, and pu s his hands in again o o a e each ock 5 imes, so ha he 6 aces o he ock appea in he ideo. And his same p ocedu e was epea ed wi h all he ocks. A e p ocessing hese ideos wi h he p og am, 796 images o ocks we e ob ained, wi h he ollowing composi ion: 188 o blue, 375 o g es and 233 o po phy y. Mo e ocks o he g es ype we e eco ded, since i was he ype wi h he mos a ia ions o colo s and ex u es. Chap e 3. Me hodology 28 FIGURE 3.19: Rock washing and d ying p ocess. 3.5.2 Fea u e Ex ac ion Once he da abase was buil , wo ways o sol ing he classi ica ion p oblem we e s udied: Con olu ional Neu al Ne wo k (CNN) and ANN. CNN’s a e a special ype o ANN, designed o ecognize isual pa e ns di ec ly om he pixels o an image wi h almos no p ep ocessing, while ANN’s a e he basic a chi ec u e o neu al ne wo ks. In o de no o inc ease he compu a ional load o he p og am oo much, i was decided o use an ANN, since he inpu s o a CNN a e all he pixels o an image (on a e age he ock images we e 150x150, which means 22500 inpu s), while he inpu s o an ANN will be an in ini ely smalle numbe , depending on he ea u es we wan o use. The e a e many ea u es ha can be ob ained om an image: shapes, colo , ex u es. . . Howe e , no all o hese ea u es p o ide ele an in o ma ion o he classi ica ion. The e o e, i is impo an o p ope ly choose a limi ed numbe o cha ac e is ics, which help us o classi y. I was decided o choose ea u es o he colo his og ams18 o he image, because i has been shown ha aining an ANN wi h hose ea u es, p o ides be e classi ica ion accu- acy [32]. 18The his og am o an image, is a g aphic ep esen a ion o he dis ibu ion o he di e en ones o an image. Chap e 3. Me hodology 35 The inpu laye will ha e 9 nodes, one pe ea u e, and he inal laye has 3 nodes, one o each ock class. In he FIGURE 3.24 shows a g ouped ep esen a ion o he ne wo k, while in he FIGURE 3.25 shows he ac ual s uc u e o he ne wo k, wi h all o i s nodes. . . .. . .. . .. . . WR WG WB SR SG SB kR kG kB H11 H12 H164 H21 H22 H232 H31 H32 H316 H41 H42 H48 D O1 O2 O3 Inpu laye Hidden1 laye Hidden2 laye Hidden3 laye Hidden4 laye D opou laye Ou pu laye FIGURE 3.24: G ouped ep esen a ion o he neu al ne wo k. Once he model was buil , he compila ion pa ame e s we e chosen: • Adam, as an op imiza ion algo i hm (explained in sec ion 2.2). • Spa se ca ego ical c oss en opy, as a loss unc ion (explained in sec ion 2.2). Chap e 3. Me hodology 36 FIGURE 3.25: Real ep esen a ion o he neu al ne wo k. • 300 epochs27. Once he ANN was ained, i was sa ed in he H5 o ma 28. I should be men ioned ha he ne wo k had 3.411 hype pa ame e s. 27An epoch is one comple e cycle h ough he ull aining da ase . 28Hie a chical da a o ma 5 (HDF5) (.h5) is a ile o ma o s o ing s uc u ed da a, i is no a model by i sel . Ke as sa es he models in his o ma as i can easily s o e he weigh s and model con igu a ion in a single ile [38]. 37 Chap e 4 Resul s and Implemen a ion 4.1 Classi ica ion Accu acy The inal ained model ob ained an accu acy o 95% in he es 1. In he FIGURE 4.1, he con usion ma ix2o he esul s o he es da a can be seen. FIGURE 4.1: Con usion ma ix o he es da ase . Since he numbe o images o each ype o ock in he es sample is no he same, he con usion ma ix is no malized and exp essed as a pe cen age. As we obse e in he FIGURE 4.2, he ANN classi ies wi h 100% accu acy he blue ype ocks, wi h 93% accu acy he g es ype ocks, and wi h 94% accu acy he po phy y ype ocks. 1Wi h ock images om a da ase wi h which he ANN has no been ained. 2Tool o isualize he pe o mance o an algo i hm used in supe ised lea ning. Chap e 4. Resul s and Implemen a ion 38 FIGURE 4.2: No malized con usion ma ix o he es da ase (exp essed in pe cen ages). 4.2 Implemen a ion o he Sys em Once all he s ages in which he p ojec is di ided (objec de ec ion and ack- ing (3.4), and classi ica ion (3.5)) wo k sepa a ely, i is ime o pu hem all oge he in he same p og am. In his way, he ideo cap u ed by he came a will be he inpu o he p o- g am and in i , he ocks will appea on he con eyo bel . These ocks will be de ec ed, acked and, when hei cen oid is wi hin he in e media e g een s ipe, hey will be classi ied. As can be seen in he FIGURE 4.3, in he up- pe le co ne o he ame, he ANN p edic ion will appea , as well as he p obabili y (in %) ha his p edic ion is co ec . In he APPENDIX A, he e a e examples o global implemen a ion o he sys- em o ocks o g es ype (FIGURE A.1) and po phy y (FIGURE A.2). In addi ion o he o e all sys em wo king co ec ly, as he ANN was ained wi h images o he 6 aces o each ock, he seconda y objec i e o classi ying ega dless o he ock ace cap u ed by he came a (1.3) has also been ul illed. As can be seen in he FIGURE 4.4, he sys em ecognizes a ock o ype blue, ega dless o he ace shown. In he APPENDIX A, examples o classi ica ion ega dless o he ace a e ound o ocks o ype g es (FIGURE A.3) and po phy y (FIGURE A.4). Chap e 4. Resul s and Implemen a ion 39 (A) Fi s ame. (B) Second ame. FIGURE 4.3: Example o global implemen a ion o he sys em o a ock o he blue ype, in 2 consecu i e ames sepa a ed by 2 seconds. Chap e 4. Resul s and Implemen a ion 40 (A) Face 1. (B) Face 2. (C) Face 3. (D) Face 4. FIGURE 4.4: Examples o classi ica ion ega dless o he ace cap u ed by he came a o a ock o ype blue. As can be seen, he ANN p edic ion is always blue, bu he p obabili y (in %) ha his p edic ion is co ec may a y, since di e en aces a e p esen . 41 Chap e 5 Conclusion 5.1 Budge In his sec ion, an es ima ion o he cos o his p ojec is made based on he alue o he so wa e de elopmen and he p ice o he ma e ial means neces- sa y o implemen he de ice ha will pe o m he classi ica ion o he ocks au oma ically. Fo his pu pose, i is de e mined ha he basic human eam o he de el- opmen o he so wa e mus be composed o an a i icial in elligence (AI) de elope and a p og amme , who will assis him in he c ea ion o he p o- g am. In addi ion, i is conside ed ha he ad ice o a senio enginee expe in PLC p og amming is necessa y, who will p o ide echnical suppo and supe ise he execu ion. The cos o he human esou ces, based on a e age sala ies, is calcula ed in TABLE 5.1. €/ Mon h €/ Hou Junio AI de elope 4.260 €26,63 € Junio p og amme 3.180 €19,88 € Senio PLC p og amme 5.748 €35,93 € TABLE 5.1: Sala ies. The sou ces om which he da a in he i s column (€/ Mon h) ha e been ob ained a e as ollows: ju- nio AI de elope [39], junio p og amme [40], and senio PLC p og amme [41]. As o he equipmen and ma e ials necessa y o he de elopmen o he de ice, hei cos is es ima ed in TABLE 5.2. Finally, he TABLE 5.3 b eaks down he p ojec in o h ee phases o s ages o a i e a he o al budge , which amoun s o 19,914.25 €. Chap e 5. Conclusion 42 Cos Ring ligh (LDR2-90SW2) 2.756 € Powe supply (PD2-3024-2) 110 € Came a (UI-1225-LE-C) 165 € Lens (LM8JC1MS) 207 € TABLE 5.2: Ma e ial equipmen . P ices ha e been ob ained om he ollowing sou ces: ing ligh [42], powe supply [43], came a [44], and lens [45]. Hou s Cos In o ma ion ga he ing and p elimina y s udies Junio AI de elope 150 3.993,75 € Junio p og amme 100 1987,50 € Senio PLC p og amme 20 718,50 € Technical de elopmen and code gene a ion Junio AI de elope 120 3.195,00 € Junio p og amme 170 3.378,75 € Senio PLC p og amme 25 898,13 € Tes pe o mance Junio AI de elope 50 1.331,25 € Junio p og amme 50 993,75 € Senio PLC p og amme 5 179,63 € Ma e ial cos s 3.238,00 € To al budge 19.914,25 € TABLE 5.3: S ages o budge . Fo each o he 3 s ages, junio AI de elope , junio p og amme and senio PLC p og amme hou s we e equi ed. The cos o each o hese is ob ained by mul iplying he hou s by he €/Hou o he TABLE 5.1. Finally, he o al budge is ob ained by adding he ma e ial cos (TABLE 5.2). Chap e 5. Conclusion 43 5.2 P o i abili y S udy The p o i abili y s udy aims o demons a e he economic in e es , o he Ni- js company, o acing he cos o he de elopmen o his p ojec . I will be ob ained by es ima ing he di e ence be ween he cos o he manual so ing ac i i y wi h wo ope a o s (cu en ly pe o med) and he au oma ic classi i- ca ion p oposed in his p ojec . The o al cos budge ed in sec ion 5.1, amoun ing o 19,914.25 €, is consid- e ed as he ini ial in es men . The sa ings gene a ed annually by a oiding he labo cos s o 2 ope a o s a e aken as sa ings. Each ope a o , has a emu- ne a ion o 40 eu os hou and wo ks 8 hou s day , 200 days yea 1. The e o e, he o al annual sa ings a e ob ained wi h he EQUATION 5.1. 2  ope a o s ∗40eu os   hou ∗  ope a o ∗8  hou s   day ∗200  days yea =128.000eu os yea (5.1) To simpli y he p o i abili y calcula ion, i is conside ed ha he use ul li e o his p ojec is 5 yea s, and ha he esidual alue a he end o hese yea s is ze o, assuming ha his sys em could become obsole e a e his pe iod, due o possible echnical ad ances in his ield. The TABLE 5.4, will be used o calcula e indica o s o he p o i abili y o he in es men . In es men (I) Sa ings Balance (V) Yea 0 - 19.914,25 €- 19.914,25 € Yea 1 128.000,00 €128.000,00 € Yea 2 128.000,00 €128.000,00 € Yea 3 128.000,00 €128.000,00 € Yea 4 128.000,00 €128.000,00 € Yea 5 128.000,00 €128.000,00 € To al - 19.914,25 €640.000,00 €620.085,75 € TABLE 5.4: Table o calcula ing p o i abili y indica o s. E olu- ion o sa ings and o al balance in he i s 5 yea s. 1Da a p o ided by he company Nijs Chap e 5. Conclusion 44 A e 5 yea s and a annual discoun a e (k) o 10 %, he ne p esen alue (NPV), co esponding o he alue o he ne cash lows (income - expenses) o igina ed by he ini ial in es men in es men , will be 423.005,87 €( EQUA- TION 5.2 ). NPV = n ∑ =1 V (1+k) −I0(5.2) The in e nal a e o e u n (IRR) which is a me hod o calcula ing an in es - men ’s a e o e u n, will be 642,73 % ( EQUATION 5.3 ). NPV = n ∑ =0 V (1+IRR) =0 (5.3) The p o i abili y index (PI), a a io o payo o in es men o a p ojec , used o quan i y he amoun o alue c ea ed pe uni o in es men , is 22,24 ( EQUATION 5.4 ). PI =1+NPV |I0|(5.4) And inally, he p ojec payback will be less han 2 mon hs (EQUATION 5.5). 1  yea 128.000 eu os ∗12mon hs 1  yea ∗19.914, 25 eu os =1, 86mon hs (5.5) In iew o hese indica o s, i can be concluded ha he p ojec is ex ao di- na ily p o i able. In addi ion o educing cos s, he au oma ic so ing sys em will inc ease p o- duc ion speed by 265 %, om 15.3 o 40.6 ons hou 2. 5.3 Summa y In iew o he high classi ica ion accu acy (4.1), he co ec global pe o - mance o he sys em (4.2), and he ex ao dina y p o i abili y (5.2), his p ojec has been a ema kable success, he alding a p omising u u e o a i icial neu al ne wo ks o classi ica ion asks. 2Es ima e ob ained om: a e age weigh o a ock (11.72 kg), speed o he con eyo bel (0.4 m s) and ocks p esen in each me e ape (∼2.4). Appendix A. Addi ional Da a 51 RockType MeanR MeanG MeanB SkewR SkewG SkewB Ku osisR Ku osisG Ku osisB 00 39.21 46.45 70.54 2.2 2.04 1.55 3.76 3.12 1.43 10 120.13 135.82 198.69 1.42 1.16 0.48 0.54 -0.18 -1.28 20 96.16 110.63 157.79 1.94 1.8 1.26 2.47 1.91 0.16 30 77.56 85.03 114.85 2.4 2.39 1.77 4.47 4.44 1.7 40 69.98 74.65 94.87 2.34 2.16 1.54 4 3.15 0.74 ... ... ... ... ... ... ... ... ... ... ... 791 2 12.35 14.46 19.52 3.65 3.59 3.07 13.8 13.39 9.56 792 2 19.69 22.36 29.68 2.09 2.07 1.67 2.99 2.96 1.47 793 2 15.76 18.44 24.31 4.03 4.03 3.72 17.41 17.44 14.62 794 2 17.33 19.79 26.34 3.07 3.08 2.62 8.9 8.94 5.99 795 2 13.97 16.04 21.71 3.51 3.52 3.12 12.46 12.5 9.48 TABLE A.1: Fea u es able o ANN aining. This able con ains: in i s i s column, he ock ype coded; and he emaining columns, co esponding o he 9 ex ac ed ea u es. To show he able as an example, he alue o he ea u es was ounded o 2 decimal places, howe e , o he ANN aining 6 decimal places we e used. 52 Bibliog aphy [1] Ma lab. (2021). “Rgb2g ay,” [Online]. A ailable: h ps://nl.ma hwo ks. com/help/ma lab/ e / gb2g ay.h ml. Accessed: 16/06/2021. [2] L. Shapi o and C Geo ge, “S ockman g: Compu e ision,” in P en ice Hall, 2002. [3] N. O su, “A h eshold selec ion me hod om g ay-le el his og ams,” IEEE ansac ions on sys ems, man, and cybe ne ics, ol. 9, no. 1, pp. 62–66, 1979. [4] OpenCV. (2021). “Adap i e h esholding,” [Online]. A ailable: h ps: //docs.openc .o g/mas e /d7/d4d/ u o ial_py_ h esholding. h ml. Accessed: 14/06/2021. [5] Wikipedia con ibu o s, G adien — Wikipedia, he ee encyclopedia, [On- line; accessed 14-June-2021], 2021. [Online]. A ailable: h ps://en. wikipedia.o g/w/index.php? i le=G adien &oldid=1024885933. [6] I. Sobel, R Duda, P Ha , and J. Wiley, “Sobel- eldman ope a o ,” [7] OpenCV. (2021). “Image g adien s,” [Online]. A ailable: h ps://docs. openc .o g/mas e /d5/d0 / u o ial_py_g adien s.h ml. Accessed: 16/06/2021. [8] J. Canny, “A compu a ional app oach o edge de ec ion,” IEEE T ansac- ions on pa e n analysis and machine in elligence, no. 6, pp. 679–698, 1986. [9] OpenCV. (2021). “Mo phological ans o ma ions,” [Online]. A ailable: h ps://docs.openc .o g/mas e /d9/d61/ u o ial_py_mo phological_ ops.h ml. Accessed: 14/06/2021. [10] S. Russell and P. No ig, “A i icial in elligence: A mode n app oach,” 2002. Bibliog aphy 53 [11] D. W. Hosme J , S. Lemeshow, and R. X. S u di an , Applied logis ic eg ession. John Wiley & Sons, 2013, ol. 398. [12] P. Ramachand an, B. Zoph, and Q. V. Le, “Sea ching o ac i a ion unc- ions,” a Xi p ep in a Xi :1710.05941, 2017. [13] K. Janocha and W. M. Cza necki, “On loss unc ions o deep neu al ne wo ks in classi ica ion,” a Xi p ep in a Xi :1702.05659, 2017. [14] S. Rude , “An o e iew o g adien descen op imiza ion algo i hms,” a Xi p ep in a Xi :1609.04747, 2016. [15] R. Hech -Nielsen, “Theo y o he backp opaga ion neu al ne wo k,” in Neu al ne wo ks o pe cep ion, Else ie , 1992, pp. 65–93. [16] M. Ba , “Pulse wid h modula ion,” Embedded Sys ems P og amming, ol. 14, no. 10, pp. 103–104, 2001. [17] Wikipedia con ibu o s, Ac i e-pixel senso — Wikipedia, he ee encyclo- pedia, [Online; accessed 7-June-2021], 2021. [Online]. A ailable: h ps: //en.wikipedia.o g/w/index.php? i le=Ac i e-pixel_senso & oldid=1018753557. [18] ——, Focal leng h — Wikipedia, he ee encyclopedia, [Online; accessed 7- June-2021], 2021. [Online]. A ailable: h ps://en.wikipedia.o g/w/ index.php? i le=Focal_leng h&oldid=1024363691. [19] ——, F-numbe — Wikipedia, he ee encyclopedia, [Online; accessed 7- June-2021], 2021. [Online]. A ailable: h ps://en.wikipedia.o g/w/ index.php? i le=F-numbe &oldid=1025188269. [20] G. B adski and A. Kaehle , Lea ning OpenCV: Compu e ision wi h he OpenCV lib a y. " O’Reilly Media, Inc.", 2008. [21] M. Abadi, P. Ba ham, J. Chen, Z. Chen, A. Da is, J. Dean, M. De in, S. Ghemawa , G. I ing, M. Isa d, e al., “Tenso low: A sys em o la ge- scale machine lea ning,” in 12 h {USENIX}symposium on ope a ing sys- ems design and implemen a ion ({OSDI}16), 2016, pp. 265–283. [22] E. Bisong, “Google colabo a o y,” in Building Machine Lea ning and Deep Lea ning Models on Google Cloud Pla o m, Sp inge , 2019, pp. 59–64. Bibliog aphy 54 [23] Wikipedia con ibu o s, Dynamic-link lib a y — Wikipedia, he ee ency- clopedia, [Online; accessed 7-June-2021], 2021. [Online]. A ailable: h ps: //en.wikipedia.o g/w/index.php? i le=Dynamic-link_lib a y& oldid=1017853698. [24] OpenCV. (2021). “Con ou s : Ge ing s a ed,” [Online]. A ailable: h ps: //docs.openc .o g/mas e /d4/d73/ u o ial_py_con ou s_begin. h ml. Accessed: 15/06/2021. [25] M. H. P o e and C. B. Mo ey, College calculus wi h analy ic geome y. Addison-Wesley, 1977. [26] jdhao. (2019). “C opping o a ed ec angles om image wi h openc ,” [Online]. A ailable: h ps://jdhao.gi hub.io/2019/02/23/c op_ o a ed_ ec angle_openc /. Accessed: 24/04/2021. [27] M. Mu uga el. (2019). “Objec acking — e e enced wi h he p e i- ous ame using euclidean dis ance,” [Online]. A ailable: h ps : / / mani annan- ai.medium.com/objec - acking- e e enced- wi h- he - p e ious - ame - using - euclidean - dis ance - 49118730051a. Accessed: 24/05/2021. [28] Wikipedia con ibu o s, Py hago ean heo em — Wikipedia, he ee en- cyclopedia, [Online; accessed 11-June-2021], 2021. [Online]. A ailable: h ps : / / en . wikipedia . o g / w / index . php ? i le = Py hago ean _ heo em&oldid=1027893764. [29] S. Canu. (2021). “Objec acking wi h openc and py hon,” [Online]. A ailable: h ps://pysou ce.com/2021/01/28/objec - acking- wi h-openc -and-py hon/. Accessed: 24/05/2021. [30] H. B. Ba low, “Unsupe ised lea ning,” Neu al compu a ion, ol. 1, no. 3, pp. 295–311, 1989. [31] Wikipedia con ibu o s, K-means clus e ing — Wikipedia, he ee encyclo- pedia, [Online; accessed 7-June-2021], 2021. [Online]. A ailable: h ps: //en.wikipedia.o g/w/index.php? i le=K- means_clus e ing& oldid=1021685301. Bibliog aphy 55 [32] D. Jadha , G. Phadke, and S. De ane, “Colou and ex u e ea u e based hyb id app oach o image e ie al,” in Ad ances in Compu e Science, Enginee ing & Applica ions, Sp inge , 2012, pp. 101–111. [33] S. B own, “Measu es o shape: Skewness and ku osis,” Re ie ed on Augus , ol. 20, p. 2012, 2011. [34] V. López, A. Fe nández, and F. He e a, “On he impo ance o he al- ida ion echnique o classi ica ion wi h imbalanced da ase s: Add ess- ing co a ia e shi when da a is skewed,” In o ma ion Sciences, ol. 257, pp. 1–13, 2014. [35] Wikipedia con ibu o s, En opy (in o ma ion heo y) — Wikipedia, he ee encyclopedia, [Online; accessed 7-June-2021], 2021. [Online]. A ail- able: h ps://en.wikipedia.o g/w/index.php? i le=En opy_ (in o ma ion_ heo y)&oldid=1024571966. [36] S. (h ps://s a s.s ackexchange.com/use s/2719/simone), A gene al mea- su e o da a-se imbalance, C oss Valida ed, ( e sion: 2016-10-13). [On- line]. A ailable: h ps://s a s.s ackexchange.com/q/239982. [37] I. Guyon e al., “A scaling law o he alida ion-se aining-se size a io,” AT&T Bell Labo a o ies, ol. 1, no. 11, 1997. [38] S. (h ps://s acko e low.com/use s/349130/d snoopy), Wha is h5 model in ke as, C oss Valida ed, ( e sion: 2018-01-18). [Online]. A ail- able: h ps://s acko e low.com/ques ions/48320588/wha - is- h5-model-in-ke as. [39] Sala yexplo e . (2021). “A i icial in elligence de elope a e age sala y in belgium 2021,” [Online]. A ailable: h p://www.sala yexplo e . com/sala y-su ey.php?loc=21&loc ype=1&job=12098&job ype=3. Accessed: 14/06/2021. [40] S. explo e . (2021). “De elope / p og amme a e age sala y in bel- gium 2021,” [Online]. A ailable: h p://www.sala yexplo e .com/ sala y- su ey.php?loc=21&loc ype=1&job=783&job ype=3. Ac- cessed: 14/06/2021. Bibliog aphy 56 [41] Paylab. (2021). “Plc p og amme . elec ical powe enginee ing,” [On- line]. A ailable: h ps://www.paylab.com/be/sala yin o/elec ical- powe -enginee ing/plc-p og amme . Accessed: 14/06/2021. [42] Oc opa . (2021). “Om on ccs-ld 2-90sw2 p icing and a ailable in en- o y.,” [Online]. A ailable: h ps://oc opa .com/ccs-ld 2-90sw2- om on-71332513. Accessed: 14/06/2021. [43] Ebay. (2021). “One ccs pd2-3024-2 led ligh sou ce con olle ,” [Online]. A ailable: h ps : / / www . ebay . com / i m / ONE - CCS - PD2 - 3024 - 2 - LED- ligh - sou ce- con olle - DC24V- 2- ou pu - /283288537781. Accessed: 14/06/2021. [44] ebay. (2021). “Ids ui-1225le-c-hq cama a usb 2.0,” [Online]. A ailable: h ps://www.ebay.es/i m/154089823902?hash=i em23e0781e9e:g: ~mUAAOSwy~N Ye3w. Accessed: 14/06/2021. [45] Kowa. (2021). “Lm8jc1ms | 2/3" 8mm 2mp c-moun lens,” [Online]. A ailable: h ps : / / www . kowa - lenses . com / en / lm8jc1ms - 2mp - indus ial-lens-c-moun . Accessed: 14/06/2021.