Panoramic Background Modeling for PTZ Cameras with Competitive Learning Neural Networks

Pano amic Backg ound Modeling o PTZ Came as wi h Compe i i e Lea ning Neu al Ne wo ks Ka l Thu nho e -Hemsi∗, Ezequiel L´ opez-Rubio∗, En ique Dom´ ınguez∗, Ra ael Ma cos Luque-Baena∗and Miguel A. Molina-Cabello∗ ∗Depa men o Compu e Languages and Compu e Science Uni e si y o M´ alaga, Bule a Louis Pas eu , 35, 29071 M´ alaga, Spain Emails: {ka lkhade ,ezeql ,en iqued, mluque,miguelangel}@lcc.uma.es Abs ac —The cons uc ion o a model o he backg ound o a scene s ill emains as a challenging ask in ideo su eillance sys ems, in pa icula o mo ing came as. This wo k p esen s a no el app oach o cons uc ing a pano amic backg ound model based on compe i i e lea ning neu al ne wo ks and a subsequen piecewise linea in e pola ion by Delaunay iangu- la ion. The app oach can handle a bi a y came a di ec ions and zooms o a Pan-Til -Zoom (PTZ) came a-based su eil- lance sys em. A e es ing he p oposed app oach on se e al indoo sequences, he esul s demons a e ha he p oposed me hod is e ec i e and sui able o use o eal- ime ideo su eillance applica ions. 1. In oduc ion Wi h hei high mobili y and zoom capabili y, pan- il - zoom (PTZ) came as ha e become inc easingly popula in moni o ing public a eas [1], [2], [3]. Omnidi ec ional came as a e p omising candida es o moni o ing mul iple la en ac i i ies in he a ea o in e es [4]. Howe e , hese kind o came as ha e nonuni o m esolu ion and a e unable o p o ide close obse a ions o pa icula a ge s. In hese cases, whe e PTZ came as a e needed, he combina ion o hese wo ypes o came as (omnidi ec ional and PTZ) is p oposed in o de o acili a e a con inuous moni o ing o he whole su eillance a ea and de ailed obse a ions o speci ic a ge s simul aneously [5]. Ne e heless, his dual-came a sys em may be s ill an expensi e and complex solu ion in some scena ios. Fo his eason, in his pape we a e ocusing on an ac i e sensing app oach o mul iple objec de ec ion and acking using a single PTZ came a. Mos came as used in su eillance a e s a ic, and he scenes aken om his ype o came as only show one spe- ci ic iew o he su eillance a ea. Fo hese images/ ideos, he mos common and e icien app oach o mo ing objec de ec ion is backg ound sub ac ion, ha consis s in main- aining an up- o-da e model o he ixed backg ound and de ec ing mo ing objec s as hose ha de ia e om such model. Compa ed o o he app oaches, such as op ical low, his app oach is compu a ionally a o dable o eal- ime applica ions, is independen o mo ing objec eloci y, and is no subjec o he o eg ound ape u e p oblem. Howe e , adi ional backg ound sub ac ion algo i hms assume he came as a e s a ic and his leads o alse de ec- ion when he came a mo es [6], [7]. Due o his came a mo emen , e en pixels belonging o s a ic objec s appea o mo e in he came a ame (called ego-mo ion e ec ). Ex ensi e esea ch has been ca ied ou ega ding objec de ec ion o mo ing came as. Some p oposal a e based on he op ical low clus e ing, ha consis s in calcula ing dense o spa se op ical lows and clus e ing hem o iden i y mo ing objec egions [8]. Ano he me hods a e based on he es ima ion o he ans o ma ion pa ame e s be ween consec- u i e ames [9]. Ou app oach is based on mosaicing he backg ound [10], [11], ha consis in c ea ing a mosaiced o pano amic backg ound image and hen using a backg ound sub ac ion echnique o ex ac mo ing objec egions. The p oblem o mo ing objec s de ec ion o PTZ cam- e as is add essed in his pape , and we p opose a me hod based on building a pano amic backg ound model using a compe i i e neu al ne wo k. Apa o he adi ional and equen ly ci ed seminal pape s ela ed o compe i i e lea n- ing [12], [13], [14], ecen success ul applica ions in he compu e ision ield can be ound in he li e a u e [15], [16], [17], [18]. In ou app oach, a compe i i e neu al ne - wo k is used o build a pano amic backg ound model o objec de ec ion. Due o he huge inpu in o ma ion, a la ge numbe o neu ons has been used and he neu on p o o ypes ha e been o ganized in a quad- ee in o de o be quickly e alua ed. The es o he pape is o ganized as ollows. In sec ion 2, a mo e de ailed desc ip ion o he p oposed neu al model is p esen ed. In sec ion 3, we p esen he esul s achie ed wi h he implemen a ion o he p oposed app oach. Finally, sec ion 4 includes some concluding ema ks. 2. The model In his sec ion a compe i i e lea ning based sys em o lea n he backg ound o a pano amic scene om he inpu o a PTZ came a is p oposed. Fi s he da a acquisi ion p ocedu e o ans o m he inpu ideo ames in o inpu samples o he compe i i e lea ning ne wo k is conside ed (Subsec ion 2.1). Then he compe i i e lea ning model is de- sc ibed (Subsec ion 2.2). Finally, an in e pola ion p ocedu e 978-1-5090-6182-2/17/$31.00 ©2017 IEEE 396 is designed o es ima e he backg ound om he inal s a e o he compe i i e lea ning ne wo k, which is based on a Delaunay iangula ion (Subsec ion 2.3). 2.1. Da a acquisi ion The da a om which we s a a e he acqui ed ideo ames o he PTZ came a. These a e ideo ames wi h ixed wid h and heigh , and e e y pixel has wo ame coo dina es (x, y). To ans o m hese ame coo dina es o ano he coo dina e sys em in he pano amic image, he i s s ep is o ob ain he sphe ical coo dina es associa ed o he ame coo dina es and hen ca y ou a scala ans o ma ion o he dimensions o he pano amic image. In o de o ind ou equi ed pola coo dina es (θ,φ) o an a bi a y poin Abelonging o p ojec ion plane (x, y) a pinhole came a model is used, consis ing o a sphe e cen e ed on he coo dina e o igin and a p ojec ion plane (see Fig. 1a), whose bounds a e he wid h and heigh o he ame window in pixels: w,h. In his wo k he i ual PTZ lib a y [19] is employed, so we mus ollow he coo dina e sys em c i e ia used by ha lib a y. We s a wi h he case o came a o ien a ion (pan, il ) = (0,0), which co esponds o see he posi i e Z axis om he o igin. The gene al case, when pan 6= 0, il 6= 0 is ob ained om he pa icula case by means o a coo dina e sys em ans o ma ion. The coo dina es o ec o OA in coo dina e sys em O can be ound as p ojec ions on he coo dina e axes: ( x, y, z)=(x−w/2, h/2−y, )(1) whe e z = , since poin s Aand Ca e loca ed on he same plane. x and y is calcula ed aking in accoun ha Aon he p ojec ion plane is based ela i ely o he le op co ne o he ame. Radius is calcula ed using he e ical ield o iew (F OV ), which is known. We only need o no ice ha FOC = FOV /2, and F OC is a igh iangle, so an( FOV /2) = FC ⇔ =h/2 an( FOV /2) (2) This way he came a coo dina es a e compu ed. In o de o ex end he pa icula case o he gene ic case and ob ain he wo ld coo dina es we need o ecalcula e ec o OC om he Ocoo dina e sys em in o he o a ed coo dina e sys em O0. The coo dina es a e ound by means o mul ipli- ca ion by he in e se o a double o a ion ma ix. This ma ix can be ound by mul iplica ion o wo o a ion ma ices: a ound Xand Zaxes by (pan, il ): R=  1 0 0 0 cos( il )−sin( il ) 0 sin( il ) cos( il ) · ·  cos(pan)−sin(pan) 0 sin(pan) cos(pan) 0 0 0 1  (3) A0= ( x0, y0, z0) = R−1·( x, y, z)(4) Now, when he new coo dina es A0a e compu ed, we only need he sphe ical coo dina es ans o ma ion o ob ain θand φ:    x0=Rcos(φ) sin(θ) y0=Rsin(φ) sin(θ) z0=Rcos(θ) (5) By ea anging he o mulas: (φ= a c an( y0 x0) θ= a ccos( z0 R) = a ccos( z0 √ x02+ y02+ z02)(6) The ob ained samples a e eal alued ec o s, whose wo i s componen s a e he sphe ical coo dina es φand θo a pixel, while he hi d las ones a e he obse ed RGB colo alues a ha pixel: x= (θ, φ, , g, b)=(x1, x2, x3, x4, x5)(7) so ha x∈R5. A each ime ins an , a new episode S ⊂ R5is acqui ed which con ains one sample o each pixel o he incoming ideo ame a ime . The e o e he ca dinal o S is he numbe o pixels in he ideo ame. 2.2. Compe i i e lea ning ne wo k Nex a compe i i e lea ning model o pano amic scenes cap u ed by PTZ came as is de eloped. In o de o lea n he de ails o he scene, a la ge numbe o neu ons N is employed, which is lowe bu in he same o de o magni ude as he numbe o pixels o he ull pano ama. Following he s a egy in [20], he inpu ec o s x∈R5 a e di ided in o wo sec ions. The i s sec ion con ains he posi ional in o ma ion in he ideo ame, while he second sec ion con ains he colo ea u es. In ou case, he i s sec ion comp ises he wo sphe ical coo dina es φ and θ, while he second sec ion con ains he RGB colo alues. All he componen s o he inpu ec o s a e used o upda e he neu on p o o ypes wi∈R5, bu only he wo i s ones pa icipa e in he compe i ion. This way, only he posi ional in o ma ion is employed o de e mine which neu on is he winne . The e o e, each neu on ep esen s he a e age colo in a ecep i e ield which is one egion o a Vo onoi esella ion o he ull pano ama. Tha is, he neu ons specialize on small pixel neighbo hoods o he pano ama. Since he posi ional componen s can ha e ac ional alues, (x1, x2)∈R2, he ne wo k can lea n ac ional posi ional in- o ma ion wi hou ha ing o ound o in ege pixel posi ions in he pano ama, which a oids losing aluable in o ma ion. Each neu on con ains a p o o ype wi∈R5and also a Boolean lag bi∈ { ue, alse}. The lag is equi ed o con ol he ini ializa ion o he neu on. Since he PTZ came a does no co e he en i e pano ama a a ime, i is no possible o ini ialize all he neu ons a he same ime. They can only be ini ialized while hei posi ional componen s all in o he cu en ield o iew. To his end, he Boolean lags a e ini ialized o alse. The i s ime ha a neu on wins, i s p o o ype is se o he inpu sample, and i s lag is se o ue. 397 Y -X Z A=(x,y) x z = y C F V O θ X φ h w X Z Y z’ x’ y’ A’=( x’, y’, z’) φ θ Figu e 1. Schemes o he PTZ came a model. a) Le : pinhole came a model wi h he o iginal o ien a ion. b) Righ : eo ien ed scheme o ob ain sphe ical coo dina es. F om ha poin on, he neu on will be upda ed acco ding o he compe i i e lea ning ule. The p oposed lea ning algo i hm is as ollows: 1) D aw a aining sample xa andom om om he cu en episode S , which has no been conside ed be o e, i.e. a andom sampling wi hou eplacemen is done. 2) Find he nea es neu on qin e ms o Euclidean dis ance acco ding o he i s wo ec o componen s: q= a g min i∈{1,...,N}k(wi,1, wi,2)−(x1, x2)k(8) 3) I he lag bqis ue, hen go o s ep 4. O he wise, se he p o o ype o he winning neu on q o he aining sample and se i s lag o ue: wq=x(9) bq= ue (10) Then go o s ep 5. 4) Upda e he winning neu on p o o ype acco ding o he s anda d compe i i e lea ning ule: ∆wq=η( ) (x−wq)(11) whe e η( )is a decaying lea ning a e ha a ies depending on he ime s ep . 5) I all he samples o he cu en episode S ha e al- eady been p ocessed, hen go o s ep 6. O he wise, go o s ep 1. 6) I he las ime ins an has been eached, hen s op. O he wise, inc emen he ime ins an coun e , load he nex episode and go o s ep 1. The e a e wo phases in he lea ning p ocess: i s he o de ing phase whe e ηexpe iences a linea decay (ini ial lea ning a e ηI); and hen he con e gence phase whe e η emains cons an a a small alue (ηC). This is because he o de ing phase is equi ed o he wa m-up o he algo i hm only, and a e ha he sys em uns o an inde ini ely long ime. The change o phase is moni o ed by a s ep pa ame e n. η( ) = ηI(1 − /n),i <n ηC, o he wise (12) A heo e ical analysis o he abo e algo i hm can be ca ied ou . Since he compe i ion is done on he i s wo componen s, he algo i hm seeks a local minimum o an ene gy unc ion Ewhich only akes in o accoun hese componen s: E=X i∈{1,...,N}X x∈Fi k(wi,1, wi,2)−(x1, x2)k2(13) whe e Fiis he ecep i e ield o he i- h neu on: Fq=x|q= a g min i∈{1,...,N}k(wi,1, wi,2)−(x1, x2)k (14) On he o he hand, he p o o ype upda e is ca ied ou on he h ee las componen s oo, so hose componen s app oxima e he a e age colo o he ecep i e ield: (wi,3, wi,4, wi,5)≈E[(x3, x4, x5)|Fi](15) 398 As a las ema k, i mus be poin ed ou ha he la ge numbe o neu ons N o be used o his applica ion equi es a conside able op imiza ion o he compe i ion equa ion (8). This is accomplished by inse ing all he i s sec ions (wi,1, wi,2)o he neu on p o o ypes in o a quad- ee [21]. This way (8) is e alua ed e y quickly, e en o alues o Nin he millions. 2.3. Delaunay iangula ion based in e pola ion Since he posi ions o he neu on p o o ypes a e gi en by eal numbe s, he e is no di ec way o ob ain he es i- ma ed colo o he in ege alued pixel coo dina es o he pano ama. In o de o o e come his di icul y, we p opose o build he Delaunay iangula ion [22] o he se o med by he i s sec ions (wi,1, wi,2)o each neu on p o o ype. This way, a iangula ion o he pano ama is ob ained. Then, o each in ege alued pai o pixel coo dina es (y1, y2)∈N2, he iangle which i belongs o is compu ed, along wi h i s ba ycen ic coo dina es wi h espec o ha iangle: y=λi(wi,1, wi,2)+λj(wj,1, wj,2)+λk(wk,1, wk,2)(16) i, j, k ∈ {1, ..., N}(17) λi, λj, λk≥0(18) λi+λj+λk= 1 (19) Then he es ima ed colo (y3, y4, y5)∈R3is ob ained by linea in e pola ion wi h weigh s equal o he ba ycen ic coo dina es: (y3, y4, y5) = λi(wi,3, wi,4, wi,5) + λj(wj,3, wj,4, wj,5) + λk(wk,3, wk,4, wk,5)(20) The e o e a con inuous, piecewise linea unc ion is employed o es ima e he colo o e he pano ama. 3. Expe imen al esul s In his sec ion we epo he compu a ional expe imen s we ha e ca ied ou and hei esul s. The so wa e and ha dwa e ha ha e been used a e speci ied in Subsec ion 3.1. Then, he es ed ideo sequences a e desc ibed in 3.2. The desc ip ions o he used pa ame e s a e in Subsec ion 3.3 and he desc ip ions o he compe i o s in 3.4. Finally, he ob ained esul s om he expe imen s a e epo ed in Subsec ion 3.5. 3.1. Me hods The came a con ol module is based on he i ualp z lib a y [19]. I simula es a PTZ came a om a pano amic ideo sequence, and i is accessible om i s websi e 1. The implemen a ion is w i en in C++ and i uses he OpenCV and OpenGL lib a ies. This i ual came a has limi a ions in i s e ical mo emen , going om 0 (up) o 180 (down) deg ees. To gene a e he inpu da a o he compe i i e neu al ne and o he compe i o s, a exhaus i e scanning o he scene has been ca ied ou . To simula e he eal beha io o a PTZ came a in a p ac ical se ing, some limi a ions in he scan ha e been imposed. S a ing a he op ( e ical 0 deg ees), we u n 360 deg ees o he igh wi h a s ep o 10 deg ees, and when we come back o he ini ial poin , we go down 10 deg ees. This p ocess is epea ed un il we a i e o he bo om and we s a o go up again. Fu he mo e, a andom zoom ( e ical ield o iew) is applied o each ame, bu again, we y o simula e a eal si ua ion. To his end, we gene a e a andom numbe in an in e al [F OV, FOV + 5] and mo e his in e al by s eps o 5 deg ees be ween a minimum and maximum alue o he ield o iew, [70,140] deg ees, in o de o a oid s ange images wi h i egula i ies. A o al o 3000 came a ames ha e been sa ed in bina y iles o ead hem synch onously by all he me hods, which ha e been implemen ed in Ma lab R2015b. The epo ed expe imen s ha e been ca ied ou on a 64-bi Pe sonal Compu e wi h an eigh -co e In el i7 3.60GHz CPU, 32 GB RAM and s anda d ha dwa e. The implemen a ion o ou app oach does no use any GPU esou ces. Pano amic g ound u h (GT) has been calcula ed doing he median o he aw pano amic ideo ames. Fo each me hod, we compa ed he pano amic image ob ained wi h he g ound u h. Th ee quali y measu es we e used o e alua e he p oposed app oach: he i s was he Mean Squa ed E o (MSE) me ic (lowe is be e ), which is commonly used in image p ocessing; he second was he S uc u al Simila i y index (highe is be e ), which ocuses on s uc u al simila i ies be ween images: SSIM(x, y) = (2µxµy)(2σxy +c2) (µ2 x+µ2 y+c1)(σ2 x+σ2 y+c2)(21) whe e µxand µya e he mean alue o images xand y, σxand σya e he s anda d de ia ion o images xand y,σxy is he co a iance o xand y,c1= (k1L)2and c2= (k2L)2(Lis he dynamic ange, k1= 0.01 and k2= 0.03). The alues ob ained om (21) a e a e aged o e he h ee RGB channels o ob ain he pe o mance o colo images. The hi d quan i a i e pe o mance measu e is he Bha acha ya coe icien BC [23], which measu es he closeness o he wo disc e e pixel p obabili y dis ibu ions Pand ˆ Pco esponding o he GT and modeled images: BC = 255 X j=0 P(j)ˆ P(j)(22) 1. h ps://bi bucke .o g/pie e luc s cha les/ i ualp z s andalone 399 TABLE 1. CONSIDERED PARAMETER VALUES FOR OUR COMPETITIVE LEARNING METHOD Pa ame e Value Numbe o neu ons N(Numbe o pixels)/2 Ini ial lea ning a e ηI0.4 Con e gence lea ning a e ηC0.01 Numbe o s eps n1000 whe e BC ∈[0,1] and highe is be e . In addi ion, CPU imes we e measu ed o line o each inpu bina y ile o ob ain he eal unning ime o each algo i hm pe ame, wi hou including he ex a wai ing ime o acqui e he nex ame ha would be equi ed i we did i online. 3.2. Sequences Fo he expe imen s we used wo ideo da abases. One o hem has h ee ideos which a e a ailable on he i - ualp z websi e. They a e h ee indoo sequences, wo o hem wi h he same scene, and hey a e named scena io3, scena io4 and scena io5. Because he wo i s ones a e e y simila , we ha e jus shown he image esul s o he scena io3 (3500x1750 pixels and 566 ideo ames). The ideo scena io5 (3500x1750 pixels and 1957 ames) shows a oom wi h people mo ing on in and doing di e en ac ions. On he o he hand, we used one ideo sequence o he Li ls a web page2. We named his ideo as scena io6 (2880x1440 pixels and 1169 ames) and shows a beach wi h people mo ing and playing beach oleyball. S a is ics we e calcula ed using he ou sequences. 3.3. Pa ame e selec ion A se o uned pa ame e s is needed o de ine he com- pe i i e lea ning model. These ixed pa ame e s a e epo ed in Table 1. In addi ion, a s udy o he numbe o neu ons has been done and he esul s a e displayed in Fig. 2. As seen, he equilib ium s a e is eached quickly and excellen esul s a e ob ained om only 612500 neu ons (10% o o al pixels). 3.4. Compe i o s We ha e compa ed ou p oposed me hod wi h o he h ee me hods. The compe ing me hods ead each ame, hen posi ion i in he pano amic image ma ix and inally compu e he mean o e all he ames. Since da a inpu coo dina es a e almos always ac ional numbe s, he pano amic in ege coo dina es whe e he in- coming ame mus be placed we e calcula ed by ounding he o iginal ac ional coo dina es. Besides, he coo dina e pai s do no de ine a egula s ic ly mono onic g id, i.e., he poin s ha e no s uc u e o o de be ween hei ela i e 2. h ps://li ls a .com/ loca ions, so ha he usual in e pola ion me hods de ined o ob ain he in e pola ed RGB alues in a pano ama canno be used. The e o e, he sca e ed in e polan me hod om MAT- LAB was employed o manage his si ua ion. I p o ides he ollowing in e pola ion a ian s: ’nea es ’ (nea es -neighbo in e pola ion), ’linea ’ (linea in e pola ion), and ’na u al’ (na u al-neighbo in e pola ion). The same inpu da a was p o ided o all he compe i o s as well as ou me hod, i.e. he 3000 came a ames sa ed in bina y iles. 3.5. Resul s A compa ison o all he e alua ed me hods o one o he selec ed ideos is epo ed in Fig. 3. This igu e shows he compa a i e e olu ion o he me hods wi h espec o he numbe o cap u ed PTZ came a ames, i.e. he ame index in he ideo sequence. I can be obse ed ha when a ce ain numbe o ames is eached ha co e s almos all he pano amic image, which is a ound 500 ames, ou me hod (in ed) a ains he bes pe o mance alues, as compa ed o he compe ing me hods. In pa icula , MSE enhances subs an ially, whe e lowe alues indica es ha he ob ained RGB pixel alues a e mo e p ecise. When he numbe o acqui ed ames inc eases o mo e han 2500, all me hods end o pe o m simila ly, bu ou me hod s ill emains as he bes one. Some hing simila happens o he o he h ee ideos. To summa ize i , we ha e calcula ed he mean and s anda d de ia ion o he ou sequences o each me hod and o each pe o mance measu e. These quali a i e esul s a e shown in Table 2. As we can see, ou me hod ou pe o ms he compe ing me hods, pa icula ly in he mean squa ed e o . Highe alues o SSIM and BC con i m ha ou compe i i e neu al model p oduces he bes app oxima ion o he backg ound o he scene. The CPU ime equi ed o p ocess one ame is a e y impo an ea u e o be assessed. Table 2 shows he mean equi ed ime o p ocess a bina y ile o a ame. Ou me hod is a ound 85% as e han he in e pola ion me hods. I compu es he winne neu ons and upda es he quad- ee whe e hey a e s o ed in jus o e a second. I we conside ha a mo emen o he i ual PTZ came a and he gene - a ion o he bina y ile o he cu en ame akes be ween one and h ee seconds, i u ns ou ha ou me hod is he only one ha can be execu ed concu en ly wi h he PTZ came a ame acquisi ion p ocess. The compe i o s iplica e he equi ed ime and incu in a big ime delay o ob ain he pano amic backg ound. The u iliza ion o a GPU would imp o e be ween 25 and 50 imes he p ocessing a e. In o de o ge a quali a i e poin o iew abou he sui abili y o ou app oach, he pano amic backg ounds gene a ed by each me hod ha e been compa ed. In Fig. 5 a window o 50x50 pixels o wo scena ios a e displayed, wi h hei espec i e aw pano amic ideo ame. As i can be obse ed, he quali a i e esul s o e ed by ou app oach a e he mos simila o he g ound u h. This can be be e app ecia ed on he op and he bo om o he pano amic image, whe e he compe i o s p oduce black pixels ha 400 0246 Numbe o Neu ons 10 6 1.9 2 2.1 2.2 2.3 2.4 log10(MSE) 0246 Numbe o Neu ons 10 6 0.99 0.9 0 SSIM 0246 Numbe o Neu ons 10 6 0.999 0.99 0.9 BC 0246 Numbe o Neu ons 10 6 0.2 0.4 0.6 0.8 1 1.2 1.4 CPU ime (sec.) Figu e 2. E olu ion o he MSE, SSIM, BC and CPU ime a ying he numbe o neu ons. scena io3 was used o he analysis. MSE has been displayed in a loga i hmic scale (lowe is be e ), SSIM and BC in a e e se loga i hmic scale (highe is be e ). 0 1000 2000 3000 NumF ames 1.5 2 2.5 3 3.5 4 4.5 log10(MSE) 0 1000 2000 3000 NumF ames 0.99 0.9 0 SSIM 0 1000 2000 3000 NumF ames 0.999 0.99 0.9 0 BC Ou s linea nea es na u al Figu e 3. Compa a i e o he MSE, SSIM and BC o he ou me hods wi h scena io3. MSE has been displayed in a loga i hmic scale (lowe is be e ), SSIM and BC in a e e se loga i hmic scale (highe is be e ). should no be he e. The window o scena io6 is a clea example o his. Also, in he le and he igh o he images p oduced by he compe i o s he e a e mo e black pixels han in ou app oach. Wi h espec o he mo ing people o he ideo ame, all me hods emo e hem e icien ly and no big di e ences a e no iced. An example o he esul o ou me hod is shown in Fig. 4. 4. Conclusion In his wo k, a me hodology o model he pano amic backg ound o PTZ came as is p esen ed. I consis s o an online lea ning me hod based on a compe i i e neu al ne - wo k o ead each came a ame and p ocess i o gene a e a pano amic image o he scene. Fou scenes ha e been es ed o check he easibili y o he sys em, ob aining sui able and success ul esul s. Also, i has been demons a ed ha ou app oach ou pe o ms se e al compe ing me hods. I is ema kable ha he p oposed model uses e y li le CPU ime o p ocess each inpu ideo ame, which pe mi s a easy in eg a ion in a eal ime PTZ came a ideo su eillance sys em. Acknowledgmen s This wo k is pa ially suppo ed by he Minis y o Econ- omy and Compe i i eness o Spain unde g an TIN2014- 53465-R, p ojec name Video su eillance by ac i e sea ch o anomalous e en s. I is also pa ially suppo ed by he Au onomous Go e nmen o Andalusia (Spain) unde p ojec s TIC-6213, p ojec name De elopmen o Sel - O ganizing Neu al Ne wo ks o In o ma ion Technologies; and TIC-657, p ojec name Sel -o ganizing sys ems and obus es ima o s o ideo su eillance. Finally, i is pa - ially suppo ed by he Au onomous Go e nmen o Ex- emadu a (Spain) unde he p ojec IB13113. All o hem include unds om he Eu opean Regional De elopmen Fund (ERDF). The au ho s hank ully he g an o he Uni e si y o M´ alaga and acknowledge he compu e e- sou ces, echnical expe ise and assis ance p o ided by he SCBI (Supe compu ing and Bioin o ma ics) cen e o he Uni e si y o M´ alaga. They also g a e ully acknowledge he suppo o NVIDIA Co po a ion wi h he dona ion o he Ti an X GPU. Ka l Thu nho e -Hemsi is unded by a PhD schola ship om he Spanish Minis y o Educa ion, Cul u e and Spo unde he FPU p og am. Re e ences [1] Chung-Chen Chen, Yi Yao, A. D i a, A. Koschan, and M. Abidi, “Co- ope a i e mapping o mul iple PTZ came as in au oma ed su eillance sys ems,” in 2009 IEEE Con e ence on Compu e Vision and Pa e n Recogni ion. IEEE, 2009, pp. 1078–1084. [2] C. Ding, B. Song, A. Mo ye, J. Fa ell, and A. Roy-Chowdhu y, “Collabo a i e sensing in a dis ibu ed p z came a ne wo k,” IEEE 401 TABLE 2. MEAN (STANDARD DEVIATION)MEASURES FOR EACH METHOD ALONG THE FOUR SCENARIOS MSE SSIM BC T ime(sec/ ame) Ou s 106.87 (23.09) 0.9184 (0.0371) 0.9703 (0.0466) 1.1427 (0.1284) Linea 233.92 (35.60) 0.9167 (0.0288) 0.9701 (0.0460) 7.6549 (0.8106) Nea es neighbo 235.00 (35.12) 0.9176 (0.0282) 0.9701 (0.0463) 7.7251 (0.8171) Na u al neighbo 234.00 (35.63) 0.9162 (0.0292) 0.9700 (0.0459) 9.1097 (1.0477) F ame P oposed me hod Figu e 4. Pano amic images o scena io5 o ame 153 and he p oposed me hod, esp. T ansac ions on Image P ocessing, ol. 21, no. 7, pp. 3282–3295, 2012. [3] K. Konda, N. Conci, and F. De Na ale, “Global co e age maximiza- ion in p z-came a ne wo ks based on isual quali y assessmen ,” IEEE Senso s Jou nal, ol. 16, no. 16, pp. 6317–6332, 2016. [4] T. Boul , X. Gao, R. Micheals, and M. Eckmann, “Omni-di ec ional isual su eillance,” Image and Vision Compu ing, ol. 22, no. 7, pp. 515–534, 2004. [5] C.-H. Chen, Y. Yao, D. Page, B. Abidi, A. Koschan, and M. Abidi, “He e ogeneous usion o omnidi ec ional and p z came as o mul i- ple objec acking,” IEEE T ansac ions on Ci cui s and Sys ems o Video Technology, ol. 18, no. 8, pp. 1052–1063, 2008. [6] S. Kim, K. Yun, K. Yi, S. Kim, and J. Choi, “De ec ion o mo ing objec s wi h a mo ing came a using non-pano amic backg ound model,” Machine Vision and Applica ions, ol. 24, no. 5, pp. 1015– 1028, 2013. [7] K. Xue, Y. Liu, G. Ogunmakin, J. Chen, and J. Zhang, “Pano amic gaussian mix u e model and la ge-scale ange backg ound subs ac- ion me hod o p z came a-based su eillance sys ems,” Machine Vision and Applica ions, ol. 24, no. 3, pp. 477–492, 2013. [8] P. D. Z. Va cheie and G. A. Bilodeau, “Adap i e uzzy pa icle il e acke o a PTZ came a in an IP su eillance sys em,” IEEE T ansac ions on Ins umen a ion and Measu emen , ol. 60, no. 2, pp. 354–371, 2011. [9] F. J. L´ opez-Rubio and E. L´ opez-Rubio, “Fo eg ound de ec ion o mo ing came as wi h s ochas ic app oxima ion,” Pa e n Recogni ion Le e s, ol. 68, pp. 161–168, 2015. [10] P. Azza i and A. Be ilacqua, “Join spa ial and onal mosaic alignmen o mo ion de ec ion wi h PTZ came a,” Lec u e No es in Compu e Science, ol. 4142, pp. 764–775, 2006. [11] A. Be ilacqua and P. Azza i, “High-Quali y Real Time Mo ion De- ec ion Using PTZ Came as,” P oceedings o he IEEE In e na ional Con e ence on Video and Signal Based Su eillance (AVSS 2006), pp. 1–6, 2006. [12] S. Ahal , A. K ishnamu hy, P. Chen, and D. Mel on, “Compe i i e lea ning algo i hms o ec o quan iza ion,” Neu al Ne wo ks, ol. 3, no. 3, pp. 277–290, 1990. [13] T. Uchiyama and M. A bib, “Colo image segmen a ion using compe - i i e lea ning,” IEEE T ansac ions on Pa e n Analysis and Machine In elligence, ol. 16, no. 12, pp. 1197–1206, 1994. [14] E. Yai , “Compe i i e lea ning and so compe i ion o ec o quan- ize design,” IEEE T ansac ions on Signal P ocessing, ol. 40, no. 2, pp. 294–309, 1992. [15] H.-P. Chen, X.-J. Shen, and J.-W. Long, “His og am-based colou im- age uzzy clus e ing algo i hm,” Mul imedia Tools and Applica ions, ol. 75, no. 18, pp. 11 417–11 432, 2016. [16] E. Ozan, S. Ki anyaz, and M. Gabbouj, “Compe i i e quan iza ion o app oxima e nea es neighbo sea ch,” IEEE T ansac ions on Knowledge and Da a Enginee ing, ol. 28, no. 11, pp. 2884–2894, 2016. [17] R. Valen e and T. Ab ao, “MIMO ansmi scheme based on mo - phological pe cep on wi h compe i i e lea ning,” Neu al Ne wo ks, ol. 80, pp. 9–18, 2016. [18] H. Xie, X. Luo, C. Wang, S. Liu, X. Xu, and X. Tong, “Mul ispec al emo e sensing image segmen a ion using i al penalized con olled compe i i e lea ning and uzzy en opy,” So Compu ing, ol. 20, no. 12, pp. 4709–4722, 2016. [19] G. Chen, P. S -Cha les, W. Bouachi , T. Joeissein , G. Bilodeau, and R. Be ge in, “Rep oducible e alua ion o pan- il -zoom acking,” CoRR, ol. abs/1505.04502, 2015. [Online]. A ailable: h p://a xi .o g/abs/1505.04502 [20] R. M. Luque-Baena, E. L´ opez-Rubio, E. Dom´ ınguez, E. J. Palomo, and J. M. Je ez, “A sel -o ganizing map o imp o e ehicle de ec ion in low moni o ing sys ems,” So Compu ing, ol. 19, no. 9, pp. 2499– 2509, 2015. [21] S. F. F isken and R. N. Pe y, “Simple and e icien a e sal me hods o quad ees and oc ees,” Jou nal o G aphics Tools, ol. 7, pp. 1– 11, 2002. [22] D. Lee and B. Schach e , “Two algo i hms o cons uc ing a delaunay iangula ion,” In e na ional Jou nal o Compu e & In o ma ion Sciences, ol. 9, no. 3, pp. 219–242, 1980. [23] A. Bha acha yya, “On a measu e o di e gence be ween wo mul inomial popula ions,” Sankhy: The Indian Jou nal o S a is ics (1933-1960), ol. 7, no. 4, pp. 401–406, 1946. [Online]. A ailable: h p://www.js o .o g/s able/25047882 402 scena io3 scena io6 F ame GT P oposed linea nea es na u al Figu e 5. G aphical depic ion o he ope a ion o he p oposed me hod. Each ow show, om le o igh , up o down: a ame o a aw pano amic ideo sequence, and sec ions ( ed squa e) o i s g ound u h, ou algo i hm, and he h ee compe i o s algo i hms. F ame 153 o scena io3 and ame 859 o scena io6 a e shown. 403