CowLog - cross-platform application for coding behaviours from video

Pas ell, M 2016 CowLog – C oss-Pla o m Applica ion o Coding Beha iou s om Video. Jou nal o Open Resea ch So wa e, 4: e15, DOI: h p://dx.doi.o g/10.5334/jo s.113 Jou nal o open esea ch so wa e SOFTWARE METAPAPER CowLog – C oss-Pla o m Applica ion o Coding Beha iou s om Video Ma i Pas ell1 1 Na u al Resou ces Ins i u e Finland (Luke), G een Technology. Viikinkaa i 4, 00790 Helsinki, Finland ma i.pas ell@luke. i CowLog is a c oss-pla o m applica ion o code beha iou s om ideo eco dings o use in beha iou al esea ch. The so wa e has been used in se e al s udies e.g. o s udy sleep in dai y cal es, emo ions in goa s and he mind wande ing ela ed o compu e use du ing lec u es. CowLog 3 is implemen ed using Ja aSc ip and HTML using he Elec on amewo k. The amewo k allows he de elopmen o packaged c oss-pla o m applica ions using ea u es om web b owse ( Ch omium) as well as se e side Ja aSc ip om Node.js. The p og am suppo s using mul iple ideos simul aneously and HTML5 and VLC ideo playe s. CowLog can be used o any p ojec ha equi es coding he ime o e en s om digi al ideo. I is eleased unde GNU GPL 2 making i possible o use s o modi y he applica ion o hei own needs. The so wa e is a ailable h ough i s websi e h p://cowlog.o g. Keywo ds: Beha iou al coding; ideo coding; Elec on F amewo k (1) O e iew In oduc ion CowLog is a c oss-pla o m applica ion o code beha iou s om ideo eco dings. I can be used in any applica ion whe e coding beha iou s om ideo is needed. The i s e sion o he applica ion [1] o was de eloped in 2009 o a p ojec whe e we needed o code he sleep beha - iou o cal es om ideo eco dings in o de o de elop a sleep ac ig aph [2]. The so wa e has also been used in se - e al o he s udies e.g. o s udy emo ions in goa s [3], he mind wande ing ela ed o compu e use du ing lec u es [4] and e ec s o simula ed anspo on he beha iou o eas e n blue ongued liza ds [5]. The e we e 33 pape s indexed in Scopus ci ing he i s pape abou CowLog [1] a he ime o he w i ing. The ini ial elease un only on Linux and had limi ed con igu a ion op ions [1]. I soon became clea om ou own p ojec s and use eedback om o he g oups a e he ini ial elease ha he e was a clea need o a c oss- pla o m e sion. Two e sions o CowLog we e de eloped o add ess hese issues: Py hon based desk op e sion and HTML5 applica ion ha uns in he b owse . Se e al use s we e conce ned abou he p i acy issues o he web applica ion al hough i only used single sc ip o o ma he ou pu ile and e e y hing else un on clien side. I was decided o di ec de elopmen e o only o Ja aSc ip and in 2015 he web e sion o he applica- ion was po ed o he Elec on F amewo k and packaged be a e sion as a desk op applica ion. This was eleased as CowLog 3 in ea ly 2016. The aim o his pape is o desc ibe he implemen a ion o CowLog 3 in o de o make i easie o use s o modi y he p og am o hei needs o pa icipa e in de elopmen o he eleased e sion. Implemen a ion and a chi ec u e CowLog e sion 3 desc ibed in his pape was imple- men ed using Ja aSc ip and HTML5 using he Elec on amewo k. The amewo k allows he de elopmen o packaged c oss-pla o m applica ions using ea u es om he Ch omium web b owse as well as se e side Ja aSc ip om Node.js. This e sion is based on ea lie e sion o CowLog ha uns in he web b owse . An applica ion implemen ed using he Elec on amewo k has a main p ocess and one o se e al en- de e p ocesses which display he use in e ace. The main p ocess, based on Node.js, is used o c ea e he use in e ace windows, con ols he applica ion li e ime and p o ides ea u es such as ile sys em access and ia Node.js modules. The ende e p ocess displays he use in e ace as a websi e, each c ea ed window uns in i s own p ocess. The ende e p ocess and main p ocess communica e wi h e en based sys em using he Elec on in e p ocess communica ion module. An applica ion using Elec on can also use almos any Node.js module o ex end he unc ionali y. Pas ell: CowLog – C oss-Pla o m Applica ion o Coding Beha iou s om VideoA . e15, pp.  2 o 4 The used a chi ec u e makes he applica ion e y modula and he design and implemen a ion in one window can be changed easily wi hou a ec ing he o he windows as long as he messages exchanged be ween p ocesses a e no changed. The use in e ace makes ex ensi e use o he jQue y Ja aSc ip lib a y. Fo ins ance he coding bu ons in he use in e ace a e gene a ed dynamically based on he p ojec con igu a ion and he p og am indica es he cu - en ly ac i e code o each beha iou al class. The jQue y UI lib a y is used o he ideo p og ess ba and da e picke componen s. The use in e ace uses he Boo s ap amewo k o layou and design o he elemen s such as bu ons and o ms. The e hog am, olde s o ideos and ou pu , used ideo playe , modi ie s o beha iou s and keyboa d sho cu s o coding can be de ined and modi ied using CowLog p ojec s. The p ojec se ings a e sa ed o JSON ile which can be used o ans e he se ings be ween compu e s and e.g. sha ed oge he wi h publica ions. The so wa e makes use o wo ideo playe s and he use can selec he mos sui able one o hei applica ion: 1. Na i e HTML5 ideo playe is able o play ogg, mp4 (h.264) and webm ideo o ma s. The Ja aS- c ip API o he playe is easy o use and has good documen a ion. 2. VLC playe con olled using WebChime a.js, which is able o ende ideo ames di ec ly on HTML5 can as gi ing good pe o mance. CowLog bina ies on Windows and Mac OS X a e bundled wi h VLC playe and p ecompiled WebChime a.js. On Linux he use should ins all VLC playe sepa a ely. The p og am suppo s using mul iple ideos simul ane- ously, and all o he ideos a e played in single window. The playback speed o he ideo can be changed o make i slowe o as e . The e is no limi o he numbe o simul- aneous ideos used, he p ac ical limi a ions depends on he capaci y o he used compu e . In ou p ac ical p o- jec s we ha e used a mos 3 came as a he same ime o co e la ge enough a ea. The layou o he ideos is also use selec able. CowLog use in e ace o he ac ual beha iou al coding is ocused a ound bu ons ep esen ing each beha iou . The use can also con igu e and use keyboa d sho cu s o he coding. Dialogs a e used o c ea e and edi p ojec s and s a - ing o esuming coding sessions. Coding bu ons and he ideos a e displayed in sepa a e windows o make i easie o use he so wa e wi h mul iple displays. Figu e 1 shows he CowLog main window con igu ed wi h a simple e ho- g am o s udy he beha iou o sows [6] a ound a owing and he ideo window. Figu e 1: CowLog coding window (le ) con igu ed o s udy he beha iou o pigs be o e a owing and he ideo window ( igh ). The coding window shows he las en e ed code and allows he con ol o ideo. Pas ell: CowLog – C oss-Pla o m Applica ion o Coding Beha iou s om Video A . e15, pp.  3 o 4 Releases o he so wa e a e packaged o di e en pla o ms using elec on-package , which makes i simple o package he elease o Mac, Windows and Linux a chi ec u es wi h a single command. The Windows e sion is also eleased wi h an ins alle buil using Inno Se up. Quali y con ol The so wa e goes ough unc ional es ing and a be a elease be o e each inal elease using exis ing p ojec con igu a ions and se e al ideo o ma s. The e a e cu - en ly no au oma ed es s. The de elopmen akes place on Ubun u Linux, Mac OS X and Windows and he builds a e es ed on each o he pla o ms. The so wa e is used in ou own esea ch and we ge immedia e eedback om use s. Se e al o he esea ch g oups also use CowLog and ha e epo ed om hei expe iences. All use s can epo bugs and ask o ea u es on Gi Hub o using e-mail. (2) A ailabili y Ope a ing sys em Windows, 7, 8 and 10. Mac OS X (> = 10.8). Linux 32bi and 64bi , es ed on Ubun u 14.04 and 15.10. Elec on uns also on Linux ARM 7, bu CowLog has no been es ed on ha pla o m. P og amming language Ja aSc ip and HTML5 unning on he Elec on amewo k. Addi ional sys em equi emen s A ypical coding session has modes sys em equi emen s. The sys em equi emen s depend on he codec, esolu ion and he numbe o simul aneous ideos used. The p o- g am can un on all Elec on suppo ed pla o ms. Dependencies – Elec on 0.36.2 – jQue y 2.1.4 – jQue y UI 1.8.21 – Boo s ap 3.3.5 – WebChime a.js 0.1.47 – VLC playe 2.2 (Op ional) Lis o con ibu o s – D . Ma i Pas ell (Na u al Resou ces Ins i u e Finland, Luke). De elope . – D . Lau a Hänninen (Uni e si y o Helsinki). Requi e- men s o he i s e sion – D . And ew Robins (The Uni e si y o Queensland). CowLog logo, es ing and eedback. So wa e loca ion A chi e Name: Gi Hub Pe sis en iden i ie : h ps://gi hub.com/mpas ell/ CowLog/ eleases Licence: GPL- 2.0 Publishe : Ma i Pas ell Da e published: 13/01/2016 ( 3.0.1) Code eposi o y Name: Gi Hub Iden i ie : h ps://gi hub.com/mpas ell/CowLog Licence: GPL- 2.0 Da e published: 12/09/2015 Language English (3) Reuse po en ial CowLog is gene al pu pose applica ion in beha iou al coding. I can be used o any p ojec ha equi es cod- ing he ime o e en s om digi al ideo. The so wa e has been used in s udies abou animal beha iou as well acial exp essions in humans. The so wa e could easily be mod- i ied o do also li e coding. New de elope s a e welcome o make con ibu ion o he p og am ia Gi Hub. Compe ing In e es s The au ho decla es ha hey ha e no compe ing in e es s. Acknowledgemen s I acknowledge all CowLog use s who ha e sen me hei eedback and used he so wa e in in e es ing esea ch p ojec s. Re e ences 1. Hänninen, L and Pas ell, M 2009 CowLog: Open sou ce so wa e o coding beha io s om digi al ideo. Beha io Resea ch Me hods, 41 (2): 472–476. DOI: h p://dx.doi.o g/10.3758/BRM.41.2.472 2. Hokkanen, A-H, Hänninen, L, Tiusanen, J and Pas ell, M 2011 P edic ing sleep and lying ime o cal es wi h a suppo ec o machine classi ie using accele ome e da a. Applied Animal Beha iou Science, 134(1–2): 10–15, ISSN 0168-1591. DOI: h p://dx.doi. o g/10.1016/j.applanim.2011.06.016 3. B ie e , E F, Te aman i, F and McElligo , A G 2015 Emo ions in goa s: Mapping physiological, beha iou al and ocal p o iles. Animal Beha iou , 99: 131–143. DOI: h p://dx.doi.o g/10.1016/j.anbeha .2014.11.002 4. Risko, E F, Buchanan, D, Medimo ec, S and Kings one, A 2013 E e yday a en ion: Mind wan- de ing and compu e use du ing lec u es. Compu e s and Educa ion, 68: 275–283. h p://dx.doi.o g/DOI: 10.1016/j.compedu.2013.05.001 5. Mance a, K, Mu ay, P J, Gao, Y N, Lisle, A and Phillips, C 2014 The e ec s o simula ed anspo on he beha iou o eas e n blue ongued liza ds (Tiliqua scincoides). Animal Wel a e, 23 (3): 239–249. DOI: h p://dx.doi.o g/10.7120/09627286.23.3.239 6. Pas ell, M, Hie aoja, J, Tiusanen, J, Yun, J and Val os, A 2014 A Model o De ec Fa owing based on Sow Ac i i y. P oceedings o he In e na ional Con e - ence o Ag icul u al Enginee ing - AgEng 2014, Zu ich. DOI: h p://dx.doi.o g/10.13140/RG.2.1.2073.0960 Pas ell: CowLog – C oss-Pla o m Applica ion o Coding Beha iou s om VideoA . e15, pp.  4 o 4 How o ci e his a icle: Pas ell, M 2016 CowLog – C oss-Pla o m Applica ion o Coding Beha iou s om Video. Jou nal o Open Resea ch So wa e, 4: e15, DOI: h p://dx.doi.o g/10.5334/jo s.113 Submi ed: 14 Janua y 2016 Aceep ed: 13 Ap il 2016 Published: 27 Ap il 2016 Copy igh : © 2016 The Au ho (s). This is an open-access a icle dis ibu ed unde he e ms o he C ea i e Commons A ibu ion 4.0 In e na ional License (CC-BY 4.0), which pe mi s un es ic ed use, dis ibu ion, and ep oduc ion in any medium, p o ided he o iginal au ho and sou ce a e c edi ed. See h p://c ea i ecommons.o g/licenses/by/4.0/. Jou nal o Open Resea ch So wa e is a pee - e iewed open access jou nal published by Ubiqui y P ess OPEN ACCESS