Pombal’s maze: a novel decision making paradigm

Ana Ri a Secca da Fonseca Disse a ion p esen ed o ob ain he Ph.D deg ee in Biology | Neu oscience Ins i u o de Tecnologia Química e Biológica An ónio Xa ie | Uni e sidade No a de Lisboa Oei as, May, 2016 Inse he e an image wi h ounded co ne s Pombal’s maze: a no el decision- making pa adigm Ana Ri a Secca da Fonseca Disse a ion p esen ed o ob ain he Ph.D deg ee in Biology | Neu oscience Ins i u o de Tecnologia Química e Biológica An ónio Xa ie | Uni e sidade No a de Lisboa Oei as, May, 2016 Pombal’s maze: a no el decision- making pa adigm Resea ch wo k coo dina ed by: Cá den o inquie ação, inquie ação É só inquie ação, inquie ação Po quê, não sei Mas sei É que não sei ainda José Má io B anco, in “Inquie ação” À memó ia de meu pai de quem he dei a inquie ação que me ouxe a é aqui This wo k was de eloped in he con ex o he In e na ional Neu oscience Doc o al P og amme (INDP) o he Champalimaud Resea ch P og amme, Champalimaud Cen e o he Unknown, Lisbon, Po ugal. The p ojec en i led “Pombal’s maze: a no el decision-making pa adigm” was ca ied ou a Ins i u o Gulbenkian de Ciência, Oei as, Po ugal and a he Champalimaud Resea ch P og amme, Champalimaud Cen e o he Unknown, Lisbon, Po ugal, unde he scien i ic supe ision o Zacha y Mainen, Ph.D, and unde he guidance o he Thesis Commi ee composed by Ca los Ribei o, Ph.D, and Rui Cos a, Ph.D. This wo k was suppo ed by he ellowship SFRH/BD/33944/2009 om Fundação pa a a Ciência e Tecnologia, Po ugal. i ACKNOWLEDGEMENTS I eel ex emely p i ileged o ha e been o e ed he possibili y o unde ake such an inc edible in ellec ual endea o . Abo e all, I hank he people who h oughou hese yea s ha e gene ously gi en me he oppo uni y o become a be e pe son, owa ds he wo ld and owa ds mysel Zach as an example o ex a-senso ial in ui ion and in ellec ual eedom Masa as an example o igo and endless gene osi y Pa ícia o all he cogni i e ough and umble Gil o silence Ma ia o h owing ladde s a he moon Bass as an example o making-i -happen Cindy as an example o ho ough kindness owa ds Na u e E ic as an example o scien i ic cul u e Nico as an example o de ail Sa a as an example o endu ance Sam as an example o insu ec ion Alex as an example o ebellion Susana and Ma a as examples o se iously play ul science Rui and Ca los o pu ing hings in o pe spec i e INDP 2008 o all he un and suppo CNP communi y as an example o di e si y and ac i e lis ening wo ld wide web as an example o human coope a ion Alexand a Elbakyan as example o open science Ra s as eminde s o a uni e sal bond Ana C. H. o helping me disco e new ways o look wi hin ii À minha amília xoné como exemplo de amizade e es upidez na u al À minha amília de sangue po me e ensinado o be o e o iso À a ó Augus a como exemplo de le eza À a ó Luz como um exemplo do b inca Ao i mão Filipe como exemplo de des emo À mãe Ana como exemplo de esis ência A Ma ilde como exemplo de impe manência A F ede ico como exemplo de amo incondicional iii RESUMO Os animais são cons a emen e con on ados com decisões: o que come ? Onde encon a pa cei o sexual? Como alcança aquela go a de água? Ac ualmen e, deco e um deba e den o do campo cujo o objec o de es udo são as omadas de decisão, sob e quais são os mecanismos subjacen es à selecção de acções: se á que os animais escolhem de en e o que p e endem ob e - o e o no, ou escolhem de en e o que p ecisam de aze pa a alcança esse e o no? Exis em dados compo amen ais e de ac i idade neu onal que supo am a hipó ese de que há compe ição en e as acções po enciais a acon ece em á eas elacionadas com a componen e mo o a, enquan o os animais êm que decidi de en e múl iplas opções qual a que le a ao objec i o. Mais especi icamen e, em sido a ançada a hipó ese de que a selecção de acções depende dum mecanismo de compe ição en e acções po enciais, onde um sinal pola izado a o ece uma das acções, le ando à sua selecção. Po o ma a melho comp eende em que medida a selecção de acções depende e ec i amen e dum mecanismo de compe ição, é necessá io explo a melho quais os ipos de in e acção en e acções po enciais, simul aneamen e ao ní el da ac i idade neu onal e do compo amen o. Uma das o mas em que es e objec i o pode á se a ingido é a a és da in es igação da ac i idade neu onal e do compo amen o enquan o animais decidem en e múl iplas acções em que odas le am ao objec i o. Den o das neu ociências, exis em duas linhas de in es igação cujo um dos objec i os é de e mina o que acon ece no cé eb o quando múl iplas acções pe mi em a ingi o objec i o: o es udo de na egação espacial, que po um lado, p e ende comp eende a selecção de acções pela pe spec i a do p ocessamen o de in o mação espacial, ocando-se nos pon os do i espaço que ob igam a uma decisão - pon os de escolha- e co esponden es escolhas disc e as, e o planeamen o mo o , que p e ende comp eende a seleção de acções pela pe spec i a do con olo mo o , ocando-se nas dínâmicas con ínuas do mo imen o. Assim, desenhámos e desen ol emos um no o pa adigma compo amen al- a que chamámos de labi in o de Pombal- onde são o e ecidas múl iplas opções de acção pa a chega a um objec i o. Es e pa adigma ap esen a a oedo es o p oblema de escolhe en e caminhos in e connec ados quais os que lhe pe mi em chega a uma ecompensa sinalizada po um sinal luminososo, o que implica que es a a e a em simul aneamen e ca ac e ís icas de uma a e a de na egação e de uma a e a de planeamen o mo o . O pa adigma p opos o baseia-se num labi in o que ap esen a uma es u u a de uma g elha com múl iplos pon os de ecompensa e múl iplas in e secções. Es as in e secções inham o po encial pa a se o na em pon os de escolha, dando a opo unidade aos animais de aze á ias decisões ao longo do caminho. Foi obse ado que os oedo es adop am a es a égia óp ima de escolhe os caminhos que são mais cu os pa a chega à ecompensa, na maio ia das ezes. Den o do conjun o dos caminhos mais cu os, os animais mos am endências pa a ce as opções baseando-se em ca ac e ís icas geomé icas (e.g. o núme o de mudanças de di ecção). Es as endências e lec em cus os in e nos implíci os associados a di e en es ca ac e ís iscas dos caminhos que combinadas podem ge a compe ição en e di e en es acções, pois podem se mu uamen e exclusi as. Adicionalmen e, oi obse ado que os animais diminuiam a elocidade quando se ap oxima am da in e secção, o que é indica i o de que es es locais e am is os como sí ios onde ha ia a possibilidade de decidi en e acções. Compa ando ajec ó ias de caminhos em xi Figu e 15 T ials ca ego ized acco ding o di e en c i e ia.. .................. 50 Figu e 16 Example o s a -goal pai o mul iple op ion analysis.. ........ 51 Figu e 17 Example o s a -goal pai o cen e a oidance analysis. ....... 52 Figu e 18 Numbe ing o each po .. ......................................................... 53 Figu e 19 Example s a -goal pai o maze expe ience.. ........................ 52 Figu e 20 Goal p obabili ies pe s a po .. ............................................. 53 Figu e 21 Me ics associa ed wi h di ec ion o mo emen . ..................... 54 Figu e 22 Animals a oid cen e pa hs.. ................................................... 56 Figu e 23 Animals p e e ed o he go back h ough he di ec ion o a i al a he po . ..................................................................................... 57 Figu e 24 P e e ence o ype A is main ained ac oss he majo i y o s a -goal pai s.. ........................................................................................ 59 Figu e 25 Ra s p e e pa hs ha minimize dis ance o goal as e .. ......... 58 Figu e 26 Ra s p e e pa hs ha go h ough he cen e , in a h ee op ions condi ion.. ................................................................................................ 59 Figu e 27 Animals p e e o con inue in he di ec ion ha hey a i ed a he po .. ................................................................................................... 60 Figu e 28 Animals p e e pa hs wi h one u n.. ....................................... 61 Figu e 29 Animals a e as e o each he goal when aking a pa h wi h one u n. ................................................................................................... 62 Figu e 30 Di e en ac o s de ine di e en expec ed p opo ions o each op ion. ...................................................................................................... 68 Figu e 31 T ial ype A minimizes numbe o u ns and ial ype C minimizes Euclidean dis ance o he goal.. .............................................. 63 Figu e 32 Di e en ac o s de ine di e en expec ed p opo ions o each op ion.. ..................................................................................................... 64 xii Figu e 33 Ra s don' p e e pa hs ha minimize Euclidean dis ance o goal.. ......................................................................................................... 65 Figu e 34 Animals p e e pa hs wi h one u n. ........................................ 66 Figu e 35 Mo emen ime is independen o numbe o u ns................. 67 Figu e 36 The p obabili y o choice o op ion A o op ion B should be he same.. ................................................................................................. 72 Figu e 37 Window o in e es o beha io al analysis.). ......................... 77 Figu e 38 Linea ans o ma ion o ajec o ies o spa ial alignmen .. ... 78 Figu e 39 Raw eloci ies and co esponding ajec o ies.. ...................... 78 Figu e 40 Examples o binned ajec o ies and co esponding eloci ies.. .................................................................................................................. 79 Figu e 41 Example ial dynamics p o ile whe e one a le he choice poin ROI and hen e-en e ed i . ............................................................. 79 Figu e 42 Pai ing o s a -goal pai s acco ding o spa ial o e lap.. ......... 80 Figu e 43 Animals a e as e o lea e s a po when hey a e abou o do a sho es pa h.. ........................................................................................ 81 Figu e 44 Animals ake mo e ime o lea e he s a po ROI when mo e op ions a e a ailable.. .............................................................................. 83 Figu e 45 Animals we e as e o lea e s a po when he goal was close .. ...................................................................................................... 84 Figu e 46 Bell-shaped eloci y p o ile.. .................................................. 85 Figu e 47 Animals slow down while eaching he choice poin .. ............ 86 Figu e 48 Ex a non-op imal op ion does no a ec dec ease in eloci y, when one op imal op ion is a ailable....................................................... 87 Figu e 49 Ex a non-op imal op ion does no a ec dec ease in eloci y, when mo e han one op imal op ion is a ailable.. ................................... 89 xiii Figu e 50 Animals slow down mo e when choice poin o e s wo op imal pa h op ions.. ............................................................................... 90 Figu e 51 Dis ance o he ligh does no a ec eloci y dec ease a he choice poin .............................................................................................. 92 Figu e 52 T ajec o ies om compe ing op ions in a h ee op ion condi ion di e ge 20 cm a e animal has le he s a po ROI.. ........................... 94 Figu e 53 T ajec o ies om compe ing op ions in a six op ion condi ion di e ge 20 cm a e animal has le he s a po ROI. ............................ 95 xi AUTHORS CONTRIBUTION Ana Ri a Fonseca (ARF) and Zacha y F ank Mainen (ZFM) designed he expe imen s (Chap e 2). ARF conduc ed he expe imen s and analyzed he da a (Chap e 3 and 4). x TABLE OF CONTENTS ACKNOWLEDGEMENTS ______________________________________ i RESUMO _________________________________________________ iii ABSTRACT ________________________________________________ i ABREVIATION LIST _________________________________________ ix FIGURE INDEX ______________________________________________ x AUTHORS CONTRIBUTION __________________________________ xi TABLE OF CONTENTS _______________________________________ x 1 GENERAL INTRODUCTION ________________________________ 1 1.1 The en ance… _________________________________________ 2 1.1.1 Wha _____________________________________________________ 3 1.1.2 Why ______________________________________________________ 3 1.1.3 How ______________________________________________________ 3 1.2 F aming decision-making p ocesses _________________________ 4 1.3 Ac ion selec ion mechanisms ______________________________ 6 1.3.1 Compe i ion be ween ac ions _________________________________ 8 1.4 Mul iple ac ions, one goal _______________________________ 11 1.4.1 Na iga ion and ac ion selec ion _______________________________ 12 1.4.2 Mo o planning and ac ion selec ion ___________________________ 16 1.5 Guide o he eade ____________________________________ 20 2 POMBAL’S MAZE ______________________________________ 22 2.1 In oduc ion __________________________________________ 23 2.2 Me hods _____________________________________________ 24 x i 2.2.1 Animal subjec s __________________________________________ 24 2.2.2 Expe imen al appa a us ___________________________________ 25 2.2.3 Pombal’s maze pa adigm __________________________________ 26 2.2.4 Spa ial dis ibu ion o ewa d _______________________________ 27 2.2.5 T aining _________________________________________________ 28 2.2.6 Video p ocessing _________________________________________ 29 2.2.7 T ajec o ies in o sequences ________________________________ 30 2.2.8 Video and BCon ol synch oniza ion ________________________ 32 2.2.9 Da a base _______________________________________________ 32 2.2.10 Valida ion c i e ia _______________________________________ 33 2.3 Resul s _______________________________________________ 35 2.3.1 P e e ence o sho es pa hs ______________________________ 35 2.3.2 S eng h o ligh -wa e pai ing ______________________________ 38 2.3.3 Biased choices ___________________________________________ 39 2.4 Discussion ____________________________________________ 44 3 DISCRETE CHOICES _____________________________________ 47 3.1 In oduc ion __________________________________________ 48 3.2 Me hods _____________________________________________ 48 3.2.1 T ial classi ica ion _________________________________________ 49 3.2.2 Mo emen ime ___________________________________________ 54 3.2.3 Di ec ion o mo emen ____________________________________ 54 3.2.4 Measu ing p e e ence _____________________________________ 55 3.3 Resul s _______________________________________________ 55 3.3.1 P e e ence o allocen ic ea u es: cen e e sus edge ________ 55 3.3.2 P e e ence o egocen ic ea u es: backwa ds e sus o wa d _ 56 3.3.3 P e e ence o geome ical ea u es: minimizing he Euclidean dis ance o goal __________________________________________________ 57 3.3.4 P e e ence o geome ical ea u es: numbe o u ns _________ 61 x ii 3.3.5 Quali a i e amewo k _____________________________________ 67 3.3.6 Compe i ion be ween op ions ______________________________ 62 3.4 Discussion ____________________________________________ 67 4 DYNAMICS OF CHOICE __________________________________ 75 4.1 In oduc ion __________________________________________ 76 4.2 Me hods _____________________________________________ 76 4.2.1 Spa ial window o in e es __________________________________ 77 4.2.2 Spa ial da a alignmen and binning _________________________ 77 4.2.3 Veloci y exclusion c i e ion _________________________________ 79 4.2.4 ROC analysis ____________________________________________ 80 4.3 Resul s _______________________________________________ 81 4.3.1 La ency o lea e he s a po ______________________________ 81 4.3.2 Bell-shaped eloci y p o ile_________________________________ 84 4.3.3 Veloci y a he choice poin _________________________________ 86 4.3.4 T ajec o ies o compe ing pa hs ____________________________ 92 4.4 Discussion ____________________________________________ 95 5 GENERAL DISCUSSION _________________________________ 100 5.1 Concep ual aming ____________________________________ 101 5.2 O e iew o main empi ical indings ______________________ 101 5.3 C i ical poin s ________________________________________ 103 5.4 Pombal’s maze and ac ion selec ion ______________________ 105 5.5 Final ema ks and u u e di ec ions _______________________ 108 6 REFERENCES _________________________________________ 110 1 GENERAL INTRODUCTION I ge in in wha , om he ou side, i ’s called he inside sea ching o he pa h ha will ge o he cen e lea ing longing and ea on he ou side Sé gio Godinho, in “The maze” (F eely ansla ed by ARF) 2 1.1 The en ance… Na u e has p o ided humans wi h a lo o wonde abou . Fa away licke ing poin s up abo e ou heads, sea ages, clamo ous ligh ning c uising da k skies, apples ha pe sis o all and days ha pe sis o die. Bu maybe, and jus maybe, he phenomenon ha elen lessly akes mo e o ou su p ise is ou sel es, wha is wi hin us. And, wha e e ha migh be, i li es in an enclosed ini e body which in e ac s wi h o he s – conspeci ics o no - and wi h he en i onmen , h ough beha io . E en hough he e is no consensus abou a de ini ion o beha io , neu oscien is s ag ee ha i is gene a ed by di e en ypes o biological ma e (ne e cells, muscles) which wo k oge he o gene a e ac ions. The e a e many pa s bu in he end he body is jus one. I a a goes on a ques o ind wa e hen i s head, legs, ail and 200 000 000 neu ons ha e o ge he e, in ull commi men wi h ha decision, oge he . Bu animals ha e o make decisions cons an ly: wha o ea , whe e o ma e, how o each ha wa e d op... The e a e always many op ions so how do animals selec a single one? The e is an ongoing deba e wi hin he decision-making ield abou he unde lying mechanisms o ac ion selec ion: is ac ion selec ion achie ed by i s making a choice be ween abs ac ep esen a ion o ou comes and hen pe o ming a senso imo o ans o ma ion o ha abs ac ion o he ac ion space o is i achie ed by e alua ing se e al po en ial ac ions while a he same ime weighing he cos s and bene i s o each, be o e a i ing a a decision? 3 1.1.1 Wha One o he undamen al p oblems in decision-making is o unde s and how ac ions a e selec ed (1.2 F aming decision-making p ocesses). Fo a long ime i was hough ha ac ion selec ion elied mos ly on a cen al execu i e sys em ha would hen send he selec ed abs ac ac ion o be implemen ed by he mo o sys em. The e is cu en ly compelling e idence ha highe mo o a eas a e in ol ed in he ac ion selec ion p ocess. I has been hypo hesized ha ac ion selec ion is gene a ed by a compe i ion p ocess be ween po en ial ac ions which unde lying a chi ec u e elies on mu ual inhibi ion ha is ed by a alue-based bias signal (1.3 Ac ion selec ion mechanisms). 1.1.2 Why On one side, he e is a ai amoun o esea ch ha has b ough unde s anding abou he gene a ion o he bias signal. On he o he side, he e is bo h beha io and neu al da a ha suppo s he hypo hesis ha compe i ion be ween po en ial ac ions is aking place in mo o - ela ed a eas while animals a e deciding be ween mul iple op ions o each a goal (1.3.1 Compe i ion be ween ac ions). None heless, in o de o u he unde s and o wha ex en ac ion selec ion elies on compe i ion, i is necessa y o u he explo e he ypes o in e ac ions be ween po en ial ac ions bo h a he le el o neu al ac i i y and beha io . 1.1.3 How One way in which his subjec could be add essed is by in es iga ing decision-making p oblems whe e mul iple op ions lead o a goal and he animal can eely choose which op ion o ake (1.4 Mul iple ac ions, one 10 p esen ed o a monkey ha has o saccade o one o wo pa allel a ge s o implici ly epo which mo ion di ec ion i pe cei ed, allowed us o unde s and ha in la e al in apa ie al a ea (LIP) he e a e neu ons whose ac i i y co ela es wi h he in eg a ion o in o ma ion. Ano he example is he odo -mix u e ca ego iza ion ask, whe e a mix u e o wo odo s is p esen ed o a a ha has o nose poke in o one o wo pa allel po s o implici ly epo which odo iden i y i pe cei ed, allowed o unde s and ha no only SC was ac i e be o e mo emen ini ia ion bu ha his ac i i y a ied wi h senso y unce ain y and ask di icul y (Felsen & Mainen 2012). Value-based decision-making pa adigms allowed answe ing ques ions like whe e and how subjec i e expe ience in o ma ion is in eg a ed. Fo example, in a ask whe e monkeys chose be ween wo ypes o juice o e ed in di e en amoun s, OFC neu ons encode alue independen ly o isuospa ial ac o s and mo o esponses (Padoa-Schioppa & Assad 2006). Ano he example is a dynamic o aging ask, whe e a s chose eely be ween wo spa ial goals ha deli e ed a ixed amoun o wa e ewa d wi h di e en p obabili ies, allowed o unde s and ha neu ons in seconda y mo o co ex (M2) encode alue o upcoming choice (Sul e al. 2012). In summa y, using hese ca ego ies o pa adigms beha io al and neu al da a was in es iga ed while senso ial and/o subjec i e in o ma ion was manipula ed, c ea ing decision con lic s ha he subjec had o sol e. This lead o be e unde s anding on how he b ain ep esen s and in eg a es in o ma ion ha is aluable o decisions whe he his in o ma ion is senso ial o de i ed om subjec i e expe ience (Glimche e al. 2009; Sug ue e al. 2005). The p e iously desc ibed pa adigms con ibu ed o he unde s anding abou how he bias is gene a ed bu i s ill emains o be unde s ood i he e is a compe i ion ci cui ha compu es he ac ion selec ion 11 and ha would be ed wi h his bias signal. The e a e o he ela ed lines o s udy ha ha e add essed hese ques ions mo e speci ically. 1.3.1.2 Rep esen a ion o po en ial ac ions The hypo hesis ha ac ion selec ion is implemen ed h ough compe i ion elies hea ily on he implici p emise ha po en ial ac ions unde selec ion a e simul aneously ep esen ed while he e is an ongoing decision-making (Cisek 2006; Tippe e al. 1998). The heo y ha pe cei ing and ac ing on he wo ld depends on he simul aneous speci ica ion o mul iple po en ial ac ions han can be a o ded by he en i onmen has been a ound o 40 yea s (Gibson 1979). Du ing ha ime con incing e idence has been accumula ed showing ha when an animal is o e ed he possibili y o choose be ween mul iple ac ions o each a goal hese po en ial ac ions a e simul aneously ep esen ed in mo o - ela ed a eas (Cisek & Kalaska 2002; Cisek & Kalaska 2005; Basso & Wu z 1998; Houk 1995). In summa y, a leas some o he necessa y condi ions o he implemen a ion o ac ion selec ion h ough compe i ion a e me , bo h o he exis ence o a bias signal and o he simul aneous ep esen a ion o po en ial ac ions. None heless, i is no ully unde s ood how and wha ype o in e ac ions dynamically occu be ween po en ial ac ions, bo h a he le el o neu al ac i i y and beha io . I is necessa y o u he explo e he ypes o in e ac ions be ween po en ial ac ions bo h a he le el o neu al ac i i y and beha io . One way in which his subjec could be add essed is by in es iga ing neu al ac i i y and beha io while animals a e deciding be ween mul iple op ions ha all lead o goal 1.4 Mul iple ac ions, one goal 12 Wi hin he decision-making ield, he e a e wo ypes o p oblems ha equi e he subjec o make a decision abou which ac ion o ake, among al e na i es, o each he same goal: mo o planning asks and spa ial na iga ion asks. Bo h ypes o asks in ol e making choices abou which mo emen o sequence o mo emen s o op imize in o de o each a goal (Penne & Mizumo i 2012; Whi lock e al. 2008; Wolpe & Landy 2012), being he main di e ence ha , on he i s case, he mo emen s unde choice a e pe o med by one body pa (hand, a m o eye) and, in he second case, he whole body. Also, he e is an impo an line o complemen a i y be ween hese wo ields associa ed o he ype o ac ion selec ion co ela es hey ocus on. I , on one side, in na iga ion pa adigms one o he key windows o unde s anding ac ion selec ion is he choice poin , whe e disc e e choices a e measu ed and in e p e ed acco ding o di e en s a egies (Tolman 1938); in he case o mo o planning, one o he key windows is he mo emen be ween s a and goal posi ions, whe e he con inuous dynamics o mo emen is assessed (Flash & Hogan 1985). I seems ha conside ing bo h o hese wo pe spec i es can b ing a mo e comple e unde s anding abou ac ion selec ion co ela es and may lead o a mo e de ailed cha ac e iza ion o bo h he ype o choices a choice poin s and he dynamics o hose choices. Consequen ly, his can con ibu e o a mo e comple e comp ehension o how animals selec ac ions namely by exposing co ela es o he in e ac ions be ween po en ial ac ions. 1.4.1 Na iga ion and ac ion selec ion I has been p oposed ex ensi ely ha animals selec ac ions in na iga ion con ex s using wo dis inc schemes de i ed om ins umen al condi ioning: goal-di ec ed ac ion selec ion, d i en by esponse-ou come associa ions, and 13 habi ual ac ion-selec ion, d i en by s imulus- esponse associa ions (Glimche e al. 2009; Penne & Mizumo i 2012; Ni e al. 2006; Balleine & Os lund 2007; Daw & Shohamy 2008). Goal-di ec ed ac ion selec ion is sensi i e o he con ingencies be ween ac ions and hei ou comes and o he u ili ies o hese ou comes. I depends on ha ing a schema o how he wo ld wo ks ha can be used o de e mine he consequences o ac ions (Redish 2016). This a gues o animals being capable o lea ning he casual ela ionship be ween hei ac ions and he esul ing ou comes, allowing hem con ol o e hei own ac ion based on hei desi e o a pa icula ou come goal (Tolman 1948). This ype o schema con e s animals wi h he abili y o plan ac ions o maximize expec ed ewa ds wi hin hei en i onmen and lexibly in eg a e new in o ma ion like a no el s imuli and o a change in ewa d con ingency. In ha sense, his sys em does no impose ha a speci ic ac ion has o be selec ed bu animals can eso o i o guide beha io (Redish 2016; Johnson e al. 2007). I is hough ha his lexibili y comes wi h a ime cos because i in ol es delibe a ion. This sys em is c i ically dependen on in ac hippocampal unc ion (Pold ack & Packa d 2003; Packa d & McGauch 1996). On he o he side, habi ual ac ion selec ion is sensi i e o he pai ing be ween an eceden s imuli wi hin he en i onmen and ac ions wi hou ega d o hei consequen ial ou comes (Ni e al. 2006). Animals au oma e hei beha io when ac ions eliably led o goals, by de eloping s o ed ac ion chains ha can be eleased a app op ia e imes bu ha , once ini ia ed, end o un o conclusion (Hull 1943; Dez ouli e al. 2014). The habi sys em allows o as ac ion selec ion bu also pe mi s a bi a y s imulus-ac ion pai ings. Hence, i p o ides he basis o simple , non-in eg a ed na iga ion unc ions such as s imulus ecogni ion and app oach and is c i ically dependen on in ac s ia al unc ion (Penne & Mizumo i 2012). 14 F om he pe spec i e o ac ion selec ion, sol ing a na iga ion ask is also sol ing an op imiza ion p oblem ha depends on he p ocessing o spa ial con ex ual in o ma ion (Ni e al. 2006; Penne & Mizumo i 2012). Rein o cemen lea ning is a heo e ical amewo k ha aims a desc ibing he p ocess h ough which an o ganism lea ns o op imize beha io wi hin a decision en i onmen . The decision-making en i onmen s in which ein o cemen lea ning occu s is de ined by a se o s a es (Su on & Ba o 1998), which in he case o na iga ion, can be ep esen ed by loca ions on a maze, such as a co ne o an in e sec ion; a se o possible ac ions ha he decision-make can choose om, such as u n le o a el wes ; and, inally, a se o ules ha he decision-make mus lea n abou he en i onmen h ough pe sis en in e ac ion wi h i , such as whe e is ewa d loca ed and when. In ein o cemen lea ning, ou comes such as ood o wa e ha e nume ical u ili ies, and he impe a i e is o choose ac ions o maximize a long- e m measu e o o al u ili y (Ni e al. 2006; Glimche e al. 2009). The ac ions implemen ed by he decision-make implica e a change om one s a e o ano he , which leads o di e en ou comes. Then he p oblem is o decide which s a egy maximizes u ili y gi en many di e en possibili ies o s a es. In he ein o cemen lea ning amewo k, on one side, using goal- di ec ed sys em implica es ha , when a a choice poin , animals wo k ou he ul ima e ou comes consequen on a sequence o hei ac ions by sea ching h ough a ee o s a es, joined up acco ding o he ac ions ha lead be ween hem, and choosing ac ions based on he ou comes’ cu en u ili ies; on he o he side, habi ual ac ion selec ion sys em implica es ha , when a he choice poin , animals chose an ac ion by compa ing hei ela i e s o ed alues, a he han he ac ual iden i y o he ou comes consequen on sequences o ac ions . 15 In na iga ional con ex s, he ou comes a e o en dependen on he implemen a ion o whole sequences o ac ion choices dependen on he spa ial con ex . Hence, unde s anding how animals selec which pa h o ake o each a goal is also a ma e o spa ial cogni ion. This a ea o knowledge ocus, o example, in in es iga ing how spa ial in o ma ion om he en i onmen is p ocessed and ep esen ed in he b ain (Mose e al. 2008; O’Kee e 1979), how in e nal cues a e used o na iga e he en i onmen (Taube 2007), how animals in eg a e in o ma ion om a a elled pa h o hen e u n o an ini ial posi ion (McNaugh on e al. 2009) o how animals lea n o sol e a maze (Tolman & Pos man 1953). Among o he beha io al pa adigms, mazes ha e been he mos popula se ing o unde s and how animals na iga e he en i onmen (Knie im & Hamil on 2011; Sha ma e al. 2010). Usually, inside mazes, animals a e p esen ed wi h he p oblem o na iga ing h ough a ne wo k o in e connec ing pa hs and dead ends whe e only one o ew op ions lead o ewa d. Gene ally, he unde lying s a egy o hese pa adigms is o le he animal o amilia ize i sel wi h he maze by lea ning spa ial ea u es and ewa d con ingencies, and hen manipula e ei he o bo h. These manipula ions a e designed so ha he obse ed beha io al e ec s can be explained by di e en lea ning ac o s. Fo ins ance, in he case o Tolman–Hull plus maze whe e a s a e ained o collec ing ewa d acco ding wi h a ce ain ule - u n le om he sou h a m o he wes a m- o es wha animals ha e lea ned, ials s a s in a di e en a m and, hen depending on which s a egy he animal is ollowing, he beha io al e ec should be di e en (Ga dne e al. 2013; Rup ech e al. 2014). Ano he example is he Mul iple-T ask whe e a s a e ained o un h ough a cen al ack and o u n le o igh o ood (B own 1946; Schmi ze -To be & Redish 2004). On each day he cen al ack has a 16 di e en shape and he side on which he inal ewa d is placed can change, bu bo h a iables gene ally emain cons an wi hin a day. He e he aim is o de e mine wha animals lea n by obse ing, o ins ance, when animals ake w ong pa hs. Gi en ha one o he aims o his ype o s udies is o unde s and how animals p ocess and in eg a e spa ial in o ma ion, he goal is usually no cued o allow o he animals o e eal hei expec a ion o whe e he goal is, which may b ing up how he animals is using spa ial in o ma ion. An excep ion o his is he Cue-T ask whe e a s a e ained o u n le o igh a a T in e sec ion whe e he co ec choice depends on a senso y cue p o ided on he s em o he T (Johnson & Redish 2007; Tho n e al. 2010). In his case, he aim can be o be e unde s and how animals lea n o associa e s imulus wi h ewa d (Ba nes e al. 2005). O e all, hese pa adigms allow o asking di e en ques ions om how is spa ial in o ma ion p ocessed and lea ned and how possible choices map on o di e en s a egies. While sol ing a maze, he animal is aced wi h choice poin s whe e a decision akes place. A he choice poin , animals should conside which ac ion among al e na i es leads o ewa d and, depending on he unde lying s a egy and on he decision a iables ha he animal is aking in o conside a ion, he selec ed ac ion should be di e en . As in any o he decision-making s udy, he expe imen al s a egy is o pose a p oblem o he subjec whe e, gi en an op imiza ion p oblem, he di e en choices e eal he unde lying p ocesses o ac ion selec ion. 1.4.2 Mo o planning and ac ion selec ion In planning a mo emen , he b ain has o selec one o many possible mo emen plans. De e mining whe he he mo o sys em na u ally p epa es 17 mul iple po en ial mo emen s when me ely p esen ed wi h al e na i e a ge s o ac ion, one o which will subsequen ly be selec ed be o e a mo emen is e en equi ed, is c i ical o he unde s anding o he unde lying mechanisms by which he b ain ini ially ep esen s and makes decisions be ween compe ing op ions in he en i onmen (Galli an e al. 2015; Mu akami & Mainen 2015). As desc ibed be o e, i has been p oposed ha selec ion o ac ions may be pe o med h ough compe i ion among neu ons p epa ing o a ailable ac ions (Cisek 2007). This u he suppo s he iew ha mo o sys ems do no passi ely e lec he esul o comple ed cogni i e p ocesses; a he , i is c ucially linked o he dynamic decision-making p ocess i sel (McKins y e al. 2008; O’Ho a e al. 2016; Hai h e al. 2015). Conside ing he linkage be ween cogni i e and mo o p ocesses, and he concu en p ocessing o mul iple mo o plans, compe ing cogni i e s a es a e likely o p epa e co esponding plans in pa allel (Song & Nakayama 2009; Cisek 2012). Hence, he quali y o he in e ac ions be ween po en ial ac ions could, in po en ial, spill o he mo emen execu ion dynamically modula ing eac ion ime o ajec o y. In he pa icula se ing o mo o planning pa adigms, o ins ance, i has been shown ha when eaching o a a ge in he p esence o a dis ac o , whe e he dis ac o a o ded a mo emen wi h he o he hand, esul ed in highe eac ion ime in compa ison when he dis ac o s a o ded esponses o he same hand (Ray e al. 2014). Mo eo e , he e is e idence ha he numbe o ac ions unde conside a ion o selec ion modula es eac ion ime ha inc eases wi h he inc ease in he numbe o possible each a ge s (Chu chland e al. 2008). E idence o dynamical in e ac ions be ween ac ion plans has come om kinema ic s udies o eye and a m mo emen s made o a ge s in he p esence 18 o dis ac o s in ‘go-be o e-you-know’ asks. I has been obse ed ha he ajec o y o a each o a a ge is in luenced by he p esence and placemen o dis ac e s (Welsh e al. 1999; Tippe e al. 1998), sugges ing ha mul iple a ge and dis ac o ela ed di ec ional signals coexis and in e e e, in neu al popula ions speci ying each di ec ion. When a a ge and dis ac o appea in close p oximi y, he endpoin s o a ge -di ec ed eye and a m mo emen s o en land be ween hem, an obse a ion which has been e med ‘spa ial a e aging’ (Van De S igchel & Theeuwes 2006; Egge e al. 2002; Theeuwes e al. 1998). In a case whe e he p esen ed a ge s co esponded o po en ial each loca ions and no as dis ac o , i has been shown ha , ajec o ies a e spa ially sensi i e o he p obabilis ic dis ibu ion o a ge s wi hin he display. Speci ically, when p esen ed wi h wo o h ee a ge displays, subjec s ini ia e hei eaches owa d an in e media y o a e aged loca ion be o e co ec ing he ajec o y o he mo emen owa ds he cued a ge loca ion. This indica es ha he obse ed spa ial a e aging o he eaching ajec o y depends no only on he spa ial posi ion o he s imulus in he display, like in he p e iously desc ibed case o he dis ac o s, bu also on he p obabili y ha hese s imulus a e a ge s ha a o d each mo emen s (Chapman e al. 2010). The e has been some deba e abou whe he he po en ial each a ge s we e being encoded in isual e sus mo o coo dina es so ha he spa ial a e aging was a consequence o isual a e aging bu , ecen ly, i was shown ha by in oducing an obs acle in some ials while subjec s whe e sol ing a ‘go-be o e-you-know’ ask gene a ed a co ec spa ial a e aged each ajec o y p edic ed by a e aging o mo o a ge loca ions a he han a e aging o isual a ge loca ions (S ewa e al. 2014). One o he hypo hesis ha has been pu o wa d o explain all hese beha io al obse a ions is ha closely spaced isual s imuli ( a ge s o 19 dis ac o s) c ea e o e lapping hills o ac i i y in he co esponding mo o maps o s uc u es in ol ed in mo emen s o he eye in s uc u es like SC (McPeek 2003), and a m in s uc u es like mo o co ex (Geo gopoulos e al. 1988) wi h he inal mo emen ec o being de e mined by he a e aging o hese signals (Tippe e al. 2001). Unde his iew, bo h a ge s and dis ac o s a e he e o e hough o be ini ially ep esen ed as po en ial a ge s by he isuo-mo o sys em (McPeek & Kelle 2004). In ag eemen wi h his in e p e a ion, i was ound ha ac i i y in he SC appea s g ea es a an in e media y loca ion when spa ial a e aging o eye mo emen occu s (Glimche & Spa ks 1993) a guing o he simul aneous ini ia ion o mul iple mo emen s and consequen a e aging o he esul an ac i i y. On he o he side, i has also been shown ha in a cued- esponse delayed ask, when wo a ge s o di e en alue a e p esen ed be o e he go signal and, a e he goal signal, he less alued a ge is cued o each, he a e age ajec o y ends o be ini ially de ia ed owa ds he highe alue a ge (Pas o -Be nie e al. 2012). Cong uen ly wi h his iew, o ins ance, i has been obse ed ha neu al ac i i y co ela ed wi h mo emen di ec ion is p e- shaped by in o ma ion abou he a ailable mo o choices (Ku a a 1993). Fo example, neu al ac i i y ela ed o an ac ion ends o be s onge i he ac ion is mo e likely o yields highe ewa ds (Kable & Glimche 2009; Sug ue e al. 2005). This obse a ion would again en o ce he in e p e a ion ha he e is an unde lying ongoing compe i ion be ween he wo ep esen a ions o he wo each mo emen s bu he one ha is mo e alued is winning be o e he cue is p esen ed, and hen when he cued signal is p esen ed he subjec has o upda e he alue o each a ge and ake he each ha leads o ewa d. Mo e speci ically, he hypo hesis ha compe i ion is implemen ed h ough mu ual inhibi ion inds i s s onge e idence in s udies ha show ha peaks 26 wo was 75 cm. The acks ha composed he g id we e 15 cm wide. The ha d s uc u e o he maze was buil wi h me al ails and acks whe e plas ic shee s pa s ha slid in o he ails sli . Mo eo e , ~2 cm away om each po he e was a whi e isible led encapsula ed in a 10 cm heigh plas ic ube which would signal o ewa d a ailabili y. A Poin G ey FL3-U3-13S2M came a wi h lens Fujinon YV28x28SA-2 was placed a 1.5 me e s abo e he cen e o he maze and ideo eco dings whe e pe o med a 73 ps wi h esolu ion o 1280x960. Expe imen s we e pe o med unde dim ligh (~10 lux) o inc ease he con as be ween backg ound and ligh cue. The ideo was eco ded on he in a ed spec um and he maze was illumina ed by s anda d CCTV in a ed lamps. 2.2.3 Pombal’s maze pa adigm Figu e 1 Top iew o he maze. The maze was an ele a ed g id s uc u e wi h nine wa e po s embedded on he loo , wi h no walls. Each po was dis anced om he closes neighbo by 75 cm and each ack was 15cm wide. Co-localized wi h each wa e po he e was a isible led encapsula ed in a plas ic ube. 27 Each ial ini ia ed when one o he LED ligh s, co-localized wi h each po , was swi ched on. When he animal poked in ha goal po , ligh wen o and wa e ewa d was eleased (Figu e 2). I he animal was ou o he po o mo e han 0.5 s, hen a 100 ms in e - ial in e al (ITI) s a ed, a e which a new ial s a ed. A e ligh was on, he animal could eely choose which pa h o ake o each he goal (Figu e 3). The e was no penal y o no choosing he sho es pa h. I he animal ook mo e han 90 s o poke in, ligh wen o and a new ial s a ed. 2.2.4 Spa ial dis ibu ion o ewa d Spa ial dis ibu ion o ewa d was de ined acco ding o a se o p obabilis ic ules so ha i would be unlikely ha a s could p edic whe e he nex Figu e 2 T ial s uc u e. Schema ic d awing o he succession o e en s in each s a e: ligh was swi ched on in po loca ion A (o ange); a e a poked in ha po (blue ace), ligh wen o and a poked in and ou , signaling ewa d consump ion. When he a was ou o he po o mo e han 500ms, an in e - ial in e al s a ed, las ing o 100ms (g ey). Figu e 3 Example ial. Schema ic d awing o an example ial whe e he s a po co esponded o he loca ion whe e ewa d was las ly a ailable (blue ci cle) and he goal po co esponded o he loca ion whe e ligh was on (yellow shape). Bo h ewa d consump ion pe iod and ITI a e no ep esen ed. 28 ewa d was going o be a ailable be o e ligh u ned on. The aim was o make i cos ly o he a o decide which pa h o ake in he nex ial be o e he ligh was on, and, hence, inc easing he p obabili y ha he decision was happening in a de ined ime window be ween ligh is on and a lea ing s a po . Ne e heless, a he same poin exposing he a s, exclusi ely, o a uni o m dis ibu ion o loca ions implied ha all s a -goal pai s would be equally ep esen ed in ou sample which would lead o he o e ep esen a ion o ials ha we e no p opo ional in in e es o he analysis. In o de o gene a e a bias, we de ined ca ego ies o in e es o he beha io al analysis and biased he dis ibu ion o ewa d loca ion acco dingly. These ca ego ies and gene a ed biases will be explained in dep h in Chap e 3, Sec ion 3.2.1. 2.2.5 T aining The aining sequence consis ed o : (I) handling (5 sessions); (II) 20 minu es ee explo a ion o he maze (1 session); (III) nose poking wa e associa ion whe e ligh was on o six andom po loca ions simul aneously so ha i animal poked in ei he o hem ewa d would be eleased (2 sessions); (IV) ligh wa e associa ion whe e numbe o ligh s pe ial would dec ease wi h inc ease in numbe o ials; (V) when only one ligh was on pe ial he ull session. This p og ession happened ac oss se e al days and i was indi idualized o each a . The a ionale was ha as long as he animals would con inue unning a e ewa ds he exposu e o he ligh -wa e pai ing would happen. Hence, he c i e ion o aining p og ess was he numbe o ials pe day (Figu e 4). Animals signi ican ly do mo e ials in he las session han in he i s session (one-sided, pai ed - es , p=0.0020, n=8). A e his s age, animals we e conside ed o be su icien ly exposed o he maze and o he ligh -wa e pai ing. 29 Ra s ook on a e age 9±0.5 sessions (mean±S.E.M.) om i s session o s age III o he las session o s age IV o each his s age. O e all, each a pe o med ~5000 alid ials, ha is, ials whe e he animal s a ed he ial on he co ec po and inished in he co ec goal po and whe e no inco ec pokes we e pe o med. O e all, each a pe o med ~5000 alid ials, ha is, ials whe e he animal s a ed he ial on he co ec po and inished in he co ec goal po and whe e no inco ec pokes we e pe o med. 2.2.6 Video p ocessing Video acquisi ion and online analysis we e pe o med wi h Bonsai (Lopes e al. 2015), whe e x and y posi ion, body cen oid and o ien a ion o he animal we e ex ac ed om each ame using backg ound sub ac ion and bina y egion analysis by ellipsoidal app oxima ion. A ea occupied by he a co esponded o ~3500 pixels. I was es ima ed ha , om he body cen e o Figu e 4 Numbe o ials inc eases wi h expe ience. Compa ison be ween he mean numbe o ials in he i s aining session and mean numbe o ials in las aining session. E o ba s a e mean±S.E.M. 30 he ip o he snou he e was a ~5 cm dis ance. Gi en ha he e was ish eye like image dis o ion, we applied a ans o ma ion ma ix o he image using a s anda d image co ec ion p ocedu e. This ans o ma ion ma ix was ob ained a e image calib a ion using a checke boa d. A e he ans o ma ion, he e o was equal o 0.9 cm in he x di ec ion, and equal o 0.8 cm in y di ec ion. Smoo hing was applied o x, y alues o cen e o mass in o de o educe noise om di e en sou ces such as ideo ji e o body wobble (Hen e al. 2004). We used a gene alized e sion o he local weigh ed eg ession (LOWESS) echnique (Cle eland 1979) called LOESS. In he la e , each smoo hed alue is gi en by a weigh ed quad a ic leas squa es eg ession ins ead o a weigh ed linea leas squa es eg ession as in he o me . We used a lag o 35 poin s, co esponding o ~ 0.5 s window (Figu e 5). This smoo hing echnique was applied using MATLAB Cu e Fi ing Toolbox. Veloci y o e x and eloci y o e y we e de i ed om cen e o mass posi ion using a s anda d nume ical me hod and hen he module 𝑣 = 𝑠𝑞𝑟𝑡(𝑣𝑥^2 + 𝑣𝑦^2) was calcula ed. Video ame a e ji e was on a e age 3% (da a no shown). 2.2.7 T ajec o ies in o sequences Figu e 5 Example o eloci y ace o e y dimension. In g ey is ep esen ed he ace be o e smoo hing y as a unc ion o ime and in black is ep esen ed he ace a e smoo hing, o a window o ~25s. 31 A e smoo hing, we ans o med x,y coo dina es in o a sequence o numbe s co esponding o po loca ion. This p ocedu e was pe o med acco ding o he ollowing s eps: 1) x,y coo dina es in pixels whe e ans o med in o x,y coo dina es in he eal wo ld o cm, acco ding o a linea ela ion ex ac ed ia a s anda d calib a ion p ocedu e. 2) The cen e o mass posi ion was con e ed in o a sequence o loca ions in an o line p ocedu e. The loca ion o each po was manually de ined by ex ac ing x,y coo dina es ou o a dis o ion-co ec ed ame. Regions o In e es (ROI) whe e de ined by a squa e cen e ed in he x, y coo dina es o each po cen e and side size equal o 13 cm (Figu e 6). Each ROI was assigned he co esponding poke numbe be ween 1 and 9 and whene e he animal occupied one o he 9 a eas, x,y coo dina es whe e a ibu ed he co esponding numbe . When he x,y coo dina es did no ma ch any o he ROIS, 0 was he assigned alue. Finally, all consecu i e equal numbe s whe e educe o one elemen such ha sequence loses Figu e 6 Regions o in e es cen e ed on wa e po s. Example session whe e x,y posi ion (black) is de ec ed in he di e en ROIs associa ed wi h each po (colo ). 32 in o ma ion on how long he animal s ayed in he same ROI and all ze os co esponding o mo emen in be ween ROIs we e emo ed. Hence, in o ma ion abou een e ing ROIs was p ese ed. 2.2.8 Video and BCon ol synch oniza ion Video and nose poke da a whe e aligned using he led isible ligh associa ed o each po so ha in each ial he ime o onse o ligh was gi en bo h by he clock om he BCon ol and he de ec ion o ligh change in he ideo ame. The ime lag be ween he BCon ol da a and he ideo da a was on ~ 17ms. 2.2.9 Da a base All he analysis was pe o med in Ma lab® 2014b (8.4.0.150421, The Ma hWo ks, Inc). Pe each a and each session, we au oma ically gene a ed a ma ix whe e ows co esponded o ials and columns co esponded o di e en ields o wo ypes:  Raw in o ma ion such as ame numbe when ial s a ed, ame numbe when i ended, ime in session when ial s a ed, ime in session when ial ended, s a po , goal po .  P ocessed a iables: sho es pa h leng h, a pa h leng h, sho es pa h {0 – non sho es pa h, 1 – sho es pa h}, u n {0 – s aigh , 1 - u n}, di ec ion o mo emen a a i al o po {Sou h, Wes , No h, Eas }, di ec ion o mo emen a depa u e, among o he s. Each ial was agged wi h he co esponding s a e o he a iable in he co esponding column. Fo example, o selec ing ials whe e s a po was 1, goal po was 3 and a did sho es pa h - 1 we would apply simply a conjunc ion logic ule ex ac ing he ows o which he columns would mee hese c i e ia. All he analysis was pe o med using as basic uni s s a -goal 33 pai s, which implici ly means ha we aligned da a on he s a po nose poke e en un il goal po nose poke e en . 2.2.10 Valida ion c i e ia Fo he alida ing he pa adigm, we e alua ed beha io acco ding o h ee c i e ia: I. Maze amilia iza ion and isual disc imina ion o cue, whe e we ocused on choices be ween sho es pa hs and de ou s. Mo e speci ically we measu ed: a. Median la ency o lea e s a po in ials whe e a s did sho es pa hs and whe e a s did a de ou . La ency o lea e s a po was de ined has he in e al be ween ial s a ed, ha is, ligh u ned on and he ime ha a ook o lea e ROI o he s a po . Fo each s a -goal pai we calcula ed he median la ency o lea e s a po , a e aged ac oss s a -goal pai s o each a and de e mined he a e age ac oss he popula ion. b. Popula ion median p opo ion o sho es pa h ials in compa ison wi h a null model ha selec ed andomly a each choice poin om he se o closes geome ical neighbo s, ha is, a s ochas ic choice model. This sampling was un i e a i ely un il he goal was inally me . Fi s ly, his agen was un eigh imes, one o each a , whe e i was p esen ed wi h he same s a -goal pai s as each o he a s. A e de e mining he median p opo ion o sho es -pa h ials o each s a -goal pai o each un, we calcula ed he median o each un and, inally, he median o medians ac oss he eigh uns. Finally, we un his p ocedu e 1000 imes and calcula ed he 95% con idence in e al. 34 c. Popula ion median p opo ion o sho es pa h ials in compa ison wi h a s ochas ic choice model, as a unc ion o Manha an dis ance o goal. We spli he da a in b. in o ials acco ding o Manha an dis ance o he goal and de e mine median p opo ion o sho es pa h ials ac oss dis ance o goal. d. Popula ion median p opo ion o sho es pa h ials ac oss sessions, a e aining had inished. A e de e mining he median p opo ion o sho es pa h ials o each s a -goal pai o each a , we de e mined he median ac oss s a -goal pai s o each a and, inally, he median o medians ac oss he popula ion. This measu e was calcula ed o each o he i s i e sessions a e aining had ended. II. S eng h o ligh wa e pai ing, whe e we ocused on choices be ween poking o no in non-cued po s. Mo e speci ically, we measu ed he p opo ion o ials whe e animals poked a leas once in a po whe e ligh was no on. We sepa a ed ials we e animals did sho es pa hs and hen quan i ied he median p opo ion o hese whe e animals did inco ec poking. We used he median gi en ha he he e was a high p obabili y ha dis ibu ion ac oss a s would be close o ze o as ime goes by. Hence, we selec ed he i s 5 sessions, a e animals we e conside ed o be ained and om hese we selec ed ials whe e animals did sho es pa hs. We excluded de ou s om his analysis because he main conce n was ha a s would use ligh as di ec ional cue o sho es pa h choice bu hen would be s opping a each wa e po o check o ewa d. III. Biased beha io , whe e we ocused on choices be ween di e en op ions wi hin each se o possible sho es pa hs. Mo e speci ically, we sepa a ed ials in o h ee condi ions: ials whe e animals could 35 selec be ween wo sho es pa h op ions, h ee sho es pa h op ions o six sho es pa h op ions. Fo each s a -goal pai , we a bi a ily ca ego ized each o he op ions (op ion A, op ion B, e c) and de e mined he a e age p opo ion o ials consis en ly ac oss a s co esponding o each op ion. We a e aged o e all s a -goal pai s o each condi ion pe a and hen a e aged ac oss a s. In he condi ion whe e wo sho es pa hs we e a ailable, he compa ison o de e mine a bias was made wi h a shu ling o he da a, so ha in each ial o each s a -goal pai o each a he choice was sampled om a binomial dis ibu ion. Mo eo e , we epea ed he analysis bu sepa a ing da a in o h ee ba ches: he i s 1/3 o he sessions, he second 1/3 o he sessions and las 1/3 o he sessions. The da a was spli in o ba ches ins ead o doing session by session analysis because he e was no enough s a is ical powe o ha . Las ly, he o e all biases we e compa ed be ween he h ee ba ches co esponding o h ee expe ience epochs. 2.3 Resul s 2.3.1 P e e ence o sho es pa hs Fi s ly, in o de o unde s and i animals we e ollowing an op imal s a egy, we compa ed he choices o he animals wi h a null model ha did no ake in o conside a ion any in o ma ion abou he goal, ha is, a each choice poin i selec ed andomly which pa h o ake. Hence, we compa ed he median p opo ion o sho es pa h ials, ac oss he popula ion, wi h a s ochas ic choice model simula ion which sampled andomly a each choice poin (Figu e 7). Animals chose sho es pa hs mo e o en han a s ochas ic choice model simula ion (pe mu a ion es , p<0.002, n=8). Mo e speci ically, 42 The e was no signi ican di e en o e he h ee epochs, o all he h ee Figu e 13 The e is a bias owa ds one o he op ions, in a h ee op ion condi ion. (i) Example s a -goal pai whe e six sho es pa h op ions we e a ailable. Only h ee o he six sho es pa h op ions a e depic ed (A, B and C) (ii) Mean p opo ion o ials co esponding o each o he six op ions in he se o sho es pa hs o one a ac oss all s a -goal pai s ma ching he condi ion. D, E and F a e he co esponding diagonally symme ic pa hs o A, B and C (iii) Mean p opo ion o ials co esponding o each o he 6 op ions in he se o sho es pa hs (one-way ANOVA, p <0.001, n=8). E o ba s a e mean±SEM. 43 condi ions wi h associa ed numbe o op ions. The e was no signi ican di e ence ac oss he h ee ime epochs o all he h ee condi ions (one way Figu e 14 Biases owa ds ce ain op ions do no change ac oss ime. (i) Mean p opo ion o ials whe e op ion A was chosen ou o wo possibili ies ac oss a s (ho izon al line) o e session ba ches (ligh e g ey – i s ba ch, middle g ey –second ba ch, da ke g ey hi d ba ch) (one way ANOVA p=0.6759, n=8). (ii) Mean p opo ion o ials whe e op ions A, B and C whe e chosen ou o h ee possibili ies ac oss a s (ho izon al line) o e session ba ches (ligh e g ey – i s ba ch, middle g ey –second ba ch, da ke g ey hi d ba ch) ( wo way ANOVA, in e ac ion op ion x ime, p=0.1789, n=8). (iii) Mean p opo ion o ials whe e op ions A, B, C, D, E and F whe e chosen ou o six possibili ies ac oss a s (ho izon al line) o e session ba ches (ligh e g ey – i s ba ch, middle g ey – second ba ch and da ke g ey hi d ba ch) ( wo way ANOVA, in e ac ion op ion x epoch p=0.1228, n=8). E o ba s a e mean±SEM. 44 ANOVA, p=0.6759, n=8), h ee op ions ( wo way ANOVA in e ac ion op ion x epoch, p=0.1579, n=8) and, inally, six op ions ( wo way ANOVA in e ac ion op ion x epoch p=0.8322, n=8). This obse a ion indica ed ha p e e ences o ce ain op ions we e obus ac oss ime which possibly mean ha hese biases con e ed an ad an age o he beha io al s a egy ha animals we e using o sol e his ask. 2.4 Discussion Gi en ou in e es in gaining a be e unde s anding abou ac ion selec ion phenomenon, we aimed a de eloping a no el decision-making pa adigm which would ha e simul aneously na iga ion and mo o planning ea u es. Hence, we designed and de eloped wha we e med Pombal’s maze pa adigm whe e a s chose om mul iple op ions in a g id-like maze which pa h o ake o each a wa e ewa d signaled by a isual cue, on a ial-by- ial basis. In o de o alida e he pa adigm p emises, beha io was cha ac e ized and alida ed using di e en c i e ia. Animals we e able o lea n he ask bo h because hey selec ed sho es pa hs mos o he ials and because hey mos ly checked o ewa d in he po s whe e ligh was on. Addi ionally, he p opo ion o sho es pa hs inc eased wi h expe ience o he maze. This could be explained by he need o lea n o isually disc imina e he ligh posi ion being ha , gi en he maze symme ies, he goal could ha e been in one o i e possible loca ions, and/o he need o lea n he maze geome y, being ha i imposed cons ain s on which ac ion sequences could be pe o med (diagonals ac oss po s we e no an op ion) and, consequen ly, which pa hs could be aken o each he goal and om hose which we e a he sho es dis ance. 45 In compa ison wi h a s ochas ic choice model simula ion, animals chose sho es pa hs mo e o en which en o ced ou hypo hesis ha animals we e ollowing an op imal s a egy o using isual in o ma ion abou he goal o selec he sho es pa h. P edic ably, in o de o he s ochas ic choice model o gene a e choices ha would ma ch he op imal s a egy i would jus equi e he in oduc ion o he cons ain whe e op ions ha would no minimize dis ance o he ligh would ha e ze o p obabili y. This would co espond o an ideal subjec wi h a sho es pa h p opo ion equal o one. Animals can pe o m close o wha would be expec ed by an ideal subjec so i was concluded ha animals we e using isual in o ma ion o choose he op imal pa hs o each he cued goal. E en hough he lea ning p ocesses ha a s go h ough while being ained we e no subjec o in es iga ion in his p ojec , he po en ial in e es o doing so will be alluded o in Chap e 5. Pombal’s is a wo dimensional maze ha was designed so ha he e we e mul iple pa hs o each he goal. The solu ion ound was inspi ed by mo o planning asks whe e ypically he s a posi ion and he goal posi ion a e he same o all possible ac ions. Acco dingly, all pa hs we e a he same Manha an dis ance o he goal. Gi en he wo dimensional na u e o he maze, his necessa ily implica ed ha hei geome y had o be di e en and, hence, he necessa y ac ion sequences o a el hose pa hs would also be di e en . Hence, we conside ed ha e en hough he e we e di e en ac ion sequences o a el he same dis ance and o each he same goal, i could be a gued ha he di e ence in cos ac oss ac ion sequences was p obably esidual. Ne e heless, igo ously, he e we e di e ences be ween hese ac ion sequences. In ha sense i was impo an o quan i y i animals showed biases owa ds some op ions o e o he s, o ins ance, i would ha e been possible o choose always he same sho es pa h wi hin a gi en s a - goal pai . We obse ed ha animals consis en ly show biases owa ds ce ain 46 op ions wi hin he se o sho es pa hs. Hence, he obse ed pa e ns o choice seemed an indica ion ha animals we e aking in o conside a ion o he decision a iables besides dis ance o goal. Possibly, hese we e also decision a iables ele an o he op imiza ion p oblem ha animals we e sol ing and hence po en ially e ealing o he unde lying decision-making p ocesses. Ne e heless, he p oblem ha animals we e ying o sol e was also a na iga ion p oblem whe e he goal did no ha e o be in e ed om landma ks o did no ha e o be emembe ed, bu s ill in o ma ion abou he goal posi ion had o be in eg a ed wi h in o ma ion abou maze geome y, somehow. Pombal’s maze is also a cued na iga ion pa adigm. In conclusion, he s a egy ha animals we e using o sol e his op imiza ion p oblem emains o be unde s ood. Speci ically, i is necessa y o unde s and in g ea e dep h how animals we e using:  goal isual in o ma ion. Did animals disc imina e he exac loca ion o he goal among i e possibili ies and only hen hey selec ed he sho es pa h op ion among all possible pa hs? Did animals disc imina e he di ec ion o he ligh in ela ion o hei head posi ion and hen ollowed h ough he pa h ha seemed o be close o he ligh ?  maze geome y in o ma ion. Which a iables a e animals weighing when hey choose be ween di e en op ions? Answe ing hese ques ions will add o a be e unde s anding o he s a egy ha animals used o sol e he ask and i will also allow us o de e mine which pool o ac ions was being conside ed a each ial, hence, be e de ining he ac ion selec ion laye o he decision-making p oblem. These ma e s will be subjec o u he in es iga ion in he ollowing chap e s. 47 3 DISCRETE CHOICES and nea he cen e o mysel he e is an hal open ga e and I ha e ahead he unco e ed mons e he beas ha I gua d and ha gua ds wha li es wi hin Sé gio Godinho, “The maze” (F eely ansla ed by ARF) 48 3.1 In oduc ion In he p e ious chap e , we in oduced Pombal’s maze pa adigm and discussed he op imal s a egy o sol e he unde lying decision-making p oblem. In his chap e , we will in oduce he easoning we used o ame beha io al analysis o he decision-making pa adigm desc ibed in Chap e 2, and, hen, we will summa ize he beha io o a s ained on ha pa adigm. On a i s le el, in Pombal’s maze, animals could eely choose which pa h o ake o each he goal. As long as he animal poked in he po whe e ligh was on, i would ge ewa d, so, in ha sense, his ask did no implica e a o ced choice. Ne e heless, gi en he wa e dep i a ion s a e o he animal, a ionally, he op imal s a egy would be o selec pa hs ha minimize dis ance o he goal, so ha , animals would ge o goal as as as possible, maximizing he ewa d a e. On a second le el, o ials whe e mul iple sho es pa hs op ions we e a ailable, he e we e geome ical ea u es which dis inguished hese pa hs (e.g. numbe o u ns). Bu he e may ha e been o he ea u es ha we e dependen on in e nal biases o he animals, o ins ance, p e e ence o mo ing in a gi en di ec ion. Such ea u es may ha e imposed a cos o a bene i o he s a egy o he animal. Mo e speci ically, we in es iga ed which decision a iables he a s we e aking in o conside a ion, by desc ibing e ealed p e e ences o gi en ypes o pa hs. Ou app oach was based on selec ing and compa ing ial condi ions ha could isola e only one po en ial decision a iable. O e all, we we e aiming in de ining he unde lying na iga ion p oblem. 3.2 Me hods 49 3.2.1 T ial classi ica ion T ials we e collapsed depending on di e en classi ica ion c i e ia: A. Manha an dis ance o he goal: whe e ials we e g ouped acco ding o dis ance in maze segmen s be ween po loca ion and goal po (Figu e 15i); B. Euclidean dis ance o he goal: whe e ials we e g ouped acco ding o Euclidean dis ance in segmen s be ween po loca ion and goal po (Figu e 15ii); C. Numbe o pa h op ions: whe e ials we e g ouped acco ding o numbe o sho es pa h op ions o he goal (Figu e 15iii); D. Numbe o u ns: whe e ials we e g ouped acco ding o he numbe o u ns ac oss sho es pa h op ions (Figu e 15i ); E. Allocen ic ea u es: whe e ials we e g ouped acco ding o whe e he pa hs goes h ough in he maze (Figu e 15 ); F. Egocen ic ea u es: whe e ials we e g ouped acco ding o ea u es ha depend on body posi ion (Figu e 15 i); 50 Figu e 15 T ials ca ego ized acco ding o di e en c i e ia. (i) Schema ic d awings o examples o ou s a -goal pai s whe e he a had o a el be ween s a and goal one maze segmen , wo maze segmen s, h ee maze segmen s o six maze segmen s. (ii) Schema ic d awings o i e examples o s a -goal pai s whe e a would ha e o a el be ween s a and goal he Euclidean dis ance co esponding o one maze segmen , 𝟐 maze segmen s, wo maze segmen s, 𝟓 maze segmen s o 2 𝟐 maze segmen s. (iii) Schema ic d awing o ou examples o s a -goal pai s whe e he e is a se o sho es pa hs wi h one op ion, wo op ions, h ee op ions o six op ions. (i ) Schema ic d awings o h ee examples o s a -goal pai s whe e pa h equi ed one u n, wo u ns o h ee u ns. ( ) Schema ic d awing o an example s a -goal pai whe e he e we e only wo sho es pa hs and whe e one would equi e going h ough he edge and he al e na i e would equi e going h ough he cen e . Schema ic d awing o an example o a s a -goal pai whe e a could selec one o wo sho es pa h op ions whe e i would equi e o go backwa ds in he opposi e di ec ion in ela ion he di ec ion o a i al. 51 In Chap e 2, we men ioned ha spa ial loca ion o ewa d was p obabilis ic bu subjec ed o a bias ac o (Sec ion 2.2.4). He e we explain in mo e de ail wha his en ailed. The e we e 72 possible s a -goal pai s bu , o example, 32 pai s co esponded o ials whe e only one sho es pa h op ion was a ailable. So, in o de o ge a mo e e enly dis ibu ion o ial ypes we p esen ed a s ~7 session days whe e all s a -goal pai s we e equally p obable (uni o m dis ibu ion) and ~15 session days whe e he e was a bias. These biases whe e de ined o maximize he numbe o ials associa ed o di e en ca ego ies o ial ypes. We conside ed h ee main ca ego ies o ials: cen e a oidance ials, which aim was o de e mine i animals showed a oidance o he cen e ; mul iple op ion ials, which aim was o analyze beha io when mo e han wo op ions we e a ailable and, inally, expe ience ials, which aim was allowing he animals o expe ience enough ewa ds in he cen e . Spli ing ials in o hese di e en condi ions allowed o a mo e de ailed in es iga ion o he p e e ences o ce ain pa hs. Fo his analysis only sho es pa h ials we e conside ed. Acco ding o his ame, we de ined h ee main ca ego ies o ials: 1. Mul iple op ion ials, which aim was o analyze beha io when mo e han 2 op ions we e a ailable (Figu e 16). As a a ge , we alloca ed 45% o ials. Figu e 16 Example o s a -goal pai o mul iple op ion analysis. Schema ic d awing o one example s a -goal pai whe e mo e han 2 sho es pa h op ions we e a ailable. 58 ials ac oss a s whe e pa hs ha would minimize Euclidean dis ance o he ligh would be selec ed (Figu e 24). Ra s showed a signi ican p e e ence o pa hs ha minimize Euclidean dis ance o goal (pe mu a ion es , p=0<0.002, n=8). In o de o es o obus ness o his obse a ion, we spli he da a in o s a - goal pai s and measu ed he mean p opo ion o ials o ype A ac oss a s in compa ison wi h sampling om a binomial dis ibu ion (Figu e 25). I was obse ed ha 10/16 s a -goal pai s show means abo e he 95% con idence in e al (pe mu a ion es , p<0.002, n=8). This indica es ha p e e ence o ype A pa hs is obus ac oss di e en loca ions in he maze and ac oss a s, en o cing he hypo hesis ha aking his pa hs is pa o s a egy. Secondly, we analyzed mean p opo ion o ials whe e animals selec ed ype A bu sepa a ed by s a -goal pai s ha go h ough he cen e s h ough he edge and compa ed hese means in o de o unde s and i choosing A was dependen on loca ion in he maze (Figu e 26). Figu e 24 Ra s p e e pa hs ha minimize dis ance o goal as e . (a) Schema ic d awing o an example s a -goal pai whe e, a e i s segmen o pa h, a ype A pa h would ake he a close o he ligh posi ion in e ms o Euclidean dis ance and a ype B pa h would ake a u he away om he ligh posi ion in e ms o Euclidean dis ance. (b) Mean p opo ion o ials whe e a s choose ype A pa hs and he expec ed le el o chance sampled om a binomial dis ibu ion (pe mu a ion- es p<0.002, n=8). E o ba s o a s a e mean±SEM. E o ba s o binomial sampling a e 95% con idence. 59 The mean p opo ion o ype A ials ha go h ough he cen e is Figu e 25 Ra s p e e pa hs ha go h ough he cen e , in a h ee op ions condi ion. (i) Schema ic d awing o example s a -goal pai s whe e he pa hs go h ough he edge (solid line) o h ough he cen e (dashed line). (ii) Mean p opo ion o ials whe e a s choose op ion cen e and he expec ed le el o chance sampled om a binomial dis ibu ion (pe mu a ion- es p<0.002, n=8). E o ba s o a s a e mean±SEM. E o ba s o binomial sampling a e 95% con idence. Figu e 26 P e e ence o ype A is main ained ac oss he majo i y o s a -goal pai s. (i) Schema ic d awing o he numbe ing o each po loca ion. (ii) Mean p opo ion o ials whe e ype A pa hs we e selec ed ac oss a s o each s a -goal pai (whi e ba s) in compa ison wi h and he expec ed le el o chance sampled om a binomial dis ibu ion sampling om a binomial dis ibu ion. E o ba s a e 95% con idence. 60 signi ican ly highe han he mean p opo ion o ype A ials ha go h ough he edge (one sided, pai ed - es , p < 0.002). Mo eo e , gi en ha we, p e iously, obse ed a p e e ence o selec ing pa hs ha go in he opposi e di ec ion o po a i al, in wo op ion condi ion, we epea ed ha analysis o he h ee op ion condi ion. We compa ed mean p opo ion o ials ha a s chose pa hs ha go on he opposi e di ec ion o a i al a he po (Figu e 27) wi h sampling om a binomial dis ibu ion. Animals showed p e e ence o going in he same di ec ion, in he h ee op ion case (pe mu a ion es , p<0.002, n=8). Fo he condi ion whe e six op ions we e a ailable, analyzing he p e e ence o minimizing dis ance o ligh was p oblema ic because he e was he insepa able con ound o he numbe o u ns. Figu e 27 Animals p e e o con inue in he di ec ion ha hey a i ed a he po . Compa ison be ween mean p opo ion o ials ha a s le s a po in he opposi e di ec ion o po a i al wi h sampling om a binomial dis ibu ion (pe mu a ion es , p=0<0.002, n=8). E o ba s o a s a e mean±SEM. E o ba s o binomial sampling a e 95% con idence. 61 3.3.4 P e e ence o geome ical ea u es: numbe o u ns Ano he ac o ha migh ha e played a ole in p e e ences o ce ain op ions o e o he s was he numbe o u ns. I he aim o he animal was o each goal as as as possible, making a u n physically implied a educ ion in eloci y which could in oduce a cos o ce ain op ions. In o de o in es iga e i animals had p e e ence o doing less u ns, we analyzed ials whe e h ee sho es pa h op ions we e a ailable and whe e animals selec ed be ween pa hs ha ake one o wo u ns. Wi hin ials ha equi e one u n, we selec ed only he ials whe e he e was a di e gence poin be ween he pa h wi h one u n and he pa h wi h wo u ns. We obse ed ha a s p e e ed pa hs ha implica e less u ns, bo h in he ials whe e h ee op ions we e a ailable (Figu e 28) (pe mu a ion es , p<0.002, n=8). Figu e 28 Animals p e e pa hs wi h one u n. (i) In h ee op ion ials, he e we e wo ypes o condi ions: wo pa hs ha equi e one u n and one pa h ha equi ed wo u ns .(ii) Compa ison be ween mean p opo ion o ials ac oss he popula ion whe e op ion wi h one u n was selec ed and he expec ed le el o chance sampled om a binomial dis ibu ion (pe mu a ion es , p<0.002, n=8). E o ba s o a s a e mean±SEM. E o ba s o binomial sampling a e 95% con idence. 62 Addi ionally, we quan i ied i he e was a di e ence be ween he in e als o ime ha a s ook o a el h ough pa hs wi h di e en numbe o u ns. We calcula ed he mean mo emen ime o each op ion ype (Figu e 29). The mean mo emen ime o one u n was signi ican ly lowe han o pa hs whe e a s ook wo u ns (one-sided, pai ed - es , p<0.001, n=8). As p e iously, o he condi ion whe e six op ions we e a ailable, analyzing he p e e ence o numbe o u ns was p oblema ic because he e was he insepa able con ound o minimizing dis ance o ligh . 3.3.5 Compe i ion be ween op ions In he condi ion whe e h ee op ions we e a ailable, be ween he wo pa hs ha minimized he Euclidean dis ance o he ligh he e was one which had less u ns, hence, we would no expec ha hose would be compe ing choices. None heless, in he condi ion whe e six op ions we e a ailable animals we e p esen ed wi h ials whe e, he e may ha e been compe i ion be ween op ions. In he six op ions condi ion, he p e e ence o minimizing Euclidean dis ance o he ligh and he p e e ence o minimizing numbe o Figu e 29 Animals a e as e o each he goal when aking a pa h wi h one u n. Compa ing mean mo emen ime ac oss popula ion be ween ials whe e animals ook a one u n pa h and ials whe e animals ook a wo u n pa h (pai ed - es , p<0.001, n=8). E o ba is SEM. 63 u ns we e compe ing because he pa hs ha bes minimize he Euclidean dis ance o ligh a e he ones wi h mo e u ns (Figu e 30). A his poin we could pu o wa d a se o p edic ions o p e e ences in he six op ion condi ion gi en hese minimiza ion ac o s and hen check i hese would ma ch a ’s beha io (Figu e 31), seemingly o wha was p e iously obse ed o he h ee op ion condi ion (3.3.6). We obse ed ha he popula ion da a did no ma ch he p edic ions associa ed o each o he wo minimiza ion ac o s. Fo ins ance, acco ding o he minimiza ion Euclidean dis ance, pa hs A and B should ne e be chosen, which animals did, simila ly, acco ding o he minimiza ion o numbe o u ns, pa hs B and F should ne e be chosen. Figu e 30 T ial ype A minimizes numbe o u ns and ial ype C minimizes Euclidean dis ance o he goal. Schema ic d awing o wo example pa hs, in he six op ion condi ion, whe e A would minimize he numbe o u ns bu no Euclidean dis ance o goal and whe e C would minimize Euclidean dis ance o goal bu no he numbe o u ns. 64 In he end, he pa h ha was he bes comp omise be ween he wo ac o s Figu e 31 Di e en ac o s de ine di e en expec ed p opo ions o each op ion. (i) Example s a -goal whe e six sho es pa hs wi h di e en ia ed geome y (A, B and C and hei symme ical pa hs D, E, F no ep esen ed) we e a ailable (le panel) and popula ion mean p opo ion o ials co esponding o each op ion. (ii) Choice p obabili ies associa ed o each in e sec ion gi en he c i e ion o minimiza ion o Euclidean dis ance (le panel) and expec ed p opo ions o each op ion, gi en he de i ed condi ional p obabili ies ( igh panel). (iii) Choice p obabili ies associa ed o each in e sec ion gi en he c i e ion o minimiza ion o numbe o u ns (le panel) and expec ed p opo ions o each op ion, gi en he de i ed condi ional p obabili ies ( igh panel). 65 was pa h B which did no ma ch he mos p e e ed choice by he animals. In o de o quan i y wi h mo e de ail he pa e ns o choices in he six op ion condi ion, i s ly, we quan i ied he mean p opo ion o ials whe e a s choose ype A ials, a e he ha ing c ossed he i s segmen o pa h (Figu e 32). Symme ical ials we e collapsed so ha he op h ee op ions we e mapped o he h ee bo om op ions. This case is he only symme y axis ac oss op ions, ha is, he e is no pa h wi h same geome y ha could go h ough he cen e o h ough he edge. The aim o his mapping was o make i equi alen in condi ions o he analysis o h ee op ions. Figu e 32 Ra s don' p e e pa hs ha minimize Euclidean dis ance o goal. (i) Schema ic d awing o an example s a -goal pai whe e, a e he i s ini ial segmen , ype A pa h pu a u he away om ligh and ype B pa h pu a u he away om ligh , in e ms o Euclidean dis ance. (ii) Compa ison be ween mean p opo ion o ials whe e a s choose ype A ials and sampling om a binomial dis ibu ion (pe mu a ion es , p <0.002, n=8). E o ba is SEM. 66 We obse ed ha a s p e e ed o ake ype A pa hs bellow chance (pe mu a ion es , p <0.002, n=8). Fo he six op ion condi ion, animals did no show p e e ence o s aying close o he ligh posi ion. Second, we quan i ied he mean p opo ion o ials whe e a s could choose be ween aking one, wo o h ee u ns pa hs. Animals p e e o ake pa hs wi h one u n (Figu e 33) (one-way ANOVA, pos -hoc Tukey-K ame es , p < 0.001, 1 u n x 2 u ns, p <0.001, 1 u n x 3 u ns, p<0.001). We also quan i ied mo emen ime as a unc ion o he numbe o u ns o he pa h (Figu e 34) and he e was no a signi ican di e ence be ween he mo emen imes o pa hs wi h di e en numbe o u ns (one-way ANOVA, p= 0.4760, n=8). Figu e 33 Animals p e e pa hs wi h one u n. (i) In six op ion ials, he e a e h ee ypes o pa hs: pa hs ha equi e one u n, pa hs ha equi e wo u ns and pa hs ha equi e h ee u ns (ii) Compa ison be ween median p opo ion o ials ac oss he popula ion o each op ion wi h associa ed di e en numbe o u ns (one-way ANOVA, pos -hoc Tukey-K ame , p= 2.2743e-06, 1 u n x 2 u ns, p< 0.001, 1 u n x 3 u ns, p<0.001, n=8). E o ba is SEM. 67 Going back o he ques ion o wha animals would do i he e was a con lic be ween minimizing Euclidean dis ance and he numbe o u ns, i seemed ha animals p e e ed o ake pa hs wi h less u ns o e minimizing he Euclidean dis ance o he ligh , e en hough mo emen ime was no di e en o pa hs wi h di e en numbe o u ns. 3.3.6 Quali a i e amewo k We iden i ied wha we e wo po en ially key ea u es o beha io . I seems ha animals p e e ed o choose pa hs ha minimized Euclidean dis ance o he goal and pa hs ha minimize he numbe o u ns. On ha accoun , we could ask: how well can hese p emises explain all he obse ed choices? In o de o answe his ques ion we de i ed analy ically he expec ed p obabili ies associa ed o di e en biases and compa ed hose wi h he beha io al esul s. Fo each in e sec ion, depending on each bias ac o , we de ined he p obabili y o choosing one o he di e gen segmen s acco ding o each minimiza ion ac o and hen we calcula ed he espec i e expec ed Figu e 34 Mo emen ime is independen o numbe o u ns. Compa ing mean mo emen ime ac oss popula ion be ween ials whe e animals ook one u n pa hs, wo u n pa hs and ials whe e animals ook h ee u n pa hs (one-way ANOVA, p= 0.4760, n=8). E o ba is SEM. 74 A his poin , i emains o be be e unde s ood which o he abo e desc ibed algo i hms a e being implemen ed. Do animals main ain a look up able which hey consul a he beginning o each ial o conside he se o po en ial ac ion sequences o each he goal? Al e na i ely, do animals make sequen ial decisions along he pa h aking ad an age o each choice poin ? O is a combina ion o hese wo algo i hmic solu ions, whe e animals lea e he s a po wi h a pa h in mind bu hen hey ake he choice poin as a decision a o dance o e-e alua e p e ious choice? Ce ainly, in es iga ing beha io al co ela es o choice dynamics could help shed ligh in o hese ques ions, as i has been shown by he s udy o eaching and eye ajec o ies in equi alen con ex s such as mo o planning (Ray e al. 2014; Kau man e al. 2015; Cos e al. 2014; Song & Nakayama 2008). 75 4 DYNAMICS OF CHOICE I eel ha he mind is almos like a maze whe e each s ee has un undis inguishable name whe e each choice is made a he las momen Se gio Godinho, in “The maze” (F eely ansla ed by ARF) 76 4.1 In oduc ion In he p e ious chap e , we in es iga ed disc e e choices in o de o unde s and which s a egies animals we e using o sol e Pombal’s maze pa adigm and discussed which decision a iables we e animals aking in o accoun . In his chap e , we in es iga ed dynamics o ac ion such as la ency o s a mo emen , eloci y and ajec o y ac oss ypes o pa hs wi h he aim o ela e hese dynamics wi h decision-making p ocesses. Mo e speci ically, we wan ed o de e mine i ajec o y and/o eloci y p o iles could e eal co ela ions wi h con lic be ween po en ial ac ions o commi men wi h a choice. The aim was o gain a be e unde s anding o he unde lying s a egy o decision. Ou hypo hesis was ha i animals we e weighing op ions sequen ially ac oss choice poin s we would no be able o sepa a e ajec o ies o di e en ypes o pa hs un il animals eached choice poin s, simila ly wi h he phenomenon o spa ial a e aging desc ibed in mo o planning Van De S igchel & Theeuwes 2006; Egge e al. 2002; Theeuwes e al. 1998; addi ionally, we hypo hesized ha i animals we e deciding sequen ially, we would expec a eloci y dec ease while eaching a choice poin , simila ly wi h wha was obse ed in a ask whe e eac ion ime inc eased when las minu e choice was made a ailable (Kau man e al. 2015). Fi s , we will summa ize he dynamics o he animal’ beha io in di e en ial ypes. Secondly, we will in es iga e he hypo heses abou dynamics o choice abo e desc ibed. 4.2 Me hods 77 4.2.1 Spa ial window o in e es Fo his chap e , we mainly ocused on a speci ic window o in e es o analyze beha io (Figu e 37). Mainly, he aim was o s udy speci ically wha happened jus be o e he i s choice poin . In e ms o condi ions, he ocus was he case o h ee op ions whe e, a he i s choice poin , he e a e only wo possible symme ical pa hs di e ging. 4.2.2 Spa ial da a alignmen and binning In o de o allow o compa ison be ween di e en s a -goal pai s, ajec o ies we e linea ly ans o med o be aligned o he x,y coo dina es o he cen e o po 1 (le mos bo om po ), o always go upwa ds and o he le (Figu e 38). A e wa ds, aw ajec o ies and eloci ies (Figu e 39) whe e binned in he y di ec ion (Figu e 40). Figu e 37 Window o in e es o beha io al analysis. Schema ic d awing o he spa ial window used o analysis ( ed box) which co esponds o he space be ween lea ing he s a po and he i s in e sec ion (g ey dashed box). 78 I was inhe en o he binning p ocess he educ ion o in o ma ion o e he chosen dimension (y in his case) so i mean ha o analyzing pa s o he ajec o y whe e he dominan dimension o mo emen is along x dimension, his binning was no app op ia e. Figu e 38 Linea ans o ma ion o ajec o ies o spa ial alignmen . i) Two example ials whe e ajec o ies was linea ly ans o med o ge he ajec o ies all aligned ii) T ans o med ajec o ies, aligned on he same posi ion. Figu e 39 Raw eloci ies and co esponding ajec o ies. (i) T ajec o y p o ile o h ee di e en ials, whe e each colo co esponds o a ial. (ii) Veloci y p o iles co esponden o ajec o ies in (i) wi h same colo ma ch. 79 4.2.3 Veloci y exclusion c i e ion In some ials, ajec o ies show ha a s le he choice poin ROI and hen e-en e ed i (Figu e 41). This c i e ion was de ined as a ial exclusion ac o Figu e 41 Example ial dynamics p o ile whe e one a le he choice poin ROI and hen e- en e ed i . (i) T ajec o y co esponding o a ial whe e a le he in e sec ion ROI and e-en e ed i (ii) de i ed eloci y. Figu e 40 Examples o binned ajec o ies and co esponding eloci ies. (i) T ajec o ies binned o e y o all ials o one s a -goal pai o he same a (g ey aces) wi h associa ed mean (ligh blue) (ii) Co esponding eloci ies wi h he mean ace (ligh blue). 80 because hese ials could be, po en ially, beha io al co ela es o changes o he decision (Bu k e al. 2014; Song e al. 2008). These ials ep esen ed less han 1% o o e all ials pe s a -goal pai (da a no shown). 4.2.4 ROC analysis In o de o de e mine disc iminabili y be ween wo di e en ypes o pa hs we used ecei e ope an cha ac e is ic cu e and he espec i e measu e o disc iminabili y, a ea unde he cu e (AUC) (Ma ill & Elec onics 1956). This analysis was used bo h o disc imina ing ajec o ies o eloci ies. Mo e speci ically, i s , we sepa a ed ials o each s a -goal pai by pa h ype; second, we pe o med he ROC analysis o e he wo ypes o pa h pe 5cm bin o y; hi d, we de e mined he mean AUC as a unc ion y ac oss s a -goal pai s o each a and, inally, we a e aged ac oss a s. Each s a - goal pai had o ha e a leas 7 ials o each o he wo condi ions and bo h dis ibu ions had o ha e he same size. Fo ha , we de e mined he smalles sample size be ween he wo condi ions and used he n samples om he dis ibu ion wi h mo e samples, whe e n co esponded o he size o he smalles dis ibu ion. Gi en ha his ype o analysis is pe o med as a unc ion o spa ial loca ion, o he compa ison be ween o e lapping pa h ypes we pai ed each s a -goal pai o one o he pa h ypes wi h he Figu e 42 Pai ing o s a -goal pai s acco ding o spa ial o e lap. (i) Example compa ison be ween wo o e lapping pa h ypes whe e pa hs s a in he s a po and comple ely o e lap in space (ii) Two examples o pai ing be ween wo ial ypes whe e pa hs om he wo di e en ypes whe e o e lapping in space. 81 co esponding o e lapping s a -goal pai o he o he pa h ype (Figu e 42). This was o accoun o po en ial e ec s wi h image dis o ion e o s (0). 4.3 Resul s Fo decision-making p ocesses i has been shown ha ime ela ed measu es such as eac ion ime and mo emen ime ha e p o en e y in o ma i e beha io al co ela es (Ba ack & Gold 2016). Seemingly, we conside ed la ency o lea e s a po ROI, a e ligh was on, as a po en ial decision- making co ela e. 4.3.1 La ency o lea e he s a po We s a ed by compa ing he median la encies o lea e s a po in ials whe e animals selec ed sho es pa h in compa ison wi h ials we e animals did de ou s (Figu e 43). The hypo hesis was ha i , in gene al, animals we e aiming a eaching he goal as as as possible, i ials whe e animals did de ou s we e di e en al eady a beginning o he ial, indica ing an explici Figu e 43 Animals a e as e o lea e s a po when hey a e abou o do a sho es pa h. Mean la ency o lea e s a po o ials whe e a s choose sho es pa hs o de ou s (pai ed es , one-sided, p < 0.001, n=8). E o ba s a e SEM. 82 disengagemen in hose ials. Ra s we e, on a e age, signi ican ly as e o lea e he s a po in sho es pa h ials, in compa ison wi h ials whe e a s chose a de ou pa h (pai ed es , one-sided, p<0.001, n=8). This could be in e p e ed as i animals we e less mo i a ed o pu sue ewa d, om he beginning o he ial, when hey we e abou o choose a de ou . Al e na i ely, i could be hough ha his we e cases o ials we e isual s imulus was no eadily de ec ed, o ins ance, due o animal being ben o e he edge o maze; we conside ed ha , as soon as he animal le he s a po , independen ly o he posi ion, ligh would be isible so aking a sho es pa h would s ill be possible. Wi hin ials whe e animals selec ed sho es pa hs, we conside ed ha la ency o lea e s a po was po en ially associa ed wi h he di icul y o making a decision when mo e op ions we e in ol ed (Chu chland e al. 2008). Hence, we hypo hesized ha , i he e was an associa ed cogni i e weigh o conside ing mul iple op ions han la ency o lea e s a po would inc ease wi h he numbe o op imal op ions. Acco dingly, we compa ed mean la ency o ini ia e mo emen as a unc ion o he numbe o op ions (Figu e 44). Mul iple compa isons be ween pai s o numbe o op ions show signi ican di e ences excep be ween wo op ions and h ee op ions condi ion ( wo-way ANOVA, ac o numbe o op ions, p<0.001, pos -hoc Tukey K ame , *p<0.05, **p<0.01 n=8). Ne e heless, he numbe o op ions is co ela ed wi h dis ance o goal so we sepa a ed ials whe e only one op ion was a ailable bu dis ance o goal was di e en (Figu e 45i). The aim was o de e mine i , independen ly om he numbe o op ions, he e was an e ec associa ed o dis ance o he goal. Mo e speci ically, we hypo hesized ha la ency o lea e s a po would inc ease wi h he Euclidean dis ance o he ligh . 83 We compa ed mean la encies be ween ials whe e, in one case, Euclidean dis ance o he goal was one maze segmen and, in he o he case, Euclidean dis ance o he goal was wo maze segmen s bu bo h condi ions only comp ised one sho es pa h o he goal (Figu e 45ii). We obse ed ha animals we e signi ican ly as e o lea e s a po when he goal was close (pai ed - es , p= 0.0059, n=8). This would a gue o he hypo hesis ha he la ency o ini ia e mo emen was co ela ed wi h di icul y in disc imina ing o ligh posi ion. One al e na i e in e p e a ion o his obse a ion was ha animals we e mo e mo i a ed o pu sue an easie goal (Reppe e al. 2015). Figu e 44 Animals ake mo e ime o lea e he s a po ROI when mo e op ions a e a ailable. (i) Schema ic d awing o examples o s a -goal pai s whe e one, wo, h ee o six op ions a e a ailable o each he goal (ii) Compa ison be ween mean la encies o lea e s a po as a unc ion o numbe o op ions ( wo-way ANOVA, ac o numbe o op ions , p<0.001, pos -hoc Tukey K ame , *p<0.05, **p<0.01 n=8). E o ba is SEM. 90 (Figu e 50i ). The disc iminabili y p og essi ely inc eased un il a maximum a 45 cm, ha is, 9.5 cm away om he choice poin (pe mu a ion es , *p<0.01). On one way, i seems ha a i ing a he choice poin when i o e ed mo e han one op imal op ion seemed o maximally dec ease eloci y which a gued o conside ing his obse a ion as a beha io al co ela e o compe ion be ween di e en op ions. Mo eo e , eloci y associa ed o ials Figu e 50 Animals slow down mo e when choice poin o e s wo op imal pa h op ions. (i) Schema ic d awing o an example ype A pa h and ype B, whe e hese wo pa h ypes a e mu ually exclusi e in he same ial. (ii) Mean eloci y o ype A pa h (solid line) and o ype B pa h (dashed line) as a unc ion o posi ion, o one a ac oss all s a -goal pai s (n=16). The shaded a ea is SD (iii) Mean eloci y o ype A pa h (solid line) and o ype B pa h (dashed line) as a unc ion o posi ion ac oss he popula ion (n=8). The shaded a ea is SEM. (i ) ROC a ea unde he cu e as a unc ion o y posi ion be ween dis ibu ion o eloci ies o ype A pa h and dis ibu ion o eloci ies o ype B pa h ac oss popula ion (n=8) compa ed wi h shu led da a (ligh g ey shade) (pe mu a ion es , *p< 0.01). Shaded da k a ea is SEM. 91 whe e choice poin only o e ed one op imal op ion was signi ican ly highe as soon as he a le he s a po ROI (pe mu a ion es , *p<0.01, n=8). This could be in e p e ed as an inc eased mo i a ion o pu sue a close goal o an e ece o he ligh posi ion being close and hence, easie o disc imina e. To y o disen angle i dec ease in eloci y was a ligh disc imina ion co ela e, we compa ed pa h ypes whe e he i s choice poin o e ed mo e han one non-op imal op ion bu he dis ance o ligh was di e en (Figu e 51). Hence, we would expec ha animals would slow down mo e while eaching he choice poin i he ligh was u he away as in being ha de o disc imina e. The e is no signi ican di e ence be ween he eloci y p o iles o he wo ypes o pa hs which makes i less p obable ha he dec ease in eloci y is due o isual disc imina ion. Be o e, i was also conside ed he hypo hesis ha his dec ease in eloci y could be ela e o he physical p esence o he po on he loo . Agains his hypo hesis is he obse a ion ha he dec ease in eloci y a he a i al o he choice poin was bigge when he e we e mo e op imal pa h op ions. O e all, i could be a gued ha he obse ed dec ease in eloci y is ela ed o a delibe a ion p ocess ha akes place due o he p esence o choice poin which a o ds a decision. 92 4.3.4 T ajec o ies o compe ing pa hs I ha is he case hen we would expec ha he ajec o ies o compe ing pa h op ions would only become sepa able a e he delibe a ion p ocess a he a i al o he choice poin as a co ela e o esolu ion o he decision- making p ocess. This means ha i was expec ed o ind signi ican Figu e 51 Dis ance o he ligh does no a ec eloci y dec ease a he choice poin (i) Schema ic d awing o an example ype A pa h and ype B. (ii) Mean eloci y o ype A pa h (solid line) and o ype B pa h (dashed line) as a unc ion o posi ion, o one a ac oss all s a -goal pai s (n=16). The shaded a ea is SD (iii) Mean eloci y o ype A pa h (solid line) and o ype B pa h (dashed line) as a unc ion o posi ion ac oss he popula ion (n=8). The shaded a ea is SEM. (i ) ROC a ea unde he cu e as a unc ion o y posi ion be ween dis ibu ion o eloci ies o ype A pa h and dis ibu ion o eloci ies o ype B pa h ac oss popula ion (n=8) compa ed wi h shu led da a (ligh g ey shade) (pe mu a ion es , *p< 0.05, n=8). Shaded da k a ea is SEM. 93 disc iminabili y be ween wo ypes o ajec o ies only a he a i al o he choice poin . Hence, we quan i ied a which poin in space was possible o disc imina e be ween ajec o ies o wo concu en pa h ypes. The a ionale was such ha i i could be de e mined a poin whe e di e en pa h ypes could be sepa a ed using ajec o y, his was an indica ion ha a choice had been made. Mo eo e , we would expec ha conside ing he idea o spa ial a e aging (Van De S igchel & Theeuwes 2006; Egge e al. 2002; Theeuwes e al. 1998) ha he ajec o ies would o e lap un il a di e gence o one o he a ge s. Mo e speci ically, we compa ed wo po en ially compe ing pa hs ypes in a condi ion whe e h ee pa h op ions whe e a ailable, each pa h was mu ually exclusi e op ion in a condi ion such ha each ype would o e lap and geome ically di e ge a a choice poin (Figu e 52). Mean disc iminabili y be ween wo ajec o ies was i s signi ican 40 cm a e animals le he s a po ROI (pe mu a ion es , *p<0.01, n=8) which co esponds o 15.5 cm be o e a i al a he choice poin . Gi en ha he dis ance be ween he limi o he s a po ROI and he limi o he choice poin is 54.5 cm, his means ha animals a eled almos wo hi ds o he o he ack, mo e p ecisely 73% un il i was possible o an icipa e which pa h hey we e going o ake. Fu he mo e, we did he same analysis o de e mine he di e gence poin o ajec o ies bu o wo compe ing pa hs om he condi ion whe e six op ions a e a ailable (Figu e 53i). Disc iminabili y be ween ajec o ies was i s signi ican 40 cm a e lea ing he s a po ROI. In bo h condi ions o h ee and six op ions, he poin in space whe e wo compe ing ajec o ies di e ge is he same. 94 This esul is unexpec ed because i seems ha ajec o y di e gence, which we hypo hesized as a beha io al co ela e o commi men o a gi en choice, was aking place 5 cm be o e he maximum dec ease o eloci y a he choice poin , which we hypo hesized as being a beha io al co ela e o compe i ion be ween po en ial ac ions a he choice poin . Figu e 52 T ajec o ies om compe ing op ions in a h ee op ion condi ion di e ge 20 cm a e animal has le he s a po ROI. (i) Schema ic d awing o an example ype A pa h and ype B, whe e hese wo pa h ypes a e mu ually exclusi e in he same ial. (ii) Mean ajec o y o ype A pa h (solid line) and o ype B pa h (dashed line) as a unc ion o posi ion, o one a ac oss all s a - goal pai s (n=16). The shaded a ea is SD (iii) Mean ajec o y o ype A pa h (solid line) and o ype B pa h (dashed line) as a unc ion o posi ion ac oss he popula ion (n=8). The shaded a ea is SEM. (i ) ROC a ea unde he cu e as a unc ion o y posi ion be ween dis ibu ion o ajec o ies o ype A pa h and dis ibu ion o ajec o ies o ype B pa h ac oss popula ion (n=8) compa ed wi h shu led da a (ligh g ey shade) (pe mu a ion es , *p< 0.01). Shaded da k a ea is SEM. 95 4.4 Discussion In his chap e , we in es iga ed dynamics o ac ion such as la ency o s a mo emen , eloci y and ajec o y ac oss ypes o pa hs wi h he aim o ela e hese dynamics wi h decision-making p ocesses. Mo e speci ically, we wan ed o de e mine i ajec o y and/o eloci y p o iles could e eal co ela ions wi h con lic be ween po en ial ac ions o commi men wi h a choice. The main indings a e summa ized below: Figu e 53 T ajec o ies om compe ing op ions in a six op ion condi ion di e ge 20 cm a e animal has le he s a po ROI. (i) Schema ic d awing o an example ype A pa h and ype B, whe e hese wo pa h ypes a e mu ually exclusi e in he same ial. (ii) Mean ajec o y o ype A pa h (solid line) and o ype B pa h (dashed line) as a unc ion o posi ion, o one a ac oss all s a -goal pai s (n=16). The shaded a ea is SD (iii) Mean ajec o y o ype A pa h (solid line) and o ype B pa h (dashed line) as a unc ion o posi ion ac oss he popula ion (n=8). The shaded a ea is SEM. (i ) ROC a ea unde he cu e as a unc ion o y posi ion be ween dis ibu ion o ajec o ies o ype A pa h and dis ibu ion o ajec o ies o ype B pa h ac oss popula ion (n=8) compa ed wi h shu led da a (ligh g ey shade) (pe mu a ion es , *p< 0.01). Shaded da k a ea is SEM. 96  La ency o lea e s a po – we ound ha ha animals le he s a po as e i hey we e abou o do a sho es pa h han i hey we e abou o do a de ou . We in e p e ed his obse a ion as a mo i a ion co ela e such ha , a he beginning o he ial, animals had al eady decided o ake a sho es pa h o no . On he o he hand, he di icul y in disc imina ing he ligh a he beginning o he ial due o body posi ion and o ien a ion could only explain a di e ence in la ency o lea e s a po be ween sho es pa hs and de ou s on he o he di ec ion. I a s would ake less ime o lea e s a po when abou o do a de ou , i could be a gued ha hey le he s a po wi hou enough in o ma ion abou he goal and, hence, choose a non- op imal pa h a he beginning ha comp omised he o e all sho es pa h. Wi hin sho es pa h ials, we ound ha animals ake mo e ime o lea e s a po in ials whe e six op ions a e a ailable which could indica e ha he numbe o op ions a ailable could be inc easing he ime o delibe a e a he s a po . Ne e heless, he e is no di e ence in la ency be ween ials whe e wo op ions o h ee op ions a e a ailable. Gi en ha he numbe o op ions also inc eased wi h he dis ance o he goal which in oduced a po en ial con ound. Acco dingly, in ials whe e he dis ance o he goal is di e en and he e is only one op ion animals ake mo e ime o lea e he s a po which could be ei he associa ed o mo i a ion o di icul y in disc imina ing goal loca ion when he goal is u he away.  Bell-shaped eloci y – we ound ha animals exhibi a bell-shaped eloci y p o ile when a elling be ween wo po s sepa a ed by one maze segmen , as expec ed om mo o planning li e a u e (Flash & Hogan 1985). 97  Cen e a oidance – we ound ha animals each a highe eloci y peak in ials ha ake hem h ough he cen e which is cong uen wi h da a obse ed in he open ield (Lipkind e al. 2004) when he dis ance o goal is o wo segmen s and he e is only one op imal op ion. This esul did no hold h ough o he case whe e h ee op ions we e a ailable. One possible explana ion o his ype o incong uence was ha he cen e a oidance ac o was being supp essed by o he ac o s such minimizing Euclidean ligh .  Veloci y a he choice poin - animals slowed down while a i ing a he choice poin e en hough he e was only one a ailable op imal op ion a ha poin . This obse a ion could be in e p e ed as a consequence o he need o check o he ligh posi ion, gi en he oppo uni y o ee alua e a p e ious decision. Ne e heless, we obse ed no di e ence be ween eloci y p o iles in e ms o dec ease a he choice poin o wo pa h ypes which goals we e a di e en dis ances so we would a gue ha his obse a ion is no explained by isual disc imina ion e ec . Mo eo e , he dec ease in eloci y did no depend on he numbe o non-op imal choices (cen e s edge) which a gues o a gene al e ec o being p esen ed wi h a choice poin , as i he p esence o a choice poin a o ded a decision which in e e ed wi h he cu en mo o plan (Kau man e al. 2015). Adding o his, animals slowed down mo e i he choice poin a o ded mo e han one op imal op ion. A bigge dec ease in eloci y could indica e ha animals ake mo e ime o conside be ween wo op ions when bo h o hem a e op imal han when one is op imal and he o he is no . This could be in e p e ed as i he o e o an i ele an op ion o he op imal s a egy was in e e ing wi h he decision o ake he op imal pa h, equi alen ly as in mo o planning asks whe e 98 dis ac o s in e e e wi h he eaches (Welsh e al. 1999; Tippe e al. 1998). Hence, his obse a ion could be a co ela e o s ong compe i ion be ween wo op imal ac ions o a weak compe i ion be ween an op imal ac ion and a non-op imal ac ion (‘dis ac o ’). I canno be excluded ha , he dec ease in eloci y while a i ing a he choice poin does no ha e a componen associa ed o a biomechanical adap ion (Va e al. 1996) associa ed o he physical p esence o he po on he loo .  T ajec o y di e gence – we ound ha ajec o ies o compe ing pa hs di e ge a 15.5 cm be o e a i al a he choice poin . This could be in e p e ed as a co ela e ha he decision abou he choice a he choice poin was aken a ha loca ion in space, and consequen execu ion o he chosen op ion s a ed. In a way, his would be equi alen o he spa ial a e aging phenomenon obse ed in ‘go- be o e-you-know’ asks (Welsh e al. 1999; Tippe e al. 1998) o in asks whe e each ajec o y was used o epo bina y choices (McKins y e al. 2008), whe e in he la e case he cue o which a ge o mo e o was no de ined ex e nally by he expe imen e bu by in e nal p ocesses, simila ly o Pombal´s maze. Al e na i ely, i could be a gued ha animals had al eady decided which pa h o ake a he beginning o he ial and we e wi hholding he ac ion execu ion un il i was biomechanically necessa y. This seems a less p obable possibili y because i would implica e ha animals would ha e o main ain he memo y o he ou come o he decision un il i was biomechanically necessa y o execu e i . Ye ano he possibili y, is ha he di e gence happened be o e and ha he po en ially insu icien spa ial esolu ion o he ideo ame oge he wi h he small wid h o he ack did no allow o measu able sepa a ion o 99 he ajec o ies. Finally, i canno be excluded ha hese co ela es co espond o a bias o he animals, no e lec ing he commi men owa ds a decision. In conclusion, i seems ha bo h mo i a ion and Euclidean dis ance o he goal may in luence la ency o lea e s a po . Cen e a oidance seems o ha e di e en weigh on he beha io depending on he numbe o op ions and co ela ed Euclidean dis ance o he goal. Addi ionally, he e is e idence ha animals ake choice poin s as a place ha a o ds a decision, whe he he e a e only op imal pa h op ions o no . In ha sense, i could be ha e en hough he e is an ini ial decision abou wha pa h o ake a he choice poin , he o e o he choice poin may lead animals o econside . The hypo hesis o econside a ion is en o ced by he possibili y o de e mine be o e he choice poin which pa h a e animals going o ake om a se o wo compe ing op ions. O e all, i seems ha animals may be making decisions be o e he choice poin bu gi en he o e ed oppo uni y o change minds, animals ee alua e hei p e ious decision. 106 aims a unde s anding ac ion selec ion om he pe spec i e o spa ial p ocessing ocusing on choice poin s and associa ed disc e e choices; and mo o planning, which aims a unde s anding ac ion selec ion om he pe spec i e o mo o con ol ocusing on he con inuous dynamics o mo emen . Pombal’s maze is a cued na iga ion ask whe e animals ha e o ga he spa ial in o ma ion abou he goal, h ough isual s imulus, and choose one pa h o ge he e. F om he pe spec i e o an ac ion selec ion p oblem, animals a e equi ed o choose be ween di e en ac ions – going o wa d, u ning igh o le . I we would conside a s ochas ic choice model ha selec s andomly be ween po en ial ac ions a each choice poin , as i was desc ibed in Chap e 2, i would no mimic he a s’ beha io . In o de o his model o mimic he a beha io , a each choice poin he po en ial ac ions should be conside ed bu hen di e en ype o in o ma ion would bias he choice p obabili y o each ac ion: p e e ence o less u ns, p e e ence o a oiding he cen e , p e e ence o s aying close o he ligh as possible. A bias associa ed o u n a oidance can be gene a ed on he spo a he choice poin by lea ning om p e ious expe ience ha u ning educes eloci y o in oduces ene ge ic cos s – in a s imulus- esponse manne (Penne & Mizumo i 2012; Ni e al. 2006). Ne e heless, a bias associa ed o minimizing Euclidean dis ance would equi e an ex a s ep o compu a ion o e alua ing how much each po en ial ac ion would lead o s aying close o he ligh , in he u u e. Fo de e mining hese ela ionships be ween u u e ac ions and spa ial ela ion o he ligh , we would a gue would be acqui ed h ough a goal-di ec ed s a egy. I could be expec ed ha a e being o e ained in he ask he animals would become habi ual and use a s imulus- ou come associa ion as a ule (Penne & Mizumo i 2012; Ni e al. 2006). Ne e heless, his would esul in o he animal ha ing o emembe a 107 possibly ex ensi e lis o s imulus-ou come associa ions. Ano he aspec ha should bias he p obabili y o choice o a po en ial ac ion is he ac o i he animal is al eady mo ing o no , ha is, i he choice poin coincides wi h he s a loca ion o no , in which, o ins ance, making a u n would ha e a di e en associa ed cos om when he animal is al eady mo ing. O e all, in Pombal’s maze animals a e con on ed wi h he selec ion o ac ions whe e di e en biases change he p obabili y o choice o each o he po en ial ac ions. O e all, in o de o a s ochas ic choice model o mimic a ’s beha io i would ha e o weigh o e he cos and bene i o he di e en ea u es o he po en ial ac ions a he choice poin , engaging in a sequen ial choice s a egy. Ne e heless, his ma hema ical desc ip ion o he ac ion selec ion p oblem associa ed o sol ing Pombal’s maze would equi e he in eg a ion o dynamical p ope ies o p o ide wi h a mo e accu a e desc ip ion o beha io . Hence, in be ween closes neighbo s eloci y would be modelled by a bell-shaped unc ion wi h a modi ica ion because animals do no ully s op a choice poin s, hey educe eloci y and, in ac he educ ion seems o be modula ed by he numbe o op imal choices. Las ly, conside ing a componen ela ed o ajec o ies, he e is an obse a ion ha seems incong uen wi h he hypo hesis ha animals make sequen ial choices along he pa h, which is he esul ha posi ion on he ack an icipa es which choice animals a e going o ake a he choice poin . I is no comple ely clea i his di e gence o pa hs is as a co ela e o commi men owa ds a ce ain choice o a bias. I i is he case ha ajec o y di e gence co ela es wi h commi men his could implica e ha decision-making p ocesses a e no exclusi e om choice poin s bu a e ongoing while animals a e mo ing, which has been a gued is he case in mo o planning ypical asks (Song & Nakayama 2009) so ha he po en ial ac ions a e ep esen ed and in e ac ing dynamically, allowing o lexible adap ion o new po en ial ac ions o 108 changes in plan (Cisek 2012; Song & Nakayama 2009). This hypo hesis would p edic ha e en a e decision has been aken and ac ion is unde execu ion, he e is s ill he possibili y o econside ing (Kau man e al. 2015) which could be seen as in e e ence be ween po en ial ac ions. One obse a ion ha seems o go on his di ec ion is he exis ence o change in mind ype o ials, whe e animals le he choice poin ollowing one di ec ion and hen e u ned o he choice poin o go in he o he di ec ion. I seems, o e all, ha he choice poin and inhe en possibili y i o e s he animal o make choices gene a es an in e e ence wi h decisions p e iously made. This ype o dynamical in e ac ion be ween po en ial ac ions seems o be a he co e o unde s anding ac ion selec ion and he unde lying compe i ion mechanisms. 5.5 Final ema ks and u u e di ec ions The ocus o his hesis was o design and de elop a no el decision-making pa adigm o s udying beha io while animals a e selec ing be ween mul iple ac ions. The pa adigm, due o i s cha ac e is ics o being based on a maze and on a ial s uc u e whe e mul iple op ions lead o goal, s ands bo h in he ields o na iga ion and mo o planning. Hence, we aimed a aming he p oblem ha animals we e sol ing conside ing hese wo amewo ks. On one side, i was necessa y o cha ac e ize which p oblem we e he animals sol ing in e ms o op imiza ion and decision a iables, in o de o de e mine which ype o ac ions we e po en ially compe ing o be selec ed. On he o he side, i was necessa y o be e cha ac e ize he dynamics o mo emen wi h he aim o ind beha io al co ela es o ac ion selec ion ha could be in eg a ed wi h he pa e ns o disc e e choices. In he end, in eg a ing choice pa e ns wi h dynamical co ela es led o he unde s anding ha choice poin s 109 can a o d changes in mind, whe e he p e ious decision compe es wi h new possibili ies. This seems like a pa icula ly ele an window o s udying how po en ial ac ions dynamically in e ac and, hence, gaining a be e unde s anding abou he unde lying mechanisms o ac ion selec ion. In u u e di ec ions, we sugges o do elec ophysiological eco dings o neu al ac i i y in mo o - ela ed a eas while animals a e sol ing Pombal’s maze. Mo e speci ically, we conside ha he window be ween he loca ion whe e di e gence o ajec o ies occu s, un il he loca ion whe e he animal lea es he choice poin , is pa icula ly ele an o sea ching o neu al co ela es o ac ion selec ion. We conside ha one o he b ain a eas which could be a good candida e o ecoding neu al ac i i y is M2 which is an a ea associa ed wi h ac ion planning and spon aneous ac ion ini ia ion (Sul e al. 2012; Mu akami e al. 2014; E lich e al. 2011), conside ed o be homologous o p ima e supplemen a y mo o a eas (Reep e al. 1990; Reep e al. 1986). We would expec o ind ac i i y co ela ed wi h simul aneous ep esen a ion o po en ial ac ions, simila ly wi h wha has been ound in PMd (Cisek & Kalaska 2005). Mo e speci ically, we would expec o cha ac e ize he in e ac ions be ween po en ial ep esen a ions, om he momen whe e animals show ajec o y di e gence onwa ds. These obse a ions could help o disambigua e i ajec o y di e gence is associa ed o commi men owa ds a decision o no . I ha is he case, i would be ele an o see i he e a e no maliza ion e ec s be ween ep esen a ions o di e en ac ions, which would a gue o a compe i ion mechanism aking place be ween po en ial ac ions. Hence, we belie e ha hese obse a ions would con ibu e o gaining a deepe unde s anding abou ac ion selec ion mechanisms and espec i e unde lying ci cui y. 110 6 REFERENCES 111 Alexande , R.M., 1997. A minimum ene gy cos hypo hesis o human a m ajec o ies. Biological cybe ne ics, 76(2), pp.97–105. Ande sen, R.A. & Cui, H., 2009. In en ion, Ac ion Planning, and Decision Making in Pa ie al-F on al Ci cui s. Neu on, 63(5), pp.568–583. Bailey, K.R. & C awley, J.N., 2009. Anxie y- ela ed beha io s in mice. In J. J. Bucca usco, ed. Me hods o Beha io Analysis. Boca Ra on (FL): CRC p ess/Taylo & F ancis. Baldau , D., Cui, H. & Ande sen, R.A., 2008. The Pos e io Pa ie al Co ex Encodes in Pa allel Bo h Goals o Double-Reach Sequences. Jou nal o Neu oscience, 28(40), pp.10081–10089. Balleine, B.W. & Os lund, S.B., 2007. S ill a he choice-poin : ac ion selec ion and ini ia ion in ins umen al condi ioning. Annals o he New Yo k Academy o Sciences, 1104, pp.147–71. Ba ack, D.L. & Gold, J.I., 2016. Tempo al ade-o s in psychophysics. Cu en Opinion in Neu obiology, 37, pp.121–125. Ba nes, T.D. e al., 2005. Ac i i y o s ia al neu ons e lec s dynamic encoding and ecoding o p ocedu al memo ies. Na u e, 437(7062), pp.1158–61. Basso, M. a & Wu z, R.H., 1998. Modula ion o neu onal ac i i y in supe io colliculus by changes in a ge p obabili y. The Jou nal o neu oscience : he o icial jou nal o he Socie y o Neu oscience, 18(18), pp.7519–7534. Basso, M.A. & Wu z, R.H., 1997. Modula ion o neu onal ac i i y by a ge unce ain y. Na u e, 389(Sep embe ), pp.66–69. 112 Bouwknech , J.A. e al., 2007. Di e en ial e ec s o exposu e o low-ligh o high-ligh open- ield on anxie y- ela ed beha io s: Rela ionship o c- Fos exp ession in se o one gic and non-se o one gic neu ons in he do sal aphe nucleus. B ain Resea ch Bulle in, 72(1), pp.32–43. B own, W.L., 1946. The e ec s o a common s a - inish locus on o ien- a ion and beha io in a mul iple- maze. , 1, pp.331–338. Bu k, D. e al., 2014. Mo o e o al e s changes o mind in senso imo o decision making. PLoS ONE, 9(3). Chapman, C.S. e al., 2010. Reaching o he unknown: Mul iple a ge encoding and eal- ime decision-making in a apid each ask. Cogni ion, 116(2), pp.168–176. Chu chland, A.K., Kiani, R. & Shadlen, M.N., 2008. Decision-making wi h mul iple al e na i es. Na u e neu oscience, 11(6), pp.693–702. Cisek, P., 2007. Co ical mechanisms o ac ion selec ion: he a o dance compe i ion hypo hesis. Philosophical ansac ions o he Royal Socie y o London. Se ies B, Biological sciences, 362(1485), pp.1585–1599. Cisek, P., 2006. In eg a ed Neu al P ocesses o De ining Po en ial Ac ions and Deciding be ween Them: A Compu a ional Model. Jou nal o Neu oscience, 26(38), pp.9761–9770. Cisek, P., 2012. Making decisions h ough a dis ibu ed consensus. Cu en Opinion in Neu obiology, 22(6), pp.927–936. Cisek, P. & Kalaska, J.F., 2005. Neu al co ela es o eaching decisions in do sal p emo o co ex: Speci ica ion o mul iple di ec ion choices and inal selec ion o ac ion. Neu on, 45(5), pp.801–814. 113 Cisek, P. & Kalaska, J.F., 2002. Simul aneous encoding o mul iple po en ial each di ec ions in do sal p emo o co ex. Jou nal o neu ophysiology, 87(2), pp.1149–54. Cle eland, W.S., 1979. Robus locally weigh ed eg ession and smoo hing sca e plo s. Co ado, G. & Doya, K., 2007. Unde s anding neu al coding h ough he model-based analysis o decision making. The Jou nal o neu oscience : he o icial jou nal o he Socie y o Neu oscience, 27(31), pp.8178–80. Cos, I., Duque, J. & Cisek, P., 2014. Rapid p edic ion o biomechanical cos s du ing ac ion decisions. Jou nal o neu ophysiology, 112(6), pp.1256– 66. Coul ha d, E.J., Nache , P. & Husain, M., 2008. Con ol o e Con lic du ing Mo emen P epa a ion: Role o Pos e io Pa ie al Co ex. Neu on, 58(1), pp.144–157. Cou lee, C.G. & Hue el, S.A., 2012. The unc ional neu oana omy o decision making: P e on al con ol o hough and ac ion. B ain Resea ch, 1428, pp.3–12. Daw, N.D. & Shohamy, D., 2008. The Cogni i e Neu oscience o Mo i a ion and Lea ning. Social Cogni ion, 26(5), pp.593–620. Deane , R.O., Khe a, A. V. & Pla , M.L., 2005. Monkeys pay pe iew: Adap i e alua ion o social images by hesus macaques. Cu en Biology, 15(6), pp.543–548. Desimone, R. & Duncan, J.S., 1995. Neu al mechanisms o selec i e isual a en ion. Annual Re iew o Neu oscience, 18(1), pp.193–222. 114 Dez ouli, A., Lingawi, N.W. & Balleine, B.W., 2014. Habi s as ac ion sequences: hie a chical ac ion con ol and changes in ou come alue. Philosophical ansac ions o he Royal Socie y o London. Se ies B, Biological sciences, 369(1655), p.20130482–. Egge , T. e al., 2002. Di e en ial e ec o a dis ac o on p ima y saccades and pe cep ual localiza ion. Vision Resea ch, 42(28), pp.2969–2984. E lich, J.C., Bialek, M. & B ody, C.D., 2011. A co ical subs a e o memo y-guided o ien ing in he a . Neu on, 72(2), pp.330–343. Felsen, G. & Mainen, Z.F., 2012. Midb ain con ibu ions o senso imo o decision making. Jou nal o Neu ophysiology, 108(1), pp.135–147. Flash, T. & Hogan, N., 1985. The coo dina ion o a m mo emen s: an expe imen ally con i med ma hema ical model. The Jou nal o neu oscience : he o icial jou nal o he Socie y o Neu oscience, 5(7), pp.1688–1703. Galli an, J.P. e al., 2015. Ac ion plan co-op imiza ion e eals he pa allel encoding o compe ing each mo emen s. Na u e communica ions, 6(May), p.7428. Ga dne , R.S. e al., 2013. A seconda y wo king memo y challenge p ese es p ima y place s a egies despi e o e aining. Lea ning & memo y (Cold Sp ing Ha bo , N.Y.), 20(11), pp.648–56. Geo gopoulos, a P., Ke ne , R.E. & Schwa z, a B., 1988. P ima e mo o co ex and ee a m mo emen s o isual a ge s in h ee-dimensional space. II. Coding o he di ec ion o mo emen by a neu onal popula ion. J.Neu osci., 8(8), pp.2928–2937. 115 Gibson, J.J., 1979. The ecological app oach o isual pe cep ion, Hough on Mi li. Glimche , P.W. e al., 2009. Neu oeconomics, Glimche , P.W. & Spa ks, D.L., 1993. Rep esen a ion o a e aging saccades in he supe io colliculus o he monkey. Expe imen al B ain Resea ch, 95(3), pp.429–435. Gold, J.I. & Shadlen, M.N., 2007. The neu al basis o decision making. Annual e iew o Neu oscienceeu oscience, 30, pp.535–574. Hai h, A.M., Hube deau, D.M. & K akaue , J.W., 2015. Hedging You Be s: In e media e Mo emen s as Op imal Beha io in he Con ex o an Incomple e Decision. PLoS Compu a ional Biology, 11(3), pp.1–21. Hen, I. e al., 2004. The dynamics o spa ial beha io : How can obus smoo hing echniques help? Jou nal o Neu oscience Me hods, 133(1- 2), pp.161–172. Hikosaka, O., Nakamu a, K. & Nakaha a, H., 2006. Basal ganglia o ien eyes o ewa d. Jou nal o neu ophysiology, 95(2), pp.567–584. Hoshi, E. & Tanji, J., 2007. Dis inc ions be ween do sal and en al p emo o a eas: ana omical connec i i y and unc ional p ope ies. Cu en Opinion in Neu obiology, 17(2), pp.234–242. Houk, C.J., 1995. Dis ibu ed modula a chi ec u es linking basal ganglia, ce ebellum, and ce eb al co ex hei ole in planning and con olling ac ion.pd . Ce eb al Co ex, pp.95–110. Hull, C.L., 1943. P inciples o Beha io : an in oduc ion o beha io heo y, Apple on-Cen u y. 122 Taube, J.S., 2007. The head di ec ion signal: o igins and senso y-mo o in eg a ion. Annual e iew o neu oscience, 30, pp.181–207. Theeuwes, J. e al., 1998. Ou eyes do no always go whe e we wan hem o go: Cap u e o he Eyes by New Objec s. Psychological Science, 9(5), pp.379–385. Tho n, C.A. e al., 2010. Di e en ial dynamics o ac i i y changes in do sola e al and do somedial s ia al loops du ing lea ning. Neu on, 66(5), pp.781–795. Tippe , S.P., Howa d, L.A. & Hough on, G., 1998. Ac ion-based mechanisms o a en ion. Philosophical ansac ions o he Royal Socie y o London. Se ies B, Biological sciences, 353(1373), pp.1385– 1393. Tolman, E.C., 1948. Cogni i e maps in a s and men. The Psychological Re iew, 55(4), pp.189–208. Tolman, E.C., 1938. The de e mine s o beha io a a choice poin . In Ho he sall, ed. His o y o Psychology. p. 494. Tolman, E.C. & Pos man, L., 1953. Lea ning. Annual Re iew o Psychology, 5, pp.27–56. T e sky, A. & Kahneman, D., 1981. The F aming o Decisions and he Psychology o Choice. Science, 211(4481), pp.453–458. Uchida, N. & Mainen, Z.F., 2003. Speed and accu acy o ol ac o y disc imina ion in he a . Na u e neu oscience, 6(11), pp.1224–1229. Va, R. a, Ebenspe ge , L. a & Bozino ic, F., 1996. The in luence o habi a on a el speed , in e mi en locomo ion , and igilance in a diu nal 123 oden . Beha io al Ecology, 13(2), pp.182–187. Welsh, T.N., Ellio , D. & Weeks, D.J., 1999. Hand de ia ions owa d dis ac o s. E idence o esponse compe i ion. Expe imen al B ain Resea ch, 127(2), pp.207–212. Whi lock, J.R. e al., 2008. Na iga ing om hippocampus o pa ie al co ex. P oceedings o he Na ional Academy o Sciences, 105(39), pp.14755– 14762. Wolpe , D.M. & Landy, M.S., 2012. Mo o con ol is decision-making. Cu en Opinion in Neu obiology, 22(6), pp.996–1003. Wunde lich, K., Rangel, A. & O’Dohe y, J.P., 2009. Neu al compu a ions unde lying ac ion-based decision making in he human b ain. P oceedings o he Na ional Academy o Sciences o he Uni ed S a es o Ame ica, 106(40), pp.17199–17204.