ScienceDi ec
A ailable online a www.sciencedi ec .com
T anspo a ion Resea ch P ocedia 62 (2022) 719–726
2352-1465 © 2022 The Au ho s. Published by ELSEVIER B.V.
This is an open access a icle unde he CC BY-NC-ND license (h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0)
Pee - e iew unde esponsibili y o he scien i ic commi ee o he 24 h Eu o Wo king G oup on T anspo a ion Mee ing
(EWGT 2021)
10.1016/j. p o.2022.02.089
10.1016/j. p o.2022.02.089 2352-1465
© 2022 The Au ho s. Published by ELSEVIER B.V.
This is an open access a icle unde he CC BY-NC-ND license (h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0)
Pee - e iew unde esponsibili y o he scien i ic commi ee o he 24 h Eu o Wo king G oup on T anspo a ion Mee ing
(EWGT 2021)
A ailable online a www.sciencedi ec .com
T anspo a ion Resea ch P ocedia 00 (2021) 000–000 www.else ie .com/loca e/p ocedia
24 h Eu o Wo king G oup on T anspo a ion Mee ing, EWGT 2021, 8-10 Sep embe 2021,
A ei o, Po ugal
Unce ain y and Va iabili y Analysis o Agen -Based T anspo
Models
Lasse Bienzeisle a,∗, To ben Lelkea, Oska Wageb, Lena-Ma ie Hucka, Be nha d
F ied icha
aIns i u e o T anspo a ion and U ban Enginee ing, TU B aunschweig, He mann-Blenk-S . 42, 38108 B aunschweig, Ge many
bIns i u e o Ca og aphy and Geoin o ma ics, Leibniz Uni e si y Hanno e , Appels . 9a, 30167 Hanno e , Ge many
Abs ac
This pape p esen s an analysis o he ou pu a iabili y o agen -based anspo models. We simula ed a MATSim model o he
ci y o Hano e mul iple imes wi h iden ical inpu and e alua ed he esul ing a el imes on di e en le el o agg ega ion. On a
global le el, we obse ed mino a ia ions o a el imes. Howe e , he esul s show an inc eased a ia ion when examining he
ou pu on he le el o dis ic s o o indi idual agen s. A ecommenda ion o es ima ing he equi ed numbe o simula ion uns
o a s able ou pu o a el ime o he pu posed agg ega ion le el is de i ed om ou case s udy.
©2021 The Au ho s. Published by Else ie B.V.
This is an open access a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/)
Pee - e iew unde esponsibili y o he scien i ic commi ee o he 24 h Eu o Wo king G oup on T anspo a ion Mee ing.
Keywo ds: MATSim; Ou pu Va ia ion; Agen -based T anspo Simula ion
1. In oduc ion
Wi h he inc easing ele ance o agen -based simula ions, a ious app oaches ha e been de eloped, wi h MATSim
(Ho ni e al.,2016) eme ging as one o he mos equen ly used open-sou ce simula ion amewo ks. MATSim is
based on u ili y maximiza ion. Indi idual mobili y decisions on ip pu pose, des ina ion, mode, and ime choice a e
calcula ed by econome ic disc e e choice models o ep oduce a ine-g ained a ic demand. The abili y o simula e
each agen indi idually enables he conside a ion o complex linkages ac oss mul iple ips. While compe ing wi h
all o he agen s o space- ime slo s on he anspo in as uc u e, each agen epea edly op imizes i s daily ac i i y
schedule. Op imiza ion is pe o med in an i e a i e cycle wi h a p ede ined ac ion o agen s andomly changing hei
plans a each i e a ion. The amewo k e alua es he new plan using a sco ing unc ion a e he subsequen simula ion
s ep (Ho ni e al.,2016).
∗Co esponding au ho .
E-mail add ess: [email p o ec ed]
2352-1465 ©2021 The Au ho s. Published by Else ie B.V.
This is an open access a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/)
Pee - e iew unde esponsibili y o he scien i ic commi ee o he 24 h Eu o Wo king G oup on T anspo a ion Mee ing.
A ailable online a www.sciencedi ec .com
T anspo a ion Resea ch P ocedia 00 (2021) 000–000 www.else ie .com/loca e/p ocedia
24 h Eu o Wo king G oup on T anspo a ion Mee ing, EWGT 2021, 8-10 Sep embe 2021,
A ei o, Po ugal
Unce ain y and Va iabili y Analysis o Agen -Based T anspo
Models
Lasse Bienzeisle a,∗, To ben Lelkea, Oska Wageb, Lena-Ma ie Hucka, Be nha d
F ied icha
aIns i u e o T anspo a ion and U ban Enginee ing, TU B aunschweig, He mann-Blenk-S . 42, 38108 B aunschweig, Ge many
bIns i u e o Ca og aphy and Geoin o ma ics, Leibniz Uni e si y Hanno e , Appels . 9a, 30167 Hanno e , Ge many
Abs ac
This pape p esen s an analysis o he ou pu a iabili y o agen -based anspo models. We simula ed a MATSim model o he
ci y o Hano e mul iple imes wi h iden ical inpu and e alua ed he esul ing a el imes on di e en le el o agg ega ion. On a
global le el, we obse ed mino a ia ions o a el imes. Howe e , he esul s show an inc eased a ia ion when examining he
ou pu on he le el o dis ic s o o indi idual agen s. A ecommenda ion o es ima ing he equi ed numbe o simula ion uns
o a s able ou pu o a el ime o he pu posed agg ega ion le el is de i ed om ou case s udy.
©2021 The Au ho s. Published by Else ie B.V.
This is an open access a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/)
Pee - e iew unde esponsibili y o he scien i ic commi ee o he 24 h Eu o Wo king G oup on T anspo a ion Mee ing.
Keywo ds: MATSim; Ou pu Va ia ion; Agen -based T anspo Simula ion
1. In oduc ion
Wi h he inc easing ele ance o agen -based simula ions, a ious app oaches ha e been de eloped, wi h MATSim
(Ho ni e al.,2016) eme ging as one o he mos equen ly used open-sou ce simula ion amewo ks. MATSim is
based on u ili y maximiza ion. Indi idual mobili y decisions on ip pu pose, des ina ion, mode, and ime choice a e
calcula ed by econome ic disc e e choice models o ep oduce a ine-g ained a ic demand. The abili y o simula e
each agen indi idually enables he conside a ion o complex linkages ac oss mul iple ips. While compe ing wi h
all o he agen s o space- ime slo s on he anspo in as uc u e, each agen epea edly op imizes i s daily ac i i y
schedule. Op imiza ion is pe o med in an i e a i e cycle wi h a p ede ined ac ion o agen s andomly changing hei
plans a each i e a ion. The amewo k e alua es he new plan using a sco ing unc ion a e he subsequen simula ion
s ep (Ho ni e al.,2016).
∗Co esponding au ho .
E-mail add ess: [email p o ec ed]
2352-1465 ©2021 The Au ho s. Published by Else ie B.V.
This is an open access a icle unde he CC BY-NC-ND license (h p://c ea i ecommons.o g/licenses/by-nc-nd/4.0/)
Pee - e iew unde esponsibili y o he scien i ic commi ee o he 24 h Eu o Wo king G oup on T anspo a ion Mee ing.
720 Lasse Bienzeisle e al. / T anspo a ion Resea ch P ocedia 62 (2022) 719–726
2L. Bienzeisle e al. /T anspo a ion Resea ch P ocedia 00 (2021) 000–000
The a iabili y o he agen s’ choice in MATSim is based on a se ies o pseudo- andom numbe s de e mined by
a andom seed (Paulsen e al.,2018). Howe e , due o he coe olu iona y algo i hm, he choices a e no execu ed
pseudo- andomly in e e y un o he simula ion (Ho ni e al.,2011). Thus, he esul ing models a e non-de e minis ic.
The unce ain y in he e alua ion o di e en simula ion uns is a well-known p oblem ha needs o be conside ed
when conduc ing simula ion case s udies (Rasouli and Timme mans,2012). In es iga ing di e en model pa ame e s
on di e en le els o agg ega ion (LOA), we obse ed subs an ial a ia ions o di e en measu es. Howe e , he e
is a lack o sys ema ic unce ain y analysis o MATSim simula ions o acili a e mo e educa ed decision-making.
The p e equisi es o s able simula ion esul s wi h a desi ed eliabili y ha e no ye been in es iga ed o di e en
agg ega ion le els.
2. Rela ed Wo k
Quan i ica ion o he eliabili y o decisions based on ma hema ical models is a p ominen opic in he ield o
anspo modelling. Wi h he eme gence o models o inc easingly complex p oblems ha a e ha d o in e p e ,
esea che s ha e s a ed e o s o gene alize de ini ions o unce ain y and analysis me hods. Walke e al. (2003)
p esen ed a heo e ical amewo k o sys ema ic unce ain y analysis in model-based decision suppo . The ein, un-
ce ain y is de ined as ’any depa u e om he unachie able ideal o comple ely de e minis ic knowledge o he ele-
an sys em’. The au ho s di e en ia e be ween h ee dimensions o unce ain y, which hey de ine as unce ain y o
loca ion,na u e, and le el.
Mul iple s udies ha e examined a ia ions in ac i i y-based mic o-simula ions, such as he es ablished Alba oss
(A en ze and Timme mans,2004) o Fea he s models (Bao e al.,2015). Acco ding o Baus e (2021), he mos
commonly analyzed unce ain y loca ion in hese models is he simula ion e o . Reasons o his a e he ela i e ease
wi h which his loca ion can be add essed and he o en s ochas ic na u e o hese models. Cas iglione e al. (2003)
s udied he minimum numbe o uns needed o achie e obus a e age esul s. Cools e al. (2011) assessed he impac
o mic o-simula ion e o s on he a e age daily numbe o ips pe pe son as well as he a e age daily dis ance
a eled pe pe son. Thei esul s show minimal a ia ion, especially o agg ega ed alues.
Agen -based mic o-simula ions, such as MATSim, a e pa icula ly p one o model unce ain ies since hey o en
ely on disc e e choice models o pe o m mode choice and ip assignmen . Acco ding o Ho ni e al. (2016) and Ho ni
e al. (2011), he coe olu iona y algo i hm o MATSim is he majo loca ion o unce ain ies in he simula ion and
in e s di e en ypes o unce ain y in oduced by ime, ou e, and des ina ion choice modules. Caused by he andom
seed, dis inc unce ain y is in oduced o e e i e a ion o he simula ion. Di e en andom numbe s may lead he
op imiza ion algo i hm o ind o he local op ima. Mo eo e , MATSim con ains a andom a iabili y in how he e-
planning o plans is handled. Ho ni e al. (2011) demons a ed ha he esul s o simula ions can change signi ican ly
be ween mul iple uns. In hei wo k, hey s udied he impac o a ying andom seeds wi h a ocus on link loads in
wo di e en MATSim scena ios. They conside ed ha he a ia ion in daily link loads is gene ally low. Howe e ,
conside ing hou ly alues, he coe icien o a ia ion inc eases. In hei li e a u e e iew hey also concluded ha
a e age esul s gene a ed om mic o-simula ions become s able a e ’a ela i ely small numbe o simula ion uns’
(Ho ni e al.,2011, p. 8). These indings we e p oba ed by Paulsen e al. (2018). Chap e 48 o he MATSim book
(Fl¨
o e ¨
od,2016) also desc ibes he challenges in MATSims ou pu e alua ion due o he in luence o he choice o
one speci ic andom seed and elabo a es he need o u he esea ch in his pa icula a ea. Thus, we s i e o add
addi ional le els o in es iga ion o his discussion by analyzing he ou pu ’s a el ime a ia ion o MATSim and
expanding ou s udies o ega d di e en LOA.
3. Me hodology
To in es iga e he a iabili y o MATSim simula ion ou pu s, we se up a simula ion case s udy o an 10 % model o
he ci y o Hano e , Ge many (Bienzeisle e al.,2020). Using he e e enced con igu a ion pa ame e s, we epea edly
simula ed he Hano e inpu model wi h 750 i e a ions. The public anspo sys em was implemen ed as a ne wo k
mode. In addi ion, comme cial a ic was included in he model using he eigh ex ension o MATSim (Zilske e al.,
2012) sepa a ed by di e en b anches. We simula ed 16 simula ion uns wi h he same inpu pa ame e s o explo e
inconsis encies ac oss he simula ion ou pu s.
Lasse Bienzeisle e al. / T anspo a ion Resea ch P ocedia 62 (2022) 719–726 721
L. Bienzeisle e al. /T anspo a ion Resea ch P ocedia 00 (2021) 000–000 3
A e Paulsen e al. (2018) concen a ed hei wo k on he a ia ion o link loads using di e en andom seeds,
we ocused on he a ia ion o a el imes. T a el ime dis ibu ions a e a model cha ac e is ic ha can be used o
calib a ion o alida ion. Thus, he e alua ed dimension o he a el ime pe p i a e agen (p) o comme cial a ic
ehicle (c ) was de ined as he sum o all ip du a ions pe day. We conside ed he changing a el imes pe un
in he se o 16 uns Rpe agen a∈all agen s A o explo e he e ec s o he unce ain ies om he andom choice
pa s o he MATSim algo i hm. We assigned he co esponding home dis ic d∈all dis ic s o Hano e D o each
agen ap. Th ee agg ega ion le els o he analyzed a el imes we e in oduced as a se o a el imes ∈LOA . The
e alua ion o a el imes was ca ied ou sepa a ely o each LOA and each simula ion un ∈R:
•LOA1: Global a e age a el ime o Hano e : wi h o a∈A
•LOA2: A e age a el ime o each dis ic do Hano e : wi h o a∈d
•LOA3: T a el ime o each agen ao Hano e : o all a∈A
To quan i y he a ia ion o he a el ime, we applied he coe icien o a ia ion c ( ), which is de ined as he
s anda d de ia ion o he sample di ided by he sample mean, on ou de ined LOA.
4. Analysis o he Va ia ion o T a el Times
To ob ain a i s unde s anding o he a ia ion o a el imes ac oss he simula ion uns, we s a ed ou wo k by
compa ing he equency dis ibu ions o all occu ing a el imes pe agen o he p i a e a ic o each simula ion
un sepa a ely. T a el imes we e g ouped in bins o 1 minu e, each wi h hei co esponding equency pe un. Fo
a be e compa ison o he esul ing 16 a el ime dis ibu ions, we ha e combined he his og ams in he 3D ba plo
shown in Figu e 1. Each ba ep esen s he equency o occu ence o a a el ime g oup pe simula ion un.
Fig. 1: Combined His og ams o a el ime in bins o 1 minu e wi h ∈R.
The ough su ace depic ed in he igu e p o ides a isual indica ion ha he simula ion gene a es di e en dis-
ibu ions o a el imes. Especially in he ange o equen ly occu ing alues, di e en pa e ns can be obse ed.
722 Lasse Bienzeisle e al. / T anspo a ion Resea ch P ocedia 62 (2022) 719–726
4L. Bienzeisle e al. /T anspo a ion Resea ch P ocedia 00 (2021) 000–000
Howe e , he diag am also highligh s ha he e a e uns wi h a simila dis ibu ion o a el imes, whe e he su ace
o he plo is cons an and smoo h. This can be obse ed o uns 11 and 12.
To de e mine he de ia ion o he simula ion uns, we calcula ed he co esponding Roo Mean Squa e E o
(RMSE) and hus compa ed all uns o each o he (see Figu e 2). In mos cases, he RMSE a ies om 12.73 o
a maximum o 20.20. No ice ha he e a e uns wi h a RMSE o 0. This indica es an iden ical dis ibu ion o he
a el imes o he combina ion o hese speci ic uns. This obse a ion is consis en wi h he i s isual analysis. The
simula ions eplica ed exac ly he same a el imes o un 4, 9, 11, 12, 14, and 16.
Fig. 2: RMSE-Analysis o he a el ime dis ibu ion o all simula ion uns R.
Ano he inding om ou da a is ha he simula ion ou pu s can se le in se e al disc e e s a es o each agen . The
numbe o s a es spe agen ais de ined as he numbe o di e en a el ime alues o an agen occu ing o e all
simula ion uns. Figu e 3shows he numbe o di e en s a es saoccu ing o e all simula ion uns as a cumula i e
dis ibu ion plo .
A la ge g oup o p i a e agen s, 21.9 % (n=16.312) has one s a e, i.e. one cons an a el ime o e all uns. In
hese cases, he a el ime dis ibu ion does no oscilla e and he speci ic s a e eoccu s in e e y simula ion un. Fo
comme cial ehicles, his applies o 3.4 % (n=211) o he agen s.
Fig. 3: Empi ical cumula i e dis ibu ion wi h he numbe o di e en s a es saac oss all agen s g ouped by agen ype.
As shown, he ou pu o comme cial a ic ehicles di e s mo e. In pa icula cou ie s, exp ess, and pa cel se ice
(CEP) ehicles do no se le in disc e e s a es, bu show indi idual a el imes o each un. Howe e , he sample size
is signi ican ly smalle (np=74.394, nc =6.148, nCEP =98 ). To explo e his cha ac e is ic o he eigh a ic
in MATSim in de ail, we plo ed he equency dis ibu ion o a el imes pe simula ion un in Figu e 4. The mo e
homogeneous dis ibu ion o a el imes o comme cial a ic ehicles compa ed o CEP ehicles pe un is
e iden .
Lasse Bienzeisle e al. / T anspo a ion Resea ch P ocedia 62 (2022) 719–726 723
L. Bienzeisle e al. /T anspo a ion Resea ch P ocedia 00 (2021) 000–000 5
Fig. 4: Dis ibu ion o a el ime o comme cial agen s di e en ia ed by ype wi h ∈R.
To analyze he a iabili y o a el imes in de ail, we in oduced he coe icien o a ia ion c ( ) o di e en
LOA as a measu e o a iabili y and applied i o ou da a se . Figu e 5illus a es he cha ac e is ics ha led o a
pa icula ly high c ( )LOA3in ou simula ion case s udy.
Fig. 5: Dis ibu ion o he a ia ion coe icien c ( ) in ela ion o he a e age a el ime combined wi h he empi ical cumula i e dis ibu ion
unc ion o he a ia ion coe icien c ( ) wi h ∈R.
The a ious a el ime dis ibu ions obse ed p e iously a e e iden in he a ia ion o measu ed a el ime alues
ac oss all agen s and simula ion uns. The sca e plo indica es ha highe c ( ) alues usually occu a lowe a e age
a el imes. Since he analysis o a el ime equencies shows ha mos o he agen s’ a el ime end o decline
wi hin his ange o lowe a el imes, he obse ed clus e ing can be pa ly explained by he co espondingly la ge
sample size. I is appa en ha se e al agen ’s a el imes a ies conside ably be ween he simula ion uns. The
maximum alues c ( ) di e signi ican ly be ween agen ypes, i.e. c (
p)(max)=1.212 , c (
c )(max)=0.917 and
c (
cep)(max)=0.230. In o al, only nine agen s show a alue o c ( )>1. A c ( )>0.5 can be obse ed o 741
agen s (0.1 %).
Compa ing agen ypes, he a el imes o p i a e agen s show up he highes a e o de ia ion. This end is also
e iden in he cumula i e equency dis ibu ion o c ( ). Analogous o he numbe o di e en s a es pe agen , 71.0
% o he agen s o he indi idual a ic(n=52.820) ha e a alue o c (
p)>0.1 ac oss all simula ion uns. Fo
CEP- ehicles, i is close o 89.6 % (n=88). The co esponding esul s indica e ha all o hese ehicles change
hei a el ime in e e y simula ion un. Howe e , his a iance is smalle compa ed o he o he agen ypes and he
esul ing a el imes a e mo e consis en .

724 Lasse Bienzeisle e al. / T anspo a ion Resea ch P ocedia 62 (2022) 719–726
6L. Bienzeisle e al. /T anspo a ion Resea ch P ocedia 00 (2021) 000–000
Table 1: Resul ing dis ibu ion o he coe icien o a ia ion c ( ) wi h ∈R.
Le el o agg ega ion Coe icien o a ia ion S anda d de ia ion σc ( )Q1c ( )Medianc ( )Q3c ( )
c ( )|c ( )
LOA1
To al p i a e agen s o Hano e 0.0015 - - - - -
To al comme cial agen s o Hano e 0.0009 - - - - -
LOA2
P i a e agen s pe dis ic s - 0.0058 0.0031 0.0037 0.0049 0.0064
LOA3
Indi idual agen s o p i a e a ic - 0.0818 0.1074 0.0028 0.0449 0.1156
Indi idual agen s o comme cial a ic - 0.0437 0.0579 0.0092 0.0250 0.0546
A e we we e able o show ha di e en a el ime dis ibu ions occu using iden ical simula ion inpu , we s a ed
in es iga ing he h esholds o s able simula ion esul s. We de e mined he alue o c ( ) o each agen on LOA3
di e en ia ed by ype. CEP ehicles we e included in comme cial a ic. Fo a be e compa abili y, we a e aged c ( )
ac oss all agen s. E alua ing LOA2on dis ic le el, we only included p i a e agen s because comme cial ehicles
usually s a a speci ic companies wi h hei ou and a e he e o e no so widely dis ibu ed o e he simula ion a ea.
The a el imes o he co esponding agen s we e a e aged pe dis ic and he a iabili y o his a e age alue was
examined and a mean alue o c ( ) wi h ∈dwas calcula ed ac oss all dis ic s. LOA1is he a ia ion o he
global a e age a el ime ac oss all simula ion uns di e en ia ed by indi idual and comme cial agen s. The esul s
a e summa ized in Table 1. Fo LOA2and LOA3s a is ical pa ame e s o he dis ibu ion o c ( ) a e p o ided since
a single c ( ) alue was calcula ed o each dis ic o agen o Hano e .
Ou esul s suppo ou ini ial assump ions and he indings om he li e a u e e iew. The global mean o he
a e age a el imes o all p i a e agen s om Hano e emains almos cons an o e all simula ions uns c (
p)LOA1=
0.0015. The a ia ion o he comme cial agen a el imes a e smalle wi h a c (
c )LOA1=0.0009. Obse ed a ia ion
a dis ic le el inc eases sligh ly c (
p)LOA2=0.0058 and he analysis o each agen indi idually esul s in he highes
obse ed a ia ion o a el imes c (
p)LOA3=0.0818. As a compa ison o he a ia ion o he agg ega ed a el imes
pe dis ic and he co esponding a ia ion o he agen s li ing in his dis ic , we g ouped he c (
p)LOA3 alues by he
agen ’s home dis ic (Figu e 6).
Fig. 6: Boxplo o a ia ion o indi idual a el imes C ( ) g ouped by he agen ’s home loca ion wi h ∈R.
Subsequen ly, we compa ed he indi idual agen a el ime a iabili y on LOA3wi h he a iabili y o he agg e-
ga ed a el imes on LOA2. The mean dis ibu ion o a el imes o all agen s li ing in he co esponding dis ic
a ies be ween c (
p)LOA1(min)=0.048 and c (
p)LOA1(max)=0.114. The co esponding esul s o c (
p)LOA2a e 0.007
Lasse Bienzeisle e al. / T anspo a ion Resea ch P ocedia 62 (2022) 719–726 725
L. Bienzeisle e al. /T anspo a ion Resea ch P ocedia 00 (2021) 000–000 7
and 0.005. Al hough he agen s o a dis ic show a a ia ion o hei co esponding a el imes o e all simula ion
uns, he agg ega e a el ime o all esiden s o he dis ic a ies less. The esul s o ou case s udy show a com-
pensa ion o he a ia ions o a el imes o indi idual agen s on he agg ega e dimension o dis ic s. Thus, he mo e
agg ega ed e alua ion alues a e s abilizing a he as a one le el. The esul s imply ha a p edic ion abou hese
global pa ame e s, especially on LOA1and LOA2, can be made using an a e age alue o only a ew simula ion uns.
Fo p ac ical wo k wi h MATSim, i is o in e es how many simula ion uns a e necessa y o de e mine he adequa e
alue o he co esponding LOA wi h a desi ed accu acy.
5. P edic ion o Requi ed Numbe o Simula ion Runs
To allow he de i a ion o gene ally alid indica ions om ou esul s, we in es iga ed how many simula ions
a e necessa y o a i e a obus mean alues a he h ee agg ega ion le els de ined. We applied he con e gence o
subsequen mean alues n−→ c o ou da a se by o ming a mo ing mean alue nwi h a p og essing numbe o
simula ions. As soon as he de ia ion o he calcula ed mean alue o he con e gence mean alue cwas less han one
pe cen , we conside ed he ob ained mean alue o be obus . Howe e , he 16 simula ion uns we pe o med we e no
su icien o achie e a obus mean alue. Despi e his, o p edic he numbe o simula ions a which a obus mean
is eached, we used ou obse ed a el ime dis ibu ions o each agen o gene a e a i icial simula ion esul s. This
p ocess was con inuously epea ed o eplica e he obse ed a el ime dis ibu ion. We conside his me hodology o
be alid because unning a la ge se o simula ions wi h MATSim o explo e he needed numbe o simula ion uns is
no p ac ical due o he compa a i ely long compu a ion imes.
The calcula ed dis ibu ions a e summa ized in Figu e 7. Ou i s in es iga ions indica e ha he a el ime alues
on LOA1and LOA2a e al eady obus a e one i e a ion. Thus, his obus ness occu s o agg ega ed esul s. Fo
LOA3,p31 simula ions we e in a e age su icien o each a obus mean. A he maximum 36 uns we e necessa y.
Addi ionally, he g aph shows he de elopmen o he mean alue con e gence o comme cial ehicles and, as a
subse o his, o CEP- ehicles. The unc ion o LOA3,c de elops simila o LOA3,pwi h a wide ange o a ia ion,
e en hough he he obus mean alue was in a e age eached ea lie a e 20 uns.
Fig. 7: Rela i e de ia ion om calcula ed con e gence mean alues o di e en LOA.
6. Conclusion and Fu u e Wo k
The a el imes o a MATSim simula ion a y despi e cons an inpu pa ame e s. We de eloped a ecommenda ion
o he needed numbe o simula ion uns acco ding o di e en agg ega ion le els. Ou aim was o ob ain esul s wi h
a desi ed eliabili y o one pe cen de ia ion om he p edic ed a el ime alues. Acco dingly, he a ia ion o he
a el imes dec eases as he agg ega ion le el inc eases, while global agg ega ed pa ame e s such as he a e age
a el ime emain app oxima ely cons an h oughou he simula ion. By analyzing he con e ging a e age, we we e
able o show ha a single simula ion is su icien o an agg ega ed e alua ion o a el imes. These esul s a e in
726 Lasse Bienzeisle e al. / T anspo a ion Resea ch P ocedia 62 (2022) 719–726
8L. Bienzeisle e al. /T anspo a ion Resea ch P ocedia 00 (2021) 000–000
line wi h p e ious con ibu ions in his esea ch a ea, since agg ega ed mac oscopic da a is o en used o alida e
MATSim model esul s (Kagho e al.,2020). Howe e , he analysis o he coe icien o a ia ion also showed a ying
a el imes o indi idual agen s pe simula ion un. This can be pa icula ly impo an when e alua ing simula ions
ocused on speci ic popula ion g oups wi h compa a i ely small sample sizes. A possible e alua ion case applies o
CEP a ic. These ehicles a e pa o comme cial anspo and hus ha e a small numbe o ehicles compa ed o
he p i a e a ic. Ou simula ions illus a ed ha he a el imes o he eigh agen s end o be ela i ely cons an ,
al hough he a el imes o he CEP ehicles s ill a ies. Ou lie s can change he o e all esul due o he small size o
he sample. Fo hese sample sizes ou esul s lead us o ecommend o a e age a leas he esul s o wo simula ion
uns o educe he a iabili y o he e alua ed a el imes.
MATSim simula ion uns a e compu a ionally expensi e. Due o his, MATSim models a e o en scaled down. The
a ia ion o he indi idual agen a el imes on LOA3 hus has a highe in luence on he agg ega ed alues and leads o
an inhe en e o . Consequen ly, ou indings suppo he wo k o Llo ca and Moeckel (2019), who obse ed di e en
a el ime dis ibu ions o smalle scale ac o s.
The objec i e o ou u u e wo k is o p o ide an o e iew o he a iance o a MATSim model in ela ion o he
de ined le el o agg ega ion o allow mo e accu a e e alua ions wi h MATSim. T a el imes a y depending on he
agen ypes. Thus, i is app op ia e o in es iga e a ibu es causing a co esponding a iabili y and inally p edic ing
he expec ed e o o ce ain g oups o agen s. In addi ion, ou a i icial gene a ion o a el ime dis ibu ions, espec-
i ely simula ion uns, mus be alida ed wi h u he simula ion uns in o de o be able o de e mine he p edic ed
alues mo e p ecisely.
Acknowledgemen
The scien i ic esea ch published in his a icle is g an ed by he Fede al Minis y o Educa ion and Resea ch
Ge many o p ojec s USE UL and USE UL XT (g an ID 03SF0547 & 03SF0609). The au ho s co dially hank he
pa ne s and unding agency.
Re e ences
A en ze, T., Timme mans, H., 2004. Alba oss: A lea ning based anspo a ion o ien ed simula ion sys em. T anspo a ion Resea ch Pa B:
Me hodological 38, 613–633.
Bao, Q., Kochan, B., Bellemans, T., Janssens, D., We s, G., 2015. In es iga ing mic o-simula ion e o in ac i i y-based a el demand o e-
cas ing: a case s udy o he ea he s amewo k. T anspo a ion Planning and Technology 38, 425–441.
Baus e , P., 2021. De elopmen o an unce ain y analysis amewo k o model-based consequen ial li e cycle assessmen : Applica ion o
ac i i y-based modelling and li e cycle assessmen o mul imodal mobili y. Ph.D. hesis. Buil En i onmen .
Bienzeisle , L., Lelke, T., Wage, O., Thiel, F., F ied ich, B., 2020. De elopmen o an agen -based anspo model o he ci y o hano e using
empi ical mobili y da a and da a usion. T anspo a ion Resea ch P ocedia 47, 99–106.
Cas iglione, J., F eedman, J., B adley, M., 2003. Sys ema ic in es iga ion o a iabili y due o andom simula ion e o in an ac i i y-based
mic osimula ion o ecas ing model. T anspo a ion Resea ch Reco d 1831, 76–88.
Cools, M., Kochan, B., Bellemans, T., Janssens, D., We s, G., 2011. Assessmen o he e ec o mic o-simula ion e o on key a el indices:
e idence om he ac i i y-based model ea he s. P oceedings o he 90 h Annual Mee ing o he T anspo a ion Resea ch Boa d .
Fl¨
o e ¨
od, G., 2016. Ma sim as a mon e ca lo engine. The Mul i-Agen T anspo Simula ion MATSim , 327–336.
Ho ni, A., Cha ypa , D., Axhausen, K.W., 2011. Va iabili y in anspo mic osimula ions in es iga ed wi h he mul i-agen anspo simula ion
ma sim. A bei sbe ich e Ve keh s- und Raumplanung 692.
Ho ni, A., Nagel, K., Axhausen, K.W., 2016. The Mul i-Agen T anspo Simula ion MATSim. Ubiqui y P ess, London, GBR.
Kagho, G., Balac, M., Axhausen, K., 2020. Agen -based models in anspo planning: Cu en s a e, issues, and expec a ions. P ocedia
Compu e Science 170, 726–732.
Llo ca, C., Moeckel, R., 2019. E ec s o scaling down he popula ion o agen -based a ic simula ions. P ocedia Compu e Science 151,
782–787.
Paulsen, M., Rasmussen, T.K., Nielsen, O.A., 2018. Ou pu a iabili y caused by andom seeds in a mul i-agen anspo simula ion model.
P ocedia Compu e Science 130, 850–857.
Rasouli, S., Timme mans, H., 2012. Unce ain y in a el demand o ecas ing models: li e a u e e iew and esea ch agenda. T anspo a ion
Le e s 4, 55–73.
Walke , W., Ha emo¨
es, P., Ro mans, J., Sluijs, J., Assel , M., Janssen, P., K aus, M., 2003. De ining unce ain y: A concep ual basis o
unce ain y managemen in model-based decision suppo . In eg a ed Assessmen 4.
Zilske, M., Sch ¨
oe , S., Nagel, K., Lied ke, G., 2012. Adding eigh a ic o ma sim. VSP Wo king Pape 12-02 .