Papp, Tamás K.
Wo king Pape
A lexible dis ibu ion amily o es ing MCMC
implemen a ions
IHS Wo king Pape , No. 60
P o ided in Coope a ion wi h:
Ins i u e o Ad anced S udies (IHS), Vienna
Sugges ed Ci a ion: Papp, Tamás K. (2025) : A lexible dis ibu ion amily o es ing MCMC
implemen a ions, IHS Wo king Pape , No. 60, Ins i u ü Höhe e S udien - Ins i u e o Ad anced
S udies (IHS), Vienna,
h ps://i ihs.ihs.ac.a /id/ep in /7280/
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/324755
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h ps://c ea i ecommons.o g/licenses/by/4.0/
IHS Wo king Pape 60
Augus 2025
A lexible dis ibu ion amily o es ing
MCMC implemen a ions
Tamás K. Papp
All IHS Wo king Pape s a e a ailable online:
h ps://i ihs.ihs.ac.a / iew/ihs_se ies/se =5Fihswps.h ml
This pape is a ailable o download wi hou cha ge a :
h ps://i ihs.ihs.ac.a /id/ep in /7280/
Au ho (s)
Tamás K. Papp
Edi o (s)
Robe M. Kuns
Ti le
A lexible dis ibu ion amily o es ing MCMC implemen a ions
Ins i u ü Höhe e S udien - Ins i u e o Ad anced S udies (IHS)
Jose s äd e S aße 39, A-1080 Wien
T +43 1 59991-0
www.ihs.ac.a
ZVR: 066207973
License
This wo k is licensed unde he C ea i e Commons: A ibu ion 4.0 License
(h p://c ea i ecommons.o g/licenses/by/4.0/)
All con en s a e wi hou gua an ee. Any liabili y o he con ibu o s o he IHS
om he con en o his wo k is excluded.
A lexible dis ibu ion amily o es ing MCMC implemen a ions1
Tamás K. Papp ([email p o ec ed])
Ins i u e o Ad anced S udies, Vienna
Augus 2, 2025
Abs ac
We p opose a lexible, ex ensible amily o dis ibu ions o es ing Ma ko Chain Mon e Ca lo
implemen a ions. Dis ibu ionsa ec ea ed bynes ingsimple ans o ma ions,whichallow a ious
shapes,includingmul iplemodesand a ails. The esul ingdis ibu ionscanbesampledwi hhigh
p ecision using quasi- andom sequences, and ha e closed o m (log) densi y and g adien a each
poin , making i possible o es g adien -based sample s wi hou au oma ic di e en ia ion.
1 In oduc ion
Bayesian in e ence esul s in a pos e io dis ibu ion o e he pa ame e space o he model. Excep o
special cases, such pos e io dis ibu ions do no con o m o any well-known amily, and hei analysis
equi es sampling, which is a compu a ionally in ensi e ask ha equi es specialized algo i hms and
so wa e de eloped o his pu pose. The his o y o Bayesian compu a ion is a p ocession o inc eas-
ingly complex and sophis ica ed algo i hms (Robe and Casella 2011), wi h cons an new de elop-
men s (Bou-Rabee, Ca pen e , and Ma sden 2024; Bou-Rabee, Ca pen e , Liu, e al. 2025).
Tes ing so wa e implemen a ions o hese algo i hms is challenging o h ee easons. Fi s , hei
ou pu is s ochas ic, which equi es a s a is ical app oach, as opposed o simply compa ing ha he im-
plemen a ion mapsknowninpu s o known ou pu s (Whi ake 1997; Še číko á e al. 2006). Second, as
emphasized by G osse and Du enaud (2014), i is di icul o sepa a e he p oblems wi h he mixing o
he Ma ko chains om he laws in he implemen a ion. Thi d, he e y quan i ies we wan o compa e
a e, in gene al cases, unknown: as Tal s e al. (2020) pu i ,
The mos s aigh o wa d way o alida e a compu ed pos e io dis ibu ion is o compa e
compu ed expec a ions wi h he exac alues. An immedia e p oblem wi h his, howe e ,
is ha we know he ue pos e io expec a ion alues o only he simples models.
This pape add esses he las poin , by p o iding a e y gene al dis ibu ion amily which allows
1. accu a e sampling using low-disc epancy sequences, which can be compa ed o MCMC ou pu ,
2. closed o m calcula ions o he g adien o he log pos e io , allowing he use o g adien -based
me hods (Neal 2011; Ho man and Gelman 2014) wi hou au oma ic di e en ia ion, making he
so wa e implemen a ion easie .
We e iew he ela ed li e a u e in Sec ion 2. Sec ion 3 in oduces he model amily, while Sec ion 4
demons a es one possible app oach o uni es ing MCMC so wa e. Sec ion 5 concludes.
2 Rela ed li e a u e
Geweke (2004) elies on pos e io dis ibu ions being (unscaled) p oduc s o a p io and a likelihood.
Samples a e ob ained in wo ways:
1. sample p(θ) om he p io , hen sample p(x|θ) om he gene a i e model,
2. gi en a (θ,x) om he p e ious s ep, upda e θusing an MCMC ansi ion, hen gene a e da a
om p(x|θ). The combina ion o hese s eps should p ese e he join dis ibu ion p(θ,x).
1This pape documen s he in e nals and a ionale o he Julia so wa e lib a y Papp and con ibu o s (2025b). Tamás
K.Pappacknowledges he suppo o he Aus ianNa ionalBankJubileums ondsP ojek 18847. I would like o hank Robe
Kuns o commen s.
1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Figu e 1: Sobol sequence in 2D, 1024 poin s.
Bo h me hods should yield samples om exac ly he same dis ibu ion, and hey can be compa ed
using a ious s a is ics.
Cook, Gelman, and Rubin (2006) sugges he ollowing p ocedu e. Use he p io p(θ) o gene a e a
pa ame e s θ0, hen he gene a i e model p(x|θ0) o gene a e a da a sample x. Finally, ob ain pos e io
samples θi, o i= 1,2,...,L, and calcula e he quan ile ˆq(θ0) = 1
L∑︁L
i=11θ0>θi. The au ho s show ha as
L→∞,ˆq(θ0)con e ges o he uni o m dis ibu ion. This can be e alua ed g aphically o using o mal
s a is ics.
Tal s e al. (2020) sugges compa ing he da a a e aged pos e io
pD(θ)=∫︂p(θ|y)p(y|θ)d(y)p(θ)
o he p io dis ibu ion, whe e he in eg al is e alua ed nume ically, di ec sampling om he p io
p(θ)and he da a gene a ing model p(y|θ), and MCMC o p(θ|y). Thecompa isonispe o med using
one-dimensional ank s a is ics :Θ→R, using empi ical quan iles simila ly o he ˆqo Cook, Gelman,
andRubin(2006). Thep e e edme hodo compa isonis isual,and hep oposedsimula ions a egy
also allows o e o bands.
3 The dis ibu ion amily
I is assumed ha he use has access o an implemen a ion o low-disc epancy sequences2 o gene a e
quasi- andom numbe s u(i),i = 1,...,N in [0,1]m.3Figu e 1 shows samples om a commonly used
low-disc epancy sequence, he Sobol sequence, which we also use in ou implemen a ion.
Each dis ibu ion implemen s he hype cube ans o m h:[0,1]m→Rnand a densi y unc ion p:Rn→
R+ ha ha e he p ope y
∫︂Rn
g(x)p(x)dx≈1
N
N
∑︂
i=1
g(︁h(u(i)))︁)ℓ(︁h(u(i)))︁
whe e he p ope ies o he app oxima ion depend on he a ia ion o gas desc ibed in Mo oko and
Ca lisch (1995) and Ca lisch (1998).
P o ided ha a dis ibu ion has his p ope y, we a e able o
2See Niede ei e (1988).
3Lacking ha , we assume ha uia e ob ained om a sui able andom numbe gene a o , bu ha is less han ideal o
accu acy so we sugges ha i is a oided.
2
1. ob ain low-disc epancy samples using h, and
2. compa e hem o he he esul o MCMC me hods sampling om .
We desc ibe he cons uc ion o a amily o unc ions ha ha e his p ope y below. Fo nume ical
accu acy, we cons uc he log densi y unc ion
ℓ(x)=log(p(x))
and i s g adien ∇ℓ(x).
No ice he wo dimensions associa ed wi h a dis ibu ion in his amily: nis he dimension o he
domain, while m≥nis he hype cube dimension. The wo may be di e en because we need ex a alues
o gene a e mix u es, as explained in Sec ion 3.3.
3.1 P imi i es
Assume ha a uni a ia e dis ib ion Dis a ailable wi h log densi y ℓ:R→R+and in e se cumula i e
dis ibu ion unc ion I:[0,1]→R. Then, o a gi en dimension n≥1, we de ine
ℓD(x)=
N
∑︂
i=1
ℓ(xi)
∇ℓD(x)=
N
∑︂
i=1
ℓ(xi)
hD(u)=⎛
⎝
I(u1)
...
I(un)⎞
⎠
A e ycon enien p imi i e amilyis hes anda dno mal. Wein oduceS dNo mal(n)using heabo e
no a ion4wi h
ℓ(x)=−x2/2+log(2π)
2
I(u)=√2e −1(2u−1)
I ollows ha
ℓS dNo mal(x)=−∥x∥2
2+nlog(2π)
2
∇ℓS dNo mal(x)=x
hS dNo mal(u)=√2⎛
⎝
e −1(2u1−1)
...
e −1(2un−1)⎞
⎠
Figu e 2 shows he con ou plo o he highes densi y egions o he S dNo mal(2) dis ibu ion, p ima -
ily o es ablishing he con en ion o illus a ing dis ibu ions in R2wi h 90%, …, 10% highes densi y
con ou s in he pape .5
3.2 T ans o ma ions
All ans o ma ions a e de ined using a di e en iable bijec ion g:Rn→Rnwhe e nis he dimension o
he dis ibu ion. In his sec ion we use he con en ion y=g(x).
4No a ion in his pape closely ollows he so wa e package Papp and con ibu o s (2025b), wi h names expo ed om
ha package ypese in slan ed on . Uppe /lowe case and mino spelling con en ions may di e .
5Gi en he exac sampling using Sobol sequences, such con ou s a e i ial o ob ain compu a ionally.
3
−3−2−1 0 1 2 3
−3
−2
−1
0
1
2
3
Figu e 2: The s anda d no mal dis ibu ion S dNo mal(2). Con ou lines a e highes densi y egions
con aining 90%, …, 10% o he p obabili y mass.
Gi en a sou ce dis ibu ion S, cha ac e ized by ℓS,hS, he des ina ion dis ibu ion Dhas
hD(u)=g(hS(u))
ℓD(y)=ℓS(︁g−1(y))︁−log(︃de (︃∂
∂x(︁g−1(y))︁)︃)︃
⏞ ⏟⏟ ⏞
≡c
∇ℓD(y)=∇ℓS(︁g−1(y))︁∂
∂y (︁g−1(y))︁
⏞ ⏟⏟ ⏞
∂x
∂y
−∂c
∂y
whe e cis he co ec ion equi ed o ans o ma ion o he a iables. Fo all ans o ma ions, we p o-
ide analyi ical cha ac e iza ions o g−1,c,∂x
∂y , and ∂c
∂y , which makes calcula ions possible wi hou au-
oma ic di e en ia ion.
All ans o ma ions p ese e he hype cube dimension o he sou ce dis ibu ion.
3.2.1 Linea map
The linea map Linea (A)uses he mapping y=g(x)=Ax, o an n×nin e ible ma ix A. Then
x=A−1y∂x
∂y =A−1c=log(|de (A)|)∂c
∂y =0
Figu e 3 shows an example.
3.2.2 T ansla ion
T ansla ion Shi (b) akes a ec o bo leng h n, and is de ined using y=g(x)=x+b. Then
x=y−b∂x
∂y =0 c=0 ∂c
∂y =0
Shi may seem like a i ial ans o ma ion, bu i is e y use ul since i allows omi ing a loca ion
pa ame e om he o he ans o ma ions: a gene al ans o ma ion τcen e ed a ound he o igin can
be ecen e ed a ound an a bi a y loca ion bas Shi (b)◦τShi (−b). We use his ex ensi ely below.
4
−3−2−1 0 1 2 3
−3
−2
−1
0
1
2
3
Figu e 3: Linea (A)(S dNo mal(2)) wi h A=(︁0.2 0.5
0.4−0.7)︁. Con ou lines a e highes densi y egions
con aining 90%, …, 10% o he p obabili y mass.
3.2.3 Elonga e
TheElonga e(k) ans o ma ions e ches(o sh inks) he ailsa ound heo igin. Fo agi enpa ame e
k∈R, i is de ined by
y=g(x)=x·(︂1+∥x∥2
2)︂k
In o de o cha ac e ize g−1, we de ine ξ(Y,k)as he X >0 ha
Y=X·(1+X2)k(1)
o all Y > 0. The solu ion always exis s since he igh hand side o (1) is inc easing in X. Calcula ing
ξ(Y,k) equi es a uni a ia e nume ical sol e , such as bisec ion o New on’s me hod.6
Then, o a gi en y, le
κ=ξ(∥y∥2,k)2D=(1+κ)−kx=D·y
c=knlog(1+κ)2+log(1+kκ/(1+κ)) A=1+κ B =1+(1+2k)κ
Using hese, we can calcula e
∂x
∂y =(In−(2k/B)xx′)D∂c
∂y =k(2+Bn)
A2k+B2·y
Figu e 4 shows an example.
3.2.4 Funnel
The ans o ma ion o Funnel, inspi ed by he well-known example o Neal (2003), uses he mapping
y=g(x)=⎛
⎜
⎜
⎝
x1
x2exp(x1)
...
xnexp(x1)
⎞
⎟
⎟
⎠
6New on’s me hod can be se up in a way ha i is always con e gen in a ew s eps. See he ela ed sou ce code.
5
−3−2−1 0 1 2 3
−3
−2
−1
0
1
2
3
Figu e 4: Elonga e(0.5)(S dNo mal(2)). Con ou lines a e highes densi y egions con aining 90%, …,
10% o he p obabili y mass.
−3−2−1 0 1 2 3
−3
−2
−1
0
1
2
3
Figu e 5: Funnel(S dNo mal(2)). Con ou lines a e highes densi y egions con aining 90%, …, 10% o
he p obabili y mass.
which has
∂x
∂y i,j
=⎧
⎪
⎪
⎪
⎪
⎨
⎪
⎪
⎪
⎪
⎩
1i i=j=1
exp(−y1)i i=j=1
−yiexp(−y1)i i=1,j =1
0o he wise
c=(n−1)y1
∂c
∂y =⎛
⎜
⎜
⎝
n−1
0
...
0
⎞
⎟
⎟
⎠
Figu e 5 shows an example.
6
