Odhiambo, Joab; Weke, Pa ick; Nga e, Philip
A icle
A deep lea ning in eg a ed Cai ns-Blake-Dowd (CBD)
sy ema ic mo ali y isk model
Jou nal o Risk and Financial Managemen
P o ided in Coope a ion wi h:
MDPI – Mul idisciplina y Digi al Publishing Ins i u e, Basel
Sugges ed Ci a ion: Odhiambo, Joab; Weke, Pa ick; Nga e, Philip (2021) : A deep lea ning in eg a ed
Cai ns-Blake-Dowd (CBD) sy ema ic mo ali y isk model, Jou nal o Risk and Financial Managemen ,
ISSN 1911-8074, MDPI, Basel, Vol. 14, Iss. 6, pp. 1-12,
h ps://doi.o g/10.3390/j m14060259
This Ve sion is a ailable a :
h ps://hdl.handle.ne /10419/239675
S anda d-Nu zungsbedingungen:
Die Dokumen e au EconS o dü en zu eigenen wissenscha lichen
Zwecken und zum P i a geb auch gespeiche und kopie we den.
Sie dü en die Dokumen e nich ü ö en liche ode komme zielle
Zwecke e iel äl igen, ö en lich auss ellen, ö en lich zugänglich
machen, e eiben ode ande wei ig nu zen.
So e n die Ve asse die Dokumen e un e Open-Con en -Lizenzen
(insbesonde e CC-Lizenzen) zu Ve ügung ges ell haben soll en,
gel en abweichend on diesen Nu zungsbedingungen die in de do
genann en Lizenz gewäh en Nu zungs ech e.
Te ms o use:
Documen s in EconS o may be sa ed and copied o you pe sonal
and schola ly pu poses.
You a e no o copy documen s o public o comme cial pu poses, o
exhibi he documen s publicly, o make hem publicly a ailable on he
in e ne , o o dis ibu e o o he wise use he documen s in public.
I he documen s ha e been made a ailable unde an Open Con en
Licence (especially C ea i e Commons Licences), you may exe cise
u he usage igh s as speci ied in he indica ed licence.
h ps://c ea i ecommons.o g/licenses/by/4.0/
Jou nal o
Risk and Financial
Managemen
A icle
A Deep Lea ning In eg a ed Cai ns-Blake-Dowd (CBD)
Sy ema ic Mo ali y Risk Model
Joab Odhiambo * , Pa ick Weke and Philip Nga e


Ci a ion: Odhiambo, Joab, Pa ick
Weke, and Philip Nga e. 2021. A
Deep Lea ning In eg a ed
Cai ns-Blake-Dowd (CBD) Sy ema ic
Mo ali y Risk Model. Jou nal o Risk
and Financial Managemen 14: 259.
h ps://doi.o g/10.3390/j m14060259
Academic Edi o s: Michael McAlee
and Shigeyuki Hamo i
Recei ed: 8 May 2020
Accep ed: 11 June 2020
Published: 8 June 2021
Publishe ’s No e: MDPI s ays neu al
wi h ega d o ju isdic ional claims in
published maps and ins i u ional a il-
ia ions.
Copy igh : © 2021 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license (h ps://
c ea i ecommons.o g/licenses/by/
4.0/).
School o Ma hema ics, Uni e si y o Nai obi, Nai obi Ci y 30197-00100, Kenya; [email p o ec ed] (P.W.);
[email p o ec ed] (P.N.)
*Co espondence: [email p o ec ed]
Abs ac :
Many ac ua ial science esea che s on s ochas ic modeling and o ecas ing o sys ema ic
mo ali y isk use Cai ns-Blake-Dowd (CBD) Model (2006) due o i s abili y o conside he coho
e ec s. A h ee- ac o s ochas ic mo ali y model has h ee pa ame e s ha desc ibe he mo ali y
ends o e ime when dealing wi h u u e beha io s. This s udy aims o p edic he ends o
he model,
k(2)
by applying he Recu en Neu al Ne wo ks wi hin a Sho -Te m Long Memo y
(an a i icial LSTM a chi ec u e) compa ed o adi ional s a is ical ARIMA (p,d,q) models. The
no el deep lea ning (machine lea ning) echnique helps in eg a e he CBD model o enhance i s
accu acy and p edic i e capaci y o u u e sys ema ic mo ali y isk in coun ies wi h limi ed da a
a ailabili y, such as Kenya. The esul s show ha Long Sho -Te m Memo y ne wo k a chi ec u e
had highe le els o p ecision when p edic ing he u u e sys ema ic mo ali y isks han adi ional
me hods. Ul ima ely, he esul s can be implemen ed by Kenyan insu ance i ms when modeling and
o ecas ing sys ema ic mo ali y isk help ul in he p icing o Annui ies and Assu ances.
Keywo ds:
sys ema ic mo ali y isk; deep lea ning; long sho - e m memo y; CBD; ecu en neu al
ne wo ks
1. In oduc ion
Since he s a o he 21s cen u y, mo ali y a es ha e been dec easing s eadily due
o se e al ac o s such as imp o ed medical in en ions, obo ic su ge y, be e heal hca e
sys ems, and be e die s, among many o he ac o s, see (Boo and Choi 2020;Chen 2020;
Kilic 2020;Pou homayoun and Shakibi 2020). These ac o s ha e p omp ed ac ua ies,
demog aphe s, and s a is icians o hink o no el ideas o do mo ali y modeling and
o ecas ing o an inc eased le el o p ecision in he models. While his is a good idea o
he gene al global popula ion, many go e nmen s, li e assu ance i ms, and li e pension
companies ha e subs an ial inancial losses since hey canno make p ecise es ima ions
when o e ing inancial se ices. Co ec mo ali y isk es ima ion is i al in hei inancial
su i al, especially a e he ha d imes o he global Co id-19 pandemic see (Pou homay-
oun and Shakibi 2020) ha is likely o lead o massi e global economic ecession a ec ing
many na ions, bo h i s -wo ld and hi d-wo ld coun ies.
Today, in he ac ua ial li e a u e, we ha e many e ined echniques ha many ac ua ies,
s a is icians, and demog aphe s use when o ecas ing u u e mo ali y and sys ema ic
longe i y isks. We es ima e he comple e li e expec a ions o hose who wish o buy
annui ies and li e assu ance p oduc s sold in he ma ke . F om (Lee and Ca e 1992),
he e a e many s ochas ic mo ali y models cu en ly used when modeling and o ecas ing
sys ema ic mo ali y isk. Howe e , hese models ha e di e en s eng hs and weaknesses
depending on he da a a ailabili y and he numbe o a ailable pa ame e s ha need o be
de e mined o es ima ed. (Cai ns e al. 2006) model imp o es some o he weaknesses o
he (Lee and Ca e 1992) model by inco po a ing he coho e ec s and double pa ame e
abili y. Modeling a sys ema ic mo ali y isk ha is cons an o e age while p e en ing
o e lapping o he age lines du ing o ecas ing leading desi able esul s as a pa simonious
J. Risk Financial Manag. 2021,14, 259. h ps://doi.o g/10.3390/j m14060259 h ps://www.mdpi.com/jou nal/j m
J. Risk Financial Manag. 2021,14, 259 2 o 12
model was demons a ed by (Cai ns e al. 2011), which leads o high ce ain y le els and
accep abili y in he esul s.
Many esea che s, including (Hainau 2018) in his pape , p oposed a neu al ne wo k
capable o p edic ing and simula ing u u e sys ema ic mo ali y isk. Du ing his esea ch,
he au ho used a neu al analyze when de ec ing la en ime p ocesses while di ec ly
p edic ing mo ali y. The app oach did allow o iden i ica ion and duplica ion o non-
linea i y obse ed in he changes o logi o ces o mo ali y. In addi ion, (Dep ez e al.
2017) used some machine lea ning echniques o imp o e he es ima ion p ocess o he
logi mo ali y isk. This wo k was ex ended by (Le an esi and Pizzo usso 2019) o he
amewo k o mo ali y o ecas ing as in he [3] model. Fu he mo e, a ecen pape by
(Richman and Wü h ich 2018)
p oposed mul iple-dimensional popula ions o (Lee and
Ca e 1992) model whe e i es ima ed he pa ame e s using a i icial neu al ne wo ks.
Many o he ele an machine lea ning uses and applica ions in an ac ua ial ield a e
discussed by (Cas ellani e al. 2018) and (Gab ielli e al. 2020), especially when looking a
he u u e o sys ema ic mo ali y isk modeling me hodologies.
In his esea ch s udy, we use a deep lea ning echnique o imp o e he p edic i e
capabili y o he (Cai ns e al. 2006) model. To be mo e speci ic, ou app oach aims a
In eg a ing he o iginal (Cai ns e al. 2006) o mula ion by he in oduc ion o an a i icial
Recu en Neu al Ne wo ks wi h Long Sho -Te m Memo y o LSTM a chi ec u e when
o ecas ing u u e e olu ion o he
k(2)
pa ame e hus o e coming he challenges showed
by he adi ional ARIMA
(p
,
d
,
q)
ime se ies p ocess. The choice o he CBD model ins ead
o o he s anda d mo ali y models is based on he ac ha CBD sol es he p oblem
o coho e ec in mo ali y synonymous wi h o he mo ali y models. In addi ion, i
inco po a es he e ec o coho s in models compa ed o o he s used in modeling o
sys ema ic mo ali y isk.
Using LSTM allows mo e cohe ency when de e mining mo ali y o ecas s wi h high
dynamism o obse ed mo ali y, especially when dealing wi h nonlinea mo ali y ends.
To be mo e p ecise, he LTSM ne wo k is s uc u ed o help elabo a e long da a sequences
o o m a memo y capable o p ese ing he i al ela ionships be ween he a ailable
da a and e e y de ia ion wi hin hese sequences. In a simila sense, wi hin he con ex
o adi ional ime se ies, he LSTM gi es oom o p edic ing u u e mo ali y o e ime
by conside ing he subs an ial in luence o he his o ical sys ema ic mo ali y isk ends
be o e adequa ely ep oducing i in o he o ecas ed end. In addi ion, he powe o LSTM
is by p ese ing he in o ma ion o e a gi en pe iod, he e o e blocking he olde signals
om slowly disappea ing du ing p ocessing.
While he esea ch ocuses on o ecas ing sys ema ic mo ali y isk ends, pa ame e
es ima ion me hodology emains simila as o (Lee and Ca e 1992). The pape does
in oduce a new me hod o mo ali y i ing su ace as by (Hainau 2018) ha applies he
use o neu al ne wo ks o deep lea ning echnique o i ing mo ali y a es as opposed o
he con en ional SVD me hod (Singula Value Decomposi ion). This s udy in oduces a
no el me hodology s uc u e based on he LSTM ne wo k when modeling u u e common
ends o sys ema ic mo ali y isk.
2. Cai ns-Blake-Dowd (CBD) Model
De ini ion 1. Le he (Cai ns e al. 2006) be;
logi µ(x, )=α(1)
xk(1)
+α(2)
xk(2)
+α(3)
xw(3)
−x(1)
The coho e ec in luence,
w(3)
−x
, o any age-speci ic coho has been assumed o
educe o ze o wi h ime.
α(3)
x
dec eases wi h
x
as opposed o being a cons an i.e.,
α(3)
x
=
c
whe e cis a cons an i sel . The e o e, his will gi e us he model as
logi µ(x, )=α(1)
xk(1)
+α(2)
xk(2)
+α(3)
xw(3)
−x(2)
J. Risk Financial Manag. 2021,14, 259 3 o 12
whe e
α(1)
x=
1,
α(2)
x= (x−¯
x)
,
α(3)
x= (xc−x
). Wi h he eplacemen o he alues,
we ha e:
logi µ(x, )=k(1)
+k(2)
(x−¯
x) + w(3)
−x(xc−x)(3)
Du ing he analysis, we ha e o use he cons ain
∞
∑
i=1
w(3)
−x=
0 o p e en in oducing
he iden i iabili y p oblem du ing he p ocess o es ima ion as well as p ojec ion. The model
has no p oblems o iden i ica ion. In he o iginal (Cai ns e al. 2006) model, esea che s
o en used
SVD
o singula alue decomposi ion when es ima ing pa ame e s as pe he
2-s age p ocedu e. This is done by applying i o he ma ix o
logi µ(x, )
as a way o inding
alues o
k(1)
o hus ob aining alues
k(2)
and
(x−¯
x)
espec i ely. Secondly, o ensu e
ha obse ed dea hs coincide wi h he es ima ed dea hs, k(2)
is e i ed.
Lemma 1.
As pe he adi ional (Cai ns e al. 2006) o mula ion,
k(2)
is o en modeled using an
Au o- eg essi e In eg a ed Mo ing A e age (0, 1, 0)as;
k(2)
=k(2)
−1+δ+w(3)
−x(4)
whe eas δis de ined d i pa ame e and w(3)
−xa e he andomness e m and w(3)
−x∼N(0,σ2
k).
3. The Neu al Ne wo k Model
3.1. A i icial Neu al Ne wo k De ini ion
De ini ion 2.
An ANN (A i icial Neu al Ne wo k) is a se ies o algo i hms ha endea o s o
iden i y unde lying ela ionships in a gi en da a se ia a p ocess capable o mimicking how a human
b ain wo ks. A i icial neu al ne wo k a chi ec u e includes neu ons, he synap ic connec ions,
which link he neu ons, and lea ning algo i hms.
ANN is o med h ough 3 ca ego ies o laye s, known as hidden, inpu , and an ou pu laye
espec i ely whe e each one o he laye s is made up o se e al neu ons (Hassoun e al. 1995). E e y
uni in an a i icial ne wo k ob ains
“p opo ional”
in o ma ion ia synap ic links om many
o he well connec ed ones a he same ime e u ning an ou pu h ough using an a i icial ac i a ion
unc ion ha ans o ms hese p opo ional o als o he inpu signals.
3.2. Deep Lea ning Modeling
De ini ion 3. Le Q deno e a single neu on called pe cep on de ined by;
Q=Θ(ZTy+c)(5)
whe e
yeR
is he inpu and
ZeR
is he connec ed synap ic weigh ,
ψeN
a e numbe s o he
inpu signals and
Θ
is he ac i a ion unc ion. We ep esen his e m
c
as he bias ha is associa ed
wi h he model known as ac i a ion e ge o h eshold. The use mus no e ha he unc ion,
Θ
,
should ha e a di e en ial because he lea ning equa ions ha e g adien s (Minsky and Pape 2017).
We in oduce Mul ilaye Pe cep on ( MLP ) used in nonlinea sepa able p oblems
such as Exclusi e o (XOR) since ANN wi h a single laye is always inapp op ia e, hus
sol ing he s a ed p oblem. In addi ion, mos neu ons in MLP a e p edisposed on a
wide a ie y o laye s, wi h e e y uni ully connec ed o hose o he p eceding laye , as
illus a ed by (Good ellow e al. 2016). The synapses connec uni s by de ining di e en
ypes o a ailable ne wo ks in he sys em. In an ANN classical pa e n like eed- o wa d
ANN, he in o ma ion mo es in a unila e al di ec ion om an inpu o an ou pu laye a
he same ime he Recu en Neu al Ne wo ks (commonly known as RNNs) p ocesses he
in o ma ion cyclically using he ex a synapses o ensu e ha he ep ocessed ou is as a
esul o he en i e elabo a ion p ocess.
J. Risk Financial Manag. 2021,14, 259 4 o 12
Figu e 1below shows he s anda d ep esen a ion o eed- o wa d ANN. A neu on
is ep esen ed in e e y node, connec ed om one o he o he using a cs ep esen ing
all synapses. Addi ionally, he g aph ep esen s he gene al inpu , la en , as well as
ou pu a iables.
The Schema ical iew o an a i icial neu al ne wo k (ANN) below has ci cles ep e-
sen ing neu ons wi h lines ep esen ing synapses. The Synapses ake he indi idual inpu s
be o e mul iplying hem by a “weigh ” commonly known as inpu “s eng h” o de e mine
he gene al ou pu . In addi ion, Neu ons a e added o hese ou pu s om all a ailable
synapses be o e applying he ac i a ion unc ion.
Figu e 1. A No mal ep esen a ion o eed- o wa d ANN.
De ini ion 4.
F om he ou pu , le
QeRkh
deno e a gene ic hidden laye ha ing
kh
neu ons
de ined as;
Q1=Θ(ZTy+c)(6)
whe e
ZeRψ∗kh
is de ined as a weigh ma ix and
ceRkh
is called he biases ec o . Acco ding o
MLP scheme, he hidden laye ou pu becomes he inpu ins umen o he ollowing laye .
Lemma 2.
Conside ing a gi en p oblem o eg ession, whe e
eN
de ined as he numbe hidden
laye s, hen he ou pu o ˆ
yeRcan be calcula ed by:
Q1=Θ1(ZT
1y+c1)
Q2=Θ2(ZT
2Q1+c2)
Q3=Θ3(ZT
3Q2+c3)
.........
ˆ
y=Θ (ZT
Q −1+c )
whe e
Z1
,
Z2
,
Z3
, ...
Z
deno e weigh ma ix ec o s,
c1
,
c2
,
c3
, ...
c
deno e bias ec o s, and
Φ1
,
Φ2
,
Φ3, ...Φ deno e ac i a ion unc ions ha needs no be di e en om one ano he .
I is i al o no e ha all measu emen s o he weigh ma ices and bias ec o s do ely
on he uni numbe wi hin he hidden laye s; hence, by enhancing hese hidden laye s in
numbe s, he abs ac ion le els o he inpu da a also inc ease signi ican ly.
3.3. Backwa d P opaga ion o E o s
De ini ion 5.
Backp opaga ion is an algo i hm used o supe ised lea ning o a i icial neu al
ne wo ks h ough g adien descen . P o ided an a i icial neu al ne wo k (
ANN
) and an e o
unc ion, his me hod is capable o calcula ing he e o unc ion g adien wi h espec o he
espec i e weigh s o neu al ne wo ks.

J. Risk Financial Manag. 2021,14, 259 5 o 12
ANN aining in ol es he use o a gi en uncons ained op imiza ion p oblem wi h he aim o
minimizing a unc ion wi hin he high dimensional space. We s a by de ining a loss unc ion as:
B=
∑
i=1
( i−ˆ
)2
2(7)
This loss unc ion measu es he de ia ions o p edic ed alues
ˆ
om he obse ed
ones
i.e., i ob ains he absolu e e o e ms be ween hese p edic ed alues o
ˆ
as well
as obse ed alues o
. The quan i y
B
also elies on he weigh s o he ma ices namely
Z1
,
Z2
,
Z3
, ...
Z
, which ul ima ely in luences he alues o p edic ed
ˆ
. Consequen ly, he
aim o he me hod is o ind he exac synap ic weigh alues, which minimizes he alue
o quan i y B.
While machine lea ning has many algo i hms applied in i s applica ion, backp opa-
ga ion is among he mos commonly used eed- o wa d aining ANNs. The algo i hm
wo ks by compa ing he p edic ed alues e sus he expec ed ones acco ding o modi ying
he synap ic weigh s h ough back-p opaga ing he loss unc ion’s g adien .
F om Figu e 1, he p ocedu e con inuously al e na es o wa d wi h backwa d p op-
aga ion in he ollowing s eps, namely in he o wa d s ep, he p edic ed alues o
ˆ
a e
calcula ed by ixing he espec i e synap ic weigh s, and in his backwa d s ep, he adjus
weigh s hus educing he e o
B
o he ne wo k. I is impo an o no e ha ANN can
i e a i ely pe o m bo h o wa d and backwa d p opaga ion by modi ying he weigh s o
ind he combina ion, which minimizes he o e all loss unc ion.
De ini ion 6.
Analy ically, backp opaga ion algo i hm upda es all weigh s o
Z
in he las laye
by he ule o del a as ollows;
∆Z =−i∂B
∂Z0
(8)
whe e
i
is called he lea ning a e. As o o he p eceding laye s, we di e en ia e using p oduc o
chain ule o di e en ia ion. The o he weigh s ma ix Z −1a e de e mined as:
∆Z −1=−i∂B
∂Q −1
∗∂Q −1
∂Z −1
(9)
and he p ocess con inuous on o many o he laye s in he sys em.
We look in o he same idea in a igu a i e way, jus like a g adien o slope descen
simila o a “climbing down a s eep hill” so long as i eaches a local minimum o global
limi . Howe e , a e e y upda e, he sea ch does mo e in he g adien ’s opposi e di ec ion
while he slope o he g adien and lea ning a e is de e mined by he Mo emen ampli ude
(Baydin e al. 2017). Mo eo e , he choice o a e i is a i al elemen , as a small alue
can lead o se e al i e a ions simul aneously; la ge alues migh pe mi con e gence,
especially o a global minimum.
We choose om a wide ange o a chi ec u e, including he hidden laye s numbe s,
uni s o e e y laye , and he hype -pa ame e alues like lea ning a e, epochs, and
ac i a ion unc ion, which emain ano he heu is ic p oblem o ANN use s. I is impo an
o no e ha he choice will always depend on he da a ype a ailable, which migh be
a di icul s ep o jus easy. An ini ial ound o he hype -pa ame e s uning, especially
be o e he es ing, migh be highly needed. Addi ional ex ensi e desc ip ions o ANNs and
back-p opaga ion algo i hms a e explained well (Alpaydin 2016) and (James e al. 2013).
3.4. Recu en Neu al Ne wo k Using a Long Sho -Te m Memo y A chi ec u e
P oposi ion 1.
We inco po a e he concep o Deep Lea ning echniques in s ochas ic mo ali y
modelling o inc ease hei p edic abili y and o ecas ing accu acy.
J. Risk Financial Manag. 2021,14, 259 6 o 12
P oo .
The eed o wa d ANNs, which always ep esen a powe ul ool o analysis, can
be insu icien when e ec i ely managing ime sequences o he a ailable da a. Howe e ,
he ecu en connec ions be ween nodes ha ha e ea u ed he RNNs allow o an ac i e
analysis o he gi en sequen ial da a. Ne e heless, h ough applying he gi en RNN
s uc u e, we o en ace he massi e p oblem o g adien s disappea ing and weigh s change,
be o e becoming iny as o show no e ec . Consequen ly, he ne wo k will g adually lose
i s capabili y o lea ning om he pas o become ope a ionally insu icien o he mo e
p olonged da a sequences analysis and hus helping in making excellen p edic ions. I is
why we say ha RNNs possess a sho memo y only.
As a way o o e coming he s a ed p oblem, (Hoch ei e and Schmidhube 1997)
had come up wi h he Long Sho -Te m Memo y, commonly abb e ia ed as LSTM. The
LSTM is a e sion o RNN whose a chi ec u e can allow conside a e ela ionships be ween
he sequence o da a, e en i i happens in he long un, hus e adica ing he anishing
g adien p oblem in he p ocess. Simila ly, RNNs need bo h long- and sho -memo y,
hus managing o gene a e an ex ao dina y pe o mance in he analysis o ime se ies.
Howe e , se e al imp o emen s in he o iginal wo k, LSTM, ha e been imp o ed h ough
a se ies o s udies such as (Bahdanau e al. 2014) and (Cho e al. 2014). Ul ima ely, one can
de ine an excellen ly composed basic s uc u e as anilla LSTM.
De ini ion 7.
F om Figu e 2, Le
=g
,
i =
and
o
deno e he ou pu ha would be impo an
in RNN analysis. Le ou pu o he auxilia y-ou pu ga e be de ined as;
g = (Z y +U Q −1+c )(10)
= (Ziy +UiQ −1+ci)(11)
o = (Zoy +UoQ −1+co)(12)
j = (Zjy +UjQ −1+cj)(13)
The o ge ga e ou pu
g
as de ined by Equa ion (10), illus a es ac s om he p e-
ceding cell s a e as well as he one o igina ing om he p esen inpu a e mixed wi hin a
nonlinea way h ough a sigmoid ac i a ion unc ion. A e wa ds,
g
is mixed h ough
a poin -wise p oduc especially wi hin i s p e ious memo y s a e
c( −
1
)
. I s inpu ga e
, as de ined in Equa ion (11), uses an ac i e sigmoid ac i a ion, which pe mi ing o
decisions when in o ma ion is ecei ed be o e i is upda ed. The ou pu ga e
o
, as de ined
in Equa ion (12), plays he ole o p e en ing non-signi ican memo y con en ansmission
ha is s o ed in o ma ion wi hin he o he blocks. I s ole as a sigmoid unc ion is o
pass app op ia e memo y in o ma ion. As a way o egula ing p ocessed da a low, he
inpu ga e i does combines wi h ha de i ed om all linked auxilia y NN
j
as de ined in
Equa ion (13).
De ini ion 8.
Le deno e he en i e inpu block p ocessing p ocedu e ha pa icipa es in cons uc-
ion o he p esen memo y cell s a e as:
c( ) = c( −1)~g + ~j
To ge he cu en ou pu , which is a combina ion in be ween he de ined unc ion in
abo e equa ion;
Q=Φ(c( )) ∗o (14)
F om Equa ion (14), his LSTM a chi ec u e o e s an ou s anding ool when dealing
wi h o ecas ing ime se ies, pa icula ly in cases o longe ime lag connec ions, ca ching
andomness, and managemen o he noise. Ne e heless, any use o LSTM, jus ANNs
in gene al, mus ha e he ace o he classical p oblems ha conce n he hype pa ame-
e s choices.
J. Risk Financial Manag. 2021,14, 259 7 o 12
Figu e 2. A LSTM Block S uc u e wi h I s In e nal In o ma ion Fo wa d Flow Design.
4. Ma hema ical Applica ion and Resul s
In his a ea, we in oduce he LSTM and RNN a chi ec u es wi hin he s anda d
scheme o he CBD model. Mo e dis inc ly, he s udy’s objec i e is o exploi he ad an ages
and unc ionali ies o he LSTM a chi ec u e o imp o e he CBD model p edic i e capaci y.
Fo his aim, we design se e al expe imen s o es LSTM skills in o ecas ing u u e
sys ema ic mo ali y isk o e ime be o e compa ing i s pe o mance wi h he esul s
de i ed om he model o ARIMA.
Thus, he analysis o he s udy will conce n on he ime index
k(2)
end p edic ion,
bea ing in mind he ARIMA
(p
,
d
,
q)
model as he o ecas ed benchma k, whe eas o he pa-
ame e s
k(1)
and
(x−¯
x)
a e de e mined as pe he es ima ion me hod by
(Cai ns e al. 2006).
Dis inc ly, he CBD model ha applies a simple andom walk p ocess wi h d i is i al
o calib a e he bes ARIMA (p,d,q), as illus a ed by (Hyndman and Khandaka 2007). This
p ocedu e checks he ime se ies s a iona i y in he ini ial ound using a sui able uni a y
oo es be o e choosing he di e encing o de d. The 2nd s age de e mines he au o-
eg essi e bes alues and mo ing a e age o de , like p and q, espec i ely, using exac
in o ma ion c i e ia o AIC o BIC. In mos cases, he implemen ed algo i hm u ilizing he
unc ion, which is p esen in he py hon package o o ecas ing (Hyndman and Khandaka
2007); and (Baue e al. 2020).
P oposi ion 2.
The pe o mance o ARIMA (p,d,q) is compa ed wi h ha o LSTM. The LSTM
looks like a smoo h, na u al compe i o o ARIMA (p,d,q) because i can cap u e a long- e m
sequence o pa e n wi hin sequen ial da a. We s a building an LSTM model, which enume a es
he s a ed unc ion linking k(2)
o he ime lags, as:
k(2)
= (k(2)
−1,k(2)
−2,k(2)
−3,k(2)
−4, ....k(2)
−j) + w(3)
−x(15)
whe e
jeN
is de ined as he numbe o ime lags being conside ed and
w(3)
−x
is he homoschedas ic
e o o andomness e m.
P oo .
The LSTM ne wo k, jus like many o he s anda d machine lea ning me hods,
needs he da ase di iding in o es ing and aining se s. The aining se o en ep esen s
supe ised lea ning, whe eas es ing is o he alida ion o he model. Table 1shows a
supe ised lea ning da ase , which is help ul o p edic ion. Upon comple ion o aining,
he ne wo k will ha e lea ned he inpu -ou pu unc ional ela ionship, hus p edic ing
u u e alues o
k(2)
by using only he inpu . To be mo e p ac ical, aking he inpu as
(m"J)
ma ix wi h ime lags o
k(2)
as well as he ou pu as he
(m"
1
)
ec o o bes
cu en alues, wi h meNis he numbe as in Table 1.
J. Risk Financial Manag. 2021,14, 259 8 o 12
Table 1. Supe ised Lea ning Da aSe .
Ou pu Inpu
k(2)
k(2)
−1k(2)
−2.... k(2)
−j
k(2)
+1k(2)
k(2)
−1.... k(2)
−j+1
k(2)
+2k(2)
+1k(2)
.... k(2)
+j−2
k(2)
+3k(2)
+2k(2)
+1.... k(2)
+j−3
.... .... .... .... ....
k(2)
+mk(2)
+m−1k(2)
+m−2.... k(2)
+m−j
The p edic ed
k(2)
alues, a ime
m+
1,
m+
2,
m+
3, ...,
m+J
, a e done ecu si ely.
Gene ally, he p edic ed alues o
k(2)
in a gene ic ime
m+
is de e mined using he alues
o
k(2)
wi h
= (m+λ−
1,
m+λ−
2,
m+λ−
3, ...,
m+λ−J
) as inpu . The alues o
k(2)
a e de e mined by he p edic ed as opposed o obse ed alues. We s a by es ima ing
he CBD model pa ame e s
k(1)
,
(x−¯
x)
and
k(2)
using he SVD me hod. The ex ac ed
ime se ies o
k(2)
is deno ed as he i s base o ou analysis. The da a is hen spli in o
aining se and es ing se as pe 80% aining and 20% es ing ule. Consequen ly, we
de e mine he las yea
T
o obse a ion. We ha e done he analysis o he U.K. and Kenya
di e en ia ing h ough gende wi h one- ime lag (j=1)in Table 2.
Table 2. Tes ing se yea s as pe Na ions.
Na ions Numbe o Yea s Yea s o Tes ing Se
U.K. 1930–2018 1998–2018
Kenya 2010–2020 2010–2020
When selec ing he op imum hype pa ame e s combina ion o he neu al ne wo k,
i is essen ial o ca y a p elimina y ine- uning ound o all hese coun ies while dis in-
guishing hem by gende (see Table 3). In his s ep, we can ge combina ions, which will be
used du ing LSTM calib a ion du ing he o ecas ing p ocedu e. On he uning esul s, we
ha e disco e ed ha his a chi ec u e ha ing one hidden laye does pe ume be e han
o he s on ou da a and he numbe o neu ons depending on he coun y. Using a Rec i ied
Linea Uni (ReLU) as an ac i a ion unc ion ou pe o med many o he unc ions when
es ing many o he coun ies. Mo eo e , he e is no clea e idence on he in luence o he
pe o mance o hype -pa ame e s.
Table 3. ARIMA by Na ion and Gende .
Na ion ARIMA Model (p,d,q)
U.K.
Males ARIMA (1,1,0)
Females ARIMA (1,1,0)
Kenya
Males ARIMA (0,1,3)
Females ARIMA (0,1,3)
A e he calib a ion s ep, he pape ’s analysis will include nume ical and g aphical
p ocessing and p esen a ion o he goodness o i . To be speci ic, he s udy will ollow
he app oach o ou o sample, which deno es he es ing s ep wi hin he ield o machine
lea ning. The es ima ion o pa ame e
k(2)
pa ame e is de e mined using SVD, as o male
and emale espec i ely. Figu e 3dashed e ical line shows a sepa a ion o he o ecas ed
pe iod compa ed o one used in aining he LSTM ne wo k. As o ARIMA models, i