INPUT VARIABLE SELECTION FOR FORECASTING
MODELS
Manuel R. A ahal, Al onso Cepeda, Edua do F. Camacho
Dep o. Ingenie ía de Sis emas y Au omá ica. Uni e sidad de Se illa
Abs ac : The selec ion o inpu a iables plays a c ucial ole when modelling ime se ies. Fo
nonlinea models he e a e no well de eloped echniques such as AIC and o he c i e ia ha
wo k wi h linea models. In he case o Sho Te m Load Fo ecas ing (STLF) gene aliza ion
is g ea ly in luenced by such selec ion. In his pape wo app oaches a e compa ed using
eal da a om a Spanish u ili y company. The models used a e neu al ne wo ks al hough he
algo i hms can be used wi h o he nonlinea models. The expe imen s show ha ha inpu
a iable selec ion a ec s he pe o mance o o ecas ing models and hus should be ea ed
as a gene aliza ion p oblem.
Keywo ds: Au oco ela ion, Au o eg essi e models, Neu al ne wo ks, Time-se ies analysis.
1. INTRODUCTION
Some imes a model is needed o o ecas he u u e
beha io o a ime se ies. Pas alues o he ime se ies
and pas o o ecas ed alues o o he a iables can
po en ially be used as inpu s o he model. When a
g oup o (mos ly) independen a iables a e a ailable,
a iable selec ion has o be pe o med in o de o a oid
using a iables ha ha e li le in luence on he o ecas
and, a he same ime, do no neglec use ul ones.
Fo wa d inclusion and Backwa d elimina ion by means
o co ela ion ma ix a e widely used me hods. Al-
hough i has been used success ully in nonlinea sce-
na ios, i has o be no ed ha co ela ion seeks o
almos linea ela ionships among a iables. When
o he ela ionships a e p esen , such as quad a ic, he
co ela ion es can yield w ong esul s.
A b u e o ce app oach can in some cases p o ide he
bes esul s. Ei he by s ep-wise inclusion o dele ion
o a iables a each s age a ull model has o be
de eloped. The pe o mance o he model is es ed
using some con ol da a and allows o de e mine he
bes se o a iables.
This me hod has a numbe o ob ious d awbacks.
Fi s , i is e y ime consuming, and does no scale
well wi h bo h he size/complexi y o he model
and he numbe o po en ial a iables. Second, la ge
amoun s o con ol da a a e needed, o he wise he
imp o emen s in he a iable selec ion p ocedu e may
no ha e co espondence in he la e use o he model.
In many si ua ions da a is sca ce disallowing he use
o such me hod.
Sho e m elec ical load o ecas ing (STELF) is a
ask in which nonlinea i ies, model complexi y and
sca ce da a a e combined. These p oblems ha e been
add essed in (Yuan and Fine, 1998). The di e ence
index p esen ed he e is used wi h some modi ica ions
in his pape and compa ed agains he b u e o ce
app oach. A compa ison is ca ied ou using ac ual
da a om a Spanish u ili y company. Gene aliza ion
issues a e ackled wi h special ca e in he expe imen s
ha simula e he use o he models unde di e en
condi ions.
The algo i hms o a iable selec ion a e p esen ed
nex . The o ecas ing p oblem and he a ailable da a
used o he compa ison a e shown in sec ion 3. Fol-
lowing his, he esul s will be de ailed and he conclu-
sions p esen ed.
Copy igh © 2002 IFAC
www.else ie .com/loca e/i ac
Copy igh © 2002 IFAC
15 h T iennial Wo ld Cong ess, Ba celona, Spain
463
2. INPUT VARIABLE SELECTION
ALGORITHMS
Two algo i hms o a iable selec ion will be de-
sc ibed he e. The i s one consis s on s ep-wise in-
clusion o a iables in neu al models.
In bo h cases he a ailable pas da a is spli in wo se s:
one o cons uc ing models and he o he o es ing
hei goodness. E alua ion can be seen as a unc ion
ha assigns a igu e o me i o each model, allowing
hei classi ica ion. Le us deno e by E(M) he alue
o he e alua ion unc ion o model M.
Also, he se o po en ial a iables is i s p uned e-
mo ing a iables ha ha e a clea quasi-linea ela-
ionship among hem. Le us deno e by n he numbe
o emaining po en ial a iables. Ou p oblem is o
decide which o he po en ial a iables should be used
as inpu s in he model.
2.1 S ep-wise inclusion o a iables
This s a egy adds one a iable a a ime o he se o
selec ed a iables. This a iable is selec ed p obing all
possible models ha esul om he addi ion o any
a iable o he se o po en ial a iables.
The i s s age o he selec ion algo i hm is he con-
s uc ion models M1
1 o M1
nwhe e de supe index
deno es he numbe o inpu a iables used by he
model and he subindex is he a iable las added o
ha model. The model ha achie es he bes pe o -
mance indica es which a iable should be i s se-
lec ed. Ma hema ically 1=a gminjE(M1
j). Va iable
1is hen elimina ed om he se o po en ial a iables
and included in he se o selec ed ones.
In a second s ep n−1 models wi h wo inpu a iables
a e conside ed and a new a iable selec ed.
The p ocedu e can go on o e e because heo e ically
a model wi h an ex a a iable can pe o m as well as
he o iginal model. We know howe e ha gene aliza-
ion deg ades when a unnecessa y complexi y is used.
The p oblem o when o s op is some imes sol ed
plo ing he goodness o he successi e models agains
he numbe o a iables. Small changes a e expec ed
when including dispensable a iables whe eas signi i-
can imp o emen s occu when an impo an a iable
is added. Fo his eason he cu e has a la ge slope
when he i s a iables a e added and la ens succes-
si ely. The elbow o he cu e has been many imes
signaled as he co ec numbe o a iables. As we will
see in he expe imen s, his choice is coupled wi h he
pe o mance unc ion conside ed.
When using neu al ne wo ks as models o he consid-
e a ions ha e o be made. Fi s o all, he e ec s o
andom ini ial alues o weigh s and aining s opping
c i e ion cause ne wo ks wi h he same inpu s and
same aining da a o pe o m di e en ly. Second, he
numbe o nodes/connec ions also a ec he capaci y
o esul ing model. To a oid hese p oblem a numbe
o ne wo ks should be gene a ed and used as an en-
semble a e aging hei ou pu s. The ne wo ks can be
ained using di e en algo i hms and do no ha e o
sha e he same s uc u e o size.
2.2 Index based selec ion
The second algo i hm o be compa ed he e is based
on a selec ion index ha can be ob ained di ec ly
om da a wi hou he need o cons uc ing any model.
The index p oposed in (Yuan and Fine, 1998), is a
di e ence-based es ima o o esidual a iance. We
will use i he e wi h mino modi ica ions.
Gi en a se o a ailable da a o med by a pai s o inpu
ec o s and a ge alues (xi, i). The inpu ec o is
composed o ndi e en a iables. Le us deno e by
xi,j he j- h componen o he i- h da a pai .
Fo each po en ial inpu a iable jan indica ion o
i s use ulness can be ob ained by means o index
Ij. This index measu es he changes in he a ge
associa ed wi h changes in he j- h a iable. To do so,
all pai s (xi,j, i)o a iable and a ge a e conside ed
and eo de ed so ha he new pai s (xp,j, p) e i y ha
xp,j≤xq,j o any p<q.
The index is hen calcula ed as
Ij=
N
∑
h=1 (h+1)− (h)(1)
whe e Nis he numbe o da a pai s.
The way he index is used o selec ion is desc ibed
now. Po en ial inpu a iables a e i s di ided among
g oups, con aining each g oup a iables ha a e co -
ela ed. The index is ob ained o all he a iables in
each g oup. The one wi h lowe index is conside ed
he bes ep esen a i e o he g oup and is selec ed.
P oceeding in his manne , he chance o selec ing
edundan (“collinea ”) a iables is uled ou .
This p ocedu e does no cons uc any model, speed-
ing up he selec ion p ocess.
2.3 S ep-wise index based selec ion
We p opose a modi ica ion o he index based p oce-
du e. The me hod consis s on calcula ing he di e -
ence index o all a iables as be o e. Now, a e he
i s a iable has been selec ed i is elimina ed om he
se o po en ial inpu a iables. A model is cons uc ed
using his one a iable. The ou pu o he model ˆ
1is
an es ima ion o he a ge a iable. We use he e o
o he model ( he esiduals) as a new a ge : 2= −
ˆ
1. New di e ence index a e hen calcula ed o he
464
elemen s o he se o po en ial inpu a iables, and
he p ocess epea s i sel .
The numbe o models o be cons uc ed is less han in
he b u e o ce app oach ou lined in sec ion 2.1.
3. FORECASTING PROBLEM
The p oblem o sho e m load o ecas ing e e s o
he p edic ion o he ene gy demand wi hin a ho izon
o 24 o 48 hou s. Di e en app oaches ha e been used
o his p oblem anging om linea o neu al ne s and
uzzy se s. Many pape s can be ound in he li e a u e,
especially in he IEEE T ansac ions on Powe Sys ems
(see o ins ance (Chow and Leung, 1997)).
In some cases (A ahal and Camacho, 2001), he hou ly
load o ecas ing p oblem is di ided in wo: he p edic-
ion o he in eg a ed demand o a day and he no -
malized hou ly load cu e o he day. No malized load
cu es a e used o dis ibu e he in eg a ed p edic ed
demand among he 24 hou s o he day. In his pape
he o ecas ing o he load cu e o he day is no con-
side ed. The algo i hms unde conside a ion a emp
o p edic he o al (i.e. in eg a ed) ene gy demanded
o e 24 hou s o he nex wo o h ee days.
Some no a ion is needed in o de o p esen he algo-
i hm. The hou ly load ld
hdemanded a hou ho a day
dis usually he a iable used o elec ical companies
o o ecas ing pu poses. Howe e , he u ili y used as
a es bed is mainly in e es ed in p edic ing he in e-
g a ed load o a day; ha is:
c(j)=
h=24
∑
h=1
lj
h(2)
This in eg a ed load will be e e ed o as "daily load".
The sequence o alues {c(j)} o j=0,1,...,nd
cons i u es he da a base o elec ical load. Simila
da a base exis s o he empe a u es measu ed in he
egion whe e he elec ical ene gy is supplied and o
o he a iables o in e es .
Suppose ha we wish o p edic he daily load o day
k. Di e en cases appea :
(1) I kis a Monday, he p edic ion has o be made
he p e ious F iday k−3. The ad ance in he
o ecas is d=3 days.
(2) I kis a Sunday, he p edic ion has o be made
he p e ious F iday k−2. The ad ance in he
o ecas is d=2 days.
(3) I kis he day a e a holiday, he p edic ion has
o be made he wo king day be o e he holiday,
like in he p e ious case, excep in Mondays
ollowing a holiday in F iday, whe e he ad ance
is d=4 days.
(4) In he es o he cases he p edic ion is done he
p e ious day k−1. The ad ance in he o ecas
is d=1 day.
F om he cases abo e i is clea ha he alues o c(j)
a e known o all j<k−d. These pas loads can be
used o gene a e a p edic ion ˆc(k). The ha allows o
dis inguish be ween he p edic ion and he ac ual alue
o he load c(k), which is only known he day k+1.
The objec i e o a o ecas e is o p oduce ˆc(k) o
some days ahead (see abo e cases). A igu e o me i
o en used is he p edic ion e o de ined as:
ep(k)=100c(k)−ˆc(k)
c(k)(3)
his quan i y can be measu ed a e day kand is an
indica ion o how good he p edic ion was. Usually
a se o pas days wi h known load demand a e used
o e alua e he goodness o a p edic o using he oo
mean squa ed e o
E ms =1
nd
nd
∑
k=1
ep2(k)(4)
3.1 Da a se
The da a has been supplied by a Spanish elec ical
company ha does no wish o be u he iden i ied.
I consis s o hou ly powe demand o a as egion
du ing se e al yea s (see (Pa ón and A ahal, 2000)).
In o de o main ain he con iden iali y o he da a, he
plo s gi en in he pape co espond o a uniden i ied
pe iod o ime and he e ical scale is no malized.
The a ailable da a se has been spli in h ee pa s (see
igu e 1):
•Wo king se .Con ains ou yea s o dayly load.
I would be spli in wo disjoin se s, namely
Cons uc ion Se (CS) used o he aining o he
neu al ne wo ks and Valida ion Se (VS) used o
signal when o s op he aining o he ne wo ks
and o selec he mos app op ia e ne s among a
g oup o ne s.
•Tes se (TS). Con ains one yea o da a which is
pos e io o WS ha will be used o compa e he
di e en selec ion algo i hms.
The selec ion o he a iables o he algo i hm will be
ca ied ou using jus he da a in WS. On he o he
hand, he selec ion algo i hms will be es ed using
da a om he TS. I has o be ema ked ha da a
om TS will no be used (o e en disclosed) a e he
algo i hms ha e chosen hei se o inpu a iables.
4. EXPERIMENTAL RESULTS
The a iables conside ed as po en ial inpu s a e:
1=c dam Load o he p e ious day o he same
ype.
2=m7da A e age o e pas se en days.
3=m14da A e age o e pas ou een days.
465
200 400 600 800 1000 1200 1400 1600
0.75
0.8
0.85
0.9
0.95
1
1.05
1.1
1.15
1.2
1.25
days
no malized daily load
WS
TS
Fig. 1. No malized elec ical daily load in he wo king se WS=CS+VS and in he es ing se TS.
4=mdaa A e age o e a week lagged a yea .
5=mm2ma A e age o e a mon h lagged wo mon hs.
6=mm3ma A e age o e a mon h lagged h ee
mon hs.
7=mm4ma A e age o e a mon h lagged ou mon hs.
8=mm6ma A e age o e a mon h lagged six mon hs.
9= Fo ecas ed a e age daily empe a u e.
10 =m 2da A e age empe a u e o e pas wo days.
11 = d Type o day: 0-holiday {1 – 7 } Sunday
h ough Sa u day.
12 =da Day o he yea .
In o de o es he selec ion o inpu a iables a
numbe o models ha e been ob ained. Each model
di e s jus in he inpu ec o used. The models a e
neu al ne wo ks o one-hidden laye wi h 15 nodes
ained wi h he Le enbe g-Ma qua d algo i hm in
MATLAB.
T aining is pe o med using jus da a om CS o
adjus he ne wo ks pa ame e s. The E ms in he VS
is moni o ed o s op he aining i e a ions. The CS
consis s o a 20% o he da a poin s in WS andomly
selec ed. The VS is he emaining 80%.
Since he ini ial weigh s o he ne wo k can a ec he
esul s 50 ne wo ks a e cons uc ed ins ead o jus one.
Deno ing by JCS he E ms in he CS and by JVS E ms in
he VSa new igu e o me i is in oduced
JWS =0.2JCS +0.8JVS (5)
The 15 ne s ha p o ide he smalle alue o JWS a e
jus conside ed. The esul s shown in he ollowing a e
an a e age o such 15 ne s.
The numbe o possible a iables is wel e, so i would
be necessa y o gene a e 12 +11 +... +2=78 di e -
en models o ge he comple e ee o inpu a iables
ele ance. This is excessi ely ime consuming.
4.1 S ep-wise inclusion
The nex able shows he a e age alue o JWS o he
bes 15 ne s ou o 50. Each ow co espond o an inpu
a iable and each column o a s age o he inclusion
algo i hm.
I can be seen ha in he i s s age ( i s column)
all inpu a iables a e conside ed one by one. The
smalle alue appea s in he i s ow, consequen ly 1
is selec ed om he se o po en ial a iables and used
in subsequen s ages.
In he second s age 1is combined one by one wi h
he es o po en ial a iables. The smalle alue o
JWS appea s in he las ow, co esponding o 11.
This is hus he second a iable o be selec ed. I is
in e es ing o no e ha , a his s age, all models yield
be e pe o mance ha in he p e ious, as expec ed
(see sec ion 2.1).
12345
14.94
58.31 4.68 4.23 3.78 3.69
68.94 4.81 4.55 3.97 3.72
97.57 4.78 3.99 3.73
10 7.09 4.69 3.97
11 7.21 4.65
Table 1. JWS o he di e en models a each s age o
he selec ion me hod.
Using his me hod he a iables a e selec ed in his
o de : 1, 11, 10, 9and 5
4.2 Index based selec ion
Using linea co ela ion ma ix i is possible o educe
he numbe o a iables analyzed making g oups o
simila a iables as commen ed abo e. Doing so he
466
numbe o a iables can be educed o ou 1=
c dam , 4mdaa, 9= and 11 = d. These a iables
ha e p obed as he bes a iables om hei g oups.
I has o be commen ed ha his me hod and he nex
ha e a p ac ical p oblem wi h a iables ha ake dis-
c e e alues such as d. Fo his eason da is no mally
selec ed by his me hod. Howe e , in o de o compa e
wi h he p e ious one d has been selec ed ins ead o
da.
The index is calcula ed o each o he abo e p e-
selec ed a iables using da a o WS. The esul s a e
shown in he nex able.
1 4 9 11
20.64 37.58 32.96 36.03
Table 2. Values o I o he p e-selec ed a iables.
4.3 S ep-wise index based selec ion
The second index based me hod c ea es in e media e
models wi h inpu a iables elec ed ia he Ij. The
esul s a e shown in he nex able. Jus he signi ican
po ion o he able is shown o he sake o cla i y o
p esen a ion.
12345
121.2
231.6 23.8 21.0 21.3 17.6
333.5 22.6 20.6
834.2 23.5 21.0 20.9 17.4
932.3 22.4 20.6 20.6
11 36.4 21.9
Table 3. Index o he di e en a iables a each s age
o he hi d selec ion me hod.
In he i s s age he a iable wi h lowe index is 1
and hus is selec ed. Simila ly he es o a iables
conside ed a e o de ed yielding he sequence: 1, 11,
3, 9and 2.
5. COMPARISON
A di e en way o looking a he esul s o he he
di e en algo i hms is o es he imp o emen s (i
he e is any) in he models when a new inpu a iable
is added.
In igu es 2, 3 and 4 he a e age alue o JCS (da k)
and JVS ( ai g ay) is plo ed o he h ee selec ion
me hods agains he numbe o inpu a iables.
Acco ding o he wo i s me hods he bes esul s a e
ob ained wi h i e inpu a iables. The hi d me hod
sugges s ha ou a iables a e bes .
We ha e o keep in mind ha hese selec ion has
been pe o med using pas da a. The goodness o he
models when con on ed wi h new da a has ye o be
es ed. To his end we disclose he TS se and use e e y
Fig. 2. JCS (da k) and JVS ( ai g ay) o he models
de i ed o he i s selec ion me hod.
Fig. 3. JCS (da k) and JVS ( ai g ay) o he models
de i ed o he second selec ion me hod.
Fig. 4. JCS (da k) and JVS ( ai g ay) o he models
de i ed o he hi d selec ion me hod.
model o o ecas in his new condi ions. This es is a
hough one since no addi ional aining was pe o med
and he TS se span one whole yea in he u u e.
Fo each model, he abo e selec ed 15 ne s a e used
wi h he new da a se , ob aining E ms in he usual
way. The a e age o hese oo mean squa ed e o s
is JTS, which is plo ed in igu e 5. The squa e ma k
467

co espond o me hod 1, he iangle o me hod 2 and
he ci cle o me hod 3.
1 1.5 2 2.5 3 3.5 4 4.5 5
3.5
4
4.5
5
5.5
6
6.5
7
no. o a iables
RMS E o
Fig. 5. JTS e sus he numbe o inpu a iables o
each model using a iables selec ed by he i s
me hod (squa es), he second ( iangles) and he
hi d me hod (ci cles).
I can be seen ha ca e has o be aken since he
bes esul s a e ac ually ob ained wi h jus h ee inpu
a iables. I is also in e es ing o no ice ha o ou
a iables he h ee me hods ob ain e y close and
accep able solu ions.
A conclusion we d aw in he ligh o his compa ison
is ha index-based inpu a iable selec ion is easy o
pe o m and less ime consuming and can (as in his
case) lead o success ul selec ion.
Also, i is wo h no ing ha he way he neu al ne s
ha e been ained and selec ed g ea ly a ec s gene al-
iza ion. A he ime o closing his d a e sion we
ha e ye no comple e esul s on how his selec ion
should be pe o med so as o a oid (as much as ha
is possible) he misma ch be ween E(M)and JTS.Fo
ins ance, p io o he cu en ly used e alua ion unc-
ion o he pe cen ages o CS and VS whe e used wi h
poo e esul s.
I seems ha he i s index-based me hods a oids
he o e pa ame e iza ion ap by using geome ical
in o ma ion whe eas model-based selec ion me hods
need o be e y ca e ul in his espec .
6. CONCLUSIONS
Th ee me hods o inpu a iable selec ion ha e been
compa ed using eal da a in he p oblem o sho e m
load o ecas ing and he p oblems associa ed wi h
each app oach commen ed.
F om he esul s gi en i is clea ha inpu a iable se-
lec ion can se iously a ec gene aliza ion. Mo eo e ,
i has been shown ha he p oblem o inpu a iable
selec ion, neu al model assessmen and gene aliza ion
a e coupled.
Fu he esea ch will be aimed a de eloping neu al
model es ing p ocedu es ha would selec ne s ha
ha e he bes gene aliza ion capabili ies.
7. REFERENCES
A ahal, M.R. and E.F. Camacho (2001). Neighbo
his o ies o sho e m load o ecas ing. P o-
ceedings o he Eu opean Con ol Con e ence
pp. 2796–2801.
Chow, T.W.S. and C.T. Leung (1997). Neu al ne wo k
based sho - e m load o ecas ing usign wea he
compensa ion. IEEE T ansac ions on on Powe
Sys ems 11, 1736–1742.
Pa ón, F. and M.R. A ahal (2000). P edicción a co o
plazo de la demanda de la ene gía eléc ica con
siconel. (in spanish) Ac as de las XXI Jo nadas
de Au omá ica, CD-ROM 1, ja00_002.
Yuan, J.-L. and T.L. Fine (1998). Neu al-ne wo k de-
sign o small aining se s o high dimension.
IEEE T ansac ions on neu al ne wo ks 9, 266–
280.
468