ESTIMATES AND BOOTSTRAP CALIBRATION
FOR FUNCTIONAL REGRESSION
WITH SCALAR RESPONSE
Adela Ma ´ınez Cal o
Depa amen o de Es a ´ıs ica e In es igaci´on Ope a i a
Uni e sidade de San iago de Compos ela
iii
Don Wenceslao Gonz´alez Man eiga, ca ed ´a ico do Depa amen o de Es a ´ıs ica e In es igaci´on Op-
e a i a da Uni e sidade de San iago de Compos ela, Don F ´ed´e ic Fe a y, ca ed ´a ico do Ins i u de
Ma h´ema iques de Toulouse da Uni esi ´e Paul Saba ie – Toulouse III, e Don Philippe Vieu, ca ed ´a-
ico do Ins i u de Ma h´ema iques de Toulouse da Uni esi ´e Paul Saba ie – Toulouse III, in o man
que a memo ia i ulada
ESTIMATES AND BOOTSTRAP CALIBRATION
FOR FUNCTIONAL REGRESSION WITH SCALAR RESPONSE
oi ealizada baixo a s´ua di ecci´on po Dona Adela Ma ´ınez Cal o, es imando que a in e esada se
a opa en condici´ons de op a ao g ao de Dou o , polo que solici an que sexa admi ida a ´ami e pa a
a s´ua lec u a e de ensa p´ublica.
En San iago de Compos ela, a 21 de xanei o de 2013.
Os di ec o es:
P o . D . Wenceslao
Gonz´alez Man eiga P o . D . F ´ed´e ic Fe a y P o . D . Philippe Vieu
A dou o anda:
Adela Ma ´ınez Cal o
i
Con en s
P e ace xi
1 In oduc ion o FDA 1
1.1 Func ional da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Wha a e unc ional da a? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Example 1. B ownian mo ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Example 2. Canadian wea he da a . . . . . . . . . . . . . . . . . . . . . . . . . 2
Example 3. Spec ome ic da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Example 4. A mosphe ic pollu ion da a . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.3 O he unc ional da ase s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
a) Clima ology and en i onme ics . . . . . . . . . . . . . . . . . . . . . . . . . . 5
b) Chemome ics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
c) Enginee ing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
d) Econome ics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
e) Biome ics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
) Fu he applica ions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Func ional space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2.1 The Hilbe space H.................................. 6
1.2.2 Associa ed spaces and enso p oduc s . . . . . . . . . . . . . . . . . . . . . . . . 9
a) The space o Hilbe –Schmid ope a o s . . . . . . . . . . . . . . . . . . . . . 9
b) The dual space H′................................. 10
c) Tenso no a ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.2.3 Measu ing dis ances: semi–me ics . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Example 1. Semi–me ics based on FPCA . . . . . . . . . . . . . . . . . . . . . . 11
Example 2. Semi–me ics based on MPLSR . . . . . . . . . . . . . . . . . . . . . 11
Example 3. Semi–me ics based on de i a i es . . . . . . . . . . . . . . . . . . . 12
1.3 P ep ocessing unc ional da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.3.1 Smoo hing unc ional da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
a) Linea smoo hing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
b) Smoo hing by basis unc ions . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
c) Smoo hing wi h a oughness penal y . . . . . . . . . . . . . . . . . . . . . . . 19
1.3.2 Regis e ing unc ional da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
a) Ampli ude a ia ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
b) Phase a ia ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.4 Explo ing unc ional da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
1.4.1 Desc ip i e s a is ics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
a) Measu es o posi ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
b) Measu es o dispe sion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.4.2 Func ional da a classi ica ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
1.4.3 Spec al analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
a) Func ional p incipal componen analysis (FPCA) . . . . . . . . . . . . . . . . 25
i CONTENTS
b) Func ional canonical analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
c) Func ional linea disc iminan analysis . . . . . . . . . . . . . . . . . . . . . . 28
2 Func ional eg ession models 29
2.1 Wha does unc ional eg ession mean? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.2 Func ional eg ession o scala esponse . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.3 Func ional linea eg ession o scala esponse . . . . . . . . . . . . . . . . . . . . . . . 32
2.3.1 Es ima o s based on basis expansions . . . . . . . . . . . . . . . . . . . . . . . . 33
a) Leas squa es es ima o . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
b) Penalized leas squa es es ima o . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3.2 Es ima o s based on FPCA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
a) De ini ion o s anda d FPCA es ima o . . . . . . . . . . . . . . . . . . . . . . 35
b) De ini ion o gene al class o FPCA– ype es ima o s . . . . . . . . . . . . . . . 37
c) Consis ency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
d) Condi ional e o s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
e) Asymp o ic no mali y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.4 Func ional nonpa ame ic eg ession o scala esponse . . . . . . . . . . . . . . . . . . 41
2.4.1 Ke nel– ype es ima o s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
a) De ini ion o ke nel es ima o . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
b) Consis ency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
c) Bias and a iance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
d) Asymp o ic no mali y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.5 Appendix Chap e 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.5.1 Fo mula ion and p oo o Lemma 2.5.1 . . . . . . . . . . . . . . . . . . . . . . . . 47
2.5.2 P oo o Theo em 2.3.14 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
2.5.3 Fo mula ion and p oo o Lemma 2.5.2 . . . . . . . . . . . . . . . . . . . . . . . . 48
3 P esmoo hing in unc ional linea eg ession 51
3.1 Why in oduce p esmoo hing echniques? . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.2 P esmoo hing ia co a iance s uc u e . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2.1 De ini ion o es ima o . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2.2 Consis ency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.2.3 Condi ional e o s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
3.3 P esmoo hing ia esponse a iable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3.1 De ini ion o es ima o . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3.2 Consis ency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.3.3 Condi ional e o s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.4 Heu is ics on al e na i e p esmoo hing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
3.4.1 Using Pezzulli and Sil e man’ p esmoo hed FPCA . . . . . . . . . . . . . . . . . 60
De ini ion o es ima o . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
Condi ional e o s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.4.2 Using Sil e man’s p esmoo hed FPCA . . . . . . . . . . . . . . . . . . . . . . . . 62
De ini ion o es ima o . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Condi ional e o s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
3.5 Simula ion s udy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.5.1 Case A. Exis ence o null eigen alues . . . . . . . . . . . . . . . . . . . . . . . . . 65
3.5.2 Case B. Non exis ence o null eigen alues . . . . . . . . . . . . . . . . . . . . . . 65
3.6 Real da a applica ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.6.1 Canadian wea he da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.6.2 Spec ome ic da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
3.6.3 A mosphe ic pollu ion da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.7 Final conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.8 Appendix Chap e 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
CONTENTS ii
3.8.1 P oo o Theo em 3.2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.8.2 Fo mula ion and p oo o Lemma 3.8.1 . . . . . . . . . . . . . . . . . . . . . . . . 74
3.8.3 P oo o Theo em 3.2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.8.4 P oo o Co olla y 3.2.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.8.5 P oo o Theo em 3.3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
3.8.6 Fo mula ion and p oo o Lemma 3.8.2 . . . . . . . . . . . . . . . . . . . . . . . . 77
3.8.7 Fo mula ion and p oo o Lemma 3.8.3 . . . . . . . . . . . . . . . . . . . . . . . . 77
3.8.8 P oo o Theo em 3.3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.8.9 Fo mula ion and p oo o Lemma 3.8.4 . . . . . . . . . . . . . . . . . . . . . . . . 79
3.8.10 P oo o Co olla y 3.3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.8.11 P oo o Co olla y 3.3.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
3.8.12 P oo o Theo em 3.4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.8.13 P oo o Co olla y 3.4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
3.8.14 P oo o Theo em 3.4.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4 Boo s ap in unc ional linea eg ession 87
4.1 How o build con idence in e als? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.2 Asymp o ic con idence in e als o linea eg ession . . . . . . . . . . . . . . . . . . . . 88
4.3 Boo s ap con idence in e als o linea eg ession . . . . . . . . . . . . . . . . . . . . . 89
4.3.1 Nai e and wild boo s ap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
4.3.2 Asymp o ic alidi y o he boo s ap . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.4 Simula ion s udy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
4.5 Final conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.6 Appendix Chap e 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.6.1 P oo o Theo em 4.3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.6.2 Fo mula ion and p oo o Lemma 4.6.1 . . . . . . . . . . . . . . . . . . . . . . . . 103
4.6.3 Fo mula ion and p oo o Lemma 4.6.2 . . . . . . . . . . . . . . . . . . . . . . . . 103
4.6.4 Fo mula ion and p oo o Lemma 4.6.3 . . . . . . . . . . . . . . . . . . . . . . . . 104
5 Tes ing in unc ional linea eg ession 105
5.1 Tes ing in FDA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5.2 Tes o lack o dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
5.2.1 Asymp o ic heo y o es ing and boo s ap p ocedu es . . . . . . . . . . . . . . 107
Theo e ical backg ound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Linea independence es . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Tes ing p ocedu e and asymp o ic heo y . . . . . . . . . . . . . . . . . . . . . . 109
Boo s ap p ocedu es and consis ency . . . . . . . . . . . . . . . . . . . . . . . . 111
5.2.2 Boo s ap calib a ion s. asymp o ic heo y . . . . . . . . . . . . . . . . . . . . . 113
5.3 Tes o equali y o linea models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
5.3.1 Asymp o ic heo y o es ing and boo s ap p ocedu es . . . . . . . . . . . . . . 116
Theo e ical backg ound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Equali y o wo linea models es . . . . . . . . . . . . . . . . . . . . . . . . . . 116
Tes ing p ocedu e and asymp o ic heo y . . . . . . . . . . . . . . . . . . . . . . 117
Boo s ap p ocedu es and consis ency . . . . . . . . . . . . . . . . . . . . . . . . 119
5.3.2 Boo s ap calib a ion s. asymp o ic heo y . . . . . . . . . . . . . . . . . . . . . 120
5.4 Simula ion s udy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.4.1 Tes ing he lack o dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
5.4.2 Tes ing he equali y o linea models . . . . . . . . . . . . . . . . . . . . . . . . . 125
5.5 Real da a applica ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.5.1 Tes ing he lack o dependence . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
5.5.2 Tes ing he equali y o linea models . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.6 Final conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.7 Appendix Chap e 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
iii CONTENTS
5.7.1 P oo o Theo em 5.2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.7.2 P oo o Co olla y 5.2.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.7.3 P oo o Theo em 5.2.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.7.4 P oo o Theo em 5.2.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5.7.5 Fo mula ion and p oo o Lemma 5.7.1 . . . . . . . . . . . . . . . . . . . . . . . . 135
5.7.6 P oo o Theo em 5.2.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
5.7.7 Fo mula ion and p oo o Lemma 5.7.2 . . . . . . . . . . . . . . . . . . . . . . . . 136
5.7.8 P oo o Theo em 5.3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
5.7.9 P oo o Co olla y 5.3.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
5.7.10 P oo o Theo em 5.3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
5.7.11 P oo o Theo em 5.3.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
5.7.12 P oo o Theo em 5.3.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6 Th esholding in nonpa ame ic unc ional eg ession 141
6.1 Why conside a h eshold app oach? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.2 Th eshold me hodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6.2.1 Reg ession model and es ima e . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
Examples o h eshold unc ions . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
A pa icula scena io . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Mean squa e con e gence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
6.2.2 C oss– alida ion c i e ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.3 Simula ion s udy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
6.3.1 Case A. Th eshold on Y(same cu es in bo h subsamples) . . . . . . . . . . . . 150
6.3.2 Case B. Th eshold on Yand X(equally concen a ed cu es) . . . . . . . . . . . 150
6.3.3 Case C. Th eshold on Yand X(di e en ly concen a ed cu es) . . . . . . . . . 152
6.4 Real da a applica ion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
6.4.1 Canadian wea he da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.4.2 Spec ome ic da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
6.4.3 A mosphe ic pollu ion da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
6.5 Final conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
6.6 Appendix Chap e 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
6.6.1 P oo o Theo em 6.2.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.6.2 P oo o Co olla y 6.2.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.6.3 P oo o Theo em 6.2.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
6.6.4 Fo mula ion and p oo o Lemma 6.6.1 . . . . . . . . . . . . . . . . . . . . . . . . 164
6.6.5 Fo mula ion and p oo o Lemma 6.6.2 . . . . . . . . . . . . . . . . . . . . . . . . 165
6.6.6 Fo mula ion and p oo o Lemma 6.6.3 . . . . . . . . . . . . . . . . . . . . . . . . 167
6.6.7 Fo mula ion and p oo o Lemma 6.6.4 . . . . . . . . . . . . . . . . . . . . . . . . 168
6.6.8 Fo mula ion and p oo o Lemma 6.6.5 . . . . . . . . . . . . . . . . . . . . . . . . 169
6.6.9 P oo o Theo em 6.2.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
6.6.10 P oo o Theo em 6.2.9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.6.11 Auxilia y echnical lemmas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
Fo mula ion and p oo o Lemma 6.6.6 . . . . . . . . . . . . . . . . . . . . . . . . 173
Fo mula ion and p oo o Lemma 6.6.7 . . . . . . . . . . . . . . . . . . . . . . . . 174
Fo mula ion and p oo o Lemma 6.6.8 . . . . . . . . . . . . . . . . . . . . . . . . 176
Fo mula ion and p oo o Lemma 6.6.9 . . . . . . . . . . . . . . . . . . . . . . . . 176
Conclusions and u he esea ch 181
Summa y 185
Resumo en galego 191
Bibliog aphy 198
CONTENTS ix
Lis o Figu es 223
Lis o Tables 225
2CHAPTER 1. INTRODUCTION TO FDA
De ini ion 1.1.1. A andom a iable Xis called a unc ional andom a iable i i akes alues in an
in ini e dimensional space o unc ional space S. An obse a ion xo Xis called a unc ional da um.
Gi en a p obabili y space (Ω,A,P), a unc ional andom a iable Xis indeed a measu able mapping
om Ω o an in ini e dimensional space o unc ional space S. In his case, i will be said ha Xis a
unc ional andom a iable alued in S, o Xis a S– alued unc ional andom a iable.
De ini ion 1.1.2. Gi en a unc ional andom a iable Xand n∈N∗, a unc ional andom sample o
Xo lengh nis a se {Xi}n
i=1 o independen and iden ically dis ibu ed (i.i.d.) unc ional andom
a iables wi h he same dis ibu ion as X. An obse a ion {xi}n
i=1 o {Xi}n
i=1 is called a unc ional
da ase .
The gene al de ini ions gi en be o e include bo h he mos common a ailable unc ional da ase s as
cu es o su aces, and mo e gene al unc ional a iables alued in me ic o semi–me ic spaces.
Some imes, one aces high–dimensional obse a ions which canno be co ec ly analysed by means
o he classical mul i a ia e ools, due o he high co ela ion o hei componen s, bu which do no
seem o be unc ional by na u e a i s sigh . To sol e his issue, Chen e al. (2011) p oposed s inging
me hods in o de o map his high–dimensional da a in o an in ini e dimensional unc ional space in
such a way ha hey can be ea ed using FDA me hodology.
The e a e impo an links be ween FDA and Longi udinal Da a Analysis. Al hough some FDA
echniques can be used in longi udinal con ex , his canno be done in gene al since longi udinal da a
a e no able o he spa seness o he disc e iza ion g id. Some ecen con ibu ions discussed and
compa ed longi udinal and unc ional app oaches (see, o ins ance, James (2002); James and Suga
(2003); Da idian e al. (2004); Zhao e al. (2004); M¨ulle (2005); Hall e al. (2006); Yao (2007); M¨ulle
and Yang (2010); James (2011)).
1.1.2 Examples
In his subsec ion, some examples o unc ional da a a e in oduced, which will be analysed by means o
di e en s a is ical me hods h oughou he manusc ip . The i s example co esponds o a simula ed
da ase and he nex h ee o eal da a. Fu he mo e, a b ie s a e o he a on applied con ibu ions
in ol ing eal unc ional da a can be ound in he nex subsec ion.
Example 1. B ownian mo ion
A simula ed da ase is he i s example o unc ional da a included in his sec ion: a B ownian mo ion.
AB ownian mo ion de ined on T= [0,1] is a andom Gaussian elemen Bsuch ha
E(B( )) = 0 and Co (B(s), B( )) = min(s, ),∀s, ∈T.
A sample o 50 obse a ions o a B ownian mo ion is depic ed in Figu e 1.1 (see page 3). The cu es
a e disc e ized on a g id o 100 equidis an poin s.
Example 2. Canadian wea he da a
The second example comes om he Canadian wea he da a. This da ase con ains a e aged daily
empe a u e and p ecipi a ion a 35 di e en loca ions in Canada om 1960 o 1994. The Canadian
wea he da a illus a ed many unc ional me hods in he book by Ramsay and Sil e man (2005) (see
also he webpage h p://www.psych.mcgill.ca/misc/ da), and i is a ailable in he Rpackage da
(see Ramsay e al., 2011). Figu e 1.2 (see page 3) shows he 35 daily empe a u e cu es a each
wea he s a ion, being
Xi( ) : daily a e aged empe a u e in he wea he s a ion iin he day , wi h ∈ {1,...,365}.
1.1. FUNCTIONAL DATA 3
0.0 0.2 0.4 0.6 0.8 1.0
−2 −1 0 1 2
Figu e 1.1: B ownian mo ion. Sample o 50
simula ed obse a ions.
Day
Tempe a u e (ºC)
Jan Feb Ma Ap May Jun Jul Aug Sep Oc No Dec
−30 −20 −10 0 10 20
Figu e 1.2: Canadian wea he da a. Sample
o 35 daily empe a u e cu es.
Example 3. Spec ome ic da a
The hi d example co esponds o spec ome ic cu es: he po k da a. The spec ome ic da ase is a
pa o a sample which can be downloaded om h p://lib.s a .cmu.edu/da ase s/ eca o , and
i was also analysed om a unc ional poin o iew in he li e a u e (Fe a y and Vieu, 2006b; see also
he companion websi e h p://www.ma h.uni - oulouse. /s aph/np da). The da ase conce ns
a sample o 215 pieces o inely chopped mea . Fo each uni , a spec ome ic cu e is obse ed which
co esponds o he abso bance1a 100 wa eleng hs. The spec ome ic da ase was eco ded on a
Teca o In a ec Food and Feed Analyze . The le panel o Figu e 1.3 (see page 4) collec s hese 215
cu es, gi en by
Xi( ) : abso bance o he piece io mea a nm, wi h ∈ {850,...,1050}
(no e ha nm is he symbol o nanome e). In he chemome ic communi y, i is well–known ha
de i a i es o spec a a e mo e in o ma i e han he o iginal ones. This is he eason why igh panel
o Figu e 1.3 collec s he second de i a i es o he spec ome ic cu es which will play an impo an
ole la e on.
Example 4. A mosphe ic pollu ion da a
Finally, an en i onmen al example ha e been selec ed: ai pollu ion da a. The da a is a ime se ies
Zco esponding o he concen a ion o hou ly a e aged NOxmeasu ed in he neighbou hood o a
powe s a ion belongs o ENDESA, loca ed in As Pon es in he No hwes o Spain. The NOxle el
was measu ed each minu e om 2007 o 2009. The ime se ies Zhas been di ided in a ious pa hs
co esponding o 4 hou pe iods ( ha is, he cu es a e disc e ized a 240 poin s). Thus,
Xi( ) = Z( + (i−1) ×240) : hou ly a e aged NOxin he minu e (pe iod i), wi h ∈ {1,...,240}.
Figu e 1.4 (see page 4) p esen s 100 o hese cu es. No e ha , hough he NOx alues a e low in
gene al, du ing un a ou able me eo ological condi ions he NOxle els may quickly ise and cause an
1The abso bance is de ined as Aw=−log10 (I(w)/I0(w)), whe e I(w) is he in ensi y o ligh a he wa eleng h w
ha has passed h ough he sample, whe eas I0(w) is he in ensi y o he ligh be o e i en e s he sample.
4CHAPTER 1. INTRODUCTION TO FDA
Wa eleng h
Abso bance
850 900 950 1000 1050
2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5
Wa eleng h
Abso bance
850 900 950 1000 1050
−150 −100 −50 0 50 100 150
Figu e 1.3: Spec ome ic da a. Sample o 215 spec ome ic cu es (le panel) and hei second
de i a i es ( igh panel).
ai pollu ion episode2. This ac explains why he sample consis s o se e al almos cons an cu es
and e y ew inc easing/dec easing cu es which co espond o ai pollu ion episodes.
0 50 100 150 200
10 20 30 40 50
Time
NOx le el
Figu e 1.4: A mosphe ic pollu ion da a. Sample o 100 hou ly a e aged NOxle els.
1.1.3 O he unc ional da ase s
Many con ibu ions de o ed o FDA con ain applied issues, o example, Ramsay and Sil e man (2002)
o Fe a y and Vieu (2006a). These s udies conce n unc ional da a coming om a a ie y o scien i ic
ields. Nex , a selec ed collec ion o hese applied con ibu ions a e summa ized.
2An ai pollu ion episode is a pe iod o abno mally high concen a ion o ai pollu an s, o en due o low winds and
empe a u e in e sion, ha can cause illness and dea h.
1.1. FUNCTIONAL DATA 5
a) Clima ology and en i onme ics
Recen en i onmen al and clima ological p oblems ha e equi ed he use o FDA echniques (see, o
ins ance, Escabias e al., 2005; Ramsay and Sil e man, 2005). Many o he applied con ibu ions
o clima ology a e ela ed wi h El Ni˜no phenomenon (Besse e al., 2000; Valde ama e al., 2002;
An oniadis and Sapa inas, 2003; Fe a y e al., 2005), o o he clima ic da ase s (Vidako ic, 2001;
Ramsay and Sil e man, 2002; Hall and Vial, 2006b).
As a as en i onme ics applica ions a e conce ned, ai pollu ion s udies (Damon and Guillas,
2002; Fe n´andez de Cas o e al., 2005; Anei os-P´e ez e al., 2004; Ca do and Sa da, 2006; Ca do
e al., 2007b; Feb e o-Bande e al., 2007; Mei ing, 2007; Fe n´andez de Cas o and Gonz´alez-Man eiga,
2008; Fe a y and Vieu, 2009), pape s on a mosphe ic adioac i i y (Ca do e al., 2007d; Hlubinka and
P chal, 2007), and wa e quali y con ol con ibu ions (Hende son, 2006; Ne ini and Gha as, 2007)
can be ound in he li e a u e.
b) Chemome ics
Spec ome y has ocused he e o s o he s a is ical FDA communi y due o he in insic unc ional
na u e o he spec ome ic cu es. F om he ea ly pape s (Leu gans e al., 1993) o he mos ecen
wo ks (Lea di, 2003; Ama o e al., 2006; Fe a y e al., 2010a), spec ome ic da a ha e been he objec
o s udy o many au ho s, specially he po k da ase p esen ed be o e (see, amongs o he s, Bo ggaa d
and Thodbe g, 1992; Fe a y and Vieu, 2002; Fe ´e and Yao, 2005; Ama o e al., 2006; Fe a y and
Vieu, 2006b; Fe a y e al., 2006; Anei os-P´e ez and Vieu, 2006; Fe a y e al., 2007a; Mas and Pumo,
2007; Fe a y e al., 2007b; Fe a y and Vieu, 2009; Bu ba e al., 2009).
c) Enginee ing
Func ional da ase comes easily om enginee ing amewo ks as he sa elli e image y o signal ecog-
ni ion. In he li e a u e, FDA has been applied o sa elli e da a (Vidako ic, 2001; Ca do e al., 2003a;
Ca do and Sa da, 2006), ada cu es (Dabo-Niang e al., 2007) o sound signal ecogni ion da ase s
(Luce o, 1999; Hall e al., 2001; Has ie e al., 2001; Fe a y and Vieu, 2003, 2006b).
d) Econome ics
Func ional da a appea s qui e o en in he economics, whe e con inuous ime se ies can be spli and
analysed as cu es. Examples can be ound in Kneip and U ikal (2001); Ramsay and Sil e man (2002);
Fe a y e al. (2002); Laukai is and Rackauskas (2002); Kawasaki and Ando (2004); Reddy and Dass
(2006); Jank and Shmueli (2006); Wang e al. (2008); Laukai is (2008); Liu and M¨ulle (2009); Benko
e al. (2009); M¨ulle e al. (2011).
e) Biome ics
One o he mos impo an scien i ic ields in ol ing FDA is medicine. Many medical da ase s we e
analysed om a unc ional iewpoin : g ow h cu es (Ramsay e al., 1995; Gasse e al., 1998; James
e al., 2000; James and Suga , 2003; Ramsay and Sil e man, 2005; Liu and Yang, 2009), human mo ion
and pe cep ion (Ramsay e al., 1996; Ramsay and Sil e man, 2002; Spi zne e al., 2003; O monei
e al., 2005; L´opez-Pin ado and Romo, 2007; An oniadis and Sapa inas, 2007), placebo e ec s (Ta pey
e al., 2003), mo ali y (Hyndman and Ullah, 2007; Chiou and M¨ulle , 2009), gene ics (Pa ke and
Wen, 2009), cance ology (Ramsay and Sil e man, 2005; Cao and Ramsay, 2007; E bas e al., 2007),
ca diology (Clo , 2002; Ra cli e e al., 2002a,b; Cue as e al., 2004; Ha ezlak e al., 2007), neu ology
(Epi anio and Ven u a-Campos, 2011; Goldsmi h e al., 2011, 2012) o oph halmology (Locan o e e al.,
1999).
Wi h ega d o bios a is ics, FDA me hods ha e success ully been applied o many con ex s as
he handbook by H¨a dle e al. (2007) cap u ed. Among he applica ions one can emphasize wo ks
connec ed wi h animal biological p oblems (Chiou e al., 2003a,b; M¨ulle and S ad m¨ulle , 2005; Chiou
6CHAPTER 1. INTRODUCTION TO FDA
and M¨ulle , 2007), and con ibu ions on gene ics (A aki e al., 2004; Leng and M¨ulle , 2005; Opgen-
Rhein and S imme , 2006).
) Fu he applica ions
The e a e many o he sciences whe e FDA me hods ind easible applica ions: geology (Man ´e e al.,
2007), oceanology (Ne ini and Gha as, 2007), demog aphy (Hyndman and Ullah, 2007), g aphology
(Has ie e al., 1995; Ramsay, 2000a,b), e c.
Ano he sou ce o unc ional da ase s a ises om he s anda d mul i a ia e me hods. The e a e
ce ain in e es ing cu es which a e es ima ed when one wo ks wi h a mul i a ia e sample, o example
he densi y unc ion. These kinds o cu es can be ea ed as unc ional da a, and analysed by means
o FDA me hodology. Some con ibu ions can be ound in he li e a u e o densi y unc ions (Kneip
and U ikal, 2001; Ramsay and Sil e man, 2002; Ne ini and Gha as, 2007; Delicado, 2007), dis ibu ion
unc ions (Man ´e e al., 2007), eg ession unc ions (H¨a dle and Ma on, 1990; Heckman and Zama ,
2000), o o he p obabilis ic unc ional cha ac e is ics (Rossi e al., 2002).
1.2 Func ional space
Le {Xi}n
i=1 be a unc ional andom sample, ha is, ni.i.d. unc ional andom a iables wi h he same
dis ibu ion as a unc ional a iable X alued in an abs ac space S. In many common si ua ions,
unc ional da a a e cu es, so S=L2([0,1]) is qui e o en used (C ambes e al., 2009). Howe e , mo e
gene al spaces, as Hilbe spaces (S= (H,h·,·i)) o Banach spaces (S= (B,k·k)), a e conside ed some-
imes in o de o sol e ce ain echnical p oblems (Bosq, 2000). In o he cases, a b oade amewo k
is conside ed: a unc ional space Sendowed wi h a semi–me ic d(·,·) (Fe a y and Vieu, 2006b).
Th oughou his manusc ip , i has been assumed ha he space whe e unc ional a iables ake
alues is a eal sepa able Hilbe space H, al hough some imes he unc ional space has been es ic ed
o he well–known L2–space o illus a e ce ain me hods. On he o he hand, he need o measu es
o de e mine he closeness o unc ional obse a ions has mo i a ed he use o semi–me ics in FDA.
These a e he easons why his sec ion has been de o ed o he in oduc ion o some concep s connec ed
wi h bo h Hilbe spaces and semi–me ics.
1.2.1 The Hilbe space H
Fi s o all, some basic concep s, which a e necessa y o in oduce he de ini ion o Hilbe space, a e
included in o de o cla i y he kind o ea u es and p ope ies o he unc ional space.
De ini ion 1.2.1. A ield Fis a se wi h wo bina y ope a ions3, usually called addi ion (+F:
F×F→F) and mul iplica ion (·F:F×F→F), ha sa is ies he ollowing axioms o all a, b, c ∈F:
(i) Associa i i y o +Fand ·F: (a+Fb) +Fc=a+F(b+Fc), and (a·Fb)·Fc=a·F(b·Fc).
(ii) Commu a i i y o +Fand ·F:a+Fb=b+Fa, and a·Fb=b·Fa.
(iii) Iden i y elemen o +Fand ·F:∃0F∈Fsuch ha a+F0F=a= 0F+Fa,∀a∈F, and ∃1F∈F
such ha a·F1F=a= 1F·Fa,∀a∈F.
(i ) In e se elemen o +Fand ·F:∀a∈F,∃ − a∈Fsuch ha a+F(−a) = 0 = (−a) +Fa. In
addi ion, i a6= 0, hen ∃a−1∈Fsuch ha a·Fa−1= 1 = a−1·Fa.
( ) Dis ibu i i y o ·Fwi h espec o +Fand dis ibu i i y o +Fwi h espec o ·F:a·F(b+Fc) =
a·Fb+Fa·Fc, and (a+Fb)·Fc=a·Fc+Fb·Fc.
The elemen s o a ield Fa e called scala s.
3Abina y ope a ion on a non–emp y se Ais a map :A×A→Asuch ha is de ined o e e y pai o elemen s
1.2. FUNCTIONAL SPACE 7
Fo ins ance, he a ional numbe s (Q), he eal numbe s (R) o he complex numbe s (C) a e ields.
Rema k 1.2.2.To simpli y he no a ion, and whene e he e is no possible con usion, a+Fb,a·Fb,
0Fand 1Fwill be deno ed by a+b,ab, 0 and 1, espec i ely.
De ini ion 1.2.3. A ec o space Vo e a ield Fis a se wi h wo ope a ions, usually called addi ion
(+V:V×V→V) and scala mul iplica ion (·V:F×V→V), ha sa is ies he ollowing axioms o
all a, b ∈Fand x, y, z ∈V:
(i) Associa i i y o +V: (x+Vy) +Vz=x+V(y+Vz).
(ii) Commu a i i y o +V:x+Vy=y+Vx.
(iii) Iden i y elemen o +Vand ·V:∃0V∈Vsuch ha y+V0V=y= 0V+Vy,∀y∈V; i 1F∈F
deno es he mul iplica i e iden i y elemen in F, hen 1F·Vy=y,∀y∈V.
(i ) In e se elemen o +V:∀y∈V,∃−y∈Vsuch ha y+V(−y) = 0 = (−y) +Vy.
( ) Dis ibu i i y o ·Vwi h espec o +V, and ·Vwi h espec o +F:a·V(x+Vy) = a·Vx+Va·Vy,
and (a+Fb)·Vy=a·Vy+Vb·Vy.
( i) a·V(b·Vy) = (a·Fb)·Vy.
The elemen s o a ec o space Va e called ec o s.
The mos commonly used ec o spaces a e hose o e R( eal ec o spaces) o C(complex ec o
spaces). Examples o ec o spaces include he n–dimensional Euclidean spaces Rnand many unc ional
spaces, such as spaces o con inuous unc ions, spaces o measu able unc ions o spaces o summable
unc ions.
Rema k 1.2.4.Again, he no a ion will be simpli ied deno ing x+Vy,a·Vyand 0Vby x+ ,ay and
0, espec i ely, when he e is no possible con usion.
De ini ion 1.2.5. Le Vbe a ec o space and le Fbe ei he he ield o eal numbe s Ro he
ield o complex numbe s C. An inne p oduc is a map h·,·i :V×V→F ha e i ies he ollowing
p ope ies o all x, y, z ∈Vand o all a∈F:
(i) Conjuga e symme y4:hx, yi=hy, xi.
(ii) Linea i y in he i s a gumen : hx+y, zi=hx, zi+hy, zi, and hax, yi=ahx, yi.
(iii) Posi i e de ini eness: hx, xi ≥ 0 wi h equali y only o x= 0.
A ec o space Vo e he ield F oge he wi h an inne p oduc is called an inne p oduc space.
Inne p oduc spaces a e he Euclidean space Rnwi h he inne p oduc hx, yi=Pn
i=1 xiyi, o he space
o con inuous C– alued unc ions on he in e al [a, b] wi h he inne p oduc h , gi=Rb
a ( )g( )d .
Rema k 1.2.6.Fo F=R, he axioms o conjuga e symme y and linea i y in he de ini ion o inne
p oduc a e educed o symme y and bilinea i y. The e o e, he inne p oduc is a posi i e de ini e
symme ic bilinea o m. On he o he hand, i F=C, he inne p oduc is a posi i e de ini e
He mi ian o m.
in A, and uniquely essocia es each pai o elemen s in A o some elemen o A.
4The conjuga e o he complex numbe z=a+ib, whe e aand ba e eal numbe s, is z=a−ib.
8CHAPTER 1. INTRODUCTION TO FDA
De ini ion 1.2.7. Le Vbe a ec o space and le Fbe ei he he ield o eal numbe s Ro he ield
o complex numbe s C. A no m is a map k· k :V→F ha e i ies he ollowing p ope ies o all
x, y ∈Vand o all a∈F:
(i) T iangle inequali y: kx+yk ≤ kxk+kyk.
(ii) Posi i e homogenei y: kaxk=|a|kxk.
(iii) Posi i e de ini eness: kxk ≥ 0, and kxk= 0 o x= 0 only.
I k·k e i ies he p e ious p ope ies excep non–degene acy, ha is, kxk= 0 does no p eclude ha
x6= 0, hen k·k is called semi–no m.
Fu he mo e, a ec o space Vo e he ield F oge he wi h a no m ( espec i ely, semi–no m) is
called a no med space ( espec i ely, semi–no m space).
Gi en an inne p oduc space V, a no m can be gene a ed by i s inne p oduc h·,·i as ollows
kxk=hx, xi1/2,∀x∈V.
Thus, Vis a no med space wi h his induced no m. Consequen ly, he examples o inne p oduc spaces
commen ed be o e a e also examples o no m spaces when he no m induced by he inne p oduc is
conside ed.
De ini ion 1.2.8. Ame ic on a se Zis a map d(·,·) : Z×Z→R ha e i ies he ollowing p ope ies
o all x, y, z ∈Z:
(i) T iangle inequali y: d(x, y)≤d(x, z) + d(z, y).
(ii) Symme y: d(x, y) = d(y, x).
(iii) Posi i e de ini eness: d(x, y)≥0, and d(x, y) = 0 o x=yonly.
I d(·,·) e i ies he p e ious p ope ies excep non–degene acy, ha is, d(x, y) = 0 does no p eclude
ha x6=y, hen d(·,·) is called semi–me ic.
Fu he mo e, a se Zp o ided wi h a me ic ( espec i ely, semi–me ic) is called a me ic space
( espec i ely, semi–me ic space).
Rema k 1.2.9.The concep o semi–me ic included in he p e ious de ini ion is o en e e ed o as
pseudome ic in he ma hema ical li e a u e. Ne e heless, he e m semi–me ic has been chosen in
his hesis because o i s ela ion o semi–no m no ion and because o ha i was used in he FDA
monog aph by Fe a y and Vieu (2006b).
De ini ion 1.2.10. Acomple e me ic space is a me ic space in which e e y Cauchy sequence5is
con e gen .
De ini ion 1.2.11. AHilbe space His a ec o space o e Ro Cwi h an inne p oduc h·,·i such
ha he induced no m de ined by kxk=hx, xi1/2,∀x∈ H, u ns Hin o a comple e me ic space.
Fo ins ance, he complex space ℓ2, which consis s o all in ini e sequences o complex numbe s {zn}∞
n=1
5Gi en a me ic space (Z, d), a sequence {zi}i∈N∗is called Cauchy sequence i o any ε > 0 he e is a posi i e in ege
N0such ha d(zm, zn)< ε, o all m, n > N0.
1.2. FUNCTIONAL SPACE 9
such ha P∞
n=1 |zn|2con e ges, wi h he inne p oduc h{wn}∞
n=1,{zn}∞
n=1i=P∞
n=1 wnznis a Hilbe
space. Ano he example o Hilbe space is he space L2([0,1]) o squa e–in eg able unc ions wi h
espec o he Lebesgue measu e on he uni in e al. Then, he inne p oduc o , g ∈L2([0,1]) is
de ined by h , gi=R1
0 ( )g( )d in he complex case, and h , gi=R1
0 ( )g( )d in he eal case.
De ini ion 1.2.12. Asepa able space is a opological6space such ha i has a coun able7dense8
subse .
F om now on, he unc ional space whe e Xis alued will be a eal sepa able Hilbe space deno ed
by (H,h·,·i), ha is, His a ec o space wi h an inne p oduc h·,·i :H×H → Rsuch ha is also a
comple e me ic space wi h espec o he induced no m.
Rema k 1.2.13.A Hilbe space is sepa able i and only i i has a coun able o hono mal basis.
Hence, he e exis s a coun able sequence {ej}∞
j=1 o mu ually o hono mal elemen s o H, ha is,
hej1, ej2i=δj1,j2wi h δj1,j2= 0 i j16=j2and δj1,j2= 1 i j1=j2, such ha span he space. Thus,
x=∞
X
j=1 hx, ejiej,∀x∈ H,
and he induced no m can be gi en by kxk2=P∞
j=1 |hx, eji|2(Pa se al’s iden i y).
1.2.2 Associa ed spaces and enso p oduc s
Gi en a eal sepa able Hilbe space (H,h·,·i), ecall ha he induced no m has been deno ed by
k · k =h·,·i1/2. Now, o he associa ed spaces and no ms o in e es ( he space o Hilbe –Schmid
ope a o s and he dual space) and ce ain enso no a ion a e going o be p esen ed.
a) The space o Hilbe –Schmid ope a o s
Le HS be he space o Hilbe –Schmid ope a o s de ined on Hgi en by
HS ={U:H → H such ha Uis a bounded ope a o , and
∞
X
j=1 kUejk2<∞, o all {ej}∞
j=1 o hono mal basis o H}.
Fo all U∈ HS, one can conside he usual Hilbe –Schmid no m gi en by
kUkHS =
∞
X
j=1 kUejk2
1/2
,
o he uni o m no m
kUk∞= sup
kxk=1 kUxk.
Rema k 1.2.14.These wo no ms sa is y he nex inequali y: kUk∞≤ kUkHS o all U∈ HS.
6A opological space is a se Z oge he wi h a collec ion Co subse s o Z(called open se s) such ha : he emp y
se ∅is in C,Zis in C, he in e sec ion o a ini e numbe o se s in Cis in C, and he union o an a bi a y numbe o
se s in Cis also in C.
7Acoun able se is a se o he same ca dinali y as he se o na u al numbe s N.
8A subse Wo a opological space Zis dense i i s closu e sa is ies W=Z.
10 CHAPTER 1. INTRODUCTION TO FDA
b) The dual space H′
The dual space o His he space o all con inuous linea unc ions om he space Hin o R
H′={T:H → Rsuch ha Tis con inuous and linea }.
The dual space is p o ided wi h a na u al no m de ined by
kTkH′= sup
kxk=1 |Tx|,
which e i ies ha o all {ej}∞
j=1 o hono mal basis o H
kTkH′=
∞
X
j=1
(Tej)2
1/2
.
Rema k 1.2.15.Fo all T∈ H′and U∈ HS, he ollowing inequali y holds kTUkH′≤ kTkH′kUk∞.
c) Tenso no a ion
To inish his sec ion, some enso p oduc no a ion ha will be used la e is in oduced. Gi en
x, y, z ∈ H, he enso p oduc x⊗Hyis de ined as he ollowing ope a o
x⊗Hy:H → H
z→x⊗Hy(z) = hx, ziy.
Besides, o x, z ∈ H and y∈R, he enso p oduc x⊗H′yis de ined as
x⊗H′y:H → R
z→x⊗H′y(z) = hx, ziy.
These kinds o enso p oduc s will be use ul o de ine ce ain unc ional ope a o s in u u e sec ions.
1.2.3 Measu ing dis ances: semi–me ics
Soone o la e , he e a e wo ques ions ha appea when FDA comes o wo k on. The i s di icul y
is o decide how o measu e he dis ance be ween unc ional elemen s o H. The second ques ion is
ela ed o one o he g ea es wo ies in ini e dimensional spaces called he cu se o dimensionali y:
he la ge he wo king space dimension is, he la ge he spa seness o sample da a is. This c ucial
poin o ini e dimensional case is e en mo e c i ical when wo king wi h in ini e dimensional spaces.
The classic measu es o he closeness o wo elemen s in Rpa e no ms, o example, he Euclidean
no m. The choice o which no m o use is no an issue because o he equi alence among no ms in
he ini e dimensional Euclidean spaces. Ne e heless, in he in ini e dimensional spaces he no ms a e
no equi alen so he selec ion o a p elimina y no m u ns in o a p io i y p oblem. Fe a y and Vieu
(2006b) conside ed he no ion o no m o be oo es ic i e in FDA con ex , and hey sugges ed he
use o semi–me ics ( ecall De ini ion 1.2.8, page 8), which seem o be mo e sui able o his ype o
da a (see Geenens (2011) o an analysis o he use ulness o semi–me ics in o de o a oid he cu se
o dimensionali y).
In he Hilbe space case, a semi–me ic d(·,·) can be easily de ined: he me ic induced by
h·,·i is also a semi–me ic. Howe e , i could be conside ed any o he semi–me ic, no necessa -
ily de i ed om he inne p oduc o H. Nex , some examples o semi–me ics a e collec ed. They
we e p oposed by Fe a y and Vieu (2006b) o he pa icula case in which he unc ional da a a e
cu es, being H he classical L2–space and, consequen ly, hx, yi=R1
0x( )y( )d o all x, y ∈L2[0,1].
Among he p oposed semi–me ics, bo h he semi–me ics based on unc ional p incipal componen s
analysis (FPCA) and he semi–me ics based on he mul i a ia e pa ial leas squa es eg ession
1.2. FUNCTIONAL SPACE 11
(MPLSR) a e sui able o ough da ase s, whe eas he semi–me ics based on de i a i es a e well
adap ed o smoo h cu es. One mus choose he semi–me ic ha has a be e beha iou wi h e-
ga d o he da a. All he R ou ines which compu e hese semi–me ics a e a ailable in he websi e
h p://www.ma h.uni - oulouse. /s aph/np da.
In he examples below, {Xi}n
i=1 is a sample o i.i.d. cu es as he L2[0,1]– alued andom unc ional
a iable X, and xis a ixed elemen o L2[0,1].
Example 1. Semi–me ics based on FPCA
P incipal componen s analysis is a e y use ul ool o dimension educ ion in he mul i a ia e con ex .
In he unc ional case, FPCA can be used o educe he dimension and subsequen ly calcula e he
dis ances be ween he p ojec ed da a. Due o he impo ance o FPCA echniques, Sec ion 1.4.3, “ a)
Func ional p incipal componen analysis (FPCA) ”, in his chap e desc ibes FPCA in g ea de ail (see
page 25). He e he equi ed concep s in o de o build he p oposed semi–me ic a e p esen ed b ie ly,
using no a ion which will be igo ously in oduced in he abo e–men ioned sec ion.
Le {(λj, j)}∞
j=1 (wi h λ1≥λ2,...) be he eigen alues and eigen unc ions o he second momen
ope a o Γ ( o mo e de ails, see Sec ion 1.4.1, “ b) Measu es o dispe sion ”, page 22). The o hono mal
basis { j}∞
j=1 allows o w i e Xas X=P∞
j=1 (R1
0X( ) j( )d ) j, and build unca ed expansions as
ollows
X[k]=
k
X
j=1 Z1
0
X( ) j( )d j,
whe e he pa ame e kindica es he “ esolu ion le el” which has been applied. No e ha X[k]min-
imizes E(R1
0(X( )−PkX( ))2d ) o e all he p ojec ions Pko Xin o k–dimensional spaces. Hence,
he ollowing amily o semi–me ics based on he L2–no m is p oposed
dP CA
k(Xi, x) =
u
u
k
X
j=1 Z1
0
(Xi( )−x( )) j( )d 2
.
In p ac ice, he eigen unc ions o Γ a e unknown. The e o e, hey a e eplaced by he eigen unc ions
{ˆ j}∞
j=1 o he empi ical second o de ope a o Γn(see again Sec ion 1.4.1, “ b) Measu es o dispe sion ”,
page 22). Fu he mo e, he cu es a e usually obse ed in a disc e e g id { l}L
l=1 and he in eg al
in ol ed in he semi–me ic should be eplaced by an app oxima ion. Hence, dP CA
k(Xi, x) will be
app oxima ed, i he g id is ine enough, by
˜
dP CA
k(Xi, x) =
u
u
u
k
X
j=1 L
X
l=1
wl(Xi( l)−x( l))ˆ j( l)d !2
,
whe e {wl}L
l=1 a e quad a u e weigh s o he app oxima e in eg a ion.
Rema k 1.2.16.This pa ame e ized amily o semi–me ics does no equi e ha unc ional da a e i y
smoo hness condi ions. Then hese semi–me ics can be applied o qui e ough da a. Howe e , he e
a e wo implici assump ions: he da a mus be balanced9, and he g id o measu emen s mus be
su icien ly ine.
Example 2. Semi–me ics based on MPLSR
Some imes, wo a iables a e simul aneously obse ed: a esponse a iable and an i unc ional co a ia e.
This ac enables he cons uc ion o a amily o semi–me ics by means o mul i a ia e eg ession
echniques. The Mul i a ia e Pa ial Leas Squa es Reg ession (MPLSR) is a s a is ical me hod used
when he eg ession model consis s o a mul i a ia e esponse and a mul i a ia e p edic o . The key
9A unc ional da ase is balanced i all cu es a e measu ed a he same poin s { l}L
l=1.
18 CHAPTER 1. INTRODUCTION TO FDA
Rema k 1.3.2.The weigh ed leas squa es es ima o can be in e p e ed as a linea smoo he ,
whe e he ma ix S(usually called ha ma ix) is
S=Φ(Φ WΦ)−1Φ W.
(see Sec ion 1.3.1, “ a) Linea smoo hing ”, page 13). In his con ex , he ollowing o hogonal
p ope y is sa is ied (Z−ˆ
Z) Wˆ
Z= 0.
Rema k 1.3.3.In he simples case, co esponding o i.i.d. e o s {ǫl}L
l=1 wi h E(ǫl) = 0 and
Va (ǫl) = σ2
ǫ,∀l∈ {1,...,L}, he a iance–co a iance ma ix Σǫis equal o σ2
ǫ1L×L, wi h 1L×L
he L×L–iden i y ma ix (a L×L–ma ix wi h ones in he main diagonal and ze os elsewhe e).
Then, he leas squa es p oblem (1.3) u ns in o minC(Z−ΦC) (Z−ΦC), and he weigh ed leas
squa es es ima o is, in ac , he o dina y leas squa es es ima o . In pa icula , he coe icien s
a e compu ed by means o ˆ
C= (Φ Φ)−1Φ Z, and he i ed alues by ˆ
Z=Φ(Φ Φ)−1Φ Z.
No e ha he o dina y leas squa es c i e ion is inadequa e when he e o s a e co ela ed o
he e ocedas ic. In hese si ua ions, he weigh ed leas squa es i ing should be used.
(ii) Localized leas squa es es ima o . In o de o es ima e xa a ixed ime , local weigh ing
unc ions a e o en equi ed. The weigh ed leas squa es c i e ion de ined in (1.4) can be modi ied
in o de o include local weigh s as ollows
min
c1,...,cJ
L
X
l=1
wl( )
zl−
J
X
j=1
cjφj( l)
2
,
o equi alen ly,
min
C(Z−ΦC) W( )(Z−ΦC),
wi h wl( ) = K(h−1( l− )), o K(·) a ke nel unc ion and ha s ic ly posi i e bandwid h.
Fu he mo e, W( ) is a diagonal L×L–ma ix which con ains all he weigh unc ions, ha is,
W( ) = diag(w1( ),...,wL( )).
In his si ua ion, one needs o sol e he equa ion 2Φ W( )ΦC −2Φ W( )Z= 0. Hence, he
es ima ed ec o o coe icien s, which depends on , is compu ed as
ˆ
C( ) = (Φ W( )Φ)−1Φ WZ,
and ˆx( ) = Φˆ
C( ) = Φ(Φ W( )Φ)−1Φ W( )Z. This app oach allows o cap u e local ea u es
o he da a sui ably, al hough he need o epea all he calcula ions o each new ime may
inc ease he compu a ional cos conside ably.
Rema k 1.3.4.As in he p e ious case, he localized leas squa es es ima o can be seen as a
linea smoo he wi h local ha ma ix
S( ) = Φ(Φ W( )Φ)−1Φ W( ).
Ques ion 3. How o choose he numbe o basis unc ions? The pa ame e Jallows o adjus
he “smoo hness” o he da a. A la ge J i s be e he da a, bu noise and a i icial a iabili y may be
in oduced in he i ing. On he o he hand, i one chooses a small J, pe haps no able da a ea u es
ge los . Thus, he la ge J, he la ge he a iance; whe eas, he smalle J, he la ge he bias. In
o de o balance his ade–o , he key is o ind a alue o Jsuch ha an e o c i e ium is minimized:
ei he a local e o c i e ium, such as he Mean Squa e E o (MSE)
MSE(ˆx( )) = E((ˆx( )−x( ))2) = Bias2(ˆx( )) + Va (ˆx( )),
1.3. PREPROCESSING FUNCTIONAL DATA 19
o a global e o c i e ium, such as he Mean In eg a ed Squa e E o (MISE)
MISE(ˆx) = Z1
0
MSE(ˆx( )).
In p ac ice, he e a e algo i hms in he mul i a ia e li e a u e ha selec he o de o he expansion
Jwhich can be adap ed o FDA ield, o example, me hods based on s epwise a iable selec ion o
a iable–p uning p ocedu es.
c) Smoo hing wi h a oughness penal y
The smoo hing wi h oughness penal y is a powe ul ool when i comes o app oxima e disc e e da a
by unc ional da a. Besides he s anda d ad an ages o any smoo hing me hod, his echnique is
especially bene icial when he es ima ion o de i a i es is in ol ed. The oughness penal y app oach is
based on he minimiza ion o a c i e ion ha ensu es ce ain egula i y condi ions o he i ed alues.
The e o e, he goal is o ind a cu e ha gi es a good i o he da a, bu es ic ing he sea ch o
“smoo h” cu es. This kind o cons ain allows o con ol simul aneously bo h bias and a iance.
The i s issue ha a ises is how o quan i y he oughness o a cu e x. The mos widesp ead
measu es o he oughness a e based on he de i a i es o x. I he q h de i a i e o xexis s, a na u al
penal y is
PENq(x) = Z1
0x(q)(s)2ds,
whe e x(q)deno es he q h de i a i e o x. In pa icula , he in eg a ed squa ed second de i a i e
PEN2(x) = Z1
0x(2)(s)2ds
is a common choice in many p ac ical si ua ions.
Once he penal y on he oughes cu es has been chosen, he nex s ep is he in oduc ion o he
penal y in o he op imiza ion p oblem. The aim is o ind a cu e x ha sol es he penalized p oblem
min
x(Z−x( )) W(Z−x( )) + ρPENq(x),
whe e Z= (z1,...,zL) is he obse ed L– ec o , x( ) = (x( 1),...,x( L)) is he L– ec o o he alues
o xin he disc e iza ion g id, Wis he weigh ing L×L–ma ix, and ρis a smoo hing pa ame e . The
smoo hing pa ame e de e mines i he p io i y is o sui able i he da a (smalle alues o ρ) o o
educe he a iabili y o he i ed cu e (la ge alues o ρ). In p ac ice, a c oss– alida ion me hod o
a gene alized c oss– alida ion me hod can be used o selec an adequa e alue o ρ(see, o ins ance,
Ramsay and Sil e man, 2005, Chap e 5).
Coming back o he basis expansion ha we e exposed in p e ious sec ions (see Rema k 1.3.1,
page 15), he oughness penal y can be adap ed o each ixed xas ollows
PENq(x) = Z1
0(Φ(q)(s)) C2ds =C RC,
whe e Φ(q)(s) is he J– ec o de ined by Φ(q)(s) = (φ(q)
1(s),...,φ(q)
J(s)) , and Ris he J×J–ma ix
gi en by R=R1
0Φ(q)(s)(Φ(q)(s)) ds. Then, he penalized leas squa es p oblem can be exp essed as
min
C(Z−ΦC) W(Z−ΦC) + ρC RC,(1.5)
being Φ he L×J–ma ix gi en by (Φ)l,j =φj( l). No e ha he p e ious minimiza ion p oblem is
equal o he weigh ed leas squa es p oblem in (1.3) when ρ= 0. The solu ion o he minimiza ion
p oblem (1.5) e i ies 2Φ WΦC −2Φ WZ + 2ρRC = 0. Hence, he penalized es ima o o he ec o
o coe icien s is ˆ
C= (Φ WΦ +ρR)−1Φ WZ,
and he i ed alues a e ˆ
Z=Φˆ
C=Φ(Φ WΦ +ρR)−1Φ WZ.
20 CHAPTER 1. INTRODUCTION TO FDA
Rema k 1.3.5.The smoo hing wi h oughness penal y is a linea smoo hing me hod oo. In his case,
he ha ma ix is de ined as
Sρ=Φˆ
C=Φ(Φ WΦ +ρR)−1Φ W,
and depends on he smoo hing pa ame e ρ. No e ha his ha ma ix is a sub–p ojec ion ope a o
(i is no a p ojec ion because SρSρ6=Sρ).
1.3.2 Regis e ing unc ional da a
Regis a ion is a p oblem o c i ical impo ance o unc ional da a (see Ramsay and Sil e man (2005,
Chap e 7) o Ramsay (2011)). This p ep ocessing may be needed when he a ia ion in unc ional
obse a ions in ol es ampli ude and/o phase. Nex he mos common si ua ions and he p ocedu es
used o sol e hese issues a e b ie ly desc ibed.
a) Ampli ude a ia ion
The e ical a ia ion o ampli ude a ia ion co esponds o he amilia e ical shi : Xi1and Xi2
may di e a poin s { l}L
l=1 a which hey a e compa ed, bu hey exhibi he same global shape
ea u es. The ampli ude e ec s a e o en emo ed cen ing and/o escaling he da a.
A classical example o ampli ude a ia ion is he spec ome ic da a, since he e is a e ical shi
in he cu es which is called calib a ion e ec in chemome ics (see le panel o Figu e 1.3, page 4).
The ampli ude a ia ion is an a i icial e ec ha has no link wi h he s udied chemical s uc u e,
hence he need o supp ess i . In his case, one way o emo ing he shi is o conside he de i a i es
o he cu es (see igh panel o Figu e 1.3, page 4)
b) Phase a ia ion
Some imes, Xi1and Xi2canno be compa ed a he same imes { l}L
l=1 because hey ha e di e en
beha iou s, whe eas he compa ison can be done i he ime scale is p e iously ans o med. This
si ua ion happens when he unc ional da a exhibi a phase a ia ion. The mos simple phase a ia ion
co esponds o a ho izon al shi . This o en happens when one is in e es ed on a segmen o he
unc ional da a which is a bi a y loca ed inside he comple e eco ded obse a ion. The aim is o
build a new sample ˜
Xi( ) = Xi( +δi), i = 1,...,n,
whe e δiis a shi pa ame e which enables he unc ional da a alignmen . In o de o es ima e
app op ia e shi pa ame e s, one has o de ine a c i e ion which indica es when wo obse a ions a e
egis e ed. One possibili y is o iden i y a ea u e o landma k ha is a cha ac e is ic o he unc ional
da a associa ed wi h a speci ic a gumen alue (maxima, minima, ze o c ossing, e c.). Then, each
unc ional da a is shi ed so ha he selec ed ea u e occu s a a ixed poin .
Depending on he landma ks, mo e complica ed ans o ma ions o he a gumen can be equi ed.
Fo example, i F ea u es a e selec ed, o each Xi he a gumen s { i }F
=1 associa ed wi h each
landma k ha e o be iden i ied. Nex , one has o look o n ans o ma ions {wi}n
i=1 such ha he
egis e ed obse a ions ˜
Xi( ) = Xi(wi( )), i =l,...,n
ha e simila a gumen alues o he chosen landma ks. In he enginee ing li e a u e, he ans o ma-
ions {wi}n
i=1 a e called ime–wa ping unc ions.
The es ima ion o some common ea u es o he unc ional da a has gene a ed many con ibu ions
(Kneip and Gasse , 1988; Kneip and Engel, 1995; Rønn, 2001; Gamboa e al., 2007). Fu he mo e,
bo h landma k egis a ion and wa ping echniques ha e been s udied in dep h in he li e a u e. Fo
he o me , one can look up Kneip and Gasse (1992), Gasse and Kneip (1995), Wang and Gasse
(1998), and Liu and Yang (2009). Fo he la e , some in e es ing e e ences a e Wang and Gasse
(1997), Ramsay and Li (1998), Wang and Gasse (1999), and Kneip e al. (2000).
1.4. EXPLORING FUNCTIONAL DATA 21
1.4 Explo ing unc ional da a
Many me hods ha e been de eloped, o simply adap ed om he exis ing mul i a ia e me hods, o
analysing a unc ional sample. Ne e heless, he impossibili y o de ining a densi y no ion causes
se ious analy ic di icul ies when one explo es unc ional da a. The o igin o his d awback is he lack
o a na u al unc ional measu e which ca ies ou he same ole han he Lebesgue measu e does in
he mul i a ia e con ex . Con ibu ions on he unc ional densi y p oblem can be ound in Jacob and
Oli ei a (1995), Dabo-Niang (2004), Delaigle and Hall (2010) o Dabo-Niang e al. (2010).
He e, a e iew o he main ools de o ed o he explo a ion o he s uc u e o a unc ional da ase is
p esen ed: measu es o posi ion and dispe sion, unc ional da a classi ica ion and spec al analysis. In
his sec ion, {Xi}n
i=1 is a sample o i.i.d. unc ional a iables as a andom unc ional a iable X alued
in H, being (H,h·,·i) a eal sepa able Hilbe space. Ne e heless, some imes he usual unc ional
space L2[0,1] has been conside ed in o de o in oduce some concep s.
1.4.1 Desc ip i e s a is ics
a) Measu es o posi ion
Mean. In classical eal da a analysis, he cen ali y measu e pa excellence is he mean. Hence, he
main issue is o gi e a no ion o mean o unc ional da ase s. When he unc ional space is L2[0,1],
he i s a emp is o de ine he unc ional mean a each ixed ime in T= [0,1] as
E(X)( ) = E(X( )),∀ ∈T,
and calcula e a poin wise a e age ac oss he sampled elemen s, ha is,
X( ) = 1
n
n
X
i=1
Xi( ).(1.6)
This nai e app oach has been e ined by means o smoo hing p ocedu es (Rice and Sil e man, 1991;
Ge ini, 2006; Li and Hsing, 2010; Cai and Yuan, 2011; Bunea e al., 2011), and obus me hods such
as he immed mean whe e only a ce ain pe cen age o he cen al da a a e used (Cues a-Albe os
and F aiman, 2006). O he ecen al e na i e is o eplace he adi ional c oss–sec ional mean by he
mani old mean such as Chen and M¨ulle (2012) ha e p oposed. Fu he mo e, in e ence based on he
mean has ecen ly been de eloped by Ho ´a h e al. (2012) in he unc ional ime se ies con ex .
Howe e , all hese p oposals do no yield a cen al measu e om a pu ely unc ional iewpoin . A
common way o do his is based on unc ional dep h ideas (F aiman and Muniz, 2001; Cue as e al.,
2006; Feb e o-Bande e al., 2007; L´opez-Pin ado and Jo ns en, 2007; L´opez-Pin ado and Romo, 2007,
2009).
Median and quan iles. An al e na i e cen al measu e is he unc ional median. Along he yea s,
a ious median de ini ions ha e been in oduced o a iables alued in in ini e dimensional spaces
(Kempe man, 1987; Va di and Zhang, 2000; Cad e, 2001; Fe a y and Vieu, 2006b; Ge ini, 2008;
Chaouch and Goga, 2012). Mo eo e , he unc ional dep h can also be used o de ine and es ima e
o he median no ions (F aiman and Muniz, 2001; Cue as e al., 2006; L´opez-Pin ado and Romo, 2006,
2007; L´opez-Pin ado and Jo ns en, 2007). Speci ically, a unc ional dep h p o ides a c i e ion o o de
a sample o cu es om he cen e –ou wa d, ha is, om he deepes cu es (cu es which a ain he
maximum alue o he unc ional dep h) o he mos ex eme ones (cu es which a ain he minimum
alue o he unc ional dep h). Hence, he unc ional median will be he deepes cu e o a ce ain
unc ional dep h.
Apa om he unc ional median, unc ional quan iles can be also in oduced o unc ional da ase s
by means o he unc ional dep h concep and he o de ha is induced by his dep h. Mo e de ails
can be ound in F aiman and Muniz (2001); L´opez-Pin ado and Romo (2006, 2007); L´opez-Pin ado
and Jo ns en (2007); Cue as e al. (2007); Cheng and de Gooije (2007); Chaouch (2008). A new
p ojec ion–based de ini ion o quan iles o in ini e dimensional Hilbe spaces can be ound in F aiman
and Pa ei o-L´opez (2012).
22 CHAPTER 1. INTRODUCTION TO FDA
Mode. The unc ional mode is ano he cen ali y measu e, which can be de ined and es ima ed
ollowing di e en app oaches as can be seen in Gasse e al. (1998), Hall and Heckman (2002), Cue as
e al. (2006), Dabo-Niang e al. (2006, 2007), Cue as e al. (2007) o Dabo-Niang e al. (2010). Fo
ins ance, he concep o unc ional mode in oduced by Cue as e al. (2006) lies in selec ing he cu e
mos “densely su ounded” o he unc ional da ase {xi}n
i=1, in pa icula , he cu e which sol es he
maximiza ion p oblem maxi∈{1,...,n}Pn
j=1 K(h−1kxj−xik), whe e K(·) is a ke nel unc ion and ha
bandwid h.
b) Measu es o dispe sion
Fo his sec ion, i is necessa y o ecall he enso no a ion ha was in oduced in Sec ion 1.2.2, “ c)
Tenso no a ion ” (see page 10), since i will be used.
Co a iance. To in oduce he co a iance concep , i is neccesa y o assume ha he H– alued
a iable Xsa is ies E(kXk2)<∞. In he pa icula case H=L2[0,1], he dependence o eco ds
along ime can be summa ized by means o he co a iance unc ion gi en by
KX( 1, 2) = E((X( 1)−µX( 1))(X( 2)−µX( 2))) = E(X( 1)X( 2)) −µX( 1)µX( 2),∀ 1, 2∈T,
whe e µX∈ H deno es he expec ed alue o X, ha is, µX=E(X). In p ac ice, he co a iance
unc ion is es ima ed using i s empi ical coun e pa
ˆ
KX( 1, 2) = 1
n
n
X
i=1
(Xi( 1)−X( 1))(Xi( 2)−X( 2))
=1
n
n
X
i=1
(Xi( 1)Xi( 2)) −X( 1)X( 2),∀ 1, 2∈T,
wi h Xde ined as (1.6) (see page 21).
In he mos gene al case, i.e., when (H,h·,·i) is a eal sepa able Hilbe space, he co a iance ope a o
ΓXcan be de ined as ΓX=E((X−µX)⊗H(X−µX)), and he e o e
ΓX:H → H
x→ΓXx=E(hX−µX, xi(X−µX)) = E(hX, xiX)−hµX, xiµX.
No e ha ΓXis jus he unc ional e sion o he s anda d mul i a ia e a iance–co a iance ma ix.
The co a iance ope a o is a nuclea 11, sel –adjoin 12 and posi i e13 ope a o (Dauxois e al., 1982).
I s eigen alues will be deno ed by {λj}∞
j=1 (such ha λ1≥λ2≥...), and he associa ed o hono mal
eigen unc ions by { j}∞
j=1. Gi en a sample {Xi}n
i=1, he co a iance ope a o ΓXcan be es ima ed by
i s empi ical coun e pa ΓX,n =n−1Pn
i=1 (Xi−X)⊗H(Xi−X) de ined as
ΓX,n :H → H
x→ΓX,nx=1
nPn
i=1 h(Xi−X), xi(Xi−X) = 1
nPn
i=1 hXi, xiXi−hX, xiX.
Fu he mo e, he eigen alues and eigen unc ions o ΓX,n will be deno ed by {(ˆ
λj,ˆ j)}∞
j=1, espec i ely
(being ˆ
λ1≥ˆ
λ2≥...≥ˆ
λn≥0 = ˆ
λn+1 =...).
11An ope a o Uon a Hilbe space H, gi en by U:H → H, is called nuclea i i can be w i en as
U=P∞
j=1 uj j⊗Hgj, whe e { j}∞
j=1 and {gj}∞
j=1 a e o hono mal se s o H, and {uj}∞
j=1 is a se o eal numbe s
such ha P∞
j=1 uj<∞.
12Asel -adjoin ope a o is a linea ope a o Ade ined on a linea e e ywhe e–dense se D(A) in a Hilbe space
Hsuch ha i coincides wi h i s adjoin ope a o A∗, ha is, such ha D(A) = D(A∗) and hAx, yi=hx, Ayi o all
x, y ∈D(A).
13Aposi i e ope a o on a Hilbe space His a linea ope a o Asuch ha hAx, xi ≥ 0.
1.4. EXPLORING FUNCTIONAL DATA 23
Rema k 1.4.1.The co a iance ope a o will play a key ole in FPCA exposed in Sec ion 1.4.3, “ a)
Func ional p incipal componen analysis (FPCA) ” (see page 25). This ac mo i a ed pape s de o ed
o es ima e he co a iance ope a o (Rice and Sil e man, 1991; Lee e al., 2002; Ge ini, 2006), and
some benchma k o i s componen s (Hall and Vial, 2006a).
Rema k 1.4.2.Conside ing he case H=L2([0,1]), one has
(ΓXx)( ) = EZ1
0
(X(s)−µX(s))x(s)ds(X( )−µX( ))=Z1
0KX(s, )x(s)ds,
ha is, he co a iance unc ion KXis he ke nel o he co a iance ope a o ΓX. Analogously, ˆ
KXis
he ke nel o ΓX,n
(ΓX,nx)( ) = 1
n
n
X
i=1 Z1
0
(Xi(s)−X(s))x(s)ds(Xi( )−X( ))=Z1
0
ˆ
KX(s, )x(s)ds.
Rema k 1.4.3.When Xis a cen ed a iable, i.e., µX= 0, he co a iance ope a o coincides wi h
he second momen ope a o and becomes in ΓX=E(X⊗HX), hus being ΓXx=E(hX, xiX) o
all x∈ H. Consequen ly, he co a iance ope a o can be es ima ed by he empi ical second momen
ope a o ΓX,n =n−1Pn
i=1 Xi⊗HXi, ha is, ΓX,nx=n−1Pn
i=1 hXi, xiXi o all x∈ H.
Rema k 1.4.4.Whene e he e is no possible con usion, ΓXand ΓX,n will be deno ed by Γ and Γn,
espec i ely, in o de o simpli y he no a ion.
C oss–co a iance. Le Xand Ybe wo H– alued a iables such ha E(kXk2)<∞and E(kYk2)<
∞. I one wan o analyse he deg ee o dependence be ween hem when he unc ional space is
H=L2[0,1], he c oss–co a iance unc ion de ined as
KX,Y ( 1, 2) = E((X( 1)−µX( 1))(Y( 2)−µY( 2))) = E(X( 1)Y( 2)) −µX( 1)µY( 2),∀ 1, 2∈T,
can be used, being µXand µY he expec ed alues o Xand Y, espec i ely (i.e., µX=E(X) and
µY=E(Y)). S a ing om a sample {(Xi, Yi)}n
i=1 o i.i.d. unc ional a iables dis ibu ed as (X, Y ),
he c oss–co a iance unc ion can be es ima ed by
ˆ
KX,Y ( 1, 2) = 1
n
n
X
i=1
(Xi( 1)−X( 1))(Yi( 2)−Y( 2))
=1
n
n
X
i=1
(Xi( 1)Yi( 2)) −X( 1)Y( 2),∀ 1, 2∈T,
wi h Xand Yde ined as (1.6) (see page 21).
I His a gene al eal sepa able Hilbe space, besides o he c oss–co a iance unc ion, one can also
compu e he c oss–co a iance ope a o de ined by ∆X,Y =E((X−µX)⊗H(Y−µY)), ha is,
∆X,Y :H → H
x→∆X,Y x=E(hX−µX, xi(Y−µY)) = E(hX, xiY)−hµX, xiµY.
F om a sample {(Xi, Yi)}n
i=1 o (X, Y ), he c oss–co a iance ope a o can be es ima ed by means o
he empi ical ope a o ∆X,Y,n =n−1Pn
i=1 (Xi−X)⊗H(Yi−Y) gi en by
∆X,Y,n :H → H
x→∆X,Y,nx=1
nPn
i=1 hXi−X, xi(Yi−Y) = 1
nPn
i=1 hXi, xiYi−hX, xiY .
An especially in e es ing si ua ion happens when Yis a eal andom a iable. Then, i H=L2([0,1]),
he c oss–co a iance unc ion and i s empi ical es ima o u n in o
KX,Y ( ) = E((X( )−µX( ))(Y−µY)) = E(X( )Y)−µX( )µY,∀ ∈T,
ˆ
KX,Y ( ) = 1
n
n
X
i=1
(Xi( )−X( ))(Yi−Y) = 1
n
n
X
i=1
(Xi( )Yi)−X( )Y , ∀ ∈T.
24 CHAPTER 1. INTRODUCTION TO FDA
As a as a gene al eal sepa able Hilbe space His conce ned, he c oss–co a iance ope a o ∆X,Y
u ns in o ∆X,Y =E((X−µX)⊗H′(Y−µY)), de ined as
∆X,Y :H → R
x→∆X,Y x=E(hX−µX, xi(Y−µY)) = E(hX, xiY)−hµX, xiµY.
In his case, he c oss–co a iance ope a o can be es ima ed using i s empi ical coun e pa ∆X,Y,n =
n−1Pn
i=1 (Xi−X)⊗H′(Yi−Y), ha is,
∆X,Y,n :H → R
x→∆X,Y,nx=1
nPn
i=1 hXi−X, xi(Yi−Y) = 1
nPn
i=1 hXi, xiYi−hX, xiY .
Rema k 1.4.5.I H=L2([0,1]), i is easy o show ha
(∆X,Y x)( ) = EZ1
0
(X(s)−µX(s))x(s)ds(Y( )−µY( ))=Z1
0KX,Y (s, )x(s)ds,
(∆X,Y,nx)( ) = 1
n
n
X
i=1 Z1
0
(Xi(s)−X(s))x(s)ds(Yi( )−Y( ))=Z1
0
ˆ
KX,Y (s, )x(s)ds.
Hence, KX,Y and ˆ
KX,Y a e he ke nel unc ions o ∆X,Y and ∆X,Y,n, espec i ely.
Rema k 1.4.6.I bo h Xand Yha e ze o–mean hen ∆X,Y =E(X⊗HY) wi h ∆X,Y x=E(hX, xiY) o
all x∈ H. The e o e, he c oss–co a iance ope a o can be es ima ed by ∆X,Y,n =n−1Pn
i=1 Xi⊗HYi
being ∆X,Y,nx=n−1Pn
i=1 hXi, xiYi o all x∈ H. Fu he mo e, when Y∈R, he c oss–co a iance
ope a o and i s empi ical es ima o u n in o ∆X,Y =E(X⊗H′Y) and ∆X,Y,n =n−1Pn
i=1 Xi⊗H′Yi.
Rema k 1.4.7.In o de o abb e ia e he no a ion, he c oss–co a iance ope a o and i s empi ical
coun e pa will be deno ed by ∆ and ∆nwhen con usion is no possible.
1.4.2 Func ional da a classi ica ion
O he ypes o explo a o y ools a e designed o gauge he exis ence o clus e s in he unc ional
andom sample {Xi}n
i=1, and o iden i y hem when hey exis . The iden i ica ion p ocess, which
se s he di e en clus e s, is o en based on any o he dis ibu ion pa ame e s ha we e p esen ed in
he p e ious sec ion as he mean o he mode. Wi hin his scope, a gene al classi ica ion algo i hm
usable wi h any cen ali y measu e was p oposed by Fe a y and Vieu (2006b), unc ional classi ica ion
ocused on modal cu e was p esen ed by Dabo-Niang e al. (2006, 2007), and dep h–based unc ional
classi ica ion ools we e p esen ed by L´opez-Pin ado and Romo (2006), Cue as e al. (2007), Cues a-
Albe os and Nie o-Reyes (2010), Li e al. (2012), and Sgue a e al. (2012). On he o he hand,
some con ibu ions we e de o ed o he adap a ion o well–known mul i a ia e k–means p ocedu es o
unc ional classi ica ion (see, o ins ance, Ta pey and Kina ede , 2003; Ab aham e al., 2003; Mizu a,
2004; Cues a-Albe os and F aiman, 2007; Ta pey, 2007). The e we e o he pape s which analysed
he unc ional classi ica ion o spa sely obse ed cu es (James and Suga , 2003; James, 2011), o
which p oposed al e na i es app oaches, o example, me hods based on neu al ne wo ks (Rossi e al.,
2004), on eg ession ees (Ne ini and Gha as, 2007), on eg ession echniques as PLS app oach (P eda
and Sapo a, 2005a; P eda e al., 2007; Delaigle and Hall, 2012), o on simila i y measu emen s such
as a ank co ela ion (Heckman and Zama , 2000). Fu he mo e, a ecen su ey on supe ised and
unsupe ised classi ica ion wi h unc ional da a can be ound in Ba´ıllo e al. (2011).
1.4.3 Spec al analysis
F om he seminal pape s by Rao (1958) and Tucke (1958), ac o ial analysis has u ned in o a key
ool o explo ing unc ional a iables. The unde lying idea o any ac o ial analysis is he spec al
decomposi ion o a unc ional ope a o , which is chosen depending on he p oposed p oblem and he
conside ed me hodology. Nex , a e iew o he main con ibu ions ela ed o ac o ial analysis is
p esen ed. I is ocused on FPCA which will be ecalled la e o de ine unc ional linea eg ession
es ima o s (see Sec ion 2.3.2 in Chap e 2, page 35, and Chap e 3).
1.4. EXPLORING FUNCTIONAL DATA 25
a) Func ional p incipal componen analysis (FPCA)
The analysis o he co a iance s uc u e o a unc ional da ase can epo aluable in o ma ion on he
ea u es and he beha iou o he da a. Since a di ec s udy o he co a iance ope a o is no easible,
FPCA is a undamen al ool o summa izing and isualizing he unde lying amewo k o his ope a o
in a clea and easy way. One o he ad an ages o FPCA is ha he unc ional da a a e p ojec ed
on he space spanned by he eigenelemen s o he co a iance ope a o . The e o e, a educ ion o he
p oblem dimension is possible i one es ic s he p ojec ion o only he i s eigen unc ions. Nex , i
will be exposed b ie ly how FPCA wo ks ( o mo e de ails, see Ramsay and Sil e man, 2005, Chap e 8
o Hall, 2011).
De ini ion o FPCA. Le Xbe a ze o–mean unc ional a iable alued in H. FPCA can be de ined
by means o he ollowing s epwise p ocedu e:
S ep 1. Find he elemen ξ1∈ H ha sol es
max
kξ1k=1 E(hξ1, Xi2) = max
kξ1k=1 E( 2
1),
being 1=hξ1, Xi he i s p incipal componen sco e, which gi es he p ojec ion o Xon
he di ec ion ξ1.
S ep 2. Fo j∈ {2,3,...}, compu e ξj∈ H such ha ξjis solu ion o he nex op imiza ion p oblem
max
kξjk=1 E(hξj, Xi2) = max
kξjk=1 E( 2
j)
subjec o he j−1 cons ain s
hξk, ξji= 0, o all 1 ≤k < j.
In his case, j=hξj, Xiis he j h p incipal componen sco e, and co esponds o he
p ojec ion o Xon he di ec ion ξj.
The di ec ions {ξj}∞
j=1 ob ained in he cou se o he p ocedu e will o m an o hono mal basis o H. On
each s ep, he goal is o de e mine he main sou ce o esidual a ia ion in X. In ac , he pe cen age
o a iabili y explained o he i s Jcomponen s can be de e mined using he nex exp ession
PJ
j=1 E( 2
j)
P∞
j=1 E( 2
j)×100.(1.7)
Eigenanalysis. Ano he cha ac e iza ion o FPCA can be de i ed om he spec al decomposi ion
o he co a iance ope a o Γ (see Sec ion 1.4.1, “ b) Measu es o dispe sion ”, page 22). The key in ol es
ew i ing he maximiza ion p oblem as ollows
maxkξ1k=1 hξ1,Γξ1i,
maxkξjk=1 hξj,Γξji,subjec o hξk, ξji= 0, o 1 ≤k < j. (1.8)
The p e ious op imiza ion p oblem can be sol ed by conside ing he eigen unc ion p oblem
Γ =λ ,
ha is, he p incipal componen s a e p ecisely he o hono mal eigen unc ions { j}∞
j=1 o he co a iance
ope a o , a anged om la ges eigen alue o smalles one. Fu he mo e, he associa ed eigen alues
{λj}∞
j=1 ep esen he a ia ion in Xexplained by each componen j, since h j,Γ ji=λj. Hence,
he a iabili y (1.7) explained by { j}J
j=1 can also be exp essed as
PJ
j=1 λj
P∞
j=1 λj×100.
26 CHAPTER 1. INTRODUCTION TO FDA
Es ima ion. Le {Xi}n
i=1 be a cen ed sample o X. The es ima ion o FPCA can be done ollowing
he nex s eps:
S ep 1. Look o he elemen ξ1∈ H ha sol es
max
kξ1k=1
1
n
n
X
i=1 hξ1, Xii2= max
kξ1k=1
1
n
n
X
i=1
2
i1,
whe e i1=hξ1, Xii o i∈ {1,...,n}. The alues { i1}n
i=1 a e called he i s p incipal
componen sco es.
S ep 2. Fo j∈ {2,3,...}, ind ξj∈ H ha sol es
max
kξjk=1
1
n
n
X
i=1 hξj, Xii2= max
kξjk=1
1
n
n
X
i=1
2
ij
subjec o he j−1 cons ain s
hξk, ξji= 0, o all 1 ≤k < j,
being { ij}n
i=1 ={hξj, Xii}n
i=1 he j h p incipal componen sco es.
The p e ious p ocedu e is equi alen o ind he solu ion o he eigenequa ion
Γn =λ
(see Sec ion 1.4.1, “ b) Measu es o dispe sion ”, page 22). The eigen unc ions {ˆ j}∞
j=1 o Γna e
es ima o s o he unc ional p incipal componen s o X, and he empi ical eigen alues {ˆ
λj}∞
j=1 o Γn
es ima e he a iabili y o Xexplained by each componen : hˆ j,Γnˆ ji=ˆ
λj, o all j∈ {1,2,...}.
Rema k 1.4.8.As i happens in he mul i a ia e case, he a ia ion de ined by n−1Pn
i=1 2
ij is educed
s ep by s ep, so no all he p incipal componen s a e compu ed in p ac ice, bu only he i s ones.
Usually, one ob ains he unc ional p incipal componen s, one a e ano he , un il a ce ain pe cen age
Po explained a iabili y is eached, ha is, one selec s Jsuch ha
PJ
j=1 (1
nPn
i=1 2
ij)
P∞
j=1 (1
nPn
i=1 2
ij)×100 > P,
being his exp ession he empi ical e sion o (1.7). Fu he mo e, he condi ion can also be exp essed
in e ms o he empi ical eigenelemen s {ˆ
λj,ˆ j}∞
j=1 as ollows
PJ
j=1 ˆ
λj
P∞
j=1 ˆ
λj×100 > P.
I is well–known ha {ˆ j}∞
j=1 is an o hono mal basis o H. Rema k 1.2.13 (see page 9) implies
ha kXik2=P∞
j=1 hXi,ˆ ji2, so n−1Pn
i=1 kXik2=P∞
j=1 ˆ
λj. Hence he denomina o in he p e ious
exp ession could be compu ed using he empi ical es ima ion o E(kXk2).
Al hough only a ew s a is icians paid a en ion o FPCA a i s (Rao, 1958; De ille, 1974; Dauxois
e al., 1982; Ramsay, 1982), he numbe o con ibu ions de o ed o his opic inc eased conside ably
om nine ies o nowadays (see a gene al e iew in Ramsay and Sil e man, 2005). Exis ing li e a u e
includes pape s conce ned wi h he es ima ion o he co a iance ope a o (Rice and Sil e man, 1991;
Lee e al., 2002; Ge ini, 2006; Hall and Vial, 2006a), wo ks ocused on heo e ical aspec s (Hall e al.,
2006; Hall and Hosseini-Nasab, 2006, 2009) o s udies ela ed o compu a ional issues (Oca˜na e al.,
2007).
1.4. EXPLORING FUNCTIONAL DATA 27
On he o he hand, in e es ing con ibu ions ha e analysed he impo ance o he scala p oduc
in ol ed in he p ojec ion s ep (Sil e man, 1996; Oca˜na e al., 1999; Locan o e e al., 1999) o he e ec
o in oducing smoo hing echniques in he s anda d FPCA such as splines ideas (Ramsay and Dalzell,
1991; Pezzulli and Sil e man, 1993; Sil e man, 1996; Ca do , 2000; James e al., 2000; Yao and Lee,
2006; Zhou e al., 2008), local linea smoo hing (Li and Hsing, 2010) o ke nel– ype app oaches (Boen e
and F aiman, 2000). In pa icula , he me hods p esen ed in wo o hese pape s (speci ically, Pezzulli
and Sil e man, 1993, and Sil e man, 1996), which can be seen as a combina ion o he s anda d FPCA
and smoo hing echniques, a e going o be desc ibed b ie ly. These wo app oaches will be ecalled in
Sec ion 3.4 in Chap e 3 (see page 60).
(i) Pezzulli and Sil e man’ smoo hed FPCA. A oughness penal y was in oduced in he op i-
miza ion p oblem (1.8) by Pezzulli and Sil e man (1993), who conside ed he ollowing s epwise
maximiza ion p ocess
maxkξ1k=1 (hξ1,Γξ1i−αhξ1, Qξ1i),
maxkξjk=1 (hξj,Γξji−αhξj, Qξji),subjec o hξk, ξji= 0, o 1 ≤k < j,
whe e αis a posi i e smoo hing pa ame e , and Qis a symme ic nonnega i e ope a o ha
quan i ies he smoo hness o solu ions ( o ins ance, an usual choice o Qis he ou h de i a i e
ope a o ).
In p ac ice, one needs o ind he pai s {(ˆ
λα,1
j,ˆ α,1
j)}∞
j=1, which sol e he gene alized eigenp oblem
(Γn−αQ) =λ .
Each eigen unc ion ˆ α,1
jes ima es he j h unc ional p incipal componen , and i s con ibu ion
o he explained a iabili y is de e mined by he associa ed eigen alue ˆ
λα,1
j.
(ii) Sil e man’s smoo hed FPCA. In he smoo hed FPCA p oposed by Sil e man (1996), he
oughness penal y is no inse ed in o he maximiza ion p oblem, bu in o he inne p oduc
de ini ion. Fo a gi en smoo hing pa ame e α, a new inne p oduc is de ined as
hx, yiα=hx, yi+αhx, Qyi,
being Qa symme ic nonnega i e ope a o ha con ols he smoo hness es ic ions. This inne
p oduc allows o build an induced no m by means o kxkα=hx, xi1/2
α.
Once he smoo hed inne p oduc is s a ed, Sil e man (1996) p oposed o sol e
maxkξ1kα=1 hξ1,Γξ1i,
maxkξjkα=1 hξj,Γξji,subjec o hξk, ξjiα= 0, o 1 ≤k < j.
I is impo an o highligh ha he smoo hness equi emen s only a ec he o hono mali y
cons ain s. In his si ua ion, he es ima ion p ocedu e leads o he solu ion {(ˆ
λα,2
j,ˆ α,2
j)}∞
j=1 o
he eigenp oblem
Γn =λ(1H+αQ) ,
wi h 1H he iden i y ope a o on H, i.e., 1Hx=x o all x∈ H. Hence, he ob ained eigenele-
men s allow o es ima e he unc ional p incipal componen s, and he a iabili y o Xwhich is
explained using each o hem.
Finally, no e ha FPCA has been success ully ex ended o a e y wide ange o si ua ions: ime
se ies (Bosq, 1991; Aguile a e al., 1997, 1999; Bosq, 2000), bi– unc ional da a (Spi zne e al., 2003),
longi udinal da a (James, 2002; Yao e al., 2005a), o condi ional amewo ks (Ca do , 2007).
Rema k 1.4.9.Al e na i es o unc ional p incipal componen s can be ound in Pa k e al. (2009),
who ied o cons uc mo e in o ma i e s uc u al componen s, o in Chen and M¨ulle (2012), who
p oposed he use o unc ional mani old componen s.
34 CHAPTER 2. FUNCTIONAL REGRESSION MODELS
wi h B= (b1,...,bJθ) he Jθ– ec o o he coe icien s, and Φ= (φ1, . . . , φJθ) he unc ional Jθ–
ec o o he basis unc ions. The choice o Jθis a key issue, since one needs a alue la ge enough
o a oid he loss o in o ma ion, bu small enough o make he in e p e a ion o he model pa ame e
easie .
One can epea he p ocedu e o he unc ional co a ia es by means o ano he basis unc ion
sys em {ψj}∞
j(wi h {ψj}∞
j=1 no necessa ily equal o {φj}∞
j=1). Then, one ge s
X[JX]
i=
JX
X
j=1
cijψj=Ψ Ci, o i= 1,...,n,
whe e Ci= (ci1,...,ciJX) is he JX– ec o o he coe icien s o Xi, and Ψ= (ψ1,...,ψJX) is he
unc ional JX– ec o o he basis unc ions.
Consequen ly, he eg ession model (2.1) (see page 32) could be app oxima ed by
Yi=hθ[Jθ], X[JX]
ii+ǫi=
Jθ
X
j1=1
JX
X
j2=1
bj1cij2hφj1, ψj2i+ǫi=C
iMΨΦB+ǫi, o i= 1,...,n
whe e MΨΦ is he JX×Jθ–ma ix gi en by (MΨΦ)j2,j1=hψj2, φj1i. Mo eo e , i Cis he n×JX–
ma ix whose ows a e he ec o s C
i, one can conside
Y=CMΨΦB+e=ΞB +e,
whe e Yis he n– ec o o he obse ed esponses, Ξis he n×Jθ–ma ix de ined by Ξ=CMΨΦ,
and eis he n– ec o o he e o s.
Nex , wo me hods o es ima ing Bbased on he minimiza ion o he o dina y esidual sum
o squa es, and based on he minimiza ion o a penalized esidual sum o squa es, espec i ely, a e
p esen ed.
a) Leas squa es es ima o
The Kθ– ec o o coe icien s Bcan be es ima ed op imizing he s anda d leas squa es p oblem
min
B(Y−ΞB) (Y−ΞB).
I can be shown ha he solu ion o his p oblem e i ies he equa ion 2Ξ ΞB −2Ξ Y= 0. Hence, B
is es ima ed by ˆ
B= (Ξ Ξ)−1Ξ Y,
and he de i ed p edic ions ˆ
Y=Ξˆ
B=Ξ(Ξ Ξ)−1Ξ Y.
b) Penalized leas squa es es ima o
The in oduc ion o oughness penal ies in he o dina y leas squa es p oblem allows o ix egula i y
condi ions o he model pa ame e , and a oid excessi e local luc ua ion in he es ima ion. The ick
is o conside he ollowing penalized esidual sum o squa es
min
B(Y−ΞB) (Y−ΞB) + ρPENq(θ),
being ρ he smoo hing pa ame e , and PENq(·) a oughness penal y, o example, any o he penal ies
conside ed in Sec ion 1.3.1, “ c) Smoo hing wi h a oughness penal y ”, in Chap e 1 (see page 19).
When he q h de i a i e o θexis s, PENq(θ) = R1
0(θ(q)(s))2ds can be selec ed, and he penal y
can be exp essed as
PENq(θ) = Z1
0(Φ(q)(s)) B2ds =B RB,
2.3. FUNCTIONAL LINEAR REGRESSION FOR SCALAR RESPONSE 35
whe e Φ(q)(s) = (φ(q)
1(s), . . . , φ(q)
Jθ(s)) is he Jθ– ec o o he de i a i es and Ris he Jθ×Jθ–ma ix
gi en by R=R1
0Φ(q)(s)(Φ(q)(s)) ds. Consequen ly, he penalized op imiza ion p oblem can be ew i -
en as ollows
min
B(Y−ΞB) (Y−ΞB) + ρB RB.
To sol e his p oblem, one has o ind he ec o Bwhich sa is ies 2Ξ ΞB−2Ξ Y+2ρRB = 0. Hence,
he es ima ed pa ame e is ˆ
B= (Ξ Ξ+ρR)−1Ξ Y,
and he i ed esponses a e ˆ
Y=Ξˆ
B=Ξ(Ξ Ξ+ρR)−1Ξ Y.
Rema k 2.3.3.The balance be ween bias and a iance is con olled by he smoo hing pa ame e . The
choice o ρcan be done in a subjec i e way o by da a–d i en selec o s, such as a c oss– alida ion
me hod (see u he de ails in Ramsay and Sil e man, 2005, Chap e 15).
Example. The penalized B–splines es ima o . The penalized B–splines es ima o is an es ima-
o based on basis expansions. Le {Bk,j, j = 1,...,k+q}be he no malized B–splines basis o he
space Sqk o splines de ined on [0,1] wi h deg ee qand (k−1) equispaced in e io kno s. The penalized
B–splines es ima o is de ined as
ˆ
θP S =
q+k
X
j=1
ˆ
bjBk,j,(2.2)
whe e ˆ
b= (ˆ
b1,...,ˆ
bq+k) is he solu ion o he minimiza ion p oblem
min
b
1
n
n
X
i=1
Yi−
q+k
X
j=1 hbjBk,j, Xii
2
+ρk(B( )
k) bk2
,(2.3)
wi h ρ he smoo hing pa ame e , and B( )
k he unc ional ec o o de i a i es o o de o he B–splines
(see, o ins ance, Ca do e al., 2003c).
Rema k 2.3.4.Le Ξbe he n×(q+k)–ma ix de ined by (Ξ)l,j =hBk,j, Xli, le Rbe he (q+k)×
(q+k)–ma ix de ined by (R)l,j =hB(q)
k,l , B(q)
k,j i. Then he penalized B–splines es ima o is an example
o penalized leas squa es es ima o , since he minimiza ion p oblem (2.3) is equi alen o
min
b(Y−Ξb) (Y−Ξb) + ρ
nb Rb.
2.3.2 Es ima o s based on FPCA
In his sec ion, some ecen esul s on FPCA– ype es ima o s o he unc ional linea model (2.1) (see
page 32) a e b ie ly ecalled. Fi s ly, he cons uc ion o his kind o es ima o s ollowing he pape s
by Ca do e al. (1999, 2003c), and he mos gene al app oach by Ca do e al. (2007c), is p esen ed.
Then, heo e ical esul s ela ed o consis ency, condi ional e o s and some asymp o ics a e compiled.
Le (H,h·,·i) be a eal sepa able Hilbe space. Recall ha {(λj, j)}∞
j=1 deno e he eigenelemen s
o he second momen ope a o Γ = E(X⊗HX), and ∆ = E(X⊗H′Y) is he c oss second momen op-
e a o . Fu he mo e, {(ˆ
λj,ˆ j)}∞
j=1 a e he eigen alues and eigen unc ions o Γn=n−1Pn
i=1 Xi⊗HXi,
and he empi ical e sion o ∆ is deno ed by ∆n=n−1Pn
i=1 Xi⊗H′Yi. See u he de ails in Sec-
ion 1.4.1, “ b) Measu es o dispe sion ”, in Chap e 1 (see page 22).
a) De ini ion o s anda d FPCA es ima o
In o de o es ima e he model pa ame e , Ca do e al. (2003c) s udied he op imiza ion p oblem
min
β∈HE(Y−hβ, Xi)2.
When
36 CHAPTER 2. FUNCTIONAL REGRESSION MODELS
(C.2.1) Ke (Γ) = {0}, and P∞
j=1 (∆ j/λj)2<+∞,
whe e Ke (Γ) deno es he ke nel o null space o Γ de ined as Ke (Γ) = {x∈ H|Γx= 0}, he pa ame e
θis he unique solu ion o his minimiza ion p oblem, and i sa is ies he equa ion ∆x=hθ, Γxi o
all x∈ H. This ac allows o exp ess he model pa ame e as
θ=∞
X
j=1
∆ j
λj
j.
Rema k 2.3.5.The applica ion o Rema k 1.2.13 in Chap e 1 (see page 9), aking as o hono mal
basis { j}j, leads o θ=P∞
j=1 hθ, ji j. Mo eo e , he equali y ∆x=hθ, Γxi, o all x∈ H, ensu es
ha ∆ j=λjhθ, ji o all j. Thus, θ=P∞
j=1 λ−1
j∆ j j.
The e is no bounded in e se o Γ, so Ca do e al. (1999) p ojec ed he da a on he subspace
spanned by he i s kneigen unc ions o Γn. Gi en {(Xi, Yi)}n
i=1 a sample o i.i.d. andom a iables
d awn om (X, Y ), hey p oposed o es ima e θby
ˆ
θkn=
kn
X
j=1
∆nˆ j
ˆ
λj
ˆ j,(2.4)
whe e {kn}∞
n=1 is a sequence o posi i e in ege s such ha kn→+∞,kn≤n, and ˆ
λkn>0.
Rema k 2.3.6.No e ha (2.4) is he unca ed e sion o ˆ
θ=P∞
j=1 ˆ
λ−1
j∆nˆ jˆ j, which sa is ies
∆nx=hˆ
θ, Γnxi,∀x∈Im(Γn),
whe e Im(Γn) deno es he image o he ope a o Γn, i.e., Im(Γn) = {x′∈ H|x′= Γnx, x ∈ H}.
Al e na i e cons uc ion. The es ima o ˆ
θkncan be ob ained by p ojec ing he unc ional ob-
se a ions on o a ini e subspace o H, and applying he same a gumen s used in Fa aldo-Roca and
Gonz´alez-Man eiga (1987) in o de o educe he condi ional mean squa e e o o he s anda d leas
squa es es ima o as ollows. Le {ej}∞
j=1 be an o hono mal basis o H, and ix kn< n. Fo all
x∈ H, he co esponding bold aced le e xdeno es he kn– ec o gi en by x= (hx, e1i,...,hx, ekni) .
Following he s eps gi en by C is ´obal-C is ´obal e al. (1987), conside he gene alized op imiza ion
p oblem
min
bEµn( ˆmkn(X)−X b)2,wi h b= (hβ, e1i,...,hβ, ekni) ,∀β∈ H,(2.5)
whe e
•ˆmkn(x) = Pn
i=1 Yiδ(x,Xi)/Pn
i=1 δ(x,Xi) is a nonpa ame ic es ima o o he eg ession unc ion
mkn(x) = E(Y|X=x), and
•µn(x) = Rx
−∞ ˆ
n( )d is a weigh ing unc ion whe e ˆ
nis a nonpa ame ic es ima o o he densi y
o Xde ined as ˆ
n(x) = n−1Pn
i=1 δ(x,Xi),
being δ(·,·) a measu able unc ion om Rkn×Rknin o R. Sol ing (2.5) o he special case
δ(u,w) = 1,i u=w,
0,i u6=w,
one ge s ha β0∈ H is a solu ion o (2.5) i and only i b0= (hβ0, e1i, . . . , hβ0, ekni) sa is ies he
kn–dimensional no mal equa ion ∆n=Γnb0whe e
Γn=1
n
n
X
i=1
XiX
iand ∆n=1
n
n
X
i=1
XiYi.
2.3. FUNCTIONAL LINEAR REGRESSION FOR SCALAR RESPONSE 37
This amoun s o knequa ions
∆nel=
kn
X
j=1 hΓnel, ejihβ0, eji, o all l∈ {1,...,kn}.
Taking el= ˆ l, one ge s hβ0,ˆ li=ˆ
λ−1
l∆nˆ l o all l∈ {1,...,kn}, and he es ima o (2.4) appea s as
he p ojec ion o β0on o he subspace o Hspanned by {ˆ j}kn
j=1 (i.e., ˆ
θkn=Pkn
j=1hβ0,ˆ jiˆ j).
Rema k 2.3.7.The s anda d FPCA es ima o ˆ
θknis he solu ion o (2.5) o a basic nonpa ame ic
es ima o (i.e., ˆmkn(Xi) = Yi). A na u al ex ension o ˆ
θknconsis s in in es iga ing on he solu ion
o (2.5) o a gene al nonpa ame ic es ima o , which is equi alen o p esmoo h he esponses be o e
es ima ing he unc ional pa ame e θ(see mo e de ails in Chap e 3).
b) De ini ion o gene al class o FPCA– ype es ima o s
Following he s anda d FPCA app oach, Ca do e al. (2007c) analysed a la ge class o FPCA– ype
es ima o s, whe e (2.4) is included as pa icula case. They assumed ha λ1> λ2> . . . > 0, whe e
he mul iplici y o each λjis one, and hey sol ed he o iginal p oblem o ill–condi ioned in e se o
Γ by means o a egula iza ion p ocedu e o which i is necessa y o in oduce some no a ion. They
de ine
δj=λ1−λ2,i j= 1,
min(λj−1−λj, λj−λj+1),i j6= 1,
ix a s ic ly posi i e sequence c=cnsuch ha c→0 and c < λ1, and se
kc
n= sup {j:λj+δj/2≥c}.
They also conside a sequence { c
n: [c, +∞)→R}no posi i e unc ions such ha
(C.2.2) c
nis dec easing on [c, λ1+δ1],
(C.2.3) ( c
n)′(x) exis s o all x∈[c, +∞), and
(C.2.4) supx≥c|x c
n(x)−1|=o(n−1/2).
Then, he p oposed es ima o o he model pa ame e θis
ˆ
θc=
n
X
j=1
c
n(ˆ
λj)∆nˆ jˆ j.(2.6)
Rema k 2.3.8.Rega ding he suppo o c
n, one ge s
ˆ
θc=
Jc
n
X
j=1
c
n(ˆ
λj)∆nˆ jˆ j,
whe e Jc
n= sup {j:ˆ
λj≥c}. Ob iously, Jc
nis a andom sequence ha can be di e en om kc
n. How-
e e , looking a p oo o Lemma 5 by Ca do e al. (2007c), i can be deduced ha , i (kc
n)2log kc
n/√n→
0, P(Jc
n6=kc
n)→0 as enough o conside only he case Jc
n=kc
n( a e n−1/2).
Example 1. F om he p e ious ema k, i is easy o deduce ha he s anda d FPCA es ima o (2.4)
is asymp o ically equi alen o (2.6) when
n(x) = x−1I{x≥c}.
38 CHAPTER 2. FUNCTIONAL REGRESSION MODELS
Example 2. The idge– ype FPCA es ima o p oposed by Ma ´ınez-Cal o (2008) and Fe a y e al.
(2012a) also belongs o his la ge amily o es ima o s based on FPCA. In his case, one has o choose
n(x) = (x+αn)−1I{x≥c},
o αna sequence o posi i e pa ame e s.
c) Consis ency
In his sec ion, Theo em 3.2 by Ca do e al. (1999) is ep oduced. This esul ensu es he almos
su ely con e gence o he s anda d FPCA es ima o ˆ
θkn. Fi s o all, he almos su ely con e gence
and i s associa ed “big O” and “li le o” no a ion a e in oduced.
De ini ion 2.3.9. Le {Zn}n∈Nbe a sequence o eal andom a iables and le Zbe a eal andom
a iable, all o hem de ined on he same p obabili y space (Ω,A,P). {Zn}n∈Ncon e ges almos su ely
(a.s. o a.s. −P) o Z, ha is,
lim
n→∞Zn=Z a.s., o equi alen ly, Zn→Z a.s.,
i and only i
Pω∈Ω : lim
n→∞Zn(ω) = Z(ω)= 1.
De ini ion 2.3.10. Le {Zn}n∈Nbe a sequence o eal andom a iables and le Zbe a eal andom
a iable, all o hem de ined on he same p obabili y space (Ω,A,P). Le {un}n∈Nbe a de e minis ic
sequence o posi i e eal numbe s. The a e o almos su ely con e gence o {Zn}n∈N o Zis o o de
un, ha is,
Zn−Z=Oa.s.(un),
i and only i
P(ω∈Ω : ∃c < ∞,∃N, ∀n > N, |Zn(ω)−Z(ω)| ≤ cun) = 1.
Fu he mo e,
Zn−Z=oa.s.(un) i and only i (un)−1(Zn−Z)→0a.s.
On he o he hand, ecall ha k ·kH′deno es he no m o he dual space H′(see Sec ion 1.2.2, “ b)
The dual space H′”, in Chap e 1, page 10). I is also necessa y o in oduce he ollowing no a ion
aj= 2√2/δj, ha is,
aj=2√2/(λ1−λ2),i j= 1,
2√2/min(λj−1−λj, λj−λj+1),i j6= 1,
and m(·) = hθ, ·i and ˆmkn(·) = hˆ
θkn,·i. In addi ion, he ollowing assump ions a e equi ed:
(C.2.5) λ1> λ2> . . . > 0, and ˆ
λ1>ˆ
λ2> . . . > ˆ
λkn>0a.s.
(C.2.6) kXk ≤ c1a.s.
(C.2.7) ∃c2>0,∀l≥1,E(|ǫ|l)< l!c2<+∞,
(C.2.8) nλ4
kn/log n→+∞, and nλ2
kn/((Pkn
j=1 aj)2log n)→+∞.
2.3. FUNCTIONAL LINEAR REGRESSION FOR SCALAR RESPONSE 39
Theo em 2.3.11 (Ca do e al., 1999).I (C.2.1) and (C.2.5)–(C.2.8) a e sa is ied, hen
kˆmkn−mkH′→0a.s.
Rema k 2.3.12.Ac ually, Ca do e al. (1999) ob ained he p e ious esul assuming he hypo hesis
(C.2.7 ’) |ǫ| ≤ c2a.s.
ins ead o (C.2.7), bu i can be shown ha consis ency holds despi e his modi ica ion. The key
poin is o eplace he second pa o Lemma 5.3 by Ca do e al. (1999) by Lemma 2.5.1 (see page 47).
This assump ion has been al e ed in o de o ex end he o iginal esul by Ca do e al. (1999) o a
la ge class o e o s, which includes, o ins ance, non–bounded e o s such as he Gaussian ones.
Rema k 2.3.13.I kn=o(log n), (C.2.8) is sa is ied when λja e geome ically o algeb aically de-
c easing, ha is, when
λj=c1cj
2,wi h c1>0 and 0 < c2<1,
o
λj=c1j−c3,wi h c1>0 and c3>1.
d) Condi ional e o s
The nex heo em gi es he condi ional mean squa e p edic ion e o o a new esponse
Yn+1 =hθ, Xn+1i+ǫn+1,
and he condi ional mean squa e es ima ion e o . Fo any n∈N∗, deno e Xn={X1,...,Xn}, and
le EXn(·) be he expec a ion condi ionally on Xn. Fu he mo e, le ˆ
Rknbe he e m de ined as
ˆ
Rkn=X
j>knhθ, ˆ jiˆ j.(2.7)
Theo em 2.3.14 (Fe a y e al., 2012a).Fo he s anda d FPCA es ima o (2.4), i holds ha
EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2=σ2+σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
+hXn+1,ˆ
Rkni2,
EXn(kθ−ˆ
θknk2) = σ2
n
kn
X
j=1
1
ˆ
λj
+kˆ
Rknk2,
whe e ˆ
Rknis de ined in (2.7).
The p oo o he p e ious esul can be ound in he appendix o he chap e (see Sec ion 2.5.2, page 48).
Rema k 2.3.15.Theo em 2.3.14 can be de i ed di ec ly om Lemma 2.5.2 (see page 49), which allows
o ob ain he condi ional e o s o a gene al ype o es ima o s in a simple way. The condi ional
es ima ion e o o ˆ
θkngi en in Theo em 2.3.14 was al eady s udied in li e a u e. In ac , Theo em 5
in Hall and Hosseini-Nasab (2006) gi es condi ions in o de o ind ha
E(kθ−ˆ
θknk2)∼σ2
n
kn
X
j=1
1
λj+kRknk2,
whe e Rkn=Pj>knhθ, ji j, and Wn∼Znmeans ha he a io o Wnand Zncon e ges o 1 when
n→+∞.
40 CHAPTER 2. FUNCTIONAL REGRESSION MODELS
e) Asymp o ic no mali y
In his sec ion, he gene al FPCA– ype es ima o ˆ
θcde ined by (2.6) is conside ed. Ca do e al.
(2007c) p oposed in hei wo k o de i e a Cen al Limi Theo em (CLT) in he unc ional linea
eg ession model wi h scala esponse. In pa icula , hey ga e he ollowing weak con e gence esul s
o he p edic ion a a ixed x∈ H, being he ollowing assump ions equi ed:
(C.2.9) P+∞
j=1 |hθ, ji| <+∞,
(C.2.10) ∃λa con ex posi i e unc ion such ha λj=λ(j), o jla ge,
(C.2.11) E(kXk4)<∞, and supj(E(hX, ji4)/λ2
j)<+∞,
(C.2.12) (kc
n)3(log (kc
n))2/( c
n,x√n)→0 wi h c
n,x =qPkc
n
j=1 λj( c
n(λj))2hx, ji2,
(C.2.13) P+∞
j=1 (hx, ji2/λj)<+∞.
In addi ion, ecall he de ini ion o con e gence in dis ibu ion.
De ini ion 2.3.16. Le {Zn}n∈Nbe a sequence o eal andom a iables wi h dis ibu ion Fnand le
Zbe a eal andom a iable wi h dis ibu ion F.{Zn}n∈Ncon e ges in dis ibu ion o weakly (do
w) o Z, ha is,
Zn
d
→Z, o equi alen ly, Zn
w
→Z,
i and only i
lim
n→∞Fn(x) = F(x),
o e e y x∈Ra which Fis con inuous.
Theo em 2.3.17 (Ca do e al., 2007c).Gi en a alue xin H, i (C.2.1),(C.2.2)–(C.2.4), and
(C.2.9)–(C.2.13) hold, hen
√n
c
n,xσ(hˆ
θc, xi−hˆ
Πkc
nθ, xi)w
→ N(0,1),
being ˆ
Πkc
n he p ojec o on o he subspace spanned by he i s kc
neigen unc ions o Γn, and N(0,1)
he dis ibu ion o a Gaussian andom a iable wi h mean equals o 0and a iance equals o 1.
Co olla y 2.3.18 (Ca do e al., 2007c).Theo em 2.3.17 s ill holds i c
n,x (de ined in assump ion
(C.2.12)) is eplaced by i s empi ical coun e pa
ˆ
c
n,x =
u
u
kc
n
X
j=1
ˆ
λj( c
n(ˆ
λj))2hx, ˆ ji2,
and σis eplaced by a consis en es ima e ˆσ.
Rema k 2.3.19.No e ha he bias e m is andom. Ca do e al. (2007c) ema ked ha e y speci ic
assump ions on θo {λj}ja e equi ed in o de o eplace he bias by a non– andom quan i y.
2.4. FUNCTIONAL NONPARAMETRIC REGRESSION FOR SCALAR RESPONSE 41
Rema k 2.3.20.Con idence in e als o p edic ion can be de i ed om he CLT in Co olla y 2.3.18.
Le α∈[0,1] be a ce ain con idence le el, and le zαbe he quan ile o o de α om a N(0,1)
dis ibu ion. I i is assumed ha θ(o x) is e y well app oxima ed by i s p ojec ion ˆ
Πkc
nθ(o ˆ
Πkc
nx),
hen Co olla y 2.3.18 allows o e alua e asymp o ic con idence in e als o hθ, xiby compu ing
CIasy
x,α =hhˆ
θc, xi−ˆ
c
n,xˆσn−1/2z1−α/2,hˆ
θc, xi+ˆ
c
n,xˆσn−1/2z1−α/2i,(2.8)
o which P(m(x)∈CIasy
x,α)≈1−α.
2.4 Func ional nonpa ame ic eg ession o scala esponse
This sec ion is ocused on he ke nel– ype es ima o s o he unc ional nonpa ame ic model exp essed
as
Y=m(X) + ǫ, (2.9)
whe e Yis a eal andom a iable, m(·) is a unc ional eg ession ope a o which sa is ies some smoo h-
ness es ic ions, Xis a ze o–mean andom a iable alued in an abs ac space S, and ǫis a eal
andom a iable sa is ying ha E(ǫ) = 0, Va (ǫ) = σ2<∞, and E(ǫX) = 0.
The mos gene al o mula ion assumes ha he unc ional space Sis a semi–me ic space endowed
wi h a semi–me ic d(·,·), so his abs ac app oach has been almos always kep in he nex subsec ions
excep o ce ain esul s which equi e speci ically no med spaces. In o de o p ese e he no a ion
used up o now, he semi–me ic space (H, d(·,·)), whe e His s ill a eal sepa able Hilbe space, and
d(·,·) is he semi–me ic induced by he inne p oduc , i.e., d(x, y) = kx−yk=hx−y, x −yi1/2
(in ac , d(·,·) is a me ic wi h his cons uc ion), is going o be selec ed. Howe e , no e ha any
o he semi–me ic space could be conside ed (see some examples o semi–me ics in Sec ion 1.2.3 in
Chap e 1, page 10).
Among he di e en exis ing nonpa ame ic me hods, he sec ion is ocused on he ke nel– ype
es ima o s based on condi ional expec a ion in oduced by Fe a y and Vieu (2006b), and s udied by
Fe a y and Vieu (2004) and Fe a y e al. (2007a), al hough o he nonpa ame ic echniques based on
condi ional median o condi ional mode we e also de eloped in he li e a u e (see, o ins ance, Fe a y
and Vieu, 2006b). Nex , some ou s anding ad ances ela ed wi h his unc ional ke nel me hodology
ha e been summed up.
2.4.1 Ke nel– ype es ima o s
Gi en he unc ional model (2.9), he goal is o es ima e he eg ession ope a o m(·) de ined by
m:H → R
x→m(x) = E(Y|X=x).
As men ioned be o e, he eg ession ope a o does no belong o any pa ame ic amily in his case,
bu m(·) sa is ies ce ain egula i y assump ions. Usually, one conside s con inui y– ype condi ions
(C.2.14) m∈CH,0={ :H → Rsuch ha limd(x,x′)→0 (x′) = (x)},
o Lipschi z– ype condi ions, ha is, ∃β > 0 such ha
(C.2.15) m∈LipH,β ={ :H → Rsuch ha ∃c > 0,| (x)− (x′)|< cd(x, x′)β}.
In he ollowing subsec ions, he unc ional ke nel es ima o is p esen ed, oge he wi h some in-
e es ing esul s. In some o hem, he small ball p obabili y unc ion ϕx(·) will play a decisi e ole.
Le B(x, h) be a ball cen ed a x∈ H wi h adius h, ha is, B(x, h) = {x′∈ H|d(x, x′)≤h}. Then
ϕx(h) will deno e he p obabili y o he ball B(x, h), i.e.,
ϕx(h) = P(X∈B(x, h)).
Mo e de ails abou he small ball p obabili y no ion and examples o wha ϕx(h) is some speci ic cases
can be ound in Li and Shao (2001), and Fe a y and Vieu (2006b, 2011a).
42 CHAPTER 2. FUNCTIONAL REGRESSION MODELS
a) De ini ion o ke nel es ima o
Gi en x∈ H and a sample {(Xi, Yi)}n
i=1 d awn om he pai (X, Y ), Fe a y and Vieu (2006b)
p oposed he ollowing es ima o o he eg ession ope a o
ˆmh(x) = Pn
i=1 YiK(h−1d(x, Xi))
Pn
i=1 K(h−1d(x, Xi)) ,(2.10)
whe e K(·) is an asymme ic ke nel, and h=h(n) is a s ic ly posi i e eal bandwid h. The ke nel
es ima o (2.10) is he na u al adap a ion o he Nada aya–Wa son es ima o o he unc ional con ex ,
whe e he o dina y mul i a ia e no m was eplaced by he unc ional semi–me ic d(·,·) which measu es
he dis ances be ween unc ional obse a ions. Gi en ha d(x, y)≥0 o all x, y ∈ H, he suppo o
K(·) should be posi i e. Hence, asymme ic ke nels a e he mos adequa e choice: o ins ance, he
asyme ic e sions o he ke nel unc ions in oduced in Figu e 1.5 (see Chap e 1, page 15), which a e
plo ed in Figu e 2.1.
−1 0 1 2 3
0.0 0.5 1.0 1.5 2.0
Uni o m
−1 0 1 2 3
0.0 0.5 1.0 1.5 2.0
T iangle
−1 0 1 2 3
0.0 0.5 1.0 1.5 2.0
Quad a ic
−1 0 1 2 3
0.0 0.5 1.0 1.5 2.0
Gaussian
Figu e 2.1: Usual asymme ic ke nels: asymme ic uni o m ( op le panel), asymme ic iangle ( op
igh panel), asymme ic quad a ic (bo om le panel) and asymme ic Gaussian (bo om igh panel).
This ke nel es ima o is ac ually a weigh ed a e age o {Yi}n
i=1. This issue can be easily seen i one
2.4. FUNCTIONAL NONPARAMETRIC REGRESSION FOR SCALAR RESPONSE 43
conside s he weigh s
wih(x) = K(h−1d(x, Xi))
Pn
i′=1 K(h−1d(x, Xi′)),
which sa is y Pn
i=1 wih(x) = 1. Hence, he es ima o (2.10) can be exp essed as he weigh ed a e age
o he obse ed esponses, ha is,
ˆmh(x) =
n
X
i=1
wih(x)Yi.
I is clea ha he ke nel es ima o depends on he choice o he ke nel unc ion K(·), and he
smoo hing pa ame e h. Tha is why some no es on K(·) and hselec ion a e p esen ed below.
Ke nel unc ions. The ke nel K(·) is usually an asymme ic e sion o ke nel unc ions exposed
in Sec ion 1.3.1, “ a) Linea smoo hing ”, in Chap e 1 (see page 13). In gene al, he mos common
ke nels belong o one o he ollowing amilies (see Fe a y and Vieu, 2006b).
De ini ion 2.4.1. A unc ion K:R→[0,+∞) is called a ke nel o ype I i RK(u)du = 1, and i
he e exis c1, c2∈Rsuch ha 0 < c1< c2<+∞and
c1I{u∈[0,1]}≤K(u)≤c2I{u∈[0,1]},
being I he indica o unc ion.
De ini ion 2.4.2. A unc ion K:R→[0,+∞) is called a ke nel o ype II i RK(u)du = 1, i s
suppo is [0,1], and i s i s de i a i e K′exis s on [0,1] and sa is ies
c1≤K′≤c2,
o c1, c2∈Rsuch ha −∞ < c1< c2<0.
The main discon inuous asymme ic ke nels belong o he i s amily (e.g., asymme ic uni o m),
whe eas he second amily con ains he usual asymme ic con inuous ones (e.g., asymme ic iangle
o asymme ic quad a ic). The e is a close link be ween hese ke nel amilies and he small ball
p obabili y ϕxexac ly as he nex lemmas s a e.
Lemma 2.4.3 (Fe a y and Vieu, 2006b).Le K(·)be a ke nel o ype I. Then he e exis c1, c2∈R
wi h 0≤c1, c2<+∞such ha
c1ϕx(h)≤E(K(h−1d(x, X))) ≤c2ϕx(h).
The p e ious esul can be ex ended o ke nel o ype II. Fo his pu pose, i is necessa y ha he nex
assump ion
(C.2.16) ∃c > 0,∃ε0,∀ε < ε0,Rε
0ϕx(u)du > cεϕx(ε)
is sa is ied by he small ball p obabili y ϕx.
Lemma 2.4.4 (Fe a y and Vieu, 2006b).Le K(·)be a ke nel o ype II. I ϕx(·) e i ies (C.2.16)
hen he e exis c1, c2∈Rwi h 0≤c1, c2<+∞such ha , o hsmall enough,
c1ϕx(h)≤E(K(h−1d(x, X))) ≤c2ϕx(h).
50 CHAPTER 2. FUNCTIONAL REGRESSION MODELS
and, applying (2.12) again,
EXn(kθ−ˆ
θk2) = σ2
n
kn
X
j1=1
kn
X
j2=1
γj1γj2hΓnwj1, wj2ihwj1, wj2i+kR(γ,w)
knk2.
Chap e 3
P esmoo hing in unc ional linea
eg ession
In his chap e , he unc ional linea model wi h scala esponse and explana o y a iable
alued in a unc ional space is conside ed. In ecen s a is ical li e a u e, FPCA has been
used o es ima e he model unc ional pa ame e . This app oach can be modi ied by using
p esmoo hing echniques: ei he p esmoo hing ia co a iance s uc u e o p esmoo hing
ia esponse a iable. Fo hese new es ima o s, consis ency is s a ed and e iciency by
compa ison wi h he s anda d FPCA es ima o is s udied om a heo e ical poin o iew.
Fu he mo e, he ini e sample pe o mance o he p oposed p esmoo hed es ima o s is
also analysed by means o a simula ion s udy and h ee eal da a applica ions. Finally, he
p oo s o he main esul s in he chap e and some echnical lemmas a e ga he ed oge he
in he appendix.
The i s ools o de eloping he me hodology which is compiled in his chap e we e
p esen ed in Ma ´ınez-Cal o (2008). La e , he comple e esea ch on he app oach based
on p esmoo hing ia co a iance s uc u e was published in Fe a y e al. (2012a)
3.1 Why in oduce p esmoo hing echniques?
In he p e ious chap e , i was showed how classical mul i a ia e me hods ha e been adap ed o he
unc ional con ex whe e he esponse Yand/o he explana o y a iable Xa e alued in a unc ional
space. Pa icula ly, he unc ional linea model wi h scala esponse was speci ied o being he subjec
o se e al s udies in he ecen li e a u e, as i is in his chap e . Recall ha , gi en a eal sepa able
Hilbe space (H,h·,·i) (k·kdeno es he induced no m), he unc ional linea model wi h scala esponse
was in oduced in (2.1) (see Chap e 2, page 32) as
Y=m(X) + ǫ=hθ, Xi+ǫ,
whe e Yis a eal andom a iable, m(·) = hθ, ·i is a linea eg ession ope a o such ha θ∈ H and
kθk2<∞,Xis a ze o–mean andom a iable alued in Hsa is ying ha E(kXk2)<∞, and ǫis a
eal andom a iable such ha E(ǫ) = 0, Va (ǫ) = σ2, and E(ǫX) = 0. Rema k 2.3.1 (see Chap e 2,
page 33) showed ha he e is no loss o gene ali y in he conside a ion o ze o–mean a iables in he
model, whe eas Rema k 2.3.2 (see Chap e 2, page 33) p o ides he pa icula exp ession o he model
when H=L2([0,1]).
As i was commen ed in Chap e 2, es ima o s o he unc ional linea model wi h scala esponse
can be ob ained based on FPCA (Ca do e al., 1999, 2003c; Cai and Hall, 2006; Hall and Hosseini-
Nasab, 2006; Hall and Ho owi z, 2007; Ca do e al., 2007c). Since he FPCA es ima o is ein oduced
in his chap e , ecall ha when (C.2.1) holds (see Chap e 2, page 36), he model pa ame e can be
51
52 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
exp essed as θ=P∞
j=1 λ−1
j∆ j j. The e o e, gi en {(Xi, Yi)}n
i=1 a andom sample o i.i.d. a iables
d awn om (X, Y ), he s anda d FPCA es ima o was de ined in (2.4) (see Chap e 2, page 36) by
ˆ
θkn=
kn
X
j=1
∆nˆ j
ˆ
λj
ˆ j,
whe e {(ˆ
λj,ˆ j)}∞
j=1 a e he eigen alues and eigen unc ions o he empi ical second momen ope a-
o Γn=n−1Pn
i=1 Xi⊗HXi, ∆n=n−1Pn
i=1 Xi⊗H′Yiis he c oss second momen ope a o , and
{kn}∞
n=1 is a sequence o posi i e in ege s such ha kn→+∞,kn≤n, and ˆ
λkn>0. Fu he de ails
ela ed o his es ima o can be ound in Sec ion 2.3.2 (see Chap e 2, page 35), o ins ance, he no -
mal equa ion ha ˆ
θknsa is ies (see Rema k 2.3.6 in Chap e 2, page 36), he al e na i e cons uc ion
o ˆ
θknas he p ojec ion o he model pa ame e on o he subspace spanned by ˆ 1,...,ˆ kn(see Chap-
e 2, page 36), o he exp essions o he condi ional e o s o ˆ
θkn(see Theo em 2.3.14 in Chap e 2,
page 39). In his chap e , he FPCA es ima o is e isi ed in o de o imp o e i s beha iou in e ms
o condi ional mean squa e e o s by in oducing p esmoo hing echniques.
The choice o p esmoo hing me hods was decided on in ligh o Fa aldo-Roca and Gonz´alez-
Man eiga (1987) and C is ´obal-C is ´obal e al. (1987). These au ho s p oposed he applica ion o
he leas squa es p inciple on he pai s (Xi,ˆmn(Xi)) ins ead o (Xi, Yi), whe e ˆmnis a nonpa ame ic
ke nel– ype es ima o o he eg ession unc ion m(·). This al e a ion o he minimiza ion p oblem
p oduced e icien es ima o s ha educed he mean squa e e o o he classical leas squa es es ima-
o s o he linea eg ession model, excep in he compac suppo case. La e , Janssen e al. (2001)
showed ha he ine iciency p oblem in he compac suppo case could be ec i ied using bounda y
ke nels. A simila p ocedu e was de eloped by Ak i as (1996) o i polynomial eg ession models
o da a wi h incomple e obse a ions. Since hen, he p esmoo hing me hods ha e been success ully
applied in a eas such as model selec ion p ocedu es (see Ae s e al., 2010, who smoo hed he esponse
da a p io o model selec ion by Akaike’s In o ma ion C i e ion), and censo ed/ unca ed su i al da a
analysis (see Cao-Abad e al., 2005; J´acome and Iglesias-P´e ez, 2008; J´acome e al., 2008, who eplaced
censo ing indica o a iables by alues o a nonpa ame ic eg ession es ima o ).
In FDA, p esmoo hing p ocesses a e usually included as p elimina y s eps in such a way ha
he obse a ions a e eplaced by hei smoo hed app oxima ions. Fo ins ance, Hi chcock e al. (2006)
examined he e ec o his subs i u ion on es ima ing he dissimila i ies among elemen s in he da ase .
Ano he way o use p esmoo hing me hods has been conside ed by Zhang and Chen (2007) when
dealing wi h he eg ession model Yi( ) = X
iθ( ) + Vi( ) + ǫi( ), i= 1, . . . , n, whe e he co a ia e Xi
is mul idimensional and independen o (no e ha X
ideno es he anspose o he ec o Xi), he
p ocess Vi ep esen s he i h indi idual a ia ion, and Yiis he i h esponse p ocess. These au ho s
p oposed an es ima o o θbased on a local polynomial ke nel econs uc ion o i( ) = X
iθ( )+ Vi( ).
In his chap e , p esmoo hing is no used as a p ep ocessing ool, bu a he as a way o build a
new e icien FPCA– ype es ima o , ha educes he condi ional mean squa e e o s o he s anda d
FPCA one, ollowing he ideas o Fa aldo-Roca and Gonz´alez-Man eiga (1987) in he eal case. The
key he e is simila o he one ha mo i a es mul i a ia e idge eg ession: ci cum en he p oblem o
an ill–condi ioned co a iance ope a o by means o an a i icial pe u ba ion o i s eigen alues. To see
he use ulness o he p oposed app oach, he e is an example ha shows he ins abili y o he FPCA
es ima o when he eigen alues a e close o 0; i will be analysed in de ail in he simula ion s udy (see
Sec ion 3.5).
Example. Conside H=L2([0,1]) and he linea eg ession model Y=R1
0θ( )X( )d +ǫ(see
Rema k 2.3.2 in Chap e 2, page 33) whe e he explana o y cu es a e
X( ) = a1√2 sin(π ) + a2√2 cos(π ) + a3√2 sin(2π ) + a4√2 cos(2π ), ∈[0,1]
wi h al∼ U(−1/3l−1,1/3l−1) o all l∈ {1,...,4}, he model pa ame e is
θ( ) = 2√2 cos(2π ), ∈[0,1]
3.2. PRESMOOTHING VIA COVARIANCE STRUCTURE 53
and ǫ∼ N(0, σ2) wi h σ= 0.2pE(hX, θi2). The calcula ion o he s anda d FPCA es ima o in ol es
he eigenelemen s o he second momen ope a o o X(see (2.4) in Chap e 2, page 36) and, in his
example, i can be shown ha only he i s ou eigen alues a e di e en om ze o. To analyse
he e ec o null eigen alues, 200 samples o 100 obse a ions a e simula ed, and he mean squa e
p edic ion and es ima ion e o s (see (3.4), page 63) when kn∈ {1,...,8}eigenelemen s o he second
o de ope a o a e in ol ed in he FPCA es ima o a e compu ed. Figu e 3.1 p esen s he ob ained
esul s (see black solid line). The e o s when he eigen alues a e sligh ly pe u bed, adding α= 10−5
o hem (see g ey dashed line in Figu e 3.1) a e also calcula ed. I can be seen ha he p esence o
null eigen alues (kn>4), which ha dly a ec s p edic ion e o , conside ably inc eases he es ima ion
e o o he FPCA es ima o . Mo eo e , i seems ha pe u ba ion o eigen alues allows o keep
small es ima ion e o s, e en when null eigen alues a e in ol ed. Thus, all hese easonings led o
de eloping a FPCA– ype es ima o based on a p esmoo hing me hod ha a oids he incon enience o
ill–condi ioning.
1 2 3 4 5 6 7 8
0.0102 0.0104 0.0106 0.0108 0.0110 0.0112
1 2 3 4 5 6 7 8
0 5 10 15 20 25 30 35
Figu e 3.1: Example. Mean squa e p edic ion e o (le panel) and es ima ed mean squa e es ima ion
e o ( igh panel) o he s anda d FPCA es ima o (solid black line) and o he FPCA– ype es ima o
wi h pe u bed eigen alues (dashed g ey line).
In o de o achie e his goal, a new es ima o based on FPCA and p esmoo hing ia co a iance
s uc u e is p oposed in Sec ion 3.2. The sec ion con ains heo e ical esul s (consis ency and mean
squa e e o exp essions) ha allow o compa e i wi h he s anda d FPCA es ima o . Sec ion 3.3
p esen s an al e na i e app oach based on p esmoo hing ia esponse a iable (also including con-
sis ency and mean squa e e o exp essions), whe eas Sec ion 3.4 compiles some heu is ics on o he
p esmoo hing me hods. Sec ion 3.5 includes a simula ion s udy o analyse he beha iou o he di -
e en p oposals om a p ac ical poin o iew, and Sec ion 3.6 is de o ed o da a applica ions. Some
inal commen s can be ound in Sec ion 3.7, and an appendix compiles echnical lemmas and he p oo s
o he main esul s in Sec ion 3.8.
3.2 P esmoo hing ia co a iance s uc u e
3.2.1 De ini ion o es ima o
By Rema k 2.3.6 (see Chap e 2, page 36), he s anda d FPCA es ima o ˆ
θkncan be de i ed by sol ing
he equa ion
∆nx=hβ, Γnxi,∀x∈Im(Γn),
54 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
and unca ing he expansion o he solu ion o he i s kncomponen s. The key o he p oposed
me hod is o pe u b he p e ious no mal equa ion, and ind a unc ion β∈ H such ha
∆nx=hβ, (Γn+αn1H)xi,∀x∈Im(Γn),
whe e αnis a posi i e eal sequence e i ying ha αn→0 when n→ ∞, and 1His he iden i y
ope a o in H, i.e., 1Hx=x o all x∈ H. F om his equa ion, one can de i e he nex es ima o o θ
ˆ
θαn
kn=
kn
X
j=1
∆nˆ j
ˆ
λj+αn
ˆ j,(3.1)
which has he same s uc u e as he s anda d FPCA es ima o (2.4) (see Chap e 2, page 36), being
he main di e ence he p esence o pe u bed empi ical eigen alues in he denomina o .
Rema k 3.2.1.No e ha (3.1) can be seen as a unc ional e sion o he es ima o o he o dina y
mul i a ia e idge eg ession wi h penaliza ion e m equals o αn imes he usual no m o he model
pa ame e .
Al e na i e cons uc ion. In o de o build ˆ
θαn
kn, a p ocedu e simila o he one p oposed o ˆ
θkn
in Sec ion 2.3.2 (see Chap e 2, page 35) can be ollowed. The op imiza ion p oblem (2.5), ha is,
min
bEµn[( ˆmkn(X)−X b)2],wi h b= (hβ, e1i,...,hβ, ekni) ,∀β∈ H,
can be sol ed wi h
•ˆmkn(x) = Pn
i=1 Yiδ(x,Xi)/Pn
i=1 δ(x,Xi) is a nonpa ame ic es ima o o he eg ession unc ion
mkn(x) = E(Y|X=x), and
•µn(x) = Rx
−∞ ˆ
n( )d is a weigh ing unc ion whe e ˆ
nis a nonpa ame ic es ima o o he densi y
o X, namely , de ined as ˆ
n(x) = n−1Pn
i=1 δ(x,Xi),
using
δ(u,w) = hn−knK∗(h−1
n(u−w)),
whe e K∗(x) = Qkn
j=1 K(xj) o all x= (x1,...,xkn) ∈Rkn, wi h K:R→Ra symme ic posi i e
ke nel such ha RK(z)dz = 1, RzK(z)dz = 0 and Rz2K(z)dz =c(K)<∞, and hna s ic ly posi i e
sequence o bandwid hs (no e ha hnusually sa is ies hn→0 and nhn→ ∞). In his si ua ion, β0∈ H
is a solu ion o (2.5) i and only i b0= (hβ0, e1i,...,hβ0, ekni) sa is ies he associa ed kn–dimensional
no mal equa ion ∆n= (Γn+h2
nc(K)1kn×kn)b0, wi h ∆n=n−1Pn
i=1 XiYi,Γn=n−1Pn
i=1 XiX
i,
and 1kn×kn he kn×kn–iden i y ma ix (a kn×kn–ma ix wi h ones on he main diagonal and ze os
elsewhe e). This equa ion can be exp essed as
∆nel=
kn
X
j=1 hΓnel, ejihβ0, eji+h2
nc(K)hβ0, eli, o all l∈ {1,...,kn}.
By choosing el= ˆ l, i holds ha hβ0,ˆ li= (∆nˆ l)/(ˆ
λl+h2
nc(K)) and (3.1) can be de i ed wi h
αn=h2
nc(K).
3.2.2 Consis ency
In o de o s a e con e gence esul s o he p esmoo hed FPCA es ima o ˆ
θαn
kn, ecall ha k · kH′
deno es he no m o he dual space H′, and k·k∞deno es he uni o m no m o he space o Hilbe –
Schmid ope a o s de ined on H(see Sec ion 1.2.2, “ a) The space o Hilbe –Schmid ope a o s ” and
“ b) The dual space H′”, in Chap e 1, page 9).
The ollowing no a ion is also used in he nex heo em: m(·) = hθ, ·i and ˆmαn
kn(·) = hˆ
θαn
kn,·i.
Apa om he p e iously in oduced assump ions o p o ing he consis ency o he s anda d FPCA
es ima o (see u he de ails in Sec ion 2.3.2, “ c) Consis ency ”, in Chap e 2, page 38), and ex a
assump ion is equi ed in his case:
3.2. PRESMOOTHING VIA COVARIANCE STRUCTURE 55
(C.3.1) λ2
kn/αn→ ∞.
Theo em 3.2.2 (Fe a y e al., 2012a).Unde he assump ions o Theo em 2.3.11 (see Chap e 2,
page 39), i (C.3.1) is also sa is ied, i holds ha
kˆmαn
kn−mkH′→0a.s.
The p oo o p e ious heo em can be ound in he appendix o he chap e (see Sec ion 3.8.1, page 71).
3.2.3 Condi ional e o s
Nex , he condi ional mean squa e p edic ion e o and he condi ional mean squa e es ima ion e o
o he p esmoo hed FPCA es ima o ˆ
θαn
kna e p esen ed. In he heo e ical esul s below, Yn+1 =
hθ, Xn+1i+ǫn+1 is a new esponse and EXn(·) is he expec a ion condi ionally on Xn={X1,...,Xn}
o n∈N∗. In addi ion, he nex condi ions will be necessa y:
(C.3.2) ˆ
λkn/αn→ ∞ a.s.
(C.3.3) nαn→0.
On he o he hand, ecall he oa.s. no a ion in oduced in De ini ion 2.3.10 (see Chap e 2, page 38),
and ecall ha ˆ
Rknwas de ined in (2.7) (see Chap e 2, page 39) as
ˆ
Rkn=X
j>knhθ, ˆ jiˆ j.
Theo em 3.2.3 (Fe a y e al., 2012a).Fo he p esmoo hed FPCA es ima o (3.1), i (C.3.2) is
sa is ied, i holds ha
EXn+1 (Yn+1 −hˆ
θαn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=
−2αn
σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j
+ 2αnhXn+1,ˆ
RknihXn+1,ˆ
Tkni+α2
nhXn+1,ˆ
Tkni2
(1 + oa.s.(1)),
EXn(kθ−ˆ
θαn
knk2)−EXn(kθ−ˆ
θknk2) =
−2αn
σ2
n
kn
X
j=1
1
ˆ
λ2
j
+α2
nkˆ
Tknk2
(1 + oa.s.(1)),
whe e ˆ
Tkn=Pkn
j=1 ˆ
λ−1
jhθ, ˆ jiˆ j, and ˆ
Rknis de ined in (2.7) (see Chap e 2, page 39).
Co olla y 3.2.4 (Fe a y e al., 2012a).Unde he assump ions o Theo em 3.2.3, i (C.3.3) is
sa is ied, i holds ha
EXn+1 (Yn+1 −hˆ
θαn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=
−2αn
σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j
+ 2αnhXn+1,ˆ
RknihXn+1,ˆ
Tkni
(1 + oa.s.(1)),
EXn(kθ−ˆ
θαn
knk2)−EXn(kθ−ˆ
θknk2) =
−2αn
σ2
n
kn
X
j=1
1
ˆ
λ2
j
(1 + oa.s.(1)).
56 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
The p oo o he p e ious esul s can be ound in he appendix o his chap e (see Sec ion 3.8.3,
page 74, and Sec ion 3.8.4, page 75). They a e de i ed om he gene al Lemma 2.5.2 (see Chap e 2,
page 49).
Rema k 3.2.5.Co olla y 3.2.4 shows ha he bias e m ˆ
Rknplays a undamen al ole in he condi-
ional mean squa e e o o p edic ion. In ac , he p esmoo hed es ima o gi es be e o wo se
esul s han he s anda d FPCA es ima o ˆ
θkndepending on he o de o hXn+1,ˆ
Rkni. Thus i
hXn+1,ˆ
Rkni(αnhXn+1,ˆ
Tkni)−1=oa.s.(1), hen
EXn+1 (Yn+1 −hˆ
θαn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=
−2αn
σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j
+α2
nhXn+1,ˆ
Tkni2
(1 + oa.s.(1)).
Wi h (αn) = −2αnn−1σ2Pkn
j=1 ˆ
λ−2
jhXn+1,ˆ ji2+α2
nhXn+1,ˆ
Tkni2, ake αop ,1
n= a g minα (α). I
can be shown ha αop ,1
n=n−1σ2Pkn
j=1 ˆ
λ−2
jhXn+1,ˆ ji2hXn+1,ˆ
Tkni−2and, in his case
EXn+1 (Yn+1 −hˆ
θαop ,1
n
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=
−σ4
n2
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j
2
hXn+1,ˆ
Tkni−2
(1 + oa.s.(1)).
Hence, he e is a dec ease in he p edic ion e o using ˆ
θαop ,1
n
kn, and his educ ion is mo e impo an
when σ2is la ge and/o sample size nis small.
Rema k 3.2.6.Unde he assump ions o Co olla y 3.2.4, one has ha ˆ
θαn
knimp o es agains ˆ
θknin
e ms o he condi ional mean squa e e o o es ima ion, when abo e all, σ2is la ge and/o nis small.
Mo eo e , one can look o he alue o αn ha minimizes g(αn) = −2αnn−1σ2Pkn
j=1 ˆ
λ−2
j+α2
nkˆ
Tknk2.
This is αop ,2
n=n−1σ2(Pkn
j=1 ˆ
λ−2
j)kˆ
Tknk−2, o which one ge s
EXn(kθ−ˆ
θαop ,2
n
knk2)−EXn(kθ−ˆ
θknk2) =
−σ4
n2
kn
X
j=1
1
ˆ
λ2
j
2
kˆ
Tknk−2
(1 + oa.s.(1)).
P e ious ema ks ensu e second o de e iciency, ha is, ˆ
θαn
knpe o ms be e han he s anda d
FPCA es ima o in small samples, and he same as ˆ
θknin la ge ones. Second o de e iciency was
al eady achie ed o linea eg ession es ima o s based on p esmoo hing in he eal case (Fa aldo-Roca
and Gonz´alez-Man eiga, 1987; Janssen e al., 2001). P esmoo hing echniques also allowed his kind o
gain in e iciency in o he con ex s (see he p esmoo hed Nelson–Aalen es ima o e sus he classical
Nelson–Aalen es ima o in Cao-Abad e al., 2005).
3.3 P esmoo hing ia esponse a iable
3.3.1 De ini ion o es ima o
Ano he possible way o combine p esmoo hing echniques and FPCA es ima o s is o sol e he ol-
lowing no mal equa ion
∆hn
nx=hβ, Γnxi,∀x∈Im(Γn),
whe e ∆hn
n=n−1Pn
i=1 Xi⊗H′ˆmhn(Xi), wi h ˆmhn(·) he nonpa ame ic ke nel es ima o in oduced
by Fe a y and Vieu (2006b), and de ined in (2.10) (see Chap e 2, page 42) as
ˆmhn(x) = Pn
i=1 YiK(h−1
nd(x, Xi))
Pn
i=1 K(h−1
nd(x, Xi)) ,
3.3. PRESMOOTHING VIA RESPONSE VARIABLE 57
being now K(·) an asymme ic ke nel and hna s ic ly posi i e eal bandwid h such ha hn→0 when
n→+∞. This equa ion leads o he ollowing p esmoo hed es ima o o θ
ˆ
θhn
kn=
kn
X
j=1
∆hn
nˆ j
ˆ
λj
ˆ j.(3.2)
Al e na i e cons uc ion. As i was done o ˆ
θknand ˆ
θαn
kn, he es ima o ˆ
θhn
kncan also be ob ained
by means o he mul i a ia e op imiza ion p oblem (2.5) (see Chap e 2, page 36) gi en by
min
bEµn[( ˆmkn(X)−X b)2],wi h b= (hβ, e1i,...,hβ, ekni) ,∀β∈ H,
whe e
•ˆmkn(x) = Pn
i=1 Yiδ(x,Xi)/Pn
i=1 δ(x,Xi) is a nonpa ame ic es ima o o he eg ession unc ion
mkn(x) = E(Y|X=x), whe eas, in his case,
•µn(x) = n−1Pn
i=1 I{Xi∈(−∞,x]}is he weigh ing unc ion, being Iis he indica o unc ion.
Fu he mo e, i can be selec ed again
δ(u,w) = h−kn
nK∗(h−1
n(u−w)),
wi h K∗(x) = Qkn
j=1 K(xj), being K(·) a symme ic posi i e ke nel which e i ies ha RK(z)dz = 1,
RzK(z)dz = 0 and Rz2K(z)dz =c(K)<∞, and hna s ic ly posi i e sequence o bandwid hs
such ha hn→0 and nhn→ ∞. Consequen ly, β0∈ H is a solu ion o (2.5) i and only i b0=
(hβ0, e1i,...,hβ0, ekni) sa is ies he associa ed kn–dimensional no mal equa ion ∆hn
n=Γnb0, whe e
∆hn
n=n−1
n
X
i=1
Xiˆmkn(Xi) and Γn=n−1
n
X
i=1
XiX
i.
This ac ensu es ha
∆hn
nel=
kn
X
j=1 hΓnel, ejihβ0, eji, o all l∈ {1,...,kn}.
I el= ˆ l, one ge s hβ0,ˆ li= (∆hn
nˆ l)/ˆ
λl, being he associa ed es ima o he one p esen ed in (3.2).
k–NN app oach. Al hough all he heo e ical ad ances below in ol e ˆ
θhn
kn, which is based on he
ke nel es ima o ˆmhn, a di e en app oach in he simula ion s udies has been used (see Sec ion 3.5,
page 63): a k–nea es neighbou s es ima o (k–NN). The key idea o he k–NN p ocedu e is o eplace
he global bandwid h hnby a local bandwid h hk(x) which ensu es ha , o any x, a ixed numbe k
o obse a ions a e aken in o accoun o calcula e he alue o he es ima o , ha is,
ˆmk–NN(x) = Pn
i=1 YiK(hk(x)−1d(x, Xi))
Pn
i=1 K(hk(x)−1d(x, Xi)) ,
whe e hk(x) is a bandwid h such ha he e a e exac ly kelemen s in he sample which e i y ha
d(x, Xi)< hk(x). Hence, he nex es ima o o θcan be de ined
ˆ
θk–NN
kn=
kn
X
j=1
∆k–NN
nˆ j
ˆ
λj
ˆ j,(3.3)
wi h ∆k–NN
n=n−1Pn
i=1 Xi⊗H′ˆmk–NN(Xi).
Rema k 3.3.1.The es ima o ˆ
θhn
knha e been conside ed in o de o s a e con e gence and o he heo-
e ical esul s, due o he easiness o calcula ions when he bandwid h is global and independen o x.
Ne e heless, some p e ious simula ions e ealed ha ˆ
θk–NN
knp o ides mo e educed e o s han ˆ
θhn
kn,
as can be seen in he ollowing example. This is he eason why he esul s o ˆ
θk–NN
kna e included in
he simula ion s udies (see Sec ion 3.5, page 63) ins ead o he esul s ob ained by ˆ
θhn
kn.
58 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
1 2 3 4 5 6 7 8
0.011 0.012 0.013 0.014
1 2 3 4 5 6 7 8
0 10 20 30 40
1 2 3 4 5 6 7 8
0.011 0.012 0.013 0.014
1 2 3 4 5 6 7 8
0 10 20 30 40
Figu e 3.2: Example. Mean squa e p edic ion e o (le panels) and es ima ed mean squa e es ima ion
e o ( igh panels) o ˆ
θhn
kn( i s ow) and o ˆ
θk–NN
kn(second ow). Each g ey line co esponds o a
di e en alue o he smoo hing pa ame e (i.e., hn o ˆ
θhn
knand neig o ˆ
θk–NN
kn), whe eas solid black
line co esponds o cu e which gi es he minimum alue o each e o .
Example. Recall he example in oduced a he beginning o his chap e in which
X( ) = a1√2 sin(π ) + a2√2 cos(π ) + a3√2 sin(2π ) + a4√2 cos(2π ), ∈[0,1]
being al∼ U(−1/3l−1,1/3l−1) o all l∈ {1,...,4}, he model pa ame e is θ( ) = 2√2 cos(2π ) and
ǫ∼ N(0, σ2) wi h σ= 0.2pE(hX, θi2). F om he model Y=R1
0θ( )X( )d +ǫ, 200 samples o 100
obse a ions a e gene a ed, and ˆ
θhn
knand ˆ
θk–NN
kna e compu ed o kn∈ {1, . . . , 8},neig ∈ {2,...,21},
and hn alued in a g id in [0.75,2]. Fo bo h ˆmhnand ˆmk–NN, he quad a ic ke nel and he semi–me ic
d(·,·) based on he i s de i a i e a e selec ed. Fo each sample, he mean squa e p edic ion e o
and he mean squa e es ima ion e o (see (3.4), page 63) a e ob ained, and he mean o hese e o s
o e he 200 samples is plo ed in Figu e 3.2, whe e each cu e co esponds o a di e en alue o hn
( espec i ely, neig). Fu he mo e, he cu es whe e he minimum alue is eached a e highligh ed by
means o a solid black line. I seems ha bo h es ima o s gi e simila esul s in e ms o he es ima ion
e o , whe eas he p edic ion e o s o he k–NN e sion a e gene ally smalle han e o s o ˆ
θhn
kn. On
3.3. PRESMOOTHING VIA RESPONSE VARIABLE 59
he o he hand, bo h es ima o s a e su e ing he e ec on es ima ion e o o he null eigen alues when
kn>4, unlike he i s p esmoo hing p oposal ˆ
θαn
kn(see Figu e 3.1, page 53).
3.3.2 Consis ency
The ollowing heo em can be de i ed o m(·) = hθ, ·i and ˆmhn
kn(·) = hˆ
θhn
kn,·i using he same no a ion
as in he p e ious sec ion and he nex condi ions
(C.3.4) P(X∈ C) = 1 being Ca compac subse o Hsa is ying (C.2.21) (see
hypo heses o Theo em 2.4.11 in Chap e 2, page 45),
(C.3.5) λkn/hβ
n→+∞and λknpnφ(hn)/log n→+∞(being φ(·) de ined in
(C.2.23) in Chap e 2, page 45).
Theo em 3.3.2. Unde he assump ions o Theo em 2.3.11 (see Chap e 2, page 39) and Theo-
em 2.4.11 (see Chap e 2, page 45), i (C.3.4)–(C.3.5) a e also sa is ied, i holds ha
kˆmhn
kn−mkH′→0a.s.
The p oo o Theo em 3.3.2 can be ound in he appendix o he chap e (see Sec ion 3.8.5, page 76).
3.3.3 Condi ional e o s
The ollowing heo e ical ad ances in oduce he exp essions o he condi ional e o s o p edic ion and
es ima ion o he p esmoo hed FPCA es ima o ˆ
θhn
kn.
Theo em 3.3.3. Fo he p esmoo hed FPCA es ima o (3.2), i holds ha
EXn+1 (Yn+1 −hˆ
θhn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=
kn
X
j1=1
kn
X
j2=1
Uhn
n(ˆ j1)Uhn
n(ˆ j2)
ˆ
λj1ˆ
λj2hXn+1,ˆ j1ihXn+1,ˆ j2i
+σ2
kn
X
j1=1
kn
X
j2=1
Whn
n(ˆ j1,ˆ j2)
ˆ
λj1ˆ
λj2hXn+1,ˆ j1ihXn+1,ˆ j2i− σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
−2hXn+1,ˆ
RknihXn+1,
kn
X
j=1
Uhn
n(ˆ j)
ˆ
λj
ˆ ji,
EXn(kθ−ˆ
θhn
knk2)−EXn(kθ−ˆ
θknk2) =
kn
X
j=1
(Uhn
n(ˆ j))2
ˆ
λ2
j
+σ2
kn
X
j=1
Whn
n(ˆ j,ˆ j)
ˆ
λ2
j−σ2
n
kn
X
j=1
1
ˆ
λj
,
whe e
Uhn
n(x) = 1
n
n
X
i=1 hXi, xi n
X
l=1
wl,hn(Xi)(m(Xl)−m(Xi))!,∀x∈ H,
and
Whn
n(x, y) = 1
n2
n
X
i1=1
n
X
i2=1 hXi1, xihXi2, yi n
X
l=1
wl,hn(Xi1)wl,hn(Xi2)!,∀x, y ∈ H,
being wl,hn(x) = K(h−1
nd(x, Xl))/Pn
l′=1 K(h−1
nd(x, Xl′)) he weigh s o he ke nel es ima o ˆmhn, and
ˆ
Rknis de ined in (2.7) (see Chap e 2, page 39).
66 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
GCV op R(Y)op R(θ)
e o ou ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
R(Y) 0.02 0% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10% 4.25 4.32 4.13 4.08 4.08 3.88 3.86 4.15 3.97 3.96
20% 10.48 10.55 10.02 10.24 10.13 9.70 9.69 10.39 9.85 9.94
0.2 0% 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
10% 4.11 4.14 4.07 4.07 4.01 3.84 3.86 4.10 3.98 3.98
20% 10.58 10.63 10.19 10.24 10.21 9.70 9.63 10.40 9.89 9.88
0.5 0% 0.07 0.07 0.06 0.06 0.06 0.06 0.06 0.07 0.06 0.06
10% 4.32 4.36 4.11 4.14 4.10 3.86 3.90 4.22 4.01 4.04
20% 10.69 10.87 10.21 10.45 10.33 9.75 9.75 10.43 9.98 10.08
1 0% 0.25 0.25 0.25 0.24 0.24 0.22 0.22 0.25 0.24 0.24
10% 4.47 4.50 4.32 4.37 4.35 4.12 4.10 4.39 4.26 4.25
20% 10.91 11.01 10.44 10.55 10.51 9.99 9.90 10.71 10.20 10.22
2 0% 1.02 1.05 1.00 0.98 0.97 0.89 0.89 0.98 0.97 0.93
10% 5.12 5.16 4.94 4.96 4.96 4.71 4.68 5.05 4.87 4.87
20% 11.78 11.77 11.10 11.35 11.33 10.54 10.48 11.52 10.79 11.02
R(θ) 0.02 0% 0.17 0.01 0.01 5.25 1.16 0.01 4.45 0.01 0.00 0.41
10% 12.62 9.02 3.89 16.55 11.16 3.97 15.93 6.12 3.40 3.59
20% 19.39 11.95 3.98 116.45 18.32 4.17 20.11 7.92 3.49 4.37
0.2 0% 1.37 3.32 2.32 3.35 3.33 1.51 3.34 1.03 0.41 0.51
10% 12.40 7.82 3.91 18.34 11.99 4.15 24.74 5.96 3.45 3.60
20% 25.28 16.52 4.06 65.36 29.25 4.25 25.93 11.00 3.52 4.25
0.5 0% 5.00 3.41 3.37 3.39 3.40 3.33 3.37 3.33 1.55 0.73
10% 13.86 8.71 3.87 14.90 10.29 4.00 9.02 6.31 3.41 3.59
20% 24.47 16.72 4.02 118.13 22.87 4.24 22.34 9.68 3.55 4.58
1 0% 5.44 3.58 3.45 3.61 3.73 3.42 3.78 3.46 3.09 1.42
10% 13.72 8.57 3.96 15.89 10.89 4.01 7.99 6.62 3.45 3.67
20% 24.26 13.95 4.04 84.46 23.84 4.24 19.57 10.51 3.48 4.42
2 0% 6.58 4.40 3.61 4.94 4.60 3.58 4.02 3.84 3.36 3.14
10% 11.52 7.65 3.90 15.17 11.19 4.00 14.23 5.58 3.41 3.65
20% 23.37 14.74 4.10 70.42 25.79 4.15 15.03 11.03 3.52 4.92
Table 3.1: Case A. Median o R(Y) and R(θ) o sample size n= 25.
wi h all eigen alues o Γ s ic ly posi i e.
F om Table 3.5, Table 3.6, Table 3.7, and Table 3.8 (see pages 70–73), one sees ha ˆ
θαn
kngi es
be e es ima es o θ han ˆ
θkn, acco ding wi h he heo e ical esul s. Again, simula ions con i m he
e ec o he sample size and noise on he expec ed educ ion o es ima ion o θ(see Rema k 3.2.6,
page 56): mo e imp o emen when nis small and when he “noise le el” is la ge. As happens in Case
A, ˆ
θknand ˆ
θαn
kngi e simila esul s in e ms o R(Y). Compa ing he p esmoo hed es ima o ˆ
θαn
knwi h
ˆ
θP S, he beha iou is simila as a as p edic ion e o is conce ned, ˆ
θαn
kngi es smalle es ima ion e o s
han he penalized B–splines es ima o when nis small o he noise is la ge.
Wi h ega d o ˆ
θk–NN
kn he conclusions a e simila o hose de i ed om he Case A: ˆ
θk–NN
kndoes
no educe he p edic ion e o s ob ained by ˆ
θkn, and i s es ima ion e o inc eases ab up ly when he
GCV me hod is conside ed and he e a e ou lie s in he sample.
These simula ions sugges ha he p esmoo hing es ima o ˆ
θαn
knimp o es he s anda d FPCA linea
es ima o , and especially when he sample size is small, whe eas ˆ
θk–NN
kndo no signi ican ly educe he
condi ional e o s o ˆ
θkn. The choice o he pa ame e s o he p oposed p esmoo hed es ima o s is o
cou se a key poin . A gene al p ac ical guideline is o choose hese pa ame e s by c oss– alida ion
echniques. E en i he esul s abo e migh poin o a new way o selec ing hese pa ame e s ha
could gi e e en mo e imp o emen (see esul s o he “op imal” choice wi h espec R(θ)), CV gi es
good da a–d i en esul s, excep o ˆ
θk–NN
kn.
3.6. REAL DATA APPLICATION 67
GCV op R(Y)op R(θ)
e o ou ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
R(Y) 0.02 0% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10% 5.03 5.05 5.00 4.99 4.97 4.85 4.87 4.98 4.92 4.94
20% 10.13 10.15 9.92 10.06 9.92 9.71 9.75 10.00 9.81 9.86
0.2 0% 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
10% 5.05 5.09 5.03 5.01 5.00 4.86 4.90 5.03 4.94 4.95
20% 10.12 10.19 10.01 10.07 9.97 9.75 9.78 10.03 9.87 9.93
0.5 0% 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06
10% 5.13 5.14 5.07 5.11 5.09 4.95 4.98 5.12 5.03 5.06
20% 10.27 10.27 10.08 10.16 10.12 9.86 9.90 10.18 9.93 10.03
1 0% 0.25 0.25 0.25 0.24 0.24 0.23 0.24 0.25 0.24 0.24
10% 5.40 5.41 5.29 5.32 5.28 5.10 5.13 5.33 5.16 5.21
20% 10.47 10.48 10.32 10.33 10.35 10.01 10.05 10.40 10.13 10.27
2 0% 0.97 0.98 0.97 0.96 0.95 0.93 0.92 0.97 0.95 0.94
10% 6.18 6.22 6.17 6.14 6.10 5.88 5.94 6.13 6.07 6.03
20% 11.14 11.24 10.96 11.01 11.01 10.70 10.70 11.10 10.79 10.87
R(θ) 0.02 0% 0.16 0.00 0.00 8.11 1.69 0.00 7.61 0.00 0.00 0.38
10% 7.85 5.31 3.68 12.75 6.12 3.79 7.16 4.56 3.35 3.62
20% 13.95 10.32 3.95 40.66 11.93 4.13 9.98 6.83 3.44 4.23
0.2 0% 0.57 0.58 0.58 3.92 3.30 0.57 3.75 0.50 0.20 0.31
10% 8.99 6.12 3.85 13.76 8.27 3.94 7.06 5.40 3.40 3.81
20% 13.63 8.90 3.94 54.58 12.64 4.12 13.17 6.24 3.44 4.20
0.5 0% 4.17 3.38 3.34 3.37 3.35 3.30 3.34 2.87 1.03 0.58
10% 8.88 5.40 3.82 13.25 9.64 3.82 7.54 4.69 3.36 3.71
20% 12.63 8.76 3.95 34.59 12.49 4.11 13.49 6.55 3.41 4.28
1 0% 5.06 3.47 3.39 3.43 3.53 3.38 3.60 3.39 2.65 1.00
10% 10.02 6.54 3.89 9.87 9.07 3.91 9.37 5.11 3.40 3.86
20% 12.65 8.39 3.93 18.03 13.90 4.01 18.03 6.55 3.47 4.40
2 0% 5.45 3.77 3.50 3.74 4.27 3.54 3.89 3.63 3.32 2.67
10% 9.38 5.97 3.86 14.52 9.35 3.89 9.48 4.96 3.39 3.71
20% 14.49 8.16 3.94 37.83 12.62 4.06 12.09 6.64 3.45 4.20
Table 3.2: Case A. Median o R(Y) and R(θ) o sample size n= 50.
3.6 Real da a applica ion
In o de o demons a e he imp o ed pe o mance o he p oposed me hodology wi h espec o he
FPCA es ima o in applica ions, h ee unc ional da ase s wi h di e en size nwe e chosen: Canadian
wea he da a (n= 35), spec ome ic da a (n= 215), and a mosphe ic pollu ion da a (n= 1,000).
The aim is o see how he sample size a ec s o he imp o emen o ˆ
θα
knwi h espec o he s anda d
FPCA es ima o , and how he beha iou o ˆ
θk–NN
knis da a applica ions.
Since he a iables in ol ed a e no cen ed, he eg ession model wi h non–ze o in e cep gi en by
Rema k 2.3.1 (see Chap e 2, page 33) was used, and he nex s eps we e ollowed.
S ep 1. Calcula e he sample mean o cu es (X) and scala esponses (Y).
S ep 2. Spli he sample in o a lea ning sample {(Xi, Yi)}i∈ILS and a es sample {(Xi, Yi)}i∈IT S .
S ep 3. Use he cen ed lea ning sample {(Xi−X, Yi−Y)}i∈ILS o build ˆ
θ(ˆ
θP S,ˆ
θkn,ˆ
θαn
kno ˆ
θk–NN
kn),
and es ima e he in e cep e m by ˆ
θ0=Y−hˆ
θ, Xi.
S ep 4. Compu e he p edic ed esponses o he es sample, ˆ
Yi=ˆ
θ0+hˆ
θ, Xii,∀i∈IT S .
S ep 5. Ob ain R(Y) = 1
#IT S Pi∈IT S (Yi−ˆ
Yi)2, whe e #IT S deno es he es sample size.
To a oid he e ec o sample selec ion, his p ocedu e was i e a ed 100 imes, and he mean, he
median, and he s anda d de ia ion o R(Y) o e hese eplica ions we e calcula ed.
The gene al guidelines o he simula ions s udy o build he di e en es ima es o θwe e ollowed.
In pa icula , he in ol ed pa ame e s we e selec ed by GCV (because a p edic ion objec i e exis s in
he eal da a applica ion).
68 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
GCV op R(Y)op R(θ)
e o ou ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
R(Y) 0.02 0% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10% 4.99 4.99 4.96 4.98 4.94 4.88 4.91 4.97 4.93 4.95
20% 9.97 9.98 9.92 9.96 9.89 9.78 9.80 9.95 9.86 9.89
0.2 0% 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
10% 5.00 5.01 4.97 4.99 4.96 4.90 4.91 4.99 4.93 4.96
20% 9.97 10.00 9.94 9.98 9.91 9.78 9.82 9.95 9.85 9.90
0.5 0% 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06
10% 5.08 5.10 5.04 5.07 5.02 4.93 4.97 5.03 4.98 5.03
20% 10.05 10.07 10.00 10.02 9.96 9.82 9.86 10.00 9.90 9.95
1 0% 0.25 0.25 0.24 0.25 0.24 0.24 0.24 0.24 0.24 0.25
10% 5.24 5.24 5.21 5.24 5.18 5.12 5.14 5.23 5.18 5.19
20% 10.21 10.27 10.17 10.23 10.16 10.00 10.06 10.19 10.09 10.15
2 0% 0.97 0.97 0.97 0.97 0.96 0.94 0.95 0.96 0.96 0.97
10% 5.92 5.93 5.91 5.92 5.87 5.82 5.79 5.89 5.86 5.84
20% 10.96 10.98 10.85 10.92 10.85 10.68 10.73 10.92 10.80 10.82
R(θ) 0.02 0% 0.16 0.00 0.00 7.64 2.78 0.00 7.69 0.00 0.00 0.37
10% 7.03 4.74 3.73 5.66 6.84 3.75 6.32 4.27 3.37 3.69
20% 8.23 5.56 3.89 8.31 6.96 3.97 8.13 4.53 3.41 3.86
0.2 0% 0.40 0.23 0.27 5.19 1.94 0.19 4.36 0.23 0.10 0.50
10% 7.22 4.54 3.85 5.14 5.87 3.72 6.26 4.07 3.34 3.51
20% 8.86 5.84 3.87 8.41 7.94 3.79 7.04 4.71 3.38 3.81
0.5 0% 1.76 3.33 3.32 3.35 3.33 3.17 3.33 1.48 0.60 0.61
10% 7.63 4.93 3.65 6.40 5.57 3.66 6.05 4.19 3.36 3.62
20% 9.04 6.38 3.85 14.14 8.71 4.01 11.36 4.89 3.38 4.01
1 0% 4.94 3.41 3.37 3.39 3.40 3.34 3.39 3.34 1.52 0.82
10% 6.94 4.63 3.74 5.09 6.24 3.70 6.00 4.07 3.36 3.52
20% 8.29 5.67 3.82 9.70 7.11 3.76 10.29 4.60 3.37 3.77
2 0% 5.23 3.58 3.43 3.69 3.70 3.41 3.60 3.45 2.97 2.98
10% 6.99 4.81 3.72 6.35 5.93 3.68 6.13 4.14 3.36 3.56
20% 9.40 6.47 3.87 8.66 8.48 3.88 8.83 5.00 3.38 3.89
Table 3.3: Case A. Median o R(Y) and R(θ) o sample size n= 100.
3.6.1 Canadian wea he da a
Fi s ly, he Canadian wea he da a is conside ed ( o u he de ails, see Sec ion 1.1.2 in Chap e 1,
page 2). He e Yis he loga i hm o o al annual p ecipi a ion a each wea he s a ion, and Xis he
daily empe a u e cu e (see Ramsay and Sil e man, 2005, Chap e 15, o a Fou ie basis app oach
o his case). The o iginal sample was spli in o wo subsamples: a lea ning sample (25 s a ions) and
a es ing one (10 s a ions). Fo ˆ
θk–NN
kn, he quad a ic ke nel, and he semi–me ic based on he i s
de i a i es we e selec ed. Table 3.9 (see page 73) shows he mean, median, and s anda d de ia ion o
R(Y). No e ha bo h ˆ
θαn
knand ˆ
θk–NN
knyield o a educ ion o he e o o he s anda d FPCA es ima o
and he penalized B–splines es ima o .
3.6.2 Spec ome ic da a
Fo he second illus a ion, spec ome ic da a was selec ed (see Sec ion 1.1.2 in Chap e 1, page 2).
The da ase con ains 215 spec ome ic cu es ob ained om pieces o inely chopped mea , and a scala
alue co esponding o he a con en o each o hem. The second de i a i es o he spec ome ic
cu e o each mea piece we e aken as he unc ional a iable X, and i s a con en as he esponse
Y(Fe a y and Vieu, 2006b also chose he second de i a i es in o de o ge be e p edic i e esul s
in he unc ional nonpa ame ic eg ession con ex ). The sample was spli in o a lea ning sample o
160 pieces and a es sample o 55 pieces. Fo ˆ
θk–NN
kn, he quad a ic ke nel, and he semi–me ic based
on FPCA wi h q= 2 we e selec ed. Table 3.10 (see page 73) con ains he esul s o his example. In
his case, he bes esul s a e p oduced by ˆ
θαn
kn.
3.6. REAL DATA APPLICATION 69
GCV op R(Y)op R(θ)
e o ou ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
R(Y) 0.02 0% 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
10% 4.95 4.95 4.94 4.94 4.92 4.90 4.91 4.93 4.92 4.93
20% 9.88 9.90 9.86 9.90 9.85 9.79 9.80 9.86 9.82 9.84
0.2 0% 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01
10% 4.96 4.96 4.95 4.95 4.95 4.92 4.92 4.96 4.94 4.95
20% 9.88 9.89 9.86 9.88 9.84 9.79 9.81 9.86 9.84 9.86
0.5 0% 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06 0.06
10% 5.00 5.01 5.00 5.01 4.99 4.95 4.96 5.00 4.98 4.99
20% 9.96 9.98 9.95 9.98 9.93 9.86 9.89 9.94 9.90 9.92
1 0% 0.24 0.24 0.24 0.24 0.23 0.23 0.23 0.24 0.24 0.24
10% 5.24 5.25 5.22 5.24 5.20 5.14 5.18 5.22 5.18 5.22
20% 10.11 10.14 10.10 10.11 10.07 10.00 10.03 10.10 10.05 10.08
2 0% 0.96 0.96 0.96 0.96 0.95 0.94 0.94 0.96 0.96 0.95
10% 5.92 5.94 5.92 5.92 5.91 5.85 5.83 5.91 5.88 5.86
20% 10.90 10.92 10.84 10.90 10.83 10.70 10.77 10.86 10.77 10.82
R(θ) 0.02 0% 0.16 0.00 0.00 8.49 2.44 0.00 8.41 0.00 0.00 0.35
10% 6.14 3.92 3.55 4.12 4.44 3.55 4.87 3.66 3.33 3.43
20% 7.38 4.55 3.70 5.57 5.04 3.65 5.01 4.12 3.35 3.60
0.2 0% 0.27 0.12 0.13 6.87 1.27 0.11 4.59 0.12 0.05 0.36
10% 5.73 3.78 3.52 3.90 4.38 3.61 4.67 3.67 3.33 3.41
20% 6.91 4.47 3.64 5.10 5.92 3.70 6.44 4.00 3.34 3.64
0.5 0% 1.02 1.93 1.48 3.34 3.33 0.95 3.35 0.82 0.31 0.35
10% 5.63 4.00 3.56 4.18 4.36 3.54 4.75 3.69 3.33 3.43
20% 7.06 4.41 3.69 4.65 5.04 3.71 5.58 4.00 3.37 3.56
1 0% 4.14 3.36 3.34 3.35 3.35 3.32 3.35 2.76 1.17 0.82
10% 5.84 3.94 3.56 4.28 5.15 3.60 4.88 3.69 3.33 3.49
20% 7.80 4.87 3.74 6.22 6.95 3.72 8.32 3.99 3.35 3.65
2 0% 5.04 3.48 3.40 3.46 3.51 3.38 3.39 3.38 2.57 1.76
10% 6.24 4.31 3.58 4.69 4.52 3.61 4.68 3.76 3.33 3.47
20% 7.97 4.92 3.72 5.78 5.73 3.69 7.31 4.28 3.38 3.67
Table 3.4: Case A. Median o R(Y) and R(θ) o sample size n= 200.
3.6.3 A mosphe ic pollu ion da a
Finally, an en i onmen al example ha e been conside ed: he a mosphe ic pollu ion da a (see Sec-
ion 1.1.2 in Chap e 1, page 2). The da a he e co espond o hou ly a e aged NOxconcen a ions
measu ed in he neighbou hood o a powe s a ion o ENDESA, loca ed in As Pon es in he No hwes
o Spain, om 2007 o 2009. The aim is o o ecas NOxwi h hal an hou ho izon o allow he powe
plan s a o p eclude NOxconcen a ions eaching he limi s ixed by he cu en en i onmen al leg-
isla ion (see a unc ional ke nel and a linea au o eg essi e app oach o his p oblem wi h SO2le els
in Fe n´andez de Cas o e al., 2005). Each cu e Xwas buil wi h 240 consecu i e minu e–by–minu e
alues o hou ly a e aged NOxconcen a ion1, and he NOx alue hal an hou ahead was aken as
esponse Y. As in p e ious examples, di e en pai s o lea ning/ es ing samples we e conside ed,
consis ed o 750 and 250 obse a ions, espec i ely. Fo ˆ
θk–NN
kn, he quad a ic ke nel, and he semi–
me ic based on he L2–no m we e selec ed. Table 3.11 (see page 73) compiles he main esul s. Fo
his da ase , he smalles alues o R(Y) co espond o ˆ
θP S. Among he FPCA– ype es ima o s, ˆ
θαn
kn
imp o es sligh ly he esul s o he s anda d es ima o , whe eas ˆ
θk–NN
kngi es he la ges e o s.
Rema k 3.6.1.In Co olla y 3.2.4 (see page 55), he second o de e iciency o he es ima o ˆ
θα
knin
e ms o he condi ional es ima ion e o was s a ed. The exis ence o an in e se ela ion be ween
he imp o emen o ˆ
θα
knwi h espec o ˆ
θkn, and he sample size nwas also indica ed. This ac is
e lec ed in he eal da a applica ions: he gain o he p esmoo hed es ima o dec eases om small o
la ge sample size (see Table 3.9, Table 3.10 and Table 3.11, page 73).
1Each alue o his hou ly a e aged NOxconcen a ion is compu ed as he a e age o 60 minu e–by–minu e NOx
alues, which co espond o he alues a he p e ious 59 ime poin s and a he cu en ime poin .
70 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
GCV op R(Y)op R(θ)
e o ou ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
R(Y) 0.02 0% 0.00 0.00 0.00 0.05 0.00 0.00 0.05 0.00 0.00 0.07
10% 4.26 4.31 4.26 4.32 4.04 3.84 3.80 4.17 4.03 4.09
20% 10.97 11.07 10.70 11.04 10.13 9.59 9.51 10.36 9.94 10.01
0.2 0% 0.12 0.12 0.12 0.16 0.10 0.10 0.15 0.11 0.11 0.18
10% 4.38 4.37 4.34 4.42 4.09 3.91 3.91 4.22 4.11 4.14
20% 10.93 10.94 10.77 11.05 10.05 9.56 9.54 10.38 10.20 10.25
0.5 0% 0.71 0.72 0.71 0.78 0.65 0.62 0.69 0.70 0.68 0.76
10% 5.26 5.22 5.10 5.11 4.68 4.52 4.51 4.93 4.74 4.83
20% 11.42 11.49 11.21 11.40 10.72 9.96 9.93 11.00 10.67 10.60
1 0% 2.75 2.82 2.71 2.68 2.52 2.35 2.37 2.62 2.52 2.56
10% 7.31 7.30 6.93 6.92 6.44 6.04 6.09 6.66 6.38 6.51
20% 13.96 14.18 13.59 13.44 12.70 12.06 11.94 13.20 12.71 12.69
2 0% 11.12 11.23 10.73 10.89 10.45 9.93 9.60 10.70 10.26 10.30
10% 16.60 16.85 16.04 16.14 15.19 13.69 13.86 15.76 14.83 14.97
20% 22.70 23.19 21.85 21.75 20.37 19.28 18.79 21.02 20.31 20.50
R(θ) 0.02 0% 0.06 0.47 0.45 1.39 0.43 0.43 1.45 0.42 0.40 1.31
10% 10.27 6.82 4.01 14.75 9.58 3.12 12.59 3.73 1.67 1.86
20% 17.43 11.98 6.29 128.15 18.73 4.11 24.99 6.96 1.79 2.42
0.2 0% 0.72 1.40 1.36 1.68 1.41 1.36 1.80 1.16 0.96 1.38
10% 10.07 6.56 4.07 20.24 7.47 3.22 13.20 3.78 1.70 1.86
20% 19.90 17.26 6.55 163.58 25.91 7.01 28.16 5.98 2.02 2.42
0.5 0% 1.45 1.95 2.03 2.47 2.65 1.89 2.88 1.53 1.21 1.42
10% 9.23 6.87 4.08 16.81 12.16 3.56 15.62 3.60 1.65 1.93
20% 20.72 14.72 6.29 142.17 19.25 5.14 25.76 7.75 2.07 2.50
1 0% 5.50 4.08 3.38 5.86 6.35 2.66 11.10 2.39 1.47 1.69
10% 13.43 10.46 5.06 36.89 11.20 3.50 21.48 4.95 1.67 1.90
20% 24.08 14.42 6.56 83.28 22.13 4.99 17.68 8.15 2.17 3.00
2 0% 14.97 9.47 4.48 28.14 19.50 5.46 22.56 5.52 1.84 2.34
10% 33.05 19.14 6.61 60.22 44.83 6.56 32.91 8.86 2.18 2.52
20% 43.98 27.31 6.53 141.35 36.98 6.54 41.44 13.36 2.50 2.84
Table 3.5: Case B. Median o R(Y) and R(θ) o sample size n= 25.
3.7 Final conclusions
Th oughou his chap e , new FPCA– ype es ima o s o he linea model pa ame e θbased on
di e en p esmoo hing echniques ha e been in oduced.
Fi s o all, ˆ
θαn
knhas been p oposed, which can be seen as an ex ension o he o dina y mul i a ia e
idge eg ession es ima o o gene al Hilbe spaces: one sligh ly pe u bs he eigen alues o he second
momen ope a o in o de o a oid ill–condi ioned p oblems. I has been shown ha he p esmoo hed
es ima o p ese es he consis ency p ope ies o he s anda d FPCA es ima o , and exp essions o
condi ional mean squa e e o s o p edic ion and es ima ion ha e been ob ained. A ema k highligh s
he e ec o he bias e m in he condi ional p edic ion e o : one can only ob ain clea e iciency
when he bias is negligible. As a as he condi ional es ima ion e o is conce ned, one is able o
ge imp o emen o e he FPCA es ima e, especially i he model noise is la ge and/o he sample
size is small. Then, o he p esmoo hing app oaches as ˆ
θhn
kn, o he es ima o s based on p esmoo hed
FPCA analysis by Pezzulli and Sil e man (1993) and Sil e man (1996), ha e been in oduced. The
consis ency o ˆ
θhn
kn, and he condi ional e o exp essions o all o hem, ha e been ob ained.
The e ec i eness o he p esmoo hed es ima o s ˆ
θαn
knand ˆ
θk–NN
kn ela i e o he s anda d FPCA
es ima o and he penalized B–splines es ima o ha e been es ed by means o simula ion s udies and
da a applica ions. In e ms o condi ional es ima ion e o , ˆ
θαn
kngi es be e esul s han he FPCA
es ima o , and i is a se ious i al o he spline es ima o when he sample size is small o he noise
le el is la ge. Ne e heless, second o de e iciency only gene a es a clea imp o emen o small sample
size, whe eas o nla ge enough ˆ
θknand ˆ
θα
knha e simila beha iou . On he o he hand, he e iciency
o ˆ
θk–NN
kncould no be s a ed. Fu he mo e, om a p ac ical poin o iew, i has been seen ha his
3.8. APPENDIX CHAPTER 3 71
GCV op R(Y)op R(θ)
e o ou ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
R(Y) 0.02 0% 0.00 0.00 0.00 0.02 0.00 0.00 0.02 0.00 0.00 0.03
10% 5.17 5.21 5.15 5.26 4.98 4.88 4.92 5.06 5.01 5.05
20% 10.22 10.28 10.26 10.47 9.93 9.67 9.71 10.06 9.87 9.94
0.2 0% 0.11 0.11 0.11 0.13 0.10 0.10 0.12 0.11 0.11 0.14
10% 5.32 5.28 5.27 5.31 5.10 4.98 4.99 5.20 5.10 5.13
20% 10.35 10.33 10.23 10.57 10.05 9.80 9.81 10.16 10.07 10.08
0.5 0% 0.66 0.69 0.67 0.67 0.64 0.62 0.63 0.65 0.65 0.67
10% 5.83 5.89 5.83 5.95 5.65 5.56 5.61 5.74 5.74 5.78
20% 11.07 11.29 10.96 11.25 10.75 10.41 10.40 10.86 10.63 10.68
1 0% 2.69 2.63 2.59 2.66 2.56 2.45 2.49 2.58 2.55 2.56
10% 7.79 7.79 7.67 7.67 7.41 7.23 7.13 7.59 7.48 7.36
20% 13.61 13.77 13.51 13.73 13.02 12.63 12.65 13.30 13.00 13.35
2 0% 11.09 10.99 10.70 10.76 10.27 10.07 10.30 10.51 10.41 10.54
10% 16.32 16.34 15.71 15.76 15.66 14.98 14.93 15.87 15.33 15.31
20% 21.57 21.54 21.08 21.46 20.29 19.67 19.87 20.65 20.08 20.40
R(θ) 0.02 0% 0.04 0.33 0.33 1.24 0.32 0.31 1.28 0.31 0.29 1.14
10% 5.14 4.36 3.34 20.58 5.34 3.00 7.43 2.53 1.52 1.88
20% 8.89 7.39 3.84 71.15 14.95 3.95 12.54 4.05 1.76 2.50
0.2 0% 0.64 1.28 1.26 1.37 1.23 1.18 1.39 1.03 0.76 1.17
10% 5.41 4.11 3.30 11.11 6.77 2.58 7.57 2.74 1.45 1.64
20% 9.38 6.04 3.63 55.30 7.16 3.53 17.77 3.73 1.69 1.99
0.5 0% 0.97 1.64 1.65 1.95 1.60 1.43 2.10 1.35 1.16 1.28
10% 6.34 4.85 3.70 15.70 5.35 2.44 6.36 2.81 1.46 1.77
20% 10.21 10.26 5.51 75.29 9.80 3.55 14.70 4.34 1.75 2.45
1 0% 2.68 3.26 2.92 4.18 3.51 2.20 4.06 1.94 1.37 1.49
10% 6.60 5.48 3.38 23.25 6.73 3.04 11.19 3.01 1.61 1.76
20% 12.60 8.40 4.17 74.19 15.33 4.21 12.97 4.21 1.68 2.52
2 0% 9.65 6.28 3.64 14.97 9.41 3.96 16.15 3.69 1.63 1.97
10% 9.11 7.06 4.20 23.10 12.08 3.62 26.68 4.68 1.70 2.32
20% 18.37 12.81 6.36 82.13 17.30 5.38 21.93 7.31 2.18 3.06
Table 3.6: Case B. Median o R(Y) and R(θ) o sample size n= 50.
es ima o educes he condi ional e o o ˆ
θknin some cases, whe eas i gi es la ge e o s han he
FPCA es ima o in o he ones.
3.8 Appendix Chap e 3
In his appendix, he p oo s o all he heo ems and co olla ies (in o de o appea ance) in oduced
h oughou he chap e ha e been compiled, join ly wi h he echnical lemmas equi ed o p o e hem.
3.8.1 P oo o Theo em 3.2.2
This p oo is simila o ha o Theo em 3.2 in Ca do e al. (1999).
Fi s , ake mkn= ∆Πkn(ΠknΓΠkn)−1, whe e Πknis he o hogonal p ojec ion on o he space
spanned by he i s kneigen unc ions o Γ. I is clea ha
km−ˆmαn
knkH′≤ km−mknkH′+kmkn−ˆmαn
knkH′.
Ca do e al. (1999) showed ha km−mknkH′→0, so one jus needs o show ha kmkn−ˆmαn
knkH′→0.
Le En={λkn/2<ˆ
λkn<3λkn/2}. In En, Lemma 3.8.1 (see page 74) ensu es ha
kmkn−ˆmαn
knkH′≤δnk∆kH′kΓ−Γnk∞+ 2k∆−∆nkH′
λkn
+ 2αnk∆kH′
λ2
kn
,
72 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
GCV op R(Y)op R(θ)
e o ou ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
R(Y) 0.02 0% 0.00 0.00 0.00 0.02 0.00 0.00 0.01 0.00 0.00 0.02
10% 5.01 5.05 5.00 5.02 4.94 4.89 4.89 4.98 4.96 4.96
20% 9.98 9.99 9.93 10.09 9.82 9.74 9.75 9.93 9.86 9.87
0.2 0% 0.11 0.11 0.11 0.12 0.11 0.11 0.12 0.11 0.11 0.13
10% 5.15 5.17 5.14 5.19 5.06 5.00 5.00 5.09 5.09 5.10
20% 10.16 10.20 10.14 10.31 10.02 9.91 9.96 10.12 10.06 10.07
0.5 0% 0.66 0.68 0.67 0.69 0.66 0.65 0.66 0.66 0.67 0.68
10% 5.69 5.71 5.65 5.67 5.61 5.52 5.55 5.65 5.62 5.62
20% 10.68 10.68 10.64 10.74 10.51 10.36 10.38 10.58 10.49 10.49
1 0% 2.57 2.59 2.57 2.58 2.52 2.48 2.52 2.57 2.52 2.55
10% 7.66 7.70 7.67 7.66 7.57 7.45 7.42 7.63 7.56 7.55
20% 13.00 13.01 12.86 13.07 12.55 12.44 12.56 12.75 12.67 12.72
2 0% 10.70 10.72 10.54 10.57 10.37 10.26 10.26 10.56 10.42 10.51
10% 15.68 15.80 15.51 15.41 15.28 14.88 14.99 15.51 15.14 15.37
20% 20.52 20.90 20.60 20.57 20.21 19.89 19.83 20.36 20.19 20.10
R(θ) 0.02 0% 0.02 0.25 0.24 1.10 0.24 0.24 1.12 0.23 0.23 0.96
10% 2.40 2.82 2.35 5.86 3.35 2.03 4.21 1.77 1.31 1.51
20% 4.47 3.90 2.93 30.94 5.38 2.86 6.85 2.72 1.44 1.77
0.2 0% 0.60 1.23 1.13 1.24 1.15 1.01 1.22 0.86 0.65 1.01
10% 2.43 2.63 2.43 6.47 3.45 2.11 4.03 1.85 1.31 1.43
20% 4.78 4.50 3.06 32.77 4.96 2.81 4.98 2.43 1.57 1.87
0.5 0% 0.79 1.46 1.48 1.52 1.43 1.31 1.48 1.23 1.04 1.14
10% 3.07 3.10 2.74 6.14 3.91 2.25 4.84 2.01 1.32 1.40
20% 3.68 3.10 2.56 40.80 4.56 2.57 7.37 2.24 1.39 1.73
1 0% 1.51 1.80 1.83 2.69 3.15 1.94 2.62 1.56 1.25 1.37
10% 3.96 3.42 2.68 5.35 4.65 2.42 5.50 2.36 1.53 1.62
20% 3.86 3.97 2.64 30.56 5.61 3.12 8.28 2.50 1.58 1.89
2 0% 4.42 3.60 2.77 13.52 5.58 2.83 6.18 2.39 1.44 1.77
10% 6.06 5.09 3.21 15.05 9.50 3.35 9.36 3.09 1.53 1.90
20% 9.16 7.64 4.09 30.66 10.02 3.53 9.57 4.17 1.68 2.22
Table 3.7: Case B. Median o R(Y) and R(θ) o sample size n= 100.
wi h δn= 2/λ2
kn+ 6 Pkn
j=1 aj/λkn. The e o e
P(kmkn−ˆmαn
knkH′> η)≤PkΓ−Γnk∞>η
3δnk∆kH′+Pk∆−∆nkH′>λknη
6
+I{αn>ηλ2
kn/(6k∆kH′)}+P(En).
(3.5)
I can be shown ha
P(En)≤P(kΓ−Γnk∞> λkn/2) ≤2 exp (−Cnλ2
kn),(3.6)
whe e Cis a posi i e cons an independen o n, and he las inequali y is de i ed om Lemma 5.3 in
Ca do e al. (1999). Fu he mo e,
PkΓ−Γnk∞>η
3δnk∆kH′≤2 exp −Aηn
δ2
n,(3.7)
Pk∆−∆nkH′>λknη
6≤2 exp (−Bηnλ2
kn) (3.8)
whe e Aηand Bηa e posi i e cons an s independen o n. These wo inequali ies a e ob ained using
Lemma 5.3 in Ca do e al. (1999), and Lemma 2.5.1 (see Chap e 2, page 47). Using (3.6), (3.7), and
(3.8) in (3.5), one ge s
P(kmkn−ˆmαn
knkH′> η)≤2 exp −Aηn
δ2
n+ 2 exp (−Bηnλ2
kn) + I{αn>ηλ2
kn/(6k∆kH′)}
+ 2 exp (−Cnλ2
kn).
3.8. APPENDIX CHAPTER 3 73
GCV op R(Y)op R(θ)
e o ou ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
knˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
R(Y) 0.02 0% 0.00 0.00 0.00 0.01 0.00 0.00 0.01 0.00 0.00 0.01
10% 4.95 4.97 4.94 4.97 4.91 4.89 4.89 4.94 4.93 4.93
20% 9.91 9.91 9.90 9.95 9.83 9.79 9.79 9.87 9.85 9.85
0.2 0% 0.11 0.11 0.11 0.12 0.11 0.11 0.11 0.11 0.11 0.12
10% 5.08 5.08 5.07 5.09 5.03 5.00 5.01 5.06 5.04 5.05
20% 10.00 10.02 9.99 10.04 9.92 9.86 9.86 9.96 9.95 9.94
0.5 0% 0.66 0.66 0.66 0.67 0.65 0.65 0.65 0.66 0.65 0.66
10% 5.68 5.71 5.68 5.68 5.62 5.57 5.59 5.65 5.63 5.61
20% 10.63 10.63 10.60 10.66 10.51 10.42 10.47 10.56 10.51 10.54
1 0% 2.61 2.64 2.63 2.62 2.59 2.58 2.57 2.61 2.60 2.60
20% 7.65 7.65 7.62 7.66 7.53 7.48 7.53 7.56 7.54 7.60
10% 12.44 12.47 12.42 12.45 12.35 12.28 12.27 12.40 12.37 12.35
2 0% 10.53 10.50 10.54 10.54 10.44 10.39 10.39 10.49 10.43 10.46
10% 15.41 15.47 15.37 15.43 15.24 15.18 15.19 15.28 15.32 15.34
20% 20.59 20.57 20.59 20.50 20.23 20.08 20.05 20.32 20.23 20.23
R(θ) 0.02 0% 0.01 0.22 0.21 0.95 0.21 0.21 1.00 0.21 0.21 0.88
10% 1.85 1.98 2.04 4.09 2.17 1.71 2.57 1.49 1.22 1.31
20% 1.91 2.21 2.17 11.11 3.04 1.93 5.03 1.81 1.29 1.51
0.2 0% 0.53 1.02 0.79 1.15 0.88 0.75 1.15 0.60 0.49 0.91
10% 1.51 2.04 1.92 2.99 2.17 1.65 2.39 1.56 1.21 1.36
20% 2.43 2.55 2.12 15.12 3.43 1.93 3.61 1.86 1.28 1.40
0.5 0% 0.68 1.34 1.32 1.36 1.28 1.22 1.28 1.15 0.89 1.10
10% 2.08 2.23 2.31 3.73 2.43 1.75 2.56 1.57 1.24 1.33
20% 2.46 2.46 2.26 11.24 2.99 1.94 4.12 1.71 1.26 1.48
1 0% 1.23 1.65 1.59 1.76 1.69 1.46 2.03 1.33 1.14 1.22
10% 2.36 2.23 2.39 4.88 2.73 1.86 3.61 1.63 1.27 1.35
20% 2.86 2.68 2.58 8.20 4.05 2.22 4.32 2.09 1.38 1.47
2 0% 2.43 2.44 2.32 3.71 3.57 2.19 3.78 1.81 1.29 1.46
10% 3.75 3.43 2.80 6.60 4.64 2.84 6.45 2.19 1.49 1.66
20% 4.85 3.85 3.21 17.51 5.72 2.48 5.96 2.47 1.44 1.76
Table 3.8: Case B. Median o R(Y) and R(θ) o sample size n= 200.
GCV
e o ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
mean(R(Y)) 0.03698 0.03566 0.03293 0.02601
median(R(Y)) 0.03626 0.03466 0.03230 0.02585
sd(R(Y)) 0.01772 0.01579 0.01436 0.01483
Table 3.9: Canadian wea he da a. Mean, median and s anda d de ia ion o R(Y).
GCV
e o ˆ
θP S ˆ
θknˆ
θαn
kn
ˆ
θk–NN
kn
mean(R(Y)) 6.656 6.926 6.604 8.072
median(R(Y)) 6.313 6.524 6.088 7.598
sd(R(Y)) 1.935 2.153 1.939 2.157
Table 3.10: Spec ome ic da a. Mean, median and s anda d de ia ion o R(Y).
GCV
e o ˆ
θP S ˆ
θknˆ
θαn
knˆ
θk–NN
kn
mean(R(Y)) 2.865 3.238 3.207 19.664
median(R(Y)) 2.780 3.074 3.062 19.202
sd(R(Y)) 0.966 0.866 0.833 5.221
Table 3.11: A mosphe ic pollu ion da a. Mean, median and s anda d de ia ion o R(Y).
74 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
Ca do e al. (1999) showed exp (−Aηn/δ2
n), exp (−Bηnλ2
kn) and exp (−Cnλ2
kn) a e gene al e ms
o con e gen se ies unde hypo hesis (C.2.8). On he o he hand, unde (C.3.1), he e is an n0
such ha αn< ηλ2
kn/(6k∆kH′) o all n > n0. The e o e, Pn∈N∗P(kmkn−ˆmαn
knkH′> η)<∞, and
Bo el–Can elli Lemma gi es kmkn−ˆmαn
knkH′→0a.s.
3.8.2 Fo mula ion and p oo o Lemma 3.8.1
Lemma 3.8.1. Wi h γn=k∆kH′{1/(λknˆ
λkn) + 2(1/λkn+ 1/ˆ
λkn)Pkn
j=1 aj},
kmkn−ˆmαn
knkH′≤γnkΓ−Γnk∞+k∆−∆nkH′
ˆ
λkn
+αnk∆kH′
λknˆ
λkn
.
P oo . Take ˜
Γkn=Pkn
j=1 λjˆ j⊗Hˆ j. The i s s ep is o w i e
kmkn−ˆmαn
knkH′≤ k∆ΠknkH′k(ΠknΓΠkn)−1−˜
Γ−1
knk∞+k∆ΠknkH′k˜
Γ−1
kn−(ˆ
Πkn(Γn+αnI)ˆ
Πkn)−1k∞
+k∆Πkn−∆nˆ
ΠknkH′k(ˆ
Πkn(Γn+αnI)ˆ
Πkn)−1k∞.
(3.9)
F om (11) and (14) in he p oo o Lemma 5.1 in Ca do e al. (1999),
k∆ΠknkH′k(ΠknΓΠkn)−1−˜
Γ−1
knk∞≤2k∆kH′
λknkΓ−Γnk∞
kn
X
j=1
aj,(3.10)
k∆Πkn−∆nˆ
ΠknkH′≤2k∆kH′kΓ−Γnk∞
kn
X
j=1
aj+k∆−∆nkH′.(3.11)
Mo eo e , wi h a gumen s as in he p oo o Lemma 5.1 in Ca do e al. (1999), one has
k∆ΠknkH′k˜
Γ−1
kn−(ˆ
Πkn(Γn+αnI)ˆ
Πkn)−1k∞≤k∆kH′
λknˆ
λkn
(kΓ−Γnk∞+αn),(3.12)
k(ˆ
Πkn(Γn+αnI)ˆ
Πkn)−1k∞≤1
ˆ
λkn
.(3.13)
Hence, using (3.10), (3.11), (3.12), and (3.13) in (3.9), one ge s
kmkn−ˆmαn
knkH′≤ k∆kH′
1
λknˆ
λkn
+ 2 1
λkn
+1
ˆ
λkn!kn
X
j=1
aj
kΓ−Γnk∞+k∆−∆nkH′
ˆ
λkn
+αnk∆kH′
λknˆ
λkn
.
3.8.3 P oo o Theo em 3.2.3
Conside Lemma 2.5.2 (see Chap e 2, page 49), wi h γj= (ˆ
λj+αn)−1and wj= ˆ j. Then,
R(γ,w)
kn=θ−
kn
X
j=1
ˆ
λj
ˆ
λj+αnhˆ j, θiˆ j=ˆ
Rkn+
kn
X
j=1
αn
ˆ
λj+αnhˆ j, θiˆ j,
3.8. APPENDIX CHAPTER 3 75
wi h ˆ
Rknde ined in (2.7) (see Chap e 2, page 39). One hen ob ains o he condi ional p edic ion
e o
EXn+1 (Yn+1 −hˆ
θαn
kn, Xn+1i)2=σ2+σ2
n
kn
X
j=1
ˆ
λj
(ˆ
λj+αn)2hXn+1,ˆ ji2
+hXn+1,ˆ
Rkn+
kn
X
j=1
αn
ˆ
λj+αnhˆ j, θiˆ ji2.
(3.14)
Some calcula ions and (C.3.2) allow o ob ain
ˆ
λj
(ˆ
λj+αn)2=1
ˆ
λj−2αn
ˆ
λ2
j
+α2
n
ˆ
λj(ˆ
λj+αn)2 3 + 2αn
ˆ
λj!=1
ˆ
λj−2αn
ˆ
λ2
j
(1 + oa.s.(1)),(3.15)
αn
ˆ
λj+αn
=αn
ˆ
λj 1−αn
ˆ
λj+αn!=αn
ˆ
λj
(1 + oa.s.(1)).(3.16)
Using (3.15) and (3.16) in (3.14), one ge s
EXn+1 (Yn+1 −hˆ
θαn
kn, Xn+1i)2=σ2+σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj−2αn
σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j
(1 + oa.s.(1))
+hXn+1,ˆ
Rkni2+ 2αnhXn+1,ˆ
RknihXn+1,ˆ
Tkni(1 + oa.s.(1)) + α2
nhXn+1,ˆ
Tkni2(1 + oa.s.(1)),
whe e ˆ
Tkn=Pkn
j=1 ˆ
λ−1
jhθ, ˆ jiˆ j. Compa ing his exp ession wi h he condi ional p edic ion e o o
ˆ
θkngi en in Theo em 2.3.14 (see Chap e 2, page 39), one ge s
EXn+1 (Yn+1 −hˆ
θαn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2=−2αn
σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j
(1 + oa.s.(1))
+ 2αnhXn+1,ˆ
RknihXn+1,ˆ
Tkni(1 + oa.s.(1)) + α2
nhXn+1,ˆ
Tkni2(1 + oa.s.(1)).
On he o he hand, o he condi ional es ima ion e o Lemma 2.5.2 (see Chap e 2, page 49)
implies
EXn(kθ−ˆ
θαn
knk2) = σ2
n
kn
X
j=1
ˆ
λj
(ˆ
λj+αn)2+kˆ
Rknk2+
kn
X
j=1
αn
ˆ
λj+αnhˆ j, θiˆ j
2
.(3.17)
Using (3.15) and (3.16) in (3.17), one has
EXn(kθ−ˆ
θαn
knk2) = σ2
n
kn
X
j=1
1
ˆ
λj−2αn
σ2
n
kn
X
j=1
1
ˆ
λ2
j
(1 + oa.s.(1)) + kˆ
Rknk2+α2
nkˆ
Tknk2(1 + oa.s.(1)),
wi h ˆ
Tknde ined as be o e. Bea ing in mind Theo em 2.3.14 (see Chap e 2, page 39),
EXn(kθ−ˆ
θαn
knk2)−EXn(kθ−ˆ
θknk2) = −2αn
σ2
n
kn
X
j=1
1
ˆ
λ2
j
(1 + oa.s.(1)) + α2
nkˆ
Tknk2(1 + oa.s.(1)).
3.8.4 P oo o Co olla y 3.2.4
No e ha
hXn+1,ˆ
Tkni2=
kn
X
j=1
hθ, ˆ ji
ˆ
λjhXn+1,ˆ ji
2
≤
kn
X
j=1 hθ, ˆ ji2
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j
≤ kθk2
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j
,
82 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
As a esul , using (3.23), (3.24), (3.25), (3.26), and (3.27) in (3.28), one has
EXn(kθ−ˆ
θk2) =
kn
X
j1=1
kn
X
j2=1
γj1γj2Uhn
n(ωj1)Uhn
n(ωj2)hωj1, ωj2i
+σ2
kn
X
j1=1
kn
X
j2=1
γj1γj2Whn
n(ωj1, ωj2)hωj1, ωj2i−2*R(γ,ω)
kn,
kn
X
j=1
γjUhn
n(ωj)ωj++kR(γ,ω)
knk2.
3.8.10 P oo o Co olla y 3.3.4
No e ha i wl,hn(Xi) = 1 when i=l, and wl,hn(Xi) = 0 when i6=l, hen Uhn
n(x) = 0 o all x∈ H,
and Whn
n(x, y) = n−1hΓnx, yi o all x, y ∈ H. Thus, he p oo is inished by Theo em 3.3.3, gi en ha
EXn+1 (Yn+1 −hˆ
θhn
kn, Xn+1i)2=σ2+σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
+hXn+1,ˆ
Rkni2,
EXn(kθ−ˆ
θhn
knk2) = σ2
n
kn
X
j=1
1
ˆ
λj
+kˆ
Rknk2.
3.8.11 P oo o Co olla y 3.3.5
No e ha i wl,hn(Xi) = n−1 o all i= 1,...,n, hen
Uhn
n(x) = 1
n
n
X
i=1 hXi, xihθ, Xi−hθ, Xii=hθ, XihX, xi−hΓnx, θi,∀x∈ H,
Whn
n(x, y) = 1
n2
n
X
i1=1
n
X
i2=1 hXi1, xihXi2, yi n
X
l=1
1
n2!=1
nhX, xihX, yi,∀x, y ∈ H,
whe e X=n−1Pn
i=1 Xi. Consequen ly, Theo em 3.3.3 gi es
EXn+1 (Yn+1 −hˆ
θhn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=
kn
X
j=1
(hθ, XihX, ˆ ji− ˆ
λjhˆ j, θi)
ˆ
λjhXn+1,ˆ ji
2
+σ2
n
kn
X
j=1
hX, ˆ ji
ˆ
λjhXn+1,ˆ ji− σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
−2hXn+1,ˆ
Rkni*Xn+1,
kn
X
j=1
hθ, XihX, ˆ ji− ˆ
λjhˆ j, θi
ˆ
λj
ˆ j+
=hθ, XihXn+1,ˆ
Mkni−hXn+1, θ −ˆ
Rkni2+σ2
n
hXn+1,ˆ
Mkni2−
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
−2hθ, XihXn+1,ˆ
RknihXn+1,ˆ
Mkni+ 2hXn+1,ˆ
RknihXn+1, θ −ˆ
Rkni
=σ2
n
hXn+1,ˆ
Mkni2−
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
+hθ, XihXn+1,ˆ
Mkni−hXn+1, θi2
+ 2 hθ, XihXn+1,ˆ
Mkni−hXn+1, θihXn+1,ˆ
Rkni+hXn+1,ˆ
Rkni2
−2hθ, XihXn+1,ˆ
RknihXn+1,ˆ
Mkni+ 2hXn+1,ˆ
RknihXn+1, θi−2hXn+1,ˆ
Rkni2
=σ2
n
hXn+1,ˆ
Mkni2−
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
+hθ, XihXn+1,ˆ
Mkni−hXn+1, θi2−hXn+1,ˆ
Rkni2,
3.8. APPENDIX CHAPTER 3 83
and
EXn(kθ−ˆ
θhn
knk2)−EXn(kθ−ˆ
θknk2) =
kn
X
j=1
(hθ, XihX, ˆ ji− ˆ
λjhˆ j, θi)2
ˆ
λ2
j
+σ2
n
kn
X
j=1
hX, ˆ ji2
ˆ
λ2
j−σ2
n
kn
X
j=1
1
ˆ
λj
=σ2
n
kn
X
j=1
hX, ˆ ji2
ˆ
λ2
j−
kn
X
j=1
1
ˆ
λj
+hθ, Xi2
kn
X
j=1
hX, ˆ ji2
ˆ
λ2
j−2hθ, Xi*kn
X
j=1
hX, ˆ ji
ˆ
λj
ˆ j, θ++
kn
X
j=1 hθ, ˆ ji2
=σ2
n
kˆ
Mknk2−
kn
X
j=1
1
ˆ
λj
+hθ, Xi2kˆ
Mknk2−2hθ, Xih ˆ
Mkn, θi+kθk2−kˆ
Rknk2
=σ2
n
kˆ
Mknk2−
kn
X
j=1
1
ˆ
λj
+hθ, Xik ˆ
Mknk−kθk2+ 2hθ, Xikˆ
Mknkkθk−h ˆ
Mkn, θi−kˆ
Rknk2,
wi h ˆ
Mkn=Pkn
j=1 ˆ
λ−1
jhX, ˆ jiˆ j.
3.8.12 P oo o Theo em 3.4.1
Conside Lemma 2.5.2 (see Chap e 2, page 49), wi h γj= (ˆ
λαn,1
j)−1and wj= ˆ αn,1
j. The e o e, he
condi ional p edic ion e o is gi en by
EXn+1 (Yn+1 −hˆ
θP S, αn
kn, Xn+1i)2=σ2
+σ2
n
kn
X
j1=1
kn
X
j2=1
(ˆ
λαn,1
j1
ˆ
λαn,1
j2)−1hΓnˆ αn,1
j1,ˆ αn,1
j2ihXn+1,ˆ αn,1
j1ihXn+1,ˆ αn,1
j2i+hXn+1, R(γ,w)
kni2,(3.29)
wi h R(γ,w)
kn=θ−Pkn
j=1 (ˆ
λαn,1
j)−1hΓnˆ αn,1
j, θiˆ αn,1
j. Recall ha {ˆ
λαn,1
j,ˆ αn,1
j}jsa is y
Γnˆ αn,1
j=ˆ
λαn,1
jˆ αn,1
j+αnQˆ αn,1
jand hˆ αn,1
j1,ˆ αn,1
j2i=δj1j2,(3.30)
whe e δj1j2= 1 i j1=j2, and 0 o he wise. Hence,
R(γ,w)
kn=θ−
kn
X
j=1 hθ, ˆ αn,1
jiˆ αn,1
j−αn
kn
X
j=1
hθ, Qˆ αn,1
ji
ˆ
λαn,1
j
ˆ αn,1
j.(3.31)
Using (3.30) and (3.31) in (3.29), one ge s
EXn+1 (Yn+1 −hˆ
θP S, αn
kn, Xn+1i)2=σ2
+σ2
n
kn
X
j=1
hXn+1,ˆ αn,1
ji2
ˆ
λαn,1
j
+αn
σ2
n
kn
X
j1=1
kn
X
j2=1
ραn,1
j1,j2
ˆ
λαn,1
j1
ˆ
λαn,1
j2hXn+1,ˆ αn,1
j1ihXn+1,ˆ αn,1
j2i
+*Xn+1, θ −
kn
X
j=1 hθ, ˆ αn,1
jiˆ αn,1
j−αn
kn
X
j=1
hθ, Qˆ αn,1
ji
ˆ
λαn,1
j
ˆ αn,1
j+2
,
(3.32)
whe e ραn,1
j1,j2=hˆ αn,1
j1, Qˆ αn,1
j2i. On he o he hand, due o he asymp o ic expansions o ˆ
λαn,1
jand
ˆ αn,1
j, ones has
1
ˆ
λαn,1
j
=1
ˆ
λj
+αn
ρj
ˆ
λ2
j
(1 + oa.s.(1)) and ˆ αn,1
j= ˆ j−αnΠjQˆ j+oa.s.(αn).(3.33)
84 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
Thus, using (3.33) in (3.32),
EXn+1 (Yn+1 −hˆ
θP S, αn
kn, Xn+1i)2=σ2+σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
−2αn
σ2
n
kn
X
j=1
hXn+1,ˆ jihXn+1,ΠjQˆ ji
ˆ
λj
(1 + oa.s.(1)) + αn
σ2
n
kn
X
j=1
ρjhXn+1,ˆ ji2
ˆ
λ2
j
(1 + oa.s.(1))
+αn
σ2
n
kn
X
j1=1
kn
X
j2=1
ρj1,j2
ˆ
λj1ˆ
λj2hXn+1,ˆ j1ihXn+1,ˆ j2i(1 + oa.s.(1)) + hXn+1,ˆ
Rkni2
+ 2αnhXn+1,ˆ
RknihXn+1,ˆ
CP S
kni(1 + oa.s.(1)) + α2
nhXn+1,ˆ
CP S
kni2(1 + oa.s.(1)),
wi h ˆ
CP S
kn=Pkn
j=1 (hθ, ˆ jiΠjQˆ j+hθ, ΠjQˆ jiˆ j−ˆ
λ−1
jhθ, Qˆ jiˆ j), and by Theo em 2.3.14 (see Chap-
e 2, page 39)
EXn+1 (Yn+1 −hˆ
θP S, αn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=−2αn
σ2
n
kn
X
j=1
hXn+1,ˆ jihXn+1,ΠjQˆ ji
ˆ
λj
(1 + oa.s.(1)) + αn
σ2
n
kn
X
j=1
ρjhXn+1,ˆ ji2
ˆ
λ2
j
(1 + oa.s.(1))
+αn
σ2
n
kn
X
j1=1
kn
X
j2=1
ρj1,j2
ˆ
λj1ˆ
λj2hXn+1,ˆ j1ihXn+1,ˆ j2i(1 + oa.s.(1))
+ 2αnhXn+1,ˆ
RknihXn+1,ˆ
CP S
kni(1 + oa.s.(1)) + α2
nhXn+1,ˆ
CP S
kni2(1 + oa.s.(1)).
Fo he condi ional es ima ion e o , one ge s om Lemma 2.5.2 (see Chap e 2, page 49)
EXn(kθ−ˆ
θP S, αn
knk2) = σ2
n
kn
X
j1=1
kn
X
j2=1
(ˆ
λαn,1
j1
ˆ
λαn,1
j2)−1hΓnˆ αn,1
j1,ˆ αn,1
j2ihˆ αn,1
j1,ˆ αn,1
j2i+kR(γ,w)
knk2.(3.34)
Using (3.30) and (3.31) in (3.34),
EXn(kθ−ˆ
θP S, αn
knk2) = σ2
n
kn
X
j=1
1
ˆ
λαn,1
j
+αn
σ2
n
kn
X
j=1
ραn,1
j
(ˆ
λαn,1
j)2
+θ−
kn
X
j=1 hθ, ˆ αn,1
jiˆ αn,1
j−αn
kn
X
j=1
hθ, Qˆ αn,1
ji
ˆ
λαn,1
j
ˆ αn,1
j
2
,
(3.35)
whe e ραn,1
j=hˆ αn,1
j, Qˆ αn,1
ji. As a esul , eplacing (3.33) in (3.35), i can be shown ha
EXn(kθ−ˆ
θP S, αn
knk2) = σ2
n
kn
X
j=1
1
ˆ
λj
+ 2αn
σ2
n
kn
X
j=1
ρj
ˆ
λ2
j
(1 + oa.s.(1)) + kˆ
Rkn+αnˆ
CP S
knk2(1 + oa.s.(1)),
and by Theo em 2.3.14 (see Chap e 2, page 39)
EXn(kθ−ˆ
θP S, αn
knk2)−EXn(kθ−ˆ
θknk2) =
2αn
σ2
n
kn
X
j=1
ρj
ˆ
λ2
j
+ 2αnhˆ
Rkn,ˆ
CP S
kni+α2
nkˆ
CP S
knk2
·(1 + oa.s.(1)).
3.8. APPENDIX CHAPTER 3 85
3.8.13 P oo o Co olla y 3.4.2
I Q=I,ρj= 1, ρj1,j2=δj1,j2, and ΠjQˆ j= 0. Thus, Theo em 3.4.1 implies
EXn+1 (Yn+1 −hˆ
θP S, αn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=
2αn
σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λ2
j−2αnhXn+1,ˆ
RknihXn+1,ˆ
Tkni+α2
nhXn+1,ˆ
Tkni2
(1 + oa.s.(1)),
EXn(kθ−ˆ
θP S, αn
knk2)−EXn(kθ−ˆ
θknk2) =
2αn
σ2
n
kn
X
j=1
1
ˆ
λ2
j
+α2
nkˆ
Tknk2
(1 + oa.s.(1)),
whe e ˆ
Tknis de ined as in Theo em 3.2.3 (see page 55).
3.8.14 P oo o Theo em 3.4.4
In his case, Lemma 2.5.2 (see Chap e 2, page 49) is going o be applied wi h γj= (ˆ
λαn,2
j)−1and
wj= ˆ αn,2
j. Then, he condi ional p edic ion e o is
EXn+1 (Yn+1 −hˆ
θS, αn
kn, Xn+1i)2=σ2
+σ2
n
kn
X
j1=1
kn
X
j2=1
(ˆ
λαn,2
j1
ˆ
λαn,2
j2)−1hΓnˆ αn,2
j1,ˆ αn,2
j2ihXn+1,ˆ αn,2
j1ihXn+1,ˆ αn,2
j2i+hXn+1, R(γ,w)
kni2,(3.36)
whe e R(γ,w)
kn=θ−Pkn
j=1 (ˆ
λαn,2
j)−1hΓnˆ αn,2
j, θiˆ αn,2
j. No e ha {ˆ
λαn,2
j,ˆ αn,2
j}jsa is y
Γnˆ αn,2
j=ˆ
λαn,2
j(ˆ αn,2
j+αnQˆ αn,2
j),(3.37)
whe eas he o hono mali y condi ions a e based on he penalized inne p oduc h·,·iαn, i.e.,
hˆ αn,2
j1,ˆ αn,2
j2iαn=hˆ αn,2
j1,ˆ αn,2
j2i+αnhˆ αn,2
j1, Qˆ αn,2
j2i=hˆ αn,2
j1,ˆ αn,2
j2i+αnραn,2
j1,j2=δj1j2,(3.38)
whe e ραn,2
j1,j2=hˆ αn,2
j1, Qˆ αn,2
j2i(wi h ραn,2
j=hˆ αn,2
j, Qˆ αn,2
ji), and δj1j2= 1 i j1=j2, and 0 o he wise.
The e o e,
R(γ,w)
kn=θ−
kn
X
j=1 hθ, ˆ αn,2
jiˆ αn,2
j−αn
kn
X
j=1 hθ, Qˆ αn,2
jiˆ αn,2
j.(3.39)
Using (3.37), (3.38) and (3.39) in (3.36), one has
EXn+1 (Yn+1 −hˆ
θS, αn
kn, Xn+1i)2
=σ2+σ2
n
kn
X
j=1
hXn+1,ˆ αn,2
ji2
ˆ
λαn,2
j
+*Xn+1, θ −
kn
X
j=1 hθ, ˆ αn,2
jiˆ αn,2
j−αn
kn
X
j=1 hθ, Qˆ αn,2
jiˆ αn,2
j+2
.(3.40)
In his case, he asymp o ic expansions o he eigenelemen s imply
1
ˆ
λαn,2
j
=1
ˆ
λj
+αn
ρj
ˆ
λj
(1 + oa.s.(1)) and ˆ αn,2
j= ˆ j−αnρj
2+ˆ
λjΠjQˆ j+oa.s.(αn).(3.41)
As a esul , using (3.41) in (3.40), one ge s o he condi ional p edic ion e o
EXn+1 (Yn+1 −hˆ
θS, αn
kn, Xn+1i)2=σ2+σ2
n
kn
X
j=1
hXn+1,ˆ ji2
ˆ
λj
−2αn
σ2
n
kn
X
j=1 hXn+1,ˆ jihXn+1,ΠjQˆ ji(1 + oa.s.(1)) + hXn+1,ˆ
Rkni2
+ 2αnhXn+1,ˆ
RknihXn+1,ˆ
CS
kni(1 + oa.s.(1)) + α2
nhXn+1,ˆ
CS
kni2(1 + oa.s.(1)),
86 CHAPTER 3. PRESMOOTHING IN FUNCTIONAL LINEAR REGRESSION
whe e ˆ
CS
kn=Pkn
j=1 (ρjhθ, ˆ jiˆ j+ˆ
λj(hθ, ˆ jiΠjQˆ j+hθ, ΠjQˆ jiˆ j)−hθ, Qˆ jiˆ j), and by he e o ex-
p essions ob ained in Theo em 2.3.14 (see Chap e 2, page 39)
EXn+1 (Yn+1 −hˆ
θS, αn
kn, Xn+1i)2−EXn+1 (Yn+1 −hˆ
θkn, Xn+1i)2
=−2αn
σ2
n
kn
X
j=1 hXn+1,ˆ jihXn+1,ΠjQˆ ji(1 + oa.s.(1)) + 2αnhXn+1,ˆ
RknihXn+1,ˆ
CS
kni(1 + oa.s.(1))
+α2
nhXn+1,ˆ
CS
kni2(1 + oa.s.(1)).
As a as he condi ional es ima ion e o is conce ned, Lemma 2.5.2 (see Chap e 2, page 49)
ensu es
EXn(kθ−ˆ
θS, αn
knk2) = σ2
n
kn
X
j1=1
kn
X
j2=1
(ˆ
λαn,2
j1
ˆ
λαn,2
j2)−1hΓnˆ αn,2
j1,ˆ αn,2
j2ihˆ αn,2
j1,ˆ αn,2
j2i+kR(γ,w)
knk2.(3.42)
Using (3.37), (3.38) and (3.39) in (3.42), one has
EXn(kθ−ˆ
θS, αn
knk2) = σ2
n
kn
X
j=1
1
ˆ
λαn,2
j−αn
σ2
n
kn
X
j=1
ραn,2
j
ˆ
λαn,2
j
+θ−
kn
X
j=1 hθ, ˆ αn,2
jiˆ αn,2
j−αn
kn
X
j=1 hθ, Qˆ αn,2
jiˆ αn,2
j
2
.
(3.43)
Then, using (3.41) in (3.43),
EXn(kθ−ˆ
θS, αn
knk2) = σ2
n
kn
X
j=1
1
ˆ
λj
+kˆ
Rknk2+ 2αnhˆ
Rkn,ˆ
CS
kni(1 + oa.s.(1)) + α2
nkˆ
CS
knk2(1 + oa.s.(1)),
and by Theo em 2.3.14 (see Chap e 2, page 39),
EXn(kθ−ˆ
θS, αn
knk2)−EXn(kθ−ˆ
θknk2) = 2αnhˆ
Rkn,ˆ
CS
kni(1 + oa.s.(1)) + α2
nkˆ
CS
knk2(1 + oa.s.(1)).
Chap e 4
Boo s ap in unc ional linea
eg ession
Dealing wi h he unc ional linea model wi h unc ional explana o y a iable and scala
esponse, and as commen ed p e iously, one o he mos popula me hods o pa ame e
model es ima ion is based on FPCA. Weak con e gence o a wide class o FPCA– ype
es ima o s has ecen ly been p o ed and, as a esul , asymp o ic con idence se s can be
ob ained. In his chap e , an al e na i e app oach in o de o compu e poin wise con idence
in e als by means o a boo s ap p ocedu e is p oposed, ob aining also i s asymp o ic
alidi y in he sense speci ied in Theo em 4.3.6 (i.e., he condi ional dis ibu ion o he
es ima o can be app oxima ed by he boo s ap dis ibu ion). In addi ion, a simula ion
s udy allows o compa e he p ac ical pe o mance o asymp o ic and boo s ap con idence
in e als in e ms o co e age a es o di e en sample sizes.
The me hodology p esen ed in his chap e was i s ly in oduced by Gonz´alez-Man eiga
and Ma ´ınez-Cal o (2010) and i ga e ise o he con ibu ion by Gonz´alez-Man eiga and
Ma ´ınez-Cal o (2011).
4.1 How o build con idence in e als?
Cu en echnology collec s da a in such a ine g id ha eco ded measu emen s can be seen as ob-
se a ions o a iables alued in unc ional spaces. This ac has a oused g ea in e es in de eloping
echniques ha a e ocused on unc ional da a se s, and many au ho s ha e made an e o o adap he
exis ing mul i a ia e eg ession me hods o he unc ional eg ession model wi h scala esponse. As
men ioned in p e ious chap e s, ega ding pa ame ic eg ession, he mos ex ensi ely s udied model
is he unc ional linea model wi h scala esponse gi en by (2.1) (see Chap e 2, page 32), ha is,
Y=hθ, Xi+ǫ,
whe e m(·) = hθ, ·i :H → Ris a linea eg ession ope a o such ha (H,h·,·i) is a eal sepa able Hilbe
space and θ∈ H sa is ies kθk2<∞(being k·k =h·,·i1/2), Xis a ze o–mean andom a iable alued
in H, and Yand ǫa e eal andom a iables wi h he la e e i ying ha E(ǫ) = 0, E(ǫ2) = σ2<+∞,
and E(ǫX) = 0. In addi ion, E(kXk4)<∞is equi ed h oughou his chap e .
Among he di e en echniques o es ima ing θ, me hods based on FPCA a e qui e popula (Ca do
e al., 1999, 2003c; Cai and Hall, 2006; Hall and Hosseini-Nasab, 2006; Hall and Ho owi z, 2007; Ca do
e al., 2007c), and his kind o es ima o s ha e been conside ed in his chap e . Howe e , he aim is
no o es ima e he eg ession unc ion m(·) bu o ob ain poin wise con idence in e als o a ce ain
con idence le el α∈(0,1), ha is, CIx,α ⊂Rsuch ha P(m(x)∈CIx,α) = 1 −α o a ixed x∈ H.
In o de o compu e such in e als, asymp o ic and boo s ap app oaches ha e been widely used in
he mul i a ia e eg ession con ex . Fo example, i is well–known he asymp o ic no mali y o he s an-
87
88 CHAPTER 4. BOOTSTRAP IN FUNCTIONAL LINEAR REGRESSION
da d leas squa es es ima o o he linea model, o he asymp o ic no mali y o he Nada aya–Wa son
es ima o o mo e gene al eg ession unc ions in a nonpa ame ic se ing. Hence, app oxima ed con-
idence in e als can be buil using no mal quan iles. As a as boo s ap is conce ned, i s in oduc ion
by E on (1979) esul ed in a new dis ibu ion app oxima ion applicable o a la ge numbe o si u-
a ions (Bickel and F eedman, 1981; Singh, 1981; Pa , 1985). In pa icula , boo s ap alidi y was
ob ained o linea and nonpa ame ic eg ession models by F eedman (1981) and Cao-Abad (1991),
espec i ely.
Nowadays, he adap a ion o hese p ocedu es o he unc ional con ex has been ini ia ed o
pa ame ic and nonpa ame ic es ima o s o he eg ession ope a o m(·). In his sense, Ca do e al.
(2007c) p o ed weak con e gence o a la ge class o FPCA– ype es ima o s and Fe a y e al. (2007a)
ob ained no mali y esul s o he nonpa ame ic es ima o p oposed by Fe a y and Vieu (2006b).
Mo eo e , Fe a y e al. (2010c) and Fe a y e al. (2012d) showed he alidi y o he boo s ap in
nonpa ame ic unc ional eg ession wi h scala esponse and unc ional esponse, espec i ely. In his
chap e , a boo s ap p ocedu e o he unc ional linea model wi h scala esponse has been p oposed,
and i s asymp o ic alidi y has been analysed.
Al hough his chap e is ocused on he use o boo s ap echniques in he unc ional con ex , some
au ho s s udied his issue in o he con ex s, o ins ance, in unc ional es ima ion (Cue as e al., 2006).
An upda ed s a e o he a o me hodological and p ac ical de elopmen s o esampling me hods o
unc ional da a (including boo s ap) can be ound in McMu y and Poli is (2011). Fu he mo e,
con ibu ions such as Gin´e and Zinn (1990), Dudley (1990), Sheehy and Wellne (1992), Poli is and
Romano (1994) o an de Vaa and Wellne (1996) p o ided heo e ical ools which allow o ob ain
alidi y esul s o applica ions o boo s ap o FDA.
Coming back o he p oblem o mula ed in his chap e , ob aining poin wise con idence in e als
o he linea eg ession ope a o m(·) = hθ, ·i, Sec ion 4.2 is de o ed o ecall some no a ion abou he
gene al FPCA– ype es ima o in oduced in (2.6) (see Chap e 2, page 37) and p esen he asymp o ic
con idence in e als ha can be de i ed om i s weak con e gence (see Theo em 2.3.17 in Chap e 2,
page 40). In Sec ion 4.3, a nai e and a wild boo s ap p ocedu es a e desc ibed ( hey a e simila o
he ones conside ed by Fe a y e al., 2010c in he nonpa ame ic case), and i s asymp o ic alidi y is
shown. A simula ion s udy is compiled in Sec ion 4.4 in o de o compa e asymp o ic and boo s ap
in e als. Finally, some conclusions can be ound in Sec ion 4.5, whe eas appendix collec s he p oo
o he main esul o he chap e and some necessa y echnical lemmas (see Sec ion 4.6).
4.2 Asymp o ic con idence in e als o linea eg ession
Fi s o all, ecall he gene al class o FPCA– ype es ima o s in oduced in Sec ion 2.3.2, “ b) De ini ion
o gene al class o FPCA– ype es ima o s ”, in Chap e 2 (see page 37). Le {(Xi, Yi)}n
i=1 be a sample
o i.i.d. andom a iables d awn om (X, Y ). Assuming ha he eigen alues o Γ e i y ha λ1>
λ2> . . . > 0, wi h he mul iplici y o each λjequals o one, and assuming ha (C.2.1) holds (see
Chap e 2, page 36), Ca do e al. (2007c) de eloped asymp o ic heo y o a la ge class o FPCA– ype
es ima o s gi en by (2.6) (see Chap e 2, page 37), ha is,
ˆ
θc=
n
X
j=1
c
n(ˆ
λj)∆nˆ jˆ j,
whe e c=cnis a s ic ly posi i e sequence such ha c→0 and c < λ1,{ c
n: [c, +∞)→R}nis a se-
quence o posi i e unc ions such ha (C.2.2)–(C.2.4) a e sa is ied (see Chap e 2, page 37), whe eas,
as usual, ∆n=n−1Pn
i=1 Xi⊗H′Yi, and {(ˆ
λj,ˆ j)}∞
j=1 a e he eigen alues and he eigen unc ions o
Γn=n−1Pn
i=1 Xi⊗HXi.
Fu he mo e, Rema k 2.3.8 (see Chap e 2, page 37) highligh ed ha i
kc
n= sup {j:λj+δj/2≥c}
e i ies ha (kc
n)2log kc
n/√n→0, hen ˆ
θc≈Pkc
n
j=1 c
n(ˆ
λj)∆nˆ jˆ j. Hence, he s anda d FPCA es ima-
o ˆ
θkn(see (2.4) in Chap e 2, page 36) is asymp o ically equi alen o ˆ
θcwhen n(x) = x−1I{x≥c},
4.3. BOOTSTRAP CONFIDENCE INTERVALS FOR LINEAR REGRESSION 89
whe eas he p esmoo hed FPCA es ima o ˆ
θαn
kn(see (3.1) in Chap e 3, page 54) is asymp o ically
equi alen o ˆ
θcwhen n(x) = (x+αn)−1I{x≥c}.
Fo ˆ
θc, Ca do e al. (2007c) de i ed con idence se s o p edic ion by means o a CLT o he weak
opology o he unc ional space H(see Theo em 2.3.17 in Chap e 2, page 40). In pa icula , gi en
x∈ H, Co olla y 2.3.18 (see Chap e 2, page 40) ensu es ha
√n
ˆ
c
n,xˆσ( ˆmc(x)−hˆ
Πkc
nθ, xi)w
→ N(0,1),
being ˆ
c
n,x =qPkc
n
j=1 ˆ
λj( c
n(ˆ
λj))2hx, ˆ ji2, ˆσ2a consis en es ima e o σ2, ˆmc(·) = hˆ
θc,·i, and ˆ
Πkc
n he
p ojec o on o he subspace spanned by he i s kc
neigen unc ions o Γn.
Thus, he app oxima ed asymp o ic con idence in e als o m(·) in oduced in (2.8) (see Chap e 2,
page 41) can be de i ed as ollows o a ixed con idence le el α∈(0,1). Le zαbe he quan ile o o de
α om a N(0,1) dis ibu ion, and suppose ha θ(o x) can be “well” app oxima ed by i s p ojec ion
ˆ
Πkc
nθ(o ˆ
Πkc
nx). Then, Co olla y 2.3.18 (see Chap e 2, page 40) ensu es ha
CIasy
x,α ="ˆmc(x)−z1−α/2
ˆ
c
n,xˆσ
√n,ˆmc(x) + z1−α/2
ˆ
c
n,xˆσ
√n#
sa is ies ha P(m(x)∈CIasy
x,α)≈1−α.
Rema k 4.2.1.B oadly speaking, he cons uc ion o CIasy
x,α is implici ly based on he eplacemen o θ
(o x) by ˆ
Πkc
nθ(o ˆ
Πkc
nx) in Co olla y 2.3.18 (see Chap e 2, page 40). Howe e , Ca do e al. (2007c)
indica ed in hei Rema k 5 ha his subs i u ion equi es e y es ic i e condi ions ei he on θo
on he eigen alues {λj}∞
j=1. Despi e his ac , in p ac ice, in e als CIasy
x,α will be buil , bu aking in o
accoun ha hey can be decen ed due o his eplacemen .
4.3 Boo s ap con idence in e als o linea eg ession
The in oduc ion o boo s ap echniques in his chap e has as i s objec i e o build poin wise con i-
dence in e als o he eg ession ope a o , which a e able o compe e wi h he asymp o ic app oach
p esen ed in Sec ion 4.2. In Sec ion 4.3.1, he boo s ap p ocedu e ha will be conside ed in his
chap e is in oduced, and Sec ion 4.3.2 collec s he main heo em ha ensu es he asymp o ic alidi y
o he p oposed boo s ap me hod.
4.3.1 Nai e and wild boo s ap
The mul i a ia e nai e and wild boo s ap p ocedu es ha e been adap ed o he unc ional con ex ,
in he same way as Fe a y e al. (2010c) did o he nonpa ame ic case. Algo i hms o esampling
p oceeds as ollows.
Algo i hm 4.3.1 (Nai e boo s ap).
S ep 1. Cons uc a pilo es ima o o θ:ˆ
θd=Pn
j=1 d
n(ˆ
λj)∆nˆ jˆ j. Ob ain he esiduals ˆǫi=
Yi−hˆ
θd, Xii o all i= 1,...,n.
S ep 2. D aw ˆǫ∗
1,...,ˆǫ∗
ni.i.d. andom a iables om he cumula i e dis ibu ion o (ˆǫi−ˆǫ)n
i=1, whe e
ˆǫ=n−1Pn
i=1 ˆǫi.
S ep 3. De ine Y∗
i=hˆ
θd, Xii+ ˆǫ∗
i, o all i= 1,...,n.
S ep 4. Use he boo s ap sample {(Xi, Y ∗
i)}n
i=1 in o de o ob ain he ollowing es ima o o θ:ˆ
θ∗
c,d =
Pn
j=1 c
n(ˆ
λj)∆∗
nˆ jˆ j,whe e ∆∗
n=n−1Pn
i=1 Xi⊗H′Y∗
i.
90 CHAPTER 4. BOOTSTRAP IN FUNCTIONAL LINEAR REGRESSION
S ep 5. Repea S ep 2–S ep 4 a la ge numbe o imes nboo ∈Nin o de o ob ain a sequence o
alues {ˆ
θ∗,l
c,d}nboo
l=1 .
Algo i hm 4.3.2 (Wild boo s ap).
S ep 1. Cons uc a pilo es ima o o θ:ˆ
θd=Pn
j=1 d
n(ˆ
λj)∆nˆ jˆ j. Ob ain he esiduals ˆǫi=
Yi−hˆ
θd, Xii o all i= 1,...,n.
S ep 2. De ine ˆǫ∗
i= ˆǫiVi o i= 1,...,n, being V1,...,Vni.i.d. andom a iables independen o he
da a {(Xi, Yi)}n
i=1, such ha E(V1) = 0 and E(V2
1) = 1.
S ep 3. De ine Y∗
i=hˆ
θd, Xii+ ˆǫ∗
i, o all i= 1,...,n.
S ep 4. Use he boo s ap sample {(Xi, Y ∗
i)}n
i=1 in o de o ob ain he ollowing es ima o o θ:ˆ
θ∗
c,d =
Pn
j=1 c
n(ˆ
λj)∆∗
nˆ jˆ j,whe e ∆∗
n=n−1Pn
i=1 Xi⊗H′Y∗
i.
S ep 5. Repea S ep 2–S ep 4 a la ge numbe o imes nboo ∈Nin o de o ob ain a sequence o
alues {ˆ
θ∗,l
c,d}nboo
l=1 .
As i can be seen in he p e ious algo i hm, wo sequences mus be ixed: d o he pilo es ima o
ˆ
θdand c o ˆ
θ∗
c,d. In ac , he ollowing assump ion will be equi ed o gua an ee he asymp o ic alidi y
o he boo s ap me hod
(C.4.1) c=cnand d=dna e s ic ly posi i e sequence such ha c, d →0 and
c, d < λ1. Mo eo e , c≤d o all n.
Rema k 4.3.3.No e ha unde (C.4.1), ha is, i c≤d, he numbe o p incipal componen s used
o cons uc ing ˆ
θ∗
c,d is la ge han he numbe o componen s used o ˆ
θd. The e o e, in some way,
one should o e smoo h when he pilo es ima o is calcula ed. In his sense, he need o choose
he sequences dand cc ea es a simila p oblem o he selec ion o bandwid h in he nonpa ame ic
eg ession: b o he pilo es ima o and h o he boo s ap one. Howe e , he choice o op imal
smoo hing pa ame e s is s ill an open ques ion in bo h his pa ame ic con ex and nonpa ame ic one:
see Mammen (2000) o Gonz´alez-Man eiga e al. (2004) o he mul i a ia e nonpa ame ic eg ession,
and Fe a y e al. (2010c) o he unc ional nonpa ame ic eg ession.
4.3.2 Asymp o ic alidi y o he boo s ap
F om now on, xwill deno e a ixed elemen o H. Fu he mo e, ecall he con e gence in p obabili y
de ini ion and i s associa ed “big O” and “li le o” no a ion.
De ini ion 4.3.4. Le {Zn}n∈Nbe a sequence o eal andom a iables and le Zbe a eal andom
a iable. {Zn}n∈Ncon e ges in p obabili y (P) o Z, ha is,
plimn→∞Zn=Z, o equi alen ly, Zn
P
→Z,
i and only i
∀ε > 0,lim
n→∞P(|Zn−Z|> ε) = 0.
4.4. SIMULATION STUDY 91
De ini ion 4.3.5. Le {Zn}n∈Nbe a sequence o eal andom a iables, le Zbe a eal andom
a iable, and le {un}n∈Nbe a de e minis ic sequence o posi i e eal numbe s. The a e o con e gence
in p obabili y o {Zn}n∈N o Zis o o de un, ha is,
Zn−Z=OP(un),
i and only i
lim
plimn→∞P(|Zn−Z|< pun) = 1.
Fu he mo e,
Zn−Z=oP(un) i and only i (un)−1(Zn−Z)P
→0.
In o de o ob ain he main esul in his chap e , i is necessa y ha he hypo heses equi ed by
Ca do e al. (2007c) in hei weak con e gence esul s o he p edic ion a a gi en alue hold (see
Theo em 2.3.17 in Chap e 2, page 40). This ac will be essen ial o show he asymp o ic alidi y
o he boo s ap p ocedu e in he nex heo em, whe e PXnYndeno es he p obabili y condi ionally
on (Xn,Yn) = {(X1, Y1), . . . , (Xn, Yn)}, and PXndeno es he p obabili y condi ionally on Xn=
{X1,...,Xn}. In addi ion, he ollowing no a ion is also used: ˆm∗
c,d(·) = hˆ
θ∗
c,d,·i, ˆmd(·) = hˆ
θd,·i, and
ˆmc(·) = hˆ
θc,·i.
Theo em 4.3.6. Unde he assump ions o Theo em 2.3.17 (see Chap e 2, page 40), i (C.4.1) is
sa is ied, i holds ha , o bo h he nai e and he wild boo s ap,
sup
y∈R|PXnYn(√n( ˆm∗
c,d(x)−ˆmd(x)) ≤y)−PXn(√n( ˆmc(x)−hˆ
Πkc
nθ, xi)≤y)|P
→0,
whe e ˆ
Πkc
nis he p ojec o on o he subspace spanned by he i s kc
neigen unc ions o Γn.
The p oo o he p e ious heo em can be ound in Sec ion 4.6.1 (see page 102).
Le α∈(0,1) be a con idence le el. Assuming ha θ(o x) can be well app oxima ed by i s
p ojec ion ˆ
Πkc
nθ(o ˆ
Πkc
nx), Theo em 4.3.6 ensu es ha he dis ibu ion o he ue e o ( ˆmc(x)−m(x))
can be app oxima ed by he dis ibu ion o he boo s ap e o ( ˆm∗
c,d(x)−ˆmd(x)). Hence, he poin wise
α–quan ile qα(x) o he dis ibu ion o he ue e o can be app oxima ed by he co esponding
poin wise boo s ap α–quan ile q∗
α(x). The e o e, one can app oxima e he (1 −α)–con idence in e al
o m(x) by means o
CI∗
x,α =hˆmc(x)−q∗
1−α/2(x),ˆmc(x)−q∗
α/2(x)i,(4.1)
since P(m(x)∈CI∗
x,α)≈1−α.
Rema k 4.3.7.As highligh ed in Rema k 4.2.1 o he asymp o ic in e als CIasy
x,α, he eplacemen o
θ(o x) by i s p ojec ion on o he subspace spanned by he i s kneigen unc ions o Γnneeds ce ain
speci ic hypo heses on θo on he eigen alues o Γ which a e ha dly e e ul illed. Hence, a decen ing
e ec could also appea when he boo s ap con idence in e als (4.1) a e buil in p ac ice.
4.4 Simula ion s udy
Fo he simula ion s udy, H=L2([0,1]) was selec ed, wi h hx, yi=R1
0x( )y( )d o all x, y ∈L2([0,1]).
The s udy equi ed he simula ion o 500 samples, each one consis ed o nobse a ions (n=
50,100,200,500) om he model Y=hθ, Xi+ǫ=R1
0θ( )X( )d +ǫ, whe e Xis a B ownian mo ion
and ǫ∼ N(0, σ2) wi h signal– o–noise a io =σ/pE(hX, θi2) = 0.2. Fo he pa ame e θ, he
98 CHAPTER 4. BOOTSTRAP IN FUNCTIONAL LINEAR REGRESSION
αCI x1x2x3x4x5x6
5% CImc
x,α 5.6 ( 1.84) 5.2 ( 5.64) 4.6 ( 1.64) 5.2 ( 1.85) 5.4 ( 7.27) 5.8 ( 8.58)
CImc,s
x,α 4.8 ( 1.91) 4.6 ( 5.71) 5.4 ( 1.69) 6.0 ( 1.87) 5.2 ( 8.45) 4.4 ( 8.35)
CIasy
x,α 9.0 ( 1.66) 8.4 ( 4.95) 9.0 ( 1.46) 10.2 ( 1.59) 48.8 ( 4.42) 15.0 ( 6.18)
CI∗
x,α,ˆ
kGCV
n+ 3 10.8 ( 1.65) 9.6 ( 4.88) 8.4 ( 1.47) 10.4 ( 1.65) 28.6 ( 6.11) 8.4 ( 8.39)
CI∗
x,α,ˆ
kGCV
n+ 2 10.6 ( 1.66) 10.2 ( 4.90) 8.6 ( 1.47) 10.4 ( 1.65) 30.6 ( 5.96) 7.8 ( 8.21)
CI∗
x,α,ˆ
kGCV
n+ 1 10.2 ( 1.66) 9.4 ( 4.93) 9.2 ( 1.47) 10.0 ( 1.65) 33.6 ( 5.80) 9.0 ( 8.03)
CI∗
x,α,ˆ
kGCV
n10.6 ( 1.66) 8.8 ( 4.93) 8.8 ( 1.47) 10.6 ( 1.64) 34.2 ( 5.76) 8.6 ( 7.95)
CI∗
x,α,ˆ
kGCV
n−1 8.0 ( 1.84) 6.8 ( 5.42) 6.8 ( 1.63) 7.2 ( 1.80) 29.6 ( 6.27) 7.8 ( 8.28)
CI∗
x,α,ˆ
kGCV
n−2 6.6 ( 1.94) 5.8 ( 5.78) 5.8 ( 1.72) 6.6 ( 1.90) 26.6 ( 6.74) 7.4 ( 8.43)
CI∗
x,α,ˆ
kGCV
n−3 5.6 ( 2.00) 5.8 ( 5.96) 4.6 ( 1.77) 5.8 ( 1.95) 24.6 ( 7.01) 6.8 ( 8.50)
CI∗
x,α,ˆ
kGCV
n−4 5.0 ( 2.04) 4.8 ( 6.05) 5.0 ( 1.80) 5.6 ( 1.99) 25.0 ( 7.16) 5.8 ( 8.56)
CI∗
x,α,ˆ
kGCV
n−5 4.4 ( 2.07) 4.4 ( 6.10) 4.4 ( 1.83) 5.4 ( 2.01) 24.8 ( 7.30) 6.2 ( 8.61)
CI∗
x,α,ˆ
kGCV
n−6 3.8 ( 2.08) 4.4 ( 6.15) 4.6 ( 1.84) 4.6 ( 2.03) 24.6 ( 7.35) 6.2 ( 8.59)
CI∗,s
x,α,ˆ
kGCV
n+ 3 6.8 ( 1.86) 6.0 ( 5.53) 5.8 ( 1.65) 7.4 ( 1.83) 25.4 ( 6.84) 6.4 ( 8.32)
CI∗,s
x,α,ˆ
kGCV
n+ 2 6.4 ( 1.86) 4.8 ( 5.50) 5.4 ( 1.64) 7.6 ( 1.82) 26.0 ( 6.77) 6.8 ( 8.25)
CI∗,s
x,α,ˆ
kGCV
n+ 1 6.6 ( 1.85) 6.6 ( 5.49) 6.6 ( 1.63) 7.6 ( 1.81) 26.2 ( 6.68) 8.0 ( 8.13)
CI∗,s
x,α,ˆ
kGCV
n7.2 ( 1.84) 5.8 ( 5.48) 6.6 ( 1.62) 8.0 ( 1.80) 28.2 ( 6.62) 9.4 ( 8.05)
CI∗,s
x,α,ˆ
kGCV
n−1 9.0 ( 1.83) 6.8 ( 5.44) 6.8 ( 1.62) 8.0 ( 1.81) 25.6 ( 6.79) 7.8 ( 8.05)
CI∗,s
x,α,ˆ
kGCV
n−2 7.8 ( 1.82) 6.6 ( 5.44) 6.0 ( 1.62) 7.4 ( 1.81) 26.4 ( 6.79) 6.8 ( 7.99)
CI∗,s
x,α,ˆ
kGCV
n−3 8.4 ( 1.82) 6.2 ( 5.45) 6.0 ( 1.61) 8.0 ( 1.81) 24.2 ( 6.92) 6.6 ( 7.92)
CI∗,s
x,α,ˆ
kGCV
n−4 8.2 ( 1.82) 6.8 ( 5.45) 6.8 ( 1.62) 7.8 ( 1.82) 23.2 ( 6.83) 7.4 ( 7.87)
CI∗,s
x,α,ˆ
kGCV
n−5 8.4 ( 1.81) 6.2 ( 5.44) 6.2 ( 1.62) 7.2 ( 1.82) 23.6 ( 6.84) 7.0 ( 7.84)
CI∗,s
x,α,ˆ
kGCV
n−6 7.6 ( 1.80) 6.0 ( 5.43) 6.8 ( 1.61) 8.0 ( 1.81) 24.8 ( 6.78) 7.2 ( 7.80)
10% CImc
x,α 11.2 ( 1.55) 10.4 ( 4.68) 10.0 ( 1.38) 11.0 ( 1.54) 10.4 ( 5.95) 12.0 ( 6.97)
CImc,s
x,α 10.8 ( 1.59) 8.6 ( 4.75) 10.4 ( 1.40) 11.4 ( 1.56) 10.0 ( 6.94) 11.6 ( 6.98)
CIasy
x,α 16.6 ( 1.39) 14.2 ( 4.16) 15.8 ( 1.22) 17.4 ( 1.33) 56.2 ( 3.71) 25.2 ( 5.19)
CI∗
x,α,ˆ
kGCV
n+ 3 17.6 ( 1.39) 15.6 ( 4.13) 14.8 ( 1.24) 15.6 ( 1.40) 39.6 ( 5.06) 13.8 ( 6.92)
CI∗
x,α,ˆ
kGCV
n+ 2 17.8 ( 1.40) 15.6 ( 4.14) 15.0 ( 1.24) 15.0 ( 1.39) 41.8 ( 4.89) 14.0 ( 6.75)
CI∗
x,α,ˆ
kGCV
n+ 1 18.0 ( 1.40) 16.6 ( 4.17) 14.8 ( 1.24) 15.2 ( 1.39) 45.4 ( 4.73) 16.6 ( 6.56)
CI∗
x,α,ˆ
kGCV
n17.6 ( 1.41) 16.6 ( 4.18) 15.6 ( 1.24) 15.4 ( 1.38) 45.6 ( 4.68) 18.0 ( 6.44)
CI∗
x,α,ˆ
kGCV
n−1 14.6 ( 1.56) 13.2 ( 4.59) 13.6 ( 1.38) 13.0 ( 1.52) 41.0 ( 5.07) 14.8 ( 6.61)
CI∗
x,α,ˆ
kGCV
n−2 12.4 ( 1.65) 10.2 ( 4.88) 11.6 ( 1.45) 12.6 ( 1.60) 39.0 ( 5.40) 13.8 ( 6.66)
CI∗
x,α,ˆ
kGCV
n−3 11.4 ( 1.70) 8.4 ( 5.03) 9.6 ( 1.49) 11.6 ( 1.64) 37.6 ( 5.59) 14.2 ( 6.67)
CI∗
x,α,ˆ
kGCV
n−4 9.6 ( 1.73) 8.2 ( 5.12) 10.2 ( 1.52) 10.6 ( 1.67) 36.8 ( 5.70) 13.0 ( 6.68)
CI∗
x,α,ˆ
kGCV
n−5 8.6 ( 1.75) 7.8 ( 5.17) 8.6 ( 1.54) 10.0 ( 1.69) 35.8 ( 5.79) 12.8 ( 6.72)
CI∗
x,α,ˆ
kGCV
n−6 8.2 ( 1.76) 7.6 ( 5.20) 8.2 ( 1.55) 9.4 ( 1.70) 36.2 ( 5.87) 13.2 ( 6.72)
CI∗,s
x,α,ˆ
kGCV
n+ 3 11.4 ( 1.57) 10.8 ( 4.64) 10.8 ( 1.39) 11.4 ( 1.54) 35.0 ( 5.48) 13.0 ( 6.97)
CI∗,s
x,α,ˆ
kGCV
n+ 2 13.6 ( 1.56) 11.2 ( 4.62) 11.0 ( 1.38) 11.4 ( 1.52) 36.6 ( 5.41) 13.8 ( 6.92)
CI∗,s
x,α,ˆ
kGCV
n+ 1 13.2 ( 1.55) 11.6 ( 4.61) 11.8 ( 1.37) 12.4 ( 1.52) 40.0 ( 5.32) 15.8 ( 6.78)
CI∗,s
x,α,ˆ
kGCV
n14.8 ( 1.54) 12.2 ( 4.59) 11.4 ( 1.36) 13.0 ( 1.51) 40.0 ( 5.28) 16.6 ( 6.72)
CI∗,s
x,α,ˆ
kGCV
n−1 14.4 ( 1.54) 11.2 ( 4.57) 12.4 ( 1.36) 12.4 ( 1.52) 35.4 ( 5.53) 15.6 ( 6.75)
CI∗,s
x,α,ˆ
kGCV
n−2 14.4 ( 1.53) 12.8 ( 4.57) 11.4 ( 1.36) 11.4 ( 1.52) 33.8 ( 5.62) 14.0 ( 6.69)
CI∗,s
x,α,ˆ
kGCV
n−3 14.2 ( 1.53) 11.0 ( 4.58) 12.8 ( 1.36) 12.4 ( 1.52) 32.4 ( 5.69) 14.8 ( 6.63)
CI∗,s
x,α,ˆ
kGCV
n−4 14.2 ( 1.52) 11.4 ( 4.57) 12.4 ( 1.36) 12.8 ( 1.53) 31.2 ( 5.68) 15.0 ( 6.58)
CI∗,s
x,α,ˆ
kGCV
n−5 14.2 ( 1.52) 11.4 ( 4.57) 12.6 ( 1.36) 12.8 ( 1.53) 31.2 ( 5.73) 14.4 ( 6.58)
CI∗,s
x,α,ˆ
kGCV
n−6 14.8 ( 1.51) 11.0 ( 4.55) 13.0 ( 1.35) 13.2 ( 1.52) 31.8 ( 5.73) 14.8 ( 6.54)
Table 4.5: Empi ical co e age and mean leng h (×102) o con idence in e als o he model pa ame e
θ2and sample size n= 50, in b acke s.
4.4. SIMULATION STUDY 99
αCI x1x2x3x4x5x6
5% CImc
x,α 7.0 ( 1.26) 5.0 ( 3.83) 5.8 ( 1.13) 4.0 ( 1.28) 3.8 ( 5.55) 3.8 ( 6.32)
CImc,s
x,α 6.6 ( 1.27) 4.4 ( 3.82) 5.6 ( 1.13) 4.8 ( 1.27) 4.6 ( 6.34) 3.8 ( 6.08)
CIasy
x,α 8.2 ( 1.19) 5.6 ( 3.56) 7.6 ( 1.05) 7.0 ( 1.16) 53.2 ( 3.46) 11.0 ( 4.77)
CI∗
x,α,ˆ
kGCV
n+ 3 9.0 ( 1.18) 6.0 ( 3.56) 6.8 ( 1.05) 6.2 ( 1.20) 34.6 ( 4.67) 4.6 ( 6.30)
CI∗
x,α,ˆ
kGCV
n+ 2 8.0 ( 1.18) 6.4 ( 3.56) 6.4 ( 1.06) 6.4 ( 1.20) 37.2 ( 4.50) 5.0 ( 6.10)
CI∗
x,α,ˆ
kGCV
n+ 1 9.4 ( 1.18) 6.4 ( 3.57) 7.0 ( 1.06) 6.4 ( 1.20) 42.0 ( 4.35) 6.0 ( 5.95)
CI∗
x,α,ˆ
kGCV
n8.6 ( 1.19) 6.4 ( 3.57) 7.0 ( 1.06) 6.2 ( 1.20) 42.2 ( 4.28) 6.0 ( 5.85)
CI∗
x,α,ˆ
kGCV
n−1 6.6 ( 1.26) 5.8 ( 3.78) 6.0 ( 1.12) 5.4 ( 1.25) 43.2 ( 4.37) 4.8 ( 5.81)
CI∗
x,α,ˆ
kGCV
n−2 5.8 ( 1.34) 4.2 ( 4.00) 5.2 ( 1.18) 4.4 ( 1.32) 37.4 ( 4.66) 4.8 ( 5.91)
CI∗
x,α,ˆ
kGCV
n−3 4.8 ( 1.40) 4.4 ( 4.17) 4.2 ( 1.24) 3.4 ( 1.37) 35.0 ( 4.90) 5.8 ( 6.02)
CI∗
x,α,ˆ
kGCV
n−4 4.4 ( 1.43) 4.0 ( 4.26) 3.6 ( 1.26) 3.8 ( 1.39) 33.8 ( 5.04) 5.4 ( 6.01)
CI∗
x,α,ˆ
kGCV
n−5 3.8 ( 1.46) 3.4 ( 4.34) 3.8 ( 1.28) 3.4 ( 1.41) 33.8 ( 5.16) 5.0 ( 6.05)
CI∗
x,α,ˆ
kGCV
n−6 3.8 ( 1.46) 3.2 ( 4.38) 4.0 ( 1.29) 4.0 ( 1.43) 32.6 ( 5.20) 5.6 ( 6.08)
CI∗,s
x,α,ˆ
kGCV
n+ 3 7.2 ( 1.25) 5.2 ( 3.77) 5.8 ( 1.12) 5.8 ( 1.25) 32.8 ( 4.74) 6.6 ( 5.95)
CI∗,s
x,α,ˆ
kGCV
n+ 2 7.2 ( 1.25) 5.4 ( 3.75) 6.2 ( 1.11) 5.4 ( 1.25) 35.2 ( 4.63) 5.4 ( 5.90)
CI∗,s
x,α,ˆ
kGCV
n+ 1 7.2 ( 1.25) 5.4 ( 3.75) 6.8 ( 1.11) 6.0 ( 1.25) 37.8 ( 4.53) 6.8 ( 5.85)
CI∗,s
x,α,ˆ
kGCV
n7.8 ( 1.25) 5.4 ( 3.74) 6.8 ( 1.11) 6.2 ( 1.24) 39.2 ( 4.50) 8.4 ( 5.79)
CI∗,s
x,α,ˆ
kGCV
n−1 8.2 ( 1.24) 5.2 ( 3.73) 6.4 ( 1.10) 5.4 ( 1.24) 41.8 ( 4.49) 6.0 ( 5.82)
CI∗,s
x,α,ˆ
kGCV
n−2 7.4 ( 1.23) 5.2 ( 3.72) 6.0 ( 1.10) 6.2 ( 1.25) 34.8 ( 4.67) 6.0 ( 5.82)
CI∗,s
x,α,ˆ
kGCV
n−3 8.0 ( 1.23) 5.2 ( 3.70) 5.8 ( 1.11) 5.4 ( 1.25) 33.8 ( 4.77) 5.2 ( 5.79)
CI∗,s
x,α,ˆ
kGCV
n−4 8.2 ( 1.23) 5.8 ( 3.71) 6.8 ( 1.10) 5.6 ( 1.25) 33.2 ( 4.82) 5.6 ( 5.79)
CI∗,s
x,α,ˆ
kGCV
n−5 8.2 ( 1.23) 5.4 ( 3.70) 6.4 ( 1.10) 5.4 ( 1.25) 33.2 ( 4.90) 5.4 ( 5.78)
CI∗,s
x,α,ˆ
kGCV
n−6 7.0 ( 1.23) 5.2 ( 3.71) 6.0 ( 1.10) 5.6 ( 1.25) 32.0 ( 4.94) 5.6 ( 5.78)
10% CImc
x,α 11.6 ( 1.06) 8.2 ( 3.20) 9.4 ( 0.94) 9.6 ( 1.07) 8.2 ( 4.59) 9.0 ( 5.17)
CImc,s
x,α 11.4 ( 1.06) 8.4 ( 3.20) 9.6 ( 0.95) 8.0 ( 1.06) 8.0 ( 5.28) 8.8 ( 5.12)
CIasy
x,α 13.8 ( 1.00) 10.4 ( 2.99) 11.0 ( 0.88) 10.6 ( 0.97) 60.2 ( 2.91) 17.8 ( 4.00)
CI∗
x,α,ˆ
kGCV
n+ 3 13.0 ( 0.99) 9.8 ( 3.01) 11.8 ( 0.89) 10.0 ( 1.01) 44.6 ( 3.89) 9.4 ( 5.21)
CI∗
x,α,ˆ
kGCV
n+ 2 14.2 ( 1.00) 10.4 ( 3.01) 11.4 ( 0.89) 10.2 ( 1.01) 48.8 ( 3.72) 9.6 ( 5.05)
CI∗
x,α,ˆ
kGCV
n+ 1 13.6 ( 1.00) 10.2 ( 3.01) 10.6 ( 0.89) 10.6 ( 1.01) 50.0 ( 3.58) 11.4 ( 4.90)
CI∗
x,α,ˆ
kGCV
n14.0 ( 1.00) 10.6 ( 3.01) 11.4 ( 0.89) 10.8 ( 1.01) 51.4 ( 3.52) 12.8 ( 4.80)
CI∗
x,α,ˆ
kGCV
n−1 11.0 ( 1.06) 9.2 ( 3.19) 9.8 ( 0.94) 9.6 ( 1.05) 53.0 ( 3.53) 11.6 ( 4.71)
CI∗
x,α,ˆ
kGCV
n−2 9.6 ( 1.13) 8.6 ( 3.38) 8.8 ( 1.00) 8.0 ( 1.11) 52.6 ( 3.75) 12.2 ( 4.73)
CI∗
x,α,ˆ
kGCV
n−3 9.2 ( 1.18) 6.2 ( 3.52) 7.2 ( 1.04) 6.4 ( 1.15) 47.6 ( 3.93) 11.8 ( 4.77)
CI∗
x,α,ˆ
kGCV
n−4 8.8 ( 1.21) 6.4 ( 3.59) 7.4 ( 1.06) 6.0 ( 1.17) 47.6 ( 4.03) 12.2 ( 4.76)
CI∗
x,α,ˆ
kGCV
n−5 8.0 ( 1.23) 5.8 ( 3.66) 6.8 ( 1.08) 6.4 ( 1.19) 47.6 ( 4.09) 12.4 ( 4.77)
CI∗
x,α,ˆ
kGCV
n−6 8.4 ( 1.24) 6.2 ( 3.69) 7.0 ( 1.09) 6.0 ( 1.20) 47.6 ( 4.14) 11.4 ( 4.78)
CI∗,s
x,α,ˆ
kGCV
n+ 3 12.4 ( 1.05) 8.2 ( 3.16) 10.2 ( 0.94) 8.6 ( 1.05) 42.0 ( 3.93) 11.4 ( 5.01)
CI∗,s
x,α,ˆ
kGCV
n+ 2 12.4 ( 1.05) 8.8 ( 3.16) 10.0 ( 0.93) 8.4 ( 1.05) 44.6 ( 3.82) 10.8 ( 4.97)
CI∗,s
x,α,ˆ
kGCV
n+ 1 12.0 ( 1.05) 9.6 ( 3.16) 9.4 ( 0.93) 9.2 ( 1.05) 46.0 ( 3.72) 12.6 ( 4.91)
CI∗,s
x,α,ˆ
kGCV
n12.4 ( 1.05) 8.8 ( 3.15) 9.8 ( 0.93) 9.0 ( 1.05) 48.2 ( 3.70) 12.6 ( 4.85)
CI∗,s
x,α,ˆ
kGCV
n−1 12.2 ( 1.04) 9.4 ( 3.13) 9.8 ( 0.93) 9.0 ( 1.05) 50.0 ( 3.77) 11.2 ( 4.90)
CI∗,s
x,α,ˆ
kGCV
n−2 12.4 ( 1.04) 10.0 ( 3.13) 9.6 ( 0.93) 8.8 ( 1.05) 45.8 ( 3.94) 11.2 ( 4.90)
CI∗,s
x,α,ˆ
kGCV
n−3 13.2 ( 1.04) 9.2 ( 3.12) 10.2 ( 0.93) 8.0 ( 1.06) 43.4 ( 4.04) 11.2 ( 4.89)
CI∗,s
x,α,ˆ
kGCV
n−4 12.8 ( 1.04) 9.0 ( 3.12) 10.4 ( 0.93) 8.6 ( 1.05) 42.2 ( 4.10) 11.8 ( 4.89)
CI∗,s
x,α,ˆ
kGCV
n−5 13.0 ( 1.04) 9.0 ( 3.12) 11.2 ( 0.93) 8.6 ( 1.06) 42.0 ( 4.17) 11.8 ( 4.87)
CI∗,s
x,α,ˆ
kGCV
n−6 13.4 ( 1.04) 10.6 ( 3.12) 11.0 ( 0.93) 8.4 ( 1.05) 43.2 ( 4.20) 12.0 ( 4.87)
Table 4.6: Empi ical co e age and mean leng h (×102) o con idence in e als o he model pa ame e
θ2and sample size n= 100, in b acke s.
100 CHAPTER 4. BOOTSTRAP IN FUNCTIONAL LINEAR REGRESSION
αCI x1x2x3x4x5x6
5% CImc
x,α 3.6 ( 0.87) 3.4 ( 2.64) 4.4 ( 0.78) 5.2 ( 0.89) 6.4 ( 4.25) 5.2 ( 4.69)
CImc,s
x,α 5.0 ( 0.87) 3.6 ( 2.62) 4.8 ( 0.78) 6.2 ( 0.88) 6.8 ( 4.89) 4.4 ( 4.57)
CIasy
x,α 5.6 ( 0.85) 3.8 ( 2.54) 6.0 ( 0.75) 7.4 ( 0.84) 49.6 ( 2.74) 10.2 ( 3.78)
CI∗
x,α,ˆ
kGCV
n+ 3 5.4 ( 0.84) 4.2 ( 2.51) 6.2 ( 0.75) 7.0 ( 0.85) 31.4 ( 3.66) 4.6 ( 4.83)
CI∗
x,α,ˆ
kGCV
n+ 2 5.0 ( 0.85) 4.8 ( 2.52) 6.0 ( 0.75) 7.8 ( 0.85) 36.2 ( 3.52) 6.0 ( 4.70)
CI∗
x,α,ˆ
kGCV
n+ 1 5.2 ( 0.85) 4.8 ( 2.52) 6.2 ( 0.76) 7.6 ( 0.85) 37.8 ( 3.42) 6.0 ( 4.59)
CI∗
x,α,ˆ
kGCV
n5.4 ( 0.85) 3.6 ( 2.52) 6.0 ( 0.75) 7.6 ( 0.85) 39.2 ( 3.36) 6.2 ( 4.53)
CI∗
x,α,ˆ
kGCV
n−1 5.4 ( 0.86) 3.6 ( 2.57) 6.0 ( 0.77) 6.4 ( 0.86) 41.4 ( 3.20) 6.4 ( 4.32)
CI∗
x,α,ˆ
kGCV
n−2 4.8 ( 0.91) 3.2 ( 2.72) 5.2 ( 0.81) 6.2 ( 0.90) 37.4 ( 3.32) 7.4 ( 4.30)
CI∗
x,α,ˆ
kGCV
n−3 3.6 ( 0.96) 2.0 ( 2.85) 3.8 ( 0.85) 5.2 ( 0.94) 35.4 ( 3.49) 7.2 ( 4.33)
CI∗
x,α,ˆ
kGCV
n−4 3.4 ( 0.98) 2.0 ( 2.92) 3.8 ( 0.86) 4.4 ( 0.95) 34.8 ( 3.54) 8.6 ( 4.34)
CI∗
x,α,ˆ
kGCV
n−5 3.4 ( 1.00) 2.0 ( 2.98) 3.2 ( 0.88) 3.4 ( 0.98) 32.6 ( 3.64) 7.0 ( 4.35)
CI∗
x,α,ˆ
kGCV
n−6 2.8 ( 1.02) 1.8 ( 3.04) 2.8 ( 0.90) 2.6 ( 0.99) 33.6 ( 3.72) 7.2 ( 4.36)
CI∗,s
x,α,ˆ
kGCV
n+ 3 5.2 ( 0.87) 3.6 ( 2.59) 5.0 ( 0.77) 6.2 ( 0.87) 29.2 ( 3.76) 6.0 ( 4.60)
CI∗,s
x,α,ˆ
kGCV
n+ 2 5.0 ( 0.87) 3.6 ( 2.59) 5.2 ( 0.77) 6.4 ( 0.87) 34.0 ( 3.67) 7.0 ( 4.54)
CI∗,s
x,α,ˆ
kGCV
n+ 1 4.8 ( 0.87) 3.6 ( 2.59) 5.8 ( 0.77) 5.8 ( 0.87) 36.8 ( 3.59) 8.0 ( 4.52)
CI∗,s
x,α,ˆ
kGCV
n5.6 ( 0.87) 4.2 ( 2.59) 5.6 ( 0.77) 6.8 ( 0.87) 37.4 ( 3.56) 7.6 ( 4.48)
CI∗,s
x,α,ˆ
kGCV
n−1 4.8 ( 0.86) 3.6 ( 2.58) 5.6 ( 0.77) 6.6 ( 0.87) 40.4 ( 3.46) 6.8 ( 4.52)
CI∗,s
x,α,ˆ
kGCV
n−2 5.6 ( 0.87) 3.6 ( 2.58) 6.0 ( 0.77) 6.8 ( 0.87) 39.0 ( 3.52) 6.4 ( 4.50)
CI∗,s
x,α,ˆ
kGCV
n−3 6.2 ( 0.86) 3.4 ( 2.57) 5.6 ( 0.77) 5.4 ( 0.88) 35.8 ( 3.63) 6.2 ( 4.52)
CI∗,s
x,α,ˆ
kGCV
n−4 5.4 ( 0.86) 3.4 ( 2.58) 5.6 ( 0.77) 6.0 ( 0.88) 33.8 ( 3.65) 6.4 ( 4.53)
CI∗,s
x,α,ˆ
kGCV
n−5 6.0 ( 0.86) 3.2 ( 2.58) 5.4 ( 0.77) 6.4 ( 0.89) 32.4 ( 3.72) 5.6 ( 4.52)
CI∗,s
x,α,ˆ
kGCV
n−6 5.4 ( 0.86) 3.8 ( 2.57) 5.2 ( 0.77) 6.2 ( 0.89) 33.2 ( 3.77) 5.2 ( 4.50)
10% CImc
x,α 9.0 ( 0.74) 6.4 ( 2.21) 10.0 ( 0.66) 10.0 ( 0.75) 11.2 ( 3.54) 10.2 ( 3.86)
CImc,s
x,α 8.8 ( 0.73) 8.0 ( 2.20) 9.4 ( 0.66) 10.8 ( 0.74) 11.8 ( 4.12) 10.4 ( 3.84)
CIasy
x,α 11.0 ( 0.71) 9.4 ( 2.13) 11.4 ( 0.63) 12.0 ( 0.71) 57.6 ( 2.30) 16.0 ( 3.17)
CI∗
x,α,ˆ
kGCV
n+ 3 11.4 ( 0.71) 9.4 ( 2.12) 11.0 ( 0.63) 12.4 ( 0.72) 40.2 ( 3.06) 9.8 ( 4.02)
CI∗
x,α,ˆ
kGCV
n+ 2 10.8 ( 0.71) 9.4 ( 2.13) 11.6 ( 0.63) 12.8 ( 0.72) 43.8 ( 2.92) 10.4 ( 3.90)
CI∗
x,α,ˆ
kGCV
n+ 1 11.6 ( 0.71) 9.6 ( 2.12) 11.0 ( 0.63) 12.8 ( 0.72) 45.4 ( 2.84) 11.0 ( 3.80)
CI∗
x,α,ˆ
kGCV
n11.2 ( 0.71) 9.0 ( 2.12) 11.4 ( 0.63) 13.4 ( 0.71) 47.8 ( 2.78) 12.6 ( 3.73)
CI∗
x,α,ˆ
kGCV
n−1 10.4 ( 0.73) 9.4 ( 2.16) 10.6 ( 0.65) 12.6 ( 0.72) 51.6 ( 2.62) 13.8 ( 3.54)
CI∗
x,α,ˆ
kGCV
n−2 9.0 ( 0.77) 7.0 ( 2.29) 9.6 ( 0.68) 10.6 ( 0.75) 51.0 ( 2.69) 14.2 ( 3.49)
CI∗
x,α,ˆ
kGCV
n−3 7.6 ( 0.81) 6.0 ( 2.40) 8.4 ( 0.71) 8.8 ( 0.79) 48.2 ( 2.81) 13.8 ( 3.48)
CI∗
x,α,ˆ
kGCV
n−4 6.8 ( 0.82) 4.6 ( 2.46) 7.0 ( 0.73) 8.4 ( 0.80) 49.6 ( 2.85) 15.0 ( 3.46)
CI∗
x,α,ˆ
kGCV
n−5 6.0 ( 0.84) 4.4 ( 2.51) 6.2 ( 0.74) 9.0 ( 0.82) 46.6 ( 2.91) 14.6 ( 3.45)
CI∗
x,α,ˆ
kGCV
n−6 5.4 ( 0.86) 4.0 ( 2.56) 5.8 ( 0.76) 8.0 ( 0.83) 44.8 ( 2.96) 13.8 ( 3.44)
CI∗,s
x,α,ˆ
kGCV
n+ 3 10.4 ( 0.73) 8.4 ( 2.18) 9.8 ( 0.65) 11.6 ( 0.73) 38.6 ( 3.12) 12.4 ( 3.87)
CI∗,s
x,α,ˆ
kGCV
n+ 2 10.2 ( 0.73) 9.0 ( 2.18) 10.2 ( 0.65) 12.4 ( 0.73) 42.4 ( 3.03) 11.2 ( 3.82)
CI∗,s
x,α,ˆ
kGCV
n+ 1 10.6 ( 0.73) 9.0 ( 2.18) 10.6 ( 0.65) 12.4 ( 0.73) 45.2 ( 2.98) 13.4 ( 3.80)
CI∗,s
x,α,ˆ
kGCV
n10.0 ( 0.73) 8.8 ( 2.18) 11.0 ( 0.65) 13.0 ( 0.73) 47.2 ( 2.94) 14.2 ( 3.76)
CI∗,s
x,α,ˆ
kGCV
n−1 10.8 ( 0.73) 9.4 ( 2.17) 11.2 ( 0.65) 12.6 ( 0.73) 47.4 ( 2.90) 12.0 ( 3.79)
CI∗,s
x,α,ˆ
kGCV
n−2 10.2 ( 0.73) 8.8 ( 2.17) 11.6 ( 0.65) 11.6 ( 0.73) 45.4 ( 2.96) 12.4 ( 3.79)
CI∗,s
x,α,ˆ
kGCV
n−3 11.0 ( 0.73) 8.6 ( 2.17) 10.6 ( 0.65) 11.2 ( 0.74) 43.0 ( 3.06) 11.2 ( 3.80)
CI∗,s
x,α,ˆ
kGCV
n−4 9.4 ( 0.73) 8.4 ( 2.17) 11.2 ( 0.65) 11.4 ( 0.74) 40.2 ( 3.10) 11.6 ( 3.81)
CI∗,s
x,α,ˆ
kGCV
n−5 10.8 ( 0.73) 8.6 ( 2.17) 10.6 ( 0.65) 11.6 ( 0.74) 40.4 ( 3.16) 10.8 ( 3.81)
CI∗,s
x,α,ˆ
kGCV
n−6 10.6 ( 0.73) 8.6 ( 2.17) 10.8 ( 0.65) 10.8 ( 0.75) 39.6 ( 3.21) 10.4 ( 3.80)
Table 4.7: Empi ical co e age and mean leng h (×102) o con idence in e als o he model pa ame e
θ2and sample size n= 200, in b acke s.
4.4. SIMULATION STUDY 101
αCI x1x2x3x4x5x6
5% CImc
x,α 5.8 ( 0.55) 4.8 ( 1.64) 4.4 ( 0.49) 4.2 ( 0.56) 6.0 ( 3.01) 5.0 ( 3.22)
CImc,s
x,α 5.6 ( 0.55) 4.2 ( 1.64) 4.6 ( 0.49) 4.4 ( 0.56) 5.6 ( 3.52) 5.8 ( 3.09)
CIasy
x,α 6.4 ( 0.54) 4.4 ( 1.62) 5.6 ( 0.48) 4.6 ( 0.55) 56.2 ( 1.92) 11.2 ( 2.65)
CI∗
x,α,ˆ
kGCV
n+ 3 7.2 ( 0.54) 4.6 ( 1.61) 5.4 ( 0.48) 4.8 ( 0.55) 37.4 ( 2.53) 5.0 ( 3.30)
CI∗
x,α,ˆ
kGCV
n+ 2 6.8 ( 0.54) 4.8 ( 1.62) 5.6 ( 0.48) 4.6 ( 0.55) 41.6 ( 2.45) 4.8 ( 3.22)
CI∗
x,α,ˆ
kGCV
n+ 1 6.6 ( 0.54) 4.8 ( 1.61) 6.0 ( 0.48) 4.6 ( 0.55) 43.2 ( 2.37) 6.2 ( 3.13)
CI∗
x,α,ˆ
kGCV
n6.8 ( 0.54) 4.8 ( 1.61) 5.8 ( 0.48) 4.6 ( 0.55) 43.6 ( 2.34) 6.4 ( 3.09)
CI∗
x,α,ˆ
kGCV
n−1 6.2 ( 0.54) 4.2 ( 1.62) 5.4 ( 0.49) 4.8 ( 0.55) 49.4 ( 2.16) 8.0 ( 2.92)
CI∗
x,α,ˆ
kGCV
n−2 6.2 ( 0.55) 4.4 ( 1.64) 5.6 ( 0.49) 5.0 ( 0.55) 55.0 ( 2.07) 8.6 ( 2.81)
CI∗
x,α,ˆ
kGCV
n−3 5.0 ( 0.57) 3.2 ( 1.71) 3.8 ( 0.51) 4.0 ( 0.57) 54.0 ( 2.10) 11.6 ( 2.76)
CI∗
x,α,ˆ
kGCV
n−4 5.2 ( 0.59) 3.2 ( 1.76) 4.4 ( 0.52) 4.0 ( 0.58) 53.4 ( 2.12) 11.4 ( 2.70)
CI∗
x,α,ˆ
kGCV
n−5 4.4 ( 0.62) 2.2 ( 1.84) 3.2 ( 0.55) 3.8 ( 0.61) 48.2 ( 2.23) 9.6 ( 2.73)
CI∗
x,α,ˆ
kGCV
n−6 3.6 ( 0.64) 1.8 ( 1.91) 2.2 ( 0.56) 2.8 ( 0.63) 44.4 ( 2.33) 9.8 ( 2.76)
CI∗,s
x,α,ˆ
kGCV
n+ 3 7.2 ( 0.55) 4.2 ( 1.64) 4.8 ( 0.49) 4.2 ( 0.56) 33.8 ( 2.59) 8.0 ( 3.11)
CI∗,s
x,α,ˆ
kGCV
n+ 2 5.8 ( 0.54) 4.2 ( 1.63) 5.8 ( 0.49) 5.0 ( 0.56) 38.0 ( 2.54) 7.2 ( 3.09)
CI∗,s
x,α,ˆ
kGCV
n+ 1 6.6 ( 0.55) 4.8 ( 1.63) 5.2 ( 0.49) 5.0 ( 0.56) 38.6 ( 2.48) 7.6 ( 3.07)
CI∗,s
x,α,ˆ
kGCV
n6.6 ( 0.55) 4.4 ( 1.63) 4.6 ( 0.49) 5.0 ( 0.56) 40.2 ( 2.46) 7.6 ( 3.06)
CI∗,s
x,α,ˆ
kGCV
n−1 6.2 ( 0.55) 4.4 ( 1.63) 5.8 ( 0.49) 4.6 ( 0.56) 44.4 ( 2.36) 8.0 ( 3.05)
CI∗,s
x,α,ˆ
kGCV
n−2 5.8 ( 0.54) 5.0 ( 1.63) 5.0 ( 0.49) 4.8 ( 0.56) 47.6 ( 2.31) 6.4 ( 3.03)
CI∗,s
x,α,ˆ
kGCV
n−3 6.8 ( 0.54) 4.2 ( 1.63) 5.8 ( 0.49) 4.6 ( 0.56) 48.0 ( 2.34) 5.8 ( 3.03)
CI∗,s
x,α,ˆ
kGCV
n−4 6.4 ( 0.54) 4.8 ( 1.63) 5.6 ( 0.49) 4.6 ( 0.56) 46.0 ( 2.38) 6.6 ( 3.04)
CI∗,s
x,α,ˆ
kGCV
n−5 6.6 ( 0.54) 4.8 ( 1.62) 5.2 ( 0.49) 5.0 ( 0.57) 40.2 ( 2.48) 5.8 ( 3.07)
CI∗,s
x,α,ˆ
kGCV
n−6 6.8 ( 0.54) 4.6 ( 1.63) 5.2 ( 0.49) 4.4 ( 0.57) 38.4 ( 2.56) 6.0 ( 3.10)
10% CImc
x,α 10.6 ( 0.46) 11.0 ( 1.38) 10.4 ( 0.41) 8.2 ( 0.47) 10.4 ( 2.53) 11.2 ( 2.67)
CImc,s
x,α 11.2 ( 0.46) 10.8 ( 1.38) 11.4 ( 0.41) 8.8 ( 0.47) 10.8 ( 3.00) 11.6 ( 2.61)
CIasy
x,α 11.4 ( 0.45) 11.2 ( 1.36) 11.4 ( 0.41) 9.8 ( 0.46) 63.8 ( 1.61) 17.6 ( 2.23)
CI∗
x,α,ˆ
kGCV
n+ 3 11.0 ( 0.45) 11.6 ( 1.36) 11.6 ( 0.41) 8.8 ( 0.47) 47.4 ( 2.12) 10.4 ( 2.75)
CI∗
x,α,ˆ
kGCV
n+ 2 11.6 ( 0.45) 11.4 ( 1.36) 11.0 ( 0.41) 9.2 ( 0.46) 49.0 ( 2.05) 11.6 ( 2.68)
CI∗
x,α,ˆ
kGCV
n+ 1 11.4 ( 0.45) 12.0 ( 1.36) 11.6 ( 0.41) 9.4 ( 0.46) 50.2 ( 1.97) 12.0 ( 2.60)
CI∗
x,α,ˆ
kGCV
n12.4 ( 0.45) 11.4 ( 1.36) 11.2 ( 0.41) 9.0 ( 0.46) 52.0 ( 1.94) 14.2 ( 2.56)
CI∗
x,α,ˆ
kGCV
n−1 11.6 ( 0.46) 11.4 ( 1.37) 10.4 ( 0.41) 9.2 ( 0.46) 58.8 ( 1.78) 15.2 ( 2.41)
CI∗
x,α,ˆ
kGCV
n−2 11.6 ( 0.46) 11.2 ( 1.38) 10.8 ( 0.41) 9.6 ( 0.47) 61.4 ( 1.69) 16.0 ( 2.30)
CI∗
x,α,ˆ
kGCV
n−3 9.4 ( 0.48) 7.4 ( 1.44) 9.4 ( 0.43) 7.8 ( 0.48) 64.2 ( 1.70) 17.8 ( 2.23)
CI∗
x,α,ˆ
kGCV
n−4 8.6 ( 0.49) 6.6 ( 1.48) 9.4 ( 0.44) 7.4 ( 0.49) 64.8 ( 1.71) 18.2 ( 2.17)
CI∗
x,α,ˆ
kGCV
n−5 7.4 ( 0.52) 5.0 ( 1.55) 7.6 ( 0.46) 6.4 ( 0.51) 63.6 ( 1.79) 18.2 ( 2.18)
CI∗
x,α,ˆ
kGCV
n−6 6.4 ( 0.54) 5.2 ( 1.61) 6.0 ( 0.47) 4.8 ( 0.53) 60.8 ( 1.87) 18.2 ( 2.19)
CI∗,s
x,α,ˆ
kGCV
n+ 3 11.0 ( 0.46) 11.2 ( 1.37) 11.4 ( 0.41) 9.6 ( 0.47) 43.8 ( 2.14) 12.6 ( 2.62)
CI∗,s
x,α,ˆ
kGCV
n+ 2 12.0 ( 0.46) 10.8 ( 1.37) 10.6 ( 0.41) 9.4 ( 0.47) 46.6 ( 2.10) 12.6 ( 2.60)
CI∗,s
x,α,ˆ
kGCV
n+ 1 11.2 ( 0.46) 10.6 ( 1.37) 11.0 ( 0.41) 9.4 ( 0.47) 47.2 ( 2.05) 14.0 ( 2.58)
CI∗,s
x,α,ˆ
kGCV
n11.0 ( 0.46) 10.8 ( 1.37) 11.2 ( 0.41) 9.0 ( 0.47) 47.6 ( 2.03) 14.6 ( 2.57)
CI∗,s
x,α,ˆ
kGCV
n−1 11.6 ( 0.46) 11.8 ( 1.37) 10.8 ( 0.41) 8.0 ( 0.47) 51.6 ( 1.97) 14.2 ( 2.57)
CI∗,s
x,α,ˆ
kGCV
n−2 11.2 ( 0.46) 11.2 ( 1.37) 10.6 ( 0.41) 8.4 ( 0.47) 55.6 ( 1.93) 13.0 ( 2.55)
CI∗,s
x,α,ˆ
kGCV
n−3 12.0 ( 0.46) 11.0 ( 1.37) 12.0 ( 0.41) 8.8 ( 0.47) 56.4 ( 1.97) 13.8 ( 2.55)
CI∗,s
x,α,ˆ
kGCV
n−4 11.6 ( 0.46) 10.0 ( 1.37) 11.6 ( 0.41) 8.6 ( 0.47) 56.4 ( 2.01) 13.8 ( 2.56)
CI∗,s
x,α,ˆ
kGCV
n−5 11.2 ( 0.46) 11.2 ( 1.37) 11.0 ( 0.41) 7.6 ( 0.48) 50.6 ( 2.10) 13.6 ( 2.59)
CI∗,s
x,α,ˆ
kGCV
n−6 11.2 ( 0.46) 12.0 ( 1.37) 11.2 ( 0.41) 7.8 ( 0.48) 48.2 ( 2.18) 13.0 ( 2.61)
Table 4.8: Empi ical co e age and mean leng h (×102) o con idence in e als o he model pa ame e
θ2and sample size n= 500, in b acke s.
102 CHAPTER 4. BOOTSTRAP IN FUNCTIONAL LINEAR REGRESSION
4.5 Final conclusions
In his chap e , nai e and wild boo s ap echniques ha e been p oposed in o de o ge poin wise
con idence in e als o he eg ession ope a o m(x) in he unc ional linea model wi h scala e-
sponse. Fo hese wo boo s ap app oaches, algo i hms ha e been p esen ed (see Algo i hm 4.3.1
and Algo i hm 4.3.2, pages 89–90) and hei asymp o ic alidi y has been p o ed (see Theo em 4.3.6,
page 91).
F om a p ac ical poin o iew, a simula ion s udy has been ca ied ou , allowing o con i m he
good beha iou o boo s ap con idence in e als. Mo eo e , i was obse ed ha empi ical co e age
a es o hese in e als a e close o nominal α han he co e age a es o con idence in e als based on
asymp o ic no mali y esul s when an op imal alue o he pilo pa ame e kd
nis conside ed, specially
o small sample sizes. Howe e , u he esea ch is equi ed in o de o ind a da a–d i en me hod
which allows selec ing his op imal alue.
4.6 Appendix Chap e 4
This appendix includes he p oo o Theo em 4.3.6, he main esul o he chap e , oge he wi h he
echnical lemmas which a e necessa y o p o e i .
4.6.1 P oo o Theo em 4.3.6
In o de o p o e Theo em 4.3.6, he p oo o Theo em 3.2 in Fe a y e al. (2010c) will be mimicked.
Fi s ly, i is de ined
φc,d(y) = Φ
y−√n(EXnYn( ˆm∗
c,d(x)) −ˆmd(x))
qnVa XnYn( ˆm∗
c,d(x))
,
φc(y) = Φ y−√n(EXn( ˆmc(x)) −hˆ
Πkc
nθ, xi)
pnVa Xn( ˆmc(x)) !,
(4.7)
whe e Φ is he dis ibu ion unc ion o he s anda d no mal dis ibu ion N(0,1), EXnYnand Va XnYn
deno e he expec a ion and he a iance condi ionally on (Xn,Yn) = {(X1, Y1),...,(Xn, Yn)}, and
EXnand Va Xndeno e he expec a ion and he a iance condi ionally on Xn={X1,...,Xn}. Then,
one can w i e
PXnYn(√n( ˆm∗
c,d(x)−ˆmd(x)) ≤y)−PXn(√n( ˆmc(x)−hˆ
Πkc
nθ, xi)≤y)
= (PXnYn(√n( ˆm∗
c,d(x)−ˆmd(x)) ≤y)−φc,d(y)) + (φc,d(y)−φc(y))
+ (φc(y)−PXn(√n( ˆmc(x)−hˆ
Πkc
nθ, xi)≤y))
=T1(y) + T2(y) + T3(y).
Lemma 4.6.1 ensu es T1(y)→0a.s. and T3(y)→0a.s. o a ixed y, and he uni o m con e gence
can be ob ained using he con inui y o Φ and Polya’s Theo em. Finally, Lemma 4.6.3 (see page 104)
allows o ob ain supy|T2(y)|P
→0.
4.6. APPENDIX CHAPTER 4 103
4.6.2 Fo mula ion and p oo o Lemma 4.6.1
Lemma 4.6.1. When he assump ions o Theo em 4.3.6 (see page 91) hold, hen
ˆmc(x)−EXn( ˆmc(x))
pVa Xn( ˆmc(x))
w
→ N(0,1) and ˆm∗
c,d(x)−EXnYn( ˆm∗
c,d(x))
qVa XnYn( ˆm∗
c,d(x))
w
→ N(0,1),
whe e EXnand Va Xndeno e he expec a ion and he a iance condi ionally on Xn={X1,...,Xn},
whe eas EXnYnand Va XnYndeno e he expec a ion and he a iance condi ionally on (Xn,Yn) =
{(X1, Y1),...,(Xn, Yn)}.
P oo . The i s s a emen is shown by Ca do e al. (2007c) (see Theo em 2.3.17 in Chap e 2,
page 40). The second one ollows a e conside ing Lemma 4.6.2 and checking he nex Liapuno ’s
condi ion Pn
i=1 EXnYn|(n−1Pn
j=1 c
n(ˆ
λj)hXi,ˆ jihx, ˆ ji)ǫ∗
i|3
(n−1σ2(ˆ
c
n,x)2)3/2
P
→0.
I can be shown ha he nume a o is OP(n−2). Mo eo e , using (C.2.13) and [ˆ
c
n,x]2/[ c
n,x]2P
→1 (see
p oo o Co olla y 2 by Ca do e al., 2007c), one has ha he denomina o is OP(n−3/2). The e o e,
Liapuno ’s condi ion is e i ied.
4.6.3 Fo mula ion and p oo o Lemma 4.6.2
Lemma 4.6.2. When he assump ions o Theo em 4.3.6 (see page 91) hold, hen
EXn( ˆmc(x)) = hˆ
Πkc
nθ, xi+oP(n−1/2),Va Xn( ˆmc(x)) = σ2
n(ˆ
c
n,x)2+oP(n−3/2),
EXnYn( ˆm∗
c,d(x)) = ˆmd(x) + oP(n−1/2),Va XnYn( ˆm∗
c,d(x)) = σ2
n(ˆ
c
n,x)2+oP(n−1),
whe e ˆ
Πkc
nis he p ojec o on o he subspace spanned by he i s kc
neigen unc ions o Γn, and ˆ
c
n,x =
qPkc
n
j=1 ˆ
λj( c
n(ˆ
λj))2hx, ˆ ji2.
P oo . Rew i ing ˆmc(x) = n−1Pn
i=1 (Pn
j=1 c
n(ˆ
λj)hXi,ˆ jihx, ˆ ji)Yi, hen
EXn( ˆmc(x)) = 1
n
n
X
i=1
n
X
j=1
c
n(ˆ
λj)hXi,ˆ jihx, ˆ ji
hθ, Xii=
n
X
j=1
ˆ
λj c
n(ˆ
λj)hθ, ˆ jihx, ˆ ji
=hˆ
Πkc
nθ, xi+oP(n−1/2),
whe e he las equali y ollows om assump ion (C.2.4) (see Chap e 2, page 37) and Rema k 2.3.8
(see Chap e 2, page 37). Fo he a iance, using Rema k 2.3.8 again, one ge s
Va Xn( ˆmc(x)) = σ2
n2
n
X
i=1
n
X
j=1
c
n(ˆ
λj)hXi,ˆ jihx, ˆ ji
2
=σ2
n
n
X
j1=1
n
X
j2=1
c
n(ˆ
λj1) c
n(ˆ
λj2)hΓnˆ j1,ˆ j2ihx, ˆ j1ihx, ˆ j1i=σ2
n
n
X
j=1
ˆ
λj( c
n(ˆ
λj))2hx, ˆ ji2
=σ2
n(ˆ
c
n,x)2+oP(n−3/2).
104 CHAPTER 4. BOOTSTRAP IN FUNCTIONAL LINEAR REGRESSION
On he o he hand, ˆm∗
c,d(x) = n−1Pn
i=1 (Pn
j=1 c
n(ˆ
λj)hXi,ˆ jihx, ˆ ji)Y∗
i, so one can ep oduce he
easoning which was done o ˆmcand ob ain
EXnYn( ˆm∗
c,d(x)) = 1
n
n
X
i=1
n
X
j=1
c
n(ˆ
λj)hXi,ˆ jihx, ˆ ji
hˆ
θd, Xii=
n
X
j=1
ˆ
λj c
n(ˆ
λj)hˆ
θd,ˆ jihx, ˆ ji
=hˆ
Πkc
n
ˆ
θd, xi+oP(n−1/2) = ˆmd(x) + oP(n−1/2),
whe e he las equali y comes om (C.4.1). Mo eo e , o he nai e boo s ap one has ha
Va XnYn( ˆm∗
c,d(x)) = 1
n2
n
X
i=1
n
X
j=1
c
n(ˆ
λj)hXi,ˆ jihx, ˆ ji
2 1
n
n
X
i=1
(ˆǫi−ˆǫ)2!
=σ2+oP(1)
n
n
X
j=1
ˆ
λj( c
n(ˆ
λj))2hx, ˆ ji2=σ2
n(ˆ
c
n,x)2+oP(n−1),
and simila calcula ions allow o each he same esul when one conside s he wild boo s ap p ocedu e.
4.6.4 Fo mula ion and p oo o Lemma 4.6.3
Lemma 4.6.3. When he assump ions o Theo em 4.3.6 (see page 91) hold, hen
sup
y∈R|φc,d(y)−φc(y)|P
→0,
wi h φc,d and φcde ined in (4.7) (see page 102).
P oo . Fi s o all, i is necessa y o p o e ha o any a∈Rand b > 0
sup
z∈R|Φ(a+bz)−Φ(z)| ≤ |a|+ max(b, b−1)−1.(4.8)
In o de o show (4.8), le zbe a ixed alue z∈R. One has ha
|Φ(a+bz)−Φ(z)| ≤ |Φ(a+bz)−Φ(bz)|+|Φ(bz)−Φ(z)|.(4.9)
I can be ound ha
|Φ(a+bz)−Φ(bz)| ≤ |a|.(4.10)
On he o he hand, |Φ(bz)−Φ(z)| ≤ |b−1|max (b−1,1). The e o e,
|Φ(bz)−Φ(z)| ≤ max (b, b−1)−1.(4.11)
Then, one ge s he inequali y (4.8) eplacing (4.10) and (4.11) in (4.9).
Conside a0=−(EXnYn( ˆm∗
c,d(x)) −ˆmd(x)−EXn( ˆmc(x)) + hˆ
Πkc
nθ, xi)/qVa XnYn( ˆm∗
c,d(x)), b0=
qVa Xn( ˆmc(x))/Va XnYn( ˆm∗
c,d(x)), and z0= (y−√n(EXn( ˆmc(x)) −hˆ
Πkc
nθ, xi))/pnVa Xn( ˆmc(x)).
Subs i u ing hese alues in (4.8), one has
sup
y∈R|φc,d(y)−φc(y)|= sup
z0∈R|Φ(a0+b0z0)−Φ(z0)| ≤ |a0|+ max(b0, b−1
0)−1.
Thus, he con e gence ollows om Lemma 4.6.2 (see page 103).
Chap e 5
Tes ing in unc ional linea
eg ession
As indica ed be o e, unc ional da a ha e been he subjec o many esea ch wo ks o e
he las yea s, being unc ional eg ession one o he mos discussed issues. In his chap e ,
he unc ional linea model wi h scala esponse is conside ed bu , unlike Chap e 3 and
Chap e 4, including an in e cep e m, ha is, Y=hθ, Xi+b+ǫ, whe e Yand ǫa e eal
andom a iables, Xis a andom a iable alued in a sepa able Hilbe space (H,h·,·i),
and he model pa ame e s band θbelong o Rand H, espec i ely. In his con ex ,
a consis en boo s ap me hod o calib a e he dis ibu ion o es s a is ics o assesing
H0:θ= 0 e sus H1:θ6= 0 (i.e., o es ing he lack o dependence) is de eloped, and he
ela ed asymp o ic heo y is p esen ed. Nex , wo linea models, Y1=hθ1, X1i+b1+ǫ1
and Y2=hθ2, X2i+b2+ǫ2, sa is ying ha X1and X2ha e he same co a iance ope a o ,
and ǫ1and ǫ2ha e he same a iance, a e aken. Then, a boo s ap me hod o checking
he equali y o he wo linea models, i.e., o es ing H0:θ1=θ2 e sus H1:θ16=θ2,
is in oduced, and i s asymp o ic p ope ies a e s udied. Finally, a simula ion s udy and
a eal da a example illus a e he pe o mance o he p oposed boo s ap echniques in
p ac ice.
The applica ions o boo s ap calib a ion o es ing in unc ional linea eg ession compiled
in his chap e we e i s ly in oduced in Gonz´alez-Man eiga and Ma ´ınez-Cal o (2010).
Fu he mo e, he asymp o ic heo y o he i s es ing p oblem, i.e., he lack o dependence
es , was de eloped in Gonz´alez-Man eiga e al. (2012)1.
5.1 Tes ing in FDA
Al hough he li e a u e on model cons uc ion and es ima ion me hods o FDA has inc eased consid-
e ably du ing he las yea s, he e a e s ill ew con ibu ions on es ing p ocedu es. This ac makes
his ield an impo an challenge o he s a is ical communi y nowadays. Among he es ing issues
which esea che s al eady s a ed s udying can be highligh ed cu e compa ison p oblems (Cue as
e al., 2004; Delicado, 2007; Hall and Van Keilegom, 2007; Fe a y e al., 2007b; Zhang and Chen,
2007; Be kes e al., 2009; Bugni e al., 2009; Cues a-Albe os and Feb e o-Bande, 2010; Zhang, 2011;
Ho ´a h e al., 2012; Gonz´alez-Rod ´ıguez e al., 2012), hypo hesis es ing on he s uc u e o he dis-
ibu ion o unc ional da a (Viele, 2001; James and Sood, 2006; Hall and Vial, 2006b; Mas, 2007a),
goodness–o – i es s o pa ame ic dis ibu ions (Cues a-Albe os e al., 2007), o es ing a speci ic
o m o he condi ional densi y unc ion (Fe a y e al., 2012c).
1In pa icula , Theo em 5.2.3, Co olla y 5.2.4, Theo em 5.2.5, Theo em 5.2.7, Theo em 5.2.9, Lemma 5.7.1 and
Lemma 5.7.2 we e de eloped by P o . Gil Gonz´alez–Rod ´ıguez, who has allowed o include hem in his hesis in o de
o en ich he heo e ical con en s o his chap e .
105
106 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
O he es ing p oblems analysed in he li e a u e a e es ing p ocedu es in eg ession models wi h
a unc ional componen . In his con ex , es s o no e ec o eg ession o no e ec o any o he
e ms included in he model (Ca do e al., 2003b, 2004; Gadiaga and Ignaccolo, 2005; M¨ulle and
S ad m¨ulle , 2005; An oniadis and Sapa inas, 2007; Ca do e al., 2007d; Kokoszka e al., 2008; Delsol
e al., 2011b; Ho ´a h and Reede , 2011; Hilge e al., 2012; Anei os-P´e ez and Vieu, 2012), as well
as es s o a speci ic pa ame ic o m o he eg ession ope a o (Ca do e al., 2007d; B¨uche e al.,
2011; Delsol e al., 2011b; Ga c´ıa-Po ugu´es e al., 2012) can be emphasized. O he con ibu ions
ocused on es ing he lack o i by examining he esiduals o he eg ession model (Chiou and
M¨ulle , 2007; Gab ys e al., 2010; Pa ilea e al., 2012). Mo eo e , es s o dimension educ ion (Delsol
e al., 2011b), o de ec ing change–poin s in he eg ession ope a o (Ho ´a h and Reede , 2012), o
choosing among wo nes ed linea models (Shen and Fa away, 2004), o o es ing equali y o wo
linea models (Ho ´a h e al., 2009) we e also de eloped.
Many o hese es ing me hods equi e he use o esampling echniques, such as boo s ap, in
o de o be applied in p ac ice since ei he hei asymp o ic heo y has no been de eloped, o i is
di icul o compu e, o i does no exhibi an accep able beha iou o small sample size cases. In
his ega d, he applica ion o boo s ap o he unc ional ield has been success ully ini ia ed (see a
gene al o e iew o esampling me hods o unc ional da a in McMu y and Poli is (2011)). Fo in-
s ance, Poli is and Romano (1994) de i ed heo e ical esul s ha suppo he asymp o ic alidi y o a
s a iona y boo s ap me hod o a b oad class o es ima o s. Cue as e al. (2006) p oposed boo s ap
con idence bands o se e al unc ional es ima o s such as he sample and he immed unc ional
means. In he eg ession ield, Fe a y e al. (2010c), and Gonz´alez-Man eiga and Ma ´ınez-Cal o
(2011) showed he alidi y o he boo s ap in he es ima ion o nonpa ame ic unc ional eg ession
and unc ional linea model, espec i ely, o scala esponse, whe eas he asymp o ic alidi y o a
componen wise boo s ap p ocedu e was p o ed by Fe a y e al. (2012d) when a nonpa ame ic e-
g ession is conside ed and bo h he esponse and he eg esso a e unc ional. On he o he hand, as
ema ked be o e, boo s ap echniques can be also e y help ul o es ing pu poses, since hey may be
used o app oxima e he dis ibu ion o he s a is ic unde he null hypo hesis. Fo example, Cue as
e al. (2004) de eloped a so o pa ame ic boo s ap o ob ain quan iles o a unc ional ANOVA
es , and Gonz´alez-Rod ´ıguez e al. (2012) p o ed he alidi y o a esidual boo s ap in ha con ex .
Hall and Vial (2006b) and, mo e ecen ly, Ba hia e al. (2010) s udied he ini e dimensionali y o
unc ional da a using a boo s ap app oxima ion o independen and dependen samples, espec i ely.
Fu he mo e, in he nonpa ame ic eg ession model wi h unc ional esponse, B¨uche e al. (2011)
p oposed es s o he hypo hesis ha he eg ession unc ion p esen s a speci ic pa ame ic o m, and
cons uc ed hei boo s ap e sions, ob aining es s wi h an accu a e app oxima ion o he nominal
le el o small samples.
This chap e is de o ed o wo es ing p oblems in he unc ional linea model wi h scala esponse:
es ing he lack o dependence and es ing he equali y o wo models. Boo s ap me hods o hese
wo issues a e in oduced in Sec ion 5.2 and Sec ion 5.3, espec i ely. In bo h cases, hei empi ical
beha iou is analysed by means o a simula ion s udy in Sec ion 5.4, and applica ions o eal da ase s
in Sec ion 5.5. Then, a b ie inal discussion and a echnical appendix wi h he p oo s o he esul s
p esen ed in he chap e can be ound in Sec ion 5.6 and Sec ion 5.7.
5.2 Tes o lack o dependence
This sec ion is going o be ocused on he unc ional linea eg ession model wi h scala esponse ha
is desc ibed below. Le (H,h·,·i) be a sepa able Hilbe space, and le k · k be he no m associa ed
wi h i s inne p oduc . Mo eo e , le (Ω,A,P) be a p obabili y space and le (X, Y ) be a measu able
mapping om Ω o H × R, ha is, Xis an H– alued andom elemen whe eas Yis a eal andom
a iable, such ha (X, Y ) e i ies he ollowing linea model wi h scala esponse,
Y=hθ, Xi+b+ǫ, (5.1)
whe e θ∈ H is a ixed unc ional model pa ame e , b∈Ris he in e cep e m, and ǫis a eal andom
a iable such ha E(ǫ) = 0, E(ǫ2) = σ2<∞, and E(ǫX) = 0.
5.2. TEST FOR LACK OF DEPENDENCE 107
The main aim o his sec ion is o de elop a consis en gene al boo s ap esampling app oach o
calib a e he dis ibu ion o s a is ics o es ing he signi icance o he ela ionship be ween Xand Y,
ha is, o es ing H0:θ= 0 e sus H1:θ6= 0, on he basis o a simple andom sample {(Xi, Yi)}n
i=1
d awn om (X, Y ). Boo s ap echniques will become an al e na i e use ul ool when he asymp o ics
o es s a is ics a e unknown o when hey a e no accu a e enough o small sample sizes.
Tes ing he lack o dependence be ween Xand Yhas s i ed up a g ea in e es du ing he las
yea s due o i s p ac ical applica ions in he unc ional con ex . Fo ins ance, Kokoszka e al. (2008)
p oposed a es o lack o dependence in he unc ional linea model wi h unc ional esponse which
was applied o magne ome e cu es consis ing o minu e–by–minu e eco ds o he ho izon al in ensi y
o he magne ic ield measu ed a obse a o ies loca ed a di e en la i ude. The aim was o analyse
i he high–la i ude eco ds had a linea e ec on he mid–la i ude o low–la i ude eco ds. On he
o he hand, Ca do e al. (2007d) p esen ed a s a is ical p ocedu e o check i a eal– alued co a ia e
has an e ec on a unc ional esponse in a nonpa ame ic eg ession con ex , using his me hodology
o a s udy o a mosphe ic adia ion. In his case, he da ase we e adia ion p o iles cu es measu ed
a a andom ime and he au ho s es ed i he adia ion p o iles changed along he ime.
Some con ibu ions ha e s udied simila es s in o he unc ional eg ession con ex s. Fo ins ance,
in he case o scala esponse and unc ional p edic o , M¨ulle and S ad m¨ulle (2005) analysed he
gene alized unc ional linea eg ession model and es ed whe he he p edic o unc ion has any in lu-
ence on he ou come; Ho ´a h and Reede (2011) s udied a quad a ic unc ional eg ession model and
es ed he signi icance o he nonlinea e m in he model; Anei os-P´e ez and Vieu (2012) assessed he
linea componen o a unc ional pa ially linea model; and Gadiaga and Ignaccolo (2005) and Delsol
e al. (2011b) checked i he explana o y a iable has an e ec on he esponse using nonpa ame ic
me hods. Mo eo e , An oniadis and Sapa inas (2007) p o ided p ocedu es o es ing whe he ce ain
ixed–e ec s unc ional componen s o he andom–e ec s unc ional componen s a e equal o ze o in
a gene al unc ional mixed–e ec s model wi h unc ional esponse.
Rega ding he eg ession model (5.1), es ing he signi icance o he ela ionship be ween he unc-
ional co a ia e and he scala esponse has been he subjec o ecen wo ks, and asymp o ic ap-
p oaches o his p oblem can be ound in Ca do e al. (2003b, 2004), Kokoszka e al. (2008) o
Hilge e al. (2012) (Kokoszka e al. (2008) s udied indeed he mo e gene al si ua ion o unc ional
esponse). The me hods p esen ed in hese pape s a e mainly based on he calib a ion o he s a is ics
dis ibu ion by using asymp o ic dis ibu ion app oxima ions. In con as , a consis en boo s ap cal-
ib a ion is p oposed in o de o app oxima e he s a is ics dis ibu ion unde H0. Fo ha pu pose,
some no a ion and basic concep s abou he eg ession model (5.1), he asymp o ic heo y o he
es ing p ocedu e, and he consis ency o he boo s ap echniques ha a e p oposed, a e in oduced
in Sec ion 5.2.1. In Sec ion 5.2.2, he boo s ap calib a ion is p esen ed as an al e na i e o he asymp-
o ic heo y p e iously exposed. La e , in Sec ion 5.4.1 and Sec ion 5.5.1, a simula ion s udy and a
eal da a applica ion allow o show he pe o mance o he boo s ap me hodology in compa ison wi h
he asymp o ic app oach.
5.2.1 Asymp o ic heo y o es ing and boo s ap p ocedu es
Theo e ical backg ound
Riesz Rep esen a ion Theo em ensu es ha he unc ional linea model wi h scala esponse can be
handled heo e ically wi hin he conside ed amewo k. Speci ically, le Hbe he sepa able Hilbe
space o squa e Lebesgue in eg able unc ions on a gi en compac se C⊂R, deno ed by L2(C, λ),
wi h he usual inne p oduc and he associa ed no m k·k. The unc ional linea model wi h scala
esponse be ween a andom unc ion Xand a eal andom a iable Yis de ined as
Y= Ψ(X) + ǫ, (5.2)
whe e Ψ is a con inuous linea ope a o , i.e., Ψ ∈ H′being H′ he dual space o Hwi h associa ed no m
k·kH′(see Sec ion 1.2.2, “ b) The dual space H′”, in Chap e 1, page 10), and ǫis a eal cen ed andom
a iable wi h ini e a iance and independen o X. In i ue o Riesz Rep esen a ion Theo em, Hand
114 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
kˆ
θknk2=Pkn
j=1 (ˆ
λ−1
j∆nˆ j)2is an es ima o o kθk2, in o de o es he lack o dependence be ween
Xand Yone may conside he s a is ic
TInd
2,n =
kn
X
j=1 ∆nˆ j
ˆ
λj!2
.(5.10)
Un o una ely, he limi dis ibu ion o (5.10) is cu en ly unknown, so u he esea ch abou his
s a is ic is necessa y in o de o ob ain i . Meanwhile, i s asymp o ic dis ibu ion can be calib a ed
using he boo s ap app oach p oposed below.
Finally, ano he s a is ic is he one based on ideas p oposed h oughou Sec ion 5.2.1, ha is,
TInd
3,n =
1
n
n
X
i=1
(Xi−X)(Yi−Y),(5.11)
which will be deno ed by F– es om now on, since i is he na u al gene aliza ion o he well–known
F– es in he ini e dimensional con ex . Ano he possibili y is o conside he s uden ized e sion o
(5.11) de ined by
TInd
3s,n =1
ˆσ
1
n
n
X
i=1
(Xi−X)(Yi−Y),(5.12)
whe e ˆσ2is a consis en es ima o o σ2(a leas unde H0), o ins ance, he empi ical es ima ion o
σ2( ecall ha s uden ized e sions we e also compu ed o con idence in e als; see (4.5) and (4.6) in
Chap e 4, page 93).
In gene al, o he s a is ics such as (5.9), (5.10), (5.11) and (5.12), he calib a ion o he dis ibu ion
can be ob ained by boo s ap. Fo ins ance, o (5.11) and (5.12), he ollowing boo s ap s a is ics
can be conside ed
TInd,∗
3,n =
1
n
n
X
i=1
(Xi−X)(Yi−Y)ǫ∗
i,(5.13)
TInd,∗
3s,n =1
ˆσ∗
1
n
n
X
i=1
(Xi−X)(Yi−Y)ǫ∗
i,(5.14)
whe e {ǫ∗
i}n
i=1 and ˆσ∗a e buil ollowing he wild boo s ap app oach desc ibed in Algo i hm 5.2.8
(see page 112). Fu he mo e, in he p e ious sec ion, bo h nai e and wild boo s ap we e shown o be
consis en o he F– es . This ac gua an ees om a heo e ical poin o iew ha he dis ibu ion o
TInd
3,n and TInd
3s,n can be app oxima ed by he co esponding boo s ap dis ibu ion o (5.13) and (5.14),
and H0can be ejec ed when he app oxima ed p– alue o he s a is ic is smalle han α.
A simila boo s ap calib a ion is p oposed o he es s based on TInd
1,n and TInd
2,n . Al hough he
consis ency o he boo s ap p ocedu e in hese cases has no been p o ed in his chap e . Fo (5.9),
wo boo s ap s a is ics a e buil
TInd,∗(a)
1,n =1
√kn
n
ˆσ∗2
kn
X
j=1
(∆∗
nˆ j)2
ˆ
λj−kn
,(5.15)
TInd,∗(b)
1,n =1
√kn
n
ˆσ2
kn
X
j=1
(∆∗
nˆ j)2
ˆ
λj−kn
.(5.16)
The di e ence be ween he wo p oposed boo s ap app oxima ions is ha in he i s one he es ima-
ion o σ2, deno ed by ˆσ∗2, is compu ed using he boo s ap sample gene a ed in each i e a ion. On
he o he hand, o (5.10), he p oposed boo s ap s a is ic is
TInd,∗
2,n =
kn
X
j=1 ∆∗
nˆ j
ˆ
λj!2
.(5.17)
5.3. TEST FOR EQUALITY OF LINEAR MODELS 115
In o de o ob ain he boo s ap dis ibu ion o (5.15), (5.16) and (5.17), he ollowing wild boo s ap
algo i hm can be used.
Algo i hm 5.2.10 (Wild boo s ap).
S ep 1. Compu e he alue o he s a is ic TInd
1,n (o he alue o he s a is ic TInd
2,n ).
S ep 2. D aw {ǫ∗
i}n
i=1 a sequence o i.i.d. andom elemen s d awn om ǫ∗, which sa is ies E(ǫ∗) = 0,
E((ǫ∗)2) = 1 and R∞
0(P(|ǫ∗|> )1/2)<∞, and de ine Y∗
i=Yiǫ∗
i o all i= 1,...,n.
S ep 3. Build ∆∗
n=n−1Pn
i=1 Xi⊗H′Y∗
i, and compu e an=|TInd,∗
1,n |(o compu e bn=|TInd,∗
2,n |).
S ep 4. Repea S eps 2 and 3 a la ge numbe o imes nboo ∈Nin o de o ob ain a sequence o
alues {al
n}nboo
l=1 (o a sequence o alues {bl
n}nboo
l=1 ).
S ep 5. App oxima e he p– alue o he es by he p opo ion o alues in {al
n}nboo
l=1 g ea e han o
equal o |TInd
1,n |(o by he p opo ion o alues in {bl
n}nboo
l=1 g ea e han o equal o |TInd
2,n |).
5.3 Tes o equali y o linea models
As s a ed in he p e ious sec ion, le (H,h·,·i) be a sepa able Hilbe space (being k·k i s associa ed
no m), and le (Ω,A,P) be a p obabili y space. In his case, le (X1, Y1) and (X2, Y2) be wo measu able
mappings om Ω o H × R(i.e., X1and X2a e H– alued andom elemen s and Y1and Y2a e eal
andom a iables) such ha bo h o hem sa is y he unc ional linea model wi h scala esponse as
ollows Y1=hθ1, X1i+b1+ǫ1,
Y2=hθ2, X2i+b2+ǫ2,(5.18)
whe e θ1, θ2∈ H a e he ixed unc ional model pa ame e s, b1, b2∈Ra e he in e cep e ms, and ǫ1
and ǫ2a e eal andom a iables such ha E(ǫ1) = E(ǫ2) = 0, E(ǫ2
1) = σ2
1<∞,E(ǫ2
2) = σ2
2<∞, and
E(ǫ1X1) = E(ǫ2X2) = 0.
This sec ion is ocused on he in oduc ion o a boo s ap calib a ion p ocedu e o app oxima e he
dis ibu ion o s a is ics o es ing he equali y o wo unc ional linea models wi h scala esponse,
ha is, o es ing H0:θ1=θ2 e sus H1:θ16=θ2. Simple andom samples {(X1,i, Y1,i)}n1
i=1 and
{(X2,i, Y2,i)}n2
i=1 d awn om (X1, Y1) and (X2, Y2), which a e assumed o be independen , will be used
in o de o achie e his objec i e. No e ha boo s ap me hods can be an especially in e es ing op ion
when he asymp o ic dis ibu ion o a es s a is ic is ha d o compu e o i has no an app op ia e
beha iou o small sample sizes.
The p oblem o checking he equali y o wo unc ional linea models has ba ely been s udied in
FDA li e a u e. In his sense, he pape by Ho ´a h e al. (2009) is he mos no ewo hy con ibu ion
o his issue. The au ho s compa ed wo unc ional linea models in which explana o y a iables
a e cu es and esponses can be ei he scala s o cu es, and hey es ed he null hypo hesis ha
he wo eg ession ope a o s a e he same. The p oposed es s a is ics ha e asymp o ic chi–squa ed
dis ibu ion. Hence, hey used his dis ibu ion in p ac ice, when hey applied hei me hodology o
he wo eal da a applica ions conside ed in hei wo k.
In his sec ion, a es s a is ics o assesing he equali y o he linea models in (5.18) is p oposed
and a boo s ap algo i hm is in oduced o app oxima e i s dis ibu ion. Sec ion 5.3.1 compiles heo-
e ical backg ound o he eg ession model (5.18), desc ibes he es ing p ocedu e and i s asymp o ic
p ope ies, and p esen s he boo s ap echniques and hei associa ed consis ency and co ec ness
esul s. Nex , Sec ion 5.3.2 is de o ed o compa e he boo s ap calib a ion and he asymp o ic one.
Recall ha a simula ion s udy and a eal da a applica ion ela ed o his es ing p oblem can be ound
in Sec ion 5.4.2 and Sec ion 5.5.2, espec i ely.
116 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
5.3.1 Asymp o ic heo y o es ing and boo s ap p ocedu es
Theo e ical backg ound
Fi s o all, as i was done o he es o lack o dependence, he special case whe e H=L2(C, λ), i.e.,
he sepa able Hilbe space o squa e Lebesgue in eg able unc ions on a gi en compac se C⊂Rwi h
he usual inne p oduc and no m, will be analysed om a heo e ical iewpoin . In his si ua ion, he
linea dependence o he scala esponses Y1and Y2on he unc ional andom p edic o s X1and X2
is modelled by Y1= Ψ1(X1) + ǫ1,
Y2= Ψ2(X2) + ǫ2,(5.19)
whe e Ψ1,Ψ2∈ H′a e con inuous linea ope a o s ( o u he in o ma ion abou he dual space H′
and i s associa ed no m, see Sec ion 1.2.2, “ b) The dual space H′”, in Chap e 1, page 10), and
ǫ1and ǫ2a e ze o–mean eal andom a iables wi h ini e a iance and independen o X1and X2,
espec i ely. Riesz Rep esen a ion Theo em s a es ha Hand H′a e isome ically iden i ied, and
he e exis unique θ1, θ2∈ H sa is ying ha kθ1k=kΨ1kH′,kθ2k=kΨ2kH′, and Ψ1(x) = hθ1, xiand
Ψ2(x) = hθ2, xi o all x∈ H. This ac ensu es ha (5.19) can be seen as a pa icula case o (5.18).
Ho ´a h e al. (2009) assumed ha he in e cep e ms b1and b2a e equal o ze o. The in e cep
e ms in (5.18) can be embedded in he a iable coun e pa o he models i one de ines ˜
X1= (X1,1),
˜
X2= (X2,1), ˜
θ1= (θ1, b1)∈ Heand ˜
θ2= (θ2, b2)∈ He, whe e Heis he p oduc space H × R
wi h he co esponding inne p oduc h·,·ie. Hence, he wo models in (5.18) can be exp essed as
Y1=h˜
θ1,˜
X1ie+ǫ1and Y2=h˜
θ2,˜
X2ie+ǫ2, espec i ely. In his case, nei he ˜
X1no ˜
X2can be
assumed o be ze o–mean andom elemen s. Gi en ha es ing θ1=θ2is no equi alen o es ing
˜
θ1=˜
θ2, since he in e cep e ms canno be assumed o be ze o in p ac ice, bo h b1and b2has been
w i en explici ly in he ollowing. Fu he mo e, Ho ´a h e al. (2009) also supposed ha X1and X2
a e mean ze o andom elemen s. Gi en ha his condi ion could be oo es ic i e in some p ac ical
cases, he p edic o a iables a e no assumed o be cen ed in he ollowing heo e ical esul s.
Equali y o wo linea models es
Gi en he wo linea models de ined in (5.18) (see page 115), he aim is o build a co ec and consis en
boo s ap me hod ha allows es ing
H0:θ1=θ2
H1:θ16=θ2(5.20)
by means o wo independen andom samples, deno ed by {(X1,i, Y1,i)}n1
i=1 and {(X2,i, Y2,i)}n2
i=1, o
i.i.d. andom elemen s d awn om (X1, Y1) and (X2, Y2), espec i ely. In o de o ob ain he heo e -
ical esul s compiled below, i will be assumed ha he wo samples ha e he same size, i.e.,
(C.5.3) n1=n2=n, wi h n→ ∞.
Rema k 5.3.1.Assump ion (C.5.3), which is a se e e es ic ion, has been in oduced in o de o
simpli y he no a ion and he calcula ions in ol ed in he heo e ical esul s o his sec ion. Howe e ,
he de elopmen s in he p oo s seem o indica e ha (C.5.3) could be eplaced by assuming ha
he e exis s a cons an 0 < c < ∞such ha n1/n2→cwhen n1, n2→ ∞. Ne e heless, addi ional
messy no a ion and edious wo k will be su ely equi ed o his. Due o his ac , (C.5.3) has been
conside ed in his chap e , al hough al e na i e (and less es ic i e) assump ions will be s udied in
de ail in u u e esea ch.
F om now on, assume ha E(kX1k2)<∞and E(kX2k2)<∞. Consequen ly, E(Y2
1)<∞and
E(Y2
2)<∞by he H¨olde ’s inequali y. Fu he mo e, suppose ha he co a iance ope a o o X1and
he co a iance ope a o X2a e equal, and he e o s ǫ1and ǫ2ha e equal a iances, ha is,
(C.5.4) E((X1−µX1)⊗H(X1−µX1)) = E((X2−µX2)⊗H(X2−µX2)) = ΓX,
and σ2
1=σ2
2=σ2,
5.3. TEST FOR EQUALITY OF LINEAR MODELS 117
whe e µX1∈ H and µX2∈ H deno e he expec ed alues o X1and X2, espec i ely. Whene e
he e is no possible con usion, ΓXwill be abb e ia ed as Γ. As usual, {(λj, j)}∞
j=1 will deno e he
eigen alues and eigen unc ions o Γ, whe e he eigen alues a e assumed o be a anged in dec easing
o de (λ1≥λ2≥...). Mo eo e , he c oss–co a iance ope a o be ween X1and Y1, and he c oss–
co a iance ope a o be ween X2and Y2a e gi en by
∆X1,Y1=E((X1−µX1)⊗H′(Y1−µY1)) and ∆X2,Y2=E((X2−µX2)⊗H′(Y2−µY2))
whe e µX1, µX2∈ H deno e he expec ed alue o X1and X2, and µY1, µY2∈Rdeno e he expec ed
alue o Y1and Y2. The c oss–co a iance ope a o s ∆X1,Y1and ∆X2,Y2will be deno ed by ∆1and ∆2,
espec i ely, in o de o simpli y he no a ion. I can be shown ha ∆1,∆2∈ H′and, in addi ion,
∆1x=hθ1,Γxiand ∆2x=hθ2,Γxi o all x∈ H. Thus, he ollowing ela ion be ween he co a iance
ope a o , he c oss–co a iance ope a o s and he eg ession pa ame e s is sa is ied
(∆1−∆2)x=hθ1−θ2,Γxi,∀x∈ H.(5.21)
As p e iously commen ed in his chap e , he Hilbe space Hcan be seen as he di ec sum o
wo o hogonal subspaces induced by he sel –adjoin ope a o Γ: he ke nel o Γ deno ed by Ke (Γ),
and he closu e o he image o Γ deno ed by Im(Γ). The e o e, θ1and θ2a e de e mined uniquely by
θ1=θ1,1+θ1,2and θ2=θ2,1+θ2,2, whe e θ1,1, θ2,1∈Ke (Γ) and θ1,2, θ2,2∈Im(Γ). I can be shown
ha Va (hθ1,1, X1i) = Va (hθ2,1, X2i) = 0, and as a esul (5.18) can be ew i en as
Y1=hθ1,2, X1i+hθ1,1, µX1i+b1+ǫ1,
Y2=hθ2,2, X2i+hθ2,1, µX2i+b2+ǫ2.
Consequen ly, i will be un easible o es whe he θ1,1=θ2,1o no , since hθ1,1, µX1iis indis inguish-
able om he in e cep e m b1and, analogously, hθ2,1, µX2iis indis inguishable om b2. Consequen ly,
he hypo hesis es will be es ic ed o check
H0:θ1,2=θ2,2
H1:θ1,26=θ2,2.(5.22)
Fu he mo e, due o (5.21), one ge s ha θ1,2=θ2,2i , and only i , ∆1x= ∆2x o all x∈ H. Hence,
he hypo hesis es in (5.22) is equi alen o
H0:k∆1−∆2kH′= 0
H1:k∆1−∆2kH′6= 0.(5.23)
Rema k 5.3.2.Ho ´a h e al. (2009) assumed ha µX1=µX2= 0. This assump ion join ly wi h he
p e ious easoning imply ha nei he θ1,1no θ2,1can be es ima ed based on he in o ma ion p o ided
by X1and X2, so he hypo hesis es ing is also es ic ed o (5.22) (o , equi alen ly, o (5.23)).
Rema k 5.3.3.I b1=b2= 0 and bo h µX1and µX2a e di e en om ze o, one could es i θ1,1=θ2,1
by checking whe he he in e cep e ms in he models a e equal o no . Howe e , his is s ill an open
p oblem in he li e a u e, which will no be sol ed in his sec ion since i has been ocused on he
es ic ed es (5.22) and no on he un es ic ed es (5.20).
Tes ing p ocedu e and asymp o ic heo y
I is s aigh o wa d o show ha k∆1−∆2kH′can be exp essed depending on he H– alued andom
elemen
TEq =E((X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2))
as ollows
k∆1−∆2kH′=kE((X1−µX1)⊗H′(Y1−µY1)−(X2−µX2)⊗H′(Y2−µY2))kH′
=kE((X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2))k=kTEqk.
118 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
Then, gi en wo independen andom samples {(X1,i, Y1,i)}n
i=1 and {(X2,i, Y2,i)}n
i=1 o i.i.d. andom
elemen s d awn om (X1, Y1) and (X2, Y2), espec i ely, kTEqkcan be es ima ed by means o i s
empi ical coun e pa kTEq
nk, whe e TEq
nis he ollowing H– alued andom elemen
TEq
n=1
n
n
X
i=1
((X1,i −X1)(Y1,i −Y1)−(X2,i −X2)(Y2,i −Y2)),
wi h X1=n−1Pn
i=1 X1,i,X2=n−1Pn
i=1 X2,i,Y1=n−1Pn
i=1 Y1,i, and Y2=n−1Pn
i=1 Y2,i. Some
p ope ies o TEq
na e s a ed in he nex heo em and co olla y, being he nex assump ion necessa y:
(C.5.5) E(kX1k4)<∞and E(kX2k4)<∞.
Fu he mo e, ecall ha , gi en an H– alued andom elemen Hsuch ha E(kHk2)<∞,ZHdeno es
a cen ed Gaussian elemen in Hwi h co a iance ope a o ΓH.
Theo em 5.3.4. Assuming ha (5.18),(C.5.3),(C.5.4) and (C.5.5) hold, hen
(i) E(TEq
n) = n−1(n−1)TEq.
(ii) TEq
ncon e ges a.s. −P o TEq as n→ ∞.
(iii) √n(TEq
n−TEq)con e ges in law, as n→ ∞, o Z(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)wi h co-
a iance ope a o 2σ2Γ + E(hθ1, X1−µX1i2(X1−µX1)⊗H(X1−µX1))+E(hθ2, X2−µX2i2(X2−
µX2)⊗H(X2−µX2)).
Co olla y 5.3.5. Unde he assump ions o Theo em 5.3.4, i he null hypo hesis in (5.23) is sa is ied
(i.e., k∆1−∆2kH′= 0), hen √n T Eq
ncon e ges in law o Z(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)wi h
co a iance ope a o 2σ2Γ+E(hθ1, X1−µX1i2(X1−µX1)⊗H(X1−µX1))+E(hθ1, X2−µX2i2(X2−µX2)⊗H
(X2−µX2)), and consequen ly, k√n TEq
nkcon e ges in law o kZ(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)k.
The p oo s o he p e ious esul s can be ound in he appendix o he chap e (see Sec ion 5.7.8,
page 137, and Sec ion 5.7.9, page 137, espec i ely).
Al hough he asymp o ic null dis ibu ion o k√n TEq
nkcanno be explici ly calcula ed om Co ol-
la y 5.3.5, his s a is ic could be used in p ac ice app oxima ing i s dis ibu ion by means o boo s ap
me hods. Ne e heless, an al e na i e s a is ic will be conside ed in he simula ion s udy o compa -
a i e pu poses.
Local al e na i es. To inish he asymp o ic analysis o he s a is ic k√n T Eq
nk, i is necessa y o
s udy i s beha iou unde local al e na i es. Fo his pu pose, i will be assumed ha θ1, θ2∈ H
sa is y ha kθ1,2k>0 and θ2= (1 −δn/√n)θ1, whe e δnis a posi i e sequence such ha (C.5.2)
holds (see page 110). Then, he modi ied andom sample
Y1,i =hθ1, X1,ii+b1+ǫ1,i,∀i∈ {1,...,n},
Yn
2,i =h(1 −δn/√n)θ1, X2,ii+b2+ǫ2,i,∀i∈ {1, . . . , n},
can be conside ed. Ob iously, H0is no e i ied in his case. Howe e , kθ1−(1 −δn/√n)θ1k=
δnkθ1k/√n→0, so he null hypo hesis is app oached a a e δn/√n. The beha iou o he p oposed
s a is ic unde his kind o local al e na i es is s a ed in he ollowing heo em.
5.3. TEST FOR EQUALITY OF LINEAR MODELS 119
Theo em 5.3.6. Unde he assump ions o Theo em 5.3.4, and wi h he abo e no a ion, i (C.5.2)
holds, hen
P √n 1
n
n
X
i=1
((X1,i −X1)(Y1,i −Y1)−(X2,i −X2)(Yn
2,i −Yn
2))!≤ !→0,∀ ∈R
as n→ ∞.
I s p oo is collec ed in Sec ion 5.7.10 (see page 138).
Boo s ap p ocedu es and consis ency
As an al e na i e o using he asymp o ic dis ibu ions in oduced in he p e ious sec ion, boo s ap
p ocedu es a e p oposed below in o de o check he null hypo hesis in (5.23).
Fi s o all, le √n(TEq
n−TEq) be he s a is ic gi en by
√n(TEq
n−TEq) = √n 1
n
n
X
i=1
((X1,i −X1)(Y1,i −Y1)−(X2,i −X2)(Y2,i −Y2)
−E((X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)))!.
(5.24)
By Theo em 5.3.4, he p e ious s a is ic con e ges in law o a cen ed Gaussian elemen wi h co a iance
ope a o 2σ2Γ + E(hθ1, X1−µX1i2(X1−µX1)⊗H(X1−µX1)) + E(hθ2, X2−µX2i2(X2−µX2)⊗H
(X2−µX2)), ega dless H0holds o no . In addi ion, i H0is sa is ied, Co olla y 5.3.5 ensu es ha
(5.24) con e ges in law o a cen ed Gaussian elemen wi h co a iance ope a o 2σ2Γ + E(hθ1, X1−
µX1i2(X1−µX1)⊗H(X1−µX1)) + E(hθ1, X2−µX2i2(X2−µX2)⊗H(X2−µX2)).
Consequen ly, his s a is ic seems o be a good be o being mimicked by a boo s ap one. The
consis ency and co ec ness o he boo s ap s a is ic will be gua an eed whene e i s limi dis ibu ion
i espec i ely o H0and i s limi dis ibu ion unde H0coincide wi h he abo e–men ioned asymp o ic
dis ibu ions. Nex , a nai e pai ed boo s ap and a wild boo s ap a e p oposed and hei asymp o ic
p ope ies a e s udied.
Nai e boo s ap. Le {(X∗
1,i, Y ∗
1,i)}n
i=1 and {(X∗
2,i, Y2,i)∗}n
i=1 wo independen collec ions o i.i.d.
andom elemen s d awn a andom om (X1, Y1) and (X2, Y2), espec i ely. Then, he nex nai e
pai ed boo s ap s a is ic can be conside ed
TEq,N∗
n=1
n
n
X
i=1
((X∗
1,i −X∗
1)(Y∗
1,i −Y∗
1)−(X∗
2,i −X∗
2)(Y∗
2,i −Y∗
2)
−(X1,i −X1)(Y1,i −Y1) + (X2,i −X2)(Y2,i −Y2)).
The ollowing algo i hm desc ibes in de ail he nai e boo s ap app oach.
Algo i hm 5.3.7 (Nai e boo s ap).
S ep 1. Compu e he alue o he s a is ic TEq
n.
S ep 2. D aw {(X∗
1,i, Y ∗
1,i)}n
i=1 and {(X∗
2,i, Y2,i)∗}n
i=1, wo independen sequences o i.i.d. andom
elemen s chose a andom om he ini ial samples (X1, Y1)and (X2, Y2), espec i ely, and
compu e an=kTEq,N∗
nk.
S ep 3. Repea S ep 2 a la ge numbe o imes nboo ∈Nin o de o ob ain a sequence o alues
{al
n}nboo
l=1 .
120 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
S ep 4. App oxima e he p– alue o he es by he p opo ion o alues in {al
n}nboo
l=1 g ea e han o
equal o kTEq
nk.
The asymp o ic consis ency and co ec ness o he nai e boo s ap app oach is s a ed in he ollowing
heo em.
Theo em 5.3.8. Unde he assump ions o Theo em 5.3.4, i holds ha √n TEq,N∗
ncon e ges in law
o Z(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)a.s. −P.
The p oo o Theo em 5.3.8 can be ound in Sec ion 5.7.11 (see page 138).
Wild boo s ap. Le {ǫ∗
i}n
i=1 be i.i.d. cen ed eal andom a iables, independen o {(X1,i, Y1,i)}n
i=1
and {(X2,i, Y2,i)}n
i=1, such ha E((ǫ∗
i)2) = 1 and R∞
0P(|ǫ∗
1|> )1/2d < ∞( o ins ance, E((ǫ∗
i)d)<∞
o ce ain d > 2 ensu es he second condi ion). Then, a wild boo s ap s a is ic can be de ined as
ollows
TEq,W ∗
n=1
n
n
X
i=1
((X1,i −X1)(Y1,i −Y1)−(X2,i −X2)(Y2,i −Y2))ǫ∗
i.
Thus, he wild boo s ap app oach can be applied by means o he ollowing algo i hm.
Algo i hm 5.3.9 (Wild boo s ap).
S ep 1. Compu e he alue o he s a is ic TEq
n.
S ep 2. D aw {ǫ∗
i}n
i=1 a sequence o i.i.d. andom elemen s d awn om a eal andom a iable ǫ∗
independen o {(X1,i, Y1,i)}n
i=1 and {(X2,i, Y2,i)}n
i=1, which sa is ies E(ǫ∗) = 0,E((ǫ∗)2) = 1
and R∞
0(P(|ǫ∗|> )1/2)<∞, and compu e an=kTEq,W ∗
nk.
S ep 3. Repea S ep 2 a la ge numbe o imes nboo ∈Nin o de o ob ain a sequence o alues
{al
n}nboo
l=1 .
S ep 4. App oxima e he p– alue o he es by he p opo ion o alues in {al
n}nboo
l=1 g ea e han o
equal o kTEq
nk.
The asymp o ic beha iou o he wild boo s ap s a is ic is analysed in he ollowing heo em.
Theo em 5.3.10. Unde he assump ions o Theo em 5.3.4, i holds ha √n T Eq,W ∗
ncon e ges in
law o Z(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)a.s. −P.
See he p oo o he p e ious heo em in Sec ion 5.7.12, page 139.
5.3.2 Boo s ap calib a ion s. asymp o ic heo y
F om now on, i is assumed ha he model (5.18) (see page 115) holds wi h b1=b2= 0 and µX1=
µX2= 0 ( hus, µY1=µY2= 0). Then, di e en es s a is ics can be conside ed in o de o check i
he wo linea models a e equal o no .
Fi s o all, no e ha Ho ´a h e al. (2009) de eloped se e al me hods o compa ing wo linea
models wi h unc ional explana o y a iables and scala esponses, based on es s a is ics wi h chi–
squa ed asymp o ic dis ibu ion, when H=L2([0,1]). This me hodology can be applied o he case
analysed he e i (C.5.3),(C.5.4) and (C.5.5) hold. Fu he mo e, i has o be assumed ha he e o s
sa is y ha E(ǫ4
1)<∞and E(ǫ4
2)<∞, and he i s keigen alues o he common co a iance ope a o
Γ a e nonze o and dis inc , ha is, λ1> λ2> . . . > λk>0. In his si ua ion, le {(ˆ
λj,ˆ j)}∞
j=1 be he
eigenelemen s o he empi ical co a iance ope a o Γncompu ed wi h espec o he pooled sample, i.e.,
Γn= (2n)−1Pn
i=1 (X1,i ⊗HX1,i +X2,i ⊗HX2,i). Gi en he se o empi ical eigen unc ions {ˆ j}∞
j=1,
5.3. TEST FOR EQUALITY OF LINEAR MODELS 121
which is an o hono mal basis o L2([0,1]), le X1and X2be he n×k–ma ices gi en by (X1)i,j =
hX1,i,ˆ jiand (X2)i,j =hX2,i,ˆ ji. In addi ion, le Y1and Y2be he n– ec o s gi en by Y1=
(Y1,1,...,Y1,n) and Y2= (Y2,1,...,Y2,n) , and le ˆ
Σkbe he k×k–ma ix gi en by ( ˆ
Σk)j,j =ˆ
λ−1
j o
all j∈ {1,...,k}, and ( ˆ
Σk)j1,j2= 0 o all j1, j2∈ {1,...,k}such ha j16=j2. Then, he easoning
in Ho ´a h e al. (2009) led o he ollowing es s a is ic
TEq
1,n =n
2ˆσ2(ˆ
C1−ˆ
C2) diag(ˆ
λ1,...,ˆ
λk)( ˆ
C1−ˆ
C2),(5.25)
whe e ˆ
C1= (X
1X1)−1X
1Y1and ˆ
C2= (X
2X2)−1X
2Y2, and ˆσ2a e esidual s anda d de ia ions
om he es ima ed eg ession models compu ed wi h espec o he wo samples oge he , i.e., ˆσ2=
(2n−k)−1Pn
i=1 ((Y1,i −ˆ
Y1,i)2+ (Y2,i −ˆ
Y2,i)2). Ho ´a h e al. (2009) showed ha he dis ibu ion o
(5.25) can be app oxima ed by he chi–squa ed dis ibu ion wi h kdeg ees o eedom. Thus, H0is
ejec ed i TEq
1,n > q1−αbeing qα he α–quan ile o a χ2
k. Finally, no e ha es s a is ics p oposed by
Ho ´a h e al. (2009) can be applied o check H0:kθ1−θ2k= 0 e sus H1:kθ1−θ2k 6= 0 in mo e
gene al si ua ions ha he speci ic case which is s udied in his chap e such as: when he co a iance
ope a o s o X1and X2a e no equal, when σ1and σ2a e di e en , when he wo samples ha e
di e en sizes o oughly he same o de (i.e., n16=n2such ha n1/n2→cas n1, n2→ ∞, wi h
0< c < ∞), o when he esponse a iables Y1and Y2a e unc ional.
Ano he choice o check whe he kθ1−θ2k= 0 o no is based on he s anda d FPCA es ima o
in oduced in Sec ion 2.3.2, “ a) De ini ion o s anda d FPCA es ima o ”, in Chap e 2 (see page 35).
Recall ha , unde (C.2.1) (see Chap e 2, page 36), he model pa ame e s θ1and θ2can be exp essed
as θ1=P∞
j=1 λ−1
j∆1 j jand θ2=P∞
j=1 λ−1
j∆2 j j, espec i ely, whene e (C.5.4) holds. The e-
o e, kθ1−θ2k2=kP∞
j=1 λ−1
j(∆1 j−∆2 j) jk2=P∞
j=1 λ−2
j(∆1 j−∆2 j)2. The model pa ame e s
can be es ima ed by he s anda d FPCA es ima o (see (2.4) in Chap e 2, page 36), so he no m
o hei di e ence can be app oxima ed by kˆ
θ1,kn−ˆ
θ2,knk2=kPkn
j=1 ˆ
λ−1
j(∆1,nˆ j−∆2,nˆ j)ˆ jk2=
Pkn
j=1 ˆ
λ−2
j(∆1,nˆ j−∆2,nˆ j)2, whe e ∆1,n =n−1Pn
i=1 X1,i ⊗H′Y1,i, ∆2,n =n−1Pn
i=1 X2,i ⊗H′Y2,i,
{(ˆ
λj,ˆ j)}∞
j=1 a e he eigenelemen s o Γn= (2n)−1Pn
i=1 (X1,i ⊗HX1,i +X2,i ⊗HX2,i), and {kn}∞
n=1
is a sequence o posi i e in ege s such ha kn→+∞,kn≤n, and ˆ
λkn>0. Hence, he nex es
s a is ic can be conside ed
TEq
2,n =
kn
X
j=1 ∆1,nˆ j−∆2,nˆ j
ˆ
λj!2
.(5.26)
Since he asymp o ic dis ibu ion o (5.26) is no known, a boo s ap app oach is equi ed o use his
s a is ic in p ac ice.
Finally, he de elopmen s p esen ed in Sec ion 5.3.1 lead o p opose as es s a is ic
TEq
3,n =
1
n
n
X
i=1
((X1,i −X1)(Y1,i −Y1)−(X2,i −X2)(Y2,i −Y2)).(5.27)
The calib a ion o he dis ibu ion o (5.25), (5.26) and (5.27) unde H0can be done by means o
boo s ap echniques. Fo (5.27), he boo s ap s a is ics de ined as
TEq,∗
3,n =
1
n
n
X
i=1
((X1,i −X1)(Y1,i −Y1)−(X2,i −X2)(Y2,i −Y2))ǫ∗
i(5.28)
can be conside ed. Hence, he p– alues o TEq
3,n can be app oxima ed using he wild boo s ap app oach
desc ibed in Algo i hm 5.3.9 (see page 120), which is consis en based on he heo e ical esul s in o-
duced in he p e ious sec ion. The e o e, H0is ejec ed when he app oxima ed p– alue o (5.27) is
smalle han α.
Fo he s a is ics TEq
1,n and TEq
2,n ano he wild boo s ap p ocedu e is p oposed, al hough i s con-
sis ency and co ec ness has no been p o ed in his chap e . The boo s ap s a is ics o (5.25) a e
122 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
gi en by he ollowing exp essions
TEq,∗(a)
1,n =n
2ˆσ∗2(ˆ
C∗
1−ˆ
C∗
2) diag(ˆ
λ1,...,ˆ
λk)( ˆ
C∗
1−ˆ
C∗
2),(5.29)
TEq,∗(b)
1,n =n
2ˆσ2(ˆ
C∗
1−ˆ
C∗
2) diag(ˆ
λ1,...,ˆ
λk)( ˆ
C∗
1−ˆ
C∗
2).(5.30)
The only di e ence be ween (5.29) and (5.30) is he es ima ion o σ2 o each i e a ion: i compu ed
om each boo s ap sample in he i s one, whe eas i is ob ained om he o iginal samples in he
second one. Rega ding o (5.26), he p oposed boo s ap s a is ic is
TEq,∗
2,n =
kn
X
j=1 ∆∗
1,nˆ j−∆∗
2,nˆ j
ˆ
λj!2
.(5.31)
The boo s ap dis ibu ions o (5.29), (5.30) and (5.31) a e ob ained by means o he ollowing wild
boo s ap algo i hm. Fo he pilo es ima ion o he common model pa ame e θ=θ1=θ2, unde H0,
he s anda d FPCA es ima o is conside ed (see (2.4) in Chap e 2, page 36).
Algo i hm 5.3.11 (Wild boo s ap).
S ep 1. Compu e he alue o he s a is ic TEq
1,n (o he alue o he s a is ic TEq
2,n).
S ep 2. Cons uc a pilo es ima o o he common pa ame e θ:ˆ
θkpilo
n=Pkpilo
n
j=1 ˆ
λ−1
j∆n(ˆ j)ˆ j, whe e
{(ˆ
λj,ˆ j)}∞
j=1 a e he eigenelemen s o Γn= (2n)−1Pn
i=1 (X1,i ⊗HX1,i +X2,i ⊗HX2,i), and
∆n= (2n)−1Pn
i=1 (X1,i ⊗H′Y1,i +X2,i ⊗H′Y2,i). Ob ain he esiduals o each sample:
ˆǫ1,i =Y1,i −hˆ
θkn, X1,iiand ˆǫ2,i =Y2,i −hˆ
θkn, X2,ii o all i= 1, . . . , n.
S ep 3. D aw {ǫ∗
1,i}n
i=1 and {ǫ∗
2,i}n
i=1 wo independen sequences o i.i.d. andom elemen s d awn
om ǫ∗, which sa is ies E(ǫ∗) = 0,E((ǫ∗)2) = 1 and R∞
0(P(|ǫ∗|> )1/2)<∞, and de ine
Y∗
1,i =hˆ
θkn, X1,ii+ ˆǫ1,iǫ∗
1,i and Y∗
2,i =hˆ
θkn, X2,ii+ ˆǫ2,iǫ∗
2,i o all i= 1,...,n. Consequen ly,
de ine Y∗
1= (Y∗
1,1, . . . , Y ∗
1,n) and Y∗
2= (Y∗
2,1,...,Y∗
2,n) .
S ep 4. Build ˆ
C∗
1= (X
1X1)−1X
1Y∗
1and ˆ
C∗
2= (X
2X2)−1X
2Y∗
2(o ∆∗
1,n =n−1Pn
i=1 X1,i ⊗H′Y∗
1,i
and ∆∗
2,n =n−1Pn
i=1 X2,i ⊗H′Y∗
2,i), and compu e an=|TEq,∗
1,n |(o compu e bn=|TEq,∗
2,n |).
S ep 5. Repea S eps 3 and 4 a la ge numbe o imes nboo ∈Nin o de o ob ain a sequence o
alues {al
n}nboo
l=1 (o a sequence o alues {bl
n}nboo
l=1 ).
S ep 6. App oxima e he p– alue o he es by he p opo ion o alues in {al
n}nboo
l=1 g ea e han o
equal o |TEq
1,n|(o by he p opo ion o alues in {bl
n}nboo
l=1 g ea e han o equal o |TEq
2,n|).
5.4 Simula ion s udy
In his sec ion a simula ion s udy illus a es he pe o mance o he asymp o ic app oach and he
boo s ap calib a ion o es ing he lack o dependence (see Sec ion 5.4.1), and he equali y o wo
linea models (see Sec ion 5.4.2). In bo h cases, H=L2[0,1] was selec ed, wi h i s usual inne p oduc ,
i.e., hx, yi=R1
0x( )y( )d o all x, y ∈L2[0,1].
5.4.1 Tes ing he lack o dependence
Fo his i s issue, he conside ed eg ession model was (5.1) (see page 106) wi h b= 0, µX= 0 and
µY= 0, ha is,
Y=Z1
0
θ( )X( )d +ǫ,
5.4. SIMULATION STUDY 123
whe e Xis a cen ed andom elemen alued in L2[0,1], Yand ǫa e cen ed andom a iables alued
in R, and θ∈L2[0,1] is he ixed model pa ame e . Then, ns = 500 samples we e simula ed, each
one consis ed o nobse a ions (n= 50,100) om his unc ional linea model, being Xa B ownian
mo ion and ǫ∼ N(0, σ2) wi h signal– o–noise a io =σ/pE(hθ, Xi2). Unde H0,
θH0( ) = 0,∀ ∈[0,1]
was aken as he pa ame e o he linea model, whe eas unde H1, he selec ed pa ame e was
θH1( ) = sin(2π 3)3,∀ ∈[0,1].
Fu he mo e, σ= 1 was chosen unde H0, while in he al e na i e H1σ= pE(hθ, Xi2) wi h h ee
di e en alues o he signal– o–noise a io ( = 0.5,1,2) was conside ed. Rema k ha bo h X
and θwe e disc e ized o p= 100 equidis an design poin s in [0,1]. As done in p e ious chap e s,
quad a u e weigh s o p−1we e used o app oxima e in eg als in ol ed in he calcula ions (see mo e
de ails in Sec ion 3.5, “How o wo k wi h disc e e da a?”, in Chap e 3, page 63).
The s a is ical es s which we e in oduced in Sec ion 5.2.2 (see page 113) we e conside ed, ha
is, TInd
1,n ,TInd
2,n ,TInd
3,n and TInd
3s,n de ined as (5.9), (5.10), (5.11) and (5.12), espec i ely. Fo TInd
1,n , h ee
dis ibu ion app oxima ions we e conside ed: i s asymp o ic dis ibu ion ( ha is, N(0,2)), and he
wo calib a ions based on he boo s ap s a is ics TInd,∗(a)
1,n and TInd,∗(b)
1,n , gi en by (5.15) and (5.16).
Fo TInd
2,n ,TInd
3,n and TInd
3s,n only he boo s ap app oaches based on he s a is ics TInd,∗
2,n ,TInd,∗
3,n and
TInd,∗
3s,n (see (5.17), (5.13) and (5.14), espec i ely) we e compu ed. In mos o hese s a is ics, an
es ima ion o σ(o σ∗) is in ol ed. I was decided o use he ollowing es ima e based on he esidual
sum o squa es, which was p oposed by Ca do e al. (2003b),
ˆσ2=1
n− (Sρ)
n
X
i=1
(Yi−SρYi)2,
whe e Sρis he ha ma ix o he penalized B–splines es ima o , i.e., SρYi=hˆ
θP S, Xii( ecall (2.2) in
Chap e 2, page 35), wi h he nex choices: B–splines wi h deg ee 4 and 20 equispaced kno s, second
de i a i es o he penal y, and pa ame e ρselec ed by GCV (see Sec ion 3.5, “Pa ame e es ima ion”,
in Chap e 3, page 63). Bedides, ˆσ∗2is compu ed analogously using he boo s ap sample gene a ed
in each i e a ion.
In o de o calib a e he boo s ap dis ibu ions, he wild boo s ap algo i hm in oduced in Sec-
ion 5.2.2 (see Algo i hm 5.2.10, page 115) was used o TInd,∗(a)
1,n ,TInd,∗(b)
1,n and TInd,∗
2,n , whe eas he
wild boo s ap algo i hm in oduced in Sec ion 5.2.1 (see Algo i hm 5.2.8, page 112) was compu ed o
TInd,∗
3,n and TInd,∗
3s,n . In all hese cases, nboo = 1,000 boo s ap i e a ions we e compu ed o app oxima e
he co esponding p– alues. Fu he mo e, {ǫ∗
i}n
i=1 we e d awn om he ollowing sum o wo Di ac
dis ibu ions: 0.1(5 + √5)δ(1−√5)/2+ 0.1(5 −√5)δ(1+√5)/2(i.e., P(ǫ∗
i= (1 −√5)/2) = 0.1(5 + √5) and
P(ǫ∗
i= (1 + √5)/2) = 0.1(5 −√5) o all i= 1 ...,n).
Since he signi icance le el αand he pa ame e knin ol ed in TInd
1,n and TInd
2,n (and in hei boo s ap
e sions) mus be ixed o un he p ocedu e, he s udy was done o ou signi icance le els (α=
0.2,0.1,0.05,0.01) and o di e en numbe s o p incipal componen s (kn= 1,...,20). Ne e heless,
in o de o simpli y he p esen a ion, he in o ma ion collec ed in he ollowing ables co esponds o
only h ee o he alues o knwhich we e analyed (kn= 5,10,20), whe eas he esul s depic ed in he
igu es co espond o only wo o he alues o α(α= 0.1,0.05).
Table 5.1 (see page 124) displays he empi ical size (i.e., he pe cen age o ejec ions unde H0) o
he es s a is ics ob ained in he simula ion s udy and Figu e 5.1 (see page 124) shows he e ec o
he pa ame e kn o he es ing p ocedu es based on TInd
1,n and TInd
2,n . Fo TInd
1,n , i can be highligh ed
ha boo s ap app oaches p esen empi ical sizes close o he nominal le el α han he asymp o ic
app oxima ion o TInd
1,n , mainly when knis small and α= 0.2. I one compa es he pe o mance
o he wo boo s ap p ocedu es p oposed, i seems ha i σ2is boo s apped (TInd,∗(a)
1,n ) he esul s
a e be e han i he same es ima ion o he a iance is conside ed in all he boo s ap eplica ions
130 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
o de o allow he powe plan s a o a oid NOxconcen a ions eaching he limi alues ixed by
he cu en en i onmen al legisla ion. The e o e, i is necessa y o es ima e p ope ly he eg ession
model which de ines he ela ionship be ween he obse ed NOxconcen a ion in he las minu es (X)
and he NOxconcen a ion wi h hal an hou ho izon (Y). Fo his pu pose, a i s s ep could be o
de e mine i he e exis s a linea dependence be ween Xand Yby means o he es ing p ocedu es
p esen ed in Sec ion 5.2.
A sample o size n= 300 was buil , whe e each cu e Xco esponds o 240 consecu i e minu e–by–
minu e alues o hou ly a e aged NOxconcen a ion, and he esponse Yco esponds o he hou ly
a e aged NOx alue hal an hou ahead. Taking α= 0.05, he es s o lack o dependence ejec he
null hypo hesis in all cases ( hus, he e is a linea ela ionship be ween he a iables), excep TInd
2,n
when knis la ge (see Table 5.5 and Figu e 5.5). Ne e heless, as i was commen ed in he simula ion
s udy, his es s a is ic does no ake in o accoun he a iance e m and i s powe is clea ly lowe
han he powe o he o he es s. As a esul , i seems ha he ela ionship be ween he las 240
obse a ions o he NOxconcen a ion and he NOx alue hal an hou ahead could be modelled by
means o a unc ional linea eg ession wi h scala esponse. Indeed, he es ima ion o his linea model
was done in Sec ion 3.6 in Chap e 3 (see Table 3.11, page 73).
TInd
1,n TInd
2,n TInd
3,n TInd
3s,n
N(0,2) TInd,∗(a)
1,n TInd,∗(b)
1,n TInd,∗
2,n TInd,∗
3,n TInd,∗
3s,n
kn1 5 10 1 5 10 1 5 10 1 5 10
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.003 0.006 0.000 0.000
Table 5.5: A mosphe ic pollu ion da a. Tes ing he lack o dependence. P– alues o TInd
1,n (using
he asymp o ic dis ibu ion N(0,2) and he boo s ap dis ibu ions o TInd,∗(a)
1,n and TInd,∗(b)
1,n ), TInd
2,n
(using he boo s ap dis ibu ion o TInd,∗
2,n ), TInd
3,n (using he boo s ap dis ibu ion o TInd,∗
3,n ), and
TInd
3s,n (using he boo s ap dis ibu ion o TInd,∗
3s,n ).
P− alues
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
0.0 0.2 0.4 0.6 0.8 1.0
Figu e 5.5: A mosphe ic pollu ion da a. Tes ing he lack o dependence. P– alues o TInd
1,n (ci cle o
he asymp o ic dis ibu ion N(0,2); squa e o he boo s ap dis ibu ions o TInd,∗(a)
1,n ; diamond o
he boo s ap dis ibu ions o TInd,∗(b)
1,n ), and TInd
2,n ( iangle o he boo s ap dis ibu ion o TInd,∗
2,n ).
Ho izon al do ed line co esponds o he signi icance le el α= 0.5.
5.6. FINAL CONCLUSIONS 131
5.5.2 Tes ing he equali y o linea models
The obse ed NOxle els a e low (and p ac ically cons an ) in gene al. Howe e , unde ce ain me e-
o ological condi ions, hese concen a ions can quickly ise and, hus, cause an ai pollu ion episode.
Hence, one could hink ha he linea eg ession model which links he cu es o hou ly a e aged
NOxconcen a ions (X) and he hou ly a e aged NOxle els hal an hou ahead (Y) is di e en when
he co a ia es in he sample co esponds o low– alued cu es, medium– alued cu es o high– alued
cu es.
In o de o check his assump ion, h ee samples o size n= 100 we e conside ed, whe e he
cu es Xconsis o 240 consecu i e minu e–by–minu e alues o hou ly a e aged NOxconcen a ion,
and he esponses Ya e he associa ed hou ly a e aged NOx alues hal an hou ahead. The i s
sample, deno ed by bin 1, co esponds o cu es Xsuch ha he las measu ed NOxle el belongs o
he in e al [0 µg/m3,10 µg/m3), i.e., X(240) ∈[0 µg/m3,10 µg/m3). The second sample, o bin 2,
consis s o pai s (X, Y ) sa is ying ha X(240) ∈[10 µg/m3,20 µg/m3). Finally, he hi d sample is
called bin 3 and hei co a ia es e i ying ha X(240) is equal o highe o 20 µg/m3.
Table 5.6 and Figu e 5.6 (see page 132) show he ob ained esul s when he equali y o models
among he di e en bins is es ed using he echniques in oduced in Sec ion 5.3. No e ha TEq
1,n wi h
he asymp o ic app oach ejec s he equali y o models in all cases i espec i ely o k(excep i bin 2
and bin 3 a e compa ed and k= 1), whe eas TEq
2,n ne e ejec s H0. On he o he hand he beha iou
o TEq
1,n o boo s ap calib a ions is simila o bo h TEq,∗(a)
1,n and TEq,∗(b)
1,n :H0canno be ejec ed when
bin 1 and bin 2 a e compa ed; H0is ejec ed when bin 1 and bin 3 a e compa ed o mode a e k
alues (and no ejec ed o e y small o la ge k); and H0canno be ejec ed when bin 2 and bin 3
a e compa ed, excep i he pa ame e kis e y la ge. Finally, TEq
3,n, which does no depend on any
pa ame e , does no ejec he null hypo hesis only o he i s s udied case (bin 1 s. bin 2).
TEq
1,n TEq
2,n TEq
3,n
χ2
kTEq,∗(a)
1,n TEq,∗(b)
1,n TEq,∗
2,n TEq,∗
3,n
k/kn1 5 10 1 5 10 1 5 10 1 5 10
bin 1 s. bin 2 0.000 0.000 0.002 0.706 0.060 0.946 0.698 0.056 0.941 0.472 0.306 0.437 0.701
bin 1 s. bin 3 0.000 0.000 0.000 0.165 0.012 0.013 0.160 0.008 0.002 0.626 0.336 0.823 0.038
bin 2 s. bin 3 0.890 0.023 0.000 0.776 0.902 0.331 0.789 0.932 0.315 0.546 0.895 0.720 0.035
Table 5.6: A mosphe ic pollu ion da a. Tes ing he equali y o linea models. P– alues o TEq
1,n (using
he asymp o ic dis ibu ion χ2
kand he boo s ap dis ibu ions o TEq,∗(a)
1,n and TEq,∗(b)
1,n ), TEq
2,n (using
he boo s ap dis ibu ion o TEq,∗
2,n ), and TEq
3,n (using he boo s ap dis ibu ion o TEq,∗
3,n ) when bin 1
is compa ed wi h bin 2, bin 1 is compa ed wi h bin 3, and bin 2 is compa ed wi h bin 3.
Gi en ha his is a mul iple es ing p oblem, co ec ed p– alues we e also compu ed. Two ad-
jus emen me hods we e conside ed: Bon e oni’s me hod and Benjamini and Hochbe g’ me hod (see
Benjamini and Hochbe g, 1995). Bo h echniques we e applied using he R ou ine p.adjus a ailable
in he package s a s (see R De elopmen Co e Team, 2010). The ob ained esul s a e shown in Ta-
ble 5.7 and Figu e 5.7 (see page 132) o he Bon e oni’s me hod, and Table 5.8 and Figu e 5.8 (see
page 133) o Benjamini and Hochbe g’ me hod. No e ha simila conclusions can be de i ed using
he co ec ed p– alues o hose ob ained using he o iginal ones.
5.6 Final conclusions
The p oposed boo s ap me hods a e compe i i e al e na i es o es s based on asymp o ic dis ibu-
ions, and hey o en gi e es sizes close o he nominal ones. In e ms o powe , he s a is ic es s
which include a consis en es ima ion o he e o a iance σ2ob ain highe empi ical powe han he
es s a is ics which do no ake i in o accoun . Fu he mo e, in all he s a is ics in ol ing a k/kn
132 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
p− alues bin 1 s bin 2
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
p− alues bin 1 s bin 3
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
p− alues bin 2 s bin 3
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
Figu e 5.6: A mosphe ic pollu ion da a. Tes ing he equali y o linea models. P– alues o TEq
1,n (ci cle
o he asymp o ic dis ibu ion χ2
k; squa e o he boo s ap dis ibu ions o TEq,∗(a)
1,n ; diamond o he
boo s ap dis ibu ions o TEq,∗(b)
1,n ), and TEq
2,n ( iangle o he boo s ap dis ibu ion o TEq,∗
2,n ) when
bin 1 is compa ed wi h bin 2 (le panel), bin 1 is compa ed wi h bin 3 (cen e panel), and bin 2 is
compa ed wi h bin 3 ( igh panel). Ho izon al do ed line co esponds o he signi icance le el α= 0.5.
TEq
1,n TEq
2,n TEq
3,n
χ2
kTEq,∗(a)
1,n TEq,∗(b)
1,n TEq,∗
2,n TEq,∗
3,n
k/kn1 5 10 1 5 10 1 5 10 1 5 10
bin 1 s. bin 2 0.001 0.000 0.006 1.000 0.180 1.000 1.000 0.168 1.000 1.000 0.918 1.000 1.000
bin 1 s. bin 3 0.000 0.000 0.000 0.495 0.036 0.039 0.480 0.024 0.006 1.000 1.000 1.000 0.114
bin 2 s. bin 3 1.000 0.070 0.000 1.000 1.000 0.993 1.000 1.000 0.945 1.000 1.000 1.000 0.105
Table 5.7: A mosphe ic pollu ion da a. Tes ing he equali y o linea models. P– alues adjus ed using
Bon e oni co ec ion o TEq
1,n (using he asymp o ic dis ibu ion χ2
kand he boo s ap dis ibu ions o
TEq,∗(a)
1,n and TEq,∗(b)
1,n ), TEq
2,n (using he boo s ap dis ibu ion o TEq,∗
2,n ), and TEq
3,n (using he boo s ap
dis ibu ion o TEq,∗
3,n ) when bin 1 is compa ed wi h bin 2, bin 1 is compa ed wi h bin 3, and bin 2 is
compa ed wi h bin 3.
p− alues bin 1 s bin 2
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
p− alues bin 1 s bin 3
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
p− alues bin 2 s bin 3
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
Figu e 5.7: A mosphe ic pollu ion da a. Tes ing he equali y o linea models. P– alues adjus ed
using Bon e oni co ec ion o TEq
1,n (ci cle o he asymp o ic dis ibu ion χ2
k; squa e o he boo s ap
dis ibu ions o TEq,∗(a)
1,n ; diamond o he boo s ap dis ibu ions o TEq,∗(b)
1,n ), and TEq
2,n ( iangle o
he boo s ap dis ibu ion o TEq,∗
2,n ) when bin 1 is compa ed wi h bin 2 (le panel), bin 1 is compa ed
wi h bin 3 (cen e panel), and bin 2 is compa ed wi h bin 3 ( igh panel). Ho izon al do ed line
co esponds o he signi icance le el α= 0.5.
5.7. APPENDIX CHAPTER 5 133
TEq
1,n TEq
2,n TEq
3,n
χ2
kTEq,∗(a)
1,n TEq,∗(b)
1,n TEq,∗
2,n TEq,∗
3,n
k/kn1 5 10 1 5 10 1 5 10 1 5 10
bin 1 s. bin 2 0.001 0.000 0.002 0.776 0.090 0.946 0.789 0.084 0.941 0.626 0.504 0.823 0.701
bin 1 s. bin 3 0.000 0.000 0.000 0.495 0.036 0.039 0.480 0.024 0.006 0.626 0.504 0.823 0.057
bin 2 s. bin 3 0.890 0.023 0.000 0.776 0.902 0.497 0.789 0.932 0.473 0.626 0.895 0.823 0.057
Table 5.8: A mosphe ic pollu ion da a. Tes ing he equali y o linea models. P– alues adjus ed using
Benjamini and Hochbe g co ec ion o TEq
1,n (using he asymp o ic dis ibu ion χ2
kand he boo s ap
dis ibu ions o TEq,∗(a)
1,n and TEq,∗(b)
1,n ), TEq
2,n (using he boo s ap dis ibu ion o TEq,∗
2,n ), and TEq
3,n (using
he boo s ap dis ibu ion o TEq,∗
3,n ) when bin 1 is compa ed wi h bin 2, bin 1 is compa ed wi h bin 3,
and bin 2 is compa ed wi h bin 3.
p− alues bin 1 s bin 2
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
p− alues bin 1 s bin 3
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
p− alues bin 2 s bin 3
1 3 5 7 9 11 13 15 17 19
0.0 0.2 0.4 0.6 0.8 1.0
Figu e 5.8: A mosphe ic pollu ion da a. Tes ing he equali y o linea models. P– alues adjus ed
using Benjamini–Hochbe g co ec ion o TEq
1,n (ci cle o he asymp o ic dis ibu ion χ2
k; squa e o
he boo s ap dis ibu ions o TEq,∗(a)
1,n ; diamond o he boo s ap dis ibu ions o TEq,∗(b)
1,n ), and TEq
2,n
( iangle o he boo s ap dis ibu ion o TEq,∗
2,n ) when bin 1 is compa ed wi h bin 2 (le panel), bin 1
is compa ed wi h bin 3 (cen e panel), and bin 2 is compa ed wi h bin 3 ( igh panel). Ho izon al
do ed line co esponds o he signi icance le el α= 0.5.
pa ame e , a sui able choice o k/knseems o be a qui e impo an poin and i is cu en ly an open
ques ion.
Besides o he op imal k/knselec ion, o he issues ela ed o hese hypo heses es s equi e u he
esea ch, such as hei ex ension o unc ional linea models wi h unc ional esponse. In addi ion, o
he es o lack o dependence, i would be in e es ing o combine i wi h he unc ional ANOVA es
(see Cue as e al. (2004), and Gonz´alez-Rod ´ıguez e al. (2012)) in o de o de elop an ANCOVA es
in his con ex . On he o he hand, o he es o equali y, he ex ension o he esul s o o he es
s a is ics p oposed by Ho ´a h e al. (2009), which a e applicable o mo e gene al si ua ions (di e en
co a iance s uc u es, di e en sample sizes,. . . ), is s ill a pending issue.
5.7 Appendix Chap e 5
This sec ion con ains he p oo s o he main esul s o he chap e and some necessa y echnical lemmas.
134 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
5.7.1 P oo o Theo em 5.2.3
Since TInd
ncan be equi alen ly exp essed as
TInd
n=1
n
n
X
i=1
(Xi−µX)(Yi−µY)−(X−µX)(Y−µY)
=n−1
n2
n
X
i=1
(Xi−µX)(Yi−µY)−1
n2
n
X
i=1 X
j6=i
(Xi−µX)(Yj−µY),
i is s aigh o wa d o check i em 1. Fu he mo e, he a.s. −Pcon e gence is a di ec applica ion o
he SLLN o sepa able Hilbe – alued andom elemen s.
On he o he hand, gi en ha E(k(X−µX)(Y−µY)k2)<∞, he con e gence in law can be
deduced by applying he CLT o sepa able Hilbe – alued andom elemen s (see, o ins ance, Laha
and Roha gi (1979)) oge he wi h Slu sky’s Theo em. The conc e e exp ession o he ope a o ΓZ
can be ob ained by simple compu a ions as ollows
ΓZ= Γ(X−µX)(Y−µY)= Γ(X−µX)(ǫ+hθ,X−µXi)= Γ(X−µX)ǫ+ Γ(X−µX)hθ,X−µXi
=E(ǫ2(X−µX)⊗H(X−µX)) + E(hθ, X −µXi2(X−µX)⊗H(X−µX))
=σ2Γ + E(hθ, X −µXi2(X−µX)⊗H(X−µX)).
5.7.2 P oo o Co olla y 5.2.4
Co olla y 5.2.4 can be de i ed om i em 3 in Theo em 5.2.3 because, unde he null hypo hesis,
kTIndk=k∆kH′= 0, and Y−µY=ǫ(hence, ΓZ= Γ(X−µX)(Y−µY)= Γ(X−µX)ǫ=σ2Γ).
5.7.3 P oo o Theo em 5.2.5
Theo em 5.2.5 is easily p o en aking in o accoun ha n−1Pn
i=1 (Xi−X)(Yn
i−Yn) con e ges a.s.−P
o E((X−µX)(Yn−µYn)) when n→ ∞, by he SLLN o sepa able Hilbe – alued andom elemen s,
and kE((X−µX)(Yn−µYn))k=δnkΓθk/√n.
5.7.4 P oo o Theo em 5.2.7
Fi s o all, no e ha
√n TInd,N∗
n=1
√n
n
X
i=1
(X∗
i−µX)(Y∗
i−µY)−1
√n
n
X
i=1
(Xi−µX)(Yi−µY)
−√n(X∗−µX)(Y∗−µY) + √n(X−µX)(Y−µY)
=√nS∗
n+1
√n√n(X∗−X)√n(Y∗−Y)−√n(X∗−X)(Y∗−µY)−(X∗−µX)√n(Y∗−Y),
(5.32)
whe e S∗
nis de ined as S∗
n=n−1Pn
i=1 (X∗
i−µX)(Y∗
i−µY)−n−1Pn
i=1 (Xi−µX)(Yi−µY). Gi en
ha {(X∗
i−µX)(Y∗
i−µY)}n
i=1 a e i.i.d. H– alued andom elemen s chosen a andom om he
boo s ap popula ion {(Xi−µX)(Yi−µY)}n
i=1 and E(k(X−µX)(Y−µY)k2)<∞, i em 1 in Lemma 5.7.1
gua an ees ha he boo s ap s a is ic √nS∗
ncon e ges in law o Z(X−µX)(Y−µY)a.s. −P.
On he o he hand, gi en ha E(kXk2)<∞and E(Y2)<∞, i ems 1 and 2 in Lemma 5.7.1,
oge he wi h Slu sky’s Theo em, ensu e ha he las h ee e ms in (5.32) con e ge in p obabili y o
0a.s. −P, and consequen ly he con e gence in law s a ed in he heo em is p o en.
Finally, no e ha ˆσ∗2=(Y∗)2−(Y∗)2. Then, he con e gence o ˆσ∗2holds in i ue o i ems 2
and 3 in Lemma 5.7.1, since E(Y2)<∞.
5.7. APPENDIX CHAPTER 5 135
5.7.5 Fo mula ion and p oo o Lemma 5.7.1
As was indica ed p e iously, he asymp o ic beha iou o he nai e boo s ap s a is ic is going o be
analysed h ough some esul s on boo s apping gene al empi ical measu es p esen ed by Gin´e and
Zinn (1990) (speci ically, h ough hei Theo em 2.4). Fo his eason, i is necessa y o in oduce
some no a ion used by Gin´e and Zinn (1990), and check ce ain condi ions equi ed in hei pape , in
o de o ob ain he Lemma 5.7.1 below.
I should be no ed ha he boo s ap esul s in Gin´e and Zinn (1990) e e o empi ical p ocesses
indexed by a class o unc ions F, and based on a p obabili y measu e P, ha pa icula ly ex end o
he boo s ap abou he mean in sepa able Banach (and hus Hilbe ) spaces. In o de o es ablish
his connec ion, i is enough o choose
F={ ∈ H′|k kH′≤1}
(see Gin´e (1997) and Koso ok (2008), o a gene al o e iew o indexed empi ical p ocesses). No e
ha Fis image admissible Suslin2conside ing he weak opology. This ac ensu es ha Fsa is ies
he measu abili y condi ions which a e necessa y o apply Theo em 2.4 in Gin´e and Zinn (1990). In
addi ion, le Fbe de ined as F(x) = sup ∈F | (x)|=kxk o all x∈ H, which sa is ies
E(F2(X)) = E(kXk2)<∞.
Finally, conside he bounded and linea (so con inuous) ope a o δ om H o l∞(F)3, gi en by
δ(x)( ) = δx( ) = (x) o all x∈ H and ∈ F, and deno e by Im(δ)⊂l∞(F) i s ange. Since
kδ(x)k∞=kxk o all x∈ H, hen he e exis s δ−1: Im(δ)→ H, such ha δ−1is con inuous (see, o
ins ance, Lemma 6.16 in Koso ok (2008)). In addi ion, as Im(δ) is closed, Dugundji’s Theo em allows o
conside a con inuous ex ension δ−1:l∞(F)→ H (see, o ins ance, Theo em 10.9 in Koso ok (2008)).
Thus, ollowing he ypical empi ical p ocess no a ion, he empi ical p ocess n−1/2Pn
i=1 (δXi−P)
indexed in Fis di ec ly connec ed wi h n−1/2Pn
i=1 (Xi−E(X)) by means o he con inuous mapping
δ−1and ice e sa. As a esul , he CLT o sepa able Hilbe – alued andom elemen s (see, o
ins ance, Laha and Roha gi (1979)) oge he wi h he Con inuous Mapping Theo em applied o δ
gua an ee ha F ∈ CLT(P)4, which is ano he condi ion in ol ed in Theo em 2.4 by Gin´e and Zinn
(1990).
All hese conside a ions lead o he esul s collec ed in he ollowing lemma.
Lemma 5.7.1 (Gonz´alez-Man eiga e al., 2012).Le ξbe a measu able mapping om a p obabilis ic
space deno ed by (Ω,A,P) o a sepa able Hilbe space (H,h·,·i)wi h co esponding no m k · k such
ha E(kξk2)<∞. Le {ξi}n
i=1 be a sequence o i.i.d. andom elemen s wi h he same dis ibu ion as
ξ, and le {ξ∗
i}n
i=1 be i.i.d. om {ξi}n
i=1. Then
(i) √n(ξ∗−ξ)con e ges in law o Zξa.s. −P
(ii) ξ∗con e ges in p obabili y o E(ξ)a.s. −P
(iii) kξ∗k2con e ges in p obabili y o E(kξk2)a.s. −P
2A sepa able measu able space (Y, S) will be called a Suslin space i he e is a Polish space Xand a Bo el measu able
map om Xon o Y(a Polish space is a opological space me izable as a comple e sepa able me ic space).
I (Ω,A) is a measu able space and Fa se , hen a eal– alued unc ion X: ( , ω)→X( , ω) will be called image
admissible Suslin ia (Y, S, T) i (Y, S) is a Suslin measu able space, Tis a unc ion om Yon o F, and (y, ω)→
X(T(y), ω) is join ly measu able on Y×Ω. Le Fbe a se o unc ions on Ω and X( , ω)≡ (ω). Fwill be called image
admissible Suslin i Xis image admissible Suslin ia some (Y, S, T ) as abo e.
3l∞(F) is he space o he bounded unc ions F → Rwi h he uni o m no m opology.
4ARadon measu e γis a measu e de ined on he σ–algeb a o Bo el se s o a opological space Xwhich is
locally ini e (i.e., o any x∈X he e is a neighbou hood which has ini e measu e) and inne egula (i.e.,
γ(B) = sup {γ(K) : K⊂B, K compac }).
A sequence {Yn}∞
n=1 o andom elemen s o l∞(F)con e ges weakly in l∞(F) i he e exis s a Radon p obabili y measu e
γon l∞(F) such ha o all H:l∞(F)→Rbounded and con inuous, limn→∞ E∗H(Yn) = RHdγ.
F ∈ CLT (P) i he sequence {√n(Pn−P)( ) : ∈ F} con e ges weakly in l∞(F) o a Radon cen ed Gaussian
p obabili y measu e γPon l∞(F), being Pn he empi ical measu e gi en by Pn=n−1Pn
i=1 δXi.
136 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
P oo . Taking in o accoun he easoning commen ed be o e he lemma o mula ion, Theo em 2.4
o Gin´e and Zinn (1990) ensu es ha √n(P∗
n(ω)−Pn(ω)) con e ges in law o a cen ed Gaussian
p ocess on F,G=δ(Zξ), a.s.−P, whe e Pn(ω) = n−1Pn
i=1 δξi(ω), and P∗
n(ω) is he empi ical measu e
based on {ξ∗
i}n
i=1, i.e., P∗
n(ω) = n−1Pn
i=1 δξ∗
i(ω). Consequen ly, by applying he Con inuous Mapping
Theo em o δ−1,√n(ξ∗−ξ) con e ges in law o Zξ=δ−1(G), and i em 1 is shown. No e ha i em 1
is also a di ec consequence o Rema k 2.5 o Gin´e and Zinn (1990); ne e heless i was p o en based
on Theo em 2.4 o illus a e he echnique.
I ems 2 and 3 can be checked in a simila way by applying Theo em 2.6 o Gin´e and Zinn (1990).
5.7.6 P oo o Theo em 5.2.9
Rema k ha , o all ω∈Ω,
√n TInd,W ∗
n(ω) = 1
√n
n
X
i=1
(Xi(ω)−µX)(Yi(ω)−µY)ǫ∗
i− 1
√n
n
X
i=1
(Xi(ω)−µX)ǫ∗
i!(Y(ω)−µY)
−(X(ω)−µX)1
√n
n
X
i=1
(Yi(ω)−µY)ǫ∗
i+ (X(ω)−µX)(Y(ω)−µY)1
√n
n
X
i=1
ǫ∗
i
=S∗
n− 1
√n
n
X
i=1
(Xi(ω)−µX)ǫ∗
i!(Y(ω)−µY)
−(X(ω)−µX) 1
√n
n
X
i=1
(Yi(ω)−µY)ǫ∗
i!+ (X(ω)−µX)(Y(ω)−µY) 1
√n
n
X
i=1
ǫ∗
i!,
(5.33)
whe e S∗
n=n−1/2Pn
i=1 (Xi(ω)−µX)(Yi(ω)−µY)ǫ∗
i. Acco ding o Lemma 5.7.2, o almos all ω∈Ω,
S∗
ncon e ges in law o Z(X−µX)(Y−µY), since E(k(X−µX)(Y−µY)k2)<∞.
On he o he hand, (X(ω)−µX) and (Y(ω)−µY) con e ge o 0 by SLLN. This ac join ly wi h
Lemma 5.7.2 and Slu sky’s Theo em gua an ee he con e gence in p obabili y o 0 o he las h ee
summands in (5.33), gi en ha E(k(X−µX)k2)<∞and E((Y−µY)2)<∞. Thus, he esul is
eached by applying again Slu sky’s Theo em.
5.7.7 Fo mula ion and p oo o Lemma 5.7.2
Lemma 5.7.2 (Gonz´alez-Man eiga e al., 2012).Le ξbe a measu able mapping om a p obabilis ic
space deno ed by (Ω,A,P) o a sepa able Hilbe space (H,h·,·i)wi h co esponding no m k·k such ha
E(kξk2)<∞. Le {ξi}n
i=1 be a sequence o i.i.d. andom elemen s wi h he same dis ibu ion as ξ,
and le {Wi}n
i=1 be a sequence o i.i.d. eal andom a iables independen o {ξi}n
i=1, wi h E(Wi) = 0
and R∞
0(P(|W1|> )1/2)<∞. Then he ollowing s a emen s a e equi alen
(i) E(kξk2)<∞(and, consequen ly, √n(ξ−E(ξ)) con e ges in law o Zξ).
(ii) Fo almos all ω∈Ω,n−1/2Pn
i=1 Wiξi(ω)con e ges in law o Zξ.
P oo . This lemma is a pa icula iza ion o a esul due o Ledoux, Talag and and Zinn (see Gin´e
and Zinn (1990) and Ledoux and Talag and (1988)). See also he Mul iplie CLT in Koso ok (2008)
o he empi ical p ocess indexed by a class o measu able unc ions coun e pa .
5.7. APPENDIX CHAPTER 5 137
5.7.8 P oo o Theo em 5.3.4
Rema k ha TEq
ncan be exp essed as
TEq
n=1
n
n
X
i=1
((X1,i −µX1)(Y1,i −µY1)−(X2,i −µX2)(Y2,i −µY2))
−((X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2))
=n−1
n2
n
X
i=1
((X1,i −µX1)(Y1,i −µY1)−(X2,i −µX2)(Y2,i −µY2))
−1
n2
n
X
i=1 X
j6=i
((X1,i −µX1)(Y1,j −µY1)−(X2,i −µX2)(Y2,j −µY2)),
so i is s aigh o wa d o check i em 1. Mo eo e , by he SLLN o sepa able Hilbe – alued andom
elemen s and he Slu sky’s Theo em, he a.s. −Pcon e gence in i em 2 can be s a ed.
On he o he hand, one has ha
√n(TEq
n−TEq) = √n 1
n
n
X
i=1
((X1,i −µX1)(Y1,i −µY1)−(X2,i −µX2)(Y2,i −µY2)) −TEq!
−1
√n√n(X1−µX1)√n(Y1−µY1) + 1
√n√n(X2−µX2)√n(Y2−µY2).
The las wo e ms in he igh side o he p e ious exp ession con e ge a.s. −P o 0 as n→ ∞ by he
CLT o sepa able Hilbe – alued andom elemen s, since E(kX1k2)<∞,E(kX2k2)<∞,E(Y2
1)<∞
and E(Y2
2)<∞, and he Slu sky’s Theo em. The e o e, due o Slu sky’s Theo em, i em 3 will be
p o en i i is showed he con e gence in law o
√n 1
n
n
X
i=1
((X1,i −µX1)(Y1,i −µY1)−(X2,i −µX2)(Y2,i −µY2)) −TEq!.
Gi en ha E(k(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)k2)<∞, his con e gence can be deduced
by applying again he CLT o sepa able Hilbe – alued andom elemen s. Finally, aking in o accoun
ha (C.5.4) is sa is ied, he conc e e exp ession o he ope a o ΓZcan be ob ained as ollows
ΓZ= Γ(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)= Γ(X1−µX1)(ǫ1+hθ1,X1−µX1i)−(X2−µX2)(ǫ2+hθ2,X2−µX2i)
= Γ(X1−µX1)ǫ1+ Γ(X1−µX1)hθ1,X1−µX1i+ Γ(X2−µX2)ǫ2+ Γ(X2−µX2)hθ2,X2−µX2i
=E(ǫ2
1(X1−µX1)⊗H(X1−µX1)) + E(hθ1, X1−µX1i2(X1−µX1)⊗H(X1−µX1))
+E(ǫ2
2(X2−µX2)⊗H(X2−µX2)) + E(hθ2, X2−µX2i2(X2−µX2)⊗H(X2−µX2))
= 2σ2Γ + E(hθ1, X1−µX1i2(X1−µX1)⊗H(X1−µX1))
+E(hθ2, X2−µX2i2(X2−µX2)⊗H(X2−µX2)).
5.7.9 P oo o Co olla y 5.3.5
Co olla y 5.3.5 can be de i ed om i em 3 in Theo em 5.3.4 aking in o accoun ha , unde he null
hypo hesis, kTEqk=k∆1−∆2kH′= 0.
138 CHAPTER 5. TESTING IN FUNCTIONAL LINEAR REGRESSION
5.7.10 P oo o Theo em 5.3.6
Rema k ha
1
n
n
X
i=1
((X1,i −X1)(Y1,i −Y1)−(X2,i −X2)(Yn
2,i −Yn
2))
=1
n
n
X
i=1
((X1,i −µX1)(Y1,i −µY1)−(X2,i −µX2)(Yn
2,i −µYn
2)) −((X1−µX1)(Y1−µY1)
−(X2−µX2)(Yn
2−µYn
2)).
One ge s ha he p e ious s a is ic con e ges a.s.−P o E((X1−µX1)(Y1−µY1)−(X2−µX2)(Yn
2−µYn
2)),
as n→ ∞, by he SLLN o sepa able Hilbe – alued andom elemen s and he Slu sky’s Theo em.
Hence, Theo em 5.3.6 is easily p o en gi en ha , when (C.5.4) is sa is ied, kE((X1−µX1)(Y1−µY1)−
(X2−µX2)(Yn
2−µYn
2))k=δnkΓθ1k/√n.
5.7.11 P oo o Theo em 5.3.8
Fi s o all, no e ha
√n TEq,N∗
n=1
√n
n
X
i=1
((X∗
1,i −µX1)(Y∗
1,i −µY1)−(X∗
2,i −µX2)(Y∗
2,i −µY2))
−1
√n
n
X
i=1
((X1,i −µX1)(Y1,i −µY1)−(X2,i −µX2)(Y2,i −µY2)) −√n(X∗
1−µX1)(Y∗
1−µY1)
+√n(X∗
2−µX2)(Y∗
2−µY2)) + √n(X1−µX1)(Y1−µY1)−√n(X2−µX2)(Y2−µY2)
=√nS∗
n+1
√n√n(X∗
1−X1)√n(Y∗
1−Y1)−√n(X∗
1−X1)(Y∗
1−µY1)−(X∗
1−µX1)√n(Y∗
1−Y1)
−1
√n√n(X∗
2−X2)√n(Y∗
2−Y2) + √n(X∗
2−X2)(Y∗
2−µY2) + (X∗
2−µX2)√n(Y∗
2−Y2),
(5.34)
whe e S∗
nis de ined as
S∗
n=1
n
n
X
i=1
((X∗
1,i −µX1)(Y∗
1,i −µY1)−(X∗
2,i −µX2)(Y∗
2,i −µY2))
−1
n
n
X
i=1
((X1,i −µX1)(Y1,i −µY1)−(X2,i −µX2)(Y2,i −µY2)).
Gi en ha {(X∗
1,i −µX1)(Y∗
1,i −µY1)−(X∗
2,i −µX2)(Y∗
2,i −µY2)}n
i=1 a e i.i.d. H– alued andom elemen s
chosen a andom om he boo s ap popula ion {(X1,i −µX1)(Y1,i −µY1)−(X2,i −µX2)(Y2,i −µY2)}n
i=1
and E(k(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)k2)<∞, he boo s ap s a is ic √nS∗
ncon e ges
in law o Z(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)a.s. −Pdue o i em 1 in Lemma 5.7.1 (see page 135).
In addi ion, he las six e ms in (5.34) con e ge in p obabili y o 0 a.s. −Pby i ems 1 and 2 in
Lemma 5.7.1, oge he wi h Slu sky’s Theo em, since E(kX1k2)<∞,E(kX2k2)<∞,E(Y2
1)<∞
and E(Y2
2)<∞. As a esul , he con e gence in law s a ed in he heo em is p o en applying again
he Slu sky’s Theo em.
5.7. APPENDIX CHAPTER 5 139
5.7.12 P oo o Theo em 5.3.10
Rema k ha , o all ω∈Ω,
√n TEq,W ∗
n(ω) = 1
√n
n
X
i=1
((X1,i(ω)−µX1)(Y1,i(ω)−µY1)−(X2,i(ω)−µX2)(Y2,i(ω)−µY2))ǫ∗
i
− 1
√n
n
X
i=1
(X1,i(ω)−µX1)ǫ∗
i!(Y1(ω)−µY1)−(X1(ω)−µX1)1
√n
n
X
i=1
(Y1,i(ω)−µY1)ǫ∗
i
+ (X1(ω)−µX1)(Y1(ω)−µY1)1
√n
n
X
i=1
ǫ∗
i+ 1
√n
n
X
i=1
(X2,i(ω)−µX2)ǫ∗
i!(Y2(ω)−µY2)
+ (X2(ω)−µX2)1
√n
n
X
i=1
(Y2,i(ω)−µY2)ǫ∗
i−(X2(ω)−µX2)(Y2(ω)−µY2)1
√n
n
X
i=1
ǫ∗
i
=S∗
n− 1
√n
n
X
i=1
(X1,i(ω)−µX1)ǫ∗
i!(Y1(ω)−µY1)−(X1(ω)−µX1)1
√n
n
X
i=1
(Y1,i(ω)−µY1)ǫ∗
i
+ 1
√n
n
X
i=1
(X2,i(ω)−µX2)ǫ∗
i!(Y2(ω)−µY2) + (X2(ω)−µX2)1
√n
n
X
i=1
(Y2,i(ω)−µY2)ǫ∗
i
+ ((X1(ω)−µX1)(Y1(ω)−µY1)−(X2(ω)−µX2)(Y2(ω)−µY2)) 1
√n
n
X
i=1
ǫ∗
i,
(5.35)
whe e S∗
n=n−1/2Pn
i=1 ((X1,i(ω)−µX1)(Y1,i(ω)−µY1)−(X2,i(ω)−µX2)(Y2,i(ω)−µY2))ǫ∗
i. Apply-
ing Lemma 5.7.2 (see page 136), S∗
ncon e ges in law o Z(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2), o almos
all ω∈Ω, since E(k(X1−µX1)(Y1−µY1)−(X2−µX2)(Y2−µY2)k2)<∞.
Fu he mo e, (X1(ω)−µX1), (Y1(ω)−µY1), (X2(ω)−µX2) and (Y2(ω)−µY2) con e ge o 0
by he SLLN. This ac , Lemma 5.7.2 and Slu sky’s Theo em ensu e he con e gence in p obabili y
o 0 o he las six summands in (5.35), gi en ha E(kX1−µX1k2)<∞,E((Y1−µY1)2)<∞,
E(kX2−µX2k2)<∞and E((Y2−µY2)2)<∞. Consequen ly, he heo em is p o en by he applica ion
o he Slu sky’s Theo em.
