All about the ⊥ with its applications in the linear statistical models

© 2015 Augus yn Ma kiewicz and Simo Pun anen licensee De G uy e Open. This wo k is licensed unde he C ea i e Commons A ibu ion-NonComme cial-NoDe i s 3.0 License. Open Ma h. 2015; 13: 33–50 Open Ma hema ics Open Access Re iew A icle Augus yn Ma kiewicz and Simo Pun anen* All abou he ?wi h i s applica ions in he linea s a is ical models Abs ac : Fo an nm eal ma ix A he ma ix A?is de ined as a ma ix spanning he o hocomplemen o he column space o A, when he o hogonali y is de ined wi h espec o he s anda d inne p oduc hx;yi D x0y. In his pape we collec oge he a ious p ope ies o he ?ope a ion and i s applica ions in linea s a is ical models. Resul s co e ing he mo e gene al inne p oduc s a e also conside ed. We also p o ide a a he ex ensi e lis o e e ences. Keywo ds: Bes linea unbiased es ima o , Column space, Gene alized in e se, Linea s a is ical model, O hocom- plemen , O hogonal p ojec o MSC: 15A99, 62H12, 62J05 DOI 10.1515/ma h-2015-0005 Recei ed Decembe 13, 2013; accep ed Ap il 14, 2014. 1 In oduc ion Conside a columnwise pa i ioned ma ix AD.a1W: : : Wam/2Rnm( he se o nmma ices wi h eal elemen s). Then he column space o Ais de ined as C.A/D z2RnWzDA Da1 1CCam m o some 2Rmg: The no a ion C.A/? e e s o he o hocomplemen o C.A/, i.e., he se o ec o s which a e o hogonal (wi h espec o he s anda d inne p oduc ) o e e y ec o o C.A/: C.A/?D u2RnWu0A D0 o all 2Rmg: Thus, because C.A/?D u2RnWu0A D0 o all 2Rmg D u2RnWA0uD0g; we ha e C.A/?DN.A0/D he null space o A0: Now A?is de ined as a ma ix whose column space is C.A?/DC.A/?DN.A0/. In iew o decomposi ion RnDC.A/˚C.A/?;whe e ˚ e e s o he di ec sum, he ank o A?is ank.A?/Dn ank.A/: The se o all ma ices A?is deno ed as A?gand hence: Z2 A?g ” (a) A0ZD0and (b) ank.Z/Dn ank.A/: (1) Augus yn Ma kiewicz: Depa men o Ma hema ical and S a is ical Me hods, Pozna´ n Uni e si y o Li e Sciences, Wojska Polskiego 28, PL-60637 Pozna´ n, Poland, E-mail: [email p o ec ed] *Co esponding Au ho : Simo Pun anen: School o In o ma ion Sciences, FI-33014 Uni e si y o Tampe e, Finland, E-mail: [email p o ec ed] 34 A. Ma kiewicz, S. Pun anen We immedia ely obse e ha Z2 A?g ” A2 Z?g:T i ially A?is unique only when Ais a nonsingula squa e ma ix in which case A?D0. No ice ha A2RnmH) A?2Rns;whe e sn ank.A/: In his pape ou pu pose is o e iew a ious ea u es o he ?ope a ion, he “pe p-ope a ion”, say, and in pa icula , o p esen se e al use ul applica ions ela ed o linea s a is ical models. Resul s co e ing he mo e gene al inne p oduc s a e also conside ed. We belie e ha ou e iew p o ides a use ul summa y o he ?ope a ion and he eby inc eases he insigh s and app ecia ion o his, seemingly simple, ope a ion. 2A?in e ms o gene alised in e ses The gene alised in e ses o e a e y handy ool o explici exp essions o he A?, and in his sec ion we gi e a sho ou in o such possibili ies. Ma ix G2Rmnis a gene alised in e se o A2Rnmi AGA DA;(mp1) and i is he Moo e–Pen ose in e se, deno ed as AC, i i also sa is ies he ollowing h ee condi ions: (mp2)GAG DG;(mp3).AG/0DAG ;(mp4).GA/0DGA : I Gsa is ies he condi ion AGA DA, we may deno e GDA, o G2 Ag. As he excellen e e ences o he gene alised in e ses, see Ben-Is ael & G e ille [10] and Rao & Mi a [46]. In pa icula , o mo e o abou he Moo e (o Moo e & Pen ose), see Ben-Is ael [11]. I is well known ha he nullspace N.A/can be exp essed as N.Anm/DC.ImAA/ ; whe e Acan be any gene alized in e se o A. Hence we can exp ess C.A/?in e ms o A: C.A/?DN.A0/DCŒIn.A0/A0DCŒIn.A/0A0 : (2) The las equali y abo e ollows om he ac .A/0g D .A0/g:(3) No ice ha i is a bi ques ionable o w i e .A/0D.A0/because (3) means he equali y be ween wo se s. Howe e , o he (unique) Moo e–Pen ose in e se we always ha e .AC/0D.A0/C:In ligh o (2), we ha e, o example, he ollowing choices o A?( ecalling ha A2Rnm): In.A0/A0;In.A/0A0;InA.A0A/A0;(4) whe e we ha e used he ac A.A0A/2 .A0/g. By eplacing .A0/wi h .A0/Cin (4) and using .A0/CD.AC/0; .A0/CA0DAAC; we ge InAACWD InPAWD QA2 A?g:(5) I can be shown ha i Gsa is ies he condi ions (mp1) and (mp3), i.e., G2 A 13g hen AG is unique and he eby AA 13 DAAC;and hence InAA 13 is one choice o A?. The no a ions PAand QAin (5) e e o he o hogonal p ojec o s on o C.A/(wi h espec o he s anda d inne p oduc ) and C.A/?, espec i ely. Ma ix Pis de ined as he o hogonal p ojec o on o C.A/i i sa is ies he ollowing condi ions: PDP0DP2and C.P/DC.A/; (6) All abou he ?wi h i s applica ions in he linea s a is ical models 35 which can be shown o be equi alen o P.AWA?/D.AW0/; whe e .AWA?/and .AW0/deno e pa i ioned ma ices. The ma ix Psa is ying (6) is unique and can be w i en as PADAACDAA 13 DA.A0A/A0, whe e he las exp ession is in a ian o any choice o .A0A/; his ollows om Rao & Mi a [46, Lemma 2.2.4], which says ha o nonnull Aand C, he ma ix p oduc ABCis in a ian wi h espec o he choice o he gene alized in e se B i and only i C.C/C.B/and C.A0/C.B0/. No ice ha AAis no necessa ily an o hogonal p ojec o : i is idempo en and i sa is ies C.AA/DC.A/bu i is no necessa ily symme ic. Below is a summa y o some o he exp essions o A?wi h ob ious ex ensions o .A0/?in e ms o gene alised in e ses. Theo em 2.1. Fo A2Rnm, deno e QADInAACDInPA. Then (a) In.A0/A02 A?g; (b) In.A/0A02 A?g; (c) InPADQA2 A?g; (d) InAA 13 2 A?g; (e) ImAA2 .A0/?g; ( ) ImA0.AA0/ADImACADImPA0DQA02 .A0/?g: Ob iously he o hogonal p ojec o QADInAACis o en a con enien choice o A?because i is symme ic and idempo en . 3 Some speci ic o mulas Suppose ha Zis a choice o A?. Then, o a com o mable ma ix B, we ha e ZB 2 A?g(7) whene e ank.ZB/D ank.Z/. Acco ding o Ma saglia & S yan [34, Co . 6.2] (see Theo em 4.1 below), ank.ZB/D ank.Z/dim C.Z0/ C.B/?; and he eby (7) holds i and only i C.Z0/ C.B/?D 0g:Thus we ha e he ollowing simple esul : Theo em 3.1. Le A2Rnm, and B2Rnq. Then o any A?2Rnnwe ha e A?B2 A?g ” CŒ.A?/0 C.B/?D 0g: In pa icula , choosing QAas A?yields QAB2 A?g ” C.AWB/DRn; whe e .AWB/deno es he pa ioned n.m Cq/ ma ix. In he nex heo em we ake a look a he pe ps o some pa icula pa i ioned ma ices. Theo em 3.2. Le A2Rnm, and B2Rnq. Then o any A?we ha e (a) A?0 0 Iq!28 < : Anm 0qm!?9 = ; ; (b) In B0!A?28 < : AnmBnq 0 Iq!?9 = ; ; (c) In B0!28 < : Bnq Iq!?9 = ; : 36 A. Ma kiewicz, S. Pun anen P oo . Pa (a) is ob ious as he o hogonali y condi ion co esponding o (1a) i ially holds and ank A?0 0 Iq!Dn ank.A/Cq: To p o e (b), we obse e ha Œ.A?/0W .A?/0B A B 0 Iq!D0: Mo eo e , he ank o In B0A?is ank.A?/0W .A?/0BD ankŒ.A?/0Dn ank.A/ ; while ank AnmBnq 0 Iq!? DnCq ank Anm 0! ank Bnq Iq! DnCq ank.A/qDn ank.A/ : Thus (b) is con i med. Pa (c) can be p o ed in he co esponding way. Theo em 3.3. Conside A2Rnmand B2R m. Then o any A?and B?we ha e A?0 0 B?!28 < : A B!?9 = ; i and only i C.A0/ C.B0/D 0g: P oo . The o hogonali y condi ion (1a) ob iously holds while ank A?0 0 B?!DnC  ank.A/ ank.B/; ank A B!? DnC  ank.A/ ank.B/Cdim C.A0/ C.B0/: Thus he p oo is comple ed. Rema k 3.4. I migh be a bi emp ing o ew i e pa (a) o Theo em 3.2 as Anm 0qm!? D A?0 0 Iq!:(8) Howe e , exp ession like (8) is ob iously p oblema ic, and he meaning o he abo e no a ion should be cla i ied. One in e p e a ion o (8) migh be o ag ee ha i means ha 8 < : Anm 0qm!?9 = ;D( A?0 0 Iq!):(9) In o he wo ds, he se s o ma ices a e iden ical. Howe e , he s a emen (9) is inco ec as can be concluded by Theo em 3.5 below. Le us ask he ollowing: which ma ices B2Rnpand D2Rqpsa is y he ollowing: A?Bnp 0 Dqp!28 < : Anm 0qm!?9 = ; ‹ All abou he ?wi h i s applica ions in he linea s a is ical models 37 We i s obse e ha equa ion .A?/00 B0D0! Anm 0qm!D 0nm 0qm! holds i and only i C.B/C.A/?. Supposing ha C.B/C.A/?, hen, in iew o Ma saglia & S yan [34, Co . 19.1], he ank o A?B 0 D is addi i e on he Schu complemen , i.e., ank A?B 0 D!D ank.D/C ank.A?BD0/D ank.D/C ank.A?/D ank.D/Cn ank.A/ : On he o he hand, because ank Anm 0qm!? DnCq ank.A/ ; we immedia ely ob ain he ollowing: Theo em 3.5. Conside A2Rnm,B2Rnp, and D2Rqp. Then he ela ion A?B 0 D!28 < : A 0!?9 = ; holds i and only i C.B/C.A/?and Dhas ull ow ank. 4 Two ank o mulas and a decomposi ion o o hogonal p ojec o Two pa icula ank o mulas in e ms o he o hocomplemen a e wo h special p aising due o hei nume ous applica ions pa icula ly when dealing wi h linea s a is ical models: he ank o he p oduc AnaBamand he ank o he pa i ioned ma ix .AnaWBnb/. Theo em 4.1. The ank o he pa i ioned ma ix .AnaWBnb/can be exp essed as ank.AWB/D ank.A/C ankŒ.A?/0BD ank.A/C ank.B0A?/ ; (10) and he ank o he ma ix p oduc AnaBamis ank.AB/D ank.A/dim C.A0/ C.B/?:(11) In e ms o an a bi a y gene alized in e se A, (10) can be exp essed as ank.AWB/D ankŒAW.InAA/BD ank.A/C ankŒ.InAA/BD ank.A/C ankŒ.InPA/B : (12) As a e e ence o (10) and (12) we may men ion Ma saglia & S yan [34, Th. 5]. Fo he e e ences o (11), see, e.g., Ma saglia & S yan [34, Co . 6.2], Rao [43, p. 28], and Zyskind & Ma in [59, p. 1194]. We may also men ion ha O.M. Baksala y & T enkle [8] p o ide se e al exp essions o he anks o a p oduc o wo ma ices and o a column-wise pa i ioned ma ix as well as an ex ensi e lis o ela ed e e ences. Se e al applica ions o (10) and (11) appea in Pun anen, S yan & Iso alo [39, Ch. 5]. One example conce ns he decomposi ion o he column space C.XWVX?/, whe e X2Rnpand Vis an nn(symme ic) nonnega i e de ini e ma ix. Such a si ua ion occu s when we conside he gene al linea model yDXˇC";deno ed as MD y;Xˇ;Vg;(13) whe e Xis a known npmodel ma ix, he ec o yis an obse able n-dimensional andom ec o , ˇis a p1 ec o o unknown pa ame e s, and "is an unobse able ec o o andom e o s wi h expec a ion E."/D0;and co a iance ma ix co ."/DV. Then we ha e he ollowing; see, e.g., Rao [45, Lemma 2.1]. 38 A. Ma kiewicz, S. Pun anen Theo em 4.2. Conside he linea model MD y;Xˇ;Vg, de ined as in (13). Then C.XWV/DC.XWVX?/DC.X/˚C.VX?/ : Mo eo e , i he model is co ec , in which case i is called consis en , hen he obse ed ( ealized) alue o he andom ec o ysa is ies y2C.XWV/ : (14) Fo a discussion conce ning he consis ency concep , see, e.g., Pun anen & S yan [38], J.K. Baksala y, Rao & Ma kiewicz [5], G oß [16, p. 314], and Tian e al. [53]. In his pape , we assume ha he co esponding consis- ency holds wha e e model we ha e. When wo king wi h linea models, we o en need o conside he o hogonal p ojec o on o he column space o he pa i ioned ma ix. Then he ollowing heo em appea s o be e y con enien in a ious connec ions; see, e.g., Pun anen, S yan & Iso alo [39, Th. 8] and Sebe & Lee [50, Appendix B]. Theo em 4.3. The o hogonal p ojec o (wi h espec o he s anda d inne p oduc )on o he column space C.AnaWBnb/can be decomposed as P.AWB/DPACP.InPA/BDPACPC.AWB/ C.A/?:(15) We comple e his sec ion by some ema ks on he explici exp ession o he in e sec ion o C.A/and C.B/. Fo a e e ence, see Rao & Mi a [46, Complemen 7, p. 118]. Theo em 4.4. Conside he ma ices Anaand Bnband deno e QBDInPB. Then C.A/ C.B/DCA.A0B?/?DCA.A0QB/?DCA.IaPA0QB/DCAŒIa.A0QBA/A0QBA: I is ob ious ha C.A/ C.B/?DCA.A0B/?DCA.IaPA0B/: In pa icula , i X2Rnpand Vnnis nonnega i e de ini e, hen C.X/ C.V/?DCX.X0V/?DCX.X0VX/?DCXŒIp.X0VX/X0VX; and in iew o C.M/ C.V/?DC.XWV/?, MŒIn.MVM/MVM2˚.XWV/?; whe e MDInPX. No ice also ha acco ding o Theo em 4.3 we ha e P.XWV/DPXCPMV and he eby InP.XWV/DMPMV DM.InPMV/2˚.XWV/?: 5 O hocomplemen when he inne p oduc ma ix is V 5.1 Vis posi i e de ini e Conside now he inne p oduc in Rnde ined as hx;yiVDx0Vy ;whe e Vis a posi i e de ini e symme ic ma ix. The o hocomplemen o C.Anm/wi h espec o his inne p oduc is C.A/? VD y2RnWz0A0Vy D0 o all z2Rmg: By A? Vwe will deno e any ma ix whose column space is C.A/? V. Recall ha A? Iis sho ly deno ed as A?. We ha e C.A/? VD y2RnWA0Vy D0g D N.A0V/DC.VA/?DC.V1A?/ ; All abou he ?wi h i s applica ions in he linea s a is ical models 39 whe e he las equali y can be concluded om A0VV1A?D0H) C.V1A?/N.A0V/ ; and ank.V1A?/D ank.A?/Dn ank.A/Ddim C.VA/?: No ice ha co esponding o (1), Z2 A? Vg ” (a) A0VZ D0and (b) ank.Z/Dn ank.A/: Rema k 5.1. Ob iously we can w i e V1A?2 A? Vgand V1A?2 .VA/?g. Ques ion: Is i co ec o w i e A? Vg D .VA/?g‹ I is easy o con i m ha he answe is posi i e. Now we ha e he ollowing decomposi ion: RnDC.A/˚C.A/? VDC.A/˚C.V1A?/ ; and hence e e y y2Rnhas a unique ep esen a ion as a sum yDAb CV1A?cDyCP y; o some band c. The ec o yDAb is he o hogonal p ojec ion o yon o C.A/along C.A/? V. The o hogonal p ojec o PAIVis such a ma ix which ans o ms yin o i s p ojec ion y, i.e., PAIVyDyDAb. I s explici unique ep esen a ion is PAIVDA.A0VA/A0V: We may men ion ha pa (a) o Theo em 3.2 holds e en i he inne p oduc ma ix is V, i.e., A? V0 0 Iq!28 < : Anm 0qm!? V9 = ; : Simila ly Theo ems 3.3 and 3.5 hold also when all o hocomplemen s a e aken wi h espec o he inne p oduc ma ix V. 5.2 Vis nonnega i e de ini e, possibly singula Le Vbe a singula nonnega i e de ini e ma ix. Then h ;uiVD 0Vu is a semi-inne p oduc and he co esponding semino m (squa ed) is k k2 VD 0V . Fo a singula nonnega i e de ini e ma ix Vwe can de ine he ma ix A? Vagain as any ma ix spanning C.A/? V, and so C.A? V/DC.A/? VDN.A0V/DC.VA/?: As no ed by Pun anen, S yan & Iso alo [39, §2.5] o (e en) a singula Vwe do ha e he decomposi ion RnDC.A/CC.A? V/DC.A/CC.VA/?;(16) bu , howe e , he abo e decomposi ion is no necessa ily a di ec sum. Fo any nonnega i e de ini e Vwe ha e, on accoun o Theo em 4.1, dim C.VA/?Dn ank.VA/DŒn  ank.A/ Cdim C.A/ C.V/?; which means ha (16) becomes a di ec sum decomposi ion i and only i C.A/ C.V/?D 0g. Fo he cha ac e iza ion o he gene alized o hogonal p ojec o , see Mi a & Rao [36]. Some ela ed conside a- ions appea also in Ha ille [19, §14.12.i], Rao & Rao [47, p. 81], and Tian & Takane [54, 55]. 40 A. Ma kiewicz, S. Pun anen 5.3 Some u he conside a ions Conside he linea model MD y;Xˇ;Vg, de ined as in (13), and le Vbe posi i e de ini e. Then we ha e obse ed ha he ollowing se s a e iden ical: (a) C.X/? V1;(b) C.VX?/ ; (c) N.X0V1/ ; (d) C.V1X/?;(e) N.PXIV1/ ; ( ) C.InPXIV1/ : Fo (a), .. . , ( ) abo e, see also Pun anen, S yan & Iso alo [39, §5.13]. When Vis singula , he abo e conside a ions become mo e complica ed. A e y con enien ool appea s o be he ollowing class o ma ices: WD W2RnnWWDVCXUX0;C.W/DC.XWV/g:(17) In (17) Ucan be any ppma ix as long as C.W/DC.XWV/is sa is ied. O cou se, Ucan be chosen as 0i C.X/C.V/which happens, o example, when Vis posi i e de ini e. The se Wo ma ices has an impo an ole in he heo y o linea models. Below a e lis ed some use ul equi alen s a emen s conce ning W: C.X/C.W/ ; (18a) C.XWV/DC.W/ ; (18b) X0WXis in a ian o any choice o W;(18c) C.X0WX/DC.X0/ o any choice o W;(18d) X.X0WX/X0WXDX o any choices o he gene alized in e ses in ol ed. (18e) Mo eo e , each o hese s a emen s is equi alen o C.X/C.W0/, and hence o he s a emen s (18b’)–(18e’) ob ained om (18b)–(18e), by se ing W0in place o W. As he e e ences o (18), we may men ion J.K. Baksala y, Pun anen & S yan [4, Th. 2], J.K. Baksala y & Ma hew [3, Th. 2], and Ha ille [19, p. 468]. Acco ding o Pun anen, S yan & Iso alo [39, §5.13] he ollowing now holds. Theo em 5.2. Suppose ha Xis an npma ix, Vis an nnnonnega i e de ini e ma ix and W2W, whe e W is de ined as in (17). Then C.VX?/DC.WXWInWW/?; whe e Wis an a bi a y (bu ixed)gene alized in e se o W. The column space C.VX?/can be exp essed also as C.VX?/DC.W/0XWIn.W/0W0?: Mo eo e , le Vbe possibly singula and assume ha C.X/C.V/. Then C.VX?/DC.VXWInVV/?C.VX/?; whe e he inclusion becomes equali y i and only i Vis posi i e de ini e. Rema k 5.3. I is o in e es o no e ha he pe p symbol ?d ops down, so o say, e y “nicely” when Vis posi i e de ini e: C.VX?/?DC.V1X/ ; bu when Vis singula we ha e o use a much mo e complica ed ule o d op down he ?symbol: C.VX?/?DC.WXWInWW/ ; whe e W2W. Rema k 5.4. Le us nex p o e he ollowing: I W2W, whe e Wis de ined as in (17), hen C.VX?/DC.WX/?”C.XWV/DRn:(19) All abou he ?wi h i s applica ions in he linea s a is ical models 41 We i s obse e ha C.VX?/DC.WXWInWW/?DC.WX/? C.InWW/?: Thus we always ha e C.VX?/C.WX/?;whe e he equali y appea s only i dim C.WX/?D ank.VX?/. Now we ha e ank.VX?/D ank.W/ ank.X/ ; dim C.WX/?Dn ank.WX/Dn ank.X/ ; om which ou claim (19) ollows. Fo comple eness we s a e he ollowing ela ed esul , due o Rao & Mi a [46, p. 140]. Theo em 5.5. Conside he linea model MD y;Xˇ;Vgand deno e WDVCXUX0, whe e C.W/DC.XWV/, and le Wbe an a bi a y gene alized in e se o W. Then C.WX/˚C.X/?DRn;C.WX/?˚C.X/DRn; CŒ.W/0X˚C.X/?DRn;CŒ.W/0X?˚C.X/DRn: 6 S a is ical examples 6.1 Cen e ing We would like o s a wi h a simple bu a he same ime e y impo an o hocomplemen in s a is ics: he se o ec o s o hogonal o he ec o o ones, ha is, C.1n/?, whe e 1nD.1; 1; : : : ; 1/02Rn. In wha ollows, we mos o he ime d op o he subsc ip om he ec o 1n; om he con ex i s dimension should be ob ious. The o hogonal p ojec o on o C.1n/is P1D1 n110WD Jand he o hogonal p ojec o on o C.1n/?is In1 n110WD C; Cis he cen e ing ma ix. Conside he n2da a ma ix Upa i ioned as UD.xWy/D0 B B B B @ x1y1 x2y2 : : :: : : xnyn 1 C C C C AD0 B B B B @ u0 .1/ u0 .2/ : : : u0 .n/ 1 C C C C A : He e u.i/ Dxi yi2R2 ep esen s he i h case o he i h obse a ion in he obse a ion space, and he ec o s x;y2Rn ep esen he wo a iables in he a iable space. Le N uDNx Ny2R2deno e he mean ec o o x- and y- a iables and S he sample co a iance ma ix: N uD1 nU01nD1 n.u.1/ Cu.2/ CCu.n//D Nx Ny!; SD1 n1U0CU D1 n1 n X iD1 .u.i/ N u/.u.i/ N u/0: Now he ollowing heo em is easy o con i m; o de ails, see, e.g., Pun anen, S yan & Iso alo [39, Ch. 3]. Theo em 6.1. Fo con o mable ma ices, he ollowing s a emen s hold: (a) The ec o N N yD Ny1is he o hogonal p ojec ion o he a iable ec o yon o he column space C.1/:N N yD Ny1D Jy DP1y. (b) The cen e ed a iable ec o Q yis he o hogonal p ojec ion o yon o he column space C.1/?:Q yDyJy D Cy D.InP1/y: 48 A. Ma kiewicz, S. Pun anen Now he es ima o Byis he BLUE o Xunde he model Fi and only i Bsa is ies he equa ion B.XWVX? /D.XW0/ : (34) Subs i u ing (33) in o (34) yields B11 B12 B21 B22! X Z RM 0 IqDZ0M!D X Z 0 0 Iq0!:(35) Using (35) Hasle & Pun anen [20, Th. 1] show ha all p ope ies o BLUEs and BLUPs in mixed model Lcan be conside ed using he augmen ed model F, whe e bo h ˇand a e ixed pa ame e s. Using he connec ion be ween he mixed model L he augmen ed model F, he ollowing esul ollows om Theo em 6.8 immedia ely. Theo em 6.12. Conside wo mixed models: LiD y;XˇCZ;Di;Rig;and deno e ˙iDZDiZ0CRiand ViDRi0 0 Di; i D1; 2: Then e e y ep esen a ion o he BLUE o Xˇunde L1 emains he BLUE o Xˇ unde L2and e e y ep esen a ion o he BLUP o unde L1 emains he BLUP o unde L2i and only i any o he ollowing equi alen condi ions holds: (a) E e y ep esen a ion o he BLUE o Xunde F1 emains he BLUE o Xunde F2. (b) C.V2X? /C.V1X? /. (c) C R2M D2Z0M!C R1M D1Z0M!: (d) C ˙2M D2Z0M!C ˙1M D1Z0M!: (e) The ma ix V2can be exp essed as V2DaV1CXN1X0 CV1X? N2.X? /0V1 o some a2Rand ma ices N1and N2such ha V2is nonnega i e de ini e. Acknowledgemen : Thanks go o he e e ees o help ul ema ks. Pa o his esea ch was done du ing he mee - ing o a Resea ch G oup on Mixed and Mul i a ia e Models in he Ma hema ical Resea ch and Con e ence Cen e , Be¸dlewo, Poland, Oc obe 2013, suppo ed by he S e an Banach In e na ional Ma hema ical Cen e . Re e ences [1] Baksala y J.K., An elemen a y de elopmen o he equa ion cha ac e izing bes linea unbiased es ima o s, Linea Algeb a Appl., 2004, 388, 3–6 [2] Baksala y J.K., Ma hew T., Linea su iciency and comple eness in an inco ec ly speci ied gene al Gauss–Ma ko model, Sankhy¯ a A, 1986, 48, 169–180 [3] Baksala y J.K., Ma hew T., Rank in a iance c i e ion and i s applica ion o he uni ied heo y o leas squa es, Linea Algeb a Appl., 1990, 127, 393–401 [4] Baksala y J.K., Pun anen S., S yan G.P.H., A p ope y o he dispe sion ma ix o he bes linea unbiased es ima o in he gene al Gauss–Ma ko model, Sankhy¯ a A, 1990, 52, 279–296 [5] Baksala y J.K., Rao C.R., Ma kiewicz A., A s udy o he in luence o he “na u al es ic ions” on es ima ion p oblems in he singu- la Gauss–Ma ko model, J. S a is . Plann. In e ence, 1992, 31, 335–351 [6] Baksala y O.M., T enkle G., A p ojec o o ien ed app oach o he bes linea unbiased es ima o , S a is . Pape s, 2009, 50, 721–733 [7] Baksala y O.M., T enkle G., Be ween OLSE and BLUE, Aus . N. Z. J. S a ., 2011, 53, 289–303 [8] Baksala y O.M., T enkle G., Rank o mulae om he pe spec i e o o hogonal p ojec o s, Linea Mul ilinea Algeb a, 2011, 59, 607–625 [9] Baksala y O.M., T enkle G., Liski E.P., Le us do he wis again. S a is . Pape s, 2013, 54, 1109–1119 [10] Ben-Is ael A., G e ille T.N.E., Gene alized in e ses: heo y and applica ions, 2nd Ed., Sp inge , New Yo k, 2003 [11] Ben-Is ael A., The Moo e o he Moo e–Pen ose in e se, Elec on. J. Linea Algeb a, 9, 150–157, 2002 [12] Bhimasanka am P., Sengup a D., The linea ze o unc ions app oach o linea models, Sankhy¯ a B, 1996, 58, 338–351 All abou he ?wi h i s applica ions in he linea s a is ical models 49 [13] Ch is ensen R., Plane answe s o complex ques ions: he heo y o linea models, 4 h Ed. Sp inge , New Yo k, 2011 [14] Da idson R., MacKinnon J.G., Econome ic heo y and me hods, Ox o d Uni e si y P ess, New Yo k, 2004 [15] F isch R., Waugh F.V., Pa ial ime eg essions as compa ed wi h indi idual ends, Econome ica, 1933, 1, 387–401 [16] G oß J., The gene al Gauss–Ma ko model wi h possibly singula dispe sion ma ix, S a is . Pape s, 2004, 45, 311–336 [17] G oß J., Pun anen S., Es ima ion unde a gene al pa i ioned linea model, Linea Algeb a Appl., 2000, 321, 131–144 [18] G oß J., Pun anen S., Ex ensions o he F isch–Waugh–Lo ell Theo em, Discuss. Ma h. P obab. S a ., 2005, 25, 39–49 [19] Ha ille D.A., Ma ix algeb a om a s a is ician’s pe spec i e, Sp inge , New Yo k, 1997 [20] Hasle S.J., Pun anen S., Equali y o BLUEs o BLUPs unde wo linea models using s ochas ic es ic ions, S a is . Pape s, 2010, 51, 465–475 [21] Hauke J., Ma kiewicz A., Pun anen S., Compa ing he BLUEs unde wo linea models, Comm. S a is . Theo y Me hods, 2012, 41, 2405–2418 [22] He D.G., On he his o y o he use o geome y in he gene al linea model, Ame . S a is ., 1980, 34, 43–47 [23] Iso alo J., Pun anen S., Linea p edic ion su iciency o new obse a ions in he gene al Gauss–Ma ko model, Comm. S a is . Theo y Me hods, 2006, 35, 1011–1023 [24] Iso alo J., Pun anen S., S yan G.P.H., A use ul ma ix decomposi ion and i s s a is ical applica ions in linea eg ession, Comm. S a is . Theo y Me hods, 2008, 37, 1436–1457 [25] Kala R., P ojec o s and linea es ima ion in gene al linea models, Comm. S a is . Theo y Me hods, 1981, 10, 849–873 [26] Kha i C.G., A no e on a MANOVA model applied o p oblems in g ow h cu es, Ann. Ins . S a is . Ma h., 1966, 18, 75–86 [27] K uskal W., When a e Gauss–Ma ko and leas squa es es ima o s iden ical? A coo dina e- ee app oach, Ann. Ma h. S a is ., 1968, 39, 70–75 [28] LaMo e L.R., A di ec de i a ion o he REML likelihood unc ion, S a is . Pape s, 2007, 48, 321–327 [29] Lo ell M.C., Seasonal adjus men o economic ime se ies and mul iple eg ession analysis, J. Ame . S a is . Assoc., 1963, 58, 993–1010 [30] Lo ell M.C., A simple p oo o he FWL Theo em, J. Econ. Educ., 2008, 39, 88–91 [31] Ma golis M.S., Pe pendicula p ojec ions and elemen a y s a is ics, Ame . S a is ., 1979, 33, 131–135 [32] Ma kiewicz A., On dependence s uc u es p ese ing op imali y, S a is . P obab. Le ., 2001, 53, 415–419 [33] Ma kiewicz A., Pun anen S., S yan G.P.H., A no e on he in e p e a ion o he equali y o OLSE and BLUE, Pakis an J. S a is ., 2010, 26, 127–134 [34] Ma saglia G., S yan G.P.H., Equali ies and inequali ies o anks o ma ices, Linea Mul ilinea Algeb a, 1974, 2, 269–292 [35] Mi a S.K., Moo e B.J., Gauss–Ma ko es ima ion wi h an inco ec dispe sion ma ix, Sankhy¯ a A, 1973, 35, 139–152 [36] Mi a S.K., Rao C.R., P ojec ions unde semino ms and gene alized Moo e–Pen ose in e ses, Linea Algeb a Appl., 1974, 9, 155–167 [37] Pun anen S., S yan G.P.H., The equali y o he o dina y leas squa es es ima o and he bes linea unbiased es ima o (wi h dis- cussion), Ame . S a is ., 1989, 43, 151–161 [Commen ed by O. Kemp ho ne on pp. 161–162 and by S.R. Sea le on pp. 162–163, Reply by he au ho s on p. 164] [38] Pun anen S., S yan G.P.H., Reply [ o R. Ch is ensen (1990), R.W. Fa eb o he (1990), and D.A. Ha ille (1990)] (Le e o he Edi o ), Ame . S a is ., 1990, 44, 192–193 [39] Pun anen S., S yan G.P.H., Iso alo J., Ma ix icks o linea s a is ical models: ou pe sonal op wen y, Sp inge , Heidelbe g, 2011 [40] Rao C.R., Leas squa es heo y using an es ima ed dispe sion ma ix and i s applica ion o measu emen o signals. In: P oceed- ings o he Fi h Be keley Symposium on Ma hema ical S a is ics and P obabili y: Be keley, Cali o nia, 1965/1966, ol. 1, L.M. Le Cam and J. Neyman, eds., Uni e si y o Cali o nia P ess, Be keley, 355–372, 1967 [41] Rao C.R., A no e on a p e ious lemma in he heo y o leas squa es and some u he esul s, Sankhy¯ a A, 1968, 30, 259–266 [42] Rao C.R., Uni ied heo y o linea es ima ion, Sankhy¯ a A 1971, 33, 371–394. [Co igendum (1972), 34, p. 194 and p. 477] [43] Rao C.R., Linea s a is ical in e ence and i s applica ions, 2nd Ed., Wiley, New Yo k, 1973 [44] Rao C.R., Rep esen a ions o bes linea unbiased es ima o s in he Gauss–Ma ko model wi h a singula dispe sion ma ix, J. Mul i a ia e Anal., 1973, 3, 276–292 [45] Rao C.R., P ojec o s, gene alized in e ses and he BLUE’s, J. R. S a . Soc. Se . B S a . Me hodol., 1974, 336, 442–448 [46] Rao C.R., Mi a S.K., Gene alized n e se o ma ices and i s applica ions, Wiley, New Yo k, 1971 [47] Rao C.R., Rao M.B., Ma ix algeb a and i s applica ions o s a is ics and econome ics, Wo ld Scien i ic, Ri e Edge, NJ, 1998 [48] Sea le S.R., Casella G., McCulloch C.E., Va iance componen s, Wiley, New Yo k, 1992 [49] Sebe G.A.F., The linea hypo hesis: a gene al heo y, 2nd Ed., G i in, London, 1980 [50] Sebe G.A.F., Lee A.J., Linea eg ession analysis, 2nd Ed. Wiley, New Yo k, 2003 [51] Sengup a D., Jammalamadaka S.R., Linea models: an in eg a ed app oach, Wo ld Scien i ic, Ri e Edge, NJ., 2003 [52] Tian Y., On equali ies o BLUEs unde misspeci ied Gauss–Ma ko models, Ac a Ma h. Sin. (Engl. Se .), 2009, 25, 1907–1920 [53] Tian Y., Beisiegel, M., Dagenais E., Haines C., On he na u al es ic ions in he singula Gauss–Ma ko model, S a is . Pape s, 2008, 49, 553–564 [54] Tian Y., Takane Y., Some p ope ies o p ojec o s associa ed wi h he WLSE unde a gene al linea model. J. Mul i a ia e Anal., 2008, 99, 1070–1082 [55] Tian Y., Takane Y., On V-o hogonal p ojec o s associa ed wi h a semi-no m, Ann. Ins . S a is . Ma h., 2009, 61, 517–530 50 A. Ma kiewicz, S. Pun anen [56] T enkle G., On he singula i y o he sample co a iance ma ix, J. S a . Compu . Simul., 1995, 52, 172–173 [57] Wa son G.S., Se ial co ela ion in eg ession analysis, I, Biome ika, 1955, 42, 327–341 [58] Zyskind G., On canonical o ms, non-nega i e co a iance ma ices and bes and simple leas squa es linea es ima o s in linea models, Ann. Ma h. S a is ., 1967, 38, 1092–1109 [59] Zyskind G., Ma in F.B., On bes linea es ima ion and gene al Gauss–Ma ko heo em in linea models wi h a bi a y nonnega i e co a iance s uc u e, SIAM J. Appl. Ma h., 1969, 17, 1190–1202