Supporting vectors vs. principal components

h p://www.aimsp ess.com/jou nal/Ma h AIMS Ma hema ics, 8(1): 1937–1958. DOI:10.3934/ma h.2023100 Recei ed: 10 July 2022 Re ised: 29 Sep embe 2022 Accep ed: 08 Oc obe 2022 Published: 26 Oc obe 2022 Resea ch a icle Suppo ing ec o s s. p incipal componen s Almudena P. M´ a quez1,*, F ancisco Ja ie Ga c´ ıa-Pacheco1, M´ ı iam Mengiba -Rod ´ ıguez2and Albe o S´ anchez-Alzola3 1Depa men o Ma hema ics, College o Enginee ing, Uni e si y o Cadiz, 11519 Pue o Real, Spain 2Depa men o Compu e Science and A i icial In elligence, Uni e si y o G anada, 18071 G anada, Spain 3Depa men o S a is ics and Ope a ion Resea ch, Uni e si y o Cadiz, 11519 Pue o Real, Spain *Co espondence: Email: [email p o ec ed]. Abs ac : Le T:X→Ybe a bounded linea ope a o be ween Banach spaces X,Y. A ec o x0∈SX in he uni sphe e SXo Xis called a suppo ing ec o o Tp o ided ha kT(x0)k=sup{kT(x)k:kxk= 1}=kTk. Since ma ices induce linea ope a o s be ween ini e-dimensional Hilbe spaces, we can conside hei suppo ing ec o s. In his manusc ip , we un eil he ela ionship be ween he p incipal componen s o a ma ix and i s suppo ing ec o s. Applica ions o ou esul s o eal-li e p oblems a e p o ided. Keywo ds: bounded linea ope a o ; Hilbe space; mean ope a o ; p incipal componen s; suppo ing ec o Ma hema ics Subjec Classi ica ion: 51F30, 54E35, 54E45 1. In oduc ion Suppo ing Vec o Analysis (SVA) is a ela i ely ecen echnique ha allows one o sol e analy ically many eal-li e p oblems ha used o be ackled by means o Heu is ic me hods. The lack o ma hema ical o malism o Heu is ic me hods esul ed many imes in unp edic able solu ions, ha is, ma hema ical solu ions whose eal-li e in e p e a ions make no sense. Suppo ing ec o s came in o play o o e come his issue. This way, suppo ing ec o s we e used in a success ul way o sol e mul iobjec i e op imiza ion p oblems coming om di e en disciplines, such as Bioenginee ing, Physics, and S a is ics [4, 6–9, 15, 22], imp o ing conside ably he esul s achie ed by o he me hods like, o ins ance, Heu is ic echniques [10,11,20, 21]. In [4, 6, 15], i was p o en ha Singula Value Decomposi ion (SVD) can be seen as a pa icula case o SVA. This ac igge ed he new end o es a ing S a is ical no ions om he pe spec i e o 1938 Func ional Analysis and Ope a o Theo y. The main objec i e o his manusc ip is o s udy P incipal Componen Analysis (PCA) by means o SVA. 2. Ma e ials and me hods We will e iew se e al basic no ions om Ope a o Theo y ha will u n ou o be c ucial o he de elopmen o his manusc ip . 2.1. Cen e ing and s anda dizing I x=(x1,...,xn)∈Rn, hen he mean o xis de ined as x:=1 nPn i=1xi, and i s s anda d de ia ion is gi en by sx:=q1 nPn i=1(xi−x)2. No ice ha √nsx=x−x2,(2.1.1) whe e x:=(x,n . . ., x)deno es he cons an ec o o e m x(in gene al, i a∈R, hen a:=(a,n . . ., a) deno es he cons an ec o o e m a). We say ha x∈Rnis cen e ed p o ided ha x=0, and i is s anda dized p o ided ha x=0 and sx=1. In he la e si ua ion, kxk2=√n, in iew o (2.1.1). The subse o cen e ed ec o s o Rnis usually deno ed by cen(Rn), ha is, cen(Rn) :={x∈Rn:x=0}.The subse o s anda dized ec o s o Rnis usually deno ed by s an(Rn), ha is, s an(Rn) :={x∈Rn:x=0 and sx=1}. Acco ding o (2.1.1), s an (Rn)⊆√nS`n 2, whe e S`n 2s ands o he uni sphe e o `n 2:=(Rn,k·k2). In Topology, S`n 2is deno ed as Sn−1. 2.2. P incipal componen analysis The co a iance o wo ec o s x,y∈Rnis de ined as sx,y:=1 n n X i=1 (xi−x) (yi−y). No ice ha sx,x=s2 x, ha is, he a iance o x. The co a iance ma ix o a gi en ma ix A∈Mm×nis de ined by sa1,...,an:=sai,aji,j=1,...,n, whe e a1,...,ans and o he column ec o s o A. Conside a ma ix A∈Mm×n. The p incipal componen s o Aa e de ined as Ax1,...,Axn, whe e {x1,...,xn}is an o de ed o hono mal basis o eigen ec o s o sa1,...,an, so ing he eigen alues o sa1,...,an dec easingly. We e e he eade o [24] o a wide pe spec i e on PCA. In e es ing applica ions o PCA o ce ain Enginee ing ields, such as ideo p ocessing and Big Da a, ha e been p o ided in [3,12]. 2.3. Suppo ing ec o analysis Le X,Ybe Banach spaces. Le T:X→Ybe a bounded linea ope a o . The ope a o no m o Tis gi en by kTk:=sup{kT(x)k:kxk=1}.(2.3.1) AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1939 The ec o space CL(X,Y) o con inuous linea ope a o s om X o Ybecomes a Banach space when endowed wi h he ope a o no m (2.3.1). In he case X=Y,CL(X,Y) is simply deno ed as CL(X). I Y=K(Ro C), hen CL(X,Y) is deno ed as X∗, ha is, he dual space o X. I is also common o deno e CL(X,Y) by B(X,Y) and CL(X) by B(X). The suppo ing ec o no ion was o mally posed o he i s ime in [5]. Howe e , his concep can be ound implici ly and sca e ed h oughou he li e a u e o Banach Space Theo y [1,2, 18,19]. The se o suppo ing ec o s o a bounded linea ope a o T:X→Ybe ween Banach spaces X,Y is de ined by supp (T) :={x∈SX:kT(x)k=kTk} =a g max kxk=1kT(x)k.(2.3.2) He e, SXs ands o he uni sphe e o X, and BXdeno es he (closed) uni ball o X. In he in ini e- dimensional se ing, i may occu ha (2.3.2) is emp y. No e ha supp (T)=supp (λT) o all λ∈ K {0}, and supp (T)=SKsupp (T), whe e K=Ro C. Fo a opological and geome ical analysis o he abo e se , we s ongly e e he eade o [13,14,23]. Fo linea unc ionals, a special subse o suppo ing ec o s is wo h ega ding. Conside a con inuous linea unc ional ∈X∗in he dual X∗o a Banach space X. We de ine he se o 1- suppo ing ec o s o by supp 1( ) :={x∈SX: (x)=k k}.(2.3.3) No ice ha 1-suppo ing ec o s a e pa icula cases o suppo ing ec o s; in o he wo ds, supp 1( )⊆ supp ( ). In he upcoming sec ions, 1-suppo ing ec o s will be e y much elied on. The ollowing ema k highligh s a s anda d geome ical p ope y sa is ied by 1-suppo ing ec o s. Rema k 2.1. Conside a Banach space X and a nonze o linea unc ional ∈X∗ {0}. Fo e e y x,y∈supp 1( )and e e y λ∈[0,1], we ha e ha λx+(1 −λ)y∈supp 1( ), ha is, supp 1( )is a con ex subse o he uni sphe e SXo X. A di ec consequence o Rema k 2.1 is ha supp 1( ) is ei he emp y o a single on in s ic ly con ex Banach spaces, like, o ins ance, Hilbe spaces. 2.4. Hilbe space heo y Rep esen a ion Theo y is one o he mos impo an heo ies in Ma hema ics. A majo esul in Rep esen a ion Theo y is undoub edly he Riesz Rep esen a ion Theo em. This is a key esul in Func ional Analysis and is c ucial o wo king wi h sel -adjoin ope a o s on Hilbe spaces. Riesz Rep esen a ion Theo em. In a Hilbe space H, o e e y h∗∈H∗ he e exis s a unique h ∈H sa is ying h∗=(·|h). This assignmen be ween H and H∗is a su jec i e linea isome y. In iew o Rema k 2.1 and unde he se ings o he Riesz Rep esen a ion Theo em, o e e y h∈ H {0}, we ha e ha supp 1(h∗)=nh khko, ha is, h khkis he only 1-suppo ing ec o o h∗. On he o he hand, he o hogonal subspace o a closed subspace Vo a Hilbe space His deno ed by V⊥. The o hogonal p ojec ion o Hon o Vis usually deno ed by pV. Obse e ha H=V⊕2V⊥, ha is, o each h∈H, h=pV(h)+pV⊥(h),and khk2=kpV(h)k2+kpV⊥(h)k2.(2.4.1) AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1940 The adjoin o a bounded linea ope a o T:H→Kbe ween Hilbe spaces H,Kis de ined as he unique bounded linea ope a o T∗:K→Hsa is ying (T(h)|k)=(h|T∗(k)) o each h∈Hand each k∈K. Basic p ope ies sa is ied by he adjoin ope a o a e kT∗k=kTk,(T∗)∗=T, (T+S)∗=T∗+S∗, (T◦S)∗=S∗◦T∗and (λT)∗=λT∗. Lemma 2.2. E e y bounded linea ope a o T :H→K be ween Hilbe spaces H,K e i ies ha cl(T(H)) =ke (T∗)⊥and T(H)⊥=cl(T(H))⊥=ke (T∗). P oo . Fi s o , i is a i ial obse a ion ha T(H)⊥=cl(T(H))⊥. Fix an a bi a y h∈H. Fo e e y k∈ke (T∗), (T(h)|k)=(h|T∗(k))=(h|0) =0. This shows ha T(h)∈ke (T∗)⊥. The a bi a iness o h∈Hmeans ha T(H)⊆ke (T∗)⊥. By aking o hogonal complemen s, we ob ain ha ke (T∗)⊆T(H)⊥. Con e sely, ix an a bi a y k∈T(H)⊥. Fo e e y h∈H, 0 =(T(h)|k)=(h|T∗(k)). This shows ha T∗(k)=0, and hence k∈ke (T∗). The a bi a iness o k∈T(H)⊥means ha T(H)⊥⊆ke (T∗). By aking o hogonal complemen s, we inally ob ain ha ke (T∗)⊥⊆cl(T(H)).  A bounded ope a o T∈B(H) is said o be sel -adjoin p o ided ha T∗=T. I His complex, hen T∈B(H) is sel -adjoin i and only i (T(h)|h)∈R o all h∈H. A sel -adjoin ope a o is called posi i e p o ided ha (T(h)|h)≥0 o all h∈H. Fo e e y T∈B(H), σ(T) :={λ∈C:λI−T<U(B(H))}is he spec um o T, whe e U(B(H))is he mul iplica i e g oup o in e ible ope a o s on H. Among o he spec al p ope ies, he spec um is compac and nonemp y, and kTk ≥ max |σ(T)|. A special subse o he spec um, called he poin spec um, σp(T) :={λ∈C: ke (λI−T),{0}},whose elemen s a e called he eigen alues o T, will be e y much employed. No e ha σp(T)⊆σ(T). Fu he mo e, o each λ∈σp(T), VT(λ) := {h∈H:T(h)=λh}s ands o he subspace o eigen ec o s associa ed wi h λ. In case he e is no con usion wi h T, we will simply deno e VT(λ) by V(λ). Suppose nex ha kTkis an eigen alue o T, ha is, kTk ∈ σp(T). In his si ua ion, since kTk ≥ max |σ(T)|, we conclude ha kTkis he maximum o |σ(T)|; in o he wo ds, kTk=max |σ(T)|. In his case, we w i e kTk=λmax(T). Obse e also ha V(kTk)∩SX⊆supp (T). Indeed, i x∈V(kTk)∩SX, hen T(x)=kTkx, so kT(x)k=kTk, and hence x∈supp (T). Ne e heless, in gene al, kTk<σp(T), unless, o ins ance, Tis compac , sel -adjoin and posi i e. This is why we ha e o ely on he adjoin T∗and on he s ongly posi i e ope a o T∗◦T. I is s aigh o wa d o check ha he eigen alues o a sel -adjoin ope a o a e eal, and he eigen alues o a sel -adjoin posi i e ope a o a e posi i e. When Tis compac , i holds ha T∗◦Tis compac , sel -adjoin and posi i e. The ollowing esul , on which we will s ongly ely la e on, can be ound in [6, Theo em 4], which is i sel a e inemen o [14, Theo em 9]. Theo em 2.3. Le H,K be Hilbe spaces. Le T ∈B(H,K). Then, kTk2=kT∗◦Tk, and supp (T)⊆ supp (T∗◦T). Fu he mo e, supp (T),∅i and only i kT∗◦Tk ∈ σp(T∗◦T). In his si ua ion, kTk=√λmax (T∗◦T), and supp (T)=V(λmax(T∗◦T))∩SH. 3. Resul s In his sec ion, we will s a e and p o e all he no el heo ems o his wo k. This sec ion is di ided in o ou subsec ions, in which we will deal wi h suppo ing ec o s, p incipal componen s and he AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1941 opological s uc u e o he subse s o cen e ed and s anda dized ec o s. 3.1. Topological s uc u e o he subse s o cen e ed and s anda dized ec o s This subsec ion begins un eiling he opological s uc u e o he subse s cen(Rn) and s an(Rn) o cen e ed and s anda dized ec o s, espec i ely. No ice ha cen(R)={0}and s an(R)=∅. Theo em 3.1. I n ≥1, hen cen(Rn)is linea ly isome ic, and hence homeomo phic, o `n−1 2. I n ≥2, hen s an(Rn)is linea ly isome ic, and hence homeomo phic, o Sn−2. P oo . No ice ha x+y=x+y, and x = x o all x∈Rnand all ∈R. As a consequence, ·:Rn→R x7→ x=1 nPn i=1xi (3.1.1) is a linea unc ional (usually called he mean unc ional). Nex , simply obse e ha cen(Rn)=ke (·). Thus, by bea ing in mind (2.1.1), we immedia ely ob ain ha s an(Rn)=ke (·)∩√nS`n 2=√nSke (·). In o he wo ds, s an(Rn) is p ecisely a mul iple o he uni sphe e o he Hilbe subspace ke (·)o `n 2. Since dim (ke (·)) =n−1, we ha e ha ke (·)is linea ly isome ic o `n−1 2. As a consequence, he uni sphe e o ke (·),1 √ns an(Rn), is linea ly isome ic o he uni sphe e o `n−1 2,S`n−1 2=Sn−2. The ollowing lemma aims a compu ing he no m o he mean unc ional as an elemen o he dual space o `n 2as well as i s only 1-suppo ing ec o . Lemma 3.2. In `n 2∗,k·k=1 √nand supp 1(·)=n1 √no. P oo . H¨ olde ’s Inequali y ensu es ha |x|= 1 n n X i=1 xi≤ n X i=1 1 n|xi| ≤  n X i=1 1 n2 1 2 n X i=1 x2 i 1 2 =1 √nkxk2 o e e y x∈`n 2. This shows ha k·k≤1 √n. On he o he hand, k1k2=√n, ha is, 1 √n∈S`n 2. Finally, 1 √n=1 n n X i=1 1 √n=1 √n. As a consequence, k·k=1 √nand supp 1(·)=n1 √no. As a di ec consequence o Lemma 3.2, he s anda d de ia ion o a ec o x∈Rncan be ew i en as he ollowing (2.1.1): sx=1 √nx−x2=k·kx−x2.(3.1.2) By bea ing in mind he Riesz Rep esen a ion Theo em, Lemma 3.2 assu es ha i h:=1 n, hen h∗=·in H:=`n 2. No ice ha h:=1 n=1 n,n . . ., 1 ncan be seen as a ( ini e) con ex se ies. This ac will help us gene alize hese concep s o an in ini e-dimensional sepa able Hilbe space se ing la e on in he Discussion. AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1942 De ini ion 3.3. Fo n ≥1, he mean ope a o is de ined by •:Rn→Rn x7→ x=(x,n . . ., x),(3.1.3) and he cen e ing ope a o is de ined as cen : Rn→Rn x7→ cen(x) :=x−x=(x1−x,n . . ., xn−x).(3.1.4) Fo n ≥2, he s anda dizing ope a o is de ined as s an : Rn Rn1→Rn x7→ s an(x) :=√nx−x kx−xk2.(3.1.5) I is a i ial obse a ion ha s an(x)=√ncen(x) kcen(x)k2 =cen(x) k·kkcen(x)k2 (3.1.6) o all x∈Rn. Recall ha a p ojec ion on a Banach space Xis a con inuous, linea , and idempo en map P:X→X. I s dual ope a o P∗:X∗→X∗is also a p ojec ion. The complemen a y p ojec ion o Pis de ined as IX−P, which is also a p ojec ion. E e y non-ze o p ojec ion has no m g ea e han o equal o 1. A 1-p ojec ion is a p ojec ion o no m 1 (also called a con ac i e p ojec ion), and a (1,1)-p ojec ion is a 1-p ojec ion whose complemen a y p ojec ion is also a 1-p ojec ion (also called bicon ac i e). O hogonal p ojec ions in Hilbe spaces a e he mos ep esen a i e examples o bicon ac i e p ojec ions. The inal esul o his i s subsec ion se es o show ha bo h he mean ope a o and he cen e ing ope a o a e complemen a y p ojec ions o each o he o no m 1, ha is, bicon ac i e, o he Euclidean no m. E en mo e, he mean ope a o and he cen e ing ope a o a e o hogonal p ojec ions o each o he . This will allow us o di ec ly ob ain he K¨ onig-Huygens Theo em (3.1.7) as a di ec consequence o he Py hago ean Theo em in Hilbe spaces. The K¨ onig-Huygens Theo em p o ides he classical decomposi ion o he 2-no m o a ec o o Rnin e ms o he mean and he s anda d de ia ion. Theo em 3.4. The cen e ing ope a o is a linea p ojec ion on Rnwhose ke nel is ke (cen) =Rn1, whose ange is cen(Rn)=ke (·), and whose complemen a y p ojec ion is he mean ope a o . Fu he mo e, i we conside Rnendowed wi h he Euclidean no m, hen k•k =kcenk=1, and • and cen a e complemen a y o hogonal p ojec ions. As a consequence, o e e y x ∈Rn, kxk2 2=kxk2 2+kx−xk2 2=n|x|2+s2 x.(3.1.7) P oo . In he i s place, no ice ha •is clea ly linea , since •=·1; in o he wo ds, x=(x,n . . ., x)= x(1,n . . ., 1)=x1 o all x∈Rn. As a consequence, cen is linea as well because cen =IRn−•, ha is, he di e ence o wo linea ope a o s on Rn. On he o he hand, obse e ha i x∈Rnis a cons an ec o , ha is, x= 1= o some ∈R, hen cen(x)=0. Con e sely, i cen(x)=0, hen x=x=x1, and hence x∈Rn1is a cons an ec o . This shows ha ke (cen) =Rn1. F om Theo em 3.1, we al eady AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1943 know ha cen(Rn)=ke (·). Nex , le us p o e ha cen is a p ojec ion. Fix an a bi a y x∈Rn. No ice ha cen (cen(x))=cen x−x=cen (x)−cen x=x−x−0=x−x=cen (x). Since •=IRn−cen, we conclude ha he mean ope a o is he complemen a y p ojec ion o he cen e ing ope a o . Finally, le us compu e k•k and kcenk. In acco dance wi h Lemma 3.2, o e e y x∈Rnwe ha e ha kxk2=√n|x| ≤ √n1 √nkxk2=kxk2, meaning ha k•k ≤ 1. Nex ,  1 √n2 = 1 √n,n . . ., 1 √n!2 =√n1 √n=1. This shows ha k•k =1. In o de o p o e ha •and cen a e o hogonal p ojec ions, i only su ices o ealize ha R1and cen(Rn) a e o hogonal subspaces. Indeed, o e e y ∈Rand e e y x∈Rnwi h x=0, we ha e ha ( 1|x)= (1|x)= Pn i=1xi= nx =0, and hence R1⊆cen(Rn)⊥, o equi alen ly, cen(Rn)⊆(R1)⊥. Fu he mo e, he Py hago ean Theo em in `n 2allows ha k 1+xk2 2=k 1k2 2+kxk2 2 o e e y ∈Rand e e y x∈Rnwi h x=0. Nex , i y∈(R1)⊥, hen 0 =1 n(1|y)=1 nPn i=1yi=y, meaning ha y∈ke (·)=cen(Rn). As a consequence, cen(Rn)⊥=R1, esul ing in kxk2 2=kxk2 2+kx−xk2 2=n|x|2+s2 x o e e y x∈Rn. I only emains o show ha kcenk=1, bu his is a di ec consequence o he ac ha cen is an o hogonal p ojec ion.  3.2. Suppo ing ec o s and i s p incipal componen In his subsec ion, we will p o ide su icien condi ions o he suppo ing ec o s o coincide wi h he i s p incipal componen . Fi s , we will need some de ini ions. De ini ion 3.5. A ma ix is said o be cen e ed (s anda dized) p o ided ha all o i s column ec o s a e cen e ed (s anda dized) ec o s. The ollowing lemma displays a simple cha ac e iza ion o cen e ed ma ices. Lemma 3.6. Le A ∈Mm×n(R). The ollowing condi ions a e equi alen : 1) A is cen e ed. 2) Ax is cen e ed o all x ∈Rn. P oo . Le {e1,...,en}deno e he canonical basis o Rn. No ice ha he columns o Aa e p ecisely Aej o j=1,...,n. Suppose i s ha Ais cen e ed. Fix an a bi a y x∈Rn. The linea i y o he mean unc ional allows ha Ax = n X j=1 xjAej= n X j=1 xjAej=0. AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1944 As a consequence, Ax is cen e ed o all x∈Rn. Con e sely, suppose now ha Ax is cen e ed o all x∈Rn. In pa icula , Aej=0 o j=1,...,n, meaning ha he columns o Aa e cen e ed ec o s, ha is, Ais cen e ed.  The ollowing cha ac e iza ion o cen e ed ma ices is a bi mo e sophis ica ed. Fi s , a echnical lemma is needed. Lemma 3.7. Conside x,y∈Rm. Then, 1) msx,y=x•y i and only i ei he x o y is cen e ed. 2) x is s anda dized i and only i x is cen e ed and x •x=m. P oo . 1) Le us obse e ha msx,y= m X i=1 (xi−x) (yi−y) = m X i=1 xiyi− m X i=1 xyi− m X i=1 xiy+ m X i=1 x y = m X i=1 xiyi−x m X i=1 yi−y m X i=1 xi+x y m X i=1 1 = m X i=1 xiyi−xmy −ymx +x y =x•y−mx y. As a consequence, msx,y=x•yi and only i ei he xo yis cen e ed. 2) By de ini ion, xis s anda dized i and only i xis cen e ed and sx=1. We know ha hen kx−xk2=√msx. In he con ex o cen e ed ec o s, he p e ious exp ession becomes √x•x= kxk2=√msx. The e o e, xis s anda dized i and only i xis cen e ed and x•x=m.  Recall ha he co a iance ma ix o a gi en ma ix A∈Mm×nis de ined by sa1,...,an:=sai,aji,j=1,...,n, whe e a1,...,ans and o he column ec o s o A. Also, ecall ha i Bis a squa e ma ix, hen diag(B) s ands o he diagonal o B. P oposi ion 3.8. Le A ∈Mm×n(R). The ollowing condi ions a e equi alen : 1) A is cen e ed. 2) A A=msa1,...,an. 3) diag A A=diag msa1,...,an. AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1945 P oo . No ice ha A A=ai•aji,j=1,...,n. Suppose i s ha Ais cen e ed. In iew o Lemma 3.7, msai,aj=ai•aj o all i,j∈ {1,...,n}, meaning ha A A=msa1,...,an. Nex , i A A=msa1,...,an, hen we i ially ha e ha diag A A=diag msa1,...,an. Finally, assume ha diag A A=diag msa1,...,an. Then, msai,ai=ai•ai o all i=1,...,n. In acco dance wi h Lemma 3.7, aiis cen e ed o all i=1,...,n, meaning ha Ais cen e ed.  As an immedia e consequence o Lemma 3.7 and P oposi ion 3.8, we ob ain he ollowing co olla y. Co olla y 3.9. Le A ∈Mm×n(R). The ollowing condi ions a e equi alen : 1) A is s anda dized. 2) A A=msa1,...,anand diag A A=(m,n . . ., m). Finally, we ha e ga he ed all he necessa y ools o p o e ou main esul s o his subsec ion. Simply keep in mind he small obse a ion ha i B∈Mn(R), hen σp(αB)=ασp(B) and VαB(αλ)=VB(λ) o all α∈R {0}and all λ∈σp(B). Theo em 3.10. Le A ∈Mm×n(R). I A is cen e ed, hen supp (A)=nx∈S`n 2:Ax is he i s p incipal componen o Ao, whe e A is seen as a linea ope a o A:`n 2→`m 2 x7→ Ax.(3.2.1) P oo . Acco ding o Theo em 2.3, supp (A)=V(λmax(A∗◦A))∩S`n 2. No ice ha he adjoin o A, A∗, coincides wi h i s anspose, A . On he o he hand, since Ais cen e ed, P oposi ion 3.8 allows ha A A=msa1,...,an. Finally, supp (A)=VA∗◦A(λmax(A∗◦A))∩S`n 2 =VA Aλmax(A A)∩S`n 2 =Vmsa1,...,anλmax(msa1,...,an)∩S`n 2 =Vmsa1,...,anmλmax(sa1,...,an)∩S`n 2 =Vsa1,...,anλmax(sa1,...,an)∩S`n 2 =nx∈S`n 2:Ax is he i s p incipal componen o Ao.  We will conclude his subsec ion wi h an example o a cen e ed ma ix Awhose las p incipal componen has a leas wo dimensions. Rema k 3.11. Le T :H→K be a bounded linea ope a o be ween Hilbe spaces H,K. Then, ke (T)⊆ke (T∗◦T). As a consequence, i 0∈σp(T), hen 0∈σp(T∗◦T). No ice ha i T∈B(H) is a sel -adjoin posi i e ope a o on a Hilbe space Hsuch ha ke (T), {0}, hen 0 is he minimum o σp(T) since all he eigen alues o Ta e posi i e. AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1952 4. Discussion We will discuss how o anspo he mean unc ional, he mean ope a o and he cen e ing ope a o o abs ac se ings, such as Hilbe spaces, Banach algeb as and p obabili y spaces. 4.1. Gene aliza ion o Hilbe spaces As men ioned in Lemma 3.2, in iew o he Riesz Rep esen a ion Theo em, i h0:=1 n=1 n,n . . ., 1 n, hen h∗ 0=·is p ecisely he mean unc ional in H:=`n 2. Ac ually, kh0k2=1 √n=kh∗ 0k. A his s age, he key is o ealize ha h0can be seen as a ( ini e) con ex se ies. Recall ha a con ex se ies is a con e gen se ies P∞ n=1 nsuch ha P∞ n=1 n=1 and n≥0 o all n∈N. No ice ha i P∞ n=1 nis a con ex se ies, hen P∞ n=1 2 n≤P∞ n=1 n=1, and hence ( n)n∈N∈`2. Now, we a e in he igh posi ion o de ine he no ions o mean and s anda d de ia ion on sepa able Hilbe spaces. Le Hbe a sepa able Hilbe space wi h an o hono mal basis (en)n∈N. Fix a con ex se ies P∞ n=1 n. Le h∈Hand w i e h=P∞ n=1(h|en)en. The mean o h, wi h espec o ( n)n∈N, is de ined as h:=∞ X n=1 n(h|en).(4.1.1) The mean unc ional, wi h espec o ( n)n∈N, is gi en by ·:H→K h7→ h:=P∞ n=1 n(h|en).(4.1.2) By elying on H¨ olde ’s Inequali y, i is no ha d o check ha he mean unc ional is an elemen o H∗ whose no m is p ecisely pP∞ n=1 2 n=k( n)n∈Nk2. In acco dance wi h he Riesz Rep esen a ion Theo em, he e exis s h0∈Hsuch ha (h|h0)=h o all h∈H. I we le h=P∞ n=1(h|en)en, hen we ob ain ha ∞ X n=1 (h|en) (en|h0)= ∞ X n=1 (h|en)en h0 =∞ X n=1 n(h|en).(4.1.3) By aking h=en o e e y n∈Nin (4.1.3), we conclude ha (en|h0)= n o e e y n∈N. In pa icula , h0=P∞ n=1 nen. As expec ed, kh0k=k( n)n∈Nk2= ∞ X n=1 2 n=k·k. In iew o he Riesz Rep esen a ion Theo em, h∗ 0=(·|h0)=·. No ice ha , in o de o conclude ha h∗ 0=(·|h0)=·, he Riesz Rep esen a ion Theo em is eally no needed. Le us ge back o a second o `n 2. Fo e e y x∈`n 2, x=(x,n . . ., x)=x1=x 1 √n 1 n 1 √n =· 1 √n (x) 1 n 1 √n . Then, going back o a gene al sepa able Hilbe space H, he mean ope a o is de ined as H→H h7→ h:=h∗ 0 kh∗ 0k(h)h0 kh0k.(4.1.4) AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1953 I is no ha d o check ha he mean ope a o is an o hogonal p ojec ion on Hwhose complemen a y p ojec ion is, p ecisely, he cen e ing ope a o : H→H h7→ cen(h) :=h−h.(4.1.5) 4.2. Gene aliza ion o Banach algeb as A Banach algeb a is a eal o complex algeb a Aendowed wi h a comple e ec o no m ha is also a ing no m, ha is, kabk≤kakkbk o all a,b∈A. We say ha Ais uni al i i is uni a y, ha is, i has a uni y 1∈A, and k1k=1. In his si ua ion, acco ding o he Hahn-Banach Theo em, he e exis s a con inuous linea unc ional, which we will deno e by 1∗∈A∗, such ha k1∗k=1 and 1∗(1)=1. Then, he mean unc ional is p ecisely 1∗, ha is, de ined by A→K a7→ a:=1∗(a).(4.2.1) The mean ope a o is de ined as A→A a7→ a:=1∗(a)1.(4.2.2) Finally, he cen e ing ope a o is de ined as A→A a7→ cen(a) :=a−a.(4.2.3) Following (3.1.2), he a iance o an elemen a∈Acan be de ined by s2 a:=1∗a−a2=1∗(a2)−1∗(a)2.(4.2.4) No ice ha his gene aliza ion o uni al Banach algeb as p esen s a weakness: The exis ence o he unc ional 1∗∈A∗is gua an eed by he Hahn-Banach Theo em, bu i s uniqueness is no gua an eed. In ac , in many uni al Banach algeb as, such as `∞(Λ) o ins ance, 1is no a smoo h poin o B`∞(Λ)[16, Theo em 2.9], and hus he e a e in ini ely many unc ionals o no m 1 a aining hei no m a 1. I seems no i ial o o e come his issue. Maybe an op ion is o y o eno m equi alen ly he uni al Banach algeb a in such a way ha 1becomes a smoo h poin o he new uni ball o he algeb a o , a leas , o ind ano he smoo h poin in he uni ball. Acco ding o [16, Theo em 2.1], he canonical uni ec o eλis a smoo h poin o B`∞(Λ) o each λ∈Λ. Ano he possibili y may ely on cons uc ing he mean unc ional in a C∗-algeb a. Recall ha a C∗-algeb a is a Banach algeb a Aendowed wi h an an imul iplica i e and an ilinea in olu ion ∗:A→Asa is ying ha kaa∗k=kak2 o all a∈A. 4.3. Gene aliza ion o p obabili y spaces A p obabili y space is a 3- uple (Ω,Σ,P) whe e (Ω,Σ) is a measu able space and P:Σ→[0,1] is a p obabili y measu e, ha is, a coun ably addi i e posi i e measu e such ha P(Ω)=1. I Xis a Banach space, by L1((Ω,Σ,P),X) we deno e he Banach space o all absolu ely in eg able unc ions, ha is, L1((Ω,Σ,P),X) :=( ∈XΩ: is measu able and ZΩk kdP<∞), AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1954 endowed wi h he no m k k1:=ZΩk kdP. Fo each ∈L1((Ω,Σ,P),X), he mean o is de ined as µ( ) :=ZΩ dP.(4.3.1) The mean unc ional is gi en by µ:L1((Ω,Σ,P),X)→X 7→ µ( ) :=RΩ dP.(4.3.2) No ice ha he mean unc ional is ac ually an ope a o , bu we keep calling i unc ional no o mis ake i wi h he mean ope a o , which will be de ined nex . I can be easily shown ha he mean unc ional has no m equal o 1. Indeed, kµ( )k=ZΩ dP≤ZΩk kdP=k k1 o e e y ∈L1((Ω,Σ,P),X), meaning ha kµk ≤ 1. Now, i we choose any x∈SX, hen xχΩ∈ SL1((Ω,Σ,P),X), and µ(xχΩ)=ZΩ xχΩdP=xP(Ω)=x. Hence, kµ(xχΩ)k=ZΩ xχΩdP =kxkP(Ω)=1=kxχΩk1. This p o es ha kµk=1 and xχΩ∈supp (µ). The mean ope a o is hen de ined as L1((Ω,Σ,P),X)→L1((Ω,Σ,P),X) 7→ µ( )χΩ,(4.3.3) and he cen e ing ope a o is L1((Ω,Σ,P),X)→L1((Ω,Σ,P),X) 7→ −µ( )χΩ.(4.3.4) Finally, i Xis a uni al Banach algeb a, hen he na u al way o de ining he a iance o ∈ L1((Ω,Σ,P),X) is σ( ) :=µ( −µ( )χΩ)2=ZΩ ( −µ( )χΩ)2dP.(4.3.5) 5. Conclusions I is well known in he li e a u e o he Geome y o Banach Spaces ha Hilbe spaces a e ansi i e Banach spaces, meaning ha e e y wo elemen s o he uni sphe e o a Hilbe space can be anspo ed one in o ano he by means o a su jec i e linea isome y. This ac con e s Hilbe spaces wi h a AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1955 ce ain eedom when i comes o choosing he con ex se ies ha de ines he mean unc ional, he mean ope a o , he cen e ing ope a o and he s anda d de ia ion. Theo em 3.1, Lemma 3.2 and Theo em 3.4 can be anspo ed o he mo e gene al scope p o ided by sepa able Hilbe spaces discussed in he p e ious sec ion. In ac , in he Discussion we un eiled how o ex end he mean unc ional, he mean ope a o , he cen e ing ope a o and he a iance o spaces o absolu ely in eg able unc ions de ined on a p obabili y space and alued on a uni al Banach algeb a. On he o he hand, he s udy o P incipal Componen Analysis h ough Suppo ing Vec o Analysis is a e olu iona y end ha allows one o look a hese S a is ical concep s om a Func ional Analysis iewpoin , which is mo e gene al and wo ks in in ini e dimensional en i onmen s, making possible applica ions in e y speci ic se ings such as, o ins ance, Quan um Mechanical Sys ems. Acknowledgmen s This wo k has been pa ially suppo ed by he Resea ch G an PGC-101514-B-I00 awa ded by he Minis y o Science, Inno a ion and Uni e si ies o Spain and co- inanced by he 2014-2020 ERDF Ope a ional P og amme and by he Depa men o Economy, Knowledge, Business and Uni e si y o he Regional Go e nmen o Andalusia unde P ojec e e ence FEDER-UCA18-105867. The APCs ha e been paid by he Depa men o Ma hema ics o he Uni e si y o Cadiz. Con lic o in e es The au ho s decla e ha hey ha e no con lic o in e es . Re e ences 1. E. Bishop, R. R. Phelps, A p oo ha e e y Banach space is sub e lexi e, Bull. Ame . Ma h. Soc., 67 (1961), 97–98. h ps://doi.o g/10.1090/S0002-9904-1961-10514-4 2. E. Bishop, R. R. Phelps, The suppo unc ionals o a con ex se , In: P oceedings o Symposia in Pu e Ma hema ics, Vol. VII, P o idence, R.I.: Ame . Ma h. Soc., 1963, 27–35. 3. T. Bouwmans, S. Ja ed, H. Zhang, Z. Lin, R. O azo, On he applica ions o obus PCA in image and ideo p ocessing, P oc. IEEE,106 (2018), 1427–1457. h ps://doi.o g/10.1109/JPROC.2018.2853589 4. C. Cobos-S´ anchez, F. J. Ga cia-Pacheco, J. M. Gue e o Rod iguez, J. R. Hill, An in e se bounda y elemen me hod compu a ional amewo k o designing op imal TMS coils, Eng. Anal. Bound. Elem.,88 (2018), 156–169. h ps://doi.o g/10.1016/j.enganabound.2017.11.002 5. C. Cobos-S´ anchez, F. J. Ga c´ ıa-Pacheco, S. Mo eno-Pulido, S. S´ aez-Ma ´ ınez, Suppo ing ec o s o con inuous linea ope a o s, Ann. Func . Anal.,8(2017), 520–530. h ps://doi.o g/10.1215/20088752-2017-0016 6. C. Cobos-S´ anchez, J. A. Vilchez-Memb illa, A. Campos-Jim´ enez, F. J. Ga c´ ıa-Pacheco, Pa e o op imali y o mul iop imiza ion o con inuous linea ope a o s, Symme y,13 (2021), 661. h ps://doi.o g/10.3390/sym13040661 AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1956 7. C. Cobos-S´ anchez, M. R. Cabello, ´ A. Q. Oloz´ abal, M. F. Pan oja, Design o TMS coils wi h educed lo en z o ces: applica ion o concu en TMS- MRI, J. Neu al Eng.,17 (2020), 016056. h ps://doi.o g/10.1088/1741-2552/ab4ba2 8. C. Cobos-S´ anchez, J. J. J. Ga c´ ıa, M. R. Cabello, M. F. Pan oja, Design o coils o la e alized TMS on mice, J. Neu al Eng.,17 (2020), 036007. h ps://doi.o g/10.1088/1741-2552/ab89 e 9. C. Cobos-S´ anchez, F. J. Ga cia-Pacheco, J. M. Gue e o-Rod iguez, L. Ga cia-Ba achina, Sol ing an IBEM wi h suppo ing ec o analysis o design quie TMS coils, Eng. Anal. Bound. Elem.,117 (2020), 1–12. h ps://doi.o g/10.1016/j.enganabound.2020.04.013 10. C. Cobos-S´ anchez, J. M. Gue e o-Rod iguez, ´ A. Q. Oloz´ abal, D. Blanco-Na a o, No el TMS coils designed using an in e se bounda y elemen me hod, Phys. Med. Biol.,62 (2016), 73–90. h ps://doi.o g/10.1088/1361-6560/62/1/73 11. C. M. Eps ein, E. Wasse mann, U. Ziemann, Ox o d Handbook o T ansc anial S imula ion, New Yo k: Ox o d Uni e si y P ess, 2008. h ps://doi.o g/10.1093/ox o dhb/9780198568926.001.0001 12. J. Fan, Q. Sun, W.-X. Zhou, Z. Zhu, P incipal componen analysis o big da a, Wiley S a sRe : S a is ics Re e ence Online, in p ess. h ps://doi.o g/10.1002/9781118445112.s a 08122 13. F. J. Ga c´ ıa-Pacheco, E. Na anjo-Gue a, Suppo ing ec o s o con inuous linea p ojec ions, In e na ional Jou nal o Func ional Analysis, Ope a o Theo y and Applica ions,9(2017), 85– 95. 14. F. J. Ga c´ ıa-Pacheco, Lineabili y o he se o suppo ing ec o s, RACSAM,115 (2021), 41, h ps://doi.o g/10.1007/s13398-020-00981-6 15. F. J. Ga cia-Pacheco, C. Cobos-S´ anchez, S. Mo eno-Pulido, A. Sanchez-Alzola, Exac solu ions o maxkxk=1P∞ i=1kTi(x)k2wi h applica ions o Physics, Bioenginee ing and S a is ics, Commun. Nonlinea Sci. Nume . Simul.,82 (2020), 105054. h ps://doi.o g/10.1016/j.cnsns.2019.105054 16. F.-K. Ga siya-Pacheko, The ca dinali y o he se Λde e mines he geome y o he spaces B`∞(Λ)and B`∞(Λ)∗, (Russian), Funk sional. Anal. i P ilozhen.,52 (2018), 62–71. h ps://doi.o g/10.4213/ aa3534 17. Ins i u o de Es ad´ ıs ica y Ca og a ´ ıa de Andaluc´ ıa. A ailable om: h ps://www. jun adeandalucia.es/ins i u odees adis icayca og a ia. 18. R. C. James, Cha ac e iza ions o e lexi i y, S ud. Ma h.,23 (1964), 205–216. h ps://doi.o g/10.4064/sm-23-3-205-216 19. J. Lindens auss, On ope a o s which a ain hei no m, Is ael J. Ma h.,1(1963), 139–148. h ps://doi.o g/10.1007/BF02759700 20. L. Ma in, H. Powe , R. W. Bow ell, C. Cobos-S´ anchez, A. A. Becke , P. Glo e , e al., Nume ical solu ion o an in e se p oblem in magne ic esonance imaging using a egula ized highe -o de bounda y elemen me hod, In: Bounda y elemen s and o he mesh educ ion me hods XXIX, Sou hamp on: WIT P ess, 2007, 323–332. h ps://doi.o g/10.2495/BE070311 21. L. Ma in, H. Powe , R. W. Bow ell, C. Cobos-S´ anchez, A. A. Becke , P. Glo e , e al., Bounda y elemen me hod o an in e se p oblem in magne ic esonance imaging g adien coils, CMES Compu . Model. Eng. Sci.,23 (2008), 149–173. h ps://doi.o g/10.3970/cmes.2008.023.149 AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1957 22. S. Mo eno-Pulido, F. J. Ga cia-Pacheco, C. Cobos-S´ anchez, A. Sanchez-Alzola, Exac solu ions o he maxmin p oblem max kaxksubjec o kbxk ≤ 1, Ma hema ics,8(2020), 85. h p://doi.o g/10.3390/ma h8010085 23. A. S´ anchez-Alzola, F. J. Ga c´ ıa-Pacheco, E. Na anjo-Gue a, S. Mo eno-Pulido, Suppo ing ec o s o he `1-no m and he `∞-no m and an applica ion, Ma h. Sci.,15 (2021), 173–187. h ps://doi.o g/10.1007/s40096-021-00400-w 24. L. Su hone, M. Timpledon, S. Ma seken, P incipal componen analysis: Ka hunen-Lo`e e Theo em, Ha old Ho elling, Ka l Pea son, Explo a o y Da a Analysis, Eigendecomposi ion o a Ma ix, Co a iance Ma ix, Singula Value Decomposi ion, Fac o Analysis, Be asc ip Publishing, 2010. Supplemen a y: Py hon GUI code Le A∈Mm×n(R) be a gene al ma ix. The algo i hm o compu e he suppo ing ec o wi h i s co esponding i s p incipal componen , he second eigen ec o wi h i s co esponding second p incipal componen and he nex eigen ec o s and p incipal componen s o Ais he ollowing: de PCA(X, num_componen s): X_meaned = X - np.mean(X, axis=0) co _ma = np.co (X_meaned, ow a =False) eigen_ alues, eigen_ ec o s = np.linalg.eigh(co _ma ) so ed_index = np.a gso (eigen_ alues)[::-1] so ed_eigen ec o s = eigen_ ec o s[:, so ed_index] eigen ec o _subse = so ed_eigen ec o s[:, 0:num_componen s] X_new_ alues = np.do (eigen ec o _subse . anspose(), X_meaned. anspose()). anspose() e u n X_new_ alues, eigen ec o _subse One o he no el ies o his wo k is o apply he PCA me hod wi h a di e en p ocedu e speci ically, using an algo i hm based on he ma hema ical idea ha he second eigen ec o , wi h i s associa ed second p incipal componen , is he suppo ing ec o o he o iginal poin s p ojec ed o he o hogonal complemen o he o iginal suppo ing ec o . Consequen ly, all he p incipal componen s can be compu ed in an i e a i e p ocess ia a suppo ing ec o . Le x∈Rnbe a ec o . Fi s , we show he ollowing unc ion o calcula e he o hogonal complemen o x: de calcula e_o hogonal_complemen (x, no malize=T ue, h eshold=1e-15): x = np.asa ay(x) , c = x.shape i < c: impo wa nings wa nings.wa n(’ ewe ows han columns’, Use Wa ning) s, , d = np.linalg.s d(x) ank = ( > h eshold).sum() AIMS Ma hema ics Volume 8, Issue 1, 1937–1958. 1958 oc = s[:, ank:] i no malize: k_oc = oc.shape[1] oc = oc.do (np.linalg.in (oc[:k_oc, :])) e u n oc Nex , by using he p e ious unc ion, we calcula e he o hogonal complemen o he suppo ing ec o , ep esen ing a hype plane wi h dimension n−1. A e wa ds, he poin s o he o iginal ma ix, used o he ini ial suppo ing ec o , a e p ojec ed o his hype plane. Hence, a new ma ix is de i ed, whose i s p incipal componen co esponds o he second p incipal componen o he o iginal ma ix. Thus, by applying his i e a i e algo i hm, we can calcula e all he p incipal componen s ia he o hogonal complemen o a suppo ing ec o . I is easie o unde s and his algo i hm conside ing a special case. Le A∈M3×3(R) be a ma ix. In an in ui i e manne , each ow o he ma ix ep esen s poin s o R3. Now, we p esen a p ocess o compu ing he i s eigen ec o (suppo ing ec o ) and i s co esponding i s p incipal componen as be o e, bu wi h a di e en p ocedu e o ob ain he ollowings. As men ioned, we no ice ha he i s p incipal componen ia a suppo ing ec o o a de i ed ma ix om he o iginal one coincides wi h he second p incipal componen o he o iginal ma ix. This de i ed ma ix is cons uc ed p ojec ing he o iginal poin s o he o iginal ma ix o a plane o med by he o hogonal complemen ( wo ec o s) o he ini ial suppo ing ec o (one ec o ) and he o igin poin . The e o e, we ob ain he same esul by applying he PCA p esen ed be o e and his algo i hm de e mining he nex eigen ec o as a suppo ing ec o o he o iginal suppo ing ec o . suppo ing_ ec o = PCA(inpu _ma ix, 1) o hogonal_complemen = calcula e_o hogonal_complemen (suppo ing_ ec o ) 1 = Vec o (o hogonal_complemen .T[0, :]) 2 = Vec o (o hogonal_complemen .T[1, :]) poin = Poin ([0, 0, 0]) plane = Plane. om_ ec o s(poin , 1, 2) p ojec ed_poin s = np.a ay([plane.p ojec _poin (x) o x in inpu _ma ix]) suppo ing_ ec o _p ojec ed_poin s = PCA(p ojec ed_poin s, 1) ©2023 he Au ho (s), licensee AIMS P ess. This is an open access a icle dis ibu ed unde he e ms o he C ea i e Commons A ibu ion License (h p://c ea i ecommons.o g/licenses/by/4.0) AIMS Ma hema ics Volume 8, Issue 1, 1937–1958.