Residual spaces in latent variables model inversion and their impact in the design space for given quality characteristics

Jou nal P e-p oo Residual spaces in la en a iables model in e sion and hei impac in he design space o gi en quali y cha ac e is ics S. Ruiz, L.A. Sa abia, M.C. O iz, M.S. Sánchez PII: S0169-7439(19)30469-1 DOI: h ps://doi.o g/10.1016/j.chemolab.2020.104040 Re e ence: CHEMOM 104040 To appea in: Chemome ics and In elligen Labo a o y Sys ems Recei ed Da e: 25 July 2019 Re ised Da e: 30 Ap il 2020 Accep ed Da e: 3 May 2020 Please ci e his a icle as: S. Ruiz, L.A. Sa abia, M.C. O iz, M.S. Sánchez, Residual spaces in la en a iables model in e sion and hei impac in he design space o gi en quali y cha ac e is ics, Chemome ics and In elligen Labo a o y Sys ems (2020), doi: h ps://doi.o g/10.1016/ j.chemolab.2020.104040. This is a PDF ile o an a icle ha has unde gone enhancemen s a e accep ance, such as he addi ion o a co e page and me ada a, and o ma ing o eadabili y, bu i is no ye he de ini i e e sion o eco d. This e sion will unde go addi ional copyedi ing, ypese ing and e iew be o e i is published in i s inal o m, bu we a e p o iding his e sion o gi e ea ly isibili y o he a icle. Please no e ha , du ing he p oduc ion p ocess, e o s may be disco e ed which could a ec he con en , and all legal disclaime s ha apply o he jou nal pe ain. © 2020 Published by Else ie B.V. C edi Au ho S a emen : S. Ruiz: Concep ualiza ion, Me hodology, So wa e, Fo mal analysis, Supe ision, W i ing, Re iew & Edi ing L.A. Sa abia: Concep ualiza ion, Me hodology, Fo mal analysis, Supe ision, M.C. O iz: Concep ualiza ion, Fo mal analysis, Supe ision, W i ing - Re iew & Edi ing, Funding acquisi ion M.S. Sánchez: Concep ualiza ion, Me hodology, Fo mal analysis, So wa e, W i ing - O iginal D a , Supe ision, Re iew & Edi ing 1 Residual spaces in la en a iables model in e sion and hei impac in he design space o gi en quali y cha ac e is ics S. Ruiz a , L.A. Sa abia a , M.C. O iz b* , M.S. Sánchez a a Dp . Ma hema ics and Compu a ion, b Dp . Chemis y Facul ad de Ciencias, U NIVERSIDAD DE B URGOS , Pza. Misael Bañuelos s/n, 09001 Bu gos, Spain Abs ac The pape con ains a discussion abou he null spaces associa ed o linea p edic ion models o he pa icula case o Pa ial Leas Squa es eg ession models. The discussion sepa a ely conside s he wo exis ing null spaces: he one in he inpu space ela ed o he p ojec ion on o he la en space and he null space, coming om he p ojec ion space, co esponding o he mapping o he sco es on o he p edic ed esponses. The pape also explo es he impac o such null spaces in he de ini ion o he design space a ound some easible solu ions ob ained by in e ing he p edic ion model, ia se e al cases wi h simula ed and eal da a om he li e a u e. The case-s udies se e o illus a e he discussion and he need o conside ing poin s in he wo null spaces, a he han jus ake in o accoun he null space wi hin he la en space. They also se e o show how o add ess he use o he esul ing ec o s in he design space o main ain he desi ed quali y by modi ying he unable (maneu e able) p ocess a iables o compensa e o a ia ions due o some o he ea u e a iables no so easy o con ol. Keywo ds Pa ial Leas Squa es; P ocess Analy ical Technology; Quali y by Design; linea applica ion; null space; model in e sion. 1. In oduc ion In he con ex o p ocess con ol, p edic ion models a e commonly used o es ima e he expec ed quali y o a p oduc as a unc ion o he cha ac e is ics o he p ocess and/o some ea u e a iables, such as aw ma e ial p ope ies o en i onmen al ac o s. Gi en * Co esponding au ho , mco [email protected] 2 he equen ly la ge numbe o p ocess o ea u e a iables and quali y cha ac e is ics and hei co ela ion (in bo h spaces), hese p edic ion models a e usually La en Va iables Models (LVM). Besides hei s anda d use o p edic he expec ed quali y cha ac e is ics ( he esponse when i ing he model), hese models a e also use ul when he ocus is on main aining a gi en quali y, like in he amewo k o Quali y by Design (QbD) es ablished o pha maceu ical p ocesses. In he la e si ua ion, he p edic ion model mus be in e ed o ind he se ings o he inpu a iables necessa y o gua an ee his gi en o desi ed quali y. The linea LVM selec ed is a Pa ial Leas Squa es (PLS) eg ession model. B ie ly, a PLS model can be seen as he ma hema ical composi ion o wo linea maps: i s a educ ion o dimensionali y by p ojec ing he poin s on o a space o lowe dimension ( he la en o p ojec ion space spanned by he selec ed la en a iables) and hen eg essing he sco es on o he esponse space. The p esen wo k ocuses on he a iabili y in bo h he inpu and la en spaces due o he linea s uc u e o he PLS model being used, a he han on he a iabili y o he p edic ed esponse i sel . The eade in e es ed in he s a is ical unce ain y o he p edic ions can consul Re s. [1, 2] whe e some con idence in e als a e compu ed o he esponse p edic ed wi h PLS models. The use o subspaces o sol e linea p oblems is usual in enginee ing and signal p ocessing. In con ol sys em, which is a ea di ec ly ela ed o PAT (P ocess Analy ical Technology) and QbD, i s p inciples models (i.e. based on physical o chemical laws) ha e been adi ionally o use o desc ibe he beha io o a sys em o subsequen ly con ol i . Howe e , in he las yea s, he idea o using da a d i en models e en i hey ha e no di ec physical meaning gained in e es among enginee s, ending up in so called ’subspace’ me hods. Thei name e lec s he ac ha linea models can be compu ed om ow and column (sub)spaces o ce ain ma ices, which a e ob ained om inpu - ou pu da a o he sys em unde obse a ion. Typically, he column space o such da a ma ices con ains in o ma ion abou he model, while he ow spaces a e somehow ep esen a i e o he s a e sequence, and hey a e es ima ed di ec ly om inpu -ou pu da a wi hou ‘a p io i’ knowledge o any i s p inciples models. The decomposi ion based on canonical classical models has some al e na i es. Fo example, in he book in e . [3], he use o QR decomposi ion is p oposed o he p ojec ion (o hogonal o oblique) and gene alized singula alue decomposi ion o de e mine ini e dimensional subspace and, hen, leas squa es o ob ain he linea ela ions. Leas Squa es Reg ession (LSR) me hod is in oduced o subspace segmen a ion in e . [4], bu mul i a ia e analysis wi h p incipal componen analysis (PCA) and pa ial leas squa es (PLS) a e somehow ep esen a i e me hods [5] among se e al da a-d i en echniques o p ocess moni o ing, hanks o hei simplici y and e iciency in p ocessing huge amoun o p ocess da a. As s a ed in e . [6], a mo e impo an objec i e o p ocess moni o ing is o p o ide assu ance o good p oduc quali y ha is impac ed by he p ocess condi ions, and quali y a iables a e be e aken in o accoun wi h PLS han wi h PCA. 3 Techniques o sea ching o subspaces app op ia e o model he sys em esponse a e ex ended o non-linea models by using KPCA (Ke nel P incipal Componen Analysis) o model nonlinea dependences be ween a iables o ep oduce nonlinea dynamic beha io [7]. Rega ding decomposi ions, e . [8] is an in oduc ion ha shows he u ili y o K ylo subspaces o sol e linea sys ems Ax = b, whe e A is a squa e ma ix and b is a ec o . When A is in e ible, he solu ion is jus A -1 b bu he calcula ion o he in e se o A o la ge ma ices can be compu a ionally ine icien and space consuming. On his subjec , he book by Liesen and S akos [9] con ains a e y comple e ma hema ical and compu a ional ounda ion, along wi h some in e es ing his o ical no es. The de elopmen o e icien algo i hms o sol e linea sys em o equa ions wi h spa se ma ices is a ma e o cons an esea ch, see o example he ecen pape in [10]. Since he K ylo subspace me hods a e p ojec ion me hods on o a subspace, i s use as he p ojec ion echniques o sol e la ge-scale con ol p oblems is a na u al ‘ex ension’ o i s use in nume ical linea algeb a o ma ix p oblems. Basically, i consis s o p ojec ing he o iginal la ge con ol p oblem on o a m-dimensional K ylo subspace (m ‘small’) by cons uc ing a basis o he subspace, and hen use a s anda d well- es ablished echnique o sol e he p ojec ed smalle p oblem. In ha way, an app oxima e solu ion o he o iginal la ge p oblem is ob ained om he solu ion o he p ojec ed p oblem [11,12]. PLS was ini ially de eloped as a heu is ic echnique [13] o sol e leas squa es p oblems in mul ilinea eg ession, wi h no op imali y p ope ies and c i icized o a long ime as a somewha ‘ad-hoc’ solu ion. La e , i was shown ha i is equi alen o some o he mos sophis ica ed nume ical algo i hms o da e o sol ing sys ems o linea equa ions, such as he Lanczos bidiagonaliza ion o he conjuga e g adien me hods [14]. The p ope y ha he ec o y ˆ consis ing o he alues p edic ed by a PLS model wi h a la en a iables is he p ojec ion o he da a ec o y on o a subspace spanned by he p edic o a iables in X (which is a subspace ha can be desc ibed as he span o a ce ain K ylo sequence), is used in e . [15] o cha ac e ize se e al eg ession me hods ( idge eg ession, con inuum powe eg ession, PLS and p incipal componen s eg ession, PCR) ha a emp o educe he a iance o he OLS (o dina y leas squa es) eg esso b OLS =(X T X) −1 X T y by eplacing he ill-condi ioned ma ix (X T X) −1 by a mo e s able al e na i e. The in e p e a ion o PLS in e ms o K ylo subspaces is also used in e . [16] o show ha a ious la en a iables eg ession me hods sha e a common s uc u e, called Mul ilaye Basic Sequence (MBS), which is a pa icula case o a K ylo sequence. I allows uni ying se e al algo i hms used o es ima e he PLS coe icien s and s udying he ela ion be ween Ma ens’ PLS [17] (o hogonal MBS) and Wold’s PLS [18,19] (oblique MBS). Fo any o hogonal-MBS model, he e is an equi alen oblique-MBS model so ha hey sha e he same eg ession space (o K ylo spanned space), leading o equi alen p edic i e pe o mance. Tha means ha simila eg ession esul s can be ob ained ia mul iple linea eg essions on o he esponse a iable. Ma ens' PLS and Wold's PLS a e a special case o his si ua ion. This explains he iden ical eg ession 4 coe icien s and p edic ion esul s. The mapping na u e o o hogonal-MBS and oblique-MBS om la en sco e T-space o he o iginal X-space e lec s he co esponding de la ion echniques. Al hough, concep ually, he in e p e a ion o he la en a iables is i ele an in he me hods o leas squa es based on subspace decomposi ion, i becomes ‘ ele an ’ du ing he s age o using he model, whe e he p ac i ione looks o his in e p e a ion. This is especially ue in Chemome ics whe e PLS used o be a me hod o p edic concen a ions o samples wi h known mul i a ia e signals, ha is, as a calib a ion me hod. I is wo h men ioning ha , in his con ex wi h, usually, mo e a iables han samples, he solu ion by using s anda d linea eg ession models does no exis . When explo ing he possible in e p e a ion o he PLS calib a ion models, i u ned ou ha he subspace spanned by he a selec ed la en a iables included in o ma ion no di ec ly ela ed o he esponse y. This issue was add essed wi h wo me hodologies: i) p e ea ing he signals o elimina e ‘a i ac s’ o he signal (usually physical) no ela ed o he esponse o ii) modi ying he sea ch o he solu ion limi ing i o subspaces co ela ed wi h he esponse. The i s app oach has led o se e al p e ea men me hods o he p edic o a iables, which g ea ly impac he subsequen PLS calib a ion model [20,21]. On he second me hodology men ioned, he mos gene alizable con ibu ion is possibly he O hogonal Signal Co ec ion (OSC) by Wold e al. [22], an app op ia e modi ica ion o he PLS algo i hm o elimina e sys ema ic y-o hogonal pa s o he ma ix X. The Ne Analy e P ep ocessing (NAP) app oach [23] and i s exac ly equi alen Di ec O hogonaliza ion (DO)[24] p opose some loading weigh s (no necessa ily con ained in he ow space o X) ep esen ing phenomena conside ed o be i ele an o he modelling o y. O he p oposals, as O hogonal-PLS (O-PLS)[25], he al e na i e way o compu ing O-PLS [26] and he eg ession p ocedu e based on he O hogonal Signal Co ec ion by Fea n [27], a e discussed by Indhal [28], cla i ying he way hese OSC me hods wo k and gi ing a igo ous explana ion o he eason why he en i e OSC concep may be bo h con using and supe luous. Yu and MacG ego [29] poin ou ha ‘ he subspace o X unco ela ed o Y is o en high dimensional and i is no a all clea wha pa o his subspace is emo ed by di e en OSC algo i hms wi h di e en numbe o OSC componen s. As a esul , any in e p e a ion o he OSC componen s may be di e en ’ and p opose he applica ion o a canonical co ela ion analysis as pos -p ocessing o a PLS model (PLS-CCA). This app oach o de ine he subspace mos ela ed o he quali y a iables is used in e . [30] o model a ba ch p ocess by looking o he subspace common o he di e en slices de ined wi h he p ocessing ime. The e o e, ac i i y in he ield o leas -squa es eg ession on subspaces is ocused on modeling he subspace o he p edic o s S m mos ela ed o he esponse. In gene al, he o hogonal subspace S m⊥ is only conside ed o de ining a limi , usually known as he squa ed p edic ion e o (SPE), o decide i a new objec is compa ible wi h he i ed PLS (o PCR) model. This index is essen ial when building mul i a ia e calib a ion models o mul i a ia e con ol cha s based on la en a iables [31]. I s use has e en 5 been gene alized o he non-linea case o mul i a ia e con ol in he indus y using KPCA [7]. Besides he de ini ion o SPE, he null space o he S m⊥ eg ession model has been used, o example, o ea ly diagnosis o s uc u al damages o machine y mal unc ions esul ing in educ ion o he main enance cos and inc easing eliabili y and sa e y [32,33,34]. This subsidia y use o he null space has been su passed in he ield o QbD and PAT, because he null space is pa o he design space. This is consequence o he own de ini ion o null space: when adding any ec o o he null space (o he eg ession be ween X -con aining p ocess a iables, aw ma e ial o en i onmen al p ope ies, and Y -wi h he quali y cha ac e is ics) o any ec o o he space spanned by X, he p edic ed quali y cha ac e is ics do no change. None heless, he analyses on he null space a e limi ed o he one ela ed o eg essing Y on he la en a iables. The QbD con ex equi es o de ine he alues o p ocess a iables, aw ma e ial o en i onmen al p ope ies ha esul in he same (o simila enough) alue o he measu ed quali y cha ac e is ics and, i only he e e ed null space is used, he esul ing es ima es a e ‘indi ec ’ because he ange o admissible alues o he o iginal a iables is compu ed om hei ange on he la en a iables. The e o e, in wha ollows, he p edic ed alue emains in a ian . In ac , we se a desi ed alue o he esponse y des ( he a ge quali y) and look o he co esponding se ings o he inpu a iables whose p edic ed alue is exac ly y des . Addi ionally, a ound hese se ings, when hey exis , i is ad isable o de e mine he co esponding design space, ha is, he egion in he inpu space whe e he p edic ed esponse is he same and hus he expec ed a ge quali y is main ained. When wo king wi h linea models, his design space includes he ke nel o he mapping, also known as he null space, which we ha e al eady said ha is he ec o subspace ha con ains he poin s mapped on o ze o, i.e. he ec o s ha do no modi y he p edic ions when added o any o he . To ou knowledge, Jaeckle and MacG ego [35] in oduced o he i s ime he idea o a null space associa ed o he in e sion o a PLS model, bu i s e ec is only conside ed in he la en space. This null space is also aken in o accoun in e . [36] when backp opaga ing he con idence in e al in e . [1] wi h di e en p ocedu es o calcula e a kind o con idence egions o b acke he design space. Howe e , ocusing on he i ed PLS model, pa o he a iabili y e lec ed in he design space comes om wha we can call he ‘ esidual’ space ela ed o he inpu a iables. The e m ‘ esidual’ e e s o he ac ha i is disca ded when selec ing he numbe o la en a iables because i is supposed o be he pa ha is no ela ed o he p edic ed quali y cha ac e is ics. In pa icula , he a iabili y in he esidual space ela ed o he inpu a iables ( he pa o he p ocess o ea u e a iables space ha is disca ded a e he decomposi ion) does no ha e any impac on he p edic ed quali y. The e o e, i e lec s he pa o he space whe e he inpu a iables can be modi ied 6 wi hou al e ing he quali y o he esul ing p oduc , ha is, i is necessa ily pa o he design space. The p esen pape in ends o desc ibe he ‘shape’ o his pa o he design space h oughou some p oduc ion p ocesses wi h a ce ain p ede ined quali y. Technically, his equi es he in e sion o he eg ession model because he quali y cha ac e is ics o he p oduc a e he ones kep ixed. Gi en ha PLS is a linea model, unde ce ain condi ions ( ela ion among he dimensions o he spaces –inpu a iables, la en , and quali y cha ac e is ics), his in e sion can be done algeb aically by sol ing ma ix sys ems [37]. The e a e also some compu a ional al e na i es [38] ha do no depend on he men ioned dimensions. Rega dless o he picked solu ion, he null spaces ela ed o he linea models being used play an impo an ole in he subsequen design space. Ce ainly, his ‘di ision in o wo pa s’ o he null space can be igno ed and use di ec ly i s algeb aic cha ac e iza ion by using he eg ession coe icien s o he i ed PLS model. Howe e , no all poin s in his null space belong o he design space, because hey also mus be adequa e o he p ocess being modelled, i.e., easible se ings o he p ocess/ ea u e a iables ha , in addi ion, a e p ojec ed on o he la en space de ined by he PLS model. The basic no a ion and some heo e ical backg ound a e p esen ed in sec ion 2 and di e en small sample sized case-s udies a e analyzed in sec ion 3 o illus a e he p ope ies and p ac ical applicabili y. The pape ends wi h some conclusions. 2. Theo e ical p ope ies and de ini ions Le X (n × p) and Y (n × q) be he ma ices o p (p edic o ) inpu a iables and he co esponding q ( esponse) quali y cha ac e is ics, espec i ely. F equen ly hese a e his o ical da a o a p ocess, a anged in wo ma ices used o i , in ou case, a PLS model o p edic quali y cha ac e is ics gi en some se ings o he p ocess/ ea u e a iables. These da a a e assumed o be ep esen a i e o he p ocess because he in o ma ion hey con ain is he only one used o i he model and o make decisions, in a comple ely da a-d i en app oach. In wha ollows, we a e assuming ha p and q a e less han n, ha up o p new o hogonal a iables can be ob ained in he inpu space, and ha bo h X and Y a e au oscaled ( ha is, each column is cen e ed by sub ac ing he mean, and has a iance one a e di iding by he s anda d de ia ion). In addi ion, in any p ocess unde con ol, he p ocess/ ea u e a iables a e bounded. To emula e his si ua ion, he admissible domain o he p edic o a iables will be de ined by he ange o he obse ed alues in X. The design space we a e looking o mus be inside his domain and wi h he se ings o he inpu a iables ha de ine easible solu ions. 7 A PLS p edic ion model is i ed wi h X and Y by compu ing a o hogonal new a iables ( he la en a iables) wi h a common sco es ma ix T a (n × a) such ha T T a a X X=TP = TP +R (1) T a Y Y= TQ +R (2) Ma ices P (p × p) and Q (q × a) a e he loadings, and R X and R Y he ma ices con aining he esiduals o he decomposi ion. As usual, uppe T means ansposing he co esponding ma ix o ec o . Finally, sub-index a deno es ha we a e only conside ing he i s a columns o he co esponding ma ix, so o ins ance, T is he n × p ma ix o X sco es, whose i s a columns o m he common sco es ma ix T a . The subspace spanned by he columns o P a is he la en a iables space o p ojec ion subspace ( o now on, la en space). Once a PLS model in equa ions (1) and (2) is i ed, he e a e also some cons ain s o adequa ely apply he model. We will use he 95 % con idence limi s o he usual Q and T 2 s a is ics o de ine he p ope egion whe e he easible p edic o a iables a e p ojec ed. The alue o he T 2 s a is ic o a gi en sample is he Ho elling’s dis ance om he p ojec ion o he o igin o he la en space (i.e., all sco es equal o ze o) and indica es he posi ion o he p ojec ed sample. On he o he hand, Q- esiduals a e calcula ed as he sum o squa es o each ow in R x , eq. (1), he esiduals a e p ojec ing he sample ough he model. I is hen a measu e o he dis ance om he sample o he o hogonal p ojec ion on o he la en space. Imposing limi s in bo h s a is ics is o impose a h eshold alue o he a ia ion o poin s bo h wi hin and ou side he la en space. Acco dingly, he i ed model mus inco po a e bo h cons ain s, so ha he PLS model is p ope ly applied only o hose x wi h alues o Q and T 2 below he h eshold alues imposed in bo h s a is ics. In o he wo ds, we a e de ining a bounded domain o he applica ion o he PLS model. Geome ically, his domain can be iewed as a “PLS- box” ha includes he pa o he la en space and he pa o he esidual space bounded wi h, in ou case, he 95 % con idence le el o Q and T 2 s a is ics. In ha sense, he easible solu ions a e hose ha belong simul aneously o he domain o he inpu a iables and o wha we ha e named he PLS-box. Fo ec o calcula ions, we will espec hei s uc u e in ma ix o m. Fo example, a se o p p edic o a iables is w i en as a ec o x T because hese a iables a e ows in ma ix X. Simila ly, y T is a ow ec o con aining quali y cha ac e is ics. Wi h his no a ion, ec o x con aining some se ings o he inpu a iables is p ojec ed on o he PLS-la en space by using some weigh s in W (p × a ma ix such ha 14 Howe e , he e a e wo dimensions disca ded when i ing he PLS model ha impac he p edic o a iables space bu no he la en space. Because o he p ojec ion, he e is a plane in he h ee-dimensional inpu space whe e we can mo e wi hou modi ying he p edic ed esponse. This plane is consequence o he space ha we ha e called he W-null space, ela ed o he esidual space, which in u n is o hogonal o he la en space. To explo e he e ec o he W-null space associa ed o y des , we gene a e m = 1000 poin s uni o mly dis ibu ed inside he ange de ined by he disca ded sco es T , equa ion (8), and mul iply hem by P , he disca ded loadings ha a e a basis o he W-null space. As we ha e al eady indica ed, all he poin s hus gene a ed ha e null alue o T 2 . Nex , we add all o hem o ˆ des x , s ep 6 in ig. 1, which does no modi y he p edic ed esponse acco ding o eq. (10) al hough i does modi y he alues o Q and T 2 . The e o e, he inal s ep (s ep 7 in ig. 1) is o emo e hose esul ing poin s ou side ei he he domain o he inpu a iables o he la en space (in his case, only i hei Q- esiduals a e la ge han he c i ical alue 5.68). The emaining 726 se ings a e plo ed as small black do s in ig. 2. To acili a e isualiza ion o he plane hey all lie on, he con ex hull o he poin s is depic ed in ligh g ay and i is clea ha i bounds he whole plane (con aining ˆ des x ) o he poin s inside he de ined domain o he inpu a iables and he PLS-box. 3.1.2 PLS model wi h wo la en a iables. Unco ela ed p edic o a iables Fo he sake o illus a ion, o he same da a se s X-y, le us suppose ha we had chosen wo LV o he PLS p edic ion model. As we can see in able 1, his new model explains 99.99 % o he a iance in y and 71.96 % o he a iance o X. Due o he unco ela ed p edic o a iables, his second la en a iable only explains a iance in X, in a pe cen age simila o he i s one. Figu e 3 shows he sco es (g een diamonds) o X in he wo-dimensional la en space. Now a = 2, so he e is a one-dimensional null space in he la en space, wha we ha e called he Q-null space, esul ing om he in ini ely many solu ions o equa ion (5). One o i s alid solu ion has sco e ze o on he second LV and, hus, wi h he same 1 ˆ on he i s LV as in 3.1.1. This solu ion is in ac he solu ion in he leas squa es sense o he sys em in equa ion (5). I is ep esen ed in igu e 3 by a ed illed ci cle. The emaining sco es, solu ion o equa ion (5), heo e ically ollow a s aigh line, he one depic ed in ig. 3 wi h blue illed squa es, ob ained by aking poin s equally spaced along i , s ep 4 in ig. 1 wi h a > 1 = q. 15 This line is in ac limi ed o he segmen made up wi h he poin s inside he PLS-box in he la en space, g aphically, he segmen bounded by he ellipse depic ed in ig. 3 wi h black dashed line. The poin s enclosed by he ellipse ha e T 2 alues less han he c i ical alue es ablished when i ing he model, namely 6.51 a 95 % con idence le el. No ice ha he ellipse is almos a ci cum e ence due o he lack o co ela ion among he p edic o a iables. Figu e 3 a ound he e By using he loadings o P a in equa ion (1), his Q -null space is exp essed (s ep 5 in ig. 1) in he inpu space also as a line in he h ee-dimensional space. This line is he one de ined by he 18 blue illed squa es in ig. 4 ha co espond o he blue illed squa es in ig. 3 whose e-cons uc ed p edic o a iables a e easible solu ions (i.e., belong o bo h he PLS-box and he inpu domain). Since hey a e compu ed di ec ly om he la en space by using he loadings, hei Q- esiduals a e always null. No ice ha he cen al posi ion is occupied by he ed illed ci cle which, also in his space, is exac ly ˆ des x , he same as in sec ion 3.1.1. As in he p e ious sec ion, he e is some addi ional a ia ion allowed in he inpu space which is no being conside ed by only aking in o accoun he Q -null space de ined in he la en space. The possibili ies ha a e missing come om he W -null space, which in his case is he one-dimensional null space ela ed o he p ojec ion on o he la en space, and o hogonal o i . Wi h eq. (10) we gene a e 50 poin s in W -null space and, ollowing s ep 6 in ig. 1, add hem o each o he 19 solu ions al eady ound ( he 18 blue squa es plus he ed ci cle in ig. 3). Finally, s ep 7 in ig. 1, we keep he 760 easible solu ions. Figu e 4 a ound he e These easible solu ions a e he small black do s de ining pa allel lines in igu e 4 whe e hey clea ly show ha he di ec ion de e mined by he black poin s in W -null space is o hogonal o he a o emen ioned line de ined by he solu ions coming om he Q -null space, blue squa es. These wo di ec ions, oge he , de ine a plane, bu only a pa o i con ains he se ings o he p edic o a iables o which he p oblem makes sense, i.e., only he con ex hull depic ed in ig. 4 in ligh g ay is pa o he design space a ound ˆ des x o p edic ing y des . In any case, he bounded egions in g ay in ig. 2 and 4, inside he heo e ical planes wi h equal p edic ed esponse, illus a e ha i only he pa o ke (L) coming om Q -null space is conside ed, he design space would be unde es ima ed, i is necessa y o add poin s in W -null space as well. 16 3.1.3 Co ela ed p edic o a iables I migh appea ha he cons ain s imposed o de ining he PLS-box ha e a ce ain deg ee o a bi a iness, bu in ac hey a e he key o he con ol o a p ocess by using he i ed model. To illus a e his, in he same es ablished si ua ion, le us collec new da a o i ing he model, his ime wi h a iables highly co ela ed. Then, new 50 samples wi h h ee co ela ed a iables a e simula ed and, again, he ue alue o he esponse in y is compu ed as in eq. (11). Table 2 con ains he new s uc u e o he PLS models i ed, wi h 1 and 2 la en a iables. One o he poin s was well beyond he 95 % limi s o Q- esiduals and T 2 s a is ics and i was emo ed. Table 2. Cha ac e is ics o he PLS model i ed o he simula ed da a wi h co ela ed p edic o a iables. Numbe o LV Va iance explained in X (%) Va iance explained in y (%) RMSECV 1 95.08 81.85 0.42 9 2 98. 80 98. 64 0.127 RMSECV: Roo Mean Squa ed E o in C oss-Valida ion Compa ed o he models in able 1 o unco ela ed a iables in X , in he p esen case, he i s la en a iable is mos ly ela ed o he co ela ion among a iables in X (95.08 % o a iance in X ) and he second la en a iable is needed o be e p edic he esponse in y , up o 98.64 % o explained a iance. Figu e 5 shows he new la en space, wi h he same coding as igu e 3: G een diamonds a e he p ojec ion o he da a poin s in X , he ed ci cle is he leas squa es solu ion o eq. (5) o y des =0.28 (an in e media e non-null alue o au oscaled y ), and he blue squa es a e along he segmen consequence o he Q -null space, inside he la en space, ela ed o he ed poin . The subspace is again a line segmen bu wi h a e y di e en slope, compa ed o he one in igu e 3. Finally, he ellipse in ig. 5 is he bounda y o he egion o he la en space belonging o he PLS–box. We see ha , now, is a om ci cula . Fig.5 a ound he e 17 The egion o easible se ings o he p edic o a iables wi hin he design space ela ed o y des is depic ed in ig. 6 as he con ex hull o he solu ions compu ed ollowing he p ocedu e o ig. 1. Figu e 6 a ound he e The o hogonal posi ion o he lines de ined wi h he blue poin s and he ones de ined wi h he black poin s is s ill e iden , bu he yellow egion inside he heo e ical plane is now e y di e en om hose in ig. 2 o ig. 4. In his case i is a kind o na ow band ha ex ends in he di ec ion de ined by he Q -null space (blue squa es), wi h wid h due o he W -null space (black do s) and con olled, p ecisely, by he h eshold on he Q s a is ic. The small a ia ion allowed is no su p ising aking in o accoun ha selec ing wo la en a iables, he e is only nea 1 % o he a iance in X le in he esidual space. Summing up, e en hough he unc ional ela ion be ween X and y is he same in all he cases, he es ima ed egions inside he design space, consequence o he null space, a e no . In ac , he whole ke nel subspaces a e di e en . To p o e i analy ically, we use di ec ly he co esponding eg ession coe icien s o ob ain he wo-dimensional null space ‘a once’ wi hou making dis inc ion o he sou ce o such a iabili y ( Q - o W - null spaces). Wi h a single esponse, he ke nel o he linea map de ined by he eg ession coe icien s b can be in e p e ed as he se o poin s o hogonal o b (see o ins ance page 137 in [39]), so ha he eg ession ec o b is he no mal ec o o he plane we a e looking o . Wi h his p ope y, he analy ical equa ions o he planes in igu es 2, 4 and 6 a e, espec i ely: 1 1 2 3 2 1 2 3 3 1 2 3 :0.45 0.63 0.63 0.33 0 :0.54 0.58 0.61 0.32 0 :0.68 0.47 1.13 0.28 0 X X X X X X X X X π π π + + − = + + − = + + − = (12) The i s wo planes in eq. (12), π 1 and π 2 , a e e y simila o each o he , and co espond o he si ua ion wi h unco ela ed p edic o a iables. The eason o he di e ence is he addi ion o he second la en a iable, which ha dly modi ies he p edic ions compu ed wi h he PLS model wi h a single la en a iable (due o he independen a iables in X) , bu ha modi ies he esidual and la en spaces. These spaces a e comple ely di e en in he case o he hi d plane π 3 ob ained wi h a PLS model wi h 2 LV i ed wi h co ela ed inpu a iables. The e o e, i is clea ha he model decomposi ion is d i en by he a ailable in o ma ion, ha is, depends on he da a a hand. The co ela ion o he inpu a iables in he a ailable da a o ces he shape and o ien a ion o he PLS-box and, hus, also he ‘inhe i ed’ s uc u e imposed in he inpu space (whe e he design space lies). 18 In any case, we ha e al eady poin ed ou ha he exp essions in eq. (12) alone a e useless unless we impose he cons ain s o he model. Howe e , hey can be use ul o make decisions in some p ac ical si ua ions. Fo example, suppose ha he e was a limi a ion in he a ailabili y o -say, X 3 (a e y expensi e eac i e o sho age in some ma e ial). The co esponding equa ion gi es he condi ions o be imposed in he o he wo a iables o main ain he p edic ed esponse, p o iding al e na i es o ind easible solu ions a e applying he cons ain s. The main conclusion is ha he design space o a gi en y des should be de ined aking in o accoun all he possible a ia ions a ound a compu ed solu ion o he in e sion. The si ua ion in igu es 2, 4 and 6 illus a es ha i only he pa o ke (L) coming om Q - null space is conside ed, he design space would be unde es ima ed, i is necessa y o add poin s in W -null space as well. Besides, as expec ed, he design space is dependen no only on he alue o he desi ed esponse bu also on he s uc u e o X and y , p o ided hey a e p ope ly ‘collec ed’ ia he p edic ion model. The limi s in Q and T 2 s a is ics when using a PLS model help de ining he domain o applicabili y and con olling he p ocess. In pa icula , ‘how much’ o he esidual space is, in ac , a ec ing he model (i.e., i is inside he PLS-box) is mos ly con olled by he h eshold alue imposed in he Q s a is ic. In he cases discussed in his sec ion, i only he p edic ion abili y o he PLS model is aken in o accoun , ables 1 and 2 show ha we should selec one la en a iable o he case o unco ela ed p edic o a iables ( ha explains 98.96 % o he a iance in y wi h less han 37 % in X ), and wo o he case o co ela ed a iables (explaining 98.62 % o y and 98.8 % o X) . In ha case, he e is almos a 65 % o he a iance o X unaccoun ed o in he i s case, which is ‘ e lec ed’ in he la ge egion in igu e 2, whe eas he esidual space will accoun o less han 2 % o he a iance in X in he second case, wi h a na ow egion ela ed o he in e sion in igu e 6. 3.2 Byp oduc in Alumina p oduc ion Table 2 in [40] con ain da a om a s udy o op imize he echnological condi ions in a p ocess, da a ha we will use as ep esen a i e o he p ocess. Acco ding o he au ho s, he p ocess o an alumina p oduc ion line consis s o sepa a ing ou he aluminum hyd oxide Al(OH) 3 om a sa u a ed solu ion o sodium alumina e NaAlO 2 as a esul o eac ion wi h ca bon dioxide, CO 2 . Un o una ely, he silicon dioxide SiO 2 p esen in he sodium alumina e also sepa a es ou and mixes wi h he Al(OH) 3 deg ading he quali y o he ex ac ed aluminum hyd oxide . The SiO 2 con en in Al(OH) 3 is he cha ac e is ic eco ded in he da ase . I consis s o 31 samples o he p oduc ion p ocess cha ac e ized by se en ea u e a iables, hose 19 lis ed in able 3, whe e we can see ha he e a e p ope p ocess a iables as well as some ea u es measu ed in he aw ma e ials. Table 3. P ocess and ea u e a iables o he Alumina p oduc ion p ocess No a ion Desc ip ion X l he o al alkaline concen a ion o he o iginal solu ion X 2 he aluminum concen a ion o he o iginal solu ion X 3 he a io o SiO 2 and Al 2 O 3 concen a ions X 4 he o al alkaline concen a ion o he solu ion a e discon inuing he connec ion o CO 2 X 5 he decomposi ion a e X 6 he es ime o he solu ion be o e connec ing he CO 2 X 7 he ime connec ing he CO 2 The se en ea u e a iables wo k as he p edic o a iables o model he quan i y o he byp oduc SiO 2 ha should be as low as possible. In he expe imen al da a his quan i y a ies in [0.0302, 0.0653]. A e au oscaling da a o ob ain ma ices X (31 × 7) and y (31 × 1), a PLS model is i ed wi h he cha ac e is ics in able 4. Looking a he pe cen age o a iance explained by adding la en a iables, i is obse ed ha , al hough he i h la en a iable cap u es mo e han 10 % o a iance in X , his is no ela ed o he SiO 2 con en we a e i ing (which is no su p ising since he p ocess is o alumina p oduc ion, no SiO 2 ). Besides, c oss alida ion wi h ene ian blinds shows a sus ained dec ease o RMSECV (Roo Mean Squa ed E o in C oss Valida ion) up un il he ou h la en a iable, hen no imp o emen is obse ed by adding mo e la en a iables. Table 4. Va iance cap u ed by PLS when adding la en a iables and he co esponding Roo Mean Squa ed E o in C oss-Valida ion (RMSECV). Numbe o la en a iables Va iance cap u ed in X (%) Cumula i e a iance cap u ed in X (%) Va iance cap u ed in Y (%) Cumula i e a iance cap u ed in Y (%) RMSECV 1 36.77 36.77 74.20 74.20 0.548 2 17.16 53.93 14.18 88.38 0.408 3 13.14 67.07 5.36 93.74 0.295 4 5.72 72.79 3.64 97.38 0.223 5 11.43 84.23 0.23 97.61 0.200 6 8.59 92.81 0.05 97.66 0.200 7 7.19 100.00 0.00 97.66 0.201 20 The e o e, he PLS p edic ing model has 4 la en a iables explaining 72.79 % o he a iance in he p edic o a iables ela ed o 97.38 % o he a iance in he esponse y . The c i ical alues a 95 % con idence le el o Q and T 2 s a is ics a e 5.26 and 12.12 espec i ely. Wi h he no a ion es ablished in sec ion 2, we ha e p = 7, a = 4, q = 1, so bo h he W - null space ela ed o he ‘ esidual’ space disca ded wi h he p ojec ion and he Q -null space inside he la en space a e h ee-dimensional subspaces. In his case, we explo e deepe he scope o knowing he ‘shape’ o he allowed a ia ions a ound any gi en poin , as consequence o he null space, and whe he i can be use ul o a gene al desc ip ion o he design space. We ha e al eady s a ed ha he a iabili y a ound a gi en poin due o he di e en null spaces does no depend on he alues o y des o he compu ed ˆ a o ˆ des x . Tha means ha we can explo e he poin s ha belong o he di e en null spaces wi hou he need o de ining a a ge quali y, o equi alen ly, conside ing ha he a ge alue is ze o ( he mean alue o he esponse when i is seen in he aw scales o he ea u e a iables). Wi h his aim and s a ing wi h W -null space (s ep 6 in ig. 1), we ake 100 poin s, uni o mly dis ibu ed, wi hin he hype cube de ined by he sco es on he h ee disca ded la en a iables, T in eq. (8), and hen econs uc he co esponding p edic o a iables, X in eq. (9), by mul iplying by he loadings P . All he ows in X a e poin s o ke (L), in o he wo ds, ep esen a ia ions ha added o any se ing o inpu a iables do no modi y he esponse p edic ed wi h hose se ings. On he o he hand, o Q -null space, we can apply he s eps in ig. 1 o y des = 0, and compu e sco es o p edic y des by using he loadings Q o eq. (2). I espec i e o he alue o y des , when ma hema ically in e ing he model, he e is no unique solu ion bu in ini ely many ec o s ha belong o he h ee-dimensional Q -null space inside he ou -dimensional la en space. To explo e his space, m=100 poin s a e also gene a ed in i , limi ing hei ange o he ange o he ac ual sco es T a in eq. (8). The p edic o a iables a e hen compu ed by using he loadings P a o ha e X 0 . Fo bo h X 0 and X , he p e en ion o using he ange o alues wi hin he co esponding sco es does no gua an ee ha he solu ions ob ained will be di ec ly alid. So, by disca ding hose objec s ou side he inpu domain o ou side he PLS-box, X 0 s ill con ain 88 ec o s (poin s) coming om Q -null space, and X ano he 71 belonging o W -null space. Figu e 7 is he pa allel coo dina es plo o he compu ed poin s, in he scale used wi h he PLS model (i.e., au oscaled alues). In he plo , each poin is ep esen ed by one b oken line ha in e sec s each coo dina e a he heigh co esponding o he alue o he poin in his coo dina e. The ed dashed lines in ig. 7 co espond o samples in X as a e e ence o he domain o he p ocess/ ea u e a iables. The con inuous blue lines a e o X 0 , compu ed om Q -null space, in ig. 7a); and o X , he se ings in W -null space, in ig. 7b). 21 Figu e 7 a ound he e Figu e 7 shows ha , in bo h cases, he poin s a e a ound he null ec o ( ha always belongs o he null space) bu a ec di e en ly depending on he a iable we look a , and o di e en a iables depending on he null space in ques ion. The lines in ig. 7a), ela ed o Q -null space inside he la en space, show ha he e is a sho e ange o allowed a ia ion o X 3 and X 4 han o he es , especially X 5 . Howe e , in ig. 7b), ela ed o W -null space, we see less a ia ion pe missible o X 2 , X 3 , p ac ically no a ia ions a ound X 5 , bu a la ge a ia ion in X 4 han in ig. 7a). Compa ing ig. 7a) and 7b) i is clea ha he wo ypes o solu ions wi h null p edic ion a e di e en and, hus, will exe a di e en impac on he design space. In addi ion o he poin s ep esen ed, any linea combina ion o hem also belongs o he ke nel, ha is, any o hem can be added o any se ings o a iables wi hou a ec ing he in ended esponse, hough he addi ion does modi y he alues o Q and T 2 s a is ics and e en he new poin s may ha e mo ed ou side he domain. O cou se, we can educe somehow he magni ude o he alues in X 0 and X so ha he egion co e ed by he poin s wi h null p edic ion would be smalle , i.e., a be e es ima ion o he pa o ke (L) ela ed o he de ini ion o he co esponding design space. Rega dless, once he solu ions in X 0 and X a e a ailable, and inside bo h he domain and he PLS-box, i is impo an o emembe ha , al hough any linea combina ion o poin s in ke (L) also belongs o ke (L), only he con ex combina ions o poin s in X 0 o X a e pa o he alid a ia ions ha can be added o a easible solu ion when de ining he design space. To illus a e he p ac ical meaning o ha ing such a desc ip ion, we will de ine a a ge esponse, say 0.0302 ( he ac ual minimum in he domain) and look o he se ings o ea u e a iables o ob ain his small quan i y o he subp oduc o he eac ion, as p edic ed wi h he PLS model. Fo he algeb aic in e sion, ig. 1, we ha e o sol e eq. (5) looking o sco es ˆ a such ha ˆ T des a y= Q , whe e y des is he au oscaled alue (using he mean and s anda d de ia ion o he o iginal esponses) o he desi ed esponse 0.0302. Sol ing he equa ion by leas squa es p o ides a unique solu ion ( , 0, 0, 0), he one wi h only one sco e non- null. The econs uc ed se ings ˆ x de ine an un easible solu ion because i is ou side he domain de ined by he ange o X , al hough i has null Q- esidual and he alue o he T 2 s a is ic is less han he h eshold alue. The e o e, ano he solu ion o eq. (5) is needed ela ed o a easible solu ion when econs uc ing he co esponding p edic o a iables. We deno e his solu ion as ˆ des x . Undoing he scaling, hei co esponding p ocess/ ea u e a iables a e w i en in he second ow in able 5. Fo compa a i e pu poses, ow 1 con ains he ac ual alues o he 22 a iables wi h which 0.0302 o SiO 2 con en in AL(OH) 3 was ob ained. We see ha he se ings in ow 1 (expe imen al) and ow 2 (calcula ed) a e no e y di e en . In addi ion, he las wo ows in able 5 desc ibe he domain es ablished o he p oblem, ha is, he ange o each p ocess a iable in he aining se o his o ical da a. P o ided ha we ollow he ows o he ma ices depic ed in igu e 7, we can add any o hem o ˆ des x wi hou modi ying he p edic ed y des . Howe e , again, e en in his case ha we made su e ha bo h ma ices X 0 and X a e in he domain and in he PLS-box, we s ill do no ha e any gua an ee ha he addi ion o hese poin s o an ac ual ˆ des x (no null) will p o ide easible solu ions. I only means ha he addi ion o such poin s does no al e he p edic ed esponse. In o he wo ds, he heo e ical p ope y o being in ke (L), by i sel , does no gi e any 'hin ' abou he easibili y o he solu ions ob ained when adding ˆ des x o one o hese poin s, i espec i e o whe he i belongs o W -null space o comes om Q -null space. Table 5. Se ings o he se en ea u es o ob ain SiO 2 con en o 0.0302. The i s ow con ains, in ac , he expe imen al condi ions in he aining se . # X 1 X 2 X 3 X 4 X 5 X 6 X 7 1 108.8 89.6 627 117.6 77.6 2.00 1.60 2 110.1 85.3 626 117.1 81.8 2.37 2.33 3 107.0 83.8 577 118.5 81.7 1.60 1.52 4 109.2 87.9 623 118.2 79.3 2.06 1.62 5 106.7 81.0 570 116.8 83.9 1.77 2.38 6 111.5 86.8 636 119.9 81.2 2.32 1.34 7 110.7 83.4 623 117.8 83.7 2.44 2.32 8 113.7 90.8 645 120.6 77.4 1.48 2.27 9 107.8 89.8 640 117.4 77.9 2.04 1.54 10 103.4 89.2 611 118.3 77.4 2.45 1.34 11 117.1 90.2 635 117.8 77.4 1.32 2.12 12 115.2 90.0 645 116.3 77.8 1.47 1.85 13 113.8 88.6 611 113.3 77.5 1.77 1.36 14 114.6 90.0 632 117.7 77.4 1.53 1.96 Minimum 103.4 81.0 570 113.3 77.4 1.32 1.34 Maximum 117.1 90.8 645 120.6 83.9 2.45 2.38 Minimum in domain 94.5 78.8 193 104.6 77.3 0.20 1.31 Maximum in domain 119.4 101.4 647 123.1 91.8 2.77 4.14 In ac , a e imposing he cons ains o he ec o s ob ained by adding elemen s o X 0 and X and sums o hem o ˆ des x , ows 3 o 14 in able 5 show some o he esul ing easible se ings o he a iables ha a e expec ed o gi e SiO 2 con en o 0.0302. Rows 15 and 16 summa ize he ange o hese solu ions. 23 Compa ing he ange in ows 15 and 16 wi h he allowed ange (las wo ows), we can see ha he design space ela ed o he minimum con en o byp oduc is a sub egion o he hype cube we ha e de ined as he domain o wo k inside. In ac , his egion is also a hype cube because any con ex combina ion o ows 2-14 in able 5 also p o ides a easible solu ion o he in e sion p oblem we a e ackling. In e ms o he p ocess analy ical echnology (PAT), able 5 can be seen as he possibili ies o use he p ocess i sel o co ec de ia ions in some p ocess a iables (o in some ea u e a iables) abo e all hose which a e no di ec ly con ollable. Fo example, say ha o some eason, he a io o SiO 2 and Al 2 O 3 concen a ions (X 3 ) dec eases om 623 o 570, as om ow 4 o ow 5. Then, ollowing he men ioned ows, we can compensa e his dec ease by educing bo h he o al alkaline concen a ion o he o iginal solu ion (X l ) un il 106.7 and he aluminum concen a ion o he o iginal solu ion, X 2 , o 81; oge he wi h a sligh educ ion o X 4 , he o al alkaline concen a ion o he solu ion a e discon inuing he connec ion o CO 2 , and less wai ing ime be o e connec ing he CO 2 , X 6 . A he same ime X 5 , he decomposi ion a e, should inc ease up o 83.9 and longe ime connec ing he CO 2 , X 7 , up o 2.38. Besides, no necessa ily he selec ed alues o he a iables mus be exac ly one o hose w i en in able 5. We ha e al eady said ha any con ex combina ion also p o ides a easible solu ion. In he p ocess, ha means ha we can use he in o ma ion in able 5 o any alue o X 3 be ween 570 (minimum) and 645 (maximum), p o ided we main ain he ela ion among ows. Fo example, X 3 = 615 is no in able 5 bu i is he esul o he con ex combina ion 570 × 0.4 + 645 × 0.6 ( alues in ows 5 and 8). Then, wi h he same con ex combina ion, he emaining p ocess a iables should be X 1 = 110.9, X 2 = 86.9, X 4 = 119.1, X 5 = 80.0, X 6 = 1.60 and X 7 = 2.31, and we use he con ollable p ocess a iables o compensa e he a iabili y in he aw ma e ial. 3.3 Supe c i ical Ca bon Dioxide Ex ac ion Zhang e al. [41] desc ibe he op imiza ion o a p ocess o oil ex ac ion using supe c i ical ca bon dioxide. Table 1 in [41] shows he se ings o h ee p ocess a iables and he co esponding esponses (expe imen al esul s) ob ained when conduc ing a Box-Behnken design in a cubic domain o i a second o de polynomial model, in he con ex o he well-known expe imen al s a egies inside he so-called Response Su ace Me hodology ( o de ails abou RSM, see o example, [42]). The p ocess depends on h ee ac o s (p ocess a iables) ha a e desc ibed in he i s h ee ows o able 6 along wi h he minimum and maximum alues o each p ocess a iable, which de ine he expe imen al domain. The las ow in able 6 co esponds o he esponse, namely he pe cen age o oil yield o N. glanduli e a seed. Table 6 also shows he ange o he pe cen age o oil yield ob ained wi h he expe imen s, as epo ed in he o iginal pape . 30 design space depends on he desi ed quali y cha ac e is ic. Besides, PAT amewo k includes he use o he p ocess o compensa e o de ia ions in some alues o he p ocess o ea u e a iables (abo e all o he a iables ha canno be di ec ly modi ied). Case 3.2 exempli ies how o do i wi h he in o ma ion ob ained om he poin s in he ke nel o he co esponding PLS model o i he con en o a byp oduc in a s ep o a p oduc ion p ocess o Al(OH) 3 . In Case 3.3 wi h a p ocess o ex ac ing oil yield o N. glanduli e a, PLS is used o desc ibe a nonlinea ela ion be ween p ocess o ea u es a iables and quali y cha ac e is ics. The e o e, he e is no algeb aic solu ion o he in e sion, and he explo a ion o he design space is based on a compu a ional in e sion. Case 3.4 illus a es he si ua ion o la ge dimensions, he es ima ion p ocedu e is he same, bu he isualiza ion o he con ex space de ined inside he design space whe e he p edic ed esponse does no change is no iable in Ca esian coo dina es. Acknowledgemen s This wo k was suppo ed by he Eu opean Regional De elopmen Fund (FEDER) h ough Spanish Agencia Es a al de In es igación (p ojec CTQ2017-88894-R). Re e ences [1] K. Fabe , B.R. Kowalski, P opaga ion o measu emen e o s o he alida ion o p edic ions ob ained by p incipal componen eg ession and pa ial leas squa es, J. Chemome ics 11 (1997) 181-238. [2] P. Facco, F. Dal Pas o, N. Meneghe i, F. Bezzo, M. Ba olo, B acke ing he Design Space wi hin he Knowledge Space in Pha maceu ical P oduc De elopmen , Ind. Eng. Chem. Res. 54 (2015) 5128−5138. [3] P. Van O e schee, B. De moo , Subspace Iden i ica ion o Linea Sys ems Theo y Implemen a ion Applica ions (1996) Kluwe Academic Publishe s. [4] Lu CY., Min H., Zhao ZQ., Zhu L., Huang DS., Yan S., Robus and E icien Subspace Segmen a ion ia Leas Squa es Reg ession. In: Fi zgibbon A., Lazebnik S., Pe ona P., Sa o Y., Schmid C. (eds) Compu e Vision – ECCV 2012. ECCV 2012. Lec u e No es in Compu e Science, ol 7578. Sp inge , Be lin, Heidelbe g. [5] S.X. Ding, Da a-d i en design o moni o ing and diagnosis sys ems o dynamic p ocesses: A e iew o subspace echnique based schemes and some ecen esul s, Jou nal o P ocess Con ol 24 (2014) 431–449. [6] Gang Li, S. Joe Qin, Donghua Zhou, Geome ic p ope ies o pa ial leas squa es o p ocess moni o ing, Au oma ica 46 (2010) 204-210. [7] V.-H. Nguyen, C. Ru en, J.-C. Golin al, Compa ison o subspace analysis me hods o aul de ec ion in indus ial sys ems, P oceedings o ISMA2010 including USD2010, (2010) 985-998. 31 [8] S. Fan, An In oduc ion o K ylo Subspace Me hods A Less Ma hema ical Way o Unde s and, a Xi :1811.09025 1 [ma h.OC] (2018). [9] J. Liesen, Z. S akos, K ylo Subspace Me hods. P inciples and Analysis, (2013) Ox o d Uni e si y P ess, Ox o d. [10] I. Fe nández de Bus os, V. Ga cía-Ma ina, G. U kullu, M. Abasolo, An e icien LDU algo i hm o he minimal leas squa es solu ion o linea sys ems, Jou nal o Compu a ional and Applied Ma hema ics 344 (2018) 346–355. [11] B.N. Da a, K ylo subspace me hods o la ge-scale ma ix p oblems in con ol, Fu u e Gene a ion Compu e Sys ems 19 (2003) 1253–1263. [12] B.N. Da a, Nume ical me hods o linea con ol sys ems design and analysis, (2004) Else ie Academic P ess, San Diego. [13] H. Wold, Es ima ion o p incipal componen s and ela ed models by i e a i e leas squa es. In: K ishnaiah, P.R. (ed.) Mul i a ia e Analysis, pp. 391–420. (1966) Academic, New Yo k. [14] Y Takane, S. Loisel On he PLS Algo i hm o Mul iple Reg ession (PLS1), in H. Abdi, V. Esposi o Vinzi, G. Russolillo, G. Sapo a, L. T inche a (eds) The Mul iple Face s o Pa ial Leas Squa es and Rela ed Me hods (2016) Sp inge . [15] A. Bjö ks öm K ylo Sequences as a Tool o Analysing I e a ed Reg ession Algo i hms, Scandina ian Jou nal o S a is ics 37 (2010) 166–175. [16] T. N. T an, L. Blanche , N.L. A anado , L. M.C. Buydens, No el uni ied amewo k o la en modeling and i s in e p e a ion, Chemome ics and In elligen Labo a o y Sys ems 149 (2015) 127–139. [17] H. Ma ens, T. Naes, Mul i a ia e Calib a ion, (1989) John Wiley & Sons, Chiches e . [18] S. Wold, E. Johansson, M. Cocchi, PLS-pa ial leas -squa es p ojec ions o la en s uc u es, pp. 523–550 in H. Kubinyi (ed.) 3D QSAR in D ug Design, (1994) Kluwe /Escom Sp inge . [19] S. Wold, L. E iksson, J. T ygg, N. Ke aneh, The PLS me hod -pa ial leas squa es p ojec ions o la en s uc u es- and i s applica ions in indus ial RDP ( esea ch, de elopmen , and p oduc ion PLS). Indus ial RPD o P ague 1 (2004), 1-44. [20] J. Engel, J. Ge e zen, E. Szymanska, J. J. Jansen, G. Downey, L. Blanche , L. M. C. Buydens, B eaking wi h ends in p e-p ocessing? T ends Anal. Chem. 50 (2013) 96–106. [21] C.Mas, L.Rubio, L.Val e de-Som, L.A.Sa abia, M.C.O iz, Impac o he p e ea men o ATR-FTIR signals on he igu es o me i when PLS is used, Chemome ics and In elligen Labo a o y Sys ems 201 (2020) 104006. [22] S. Wold, H. An i, F. Lindg en, J. Ohman, O hogonal signal co ec ion o nea - in a ed spec a, Chemome ics and In elligen Labo a o y Sys ems 44 (1998) 175– 185. [23] H.C. Goicoechea, A.C.Oli ie i, A compa ison o o hogonal signal co ec ion and ne analy e p ep ocessing me hods. Theo e ical and expe imen al s udy. Chemom In ell Lab Sys . 56 (2001) 73–78. [24] C.A. Ande sson, Di ec o hogonaliza ion, Chemome ics and In elligen Labo a o y Sys ems 47 (1999) 51– 63. [25] J. T ygg, S. Wold, O hogonal p ojec ions o la en s uc u es (OPLS), Jou nal o Chemome ics 16 (2002) 119–128. [26] R. E gon, PLS pos -p ocessing by simila i y ans o ma ion (PLS+ST): A simple al e na i e o OPLS. J. Chemom. 96 (2005) 1–4. 32 [27] T. Fea n, On o hogonal signal co ec ion, Chemome ics and In elligen Labo a o y Sys ems 50 (2000) 47– 52. [28] U.G Indhal, The O-PLS me hodology o o hogonal signal co ec ion -is i co ec ing o con using? Jou nal o Chemome ics 34 (2020) e2884. [29] H. Yu, J. F. MacG ego , Pos p ocessing me hods (PLS–CCA): simple al e na i es o p ep ocessing me hods (OSC–PLS) Chemome ics and In elligen Labo a o y Sys ems 73 (2004) 199– 205. [30] Y. Qin, C. Zhao, X. Wang, F. Gao, Subspace decomposi ion and c i ical phase selec ion based cumula i e quali y analysis o mul iphase ba ch p ocesses. Chemical Enginee ing Science 166 (2017) 130–143. [31] T. Kou i, Mul i a ia e S a is ical P ocess Con ol and P ocess Con ol, Using La en Va iables in S.D. B own, R. Taule , B. Walczak (eds.) Comp ehensi e Chemome ics Vol. 4, pp. 21-54. [32] A.M. Yan, F. Ma in, J.C. Golin al, Null-Subspace Analysis o S uc u al Damage Moni o ing in Fa igue Tes ing o Luminai es, p oceedings o ISMA2004, 557-587. [33] A.M. Yan, J.C. Golin al, Null subspace-based damage de ec ion o s uc u es using ib a ion measu emen s, Mechanical Sys ems and Signal P ocessing 20 (2006) 611–626. [34] A. R. M. Rao, V. Kasi eddy, N. Gopalak ishnan, K. Lakshmi, Senso aul de ec ion in s uc u al heal h moni o ing using null subspace–based app oach, Jou nal o In elligen Ma e ial Sys ems and S uc u es 26 (2015) 172–185. [35] C.M. Jaeckle, J.F. MacG ego , Indus ial applica ions o p oduc design h ough he in e sion o la en a iable models, Chemome ics and In elligen Labo a o y Sys ems 50 (2000) 199–210. [36] G. Bano, P. Facco, N. Meneghe i, F. Bezzo, M. Ba olo, Unce ain y back- p opaga ion in PLS model in e sion o design space de e mina ion in pha maceu ical p oduc de elopmen , Compu e s & Chemical Enginee ing 101 (2017) 110-112. [37] M. O a ian, E. Tomba, M. Ba olo, Ad anced p ocess decision making using mul i a ia e la en a iable me hods, pp. 159-189 in M. G. Ie ape i ou, R. Ramachand an (Eds.), P ocess simula ion and da a modeling in solid o al d ug de elopmen and manu ac u e, (2016) Humana P ess, Sp inge , New Yo k. [38] S. Ruiz, M.C. O iz, L.A. Sa abia, M.S. Sánchez, A compu a ional app oach o pa ial leas squa es model in e sion in he amewo k o he p ocess analy ical echnology and quali y by design ini ia i es, Chemome ics and In elligen Labo a o y Sys ems 182 (2018) 70–78. [39] S. Lang, In oduc ion o Linea Algeb a, second Ed., 1986, Sp inge -Ve lag New Yo k Inc. [40] Z. Cheng, E. Zhu, N. Chen, Applica ion o o hogonal expansion o mapping and modelling, Jou nal o Chemome ics 7 (1993) 243-253. [41] J-P. Zhang, X-L. Hou, T. Yu, Y. Li, H-Y. Dong, Response Su ace Op imiza ion o Nigella glanduli e a F eyn Seed Oil Yield by Supe c i ical Ca bon Dioxide Ex ac ion, Jou nal o In eg a i e Ag icul u e 11 (2012) 151-158. [42] L.A. Sa abia, M.C. O iz, Response Su ace Me hodology, pp. 345-390 in B own S, Taule R, Walczak R (eds.) Comp ehensi e Chemome ics, 1s Edi ion, (2009), Else ie , Ox o d. 33 Figu e cap ions Figu e 1. Diag am ha summa izes he p ocedu e o adding poin s in null spaces o he in e sion o a PLS model o p edic a gi en alue y des. Figu e 2. Th ee-dimensional inpu space in Case 3.1.1. G een diamonds ep esen he samples in X , he ed illed ci cle is ˆ des x , black poin s belong o he W -null space. The con ex hull o he black poin s is in g ay. Figu e 3. La en a iables space in Case 3.1.2. G een diamonds a e he sco es o X , he ed illed ci cle is he LS solu ion o eq. (5) and he blue illed squa es ollow he line along which he p edic ed esponse is he same. The ellipse ma ks he limi o he T 2 s a is ic (95 % con idence le el). Figu e 4. Th ee-dimensional inpu space in Case 3.1.2. G een diamonds ep esen he samples in X , he ed illed ci cle is ˆ des x , blue illed squa es a e he p edic o a iables ha co espond o he Q -null space, black poin s a e in he W -null space. In g ay, hei con ex hull. Figu e 5. La en a iables space in Case 3.1.3. G een diamonds a e he sco es o X , blue squa es a e poin s in he Q -null space and he ed ci cle is he LS solu ion o eq. (5). The ellipse ma ks he limi o he T 2 s a is ic a 95 % con idence le el. Figu e 6. Th ee-dimensional inpu space in Case 3.1.3. In yellow, he egion o easible solu ions inside he design space o y des . G een diamonds ep esen he poin s in X . Blue squa es come om he Q -null space. Black poin s come om he W -null space. Figu e 7. Case 3.2. Pa allel coo dina es plo o X in ed dashed lines. In blue some se ings: (a) coming om Q -null space, and (b) belonging o W -null space. Figu e 8. Th ee-dimensional sco es space in Case 3.3. G een diamonds a e he sco es o X , ed poin s a e inside Q -null space, wi h hose bigge ( illed ci cles) also inside he PLS-box (wi h alues o T 2 below he 95 % con idence limi ). Figu e 9. Expe imen al domain in Case 3.3. G een diamonds a e he expe imen s conduc ed. Red ci cles co espond o he ed ci cles in ig. 8 ( Q -null space), and black poin s a e in W -null space. Magen a squa es a e he poin s ound wi h he compu a ional in e sion in sec ion 3.2.2. The blue iangle ma ks he se ings ob ained wi h he compu a ional in e sion o maximizing yield in sec ion 3.2.3. Figu e 10. P ocess a iables space o case 3.4. Pa allel coo dina es plo o poin s in he ke nel o he i ed PLS model. Blue con inuous lines a e o poin s om Q -null space, do ed ed lines a e om poin s in W -null space. Decla a ion o in e es s ☒ The au ho s decla e ha hey ha e no known compe ing inancial in e es s o pe sonal ela ionships ha could ha e appea ed o in luence he wo k epo ed in his pape . ☐The au ho s decla e he ollowing inancial in e es s/pe sonal ela ionships which may be conside ed as po en ial compe ing in e es s: