Jou nal P e-p oo
Residual spaces in la en a iables model in e sion and hei impac in he design
space o gi en quali y cha ac e is ics
S. Ruiz, L.A. Sa abia, M.C. O iz, M.S. Sánchez
PII: S0169-7439(19)30469-1
DOI: h ps://doi.o g/10.1016/j.chemolab.2020.104040
Re e ence: CHEMOM 104040
To appea in: Chemome ics and In elligen Labo a o y Sys ems
Recei ed Da e: 25 July 2019
Re ised Da e: 30 Ap il 2020
Accep ed Da e: 3 May 2020
Please ci e his a icle as: S. Ruiz, L.A. Sa abia, M.C. O iz, M.S. Sánchez, Residual spaces
in la en a iables model in e sion and hei impac in he design space o gi en quali y
cha ac e is ics, Chemome ics and In elligen Labo a o y Sys ems (2020), doi: h ps://doi.o g/10.1016/
j.chemolab.2020.104040.
This is a PDF ile o an a icle ha has unde gone enhancemen s a e accep ance, such as he addi ion
o a co e page and me ada a, and o ma ing o eadabili y, bu i is no ye he de ini i e e sion o
eco d. This e sion will unde go addi ional copyedi ing, ypese ing and e iew be o e i is published
in i s inal o m, bu we a e p o iding his e sion o gi e ea ly isibili y o he a icle. Please no e ha ,
du ing he p oduc ion p ocess, e o s may be disco e ed which could a ec he con en , and all legal
disclaime s ha apply o he jou nal pe ain.
© 2020 Published by Else ie B.V.
C edi Au ho S a emen :
S. Ruiz: Concep ualiza ion, Me hodology, So wa e, Fo mal analysis, Supe ision,
W i ing, Re iew & Edi ing
L.A. Sa abia: Concep ualiza ion, Me hodology, Fo mal analysis, Supe ision,
M.C. O iz: Concep ualiza ion, Fo mal analysis, Supe ision, W i ing - Re iew & Edi ing,
Funding acquisi ion
M.S. Sánchez: Concep ualiza ion, Me hodology, Fo mal analysis, So wa e, W i ing -
O iginal D a , Supe ision, Re iew & Edi ing
1
Residual spaces in la en a iables model in e sion and hei
impac in he design space o gi en quali y cha ac e is ics
S. Ruiz
a
, L.A. Sa abia
a
, M.C. O iz
b*
, M.S. Sánchez
a
a
Dp . Ma hema ics and Compu a ion,
b
Dp . Chemis y
Facul ad de Ciencias, U
NIVERSIDAD DE
B
URGOS
, Pza. Misael Bañuelos s/n, 09001
Bu gos, Spain
Abs ac
The pape con ains a discussion abou he null spaces associa ed o linea p edic ion
models o he pa icula case o Pa ial Leas Squa es eg ession models. The
discussion sepa a ely conside s he wo exis ing null spaces: he one in he inpu space
ela ed o he p ojec ion on o he la en space and he null space, coming om he
p ojec ion space, co esponding o he mapping o he sco es on o he p edic ed
esponses.
The pape also explo es he impac o such null spaces in he de ini ion o he design
space a ound some easible solu ions ob ained by in e ing he p edic ion model, ia
se e al cases wi h simula ed and eal da a om he li e a u e. The case-s udies se e o
illus a e he discussion and he need o conside ing poin s in he wo null spaces, a he
han jus ake in o accoun he null space wi hin he la en space.
They also se e o show how o add ess he use o he esul ing ec o s in he design
space o main ain he desi ed quali y by modi ying he unable (maneu e able) p ocess
a iables o compensa e o a ia ions due o some o he ea u e a iables no so easy o
con ol.
Keywo ds
Pa ial Leas Squa es; P ocess Analy ical Technology; Quali y by Design; linea
applica ion; null space; model in e sion.
1. In oduc ion
In he con ex o p ocess con ol, p edic ion models a e commonly used o es ima e he
expec ed quali y o a p oduc as a unc ion o he cha ac e is ics o he p ocess and/o
some ea u e a iables, such as aw ma e ial p ope ies o en i onmen al ac o s. Gi en
*
Co esponding au ho , mco [email protected]
2
he equen ly la ge numbe o p ocess o ea u e a iables and quali y cha ac e is ics
and hei co ela ion (in bo h spaces), hese p edic ion models a e usually La en
Va iables Models (LVM).
Besides hei s anda d use o p edic he expec ed quali y cha ac e is ics ( he esponse
when i ing he model), hese models a e also use ul when he ocus is on main aining a
gi en quali y, like in he amewo k o Quali y by Design (QbD) es ablished o
pha maceu ical p ocesses. In he la e si ua ion, he p edic ion model mus be in e ed
o ind he se ings o he inpu a iables necessa y o gua an ee his gi en o desi ed
quali y.
The linea LVM selec ed is a Pa ial Leas Squa es (PLS) eg ession model. B ie ly, a
PLS model can be seen as he ma hema ical composi ion o wo linea maps: i s a
educ ion o dimensionali y by p ojec ing he poin s on o a space o lowe dimension
( he la en o p ojec ion space spanned by he selec ed la en a iables) and hen
eg essing he sco es on o he esponse space.
The p esen wo k ocuses on he a iabili y in bo h he inpu and la en spaces due o he
linea s uc u e o he PLS model being used, a he han on he a iabili y o he
p edic ed esponse i sel . The eade in e es ed in he s a is ical unce ain y o he
p edic ions can consul Re s. [1, 2] whe e some con idence in e als a e compu ed o
he esponse p edic ed wi h PLS models.
The use o subspaces o sol e linea p oblems is usual in enginee ing and signal
p ocessing. In con ol sys em, which is a ea di ec ly ela ed o PAT (P ocess Analy ical
Technology) and QbD, i s p inciples models (i.e. based on physical o chemical
laws) ha e been adi ionally o use o desc ibe he beha io o a sys em o subsequen ly
con ol i . Howe e , in he las yea s, he idea o using da a d i en models e en i hey
ha e no di ec physical meaning gained in e es among enginee s, ending up in so called
’subspace’ me hods. Thei name e lec s he ac ha linea models can be compu ed
om ow and column (sub)spaces o ce ain ma ices, which a e ob ained om inpu -
ou pu da a o he sys em unde obse a ion. Typically, he column space o such da a
ma ices con ains in o ma ion abou he model, while he ow spaces a e somehow
ep esen a i e o he s a e sequence, and hey a e es ima ed di ec ly om inpu -ou pu
da a wi hou ‘a p io i’ knowledge o any i s p inciples models. The decomposi ion
based on canonical classical models has some al e na i es. Fo example, in he book in
e . [3], he use o QR decomposi ion is p oposed o he p ojec ion (o hogonal o
oblique) and gene alized singula alue decomposi ion o de e mine ini e dimensional
subspace and, hen, leas squa es o ob ain he linea ela ions. Leas Squa es Reg ession
(LSR) me hod is in oduced o subspace segmen a ion in e . [4], bu mul i a ia e
analysis wi h p incipal componen analysis (PCA) and pa ial leas squa es (PLS) a e
somehow ep esen a i e me hods [5] among se e al da a-d i en echniques o p ocess
moni o ing, hanks o hei simplici y and e iciency in p ocessing huge amoun o
p ocess da a. As s a ed in e . [6], a mo e impo an objec i e o p ocess moni o ing is o
p o ide assu ance o good p oduc quali y ha is impac ed by he p ocess condi ions,
and quali y a iables a e be e aken in o accoun wi h PLS han wi h PCA.
3
Techniques o sea ching o subspaces app op ia e o model he sys em esponse a e
ex ended o non-linea models by using KPCA (Ke nel P incipal Componen Analysis)
o model nonlinea dependences be ween a iables o ep oduce nonlinea dynamic
beha io [7]. Rega ding decomposi ions, e . [8] is an in oduc ion ha shows he u ili y
o K ylo subspaces o sol e linea sys ems Ax = b, whe e A is a squa e ma ix and b is
a ec o . When A is in e ible, he solu ion is jus A
-1
b bu he calcula ion o he in e se
o A o la ge ma ices can be compu a ionally ine icien and space consuming. On his
subjec , he book by Liesen and S akos [9] con ains a e y comple e ma hema ical and
compu a ional ounda ion, along wi h some in e es ing his o ical no es. The
de elopmen o e icien algo i hms o sol e linea sys em o equa ions wi h spa se
ma ices is a ma e o cons an esea ch, see o example he ecen pape in [10].
Since he K ylo subspace me hods a e p ojec ion me hods on o a subspace, i s use as
he p ojec ion echniques o sol e la ge-scale con ol p oblems is a na u al ‘ex ension’
o i s use in nume ical linea algeb a o ma ix p oblems. Basically, i consis s o
p ojec ing he o iginal la ge con ol p oblem on o a m-dimensional K ylo subspace (m
‘small’) by cons uc ing a basis o he subspace, and hen use a s anda d well-
es ablished echnique o sol e he p ojec ed smalle p oblem. In ha way, an
app oxima e solu ion o he o iginal la ge p oblem is ob ained om he solu ion o he
p ojec ed p oblem [11,12].
PLS was ini ially de eloped as a heu is ic echnique [13] o sol e leas squa es
p oblems in mul ilinea eg ession, wi h no op imali y p ope ies and c i icized o a
long ime as a somewha ‘ad-hoc’ solu ion. La e , i was shown ha i is equi alen o
some o he mos sophis ica ed nume ical algo i hms o da e o sol ing sys ems o
linea equa ions, such as he Lanczos bidiagonaliza ion o he conjuga e g adien
me hods [14]. The p ope y ha he ec o
y
ˆ
consis ing o he alues p edic ed by a
PLS model wi h a la en a iables is he p ojec ion o he da a ec o y on o a subspace
spanned by he p edic o a iables in X (which is a subspace ha can be desc ibed as he
span o a ce ain K ylo sequence), is used in e . [15] o cha ac e ize se e al eg ession
me hods ( idge eg ession, con inuum powe eg ession, PLS and p incipal componen s
eg ession, PCR) ha a emp o educe he a iance o he OLS (o dina y leas squa es)
eg esso b
OLS
=(X
T
X)
−1
X
T
y by eplacing he ill-condi ioned ma ix (X
T
X)
−1
by a mo e
s able al e na i e.
The in e p e a ion o PLS in e ms o K ylo subspaces is also used in e . [16] o show
ha a ious la en a iables eg ession me hods sha e a common s uc u e, called
Mul ilaye Basic Sequence (MBS), which is a pa icula case o a K ylo sequence. I
allows uni ying se e al algo i hms used o es ima e he PLS coe icien s and s udying
he ela ion be ween Ma ens’ PLS [17] (o hogonal MBS) and Wold’s PLS [18,19]
(oblique MBS). Fo any o hogonal-MBS model, he e is an equi alen oblique-MBS
model so ha hey sha e he same eg ession space (o K ylo spanned space), leading
o equi alen p edic i e pe o mance. Tha means ha simila eg ession esul s can be
ob ained ia mul iple linea eg essions on o he esponse a iable. Ma ens' PLS and
Wold's PLS a e a special case o his si ua ion. This explains he iden ical eg ession

4
coe icien s and p edic ion esul s. The mapping na u e o o hogonal-MBS and
oblique-MBS om la en sco e T-space o he o iginal X-space e lec s he
co esponding de la ion echniques.
Al hough, concep ually, he in e p e a ion o he la en a iables is i ele an in he
me hods o leas squa es based on subspace decomposi ion, i becomes ‘ ele an ’ du ing
he s age o using he model, whe e he p ac i ione looks o his in e p e a ion. This is
especially ue in Chemome ics whe e PLS used o be a me hod o p edic
concen a ions o samples wi h known mul i a ia e signals, ha is, as a calib a ion
me hod. I is wo h men ioning ha , in his con ex wi h, usually, mo e a iables han
samples, he solu ion by using s anda d linea eg ession models does no exis .
When explo ing he possible in e p e a ion o he PLS calib a ion models, i u ned ou
ha he subspace spanned by he a selec ed la en a iables included in o ma ion no
di ec ly ela ed o he esponse y. This issue was add essed wi h wo me hodologies: i)
p e ea ing he signals o elimina e ‘a i ac s’ o he signal (usually physical) no ela ed
o he esponse o ii) modi ying he sea ch o he solu ion limi ing i o subspaces
co ela ed wi h he esponse. The i s app oach has led o se e al p e ea men me hods
o he p edic o a iables, which g ea ly impac he subsequen PLS calib a ion model
[20,21]. On he second me hodology men ioned, he mos gene alizable con ibu ion is
possibly he O hogonal Signal Co ec ion (OSC) by Wold e al. [22], an app op ia e
modi ica ion o he PLS algo i hm o elimina e sys ema ic y-o hogonal pa s o he
ma ix X. The Ne Analy e P ep ocessing (NAP) app oach [23] and i s exac ly
equi alen Di ec O hogonaliza ion (DO)[24] p opose some loading weigh s (no
necessa ily con ained in he ow space o X) ep esen ing phenomena conside ed o be
i ele an o he modelling o y. O he p oposals, as O hogonal-PLS (O-PLS)[25], he
al e na i e way o compu ing O-PLS [26] and he eg ession p ocedu e based on he
O hogonal Signal Co ec ion by Fea n [27], a e discussed by Indhal [28], cla i ying he
way hese OSC me hods wo k and gi ing a igo ous explana ion o he eason why he
en i e OSC concep may be bo h con using and supe luous. Yu and MacG ego [29]
poin ou ha ‘ he subspace o X unco ela ed o Y is o en high dimensional and i is
no a all clea wha pa o his subspace is emo ed by di e en OSC algo i hms wi h
di e en numbe o OSC componen s. As a esul , any in e p e a ion o he OSC
componen s may be di e en ’ and p opose he applica ion o a canonical co ela ion
analysis as pos -p ocessing o a PLS model (PLS-CCA). This app oach o de ine he
subspace mos ela ed o he quali y a iables is used in e . [30] o model a ba ch
p ocess by looking o he subspace common o he di e en slices de ined wi h he
p ocessing ime.
The e o e, ac i i y in he ield o leas -squa es eg ession on subspaces is ocused on
modeling he subspace o he p edic o s S
m
mos ela ed o he esponse. In gene al, he
o hogonal subspace S
m⊥
is only conside ed o de ining a limi , usually known as he
squa ed p edic ion e o (SPE), o decide i a new objec is compa ible wi h he i ed
PLS (o PCR) model. This index is essen ial when building mul i a ia e calib a ion
models o mul i a ia e con ol cha s based on la en a iables [31]. I s use has e en
5
been gene alized o he non-linea case o mul i a ia e con ol in he indus y using
KPCA [7]. Besides he de ini ion o SPE, he null space o he S
m⊥
eg ession model has
been used, o example, o ea ly diagnosis o s uc u al damages o machine y
mal unc ions esul ing in educ ion o he main enance cos and inc easing eliabili y
and sa e y [32,33,34].
This subsidia y use o he null space has been su passed in he ield o QbD and PAT,
because he null space is pa o he design space. This is consequence o he own
de ini ion o null space: when adding any ec o o he null space (o he eg ession
be ween X -con aining p ocess a iables, aw ma e ial o en i onmen al p ope ies, and
Y -wi h he quali y cha ac e is ics) o any ec o o he space spanned by X, he
p edic ed quali y cha ac e is ics do no change. None heless, he analyses on he null
space a e limi ed o he one ela ed o eg essing Y on he la en a iables. The QbD
con ex equi es o de ine he alues o p ocess a iables, aw ma e ial o en i onmen al
p ope ies ha esul in he same (o simila enough) alue o he measu ed quali y
cha ac e is ics and, i only he e e ed null space is used, he esul ing es ima es a e
‘indi ec ’ because he ange o admissible alues o he o iginal a iables is compu ed
om hei ange on he la en a iables.
The e o e, in wha ollows, he p edic ed alue emains in a ian . In ac , we se a
desi ed alue o he esponse y
des
( he a ge quali y) and look o he co esponding
se ings o he inpu a iables whose p edic ed alue is exac ly y
des
. Addi ionally,
a ound hese se ings, when hey exis , i is ad isable o de e mine he co esponding
design space, ha is, he egion in he inpu space whe e he p edic ed esponse is he
same and hus he expec ed a ge quali y is main ained.
When wo king wi h linea models, his design space includes he ke nel o he mapping,
also known as he null space, which we ha e al eady said ha is he ec o subspace
ha con ains he poin s mapped on o ze o, i.e. he ec o s ha do no modi y he
p edic ions when added o any o he .
To ou knowledge, Jaeckle and MacG ego [35] in oduced o he i s ime he idea o
a null space associa ed o he in e sion o a PLS model, bu i s e ec is only conside ed
in he la en space. This null space is also aken in o accoun in e . [36] when
backp opaga ing he con idence in e al in e . [1] wi h di e en p ocedu es o calcula e
a kind o con idence egions o b acke he design space.
Howe e , ocusing on he i ed PLS model, pa o he a iabili y e lec ed in he
design space comes om wha we can call he ‘ esidual’ space ela ed o he inpu
a iables. The e m ‘ esidual’ e e s o he ac ha i is disca ded when selec ing he
numbe o la en a iables because i is supposed o be he pa ha is no ela ed o he
p edic ed quali y cha ac e is ics. In pa icula , he a iabili y in he esidual space
ela ed o he inpu a iables ( he pa o he p ocess o ea u e a iables space ha is
disca ded a e he decomposi ion) does no ha e any impac on he p edic ed quali y.
The e o e, i e lec s he pa o he space whe e he inpu a iables can be modi ied
6
wi hou al e ing he quali y o he esul ing p oduc , ha is, i is necessa ily pa o he
design space.
The p esen pape in ends o desc ibe he ‘shape’ o his pa o he design space
h oughou some p oduc ion p ocesses wi h a ce ain p ede ined quali y. Technically,
his equi es he in e sion o he eg ession model because he quali y cha ac e is ics o
he p oduc a e he ones kep ixed. Gi en ha PLS is a linea model, unde ce ain
condi ions ( ela ion among he dimensions o he spaces –inpu a iables, la en , and
quali y cha ac e is ics), his in e sion can be done algeb aically by sol ing ma ix
sys ems [37]. The e a e also some compu a ional al e na i es [38] ha do no depend on
he men ioned dimensions. Rega dless o he picked solu ion, he null spaces ela ed o
he linea models being used play an impo an ole in he subsequen design space.
Ce ainly, his ‘di ision in o wo pa s’ o he null space can be igno ed and use di ec ly
i s algeb aic cha ac e iza ion by using he eg ession coe icien s o he i ed PLS
model. Howe e , no all poin s in his null space belong o he design space, because
hey also mus be adequa e o he p ocess being modelled, i.e., easible se ings o he
p ocess/ ea u e a iables ha , in addi ion, a e p ojec ed on o he la en space de ined by
he PLS model.
The basic no a ion and some heo e ical backg ound a e p esen ed in sec ion 2 and
di e en small sample sized case-s udies a e analyzed in sec ion 3 o illus a e he
p ope ies and p ac ical applicabili y. The pape ends wi h some conclusions.
2. Theo e ical p ope ies and de ini ions
Le X (n × p) and Y (n × q) be he ma ices o p (p edic o ) inpu a iables and he
co esponding q ( esponse) quali y cha ac e is ics, espec i ely. F equen ly hese a e
his o ical da a o a p ocess, a anged in wo ma ices used o i , in ou case, a PLS
model o p edic quali y cha ac e is ics gi en some se ings o he p ocess/ ea u e
a iables. These da a a e assumed o be ep esen a i e o he p ocess because he
in o ma ion hey con ain is he only one used o i he model and o make decisions, in
a comple ely da a-d i en app oach.
In wha ollows, we a e assuming ha p and q a e less han n, ha up o p new
o hogonal a iables can be ob ained in he inpu space, and ha bo h X and Y a e
au oscaled ( ha is, each column is cen e ed by sub ac ing he mean, and has a iance
one a e di iding by he s anda d de ia ion).
In addi ion, in any p ocess unde con ol, he p ocess/ ea u e a iables a e bounded. To
emula e his si ua ion, he admissible domain o he p edic o a iables will be de ined
by he ange o he obse ed alues in X. The design space we a e looking o mus be
inside his domain and wi h he se ings o he inpu a iables ha de ine easible
solu ions.
7
A PLS p edic ion model is i ed wi h X and Y by compu ing a o hogonal new
a iables ( he la en a iables) wi h a common sco es ma ix T
a
(n × a) such ha
T T
a a
X
X=TP = TP +R
(1)
T
a
Y
Y= TQ +R
(2)
Ma ices P (p × p) and Q (q × a) a e he loadings, and R
X
and R
Y
he ma ices
con aining he esiduals o he decomposi ion. As usual, uppe T means ansposing he
co esponding ma ix o ec o . Finally, sub-index a deno es ha we a e only
conside ing he i s a columns o he co esponding ma ix, so o ins ance, T is he n ×
p ma ix o X sco es, whose i s a columns o m he common sco es ma ix T
a
. The
subspace spanned by he columns o P
a
is he la en a iables space o p ojec ion
subspace ( o now on, la en space).
Once a PLS model in equa ions (1) and (2) is i ed, he e a e also some cons ain s o
adequa ely apply he model. We will use he 95 % con idence limi s o he usual Q and
T
2
s a is ics o de ine he p ope egion whe e he easible p edic o a iables a e
p ojec ed.
The alue o he T
2
s a is ic o a gi en sample is he Ho elling’s dis ance om he
p ojec ion o he o igin o he la en space (i.e., all sco es equal o ze o) and indica es
he posi ion o he p ojec ed sample. On he o he hand, Q- esiduals a e calcula ed as he
sum o squa es o each ow in R
x
, eq. (1), he esiduals a e p ojec ing he sample
ough he model. I is hen a measu e o he dis ance om he sample o he o hogonal
p ojec ion on o he la en space. Imposing limi s in bo h s a is ics is o impose a
h eshold alue o he a ia ion o poin s bo h wi hin and ou side he la en space.
Acco dingly, he i ed model mus inco po a e bo h cons ain s, so ha he PLS model
is p ope ly applied only o hose x wi h alues o Q and T
2
below he h eshold alues
imposed in bo h s a is ics. In o he wo ds, we a e de ining a bounded domain o he
applica ion o he PLS model. Geome ically, his domain can be iewed as a “PLS-
box” ha includes he pa o he la en space and he pa o he esidual space bounded
wi h, in ou case, he 95 % con idence le el o Q and T
2
s a is ics.
In ha sense, he easible solu ions a e hose ha belong simul aneously o he domain
o he inpu a iables and o wha we ha e named he PLS-box.
Fo ec o calcula ions, we will espec hei s uc u e in ma ix o m. Fo example, a
se o p p edic o a iables is w i en as a ec o x
T
because hese a iables a e ows in
ma ix X. Simila ly, y
T
is a ow ec o con aining quali y cha ac e is ics.
Wi h his no a ion, ec o x con aining some se ings o he inpu a iables is p ojec ed
on o he PLS-la en space by using some weigh s in W (p × a ma ix such ha
14
Howe e , he e a e wo dimensions disca ded when i ing he PLS model ha
impac he p edic o a iables space bu no he la en space. Because o he
p ojec ion, he e is a plane in he h ee-dimensional inpu space whe e we can mo e
wi hou modi ying he p edic ed esponse. This plane is consequence o he space
ha we ha e called he W-null space, ela ed o he esidual space, which in u n is
o hogonal o he la en space.
To explo e he e ec o he W-null space associa ed o y
des
, we gene a e m = 1000
poin s uni o mly dis ibu ed inside he ange de ined by he disca ded sco es T
,
equa ion (8), and mul iply hem by P
, he disca ded loadings ha a e a basis o he
W-null space. As we ha e al eady indica ed, all he poin s hus gene a ed ha e null
alue o T
2
.
Nex , we add all o hem o
ˆ
des
x
, s ep 6 in ig. 1, which does no modi y he
p edic ed esponse acco ding o eq. (10) al hough i does modi y he alues o Q and
T
2
. The e o e, he inal s ep (s ep 7 in ig. 1) is o emo e hose esul ing poin s
ou side ei he he domain o he inpu a iables o he la en space (in his case, only
i hei Q- esiduals a e la ge han he c i ical alue 5.68).
The emaining 726 se ings a e plo ed as small black do s in ig. 2. To acili a e
isualiza ion o he plane hey all lie on, he con ex hull o he poin s is depic ed in
ligh g ay and i is clea ha i bounds he whole plane (con aining
ˆ
des
x
) o he poin s
inside he de ined domain o he inpu a iables and he PLS-box.
3.1.2 PLS model wi h wo la en a iables. Unco ela ed p edic o a iables
Fo he sake o illus a ion, o he same da a se s X-y, le us suppose ha we had
chosen wo LV o he PLS p edic ion model. As we can see in able 1, his new
model explains 99.99 % o he a iance in y and 71.96 % o he a iance o X. Due
o he unco ela ed p edic o a iables, his second la en a iable only explains
a iance in X, in a pe cen age simila o he i s one.
Figu e 3 shows he sco es (g een diamonds) o X in he wo-dimensional la en
space. Now a = 2, so he e is a one-dimensional null space in he la en space, wha
we ha e called he Q-null space, esul ing om he in ini ely many solu ions o
equa ion (5). One o i s alid solu ion has sco e ze o on he second LV and, hus,
wi h he same
1
ˆ
on he i s LV as in 3.1.1. This solu ion is in ac he solu ion in he
leas squa es sense o he sys em in equa ion (5). I is ep esen ed in igu e 3 by a ed
illed ci cle.
The emaining sco es, solu ion o equa ion (5), heo e ically ollow a s aigh line,
he one depic ed in ig. 3 wi h blue illed squa es, ob ained by aking poin s equally
spaced along i , s ep 4 in ig. 1 wi h
a > 1 = q.

15
This line is in ac limi ed o he segmen made up wi h he poin s inside he PLS-box
in he la en space, g aphically, he segmen bounded by he ellipse depic ed in ig. 3
wi h black dashed line. The poin s enclosed by he ellipse ha e T
2
alues less han
he c i ical alue es ablished when i ing he model, namely 6.51 a 95 % con idence
le el. No ice ha he ellipse is almos a ci cum e ence due o he lack o co ela ion
among he p edic o a iables.
Figu e 3 a ound he e
By using he loadings o
P
a
in equa ion (1), his
Q
-null space is exp essed (s ep 5 in
ig. 1) in he inpu space also as a line in he h ee-dimensional space. This line is he
one de ined by he 18 blue illed squa es in ig. 4 ha co espond o he blue illed
squa es in ig. 3 whose e-cons uc ed p edic o a iables a e easible solu ions (i.e.,
belong o bo h he PLS-box and he inpu domain). Since hey a e compu ed di ec ly
om he la en space by using he loadings, hei Q- esiduals a e always null. No ice
ha he cen al posi ion is occupied by he ed illed ci cle which, also in his space,
is exac ly
ˆ
des
x
, he same as in sec ion 3.1.1.
As in he p e ious sec ion, he e is some addi ional a ia ion allowed in he inpu
space which is no being conside ed by only aking in o accoun he
Q
-null space
de ined in he la en space. The possibili ies ha a e missing come om he
W
-null
space, which in his case is he one-dimensional null space ela ed o he p ojec ion
on o he la en space, and o hogonal o i .
Wi h eq. (10) we gene a e 50 poin s in
W
-null space and, ollowing s ep 6 in ig. 1,
add hem o each o he 19 solu ions al eady ound ( he 18 blue squa es plus he ed
ci cle in ig. 3). Finally, s ep 7 in ig. 1, we keep he 760 easible solu ions.
Figu e 4 a ound he e
These easible solu ions a e he small black do s de ining pa allel lines in igu e 4
whe e hey clea ly show ha he di ec ion de e mined by he black poin s in
W
-null
space is o hogonal o he a o emen ioned line de ined by he solu ions coming om
he
Q
-null space, blue squa es.
These wo di ec ions, oge he , de ine a plane, bu only a pa o i con ains he
se ings o he p edic o a iables o which he p oblem makes sense, i.e., only he
con ex hull depic ed in ig. 4 in ligh g ay is pa o he design space a ound
ˆ
des
x
o
p edic ing
y
des
.
In any case, he bounded egions in g ay in ig. 2 and 4, inside he heo e ical planes
wi h equal p edic ed esponse, illus a e ha i only he pa o ke (L) coming om
Q
-null space is conside ed, he design space would be unde es ima ed, i is necessa y
o add poin s in
W
-null space as well.
16
3.1.3 Co ela ed p edic o a iables
I migh appea ha he cons ain s imposed o de ining he PLS-box ha e a ce ain
deg ee o a bi a iness, bu in ac hey a e he key o he con ol o a p ocess by
using he i ed model.
To illus a e his, in he same es ablished si ua ion, le us collec new da a o i ing
he model, his ime wi h a iables highly co ela ed. Then, new 50 samples wi h
h ee co ela ed a iables a e simula ed and, again, he ue alue o he esponse in
y
is compu ed as in eq. (11).
Table 2 con ains he new s uc u e o he PLS models i ed, wi h 1 and 2 la en
a iables. One o he poin s was well beyond he 95 % limi s o Q- esiduals and T
2
s a is ics and i was emo ed.
Table 2. Cha ac e is ics o he PLS model i ed o he simula ed da a wi h
co ela ed p edic o a iables.
Numbe o LV
Va iance explained in
X
(%)
Va iance explained in
y
(%) RMSECV
1
95.08
81.85
0.42
9
2
98.
80
98.
64
0.127
RMSECV: Roo Mean Squa ed E o in C oss-Valida ion
Compa ed o he models in able 1 o unco ela ed a iables in
X
, in he p esen
case, he i s la en a iable is mos ly ela ed o he co ela ion among a iables in
X
(95.08 % o a iance in
X
) and he second la en a iable is needed o be e p edic
he esponse in
y
, up o 98.64 % o explained a iance.
Figu e 5 shows he new la en space, wi h he same coding as igu e 3: G een
diamonds a e he p ojec ion o he da a poin s in
X
, he ed ci cle is he leas squa es
solu ion o eq. (5) o y
des
=0.28 (an in e media e non-null alue o au oscaled
y
), and
he blue squa es a e along he segmen consequence o he
Q
-null space, inside he
la en space, ela ed o he ed poin . The subspace is again a line segmen bu wi h a
e y di e en slope, compa ed o he one in igu e 3. Finally, he ellipse in ig. 5 is
he bounda y o he egion o he la en space belonging o he PLS–box. We see ha ,
now, is a om ci cula .
Fig.5 a ound he e
17
The egion o easible se ings o he p edic o a iables wi hin he design space
ela ed o y
des
is depic ed in ig. 6 as he con ex hull o he solu ions compu ed
ollowing he p ocedu e o ig. 1.
Figu e 6 a ound he e
The o hogonal posi ion o he lines de ined wi h he blue poin s and he ones de ined
wi h he black poin s is s ill e iden , bu he yellow egion inside he heo e ical plane
is now e y di e en om hose in ig. 2 o ig. 4. In his case i is a kind o na ow
band ha ex ends in he di ec ion de ined by he
Q
-null space (blue squa es), wi h
wid h due o he
W
-null space (black do s) and con olled, p ecisely, by he h eshold
on he Q s a is ic. The small a ia ion allowed is no su p ising aking in o accoun
ha selec ing wo la en a iables, he e is only nea 1 % o he a iance in
X
le in
he esidual space.
Summing up, e en hough he unc ional ela ion be ween
X
and
y
is he same in all he
cases, he es ima ed egions inside he design space, consequence o he null space, a e
no . In ac , he whole ke nel subspaces a e di e en . To p o e i analy ically, we use
di ec ly he co esponding eg ession coe icien s o ob ain he wo-dimensional null
space ‘a once’ wi hou making dis inc ion o he sou ce o such a iabili y (
Q
- o
W
-
null spaces).
Wi h a single esponse, he ke nel o he linea map de ined by he eg ession
coe icien s
b
can be in e p e ed as he se o poin s o hogonal o
b
(see o ins ance
page 137 in [39]), so ha he eg ession ec o
b
is he no mal ec o o he plane we
a e looking o . Wi h his p ope y, he analy ical equa ions o he planes in igu es 2, 4
and 6 a e, espec i ely:
1 1 2 3
2 1 2 3
3 1 2 3
:0.45 0.63 0.63 0.33 0
:0.54 0.58 0.61 0.32 0
:0.68 0.47 1.13 0.28 0
X X X
X X X
X X X
π
π
π
+ + − =
+ + − =
+ + − =
(12)
The i s wo planes in eq. (12),
π
1
and
π
2
, a e e y simila o each o he , and co espond
o he si ua ion wi h unco ela ed p edic o a iables. The eason o he di e ence is he
addi ion o he second la en a iable, which ha dly modi ies he p edic ions compu ed
wi h he PLS model wi h a single la en a iable (due o he independen a iables in
X)
,
bu ha
modi ies he esidual and la en spaces. These spaces a e comple ely di e en in
he case o he hi d plane
π
3
ob ained wi h a PLS model wi h 2 LV i ed wi h
co ela ed inpu a iables.
The e o e, i is clea ha he model decomposi ion is d i en by he a ailable
in o ma ion, ha is, depends on he da a a hand. The co ela ion o he inpu a iables
in he a ailable da a o ces he shape and o ien a ion o he PLS-box and, hus, also he
‘inhe i ed’ s uc u e imposed in he inpu space (whe e he design space lies).
18
In any case, we ha e al eady poin ed ou ha he exp essions in eq. (12) alone a e
useless unless we impose he cons ain s o he model. Howe e , hey can be use ul o
make decisions in some p ac ical si ua ions. Fo example, suppose ha he e was a
limi a ion in he a ailabili y o -say, X
3
(a e y expensi e eac i e o sho age in some
ma e ial). The co esponding equa ion gi es he condi ions o be imposed in he o he
wo a iables o main ain he p edic ed esponse, p o iding al e na i es o ind easible
solu ions a e applying he cons ain s.
The main conclusion is ha he design space o a gi en
y
des
should be de ined aking
in o accoun all he possible a ia ions a ound a compu ed solu ion o he in e sion. The
si ua ion in igu es 2, 4 and 6 illus a es ha i only he pa o ke (L) coming om
Q
-
null space is conside ed, he design space would be unde es ima ed, i is necessa y o
add poin s in
W
-null space as well.
Besides, as expec ed, he design space is dependen no only on he alue o he desi ed
esponse bu also on he s uc u e o
X
and
y
, p o ided hey a e p ope ly ‘collec ed’ ia
he p edic ion model. The limi s in Q and T
2
s a is ics when using a PLS model help
de ining he domain o applicabili y and con olling he p ocess.
In pa icula , ‘how much’ o he esidual space is, in ac , a ec ing he model (i.e., i is
inside he PLS-box) is mos ly con olled by he h eshold alue imposed in he Q
s a is ic. In he cases discussed in his sec ion, i only he p edic ion abili y o he PLS
model is aken in o accoun , ables 1 and 2 show ha we should selec one la en
a iable o he case o unco ela ed p edic o a iables ( ha explains 98.96 % o he
a iance in
y
wi h less han 37 % in
X
), and wo o he case o co ela ed a iables
(explaining 98.62 % o
y
and 98.8 % o
X)
. In ha case, he e is almos a 65 % o he
a iance o
X
unaccoun ed o in he i s case, which is ‘ e lec ed’ in he la ge egion in
igu e 2, whe eas he esidual space will accoun o less han 2 % o he a iance in
X
in he second case, wi h a na ow egion ela ed o he in e sion in igu e 6.
3.2 Byp oduc in Alumina p oduc ion
Table 2 in [40] con ain da a om a s udy o op imize he echnological condi ions in a
p ocess, da a ha we will use as ep esen a i e o he p ocess. Acco ding o he au ho s,
he p ocess o an alumina p oduc ion line consis s o sepa a ing ou he aluminum
hyd oxide Al(OH)
3
om a sa u a ed solu ion o sodium alumina e NaAlO
2
as a esul o
eac ion wi h ca bon dioxide, CO
2
. Un o una ely, he silicon dioxide SiO
2
p esen in
he sodium alumina e also sepa a es ou and mixes wi h he Al(OH)
3
deg ading he
quali y o he ex ac ed aluminum hyd oxide
.
The SiO
2
con en in Al(OH)
3
is he cha ac e is ic eco ded in he da ase . I consis s o
31 samples o he p oduc ion p ocess cha ac e ized by se en ea u e a iables, hose
19
lis ed in able 3, whe e we can see ha he e a e p ope p ocess a iables as well as
some ea u es measu ed in he aw ma e ials.
Table 3. P ocess and ea u e a iables o he Alumina p oduc ion p ocess
No a ion
Desc ip ion
X
l
he o al alkaline concen a ion o he o iginal solu ion
X
2
he aluminum concen a ion o he o iginal solu ion
X
3
he a io o SiO
2
and Al
2
O
3
concen a ions
X
4
he o al alkaline concen a ion o he solu ion a e
discon inuing he connec ion o CO
2
X
5
he decomposi ion a e
X
6
he es ime o he solu ion be o e connec ing he CO
2
X
7
he ime connec ing he CO
2
The se en ea u e a iables wo k as he p edic o a iables o model he quan i y o he
byp oduc SiO
2
ha should be as low as possible. In he expe imen al da a his quan i y
a ies in [0.0302, 0.0653].
A e au oscaling da a o ob ain ma ices
X
(31
×
7) and
y
(31
×
1), a PLS model is
i ed wi h he cha ac e is ics in able 4. Looking a he pe cen age o a iance explained
by adding la en a iables, i is obse ed ha , al hough he i h la en a iable cap u es
mo e han 10 % o a iance in
X
, his is no ela ed o he SiO
2
con en we a e i ing
(which is no su p ising since he p ocess is o alumina p oduc ion, no SiO
2
). Besides,
c oss alida ion wi h ene ian blinds shows a sus ained dec ease o RMSECV (Roo
Mean Squa ed E o in C oss Valida ion) up un il he ou h la en a iable, hen no
imp o emen is obse ed by adding mo e la en a iables.
Table 4. Va iance cap u ed by PLS when adding la en a iables and he co esponding Roo
Mean Squa ed E o in C oss-Valida ion (RMSECV).
Numbe o
la en
a iables
Va iance
cap u ed in
X (%)
Cumula i e
a iance
cap u ed in X
(%)
Va iance
cap u ed in Y
(%)
Cumula i e
a iance
cap u ed in Y
(%)
RMSECV
1 36.77 36.77 74.20 74.20 0.548
2 17.16 53.93 14.18 88.38 0.408
3 13.14 67.07 5.36 93.74 0.295
4 5.72 72.79 3.64 97.38 0.223
5 11.43 84.23 0.23 97.61 0.200
6 8.59 92.81 0.05 97.66 0.200
7 7.19 100.00 0.00 97.66 0.201

20
The e o e, he PLS p edic ing model has 4 la en a iables explaining 72.79 % o he
a iance in he p edic o a iables ela ed o 97.38 % o he a iance in he esponse
y
.
The c i ical alues a 95 % con idence le el o Q and T
2
s a is ics a e 5.26 and 12.12
espec i ely.
Wi h he no a ion es ablished in sec ion 2, we ha e p = 7, a = 4, q = 1, so bo h he
W
-
null space ela ed o he ‘ esidual’ space disca ded wi h he p ojec ion and he
Q
-null
space inside he la en space a e h ee-dimensional subspaces.
In his case, we explo e deepe he scope o knowing he ‘shape’ o he allowed
a ia ions a ound any gi en poin , as consequence o he null space, and whe he i can
be use ul o a gene al desc ip ion o he design space. We ha e al eady s a ed ha he
a iabili y a ound a gi en poin due o he di e en null spaces does no depend on he
alues o
y
des
o he compu ed
ˆ
a
o
ˆ
des
x
. Tha means ha we can explo e he poin s ha
belong o he di e en null spaces wi hou he need o de ining a a ge quali y, o
equi alen ly, conside ing ha he a ge alue is ze o ( he mean alue o he esponse
when i is seen in he aw scales o he ea u e a iables).
Wi h his aim and s a ing wi h
W
-null space (s ep 6 in ig. 1), we ake 100 poin s,
uni o mly dis ibu ed, wi hin he hype cube de ined by he sco es on he h ee disca ded
la en a iables,
T
in eq. (8),
and hen econs uc he co esponding p edic o a iables,
X
in eq. (9), by mul iplying by he loadings
P
. All he ows in
X
a e poin s o ke (L),
in o he wo ds, ep esen a ia ions ha added o any se ing o inpu a iables do no
modi y he esponse p edic ed wi h hose se ings.
On he o he hand, o
Q
-null space, we can apply he s eps in ig. 1 o y
des
= 0, and
compu e sco es o p edic y
des
by using he loadings
Q
o eq. (2). I espec i e o he
alue o y
des
, when ma hema ically in e ing he model, he e is no unique solu ion bu
in ini ely many ec o s ha belong o he h ee-dimensional
Q
-null space inside he
ou -dimensional la en space. To explo e his space, m=100 poin s a e also gene a ed
in i , limi ing hei ange o he ange o he ac ual sco es
T
a
in eq. (8). The p edic o
a iables a e hen compu ed by using he loadings
P
a
o ha e
X
0
.
Fo bo h
X
0
and
X
, he p e en ion o using he ange o alues wi hin he
co esponding sco es does no gua an ee ha he solu ions ob ained will be di ec ly
alid. So, by disca ding hose objec s ou side he inpu domain o ou side he PLS-box,
X
0
s ill con ain 88 ec o s (poin s) coming om
Q
-null space, and
X
ano he 71
belonging o
W
-null space.
Figu e 7 is he pa allel coo dina es plo o he compu ed poin s, in he scale used wi h
he PLS model (i.e., au oscaled alues). In he plo , each poin is ep esen ed by one
b oken line ha in e sec s each coo dina e a he heigh co esponding o he alue o
he poin in his coo dina e. The ed dashed lines in ig. 7 co espond o samples in
X
as
a e e ence o he domain o he p ocess/ ea u e a iables. The con inuous blue lines a e
o
X
0
, compu ed om
Q
-null space, in ig. 7a); and o
X
, he se ings in
W
-null
space, in ig. 7b).
21
Figu e 7 a ound he e
Figu e 7 shows ha , in bo h cases, he poin s a e a ound he null ec o ( ha always
belongs o he null space) bu a ec di e en ly depending on he a iable we look a ,
and o di e en a iables depending on he null space in ques ion.
The lines in ig. 7a), ela ed o
Q
-null space inside he la en space, show ha he e is a
sho e ange o allowed a ia ion o X
3
and X
4
han o he es , especially X
5
.
Howe e , in ig. 7b), ela ed o
W
-null space, we see less a ia ion pe missible o X
2
,
X
3
, p ac ically no a ia ions a ound X
5
, bu a la ge a ia ion in X
4
han in ig. 7a).
Compa ing ig. 7a) and 7b) i is clea ha he wo ypes o solu ions wi h null p edic ion
a e di e en and, hus, will exe a di e en impac on he design space.
In addi ion o he poin s ep esen ed, any linea combina ion o hem also belongs o he
ke nel, ha is, any o hem can be added o any se ings o a iables wi hou a ec ing
he in ended esponse, hough he addi ion does modi y he alues o Q and T
2
s a is ics
and e en he new poin s may ha e mo ed ou side he domain. O cou se, we can educe
somehow he magni ude o he alues in
X
0
and
X
so ha he egion co e ed by he
poin s wi h null p edic ion would be smalle , i.e., a be e es ima ion o he pa o
ke (L) ela ed o he de ini ion o he co esponding design space.
Rega dless, once he solu ions in
X
0
and
X
a e a ailable, and inside bo h he domain
and he PLS-box, i is impo an o emembe ha , al hough any linea combina ion o
poin s in ke (L) also belongs o ke (L), only he con ex combina ions o poin s in
X
0
o
X
a e pa o he alid a ia ions ha can be added o a easible solu ion when de ining
he design space.
To illus a e he p ac ical meaning o ha ing such a desc ip ion, we will de ine a a ge
esponse, say 0.0302 ( he ac ual minimum in he domain) and look o he se ings o
ea u e a iables o ob ain his small quan i y o he subp oduc o he eac ion, as
p edic ed wi h he PLS model.
Fo he algeb aic in e sion, ig. 1, we ha e o sol e eq. (5) looking o sco es
ˆ
a
such
ha
ˆ
T
des a
y=
Q
, whe e y
des
is he au oscaled alue (using he mean and s anda d
de ia ion o he o iginal esponses) o he desi ed esponse 0.0302. Sol ing he equa ion
by leas squa es p o ides a unique solu ion ( , 0, 0, 0), he one wi h only one sco e non-
null. The econs uc ed se ings
ˆ
x
de ine an un easible solu ion because i is ou side he
domain de ined by he ange o
X
, al hough i has null Q- esidual and he alue o he
T
2
s a is ic is less han he h eshold alue. The e o e, ano he solu ion o eq. (5) is
needed ela ed o a easible solu ion when econs uc ing he co esponding p edic o
a iables. We deno e his solu ion as
ˆ
des
x
.
Undoing he scaling, hei co esponding p ocess/ ea u e a iables a e w i en in he
second ow in able 5. Fo compa a i e pu poses, ow 1 con ains he ac ual alues o he
22
a iables wi h which 0.0302 o SiO
2
con en in AL(OH)
3
was ob ained. We see ha he
se ings in ow 1 (expe imen al) and ow 2 (calcula ed) a e no e y di e en .
In addi ion, he las wo ows in able 5 desc ibe he domain es ablished o he p oblem,
ha is, he ange o each p ocess a iable in he aining se o his o ical da a.
P o ided ha we ollow he ows o he ma ices depic ed in igu e 7, we can add any o
hem o
ˆ
des
x
wi hou modi ying he p edic ed y
des
. Howe e , again, e en in his case
ha we made su e ha bo h ma ices
X
0
and
X
a e in he domain and in he PLS-box,
we s ill do no ha e any gua an ee ha he addi ion o hese poin s o an ac ual
ˆ
des
x
(no
null) will p o ide easible solu ions. I only means ha he addi ion o such poin s does
no al e he p edic ed esponse.
In o he wo ds, he heo e ical p ope y o being in ke (L), by i sel , does no gi e any
'hin ' abou he easibili y o he solu ions ob ained when adding
ˆ
des
x
o one o hese
poin s, i espec i e o whe he i belongs o
W
-null space o comes om
Q
-null space.
Table 5. Se ings o he se en ea u es o ob ain SiO
2
con en o 0.0302. The i s ow
con ains, in ac , he expe imen al condi ions in he aining se .
# X
1
X
2
X
3
X
4
X
5
X
6
X
7
1 108.8 89.6 627 117.6 77.6 2.00 1.60
2 110.1 85.3 626 117.1 81.8 2.37 2.33
3 107.0 83.8 577 118.5 81.7 1.60 1.52
4 109.2 87.9 623 118.2 79.3 2.06 1.62
5 106.7 81.0 570 116.8 83.9 1.77 2.38
6 111.5 86.8 636 119.9 81.2 2.32 1.34
7 110.7 83.4 623 117.8 83.7 2.44 2.32
8 113.7 90.8 645 120.6 77.4 1.48 2.27
9 107.8 89.8 640 117.4 77.9 2.04 1.54
10 103.4 89.2 611 118.3 77.4 2.45 1.34
11 117.1 90.2 635 117.8 77.4 1.32 2.12
12 115.2 90.0 645 116.3 77.8 1.47 1.85
13 113.8 88.6 611 113.3 77.5 1.77 1.36
14 114.6 90.0 632 117.7 77.4 1.53 1.96
Minimum 103.4 81.0 570 113.3 77.4 1.32 1.34
Maximum 117.1 90.8 645 120.6 83.9 2.45 2.38
Minimum in domain 94.5 78.8 193 104.6 77.3 0.20 1.31
Maximum in domain 119.4 101.4 647 123.1 91.8 2.77 4.14
In ac , a e imposing he cons ains o he ec o s ob ained by adding elemen s o
X
0
and
X
and sums o hem o
ˆ
des
x
, ows 3 o 14 in able 5 show some o he esul ing
easible se ings o he a iables ha a e expec ed o gi e SiO
2
con en o 0.0302. Rows
15 and 16 summa ize he ange o hese solu ions.
23
Compa ing he ange in ows 15 and 16 wi h he allowed ange (las wo ows), we can
see ha he design space ela ed o he minimum con en o byp oduc is a sub egion o
he hype cube we ha e de ined as he domain o wo k inside. In ac , his egion is also
a hype cube because any con ex combina ion o ows 2-14 in able 5 also p o ides a
easible solu ion o he in e sion p oblem we a e ackling.
In e ms o he p ocess analy ical echnology (PAT), able 5 can be seen as he
possibili ies o use he p ocess i sel o co ec de ia ions in some p ocess a iables (o
in some ea u e a iables) abo e all hose which a e no di ec ly con ollable.
Fo example, say ha o some eason, he a io o SiO
2
and Al
2
O
3
concen a ions (X
3
)
dec eases om 623 o 570, as om ow 4 o ow 5. Then, ollowing he men ioned
ows, we can compensa e his dec ease by educing bo h he o al alkaline concen a ion
o he o iginal solu ion (X
l
) un il 106.7 and he aluminum concen a ion o he o iginal
solu ion, X
2
, o 81; oge he wi h a sligh educ ion o X
4
, he o al alkaline concen a ion
o he solu ion a e discon inuing he connec ion o CO
2
, and less wai ing ime be o e
connec ing he CO
2
, X
6
. A he same ime X
5
, he decomposi ion a e, should inc ease up
o 83.9 and longe ime connec ing he CO
2
, X
7
, up o 2.38.
Besides, no necessa ily he selec ed alues o he a iables mus be exac ly one o hose
w i en in able 5. We ha e al eady said ha any con ex combina ion also p o ides a
easible solu ion. In he p ocess, ha means ha we can use he in o ma ion in able 5
o any alue o X
3
be ween 570 (minimum) and 645 (maximum), p o ided we
main ain he ela ion among ows. Fo example, X
3
= 615 is no in able 5 bu i is he
esul o he con ex combina ion 570
×
0.4 + 645
×
0.6 ( alues in ows 5 and 8). Then,
wi h he same con ex combina ion, he emaining p ocess a iables should be X
1
=
110.9, X
2
= 86.9, X
4
= 119.1, X
5
= 80.0, X
6
= 1.60 and X
7
= 2.31, and we use he
con ollable p ocess a iables o compensa e he a iabili y in he aw ma e ial.
3.3 Supe c i ical Ca bon Dioxide Ex ac ion
Zhang e al. [41] desc ibe he op imiza ion o a p ocess o oil ex ac ion using
supe c i ical ca bon dioxide. Table 1 in [41] shows he se ings o h ee p ocess
a iables and he co esponding esponses (expe imen al esul s) ob ained when
conduc ing a Box-Behnken design in a cubic domain o i a second o de polynomial
model, in he con ex o he well-known expe imen al s a egies inside he so-called
Response Su ace Me hodology ( o de ails abou RSM, see o example, [42]).
The p ocess depends on h ee ac o s (p ocess a iables) ha a e desc ibed in he i s
h ee ows o able 6 along wi h he minimum and maximum alues o each p ocess
a iable, which de ine he expe imen al domain.
The las ow in able 6 co esponds o he esponse, namely he pe cen age o oil yield
o N. glanduli e a seed. Table 6 also shows he ange o he pe cen age o oil yield
ob ained wi h he expe imen s, as epo ed in he o iginal pape .
30
design space depends on he desi ed quali y cha ac e is ic. Besides, PAT amewo k
includes he use o he p ocess o compensa e o de ia ions in some alues o he
p ocess o ea u e a iables (abo e all o he a iables ha canno be di ec ly
modi ied). Case 3.2 exempli ies how o do i wi h he in o ma ion ob ained om he
poin s in he ke nel o he co esponding PLS model o i he con en o a byp oduc in
a s ep o a p oduc ion p ocess o Al(OH)
3
.
In Case 3.3 wi h a p ocess o ex ac ing oil yield o N. glanduli e a, PLS is used o
desc ibe a nonlinea ela ion be ween p ocess o ea u es a iables and quali y
cha ac e is ics. The e o e, he e is no algeb aic solu ion o he in e sion, and he
explo a ion o he design space is based on a compu a ional in e sion.
Case 3.4 illus a es he si ua ion o la ge dimensions, he es ima ion p ocedu e is he
same, bu he isualiza ion o he con ex space de ined inside he design space whe e
he p edic ed esponse does no change is no iable in Ca esian coo dina es.
Acknowledgemen s
This wo k was suppo ed by he Eu opean Regional De elopmen Fund (FEDER)
h ough Spanish Agencia Es a al de In es igación (p ojec CTQ2017-88894-R).
Re e ences
[1] K. Fabe , B.R. Kowalski, P opaga ion o measu emen e o s o he alida ion o
p edic ions ob ained by p incipal componen eg ession and pa ial leas squa es,
J. Chemome ics 11 (1997) 181-238.
[2] P. Facco, F. Dal Pas o, N. Meneghe i, F. Bezzo, M. Ba olo, B acke ing he
Design Space wi hin he Knowledge Space in Pha maceu ical P oduc
De elopmen , Ind. Eng. Chem. Res. 54 (2015) 5128−5138.
[3] P. Van O e schee, B. De moo , Subspace Iden i ica ion o Linea Sys ems
Theo y Implemen a ion Applica ions (1996) Kluwe Academic Publishe s.
[4] Lu CY., Min H., Zhao ZQ., Zhu L., Huang DS., Yan S., Robus and E icien
Subspace Segmen a ion ia Leas Squa es Reg ession. In: Fi zgibbon A.,
Lazebnik S., Pe ona P., Sa o Y., Schmid C. (eds) Compu e Vision – ECCV 2012.
ECCV 2012. Lec u e No es in Compu e Science, ol 7578. Sp inge , Be lin,
Heidelbe g.
[5] S.X. Ding, Da a-d i en design o moni o ing and diagnosis sys ems o dynamic
p ocesses: A e iew o subspace echnique based schemes and some ecen
esul s, Jou nal o P ocess Con ol 24 (2014) 431–449.
[6] Gang Li, S. Joe Qin, Donghua Zhou, Geome ic p ope ies o pa ial leas squa es
o p ocess moni o ing, Au oma ica 46 (2010) 204-210.
[7] V.-H. Nguyen, C. Ru en, J.-C. Golin al, Compa ison o subspace analysis
me hods o aul de ec ion in indus ial sys ems, P oceedings o ISMA2010
including USD2010, (2010) 985-998.

31
[8] S. Fan, An In oduc ion o K ylo Subspace Me hods A Less Ma hema ical Way
o Unde s and, a Xi :1811.09025 1 [ma h.OC] (2018).
[9] J. Liesen, Z. S akos, K ylo Subspace Me hods. P inciples and Analysis, (2013)
Ox o d Uni e si y P ess, Ox o d.
[10] I. Fe nández de Bus os, V. Ga cía-Ma ina, G. U kullu, M. Abasolo, An e icien
LDU algo i hm o he minimal leas squa es solu ion o linea sys ems, Jou nal o
Compu a ional and Applied Ma hema ics 344 (2018) 346–355.
[11] B.N. Da a, K ylo subspace me hods o la ge-scale ma ix p oblems in con ol,
Fu u e Gene a ion Compu e Sys ems 19 (2003) 1253–1263.
[12] B.N. Da a, Nume ical me hods o linea con ol sys ems design and analysis,
(2004) Else ie Academic P ess, San Diego.
[13] H. Wold, Es ima ion o p incipal componen s and ela ed models by i e a i e leas
squa es. In: K ishnaiah, P.R. (ed.) Mul i a ia e Analysis, pp. 391–420. (1966)
Academic, New Yo k.
[14] Y Takane, S. Loisel On he PLS Algo i hm o Mul iple Reg ession (PLS1), in H.
Abdi, V. Esposi o Vinzi, G. Russolillo, G. Sapo a, L. T inche a (eds) The
Mul iple Face s o Pa ial Leas Squa es and Rela ed Me hods (2016) Sp inge .
[15] A. Bjö ks öm K ylo Sequences as a Tool o Analysing I e a ed Reg ession
Algo i hms, Scandina ian Jou nal o S a is ics 37 (2010) 166–175.
[16] T. N. T an, L. Blanche , N.L. A anado , L. M.C. Buydens, No el uni ied
amewo k o la en modeling and i s in e p e a ion, Chemome ics and
In elligen Labo a o y Sys ems 149 (2015) 127–139.
[17] H. Ma ens, T. Naes, Mul i a ia e Calib a ion, (1989) John Wiley & Sons,
Chiches e .
[18] S. Wold, E. Johansson, M. Cocchi, PLS-pa ial leas -squa es p ojec ions o la en
s uc u es, pp. 523–550 in H. Kubinyi (ed.) 3D QSAR in D ug Design, (1994)
Kluwe /Escom Sp inge .
[19] S. Wold, L. E iksson, J. T ygg, N. Ke aneh, The PLS me hod -pa ial leas
squa es p ojec ions o la en s uc u es- and i s applica ions in indus ial RDP
( esea ch, de elopmen , and p oduc ion PLS). Indus ial RPD o P ague 1 (2004),
1-44.
[20] J. Engel, J. Ge e zen, E. Szymanska, J. J. Jansen, G. Downey, L. Blanche , L. M.
C. Buydens, B eaking wi h ends in p e-p ocessing? T ends Anal. Chem. 50
(2013) 96–106.
[21] C.Mas, L.Rubio, L.Val e de-Som, L.A.Sa abia, M.C.O iz, Impac o he
p e ea men o ATR-FTIR signals on he igu es o me i when PLS is used,
Chemome ics and In elligen Labo a o y Sys ems 201 (2020) 104006.
[22] S. Wold, H. An i, F. Lindg en, J. Ohman, O hogonal signal co ec ion o nea -
in a ed spec a, Chemome ics and In elligen Labo a o y Sys ems 44 (1998)
175– 185.
[23] H.C. Goicoechea, A.C.Oli ie i, A compa ison o o hogonal signal co ec ion and
ne analy e p ep ocessing me hods. Theo e ical and expe imen al s udy. Chemom
In ell Lab Sys . 56 (2001) 73–78.
[24] C.A. Ande sson, Di ec o hogonaliza ion, Chemome ics and In elligen
Labo a o y Sys ems 47 (1999) 51– 63.
[25] J. T ygg, S. Wold, O hogonal p ojec ions o la en s uc u es (OPLS), Jou nal o
Chemome ics 16 (2002) 119–128.
[26] R. E gon, PLS pos -p ocessing by simila i y ans o ma ion (PLS+ST): A simple
al e na i e o OPLS. J. Chemom. 96 (2005) 1–4.
32
[27] T. Fea n, On o hogonal signal co ec ion, Chemome ics and In elligen
Labo a o y Sys ems 50 (2000) 47– 52.
[28] U.G Indhal, The O-PLS me hodology o o hogonal signal co ec ion -is i
co ec ing o con using? Jou nal o Chemome ics 34 (2020) e2884.
[29] H. Yu, J. F. MacG ego , Pos p ocessing me hods (PLS–CCA): simple
al e na i es o p ep ocessing me hods (OSC–PLS) Chemome ics and In elligen
Labo a o y Sys ems 73 (2004) 199– 205.
[30] Y. Qin, C. Zhao, X. Wang, F. Gao, Subspace decomposi ion and c i ical phase
selec ion based cumula i e quali y analysis o mul iphase ba ch p ocesses.
Chemical Enginee ing Science 166 (2017) 130–143.
[31] T. Kou i, Mul i a ia e S a is ical P ocess Con ol and P ocess Con ol, Using
La en Va iables in S.D. B own, R. Taule , B. Walczak (eds.) Comp ehensi e
Chemome ics Vol. 4, pp. 21-54.
[32] A.M. Yan, F. Ma in, J.C. Golin al, Null-Subspace Analysis o S uc u al
Damage Moni o ing in Fa igue Tes ing o Luminai es, p oceedings o ISMA2004,
557-587.
[33] A.M. Yan, J.C. Golin al, Null subspace-based damage de ec ion o s uc u es
using ib a ion measu emen s, Mechanical Sys ems and Signal P ocessing 20
(2006) 611–626.
[34] A. R. M. Rao, V. Kasi eddy, N. Gopalak ishnan, K. Lakshmi, Senso aul
de ec ion in s uc u al heal h moni o ing using null subspace–based app oach,
Jou nal o In elligen Ma e ial Sys ems and S uc u es 26 (2015) 172–185.
[35] C.M. Jaeckle, J.F. MacG ego , Indus ial applica ions o p oduc design h ough
he in e sion o la en a iable models, Chemome ics and In elligen Labo a o y
Sys ems 50 (2000) 199–210.
[36] G. Bano, P. Facco, N. Meneghe i, F. Bezzo, M. Ba olo, Unce ain y back-
p opaga ion in PLS model in e sion o design space de e mina ion in
pha maceu ical p oduc de elopmen , Compu e s & Chemical Enginee ing 101
(2017) 110-112.
[37] M. O a ian, E. Tomba, M. Ba olo, Ad anced p ocess decision making using
mul i a ia e la en a iable me hods, pp. 159-189 in M. G. Ie ape i ou, R.
Ramachand an (Eds.), P ocess simula ion and da a modeling in solid o al d ug
de elopmen and manu ac u e, (2016) Humana P ess, Sp inge , New Yo k.
[38] S. Ruiz, M.C. O iz, L.A. Sa abia, M.S. Sánchez, A compu a ional app oach o
pa ial leas squa es model in e sion in he amewo k o he p ocess analy ical
echnology and quali y by design ini ia i es, Chemome ics and In elligen
Labo a o y Sys ems 182 (2018) 70–78.
[39] S. Lang, In oduc ion o Linea Algeb a, second Ed., 1986, Sp inge -Ve lag New
Yo k Inc.
[40] Z. Cheng, E. Zhu, N. Chen, Applica ion o o hogonal expansion o mapping and
modelling, Jou nal o Chemome ics 7 (1993) 243-253.
[41] J-P. Zhang, X-L. Hou, T. Yu, Y. Li, H-Y. Dong, Response Su ace Op imiza ion
o Nigella glanduli e a F eyn Seed Oil Yield by Supe c i ical Ca bon Dioxide
Ex ac ion, Jou nal o In eg a i e Ag icul u e 11 (2012) 151-158.
[42] L.A. Sa abia, M.C. O iz, Response Su ace Me hodology, pp. 345-390 in B own
S, Taule R, Walczak R (eds.) Comp ehensi e Chemome ics, 1s Edi ion, (2009),
Else ie , Ox o d.
33
Figu e cap ions
Figu e 1. Diag am ha summa izes he p ocedu e o adding poin s in null spaces o he
in e sion o a PLS model o p edic a gi en alue y
des.
Figu e 2. Th ee-dimensional inpu space in Case 3.1.1. G een diamonds ep esen he
samples in
X
, he ed illed ci cle is
ˆ
des
x
, black poin s belong o he
W
-null
space. The con ex hull o he black poin s is in g ay.
Figu e 3. La en a iables space in Case 3.1.2. G een diamonds a e he sco es o
X
, he
ed illed ci cle is he LS solu ion o eq. (5) and he blue illed squa es ollow
he line along which he p edic ed esponse is he same. The ellipse ma ks he
limi o he T
2
s a is ic (95 % con idence le el).
Figu e 4. Th ee-dimensional inpu space in Case 3.1.2. G een diamonds ep esen he
samples in
X
, he ed illed ci cle is
ˆ
des
x
, blue illed squa es a e he p edic o
a iables ha co espond o he
Q
-null space, black poin s a e in he
W
-null
space. In g ay, hei con ex hull.
Figu e 5. La en a iables space in Case 3.1.3. G een diamonds a e he sco es o
X
, blue
squa es a e poin s in he
Q
-null space and he ed ci cle is he LS solu ion o
eq. (5). The ellipse ma ks he limi o he T
2
s a is ic a 95 % con idence le el.
Figu e 6. Th ee-dimensional inpu space in Case 3.1.3. In yellow, he egion o easible
solu ions inside he design space o y
des
. G een diamonds ep esen he poin s
in
X
. Blue squa es come om he
Q
-null space. Black poin s come om he
W
-null space.
Figu e 7. Case 3.2. Pa allel coo dina es plo o
X
in ed dashed lines. In blue some
se ings: (a) coming om
Q
-null space, and (b) belonging o
W
-null space.
Figu e 8. Th ee-dimensional sco es space in Case 3.3. G een diamonds a e he sco es o
X
, ed poin s a e inside
Q
-null space, wi h hose bigge ( illed ci cles) also
inside he PLS-box (wi h alues o T
2
below he 95 % con idence limi ).
Figu e 9. Expe imen al domain in Case 3.3. G een diamonds a e he expe imen s
conduc ed. Red ci cles co espond o he ed ci cles in ig. 8 (
Q
-null space),
and black poin s a e in
W
-null space. Magen a squa es a e he poin s ound
wi h he compu a ional in e sion in sec ion 3.2.2. The blue iangle ma ks he
se ings ob ained wi h he compu a ional in e sion o maximizing yield in
sec ion 3.2.3.
Figu e 10. P ocess a iables space o case 3.4. Pa allel coo dina es plo o poin s in
he ke nel o he i ed PLS model. Blue con inuous lines a e o poin s om
Q
-null space, do ed ed lines a e om poin s in
W
-null space.

Decla a ion o in e es s
☒
The au ho s decla e ha hey ha e no known compe ing inancial in e es s o pe sonal ela ionships
ha could ha e appea ed o in luence he wo k epo ed in his pape .
☐The au ho s decla e he ollowing inancial in e es s/pe sonal ela ionships which may be conside ed
as po en ial compe ing in e es s: