A gene al app oach o So wa e P oduc Line
es ing
El i a G. Ruiz1, Jon Aye di2, José A. Galindo1, Ai o A ie a2, Goiu ia
Saga dui2, and Da id Bena ides1
1Uni e sidad de Se illa, Dep . Lenguajes y Sis emas In o má icos, A . Reina
Me cedes s/n Se illa - España,
{eg uiz, jagalindo, bena ides}@us.es
2Mond agon Unibe si a ea, Dep . de Elec ónica e In o má ica, Goi u 2,
Mond agon - España
[email p o ec ed], [email p o ec ed],
[email p o ec ed]
Abs ac . Va iabili y is a cen al concep in So wa e P oduc Lines
(SPLs). I has been ex ensi ely s udied how he SPL pa adigm can im-
p o e bo h he e iciency o a company and he quali y o p oduc s. Ne -
e heless, his b ings se e al challenges when es ing an SPL, which a e
mainly caused by he po en ially huge amoun o p oduc s ha can be
de i ed om an SPL. Di e en s udies p oposing me hods o es ing
SPLs exis . Fu he mo e, he e a e seconda y s udies e iewing and map-
ping he li e a u e o he exis ing p oposals. Howe e , he e is a lack o
sys ema ic guidelines o p ac i ione s and esea che s wi h he di e en
s eps equi ed o pe o m a es ing s a egy o an SPL. In his pape ,
we p esen a i s p elimina y e sion o a u o ial ha summa izes he
exis ing p oposals o he SPL es ing a ea. To he bes o ou knowledge,
he e is no simila a emp in exis ing li e a u e. Ou goal is o discuss
his u o ial wi h he communi y and en ich i o p o ide a mo e solid
e sion o i in he u u e.
Keywo ds: So wa e p oduc lines, So wa e es ing, So wa e eusabil-
i y.
1 In oduc ion
So wa e p oduc lines and a iabili y in ensi e sys ems bene i s om a se o
echniques, ools and me hods ha a e used o de elop a se o di e en p oduc s
ha sha e some commonali ies [26]. The conc e e unc ionali y ha a ies ac oss
p oduc s in he SPL is encapsula ed using an abs ac ion known as ea u e.
Fea u e models a e used o encode common and a ying pa s o SPLs [17].
In he li e a u e, we ind eal examples encoding a la ge numbe o p oduc s.
Fo example, he Linux Ke nel [24] wi h mo e han 6,000 ea u es o Debian
packaging sys ems [11] wi h mo e han 27,000.
The la ge amoun o p oduc s ha an SPL can encode, makes i s analysis
a ime–consuming and e o p one ask. Then, esea che s p oposed he use o
au oma ed analysis echniques [5] o a se o ac i i ies in which es ing is usually
one o he mos ele an [12].
SPL es ing ep esen s a new challenge o so wa e es ing p ac i ione s and
esea che s [23]. When es ing SPL, each p oduc sha es some common unc-
ionali y wi h one o mo e p oduc s, while di e ing in a leas one ea u e. SPLs
add es ing complexi y because hey equi e es ing a se o p oduc s a he
han a single p oduc . These p oduc s, howe e , sha e common unc ionali y o
a i ac s, enabling he euse o some es s ac oss he en i e SPL.
Acco ding o [23], se e al s a egies can be used o es SPL p oduc s. These
es ing s a egies can be summa ized as ollows: i) es ing p oduc by p oduc ,
ii)inc emen al es ing, and iii) eusable asse ins an ia ion. Tes ing p oduc by
p oduc is a s a egy ha es s all p oduc s one by one, as i hey we e no pa
o an SPL. Wi h his s a egy he es p ocess co e s all possible in e ac ions
be ween ea u es bu g ows exponen ially in cos as a unc ion o he numbe o
ea u es in he SPL. Inc emen al es ing is a s a egy ha s a s by es ing he
i s de eloped p oduc and c ea es new uni es s o each new ea u e added.
Using his s a egy, he commonali ies in he SPL a e exploi ed o educe es ing
e o . Howe e , when a new ea u e is in oduced, all he in e ac ions be ween
he new ea u e and he old ones ha e o be es ed, which can be challenging o
la ge SPLs. Reusable asse ins an ia ion elies on da a cap u ed in he domain
analysis s age o SPL c ea ion o de elop a se o abs ac es cases ha co e
all ea u es (bu no necessa ily con igu a ions) in he SPL. These abs ac es s
cases a e mapped o conc e e equi emen s in he applica ion enginee ing s age.
These las wo es ing s a egies a e designed o educe he SPL combina o ial
explosion in es ing cos as a unc ion o he ea u e coun .
Wi hin SPL enginee ing, wo di e en p ocesses can be dis inguished: (1)
domain enginee ing and (2) applica ion enginee ing. Domain enginee ing is he
p ocess o de eloping he pla o m o building p oduc s and de ining he com-
monali ies and he a iabili y o he p oduc line. Applica ion enginee ing is he
p ocess o de i ing speci ic applica ions by using he pla o m de ined in domain
enginee ing and binding he a iabili y o sa is y he needs o each pa icula
applica ion [22].
In [15] au ho s p opose an ideal pa h o ollow when i comes o SPL es ing,
which is a W es ing model o SPLs ha conside s componen , in eg a ion
and sys em es ing o bo h domain and applica ion enginee ing. This pa adigm
maps e e y sub-p ocess o ei he domain o applica ion enginee ing. Howe e ,
and due o he complexi y o he es ing p ocesses when a iabili y is conside ed,
in many so wa e p ojec s he e is no such a clea di ision o asks. I is possible
o ind some es ing p ocesses (i.e., componen es ing) ha can be s a ed in
he domain enginee ing phase and con inued du ing he applica ion enginee ing
phase – In ac , i is ecommended o adap he pa adigm o he necessi ies o
each SPL. To he bes o ou knowledge, he e is a lack o sys ema ic guidelines
ha p e en s he p ac i ione s om hese peculia i ies. Mo i a ed by his, we
ha e o mula ed a i s app oach o wha could be a lexible – ye s ill sys ema ic
– app oach. The app oach is based on he p inciples s a ed by [23], and comple ed
2
wi h di e en s a egies s a ed o he SPL es ing li e a u e, like mappings and
e iews [8,10,28]. Ou goal is o begin a discussion a ound he model wi h he
communi y in o de o en ich i and p o ide a solid e sion ha can be used as
a e e ence o new and senio SPL de elope s.
The emainde o his pape is s uc u ed as ollows: Sec ion 2 p esen s he
in o ma ion ega ding he so wa e de eloped o his a icle. Sec ion 3 p esen s
backg ound in o ma ion on di e en SPL heo e ical es ing app oaches. Ou
p oposed app oach is desc ibed in Sec ion 4 and de ailed in Sec ion 5. Finally
in Sec ion 6 we p esen concluding ema ks and lessons lea ned.
2 Running example
Online Shop
Ca alog Paymen Secu i y P oduc Sea ch
Bank Accoun E-coins C edi Ca d High
Low
{1..3}
C edi Ca d implies High
Manda o y
Op ional
O
Al e na i e
Fig. 1. Online shop ea u e model [29].
Figu e 1 shows he ea u e model o he online shop example ha we will
use h oughou his pape . I is a con igu able online shop sys em wi h di e en
capabili ies which a e he sys em’s a iabili y poin s, as p oposed in [29]. One
o he mos common me hods o modelling a iabili y in indus y consis in
using ea u e models [6], in which a iabili y poin s a e mapped in o ea u es
and hen ep esen ed in a hie a chical diag am ha depic s he ela ionships
be ween ea u es. The ea u e model om Figu e 1 shows ha all online shops
mus ha e a ca alog lis ing all he a ailable p oduc s, a se o paymen me hods,
and a secu i y le el. Fu he mo e, an online shop can op ionally ha e a sea ch
ea u e which allows use s o ind p oduc s mo e easily. Fu he down he hie -
a chy, we can see ha he e a e h ee possible paymen me hods, a leas one o
which needs o be selec ed: bank accoun , e-coins, and c edi ca d. No e ha he
{1..3} ca dinali y anno a ion is edundan in his case, since he o pa en -child
ela ionship equi es ha a leas one o mo e sub- ea u es a e selec ed. Finally,
3
he secu i y le el o he online shop mus be ei he high o low, since he al e -
na i e pa en -child ela ionship manda es ha exac ly one o he sub- ea u es
has o be selec ed.
In addi ion o he pa en al ela ionships be ween ea u es, ea u e models
may also ha e addi ional c oss– ee cons ain s, which a e p oposi ional o mulas
ha u he educe he amoun o alid con igu a ions. In ou example, he
“C edi Ca d implies High” cons ain makes he ea u e High o be manda o y
when he ea u e C edi Ca d is selec ed. Taking all o his in o accoun , we
can de i e a o al o 20 alid online shop a ian s. Fo mo e in o ma ion abou
ea u e modeling heo y, e e o [4,5].
O de Summa y
S a
P oduc Sea ch
Ca alog
oCa alog
addToCa
iewP oduc De ails
E-coins
E-coins
P oduc De ails
Paymen Choice
Bank Accoun
Bank Accoun
C edi Ca d
C edi Ca d
Paymen Valida ion
P oduc Selec ion
Ca Con en
Checkou
Fea u e Links
P oduc Sea ch
iewP oduc De ails
oCa alog
sea chP oduc
sea chP oduc
iewCa Con en
oCa alog
cancelO de
alidPaymen
in alidPaymen
oPaymen Choice
emo eP oduc F omCa
iewO de Summa y
selec BankAccoun
selec ECoins
selec C edi Ca d
alida ePaymen
alida ePaymen
alida ePaymen
oCa alog
Fig. 2. Online shop 150% model.
One way o modeling he beha io o a so wa e sys em is by employing a
s a e machine model. This model could also be used o gene a e es cases i
we we e using Model-Based Tes ing (MBT) echniques. In he case o SPLs, he
so-called 150% model can be buil , which is a domain enginee ing asse ha
ep esen s he beha io o he whole p oduc line. 150% models in eg a e all
he a iabili y, i.e., he a iabili y ela ed o he whole p oduc line in o one
single model [2]. When a speci ic p oduc a ian is selec ed, he a iabili y o
he 150% model is bound, o ming he 100% model (i.e., he model speci ic o
ha con igu a ion) [2]. This app oach is no only conside ed o models bu can
also be used o gene ic code. Figu e 2 shows he 150% s a e machine model o
he online shop example, whe e he links o he ea u es om he ea u e model
ha e been ep esen ed wi h colo s. No e ha in many cases he 150% model
4
i sel may no be a alid p oduc a ian , since i is no always alid o selec all
he ea u es on a single p oduc .
Analogously o he 150% s a e machine model shown, o he domain engi-
nee ing asse s can be gene a ed, such as use case diag ams, class diag ams, and
e en he so wa e sou ce code i sel . These asse s should also be linked o he
ea u e model so ha hey can be au oma ically eused o di e en p oduc s.
In o de o demons a e how o manage an SPL p ojec , we ha e implemen ed
a simple, Ja a based e sion o ou online shops example using Fea u eIDE [27],
a ool which can be used o de elop SPLs using he ea u e-o ien ed so wa e
de elopmen pa adigm.
3 Top-down s Bo om-up app oach
P oduc -Speci ic
Tes Sui es
Tes
Execu ion
Top-Down
Tes Gene a ion
P oduc
Selec ion
Fea u e model
Reusable
Componen s
Domain
Enginee ing
Asse s
Applica ion
Enginee ing
Asse s
P oduc s
Domain Enginee ing
Applica ion Enginee ing
Fig. 3. SPL es ing Top-Down app oach.
P oduc Line
Tes Sui e
P oduc -Speci ic
Tes Sui es
Tes
Execu ion
Bo om-Up
Tes Gene a ion
P oduc
Selec ion
Fea u e model
Reusable
Componen s
Domain
Enginee ing
Asse s
Applica ion
Enginee ing
Asse s
P oduc s
Domain Enginee ing
Applica ion Enginee ing
Fig. 4. SPL es ing Bo om-Up app oach.
The e a e wo main app oaches a he SPL es ing: (1) he op-down app oach
and (2) he bo om-up app oach [29], which a e also e e ed o as p oduc -
cen e ed and p oduc line-cen e ed espec i ely in some publica ions [19]. On
he one hand, he op-down app oach consis s in selec ing and gene a ing he
desi ed p oduc a ian s i s , and hen gene a ing a es cases o each de i ed
p oduc indi idually. On he o he hand, he bo om-up app oach consis s in
gene a ing a da abase o gene ic es cases o he whole SPL based on he
domain enginee ing asse s, such as he 150% model. La e , a iabili y o hese
es cases is bound o es indi idual p oduc a ian s. Figu es 3 and 4 show an
o e iew o hese app oaches.
5

oCa alog, sea chP oduc , iewP oduc De ails, sea chP oduc , oCa alog, iew-
P oduc De ails, addToCa , oCa alog, iewCa Con en , emo eP oduc F om-
Ca , oCa alog, iewCa Con en , iewO de Summa y, cancelO de , oCa a-
log, iewCa Con en , iewO de Summa y, oPaymen Choice, (selec BankAc-
coun OR selec ECoins OR selec C edi Ca d), alida ePaymen , alidPaymen
Example 1. Bo om-up es case o online shops.
We de ine a es case as a speci ica ion o inpu s used o so wa e es ing.
In his pape , hey will consis o sequences o e en s ha igge ansi ions in
ou example s a e machine model.
As an example o he bo om-up app oach, i we wan ed o gene a e a p oduc
line es sui e o ob ain a high ansi ion co e age o he online shops example,
we could come up wi h he es case shown in Example 1, whe e he colo s
ep esen he same ea u e links as in he Figu e 2 model.
oCa alog, sea chP oduc , iewP oduc De ails, sea chP oduc , oCa alog, iew-
P oduc De ails, addToCa , oCa alog, iewCa Con en , emo eP oduc F om-
Ca , oCa alog, iewCa Con en , iewO de Summa y, cancelO de , oCa -
alog, iewCa Con en , iewO de Summa y, oPaymen Choice, selec ECoins,
alida ePaymen , alidPaymen
Example 2. De i ed bo om-up es case o online shops.
A e p oduc selec ion, we can de i e his es case in o p oduc -speci ic
es cases o e e y p oduc , allowing us o euse i . No e ha his es case only
co e s one o he paymen me hods, e en i mul iple a e selec ed. I , o ins ance,
we selec ed a p oduc wi h {P oduc Sea ch, ECoins, LowSecu i y}, i s de i ed
p oduc -speci ic es case would be Example 2.
As o he op-down app oach, he p oduc -speci ic es sui e will be gene -
a ed a e selec ing he p oduc a ian s, so we can jus le e age exis ing so wa e
es ing echniques and ools.
I has been obse ed by some au ho s ha he bo om-up app oach seems
o scale be e han he op-down app oach in e ms o es execu ion cos (con-
side ing he o al es case coun , numbe o s eps and numbe o con igu a-
ions) [19]. Fu he mo e, conside ing ha he op-down app oach is simila o
he adi ional es ing sys em, his pape will be mo e ocused in he bo om-up
app oach.
4 P oposed P ocess O e iew
This p oposal is based on he s a egy Design es asse s o euse [8]. This
means ha es plans and es cases a e c ea ed as soon as possible, usually in
domain enginee ing. Ne e heless, applica ion enginee ing es s a e s ill needed,
6
so i is also impo an o encou age he euse o hose p oduc -speci ic es cases
de ined in applica ion enginee ing om one o ano he p oduc .
In e ac ion
Tes ing
P oduc
Tes ing
E olu ion
Tes ing
Tes Op imiza ion (Op ional)
Tes Execu ion
Fea u e Model Consis ency Checking
Asse
Tes ing
P oduc Line Tes Gene a ion
Domain
Enginee ing
Applica ion
Enginee ing
P oduc Line Tes Gene a ion
P oduc Line Tes Gene a ionSampling and P io i iza ion
Fig. 5. SPL es ing p ocess imeline p oposal.
Figu e 5 shows an o e iew o he ac i i ies in ol ed in he es ing p ocess o
an SPL and hei mapping in o domain o applica ion enginee ing. I is possible
o di ide he SPL es ing wo k low in ou di e en p ocedu es: Asse Tes ing,
In e ac ion Tes ing,P oduc Tes ing and E olu ion Tes ing. These p ocedu es
a e so ed in ch onological o de in Figu e 5, bu in high a iabili y en i onmen s
i is usually impossible o a oid mixing hem. One example is how in [23] i is
desc ibed how i is no necessa y o de elop and es e e y asse be o e s a ing
o es a p oduc by applying es ing p io i iza ion.
The e a e SPL es ing p ocesses ha a e a ec ed and modi ied by all o he
a o emen ioned p ocedu es. They a e ep esen ed by whi e boxes in Figu e 5,
and he a ows ep esen he dependency be ween hem. In o de o achie e an
op imal es execu ion i s ly we need o ensu e ha e e y ea u e selec ed (o
added o he SPL due o la e p oduc needs) is consis en wi h he o iginal
ea u e model – Fea u e model consis ency checking [1]. A e ha , p io i iza ion
o es s should be es ablished o e e y single p ocedu e. Mos imes, and because
o he a iabili y o he SPL, sampling echniques ha e o be applied in o de o
achie e good co e age o in e ac ion es ing be ween asse s. This is e e ed o
as Sampling and P io i iza ion.
7
Once we ha e a clea iew o which in e ac ions and p ocesses should be
p io i ized on he SPL es ing, a gene ic es plan is buil . Named as P oduc
Line Tes Gene a ion, his p ocess is hea ily in luenced by bo h domain and
applica ion enginee ing, and depends on he al eady es ed asse s. I is essen ial
o uni y and euse es asse s ( es cases, es scena ios and es esul s) as much
as possible om one p oduc o ano he . To achie e his, p oduc -speci ic c ea ed
es asse s a e s o ed in he gene ic SPL es ing plan as hey appea , as his has
been p o en o impac he e o educ ion [8].
Finally, Tes Op imiza ion can op ionally be pe o med on applica ion engi-
nee ing o achie e an op imal Tes Execu ion o e e y p ocedu e.
5 P ocedu es
In his sec ion, e e y sub-p ocess om Figu e 5 is de ine. Fu he mo e, some
glimpses abou di e en echniques e e y p ocess a e gi en.
5.1 Asse Tes ing
Asse s a e de ined as a i ac s buil o he de elopmen o di e en p oduc s o
he same SPL [3]. In he li e a u e hey a e also e e ed o as domain a i ac s
o co e asse s. These asse s can be es ed independen ly h ough uni es ing
making use o adi ional so wa e echniques [8]. Resul s om hese es s a e
aluable o e e y p oduc de i ed om a SPL. This p ocess is usually pe o med
in domain enginee ing, al hough he e a e some asse s ha canno be es ed un il
he e is a p oduc [23].
Depending on he pe spec i e o he SPL de elopmen , di e en app oaches
can be made. Acco ding o [23], i is highly ecommended o es commonali ies
i s . Commonali ies a e hose asse s conside ed co e in he SPL and ha will be
p esen in e e y p oduc . This will be use ul in case ha a e e ence applica ion
[21] wan s o be used a he in e ac ion es ing p ocedu e. Once commonali ies
a e es ed, he es ing o a iabili y-a ec ed asse s can begin. In o de o achie e
an op imal co e age o in e ac ions, he e can be a p io i iza ion o which a iable
asse s need o be es ed i s . This will allow in e ac ion es ing o s a ea lie
in he es ing p ocess.
Example 3 shows a es case o he P oduc De ails class in ou online shops
example, which checks he p esence o absence o he P oduc Sea ch op ion in
he use menu. E en i P oduc De ails is a co e asse ha is always p esen , his
pa icula es case canno be execu ed un il a p oduc is selec ed because he
ansi ion o P oduc Sea ch may o may no exis .
5.2 In e ac ion Tes ing
Asse es ing is no enough o achie ing a high quali y SPL. In SPLs, in e ac-
ion be ween asse s causes ailu es, bugs and inconsis encies ha can only be
8
1public oid es Sea chT ansi ion() {
2// Selec menu op ion
3inpu . p in ln ("2"); // 2. BACK TO CATALOG
4// Run P oduc De ails
5p oduc De ails. un();
6Lis < S ing > lines = ou pu . lines () . collec () ;
7// Check ou pu
8boolean sea chP oduc P esen = alse ;
9 o ( S ing line : lines ) {
10 i ( line . equals (" 3. SEARCH PRODUCT ")) {
11 asse False(sea chP oduc P esen );
12 sea chP oduc = ue;
13 }
14 }
15 // #i P oduc Sea ch
16 asse T ue(sea chP oduc P esen );
17 // #else
18 asse False(sea chP oduc P esen );
19 // # endi
20 }
Example 3. Asse es o he P oduc De ails class.
de ec ed when ce ain ea u e combina ions a e p esen . Fo his eason, in e ac-
ion es ing is used when es ing SPLs. Va iabili y is con olled on his s age o
SPL es ing [7], and esul s will be aluable o e e y di e en p oduc s de i ed
om he SPL. In e ac ion es ing includes in eg a ion es ing – which belongs
o domain enginee ing [8] – and also binding es ing [23], which belongs o ap-
plica ion enginee ing.
In high a iabili y scena ios i is impossible o achie e a ull co e age o e e y
in e ac ion be ween asse s. This is caused by he numbe o p oduc s an SPL can
ha e, ha g ows exponen ially as he numbe o ea u es inc eases. The e o e,
sampling is necessa y o es as many di e en in e ac ions as possible while
a oiding exhaus i e es ing [28]. The idea behind sampling is o de i e a subse
o all possible p oduc s ha collec i ely co e he beha io o he SPL and e eal
mos o he aul s by only hem [28].
To sample a p oduc subse om he en i e SPL, se e al app oaches ha e
been p oposed. Va shosaz e al., p oposed a axonomy o classi y hese ap-
p oaches [28]. This axonomy included (1) inpu da a o he sampling app oach
(e.g., ea u e model), (2) ype o algo i hm used o sampling p oduc s (e.g.,
om simple g eedy-based algo i hm o mo e sophis ica ed popula ion-based al-
go i hms) and (3) ype o co e age employed (e.g., ea u e-in e ac ion co e age).
Ou o he scope o his pape , he classi ica ion also included he e alua ion
echnique and he ype o applica ion o he app oach, among which mos o
hem we e ocused on es ing.
9