Quali y e alua ion o Model-D i en Web Enginee ing me hodologies
F.J. Domínguez-Mayo
a,
, M.J. Escalona
a
, M. Mejías
a
, M. Ross
b,c
, G. S aples
c
a
Depa men o Compu e Languages and Sys em, Uni e si y o Se ille, Se ille, Spain
b
Facul y o Ma i me and Technology, Sou hamp on Solen Uni e si y, Sou hamp on, Uni ed Kingdom
c
BCS So wa e Quali y Specialis G oup, Uni ed Kingdom
Keywo ds:
So wa e enginee ing
S anda ds
Me ics/measu emen
Me hodologies
Quali y analysis and e alua ion
So wa e quali y/SQA
abs ac
Con ex : The e a e lo s o app oaches o me hodologies in he Model-D i en Web Enginee ing (MDWE)
con ex o de elop Web Applica ions wi hou eaching a consensus on he use o s anda ds and sca ci y
o bo h, p ac ical expe ience and ool suppo .
Objec i e: Model-D i en Web Enginee ing (MDWE) me hodologies a e cons an ly e ol ing. Mo eo e ,
Quali y is a e y impo an ac o o iden i y wi hin a me hodology as i defines p ocesses, echniques
and a i ac s o de elop Web Applica ions. Fo his eason, when analyzing a me hodology, i is no only
necessa y o e alua e quali y, bu also o find ou how o imp o e i . The main goal o his pape is o
de elop a se o Quali y Cha ac e is ics and Sub-Cha ac e is ics o MDWE app oaches based on ISO/
IEC s anda ds.
Me hod: F om he so wa e p oduc s con ex , some widely s anda ds p oposed, such as ISO/IEC 9126 o
ISO/IEC 25000, sugges a Quali y Model o so wa e p oduc s, al hough up o now, he e a e no s anda d
me hods o assess quali y on MDWE me hodologies. Such me hodologies can be o ganized in o P ope -
ies, hus, a me hodology has a i ac s, p ocesses and echniques. Then, each i em is e alua ed h ough a
se o app op ia e Quali y Cha ac e is ics, depending on i s na u e. This pape p oposes o e alua e a
me hodology as a p oduc i sel .
Resul s: This pape ecommends a se o Quali y Cha ac e is ics and Sub-Cha ac e is ics based on hese
s anda ds in o de o e alua e MDWE me hodologies quali y. Addi ionally, i defines an agile way o
ela e hese Quali y Sub-Cha ac e is ics o P ope ies wi h he sole pu pose o no only analyzing, bu also
assessing and imp o ing MDWE me hodologies.
Conclusions: The applica ion o hese Quali y Cha ac e is ics and Sub-Cha ac e is ics could p omo e e fi-
ciency in me hodologies since his kind o assessmen enhances bo h he unde s anding o s eng hs and
weaknesses o app oaches.
1. In oduc ion
Quali y is a ele an aspec o conside in he So wa e Engi-
nee ing con ex , al hough he e a e se e al di e en defini ions, o
example, con o mance o use expec a ions, which is o en de-
sc ibed as he ‘‘fi ness o pu pose’’ o a piece o so wa e. Ano he
quali y defini ion a ending o so wa e quali y measu es deals
wi h he high quali y o so wa e design (quali y o design) and he
high le el he so wa e con o ms wi h ha design (quali y o
con o mance). The defini ion o quali y in ISO 9000:2005, as de-
sc ibed in [21], includes he no ion o ‘‘deg ee’’ meaning ha qual-
i y is no an absolu e, bu a changeable aspec . The concep o
‘‘deg ee’’ is illus a ed in Fig. 1, showing ha needs, equi emen s
and expec a ions a e
cons an ly changing. The e o e, quali y is
he di e ence be ween he model s a e implied o equi ed, and
he model s a e eached. Thus, sa is ac o y quali y akes place
when he s a e eached is wi hin he ange o accep abili y defined
by he equi ed model; supe io quali y occu s when he s a e
eached is abo e he equi ed model and in e io quali y appea s
when he s a e eached is below he equi ed s a e.
In he final e alua ion, i is he use who se s he quali y s an-
da ds by deciding which p oduc s should be pu chased and whom
hey should be pu chased. We need o exp ess ou ela i e sa is ac-
ion wi h p oduc s and, as a consequence, use subjec i e e ms.
When a p oduc sa isfies he use ’s needs, i can be said ha he
p oduc is ei he a high quali y o a sa is ac o y quali y p oduc
and in he same way, when he use is dissa isfied wi h he p od-
uc , i can be said ha i is ei he , a poo o a low quali y p oduc .
When he p oduc exceeds he use ’s needs, i is conside ed o be
ei he a high quali y o a supe io quali y p oduc , whe eas i i
is below he use ’s expec a ions, i is conside ed o be ei he a
low quali y o unsa is ac o y quali y p oduc .
The Model-D i en Enginee ing (MDE) is a so wa e de elop-
men pa adigm dealing wi h he c ea ion o models o abs ac ions
close o a pa icula domain han o concep s o specific syn ax. A
Web Enginee ing domain specific o MDE is called MDWE (Model-
D i en Web Enginee ing). A Model-D i en A chi ec u e (MDA) [37]
is an a chi ec u e pla o m app oach o de elop so wa e sys ems
unde he MDE pa adigm which p o ides a se o guidelines o
s uc u e specifica ions ha a e exp essed as models.
In ecen yea s, he g owing in e es in he In e ne has gene -
a ed a high numbe o MDWE app oaches [45] ha o e a ame
o e e ence o Web en i onmen . Nowadays, he e a e many
app oaches and di e se MDWE me hodologies such as OOHDM
(O ien ed Hype media Design Me hod) [40], UWE (UML-based
Web Enginee ing) [44], WebML (The Web Modeling Language)
[48], OOH4RIA [34], RUX-Me hod [39] o NDT (Na iga ional De el-
opmen Techniques) [36], which do no each a consensus on he
use o s anda ds, on he one hand, and show sca ci y o bo h p ac-
ical expe ience and ool suppo , on he o he . Thus, each me hod-
ology has di e en ools, such as me amodels (some o hem a e
based on o he me hodologies) o ans o ma ions, which can
implemen di e en le els o abs ac ions, o ins ance, he Com-
pu e Independen Model (CIM), he Pla o m Independen Model
(PIM) and/o he Pla o m Specific Model (PSM), among o he s. In
his si ua ion, i is necessa y o cha ac e ize hese me hodologies
in o de o analyze and e alua e hem.
In addi ion, i is impo an o know bo h, he eal needs ha
designe s ha e o co e ega ding use s o hese app oaches and
also he Quali y Cha ac e is ics hese needs should gua an ee.
Thus, designing a clea s a egy is essen ial since i will allow
designe s o e ficien ly ou line hese me hodologies. QuEF [13] is
a amewo k which manages quali y in MDWE app oaches. I is
based on quali y managemen , bu i ocuses on he Quali y Model
li ecycle. This li ecycle is composed o a se o phases (S a egy
phase, Design phase, T ansi ion phase, Ope a ion phase and Qual-
i y Con inuous Imp o emen phase) ha helps quali y manage-
men wo k e ec i ely. In QuEF, he Quali y Model is he key
elemen in quali y managemen since i desc ibes all he necessa y
elemen s ha make he au oma ic gene a ion o a i ac s educe
he es ima ed ime and cos .
The S a egy and Design phase mus s a wi h defining all
use s’ needs in gene al. Once all hese needs a e cus omized hen,
in u n, hey a e defined wi h a Quali y Model. Du ing he Ope a-
ion phase, he mos app op ia e one is selec ed and bo h use s
and designe s can analyze, con ol and e alua e he quali y o hei
app oaches, as shown in he sample applica ion included in his
pape . Use s o me hodologies need o find ou he mos app op i-
a e one o hemsel es and hei wo k en i onmen . Due o hei
expe ience, hey would p e e some aspec s a he han o he s
In any case, use s ha e he las wo d o decide and designe s
mus o e only wha use s need.
All me ics s udied in his pape ocus on de e mining he as-
pec s ha mus be included in a me hodology. Thus, in he Ope a-
ion phase, use s selec hei p e e ences by cus omizing a se o
weigh alues associa ed o he Quali y Cha ac e is ics analyzed
in his pape as well as he P ope ies con aining use s’ needs
and en i onmen al desc ip ion o app oaches in e ms o he
impo ance gi en.
This pape ocuses on de eloping a se o Quali y Cha ac e is ics
and Sub-Cha ac e is ics o MDWE app oaches based on ISO/IEC
s anda ds. I aims o p opose he bases o a se o hese Quali y
Cha ac e is ics as pa o he S a egy and Design phase o QuEF.
We also sugges how o analyze, con ol and e alua e he quali y
o MDWE app oaches as pa o he Ope a ion phase. Besides, his
pape deals wi h defining Quali y Cha ac e is ics. The idea is o
p o ide an en i onmen ha allows use s and designe s o figu e
ou which o he Quali y Cha ac e is ics ha e o be gua an eed
when pa icula use s apply hese me hodologies.
The pape is o ganized in o he ollowing sec ions: a e his
in oduc ion, Sec ion 2p esen s a global analysis o he si ua ion
and all necessa y elemen s o elabo a e his wo k acco ding o
some quali y s anda ds and con ex s. Sec ion 3p oposes quali y
concep s such as Quali y Cha ac e is ic and Quali y Sub-Cha ac e -
is ic as pa o he S a egy and Design phase in QuEF. Sec ion 4ex-
plains how all hese concep s a e ela ed o a Ma ix o Influences
(MoIs) as well as how i p o ides a se o o mulas o analyze and
e alua e MDWE me hodologies as pa o he Ope a ion phase in
QuEF. In Sec ion 5, a se o Quali y Cha ac e is ics and Quali y
Sub-Cha ac e is ics a e iden ified. Sec ion 6p o ides a sample o
he p oposed analysis and NDT me hodology assessmen . Finally,
Sec ion 7 and 8 o e a se o conclusions and con ibu ions and
sugges possible u u e wo k.
2. Wo k con ex and ela ed wo k
Few yea s ago, se e al esea ch g oups began o analyze he
cha ac e is ics o new eme ging so wa e sys ems known as hype -
media sys ems, which ha e e en ually e ol ed in o Web sys ems.
I was he bi h o a new line o So wa e Enginee ing cu en ly
known as Web Enginee ing [16]. I is a specific domain wi hin
MDE (Model-D i en Enginee ing) pa adigm whe e an applica ion
can be used [15]. The applica ion o MDE Enginee ing o Web si es
is called Model-D i en Web Enginee ing (MDWE) and, as i can be
obse ed in di e en s udies [7,45,15], i is o e ing e y good e-
sul s. Nowadays, he e a e se e al p oposals on MDWE in he li e -
a u e ha a e e y use ul o designing such applica ions. Some o
hem almos en i ely co e e e y le el o abs ac ion and hey e en
ha e ools ha suppo ans o ma ions au oma ion in de elop-
men and e alua ion p ocesses.
The e is a a ie y o p oposals in Web Enginee ing, as shown in
[45]. This ange o possibili ies and he end owa ds using MDE in
p oposals, open such a wide ange o o e s ha , in many cases, i is
di ficul o selec he mos app op ia e one. MDE was launched by
he Objec Managemen G oup (OMG). The OMG has also de el-
oped he p oposed MDA (Model-D i en A chi ec u e) ha p o ides
s anda d pla o m a chi ec u e o p oposals based on he Model-
D i en pa adigm. MDA was c ea ed wi h he idea o sepa a ing a
sys em logical specifica ion om he ope a ional de ails ha define
how he sys em uses he echnology pla o m capabili ies o be
implemen ed. In his ega d, he goals o MDA a e po abili y,
in e ope abili y and eusabili y h ough a chi ec u al sepa a ion.
The need, equi emen o expec a ion
The pe omance le el eached
Supe io quali y
Sa is ac o y
quali y
In e io quali y
Quali y Model
Time
Fig. 1. Quali y in ISO/IEC.
Besides, he concep o pla o m independence equen ly appea s
in MDA, which models he independence o he cha ac e is ics o
any echnological pla o m in CIM and PIM le el o Abs ac ion.
Among he benefi s p o ided by MDE, i can au oma e he so -
wa e de elopmen , documen a ion, code gene a ion, es ing and
p ojec managemen . This ac a o s a g owing p oduc i i y in
so wa e de elopmen and main enance, as well as ime and cos
educ ion. Finally, i also inc eases he quali y o he p oduc s ob-
ained. By applying his pa adigm, he li ecycle o a so wa e sys-
em is comple ely co e ed, anging om equi emen s cap u e o
main enance, o code gene a ion.
The e a e se e al me hodological app oaches and many com-
pa a i e s udies in he MDWE a ea. In his sense, Schwinge e al.
[45] mus be aken in o accoun . The au ho s specifically conside
modeling concep s in hei ubiqui ous na u e. They also include a
s udy ega ding he a ailable suppo o Model-D i en De elop-
men in a comp ehensi e way, using a well-defined and fine-
g ained ca alogue wi h mo e han 30 e alua ion c i e ia.
As a as quali y is conce ned, Hei lage e al. [19] discuss ha
he amoun o e o needed o main ain a so wa e sys em is e-
la ed o he echnical quali y o he sou ce code o ha sys em. Be-
sides, he ISO/IEC 9126 does no p o ide a consensual se o
measu es o es ima ing main ainabili y on he basis o he sou ce
code o a sys em. Hei lage e al. [19] iden i y a numbe o equi e-
men s o be ulfilled by a main ainabili y model in o de o be
usable in p ac ice. They also discuss hei expe iences by using
such a sys em o IT managemen consul ancy ac i i ies. Azuma
[3], on he con a y, in oduces an o e iew o cu en ISO/IEC
9126, SQuaRE and ISO/IEC 14598 se ies, ollowed by some p ob-
lems o be sol ed. Besides, Azuma explains new pa s, such as
Quali y in Use, Quali y Requi emen s and Elemen ally Me ics.
He e a e al. [20], on hei pa , aim a defining a Quali y Model
o assess Web Po als quali y. The model is based on ISO/IEC
25010 s anda d and some ela ed wo k ound in he li e a u e. Fi-
nally, Cale o e al. [7] p esen he WQM (Web Quali y Model),
which dis inguishes h ee dimensions ela ed o web ea u es, li e-
cycle p ocesses and Quali y Cha ac e is ics, and classi y he mos
ele an web me ics h ough he amewo k.
The idea o de eloping a MDE amewo k o e alua ing quali y
has been applied in se e al s udies by Mohagheghi and Dehlen
[35], whe e hey s a e ha he quali y o models is a ec ed by
he quali y o modeling languages, ools, modeling p ocesses, he
knowledge and expe ience o modele s and he quali y assu ance
echniques applied. Mo eo e , he e a e di e en p oposals con-
ce ning concep ual amewo ks o he me ics and indica o s do-
mains, such as di e en me amodels o ools like GenMETRIC [17],
a p oposal o he in eg a ed managemen o so wa e measu e-
men . GenMETRIC aims a p o iding companies wi h a gene ic
and flexible en i onmen o so wa e measu emen , by acili a ing
and se ing he bases o a common and e ec i e measu emen
p ocess, no only es ic ed o one kind o so wa e en i y, a single
quali y o an e alua ion mode. Ano he amewo k is C-INCAMI
[6], a goal-o ien ed and o ganiza ion Measu emen and E alua ion
F amewo k. C-INCAMI amewo k c ea es an on ological domain
model defining all he concep s and ela ions needed o design
and implemen measu emen and e alua ion p ocesses. I is an ap-
p oach in which he equi emen s specifica ion, measu emen and
e alua ion design, and analysis o esul s a e elabo a ed o sa is y
specific in o ma ion in a gi en con ex . This on ology also consid-
e s sou ces in e ms o WebQEM e minology such as ISO/IEC s an-
da ds, a icles and books.
Olsina and Rossi [39] explain he use o WebQEM, a quan i a i e
e alua ion s a egy o assess web si es and he quali y o applica-
ions. Defining and measu ing quali y indica o s can help s ake-
holde s unde s and and imp o e web p oduc s. An e-comme ce
case s udy illus a es he use o me hodology in sys ema ically
assessing a ibu es ha influence he quali y o he p oduc . Olsi-
na e al. [38] desc ibe mo e han a hund ed cha ac e is ics and
a ibu es o he academic si es domain in o de o analyze he
quali y equi emen ee and he way o speci y hem. These ele-
men s a e used in a quan i a i e me hodology o assessmen ,
compa ison and anking p ocesses. The p oposed web si e Quali y
E alua ion Me hodology (QEM) can be use ul o assess quali y in
di e en phases o a web p oduc li ecycle.
Pun e e al. [43] sugges he W-p ocess. I suppo s he selec-
ion and assessmen o app op ia e echniques in an e alua ion,
o example, by choosing he igh ype o me ics o measu e so -
wa e p oduc quali y. The W-p ocess is an ex ension o ISO/IEC
14598, aking he ac i i ies o he s anda d in o accoun , bu
exp essing he ela ions among hem mo e explici ly han he
s anda d does. The p ocess ca ies ou a ca ego iza ion o he
ISO/IEC 14598 ac i i ies in h ee le els, namely: Goal, Ques ion
and Me ic. In his line, he open li e a u e ypically desc ibes
GQM (Goal Ques ion Me ic) [5] in e ms o a six-s ep p ocess
whe e he fi s h ee s eps deal wi h using business goals o d i e
he iden ifica ion o he igh me ics and he las h ee s eps deal
wi h ga he ing measu emen da a and making e ec i e use o
measu emen esul s o d i e decision-making and imp o emen s.
Plösch e al. [41] p esen a me hod o sys ema ically collec
quali y equi emen s o so wa e p oduc s based on quali y mod-
els and po en ial obs acles ha may obs uc quali y goals.
In p e ious wo k [11,10], a amewo k o analyze, e alua e and
imp o e quali y o MDWE app oaches is posed bu his e sion o
QuEF was ac ually ex ended in [13] wi h se e al phases: S a egy
phase, Design phase, T ansi ion phase, Ope a ion phase and Qual-
i y Con inual Imp o emen phase. Such phases ha e been defined
oge he wi h p ocesses and a i ac s o cope wi h a comple e
Quali y Model li ecycle. In QuEF ex ension he main di e ence
among amewo ks is ha quali y managemen is ocused on he
Quali y Model li ecycle. The pu pose o QuEF ex ension is o con-
e ge on a con inuous au oma ic quali y imp o emen by means
o gene a ing Checklis s and documen a ion as well as au oma ic
e alua ions and plans in o de o con ol and be e quali y and,
in u n, au oma ically educe e o and ime.
On he one hand, his pape poin s ou some ele an de ails
when defining a se o Quali y Cha ac e is ics and Quali y Sub-
Cha ac e is ics o he quali y assessmen o MDWE app oaches
as a pa o he S a egy and Design phase o QuEF and, on he
o he , i also p oposes a way o analyze hem as a pa o he Ope -
a ion phase.
As a as Quali y s anda ds a e conce ned, ISO published [1] an
expanded e sion, con aining ISO quali y models and a consensus
decision on in en o ies o p oposed measu es o hese models.
The cu en e sion o ISO/IEC 9126 se ies o s anda ds consis s
o ou documen s:
ISO/IEC 9126-1: Quali y Models [25].
ISO/IEC TR 9126-2: Ex e nal Me ics [26].
ISO/IEC TR 9126-3: In e nal Me ics [27].
ISO/IEC TR 9126-4: Quali y in Use Me ics [28].
As a esul o In o ma ion Technology ad ances and en i on-
men al changes, a new e sion o ISO/IEC 9126 has ecen ly ap-
pea ed, known as ISO/IEC 25000:2005 [29]. I guides on he use
o he new se ies o In e na ional S anda ds named So wa e p od-
uc Quali y Requi emen s and E alua ion (SQuaRE). The pu pose o
his guidance is o p o ide an o e iew on SQuaRE con en s, com-
mon e e ence models and defini ions, and ela ions among docu-
men s. This makes use s p ope ly unde s and hese se ies o
In e na ional S anda ds acco ding o hei pu poses. The documen
explains he ansi ion p ocess om he old ISO/IEC 9126 and ISO/
IEC 14598 [31] se ies o SQuaRE, and i also gi es in o ma ion
abou using ISO/IEC 9126 and ISO/IEC 14598 se ies in hei p e i-
ous o m. The aim is o guide he so wa e p oduc de elopmen
wi h quali y equi emen s specifica ion and e alua ion. The docu-
men se c i e ia o quali y equi emen s specifica ion in so wa e
p oduc s, hei me ics and e alua ion.
SQuaRE consis s o he ollowing di isions [30]:
ISO/IEC 2500n: Quali y Managemen Di ision.
ISO/IEC 2501n: Quali y Model Di ision.
ISO/IEC 2502n: Quali y Measu emen Di ision.
ISO/IEC 2503n: Quali y Requi emen s Di ision.
ISO/IEC 2504n: Quali y E alua ion Di ision.
One o he objec i es o ISO/IEC 25000 se ies (and wha makes
hem di e en om he cu en ISO/IEC 9126 se ies) is he ha mo-
niza ion o i s con en s wi h he so wa e measu emen e minol-
ogy o ISO/IEC 15939:2007, which de e mines a measu emen
p ocess applicable o Sys em and So wa e Enginee ing and
managemen disciplines alike. The p ocess is desc ibed h ough a
model ha defines he necessa y measu emen p ocess ac i i ies
o adequa ely speci y wha measu emen in o ma ion is equi ed,
how he measu es and analysis esul s a e applied and how o con-
side hei alidi y. The measu emen p ocess is flexible, ailo able
and adap able o di e en use s’ needs. ISO/IEC 15939:2007 iden-
ifies a p ocess ha enhances defining a sui able se o measu es
add essing specific P ope ies.
O he igo ous and disciplined me hodology is Six sigma [21]
which uses da a and s a is ical analysis o measu e and imp o e
a company’s ope a ional pe o mance by iden i ying and elimina -
ing p ocess ‘‘de ec s’’. In his line, So wa e Quali y Assu ance
(SQA) is esponsible o moni o ing So wa e Enginee ing p o-
cesses and me hods used o ensu e quali y. The me hods by which
his is accomplished a e many and a ied, and may ensu e con o -
mance o one o mo e s anda ds, such as ISO 9000 o a model such
as CMMI. SQA encompasses he en i e so wa e de elopmen p o-
cess, which includes p ocesses such as equi emen s defini ion,
so wa e design, coding, sou ce code con ol, code e iews, man-
agemen change, managemen configu a ion, es ing, managemen
elease and p oduc in eg a ion. SQA is o ganized in o goals, com-
mi men s, abili ies, ac i i ies, measu emen s and e ifica ions. In
his con ex , ISO/IEC 9126 is a s anda d Quali y Cha ac e is ic e -
minology o so wa e p oduc e alua ion.
ISO/IEC 15504 [32], also known as SPICE (So wa e P ocess
Imp o emen and Capabili y de e mina ion), consis s o a se o ech-
nical s anda d documen s o he compu e so wa e de elopmen
p ocess and he ela ed business managemen unc ions. I is he e -
e ence model o he ma u i y models (dealing wi h capabili y le els
whichin u nha e odowi h hep ocessa ibu esandgene icp ac-
ices) agains which he assesso s can place he e idence collec ed
du ing hei assessmen , so ha , hey can gi e an o e all idea o he
o ganiza ion’s abili ies o deli e ing p oduc s (so wa e, sys ems,
and IT se ices). In his line, ISO/IEC 12207:2008 [33] es ablishes a
common amewo k o so wa e li ecycle p ocesses, wi h well-de-
fined e minology, ha can be e e enced by he so wa e indus y.
ISO/IEC12207:2008alsop o idesap ocess ha canbeused o defin-
ing, con olling and imp o ing so wa e li ecycle p ocesses. P o-
cesses, ac i i ies and asks o ISO/IEC 12207:2008 - ei he alone o
in conjunc ion wi h ISO/IEC 15288 – may also be applied du ing he
acquisi ion o a sys em con aining so wa e.
ISO/IEC 14598 se ies o s anda ds p o ide me hods o measu -
ing, assessing and e alua ing so wa e p oduc s quali y. They de-
sc ibe nei he me hods o e alua ing so wa e p oduc ion
p ocesses no me hods o cos p edic ion (so wa e p oduc qual-
i y measu emen s may, o cou se, be used o bo h pu poses) [21].
ISO/IEC 19796-1:2005 is an in e es ing s anda d o conside when
de eloping a amewo k o he quali y e alua ion o MDWE
me hodologies. This s anda d is ocused on e-lea ning and aims
o ha monize he di e en app oaches used a ound he wo ld o
assessing e-lea ning quali y ini ia i es. The e-lea ning ma ke
accep ance depends on he quali y o he ela ed p oduc s, se ices
and ools. A ha monized concep ion o e-lea ning quali y is a p e-
equisi e o a p ope ly unc ioning ma ke on e-lea ning p oduc s
and se ices and hei cons an quali y imp o emen .[23]
3. A quali y managemen amewo k based on he Quali y
Model Li ecycle
Gi en he high numbe o me hodologies a ailable and ecen ly
p oposed, i has become necessa y o define objec i e managemen
ools o enable o ganiza ions o imp o e hei me hodological
en i onmen and help web me hodologies designe s c ea e new
e ec i e and e ficien ools, p ocesses and echniques. Since me h-
odologies a e cons an ly e ol ing, i is no only necessa y o man-
age and imp o e quali y bu also o op imize he p ocess in o de
o educe cos s and ime. QuEF [11,10] is a quali y managemen
amewo k o Model-D i en Web me hodologies. In he p esen
wo k, a Model-D i en Web me hodology is an app oach o Web
Applica ions de elopmen based on he MDE pa adigm ha p o-
ides a se o guidelines, echniques, p ocesses and/o ools o
s uc u e specifica ions which a e exp essed as models.
QuEF was ini ially de eloped o he analysis and e alua ion o
hese kinds o me hodologies and i has ecen ly been ex ended o co -
e di e en phases o he quali y managemen . In Domínguez-Mayo
e al. [13], all he phases ha e been desc ibed oge he wi h he objec-
i es and a i ac s hey used. These phases, as shown in Fig. 2,ha e
been iden ified in a li ecycle ocusing on he Quali y Model.
Quali y Model S a egy phase: This is he ini ial phase and he
main objec i e concludes ha quali y managemen becomes
s a egically ac i e. In his phase all he s a egic asses s ha e
o be iden ified.
Quali y Model Design phase: In his phase he Quali y Model
has o be designed in e ms o all s a egic iews esul ing om
he p e ious phase. Significan ly, a design wi hin QuEF is unde -
s ood o encompass all ele an elemen s ha a e o be
designed wi hin he Quali y Model. This Quali y Model will be
used in he Ope a ion phase o he quali y managemen .
Quali y Model T ansi ion phase: The Quali y Model ansi ion
phase includes changing he Quali y Model, wi hou influencing
he ope a ion phase. This phase is e y impo an and i needs
o keep pace wi h changes in echnology, p ocesses o ech-
niques. These changes do no ha e o influence quali y manage-
men bu do ha e o be conside ed in he Ope a ion phase.
Quali y Model Ope a ion phase: In his phase he Quali y
Model is used o manage he quali y o app oaches. All a i ac s
like Checklis s, e alua ions o plans, among o he s, a e au o-
ma ically gene a ed in e ms o he Quali y Model and designed
in he Design phase. The analysis and e alua ion o quali y in
he Ope a ion phase is ca ied ou o con ol and e alua e he
cu en s a e o quali y. Finally, all esul s a e used o he con-
inuous imp o emen o quali y in he Con inual Imp o emen
phase.
Quali y Con inual Imp o emen phase: The Quali y Model can
change due o changes o ends o echnologies. In addi ion,
his phase aims o align and ealign he Quali y Model wi h
he P ope ies o be co e ed and Quali y Cha ac e is ics o be
assu ed along wi h he imp o emen o e e y phase in he
li ecycle.
Basically, his way o manage quali y consis s o defining a
Quali y Model con aining all pas , p esen and u u e equi e-
men s, expec a ions, goals o ends o each. The idea is o cen al-
ize all e o s and cos s conce ning quali y managemen in a un-
ique model. The e o e, he e o is educed no only by
cen alizing he model bu also by gene a ing all necessa y a i-
ac s like Checklis s, Imp o emen plans o Cha s, among o he s,
by means o a Model-D i en Quali y.
QuEF can be applied o any domain ha may define goals and
equi emen s o each. The benefi s o ROI a e cos s and ime
educ ion when imp o ing he domain unde s udy. Besides,
knowledge can be sha ed by s akeholde s o he same domain. An-
o he ad an age lies in ou s a egic map o managing quali y e fi-
cien ly. Despi e he ini ial e o o implemen QuEF, i he defined
s a egy is clea and well defined, e o is a good in es men since
i inc eases quali y con ol, as e quali y imp o emen and
compe i i eness.
The li ecycle deals wi h he se o phases pe o med o ca y ou
quali y managemen . In his li ecycle, pas , p esen and u u e
iews o he Quali y Model elemen s in he domain unde s udy
mus be conside ed in o de o ge quali y managemen in an e fi-
cien way. This phase ep esen s a s a egic asses ha ocuses on
defining a quali y managemen s a egy. Mo eo e , he Quali y
Model S a egy phase o QuEF is he key issue in quali y manage-
men . The Quali y Model is finally designed in he QuEF Quali y
Model Design phase, whe e all s a egic ac i es om he p e ious
phase a e aken in o conside a ion. In ac , his Quali y Model will
be he one used o Quali y analysis, con ol and e alua ion in he
nex Ope a ion phase.
The Quali y Model Ope a ion phase is used o ca y ou he
Quali y managemen . Thus, he Analysis and e alua ion manage-
men p ocesses a e pe o med wi hin his phase which means ha
we can know he cu en s a e o he domain unde s udy. I he
domain o con ex changes due o he appea ance o new ends,
hen he Quali y Model needs o be changed in he Quali y Model
T ansi ion phase wi hou a ec ing he Ope a ion phase. Finally, he
Quali y Con inual Imp o emen phase execu es all p ocesses o
imp o e quali y in all he li ecycle p ocesses as well as in he Qual-
i y Model i sel .
The use o QuEF ocuses on medium-long e m quali y manage-
men . Addi ionally, a sho - e m use o QuEF o Quali y manage-
men is also possible, bu conside ing some aspec s. Pa icula ly,
in a sho - e m quali y managemen , Checklis s a e used o assu e
quali y (mee ing he Quali y Model, o example, in e ms o
equi emen s o objec i es, among o he s) a he han o analyze
he cu en s a e domain unde s udy. On he con a y, i a med-
ium-long e m Quali y managemen is conside ed, hen, he
Checklis s de e mine he cu en s a e a he han ensu ing com-
pliance wi h he Quali y Model. In his sense, despi e pe o ming
a sho o a medium-long e m quali y managemen , in oducing
QuEF always equi es a g ea ini ial e o ewa ded wi h he a o e-
men ioned benefi s.
In he S a egy phase o QuEF, some o he main necessa y p o-
cesses o be ca ied ou a e, among o he s, he P ope y Po olio
Managemen , he Quali y Cha ac e is ic Po olio Managemen
and he Quali y Model Po olio Managemen . Di e en po olios
a e ob ained as a esul o hese p ocesses:
The Po olio o P ope ies.
The Po olio o Quali y Cha ac e is ics.
The Po olio o Quali y Model.
These po olios a e used as s a egic asse s o ob ain he final
P ope ies, he Quali y Cha ac e is ics and he Quali y Model in
Fig. 2. Quali y model li ecycle in QuEF amewo k.

he Design phase. When he Po olios a e ac i e and ha e pas ,
p esen and u u e in o ma ion, he Design phase uses such in o -
ma ion o ou line he models.
Bo h P ope ies and Quali y Cha ac e is ics ha e o be desc ibed
and s uc u ed in a wo-le elled hie a chy which ep esen s a
highe and lowe le el o g anula i y o an app oach. P ope ies
a e used o desc ibe and ca ego ize he p oposals in wo le els
(Fea u e and Sub-Fea u e) and hey indica e he deg ee o which
a me hodology is implemen ed. Fo ins ance, he MDE Fea u e
may ha e some Sub-Fea u es such as, Language Defini ion, T ans-
o ma ions and T ace Gene a ion.
On he con a y, Quali y Cha ac e is ics and Quali y Sub-Cha -
ac e is ics ep esen he opic o his wo k. They need o be cla i-
fied, bu hey a e quali y aspec s influenced by an en i onmen
desc ip ion o P ope ies. In o he wo ds, a Quali y Cha ac e is ic
is a highe -le el quali y aspec . In a hie a chy o Quali y Cha ac e -
is ics, highe -le el a ibu es a e called Quali y Cha ac e is ics and
lowe -le el a ibu es a e called Quali y Sub-Cha ac e is ics. Fo
example, Quali y Cha ac e is ics could be Usabili y, Func ionali y
o Po abili y. Usabili y, o ins ance, is defined by di e en Quali y
Sub-Cha ac e is ics such as Lea nabili y, Unde s andabili y o
Ope abili y. Fig. 3 shows he ela ion among he aspec s high-
ligh ed in his pape . A Quali y Cha ac e is ic has a se o Quali y
Sub-Cha ac e is ics and a Fea u e has a se o Sub-Fea u es.
3.1. A S a egy phase based on a Sys ema ic Li e a u e Re iew
In QuEF, he Technological Wa ch Managemen is a p ocess o he
Quali y Model S a egy phase ocused on he obse a ion o ends
and echnologies on he ise. I also explo es he s a e o he a o
he domain. Due o he defini ion o P ope ies (whe e CheckLis s
a e defined) and Quali y Cha ac e is ics in ol es a la ge ini ial e o
his askismadebyca yingou aSys ema icLi e a u eRe iew(SLR).
The e a e many easons o unde aking a SLR such as he iden-
ifica ion o concep s and gaps o MDWE me hodologies quali y
managemen . The esul s o his p ocess a e used o cap u e in o -
ma ion o P ope ies and Quali y Cha ac e is ics Po olios.
The e iew me hod is based on he esea ch p o ocol and i is in
his sec ion whe e sou ces, s udies selec ion and selec ion execu-
ion a e defined.
Sou ces Selec ion: The objec i e o his sec ion is o selec he
sou ces whe e sea ches o p ima y s udies will be execu ed.
Fi s o all, we ca ied ou p elimina y sea ches aimed a
iden i ying exis ing sys ema ic e iews and assessing he ol-
ume o po en ially ele an s udies. In his sense, we ound
he ollowing ele an ini ia i es: Schwinge e al., 2008, Meliá
e al., 2008, P eciado e al., 2008, Escalona and Koch 2004 o
Escalona and A agón, 2008, among o he s.
The p oposed s udy by Schwinge e al. was a good basis o his
amewo k, al hough i is no su ficien because he en i on-
men does no only aim a assessing app oaches, bu managing
quali y as a whole. In addi ion, in a good s a egy, he P ope ies
ha a e shown in he s udy a e no su ficien since hey mus be
conside ed in a quali y con inuous imp o emen ega ding
pas , p esen and u u e ends. This a o s a good quali y
design and leads o a quali y managemen based on quali y
con inuous imp o emen .
A e ha , a Web-sea ch was pe o med o find o he ele an
and new concep s ela ed o ou P ope ies and Quali y Cha ac-
e is ics. Se e al se s o keywo ds we e used by combining he
concep s o ou s udy. Some examples a e: ‘‘Model-D i en’’,
‘‘Web’’, ‘‘de elopmen ’’, ‘‘me hodology’’, ‘‘me hodologies’’,
‘‘app oach’’, ‘‘app oaches’’, ‘‘CIM’’, ‘‘PIM’’, ‘‘SPM’’, ‘‘Compu e
Independen Model’’, ‘‘Pla o m Independen Model’’, ‘‘Specific
Pla o m Model’’, Code’’, ‘‘Le el o abs ac ion’’, ‘‘Tool Suppo ’’,
‘‘e alua ion’’, ‘‘assessmen ’’, ‘‘Quali y’’, ‘‘ISO 9126’’, ‘‘ISO 25000’’,
‘‘CMMI’’ o ‘‘ITIL’’, among o he s. The las s ep consis ed in look-
ing o e e ences o pape s included in he p e ious e iews in
o de o iden i y mo e concep s.
S udies selec ion: Then he selec ion c i e ia o e alua e s ud-
ies sou ces we e: he a ailabili y o consul a icles on he In e -
ne o on he digi al lib a y o he Uni e si y o Se ille, which
has e-books and also access o o he esou ces like Google Scho-
la , Scopus, Mendeley, Science Di ec , ISI Web o Knowledge,
ACM Digi al Lib a y, Ci eSee X o he IEEE digi al lib a y; a ail-
abili y o sea ch mechanism h ough keywo ds; and li e a u e
om companies, books, jou nals and con e ences published by
expe s in he field.
As he e a e plen y o wo k published, ou sea ch s a ed
excluding hose issued p io o 2006 because we we e in e -
es ed in cu en wo k. Then we analyzed which o hem co e ed
ou domain, and all s udies dealing wi h he Model-D i en Web
de elopmen en i onmen s domain we e included. The p ocess
o conside a pape was as ollows: Fi s , he i le was consid-
e ed, hen he Abs ac , Keywo ds and Con en and finally he
Conclusions o he analysis.
Selec ion execu ion: Then, he p ocedu es o s udies selec ion
we e applied o ob ain a icles in o de o e i y i he s udies fi
he inclusion and exclusion c i e ia. Finally, a ending o he
Quali y
Cha ac e is ic
Quali y
Sub-Cha ac e is ic
Fea u e Sub-Fea u e P ope y
1..*
1..*
1..*
1..*
1..*
P ope ies
Quali y Cha ac e is ics Quali y Model
Fig. 3. Quali y model me amodel.
esul s o he SLR, a se o P ope ies desc ibing he MDWE
me hodologies we e iden ified and classified in o Fea u es and
Sub-Fea u es in ela ion o wo k and he cu en li e a u e.
In addi ion, a e applying he SLR, a MDWE me hodology was
e alua ed as a p oduc i sel hus, a se o Quali y Cha ac e is-
ics based on he ISO S anda d was defined. A me hodology
esembles a so wa e p oduc in ha i defines a se o me hods,
ools o echniques ha con o ms he In e nal Quali y. Mo e-
o e , hese me hodologies a e es ablished in a wo k con ex
o en i onmen and use s beha e in a di e en manne ega d-
ing hese P ope ies depending on he wo k con ex (Ex e nal
Quali y). Finally, we e alua ed he e ec s o applying he
me hodology in a wo k con ex (Quali y in Use).
Then, hese P ope ies and Quali y Cha ac e is ics ha e o be
ela ed o his en i onmen o indica e he influence o each
Sub-Fea u e on each Quali y Sub-Cha ac e is ic. Thus, e e y Sub-
Fea u e is ela ed o e e y Quali y Sub-Cha ac e is ics so as o
poin ou i s influence.
Tha is o say ha , i designe s ha e o imp o e me hodologies,
hey only need o de elop all phases o QuEF s a ing wi h he
S a egy and Design phase in o de o define a fi s Quali y Model.
The designe can ei he sha e he Quali y Model wi h o he design-
e s o use a Quali y Model eached on consensus, ins ead. QuEF
could also be applied in o he a eas and domains such as he
implemen a ion o a ecognized s anda d in a company.
The implemen a ion can be execu ed by defining a Quali y
Model wi h he necessa y asks o be pe o med and implemen ed
wi hin he phases o QuEF o co e all he Quali y Model li ecycle.
In con as o he imma u e domain o MDWE me hodologies, a
s anda d model is a ma u e domain since i is ag eed, s able and
does no su e many changes o e ime. Ne e heless, QuEF-TS
can gene a e all a i ac s o pe o m he s anda d model in he
company.
4. Analysis and e alua ion o Quali y Cha ac e is ics
Quali y Cha ac e is ics and P ope ies a e defined in he S a -
egy and Design phases o QuEF and hei ele ance has o be de e -
mined in e ms o use ’s needs, equi emen s and expec a ions.
Al hough many me hodology designe s may ha e di e en iew-
poin s, due o hei expe ience, in ela ion o he impo ance o
P ope ies and Quali y Cha ac e is ics, me hodology designe s
ha e o iden i y hese needs.
We can e alua e hose Quali y Cha ac e is ics and P ope ies as
elemen s o MDWE me hodology om ei he a gene al o a specific
g anula i y espec i ely. Each Quali y Cha ac e is ic has a se o
Quali y Sub-Cha ac e is ics, howe e , in P ope ies, a Fea u e has
a se o Sub-Fea u es con aining P ope ies. This g anula i y de-
pends on he le el in which he e alua ion alue o MDWE me hod-
ology is ob ained. Fo ins ance, i we assess Quali y Cha ac e is ics
in gene al, we will ob ain alues o Usabili y, Func ionali y, Po a-
bili y, Reliabili y and Main ainabili y. Fo specific g anula i y, o
ins ance, pa icula ly o he Usabili y Quali y Cha ac e is ics, we
can ge e alua ion alues o Lea nabili y, Unde s andabili y, Sim-
plici y, In e p e abili y, Ope abili y and A ac i eness.
Fo ha eason, he e a e wo le els o Quali y Cha ac e is ics
and P ope ies which ha e, in u n, been di ided in o wo le els:
fi s ly, Quali y Cha ac e is ics and Sub-Cha ac e is ics o Quali y
Cha ac e is ics and, secondly Fea u es and Sub-Fea u es o P op-
e ies. In he same way, me hodology designe s could ha e di e -
en iewpoin s abou he influence o P ope ies on Quali y
Cha ac e is ics.
When he domain is imma u e o no comple ely ag eed, he
S a egy and Design phases a e essen ial as designe s ha e o
iden i y P ope ies demanded by use s in a me hodology and he
Quali y Cha ac e is ics hese P ope ies ha e o ollow o be o e ed
o use s. Once he P ope ies and Quali y Cha ac e is ics ha e been
desc ibed in a fi s e sion o he Quali y Model, use s and design-
e s ha e o define he impo ance o each elemen and he ela ion-
ship among hem i.e. how Quali y Cha ac e is ics a e influenced by
P ope ies. I is e y di ficul o define he impo ance o P ope ies
and Quali y Cha ac e is ics in an imma u e domain. A consensus
decision should be eached and delimi ed in he S a egy and De-
sign phases howe e , i he domain is oo imma u e, i is e y di -
ficul o each his consensus. The eason lies in he ac ha his
kind o domain is le o he Ope a ion phase o be sol ed. Then,
use s and designe s can define he impo ance o each elemen
in he Ope a ion phase, he Quali y Analysis managemen and
he Quali y E alua ion managemen . The e o e, once use s and
designe s ha e eached a consensus decision on he S a egy and
Design phases, i is hen fixed in he Quali y Model design. In hese
lines, he Quali y Model Managemen is a p ocess ca ied ou in he
Design phase by defining he cu en Quali y Model as well as he
associa ion links be ween P ope ies and Quali y Cha ac e is ics.
Ano he impo an a i ac o define and ha p o ides he asso-
cia ion links be ween he cu en P ope ies and Quali y Cha ac e -
is ics is he MoIs (Ma ix o Influences). Table 1 shows he
associa ion links be ween Quali y Sub-Cha ac e is ics and Sub-
Fea u es de e mined by a ma ix named MoI.
In his ma ix, elemen s o he Quali y Model a e o ganized in
ows and columns; P ope ies (Hie a chical in Fea u es and Sub-
Fea u es) a e lis ed in ows and Quali y Cha ac e is ics (Hie a chi-
cal in Quali y Cha ac e is ics and Sub-Cha ac e is ics) a e ep e-
sen ed in columns. In he MoI, he Sub-Fea u es alue is
calcula ed by he Checklis s con aining P ope ies (Hie a chical in
Fea u es and Sub-Fea u es) used o ob ain he me hodology alue.
A quali a i e alue can de e mine he deg ee o influence on
each ela ion modeled by a cell. In his case, he deg ee o influence
is an associa ion link be ween Sub-Fea u es and Sub-Cha ac e is-
ics, ega dless o he deg ee o impo ance. Thus, he MoI ep e-
sen s he deg ee o influence o each Sub-Fea u e
i;j
i
on each
Quali y Sub-Cha ac e is ic q
k;l
k
. This is a quali a i e alue ha has
o be ans o med in a quan i a i e alue
½
i;ji
;q
k;lk

. Fo example,
wo alue scales o he influence deg ee can only be defined as:
Influence (p) o No Influence (). Ano he possibili y could be se -
ing ou di e en g ades: High Influence (p"), Mode a e Influence
(p), Low Influence (p;) o No Influence ().
Some o mulas ha e been defined in o de o calcula e he al-
ues de e mining he me hodology s a e eached o Quali y Cha -
ac e is ics and Sub-Cha ac e is ics. These Quali y Cha ac e is ics
and Sub-Cha ac e is ics alues a e calcula ed in e ms o Fea u es
and Sub-Fea u es alues ha a e cu en ly being s udied in o he
Table 1
Ma ix o Influences (MOIs) be ween p ope ies (Hie a chical by Sub-Fea u es and
Fea u es) and quali y cha ac e is ics (Hie a chical by Quali y Cha ac e is ics and Sub-
Cha ac e is ics).
Q
1
Q
2
... Q
k
q
11
... q
1;l
1
q
2,1
... q
2;l
2
... q
k,1
... q
k;l
k
F
1
1,1
...
1;j
1
F
2
2,1
...
½
i;ji
;q
k;lk

2;j
2
... ...
F
i
i,1
...
i;j
i
wo k. The Quali y Sub-Cha ac e is ic alues s
q
k;lk
a e calcula ed
acco ding o o mula (1), which conside s each Sub-Fea u e alue
i;ji
o a Fea u e F
i
. This o mula calcula es he Quali y Sub-Cha ac-
e is ic alue in e ms o hose Sub-Fea u es ha influence he
Quali y Sub-Cha ac e is ic. This is defined by an associa ion link
ha is a quali a i e alue in he MoI, which is, hen, ans o med
in o a quan i a i e alue
½
i;ji
;q
k;lk

. Besides, he impo ance o each
Sub-Fea u e in he se o Sub-Fea u es is conside ed by he Sub-
Fea u e weigh alue w
i;ji
.
s
q
k;lk
¼P
n
j¼1
½
i;ji;qk;lk
i;ji
w
i;ji
nð1Þ
s
q
k;lk
Z. I is a Quali y Sub-Cha ac e is ic alue. I is be ween
ð06s
q
k;lk
61Þbecause his alue ac ually ep esen s whe he
he s a e eached by his alue sa isfies o no he equi ed
model ( alue = 1).

i;ji
Z. I is a Sub-Fea u e alue be ween ð06
i;ji
61Þsince his
alue is ac ually a deg ee ep esen ing whe he he s a e
eached by his alue sa isfies o no he equi ed model
( alue = 1).

½
i;ji
;q
k;lk

Z. I is he deg ee o influence alue o he ela ion alue
be ween he Sub-Fea u e
i;j
i
and he Quali y Sub-Cha ac e is ic
q
k;l
k
. In he MoI, i is a quali a i e alue ha has o be changed
in o a quan i a i e alue be ween ð06
½
i;ji
;q
k;lk

61Þ. I depends
on he alue scale defined indica ing he influence.
w
i;ji
Z. I is he weigh alue o he Sub-Fea u e
i;j
i
. I is be ween
ð06w
i;ji
61Þand ep esen s he impo ance o he Sub-Fea-
u e in he se o Sub-Fea u es. The sum o his alue o he
se o Sub-Fea u es o he same Fea u e has o be 1.
nZ. I is he numbe o Sub-Fea u e alues o a Fea u e F
i
whose
deg ee o influence alue
½
i;ji
;q
k;lk

is no 0.
Ne e heless, i can be gene alized and he Quali y Sub-Cha ac-
e is ic alue s
q
k;lk
can be calcula ed in e ms o a Fea u e F
i
o , on
he con a y, conside ed as a combina ion o a se o Fea u es.
Fo ins ance, hese wo Fea u es a e conside ed F
1
and F
2
,as
indica ed in o mula (2).
s
q
k;lk
¼P
n
1
j¼1
½
1;j1;qk;l1
1;j1
w
1;j1
þP
n
2
j¼1
½
2;j2;qk;lk
2;j2
w
2;j2
n
1
þn
2
ð2Þ
In o mula (2), n
1
and n
2
a e he numbe s o Sub-Fea u e alues o
each Fea u e F
1
and F
2
whose weigh alue
½
1;j1
;q
k;lk

and
½
2;j2
;q
k;lk

espec i ely, a e no 0.
In o he wo ds, i can be gene alized o his o he o mula (3)
whe e a Quali y Sub-Cha ac e is ic alue s
q
k;lk
is calcula ed in e ms
o di e en Fea u es F
1
,F
2
, ..., F
selec ed.
s
q
k;lk
¼P
n
1
j¼1
½
1;j1;qk;l1
1;j1
w
1;j1
þ...þP
n
j¼1
½
;j ;qk;lk
;j
w
;j
n
1
þ...þn
ð3Þ
In a simila manne o o mula 2, n
1
, ..., n
ep esen s he numbe o
Sub-Fea u e alues o each Fea u e F
1
, ..., F
whose deg ee o influ-
ence alues
½
1;j1
;q
k;lk

;...;
½
;j
;q
k;lk

espec i ely, a e no 0.
Finally, e e y Quali y Cha ac e is ic alue s
Q
k
is calcula ed by
conside ing each Quali y Sub-Cha ac e is ic q
k;l
k
and he necessa y
deg ee o influence i wi h a weigh alue. In he ollowing o mula
(4),w
q
k;lk
ep esen s he weigh alue ha indica es he influence o
each Quali y Sub-Cha ac e is ic alue q
k;l
k
on he Quali y Cha ac-
e is ic alue s
q
k;lk
associa ed.
s
Q
k
¼P
m
l¼1
w
q
k;lk
s
q
k;lk
mð4Þ
s
Q
k
. I is a Quali y Cha ac e is ic alue. I is be ween ð06
s
Q
k
61Þbecause his alue ac ually ep esen s a deg ee show-
ing whe he he s a e eached by his alue sa isfies o no
he equi ed model ( alue = 1).
w
q
k;lk
. I is a Quali y Sub-Cha ac e is ic weigh alue. I is
be ween ð06w
q
k;lk
61Þand ep esen s he impo ance o Qual-
i y Sub-Cha ac e is ic in he se o Quali y Sub-Cha ac e is ics.
The sum o his alue o he se o Quali y Sub-Cha ac e is ics
o he same Quali y Cha ac e is ic has o be 1.
s
q
k;lk
. I is a Quali y Sub-Cha ac e is ic alue. I is be ween
ð06s
q
k;lk
61Þbecause his alue is ac ually a deg ee ep esen -
ing whe he he s a e eached by his alue sa isfies o no he
equi ed model ( alue = 1).
m. I is he numbe o Quali y Sub-Cha ac e is ic alues o a Fea-
u e F
i
whose weigh alue w
q
k;lk
is no 0.
A o mula ion is gi en in o de o define cha ep esen a ions o
a se o alues. The main goal o his kind o o mula ion is o cla i y
he esul s showed in he cha s since he e a e se e al al e na i es
o ca y ou an analysis om di e en poin s o iew. Fo ins ance,
we can ge ei he a gene al iew o Fea u es wi h e e y Fea u e a-
lue o a se o Sub-Fea u es alue o a specific Fea u e. Besides, we
can ha e a gene al iew abou Quali y Cha ac e is ics wi h e e y
Quali y Cha ac e is ic alue o a se o Quali y Sub-Cha ac e is ics
alue o a specific Quali y Cha ac e is ic, depending on di e en
Fea u es o jus one. Thus, his las ep esen a ion case has o indi-
ca e he Sub-Fea u es o Sub-Fea u e conside ed calcula ing his
Quali y Sub-Cha ac e is ics. The no a ion is explained below:
R
F
. I ep esen s he se o Fea u e alues
F
1
;
F
2
;...;
F
n
gin a
gene al iew. All Fea u es a e conside ed wi h his e minology.
R
F
i
. I ep esen s a se o Sub-Fea u e alues
i;1i
;
i;2i
;...;
i;ji
go
a Fea u e F
i
.
In he same way, in o de o ep esen Quali y Sub-Cha ac e is-
ic alues o a Quali y Cha ac e is ic, he e m Q
k
is equi alen o
he e m q
k;l
k
.
S
QjF
Z. I ep esen s a se o Quali y Cha ac e is ic alues
s
Q
1
;s
Q
2
;...;s
Q
m
g o each Quali y Cha ac e is ic in a gene al
iew, acco ding o he Sub-Fea u e alues
i;1i
;
i;2i
;...;
i;ji
go
each Fea u e F
i
. All Fea u es a e conside ed.
S
Q
k
jF
. I ep esen s a se o Quali y Sub-Cha ac e is ic alues
s
q
k;1k
;s
q
k;2k
;...;s
q
k;lk
go a Quali y Cha ac e is ic Q
K
in e ms o
Sub-Fea u e alues
i;1i
;
i;2i
;...;
i;ji
go each Fea u e F
i
.
S
Q
k
jF
1
;F
2
;...;F
n
. I ep esen s a se o Quali y Sub-Cha ac e is ic al-
ues s
q
k;1k
;s
q
k;2k
;...;s
q
k;lk
go a Quali y Cha ac e is ic Q
K
acco ding
o a se o Sub-Fea u e alues
i;1i
;
i;2i
;...;
i;ji
go he Fea u es
F
i
conside ed.
S
Q
k
jFF
. I ep esen s a se o Quali y Sub-Cha ac e is ic alues
s
q
k;1k
;s
q
k;2k
;...;s
q
k;lk
go a Quali y Cha ac e is ic Q
K
in e ms o
he Sub-Fea u e alues
i;1i
;
i;2i
;...;
i;ji
go each Fea u e F
i
,
wi hou conside ing he Sub-Fea u e alues o he Fea u e F
.
This MoI could be used o op imize he cos s o implemen ing
se e al new Fea u es and Sub-Fea u es in a me hodology. Design-
e s could p edic he cos s o implemen ing a new Fea u e in hei
me hodologies. Fo ins ance, hey can decide o implemen he
minimal numbe o Fea u es and Sub-Fea u es which ha e influ-
ence on a majo numbe o Quali y Cha ac e is ics and Quali y
Sub-Cha ac e is ics.
5. Iden i ying Quali y Cha ac e is ics and Quali y Sub-
Cha ac e is ics o MDWE
In QuEF, he Technological Wa ch Managemen is a p ocess o
he S a egy phase used o he obse a ion o ends and echnol-
ogies on he ise. I also assesses he s a e o he a o he domain.
The esul s o his p ocess a e used o cap u e in o ma ion o he
Po olios. In hese lines, he Po olio o Quali y Cha ac e is ics is
an a i ac o he Quali y Model S a egy phase ha defines all
impo an aspec s ha designe s ha e o gua an ee o use s o
hese me hodologies. Use s, on hei pa , ha e o iden i y he
impo ance o hese elemen s in he u u e, bea ing in mind he e-
mo al o inse ion o mo e Quali y Cha ac e is ics, depending on
he cu en ends cap u ed by he Technological Wa ch
Managemen .
As a as Quali y Cha ac e is ics a e conce ned, he me hodol-
ogy can be e alua ed as a p oduc i sel . Fo ins ance, we can e al-
ua e ei he Usabili y o Func ionali y o a me hodology. Depending
on i s na u e, each i em is e alua ed h ough a se o app op ia e
Quali y Cha ac e is ics. As he e is no s anda d glossa y o MDWE
e minology, i is necessa y o s anda dize he e minology o im-
p o e he access channel o communica ion in MDWE.
This pape p esen s a se o Quali y Cha ac e is ic and Quali y
Sub-Cha ac e is ic based on ISO/IEC 25000. Consequen ly, me hod-
ologies a e e alua ed as p oduc s. To his end, a se o Quali y Cha -
ac e is ics, based on cu en li e a u e such as ISO/IEC 9126, ISO/IEC
25000 (SQuaRE), IEEE [22] and o he s anda ds adap ed o MDWE
me hodologies a e being iden ified and hie a chically classified.
ISO/IEC 2501n s anda d (Quali y Model Di ision) ha con o ms his
di ision p esen s a de ailed quali y model including cha ac e is ics
o in e nal and ex e nal quali y in use. Fu he mo e, he in e nal
and ex e nal so wa e quali y cha ac e is ics a e o ganized in o
sub-cha ac e is ics. P ac ical guidance on he use o he quali y
model is also p o ided. Quali y Cha ac e is ics a e classified acco d-
ing o ISO/IEC ISO/IEC 2501n in o h ee di e en g oups:
Ex e nal Quali y, which measu es he so wa e (ISO/IEC 2501n).
In e nal Quali y, which measu es he sys em beha io (ISO/IEC
2501n).
Quali y in Use, which measu es he e ec o using he so wa e
in a specific con ex (ISO/IEC 2501n).
Thus, he Quali y Cha ac e is ics o MDWE me hodologies
could be classified in o wo ca ego ies:
Ex e nal/In e nal Quali y, which measu es he app oach ea-
u es and hei beha io . They a e Usabili y, Func ionali y, Reli-
abili y, Main ainabili y and Po abili y.
Quali y in Use, which measu es he e ec o using he app oach
ea u es in a specific con ex . They a e E ec i eness, P oduc i -
i y and Sa is ac ion.
As Fig. 4 shows, bo h ca ego ies ha e influence on each o he .
Mo eo e , he Quali y in Use ca ego y o a Web Applica ion de-
pends on he Web Applica ion Ex e nal and In e nal Quali y
ca ego y.
We ha e selec ed he abo e Quali y Cha ac e is ics because we
conside ha a me hodology could be e alua ed as a p oduc i sel .
A me hodology esembles a so wa e p oduc in ha i defines a se
o me hods, ools o echniques ha con o ms In e nal Quali y. On
he con a y, hese me hodologies a e implan ed in a wo k con ex
o en i onmen and use s beha e in a di e en manne in ela ion
o all hese P ope ies depending on he wo k con ex (Ex e nal
Quali y). Finally, we can e alua e he e ec s o applying he me h-
odology in a wo king con ex (Quali y in Use). The e o e, in spi e o
using ISO/IEC 9126 o ISO/IEC 25000, Quali y Cha ac e is ics wo k
as a basis adap ed o he e alua ion o me hodologies. I mus be
no ed ha we do no alk abou so wa e p oduc , bu MDWE
me hodologies, ins ead.
In his line, he use o a me hodology ensu es ou pu consis-
ency, al hough i does no gua an ee Web applica ion quali y. This
is due o he ac ha using a me hodology does no ensu e Web
Applica ion quali y, bu a uni o mi y o esul s. This ac may e en
ins i u ionalize he c ea ion o inapp op ia e Web Applica ions.
Thus, in u u e wo k, applica ion ea u es will be ela ed o me h-
odology ea u es in o de o define how each me hodology ea u e
influences each Web Applica ion ea u e. Fo his eason, we a e
wo king on he analysis o e ficiency and p oduc i i y wi h he
aim o ela ing hese me hodologies me hods o Web Applica ions.
Howe e , he fi s s ep mus deal wi h cha ac e izing hese me h-
odologies o know wha P ope ies mus be o e ed o use s and
wha Quali y Cha ac e is ics mus be gua an eed acco ding o
use s’ needs. In his sense, i is essen ial o desc ibe he P ope ies
o Web Applica ion and he ela ion be ween P ope ies o me h-
odologies and P ope ies o Web Applica ions in he esul s. They
mus also be ela ed o P ope ies o me hodologies, as i is neces-
sa y o know wha P ope ies o me hodologies ha e influence on
P ope ies o Web Applica ions.
In his pape , each Quali y Cha ac e is ic in ISO/IEC 25000 is de-
sc ibed in ela ion o a so wa e p oduc , bu , in his case, all Qual-
i y Cha ac e is ics and Quali y Sub-Cha ac e is ic a e desc ibed in
ela ion o app oach cha ac e is ics as ollows:
Usabili y: In ISO/IEC 25000, i is a Quali y Cha ac e is ic defined
as: ‘‘The capabili y o he so wa e o be unde s ood, lea ned, used
and be a ac i e o he use when used unde specific condi ions’’.
This defini ion could be adap ed o mo e closely fi ou specific do-
main as: ‘‘The capabili y o an app oach ea u e o be unde s ood,
lea ned, used and a ac i e o use when applied in specific condi-
ions’’ o in a gene al way as: ‘‘A se o a ibu es ha bea s influ-
ence on he e o needed o use, and on he indi idual assessmen
o such an use, by a s a ed o implied se o use s’’.
A se o Quali y Sub-Cha ac e is ics is iden ified o e e y Qual-
i y Cha ac e is ic. Quali y Sub-Cha ac e is ics associa ed o each
Quali y Cha ac e is ic o Usabili y a e simila ly desc ibed by adap -
ing o he defini ions om ISO/IEC, IEEE, o he s anda ds and wo k
al eady published. Quali y Sub-Cha ac e is ics o Usabili y a e de-
sc ibed as:
Lea nabili y. The capabili y o an app oach ea u e o enable
he use o lea n how o use i . [Adap ed om ISO/IEC 25000].
Unde s andabili y/ Comp ehensibili y. The capabili y o being
unde s ood and he ex en o which an app oach ea u e is
clea , wi hou ambigui y, and easily comp ehensible. [Adap ed
om ISO/IEC 25000].
Simplici y. The deg ee o which an app oach ea u e has a
s aigh o wa d design easily unde s ood. [Adap ed om IEEE].
MDWE
Me hodology
Ex e nal and In e nal
Quali y
MDWE
Me hodology
Quali yinUse
Web Applica ion
Ex e nal and In e nal
Quali y
Web Applica ion
Quali y in Use
Depends on
Depends on
In luences on
In luences on
Depends on
In luences on
Fig. 4. Influences and dependencies be ween he me hodology and he p oduc .
This de elopmen occu s because NDT me hodology imple-
men s hose Sub-Fea u es o Tool Suppo ha a ec hese Quali y
Sub-Cha ac e is ics. Ne e heless, alues 2 a e close o alues 1
and he eached s a e is no he equi ed model. Thus, he design-
e s could highly imp o e NDT me hodology. I he Tool Suppo
Cha ac e is ic we e no conside ed (Fig. 7), i would be obse ed
ha his Fea u e would ge lowe alues o Main ainabili y. The
cause could be ha no all ea u es defined by NDT me hodology
a e implemen ed by he Tool suppo . In o he wo ds, he Tool Sup-
po Fea u e includes aspec s ha NDT me hodology does no
implemen . Fo his eason, he esul s a e lowe wi hou conside -
ing his Fea u e. In his sense, he amewo k could be used o p e-
dic ing and simula ing di e en si ua ions and designe s could
decide wha ea u es should be implemen ed, a ending o he
analyzed esul s. This is an impo an aspec , as hey can op imize
he implemen a ion cos o hose ea u es ha highly a ec he
Quali y Cha ac e is ic ha mus be fi s ly implemen ed.
Gene ally, NDT me hodology has sensi i ely bes sco es in
Usabili y, being an app oach easy o unde s and, lea n, use and
a ac i e o use s. Addi ionally, i also has bes sco es in Main ain-
abili y, being an easy app oach o be modified when co ec ing de-
ec s, mee ing new equi emen s, making u u e main enance
easie o coping wi h a changeable en i onmen .
Besides, i shows good pe o mance in Func ionali y, Reliabili y
andPo abili y,al houghi islesssensi i e oQuali yCha ac e is ics
such as Usabili y and Main ainabili y. Finally, i could be imp o ed
in all Quali y Cha ac e is ics due o he equi ed Quali y Model.
7. Fu u e wo k
Al hough Mic oso Excel was used o implemen he sample
applica ion, we a e cu en ly wo king on QuEF-TS, a gene al ool
suppo o implemen ing he QuEF amewo k. This ool suppo
is composed o di e en ools ha a e ocused on each phase o
QuEF (S a egy phase, Design phase, T ansi ion phase, Ope a ion
phase and Quali y Con inuous Imp o emen phase):
QuEF-S: I is he ool ha suppo s he Quali y Model S a egy
phase. (I is being de eloped).
QuEF-D: I is he ool ha suppo s he Quali y Model Design
phase. (I is no de eloped ye ).
QuEF-T: I is he ool ha suppo s he Quali y Model T ansi ion
phase. (I is no de eloped ye ).
QuEF-O: I is he ool ha suppo s he Quali y Model Ope a ion
phase. (I is being de eloped).
QuEF-QCI: I is he ool ha suppo s he Quali y Con inual
Imp o emen in quali y managemen . (I is no de eloped ye ).
QuEF-TS can gene a e all a i ac s ( o ins ance he Checklis s
and MoIs) in e ms o he Quali y Model specified in he S a egy
phase and Design phase wi h QuEF-S and QuEF-D ools. Thus, an
impo an issue is o define Quali y Model o be used in he Ope -
a ion phase wi h QuEF-O, which con ains specific Checklis s and all
he necessa y elemen s based on he Quali y Model o be analyzed,
con oled and e alua ed o his ype o me hodologies. I some
changes ha e o be applied in he Quali y Model, he QuEF-T is
he ool o be used o applying he T ansi ion phase wi hou
a ec ing he quali y managemen in he Ope a ion phase. Finally,
QuEF-QCI is a ool ha suppo s he Quali y Con inuous Imp o e-
men phase o QuEF. Using his ool suppo could help imp o e
e ficiency no only in me hodologies, bu also in o he con ex s,
since his app oach enhances he unde s anding o s eng hs and
weaknesses in di e en con ex s. In his sense, we a e wo king
o apply QuEF-TS in o he con ex s. The specific con ex s ha e only
o be defined in he Quali y Model. QuEF-TS gene a es Checklis s
and MoIs o s a wi h he Ope a ion phase (analysis, con ol,
and e alua ion).
As a u u e wo k, we a e going o use QuEF-TS o wo k wi h
o he con ex s like CMMI [8] and s anda ds like ISO/IEC 15504
and ISO/IEC 12207. This use o he ool would help companies
implemen and manage hei capabili y and ma u i y. Ano he spe-
cific con ex o apply QuEF is he G een IT aspec s o sus ainabili y
managemen and cos s educ ion in companies as well as he
implemen a ion o di e en agile so wa e de elopmen me hod-
ologies like XP (Ex emme P og amming), Sc um o RUP (Ra ional
Unified P ocess).
Nowadays, we a e also wo king in he S a egy and Design
phase wi h specific s a egic models o es ablish some c i e ia o
a me hodology designe s g oup and each a consensus on he de -
ini ion o se e al aspec s, such as he P ope ies hie a chy o Qual-
i y Cha ac e is ics, despi e he impo ance o hese elemen s and
he influence o P ope ies on Quali y Cha ac e is ics. Thus, hese
me hods could help define he weigh s o P ope ies, Quali y
Cha ac e is ics and Quali y Sub-Cha ac e is ics. Fu he mo e, his
p ocess would help each a consensus on he defini ion o Sub-Fea-
u es influence on Quali y Sub-Cha ac e is ics.
In his line, a fi s p o o ype o QuEF-S ( ool suppo o he
S a egy phase) o a fi s s a is ical s udy is being cu en ly de el-
oped o ga he he gene al opinion o MDWE expe s, designe s
and use s. We a e using a uzzy g oup analy ical hie a chy p ocess
(AHP) [44] app oach o manage his ask. In hese s udies, a se o
web pages o a fi s s a is ical s udy is being cu en ly de eloped
o know he opinion o MDWE expe s, designe s and use s. In his
s a is ical s udy, designe s and use s ha e o se he necessa y
weigh o conside a Quali y Cha ac e is ic, Quali y Sub-Cha ac e -
is ic, Fea u e and ele an Sub-Fea u e. The weigh would indica e
he deg ee o impo ance.
Ano he objec i e is ha QuEF-TS is connec ed o well known
social ne wo ks like Facebook [17] o Twi e [46] o make he
S a egy and Design phase easie and each he eal quali y man-
agemen objec i es. The use o social ne wo ks does no only in-
ol e he numbe o people ha al eady use hem, bu i en ails
he ac ha hese me hodologies a e ca ied ou by people and
hese ype o sys ems make easy he communica ion among hem.
Besides, lo s o me hods in he S a egy phase o QuEF include s a-
is ical me hods ocused on s akeholde s in o de o de ec he eal
needs and demand as well as he quali y aspec s ha ha e o be
gua an eed. Social ne wo ks, in pa icula Facebook and Twi e ,
o e a comple e API ha pe mi de elope s o imp o e he connec-
i i y o s akeholde s in o de o be e he communica ion and
se ing s a is ical me hods o he S a egy phase and he final de-
sign o he Quali y Model in he Design phase o QuEF. We hink
ha ha ing access o social ne wo ks esul s c ucial o execu e
an e ec i e quali y managemen in he u u e.
8. Conclusions
A se o Quali y Cha ac e is ics and Quali y Sub-Cha ac e is ics
ha e been p oposed acco ding o ISO/IEC 9126, SQuaRE (ISO/IEC
25000), IEEE and o he s anda ds which ha e been adap ed o
MDWE s anda d in o de o manage quali y in MDWE me hodolo-
Table 10
Quali y Cha ac e is ic alues in e ms o all Fea u es ( alues 1) and all Fea u es excep
Tool Suppo ( alues 2).
Quali y Cha ac e is ic name Values 1 Values 2
Usabili y 3/4 2/3;
Func ionali y 2/3 3/5;
Po abili y 2/3 5/8;
Reliabili y 2/3 3/5;
Main ainabili y 5/7 7/8"

gies. Al hough he e a e some wo k o con e ge owa ds a com-
mon me amodel o achie e he in e ope abili y o hese me hodol-
ogies, his kind o me hodologies has di e en me amodels and
models, ans o ma ions and ool suppo ha wo k in a di e en
way. E en hough use s a e he design a ge o hese app oaches,
his si ua ion does no le hem know how o use hese app oaches
since hey do no know how hese app oaches can help hem and
which o he app oaches is mo e sui able o hem.
In addi ion, he di e si y in some ea u es o he design o hese
app oaches eflec s ha designe s o hese app oaches do no ha e
a clea idea o use s’ needs and he Quali y Cha ac e is ics hey ha e
o gua an ee hese needs. The e o e, designe s ha e a wo old ask;
o iden i y he eal P ope ies ha co e use s’ needs and heQuali y
Cha ac e is ics ha use s demand. The con e gence owa ds a s an-
da diza ion o hese me hods oge he wi h a consensus decision on
quali y managemen and a common Quali y Model would esul in
ha bo h, use s and designe s, apply and design hese me hodolo-
gies e ficien ly. This would be possible because hey would ha e a
common s a egy, he same objec i es and pu poses as well as he
same design o he model o be achie ed. A he same ime and
due o hei sha ed knowledge, designe s would ha e a clea e idea
o he changes ha should be pe o med in he Quali y Model, in
case o changes in echnologies o ends while designing hese
me hodologies. Use s will ha e he chance o compa ing me hodol-
ogies, and designe s, on hei pa , will ha e he chance o con ol-
ling and imp o ing he design o he me hodologies.
In his line, i is necessa y o s anda dize he e minology in o -
de o imp o e he access channel o communica ion in MDWE.
Fu he mo e, a b ie example shows how hese Quali y Cha ac e -
is ics (Hie a chical by Quali y Cha ac e is ics and Sub-Cha ac e is-
ics) a e ela ed o P ope ies (Hie a chical by Fea u es and Sub-
Fea u es). A se o o mulas has been sugges ed in o de o analyze
quali y in MDWE me hodologies. Addi ionally, we ha e p oposed
some o mulas o e alua e quali y in e ms o di e en P ope ies
and define a Ma ix o Influences (MoIs). As his ma ix de e mines
he influence o each Sub-Fea u e on each Quali y Sub-Cha ac e is-
ics, designe s a e able o guess he cos s o implemen ing a new
Fea u e on hei me hodology wi h i . In u u e wo k, he Quali y
in Use is a quali y aspec ha has o be adap ed o e alua e MDWE
me hodologies in he use con ex .
Rega ding he con ibu ions ob ained om his esea ch wo k,
i can be s a ed ha an en i onmen is equi ed ei he o imp o e
he cu en MDWE app oaches o de elop new MDWE me hodol-
ogies. Despi e he ac ha he domain is imma u e, we conside
ha using his amewo k o manage quali y would enhance he
design and me hods o hese MDWE app oaches as a consequence
o a clea s a egy ocused on quali y con inuous imp o emen o
he Quali y Model.
Ano he impo an aspec is ha he use o a me hodology en-
su es ou pu consis ency. Howe e , i does no gua an ee Web
Applica ion quali y, bu uni o mi y o esul s. Thus, in u u e wo k,
applica ion ea u es will be ela ed o me hodology ea u es in o -
de o define how each me hodology ea u e influences each Web
Applica ion Fea u e. Fo his eason, we a e wo king on analyzing
e ficiency and p oduc i i y wi h he aim o ela ing hese me hod-
ologies app oaches o Web applica ions. Ne e heless, he fi s s ep
mus consis in cha ac e izing hese me hodologies o iden i y
P ope ies ha mus be o e ed o use s and Quali y Cha ac e is ics
ha mus be gua an eed acco ding o use s’ needs.
To conclude, we conside essen ial o sa is y use ’s demands
and expec a ions. This aspec can be obse ed in he Quali y Model
and designe s pe ec ly know he aspec s ha should be consid-
e ed. The e o e, i use s ulfil hei equi emen s, he esul s ob-
ained (Web applica ions) will p obably imp o e, bu always
depending on he use hey make o P ope ies. This could be in e -
es ing because i hese ela ions a e defined, i will be possible o
guess he P ope ies o a me hodology ha need o ob ain a se
o equi emen s in a Web Applica ion. E en designe s will op imize
he implemen a ions o Fea u es and Sub-Fea u es in hei me h-
odologies acco ding o he esul s hey can ge . NDT me hodology
has been analyzed and e alua ed o he Func ionali y Quali y
Cha ac e is ic in a b ie example abou he use o Quali y Cha ac-
e is ics and Sub-Cha ac e is ics. We ha e checked ha his en i e
en i onmen could be also use ul o ei he simula ing di e en
si ua ions o making decisions on P ope ies designe s mus imple-
men in me hodologies in e ms o he analyzed esul s.
Acknowledgemen s
This esea ch has been suppo ed by he P ojec QSimTes
(TIN2007-67843-C06_03) and by he Temp os P ojec o he Minis-
e io de Educación y Ciencia, (TIN2010-20057-C03-02), Spain.
Re e ences
[1] A. Ab an, R.E. Al-Qu aish, J. Desha nais, Ha moniza ion issues in he upda ing
o he ISO s anda ds on so wa e p oduc quali y, Me ics News Jou nal 10(2)
(2005) 35–44, (O o- on-Gue icke Uni e si y o Magdebu g, Ge many,
Decembe , (ISSN: 1431–8008)).
[2] M. Azuma, SQuaRE: he nex gene a ion o he ISO/IEC 9126 and 14598
in e na ional s anda ds se ies on so wa e p oduc quali y, ESCOM (Eu opean
So wa e Con ol and Me ics con e ence), 2001, pp. 337–346.
[3] Alce . Fede ación Nacional de Asociaciones pa a la lucha con a las
en e medades enales. <www.alce .o g> ( e ie ed 04.11).
[4] V. Basili, H.D. Rombach, The TAME p ojec : owa ds imp o emen -o ien ed
so wa e en i onmen s, IEEE T ansac ions on So wa e Enginee ing 14 (6)
(1989) 758–773.
[5] P. Becke , L. Olsina, Towa ds suppo p ocesses o web p ojec s, in:
P oceedings o he 10 h In e na ional Con e ence on Cu en T ends in Web
Enginee ing, Sp inge -Ve lag, 2010, pp. 102–113.
[6] C. Cale o, J. Ruiz, M. Pia ini, Classi ying Web Me ics Using he Web Quali y
Model. 29 (3) (2005) 227–248.
[7] C.P. Team, CMMI o De elopmen , Ve sion 1.2, P e ace, So wa e Enginee ing
Ins i u e, Ca negie Mellon Uni e si y, Augus , 2006.
[8] Conseje ía de Cul u a. <h p://www.jun adeandalucia.es/o ganismos/
cul u a.h ml> ( e ie ed 04.11).
[9] F.J. Domínguez-Mayo, M.J. Escalona, M. Mejías, I. Ramos, L.F. Sanz, A Quali y
E alua ion F amewo k o MDWE Me hodologies, in: P oceedings o he 18 h
In e na ional Con e ence on So w ae Quali y Managemen (SQM), The B i ish
Compu e Socie y, London, UK, 2010, pp. 171–184, (ISBN: 978-0-9557300-8-
5).
[10] F.J. Domínguez-Mayo, M.J. Escalona, M. Mejías. Quali y Issues on Model-D i en
Web Enginee ing Me hodologies, In o ma ion Sys ems De elopmen Sp inge
Ve lag, 2011, pp. 295–306, (ISBN: 10.1007/978-1-4419-7355-9_25).
[11] F.J. Dominguez-Mayo, M. Espinilla, M.J. Escalona, M. Mejias, A Weigh ed
Quali y E alua ion F amewo k by applying he Analy ic Hie a chy P ocess,
Nine een h In e na ional Con e ence on So wa e Quali y Managemen , Global
Quali y Issues, 2011, pp. 55–66 (ISBN 978 1 907382 44 4).
[12] F.J. Domínguez-Mayo, M.J. Escalona, M. Mejías, M. Ross, G. S aples, A Quali y
Managemen Based on he Quali y Model Li e Cycle, Compu e S anda ds and
In e aces, Janua y 2012, doi: 10.1016/j.csi.2012.01.004 (ISSN 0920–5489).
[13] Emasesa. <www.aguasdese illa.com> ( e ie ed 04.11).
[14] M.J. Escalona, G. A agón, NDT. A model-d i en app oach o web equi emen s,
IEEE T ansac ions on So wa e Enginee ing 34 (3) (2008) 377–390.
[15] M.J. Escalona, N. Koch, Requi emen s enginee ing o web applica ions – a
compa a i e s udy, Jou nal o Web Enginee ing 2 (3) (2004) 193–212.
[16] Facebook o De elope s. <h p://de elope s. acebook.com> ( e ie ed 04.11).
[17] I. Hei lage , T. Kuipe s, J. Visse , A P ac ical Model o Measu ing
Main ainabili y, in: 6 h In e na ional Con e ence on he Quali y o
In o ma ion and Communica ions Technology (QUATIC 2007), 2007.
[18] M. He e a, M.Á. Mo aga, I. Caballe o, C. Cale o, Quali y in Use Model o Web
Po als (QiUWeP), in: P oceedings o he 10 h In e na ional Con e ence on
Cu en T ends in Web Enginee ing, Sp inge -Ve lag, 2010, pp. 91–101.
[19] D. Hoyle, ISO 9000 Quali y Sys ems Handbook – Upda ed o he ISO
9001:2008 S anda d, Using he s anda ds as a amewo k o business
imp o emen , Else ie , 2009 (ISBN: 978-1-85617-684-2).
[20] IEEE S d 610.12-1990. IEEE S anda d Glossa y o So wa e Enginee ing
Te minology.
[21] ISO/IEC IS 9126, So wa e P oduc E alua ion – Quali y Cha ac e is ics and
Guidelines o Thei Use, Gene a, Swi ze land: In e na ional O ganiza ion o
S anda diza ion, 1991.
[22] ISO/IEC 9126–1: So wa e Enginee ing – P oduc Quali y – Pa 1: Quali y
Model, Gene a, Swi ze land: In e na ional O ganiza ion o S anda diza ion,
2001.
[23] ISO/IEC TR 9126–2: So wa e Enginee ing – P oduc Quali y – Pa 2: Ex e nal
Me ics, Gene a, Swi ze land: In e na ional O ganiza ion o S anda diza ion,
2003.
[24] ISO/IEC TR 9126–3: So wa e Enginee ing – P oduc Quali y – Pa 3: In e nal
Me ics, Gene a, Swi ze land: In e na ional O ganiza ion o S anda diza ion,
2003.
[25] ISO/IEC TR 9126–4: So wa e Enginee ing – P oduc Quali y – Pa 4: Quali y in
Use Me ics, Gene a, Swi ze land: In e na ional O ganiza ion o
S anda diza ion, 2004.
[26] ISO/IEC 25000:2005. So wa e Enginee ing – So wa e p oduc Quali y
Requi emen s and E alua ion (SQuaRE) – Guide o SQuaRE.
[27] SC7, ISO/IEC DTR 25021: So wa e Enginee ing – So wa e P oduc Quali y
Requi emen s and E alua ion (SQuaRE) – Quali y Measu e Elemen s ISO/IEC
JTC1/SC7 WG6, Janua y 11, 2006, 6N–565, 2006.
[28] ISO/IEC 14598–1:1999. In o ma ion echnology – So wa e p oduc e alua ion
– Pa 1: Gene al o e iew.
[29] ISO/IEC 15504–1:2004 In o ma ion echnology – P ocess assessmen – Pa 1:
Concep s and ocabula y.
[30] ISO/IEC 12207:2008 Sys ems and so wa e enginee ing – So wa e li e cycle
p ocesses.
[31] S. Meliá, J. Gómez, S. Pé ez, O. Díaz, A Model-D i en De elopmen o GWT-
Based Rich In e ne Applica ions wi h OOH4RIA, in: ICWE ‘08 P oceedings o
he 8 h In e na ional Con e ence on Web, Enginee ing, 2008.
[32] P. Mohagheghi, V. Dehlen, De eloping a quali y amewo k o model-d i en
enginee ing, Models in So wa e Enginee ing: Wo kshops and Symposia a
MoDELS 2007 (2008) 275–286.
[33] NDT (Na iga ional De elopmen Techniques). <h p://www.iw 2.o g/en/
nd .php> ( e ie ed 03.11).
[34] OMG: MDA. <h p://www.omg.o g/mda/> ( e ie ed 01.10).
[35] L. Olsina, G. La uen e, G. Rossi, Speci ying Quali y Cha ac e is ics and
A ibu es o Websi es, Web Enginee ing (2001) 266–278.
[36] L. Olsina, G. Rossi, Applica ion Quali y wi h WebQEM, 2002, pp. 20–29.
[37] OOHDM (O ien ed Hype media Design Me hod). <h p://www-di.in .puc-
io.b /schwabe/HT96WWW/sec ion1.h ml> ( e ie ed 03.11).
[38] R. Plösch, A. May , C. Kö ne , Collec ing Quali y Requi emen s Using Quali y
Models and Goals, in: In e na ional Con e ence on he Quali y o In o ma ion
and Communica ions Technology, IEEE, 2010, pp. 198–203.
[39] J.C. P eciado, M. Linaje, R. Mo ales-Chapa o, F. Sanchez-Figue oa, G. Zhang, C.
K oib, N. Koch, Designing Rich In e ne Applica ions Combining UWE and
RUX-Me hod, in: ICWE ‘08 P oceedings o he 2008 8 h In e na ional
Con e ence on Web Enginee ing, 2008 (ISBN: 978-0-7695-3261-5).
[40] T. Pun e , R.O.B. Kus e s, J.O.S. T ienekens, T. Bemelmans, A. B ombache , The
W-P ocess o So wa e P oduc E alua ion: A Me hod o Goal-O ien ed
Implemen a ion o he ISO 14598 S anda d, Re iew Li e a u e And A s O The
Ame icas, 2004, pp. 137–158.
[41] T.L. Saa y, In oduc ion o a modeling o social decision p ocess, Ma hema ics
and Compu e s in Simula ion 25 (1983) 105–107.
[42] W. Schwinge , W. Re schi zegge , A. Schaue hube , G. Kappel, M. Wimme , B.
P öll, C. Cache o Cas o, S. Cas eleyn, O. De T oye , P. F a e nali, I. Ga igos, F.
Ga zo o, A. Ginige, G-J. Houben, N. Koch, N. Mo eno, O. Pas o , P. Paolini, V.
Pelechano Fe agud, G. Rossi, D. Schwabe, M. Tisi, A. Vallecillo, an de Sluijs,
G. Zhang, A su ey on web modeling app oaches o ubiqui ous web
applica ions, In e na ional Jou nal o Web In o ma ion Sys ems 4 (3) (2008)
234–305.
[43] Twi e o de elope s. <h p://de . wi e .com> ( e ie ed 04.11).
[44] UWE (UML-Based Web Enginee ing). <h p://uwe.ps .ifi.lmu.de> ( e ie ed
03.11).
[45] WebML (The Web Modeling Language). <h p://www.Webml.o g> ( e ie ed
03.11).