365
23RD INTERNATIONAL CONFERENCE ON INFORMATION SYSTEMS DEVELOPMENT (ISD2014 CROATIA)
Technical Tool Su eys and Compa a i e S udies:
A Sys ema ical App oach
M.J. Escalona [email protected]
IWT2 Resea ch G oup, Uni e si y o Se ille,
Se ille, Spain
J.A. Ga cía-Ga cía [email p o ec ed]
IWT2 Resea ch G oup, Uni e si y o Se ille,
Se ille, Spain
F.J. Domínguez-Mayo jdoming[email p o ec ed]
IWT2 Resea ch G oup, Uni e si y o Se ille,
Se ille, Spain
I. Ramos i am[email p o ec ed]s
IWT2 Resea ch G oup, Uni e si y o Se ille,
Se ille, Spain
Abs ac
Compa a i e s udies and su eys on di e en echnologies a e common p ac ices in esea ch
en i onmen s, no only when planning a new esea ch line, bu also in en e p ise en i onmen ,
o example, when a new p ojec is going o be de eloped and he sui able echnology mus be
selec ed. In he esea ch con ex , an o dina y and o mal me hod equen ly ollowed o analyse
he si ua ion is he Sys ema ic Li e a u e Re iew (SLR). Howe e , SLR is no enough, i he
s udy is o ien ed owa ds compa ing echnologies o ools solu ions. This pape p esen s a
mechanism o be applied sys ema ically in su eys and compa a i e s udies o ools and
echnological solu ions. I is based on gene al concep s de ined in SLR, bu i ex ends hem in
o de o co e o he necessi ies. The pape illus a es how his mechanism is applied o a eal
p ojec named THOT and i o e s conclusions and lea ned lessons om he las ends.
Keywo ds: Su eys, Compa a i e S udies, Sys ema ic Li e a u e Re iews, En e p ise Con en
Managemen .
1. In oduc ion
A Sys ema ic Li e a u e Re iew (SLR) is mainly ca ied ou in o de o ind and de elop
inno a i e ideas o u he esea ch. In [21] au ho s conside SLRs as a means o comple ing
p ocesses based on iden i ying, e alua ing and in e p e ing all a ailable documen s ocused on
pa icula esea ch ques ions o heses in a speci ic in es iga ion a ea. Howe e , his p ocess is
no only associa ed wi h scien i ic en i onmen s. In ac , i can be applied o any domain o
en i onmen (such as esea ch, en e p ise o enginee ing, among o he s) as i is no only ela ed
o esea ch wo k. In addi ion, i is no mally used as a me hod o ca ying ou compa a i e
s udies on so wa e ools o echnology p oposals.
The e o e, SLR aims o p o ide an exhaus i e summa y o li e a u e ele an o a esea ch,
echnological o echnical ques ion.
The use o SLRs is ela i ely ecen in he So wa e Enginee ing (SE) con ex , bu i has
gained signi ican impo ance in his a ea as a means o iden i y, e alua e and in e p e all
a ailable da a o answe esea ch ques ions on a pa icula opic in SE. I has been g owing in
impo ance as a sys ema ic and s uc u ed app oach ega ding li e a u e e iews since 2004,
366
ESCALONA ET AL. TECHNICAL TOOL SURVEYS AND COMPARATIVE STUDIES…
when Ba ba a Ki chenham [9] p oposed special guidelines ha we e adap ed o cope wi h
speci ic p oblems in he SE a ea. These guidelines ha e been used and e alua ed [3] [10] [11]
[18] in many con ex s. La e , in 2007, Ki chenham’s guidelines we e upda ed [10]. Las yea ,
Ki chenham’s p oposal was upda ed again [13] in o de o be implemen ed, aking in o accoun
ecen esul s published by so wa e enginee ing esea che s conce ning hei expe iences when
pe o ming SLRs, as well as hei ad ices o imp o ing he SLR p ocess.
Mo eo e , he e a e o he ideas o iews o conduc sys ema ic e iews. Fo ins ance, in [20]
[21], au ho s in oduce di e en pe spec i es o SLRs. They issue hei p oposal a e
sys ema ically selec ing and analy ically s udying a la ge numbe o pape s (SLRs) in o de o
unde s and he s a e-o - he-p ac ice o sea ch s a egies in e idence-based SE.
In consequence, hese SLRs p oposals a e highly di ec ed owa ds answe ing esea ch
ques ions on some scien i ic knowledge. Ne e heless, SLR is no enough o a s udy led o
compa e echnologies o ools solu ions.
We ha e eached his conclusion du ing he execu ion o he THOT p ojec . Such p ojec is
bo n because documen p ocesses managemen is becoming essen ial and c i ical wi hin
Andalusian Public Adminis a ion, due o e-Go e nmen is pe o ming a key ole in se ing
s a egic plans.
A p esen , he THOT p ojec is being execu ed by he Uni e si y o Se ille in liaison wi h
he AOPJA (Agencia de Ob as Públicas de la Jun a de Andalucía). This is an inno a i e p ojec
on documen managemen applied o se ice ag eemen eco ds and anspo in as uc u e
p ojec s. I aims o analyze in de ail s a egies and documen managemen sys ems (also known
as ECM1 o En e p ise Con en Managemen ), in o de o in es iga e and de ine an inno a i e
solu ion o imp o e eco ds managemen .
The e a e many easons o unde aking a SLR dealing wi h ECM solu ions and i s
applica ion o he THOT p ojec . The mos common ones a e: (i) o iden i y wha ECM sys ems
cu en ly exis in he ma ke and wha hey o e o eco ds managemen and documen al
p ocesses; (ii) o iden i y how ECM sys ems can be adap ed o he gene al guidelines o he
Andalusian Public Adminis a ion in o de o imp o e he eco ds managemen ; and (iii) o
p o ide a amewo k o app op ia ely and objec i ely compa e and acili a e decision-making
ega ding he mos sui able ECM solu ion o he THOT p ojec .
We ha e adap ed he me hod p oposed by Ki chenham o ca y ou he compa a i e s udy
among ECM solu ions. Howe e , we ha e aced some ouble o success ully apply SLR due
o ou s udy’s cha ac e is ics. We ha e also aken in o accoun o he au ho s such as Zhang [20]
[21], Da Sil a e al. [4] and Wohlin and P ikladnicki [19]. These wo ks a e summa ized in
Sec ion 2.
In his pape we sugges a mechanism ha can be used sys ema ically in su eys and
compa a i e s udies o ools and echnological solu ions. I is based on gene al concep s de ined
in SLR, bu i is ex ended o co e o he necessi ies.
The pape is s uc u ed as ollows: Sec ion 2 shows he backg ound o ou p oposal, which
is desc ibed in Sec ion 3. Finally, Sec ion 4 and Sec ion 5 p esen a success ul case in which
we ha e applied ou p oposal and some conclusions as well as u u e wo k, espec i ely.
2. Backg ound
We ha e s udied he mos ecen wo ks on SLR guidelines o lay he ounda ions o ou
p oposal, be o e desc ibing he mechanism we ecommend o be applied sys ema ically in
su eys and compa a i e s udies o ools and echnological solu ions.
Zhang and Baba conside ha SLRs a e no mally ca ied ou in o de o ind ou and
de elop inno a i e ideas o u he esea ch. A SLR is a means o comple ing p ocesses based
on iden i ying, e alua ing and in e p e ing all a ailable documen s ela ed o a pa icula hesis
1 Acco ding o AIIM, Associa ion o In o ma ion and Image Managemen , [1] ECM consis s o s a egies, me hods
and sys ems used o cap u e, manage, s o e, p ese e and deli e con en , and documen s ela ed o o ganiza ional
p ocesses. ECM sys ems and s a egies acili a e he managemen o an o ganiza ion's uns uc u ed in o ma ion,
whe e e ha in o ma ion exis s.
367
ISD2014 CROATIA
in a speci ic in es iga ion a ea. This p ocess has i s own e minology and is applied in di e en
ways. In his line, Zhang and Baba in oduce new ocabula y concep s and di e en
pe spec i es on SLRs [21].
As he SLR p ocess conce ns, Zhang e al assume ha designing and execu ing an
app op ia e and e ec i e sea ch s a egy is a key s ep. [20]. Au ho s a gue ha hese a e ime-
consuming and e o -p one ac i i ies and consequen ly, hey need o be planned and
implemen ed ca e ully. Au ho s also explain ha he e is an appa en need o a sys ema ic
app oach o designing, execu ing, and e alua ing a sui able sea ch s a egy o op imally
e ie ing he a ge li e a u e. In ligh o his, hey sugges an app oach, which consis s o a
collec ion o known s udies, and co esponding ‘quasi-sensi i i y’ in o he sea ch p ocess o
e alua ing sea ch pe o mance.. This idea was p oposed as a esul o selec ing and s udying a
wide ange o SLRs in o de o unde s and he s a e-o - he-p ac ice o sea ch s a egies in
e idence-based so wa e enginee ing.
Mo eo e , some au ho s pu o wa d ha he so wa e enginee ing esea ch communi y is
s a ing o adop SLRs consis en ly as a esea ch me hod [4]. Howe e , he majo i y o SLRs
do no e alua e p ima y s udies quali y and ail o p o ide guidelines o p ac i ione s, hus
dec easing hei po en ial impac on so wa e enginee ing p ac ice.
In [19], au ho s con i m ha he sea ch s a egy is key o ensu e a good s a ing poin o he
iden i ica ion o s udies and ul ima ely o he ac ual ou come o a pa icula s udy.
Finally, we ha e also conside ed he guidelines o he sys ema ic e iew p oposed in his
pape acco ding o he p o ocol de ined by Ki chenmham e al. [13]. These au ho s es ablish
ha a SLR essen ially in ol es h ee phases: (i) planning he e iew, which aims o decide
which me hod will be used o ca y ou he e iew as well as iden i y and o mula e he hesis
ha he sys ema ic e iew mus alida e; (ii) conduc ing he e iew, which consis s in inding
and e alua ing whe he many p ima y s udies associa ed wi h he esea ch ques ions a e
adequa e and ele an enough o be possible sou ces o u he analysis; and (iii) epo ing
he e iew, which deals wi h w i ing up he esul s o he e iew and epo ing hem o
po en ially in e es ed pa ies.
Figu e 1 shows his p ocess h ough a imeline ha indica es when each SLR phase was
applied wi hin he THOT p ojec .
Fig. 1. SLR phases
SLR is seen as a alid o mal me hod o ca y ou compa a i e s udies and su eys o
di e en ools o echnological solu ions, despi e all hese o mal p oposals. Howe e , i is no
enough since o he necessi ies mus also be conside ed. All hese easons make us ecommend
a mechanism (which is based on he well-known p oposals conce ning SLR) o be applied
sys ema ically in su eys and compa a i e s udies o ools and echnological solu ions. Sec ion
3 will u he desc ibe ou p oposal.
368
ESCALONA ET AL. TECHNICAL TOOL SURVEYS AND COMPARATIVE STUDIES…
3. Ou App oach
As i was p e iously men ioned, ca ying ou a compa a i e s udy o echnical solu ions can be
e y simila o conduc ing a SLR. Howe e , he cha ac e is ics o echnical compa a i e s udies
equi e special a en ion o some aspec s.
This sec ion desc ibes he se o phases ocused on he SLR app oach p esen ed in Figu e 1.
Then, we p opose a cha ac e iza ion scheme o e alua e di e en al e na i es o ECM sys ems,
wi h he aim o selec ing he mos sui able one o he scope and con ex o his pape . This
gua an ees quali y in he quali y con inual imp o emen p ocess o hese sys ems.
PHASE I. De ining he e iew p o ocol
In his i s phase, he eam has o plan he Re iew p o ocol o he s udy in o de o delimi he
speci ic con ex o he wo k. Fo his aim, we p opose he use o classical elici a ion echniques
such as in e iews, con ex analysis o checklis s.
Th ee impo an elemen s ha e o be in e ed om his phase:
1. Technical and Resea ch Ques ions (TRQ). They ep esen which speci ic ques ions
should he s udy answe .
2. Quali y Assessmen (QA). I de ines each cha ac e is ic ha has o be alued o each
echnical solu ion. They ha e o be aced wi h TRQ in o de o answe hem.
3. Cha ac e iza ion schema. I is he global esul o his i s phase. We p opose o de ine
a schema whe e each QA is ep esen ed. This schema has o be ins anced and comple ed
o each echnical solu ion unde s udy.
A he end o his phase, he eam has o ge a cha ac e iza ion schema simila o a check
lis , whe e each QA is ep esen ed in o de o answe each TRQ.
PHASE II. Conduc ing he e iew
In his phase h ee di e en ac i i ies mus be execu ed:
Ac i i y 1. P epa ing he sea ch. A his poin , he eam has o de ine wo di e en asks:
1. To es ablish which sea ch c i e ia will be used.
2. To es ablish which sea ch sou ces will be applied.
Acco ding o he sea ch c i e ia de ini ion, we p opose o ollow he ins uc ions e e ed in
[17]. Sea ch c i e ia ha e o be o ien ed o answe TRQ de ined in Phase I.
Wi h ega d o sea ch sou ces, i mus be s a ed ha he esea ch sea ch engines used in a
SLR, such as ACM Digi al Lib a y, IEEE Xplo e, ISI Web o Knowledge and so on, a e
ob iously alid in a echnical con ex . Howe e , i we analyze he cu en si ua ion o echnical
solu ions, we assume ha he use o o he sou ces, such as Google o communi y blogs, among
o he s, is comple ely necessa y.
Ac i i y 2. Execu ing he sea ch. The sea ch will ake place, once he mos sui able c i e ia
and sou ces a e de ined.
Including o he sea ch sou ces, like Google o blogs, can inc ease he numbe o esul s o
a g ea e ex end han only ocusing on esea ch sea ch engines. In any case, his ac i i y has o
be execu ed so many imes as equi ed in o de o delimi which echnical solu ion has o be
conside ed.
Some imes, i can be e y use ul o e-analyze he QA de ined in Phase I, because hey a e
oo ambiguous o conc e e, which is likely o p o oke wo dissimila consequences; ha a la ge
numbe o echnical solu ions o none o hem can be ound.
Ano he echnique ha can be use ul o delimi he se o echnical solu ions unde s udy is
he analysis o he unc ional con ex . We ha e o keep in mind ha ou app oach is o ien ed
owa ds compa ing echnical solu ions o a speci ic p oblem. Thus, he eam could discuss
wi h unc ional use s he se o QA and echnical solu ions ound in o de o analyse hei
sui abili y o he conc e e unc ional con ex .
Ac i i y 3. Compa a i e e alua ion o ECM sys ems. The cha ac e iza ion scheme ob ained
om Phase I has o be ins anced in each app oach, a e he echnical solu ions unde s udy
(TUS) ha e been de ined.
Howe e , we p opose ha he eam execu e ano he ac i i y be o e ins an ia ing he schema
in o de o e alua e he echnical solu ions.
369
ISD2014 CROATIA
I is usual o ind cha ac e is ics ha depend on he pe son who ca ies i ou in he echnical
e alua ion. Fo ins ance, usabili y depends on inal use s expe ise. Fo his eason, his ac i i y
should be di ided in o h ee asks:
1. To de ine e alua ion eams. S udying he se o cha ac e is ics, we p opose o de ine
he g oup o people who has o assess each o hem. Thus, e alua ion subse can be
de ined in o de o ob ain di e en poin o iews. As i is p esen ed in ou example,
o ins ance, we ha e h ee di e en e alua ion g oups, depending on hei expe ise on
ECM solu ions.
2. To ins ance he schema o each app oach in e e y e alua ion g oup. They,
independen ly, ha e o alue each cha ac e is ic in he schema o each TUS.
3. To consolida e he esul s. A e he independen alua ion, e alua ion eams mus
a ange a mee ing o analyze he indi idual esul s and ag ee a common and unique
consolida ed esul o all e alua ion g oups.
PHASE III. Repo ing
The las phase equi es he ac i e pa icipa ion o he unc ional eam. In his phase, he esul s
ob ained om phase II a e compa ed and s udied, and each TUS is analyzed in de ail in
compa ison wi h he es o TUS.
Using a cha ac e iza ion schema is essen ial o his phase because i o e s a conc e e and
simila assessmen o each app oach. The e alua ion eam can compa e e e y TUS using he
same c i e ia and ocabula y o each o hem, which helps se ing he inal conclusions.
Final epo s o he compa a i e s udy mus be issued a his phase.
4. An Example
The e iew has been conduc ed o an e ec i e cha ac e iza ion o ECM sys ems. We ha e
mul iple c i e ia o calcula e he p e e ences he cha ac e iza ion scheme elemen s demand.
Thus, i is a mul i-objec i e op imiza ion p oblem, o i is no only impo an o implemen he
mos aluable ECM sys em, bu also o educe cos , isk and ince i ude.
We will explain how he ECM sys ems ha e been analyzed and e alua ed and how he
compa a i e e alua ion has been ca ied ou .
The guidelines o he sys ema ic e iew ha e been explained in p e ious sec ion.
Ne e heless, his p oposal ini ially cen e s on he sys ema ic e iew o esea ch s udies. Fo
his aim, we ha e adap ed his p oposal o ocus on s udies o ECM sys ems and all hose ela ed
ields. A SLR essen ially in ol es h ee phases: (i) planning he e iew, (ii) conduc ing he
e iew and (iii) epo ing he e iew.
• The s ages associa ed wi h planning he e iew a e: iden i ica ion o he e iewe ’s
needs, speci ica ion o he esea ch ques ion(s), de elopmen and e alua ion o a e iew
p o ocol.
• The s ages ela ed o conduc ing he e iew a e: iden i ica ion o he esea ch, selec ion
o p ima y s udies, s udy o quali y assessmen , da a ex ac ion and moni o ing, and
da a syn hesis.
• The s ages conce ning epo ing he e iew a e: speci ica ion o dissemina ion
mechanisms, o ma ing he main epo and e alua ion o ha epo .
The planning phase has wo main goals; on he one hand, deciding on which me hod will be
used o conduc he e iew and on he o he hand, iden i ying and o mula ing he hesis ha
he sys ema ic e iew will p o e. Rega ding he i s goal, as s a ed abo e, he me hod p oposed
by Ki chenham e al. [13] will be ollowed, bu aking in conside a ion o he au ho s like Zhang
e al. [20] [21], Da Sil a e al. [4] and Wohlin and P ikladnicki [19]. Ne e heless, wi h his
sys ema ic li e a u e e iew, his wo k aims o answe he nex ques ions in ela ion o he
second goal.
• Ques ion 1.- Wha ECM sys ems cu en ly exis in he ma ke and wha do hey o e ?
• Ques ion 2.- How ECM sys ems can be adap ed o he gene al guidelines o he
Andalusian Public Adminis a ion?

370
ESCALONA ET AL. TECHNICAL TOOL SURVEYS AND COMPARATIVE STUDIES…
• Ques ion 3.- Wha is he mos app op ia e ECM sys em he Andalusian Public
Adminis a ion, and mo e speci ically, he Con ac ing Se ices o T anspo and
In as uc u e Cons uc ions mus use?
• Ques ion 4. - Wha a eas o imp o emen a e needed o he selec ed ECM sys em?
A la ge numbe o iden i ied sea ch keywo ds picked up om hese ques ions ha e been
used in he e iew p ocess. Some o hem a e: “Solu ions o Managing En e p ise Con en s”,
“ECM in Real En i onmen ”, “So wa e o Managing Documen P ocesses”. In addi ion, he
ollowing da abases ha e been conside ed in he sys ema ic li e a u e e iew: ACM Digi al
Lib a y, Ei Compendex, IEEE Xplo e, ISI Web o Knowledge, Science Di ec , SCOPUS,
Sp inge Link and Wiley In e Science Jou nal Finde .
Once all planning phase goals ha e been achie ed, he e ision p ocess en e s in he e iew
phase, which consis s in inding and e alua ing he adequacy and ele ance o many p ima y
s udies associa ed wi h he esea ch ques ions as possible sou ces o u he analysis. The
p ima y s udies will be sea ched h ough he a o emen ioned da abases by means o ou
keywo ds. Then, a s a egic plan o e alua ing he adequacy and ele ance o he s udies is
needed a e he sea ch is execu ed.
Fi s ly, keywo ds a e looked o each logical c i e ion in he sea ch ield included wi hin he
a o emen ioned da abases. Secondly, he se wi h he p e ious p ima y s udies is educed
acco ding o he ollowing inclusion c i e ia:
(a) The p ima y s udy mus ha e been published in he las ou yea s, ha is, om 2010
o 2013 (bo h included). This exclusion c i e ion is conside ed ealis ic and accep able
in he con ex o his wo k, because he numbe o ECM sys ems and e sions has
inc eased in he las yea s. The e o e, he selec ed p ima y s udy mus be ecen in o de
o in e use ul conclusions.
(b) The pape mus ocus on Compu ing Science.
(c) The pape mus ha e been published in any in luen ial magazine ( o ins ance, JCR o
Jou nal Ci a ion Repo s).
Thi dly, a new disc imina ion has been conduc ed by means o a as eading o each p ima y
s udy. Tha means, he i le heme o he p ima y s udy mus be linked o he opic o his wo k.
Fo example: «ECM», «En e p ise Con en s Managemen » o «Documen P ocess». Once his
condi ion was sa is ied and his p ima y s udy ca aloged as p omising, he in oduc ion and
abs ac mus men ion he goals o he esea ch ques ions posed in his sec ion.
Finally, a e ca ying ou his e iew, we ha e nei he ound conc e e solu ions o ECM
sys ems no s udies on hem. In con as , we ha e disco e ed some wo k associa ed wi h
heo e ical p oposals in he con ex o ECM sys ems. The e o e, classic sea ch engines and he
In e ne ha e been selec ed in o de o ca y ou a speci ic su ey o his ype o sys ems. In
addi ion, we ha e also conside ed he las Ga ne ’s analysis conce ning En e p ise Con en
Managemen . Ga ne [7] is an in e na ional esea ch and consul ing company dealing wi h
In o ma ion Technology. This s udy is popula ly known as Magic Quad an and was p esen ed
in Oc obe 2012. I consis s in e alua ing 15 weigh ed c i e ia ha a e based on hei ela i e
s eng hs in he ma ke . They depic ma ke s by using a wo-dimensional ma ix ha e alua es
endo s acco ding o hei comple eness o ision and abili y o execu e. Mo eo e , in his
second sea ch, we ha e conside ed se e al p e-selec ion c i e ia.
A e his second sea ch and in acco dance wi h Ga ne ’s Magic Quad an o ECM, we
will e alua e he ollowing sys ems (in alphabe ical o de ): Al esco [2], Documen um [5],
Nuexo/A hen o [14], IBM FileNe [8] and OpenTex [15].
Acco ding o all hese analyzed sys ems and he s a egy ha we will ollow, we p opose a
se o p e e ences o each elemen o he cha ac e iza ion scheme ha has been de ined (we
will cope wi h his de ini ion in he nex sec ion). Fo ins ance, he weigh ing es ablished by
Ga ne o basic unc ionali ies lis ed abo e is: Documen Managemen : 0.15, Documen
Imaging / Image-p ocessing applica ions: 0.18, Wo k low / BPM: 0.22, Reco ds Managemen :
0.13, Web Con en Managemen (WCM): 0.7, Social Con en / Collabo a ion: 0.15 and
In e ope abili y / ex ended componen s: 0.10.
Thus, ollowing Ga ne ’s c i e ia, his se o p e e ences will be de ined o adap he
cha ac e iza ion scheme o ou p ojec scope. Taking in o conside a ion ha we a e looking o
371
ISD2014 CROATIA
he mos comple e sys em, he es o p e e ences will be hough as impo an as each se o
elemen s.
The cha ac e iza ion scheme is unde s ood o encompass all he ele an cha ac e is ics o
analyze ECM sys ems. We ha e de ined basic cha ac e is ics ollowing he ecommenda ions
o SEG esea ch g oup de ined in [17] o es ablish an objec i e compa a i e amewo k. The
goal o his pape is o in oduce he me hodology o pe o ming igo ous e iews o exis ing
empi ical e idences in o he so wa e enginee ing communi y. The ad an ages o de ining hese
cha ac e is ics a e o p esen and e alua e schema ically and homogeneously each o he
solu ions unde s udy. Mo eo e , his cha ac e iza ion scheme allows se ing each e alua ion
c i e ia in a clea , concise and uni ied way.
Ou cha ac e iza ion scheme is composed o en p io i ized ea u es, which answe he
ques ions de ined in he p e ious sec ion. This p io i y is con ex ualized wi hin he needs o ou
p ojec : o de ine an inno a i e solu ion o documen managemen applied o p ocu emen o
se ices and anspo in as uc u e p ojec s wi hin he Regional Go e nmen o Andalusia
(Spain). Below, we illus a e his cha ac e iza ion scheme: Fea u es (FT) and hei Sub-Fea u es
(SF).
FT01: Func ional modules. The esul s ob ained in he cha ac e iza ion schema S a egy
phase poin ou ha a alid ECM sys em mus include na i ely and minimally he ollowing
unc ionali ies. Nex , we desc ibe he Sub-Fea u es o his Fea u e: SF01: Documen
Managemen , SF02: Reco ds Managemen , SF03: BPM (Wo k low/ Business P ocess
Managemen ), SF04: Documen Imaging/ Image-p ocessing applica ions, SF05:
In e ope abili y unc ions/ ex ended componen s, SF06: WCM (Web Con en Managemen )
and SF07: Social con en / collabo a ion.
FT02: Use O ien a ion. Al hough ECM sys ems o e s anda d solu ions on i s o ien a ion
owa ds he end use , many companies need using easy and e sa ile sys ems because no all
hei employees ha e he same use p o ile o handle compu e ools. Nex , he Sub-Fea u es o
his Fea u e a e desc ibed below: SF01: Usabili y compliance, SF02: Accessibili y, SF03:
Documen p e iew, SF04: D ag & D op, SF05: Bulk uploads, SF06: Undo, SF07: WYSIWYG
Edi o (Wha You See Is Wha You Ge ), SF08: Cus omiza ion, SF09: G oups and Social
ne wo ks and SF10: Mul ilanguage.
FT03: Func ionali y o cap u e, access, e ie e and iew documen s. The abili y o
cap u e, access, e ie e and display, wi hin he g oup, includes hose ea u es ha le anyone
cus omize he sys em acco ding o he p e e ences o use s o he o ganiza ion being
implemen ed. Nex , we desc ibe he Sub-Fea u es o his Fea u e: SF01: Deg ee o Ca aloguing,
SF02: Ag upa ion, SF03: Thesau us Suppo , SF04: Digi aliza ion, SF05: Bulk Upload, SF06:
Con en Gene a ion, SF07: O ice In eg a ion, SF08: Fo ms and Templa es, SF09: In eg a ion
Fo ms Manage s, SF10: Ad anced Sea ch Me hods, SF11: Sea ch Algo i hms and SF12:
Display Fo ma s.
FT04: Documen al Li e Cycle. This Fea u e enables us o assess he le el o deg ee o
suppo he sys em o e s o he documen cycle. Nex , we desc ibe he Sub-Fea u es o his
Fea u e: SF01: Check-in Check-ou , SF02: Li e cycle, SF03: Ve sioning Suppo , SF04:
Ac ions aceabili y, SF05: Inconsis encies managemen , SF06: Dissemina ion managemen ,
SF07: Conse a ion, SF08: Des uc ion and SF09: Physical ac ions.
FT05: Wo k lows. This Fea u e can assess whe he he ool suppo s managemen wi h
business p ocesses. Nex , we desc ibe he Sub-Fea u es o his Fea u e: SF01: Suppo ed
s anda ds, SF02: Managemen suppo , SF03: A ailable op ions, SF04: Mo o iza ion, SF05:
Simula ion and modeling, SF06: G aphical u ili ies and SF07: Task managemen .
FT06: eGo e nmen . Measu ing he suppo and ea u es in ol ing eGo e nmen is
essen ial, gi en he con ex in which his p ojec is de eloped. This g oup o ea u es measu es
he deg ee o suppo o e ed in his con ex . The e o e, we can e alua e whe he he sys em
allows accessing and using alid elec onic documen s acco ding o he Spanish Na ional
In e ope abili y Scheme guidelines. The Sub-Fea u es o his Fea u e a e desc ibed below:
SF01: Elec onic documen s, SF02: Digi al signa u e, SF03: Acc edi a ion and ep esen a ion,
SF04: Indexes suppo , SF05: Unique iden i ica ion, SF06: Minimum me ada a, SF07:
Classi ica ion plan suppo and SF08: O icial ime synch oniza ion.
372
ESCALONA ET AL. TECHNICAL TOOL SURVEYS AND COMPARATIVE STUDIES…
FT07: In e ope abili y Compliance. A speci ic sec ion dealing wi h in e ope abili y has
been included: In eg a ion wi h ools. This Sub-Fea u e e alua es whe he he ECM sys em
p o ides mechanisms (e.g. APIs) o in eg a e wi h hi d-pa y ools. Nex , we desc ibe he Sub-
Fea u es o his Fea u e: SF01: Connec ion, SF02: ERPs, SF03: Cap u e ools, SF04: Email,
SF05: CMIS eposi o y, SF06: Web se ices, SF07: Single window, SF08: Elec onic eco d,
SF09: Reco ds manage , SF10: Elec onic iles, SF11: Digi al signa u e, SF12: EAI and SF13:
S eaming.
FT08: Secu i y and Con ol. One o he majo objec i es o documen managemen
solu ions is o ensu e in o ma ion secu i y, by con olling access o he sys em om inside and
ou side he o ganiza ion and manage he documen s including such in o ma ion ei he o
a chi e o des uc hem. Consequen ly, hese solu ions mus p o ide se ices ha ensu e ha
he in o ma ion s o ed is secu e. I e alua es whe he he sys em is unc ional enough o analyze
da a, o o he wise, whe he he sys em allows using hi d-pa y ools. Nex , we desc ibe he
Sub-Fea u es o his Fea u e: SF01: Expo a ion, SF02: Da a Analysis, SF03: Ac i i y
indica o s, SF04: G anula i y, SF05: LOPD, SF06: Logs. SF07: SSO and SF08: No i ica ions.
FT09: A chi ec u e. Open a chi ec u e. I e alua es whe he he sys em has an open
a chi ec u e (i.e., he sys em allows adding, upg ading and changing i s componen s) o a closed
a chi ec u e (i.e., he so wa e manu ac u e chooses he componen s, and he end use does no
in end o upg ade hem). Nex , we desc ibe he Sub-Fea u es o his Fea u e: SF01: Open
a chi ec u e, SF02: B owse s, SF03: Mobili y, SF04: De elopmen ki , SF05: Cloud solu ion,
SF06: Adminis a i e capabili ies, SF07: P og amming language, SF09: Mul ipla o m, SF10:
Ex ensibili y, SF11: Volumes, SF12: High a ailabili y and SF13: Scalabili y.
FT10: Cos . Cos (bo h ini ial and long- e m by main enance) is one o he mos impo an
ac o s any o ganiza ion mus ake in o accoun when choosing an ECM solu ion. Nex , we
desc ibe he Sub-Fea u es o his Fea u e: SF01 Licenses, SF02 In as uc u e, SF03 Open
sou ce, and SF04 Main enance and suppo .
FT11: Assis ance and RM (Roadmap) Suppo . This las Fea u e lis ed in he la e g oup
includes aspec s o he e alua ion o he Cha ac e is ics suppo , assis ance and oadmap
p o ided by he ECM solu ion. Nex , we desc ibe he Sub-Fea u es o his Fea u e: SF01
Ce i ica ion p og am, SF02 Use Manuals, SF03 Se ice suppo , SF04 Fo ma ion se ice,
SF05 Roadmap and SF06 Manu ac u e online assis ance.
Along his i s i e a ion, lo s o new ends we e sensi i e o being included in he
cha ac e iza ion. In ac , we s a ed he p ojec jus conside ing Al esco [2], Documen um [5],
IBM FileNe [8] and OpenTex [15]. Then, in a second i e a ion we included Nuxeo/A hen o
[14] and KM [16]. Ne e heless, KM was disca ded in he cha ac e iza ion phase because his
sys em did no comply wi h ou p ojec scope pu poses, hus Nuxeo/A hen o was inally
conside ed.
In he THOT p ojec , we needed o decide wha ECM sys em was he mos sui able o ou
p ojec scope (Andalusian Public Adminis a ion, in Spain). As a esul , we de ined ou
cha ac e iza ion scheme and each se o p e e ences o ou cha ac e iza ion scheme elemen s.
The e o e, we used a Quali y E alua ion F amewo k (named QuEF) [6] o analyzing and
e alua ing ECM sys ems. Mo eo e , QuEF p o ides a lexible and e icien solu ion based on
a Web en i onmen , so ha o ganiza ions can choose he mos sui able ECM sys em o hei
pu poses as well as en o ce he quali y con inual imp o emen o hese sys ems in he
o ganiza ion.
In he QuEF amewo k, we used his weigh ed cha ac e iza ion scheme as a quali y model
in o de o analyze and e alua e he di e en sys ems. Thus, in his amewo k, he quali y
model was used o manage he compa a i e e alua ion o ECM sys ems, which we e analyzed
by means o checklis s. These checklis s we e a i ac s gene a ed au oma ically by he ool
suppo o QuEF, which con ained all Fea u es and Sub-Fea u es ha we de ined o analyze an
en i y. Consequen ly, we used all hese checklis s in o de o know he cu en s a e o an ECM
sys em and selec he mos sui able one.
373
ISD2014 CROATIA
5. Conclusions and Fu u e Wo k
This pape p esen s he esul s o a esea ch p ojec ha has been ca ied ou in a eal
en i onmen . The THOT p ojec is an e-Go e nmen p ojec ha aims o implemen an ECM
sys em in he Andalusian Public Adminis a ion, in Spain.
This pape ocuses on he i s s age o he p ojec and explains how he echnological and
unc ional s a us o ECM sys ems has been s udied. S a egically, i is e y impo an o
e alua e all exis ing al e na i es in he ma ke in o de o align he scope and pu pose o he
o ganiza ion. Besides, a s a ic e alua ion is no enough because new imp o emen s o exis ing
sys ems a e con inuously appea ing and i is necessa y o be able o compa e al e na i es
dynamically and objec i ely. In addi ion, we ha e o conside di e en p e e ences o he
elemen s ha con ain an ECM sys em in e ms o ou speci ic con ex .
The THOT p ojec challenges o inno a e esea ch. Al hough ou esea ch g oup ( he Web
Enginee ing and Tes ing Ea ly, IWT2) has al eady wo ked and has ma u e expe ience in many
aspec s o he p ojec , he in eg a ion o all he elemen s in o a p ojec o his size and he abili y
o alida e esea ch esul s and ans e s a ouse in e es o he use and p o ide elemen s ha
will enhance he u u e wo k o his esea ch g oup.
Finally, he p ojec ends o sol e he en i onmen al p oblem by applying p inciples o e-
Go e nmen in he public p ocu emen ac i i y, p omo ing he con e sion om pape o
elec onic media in w i en communica ions be ween manage s as well as an e icien and
e ec i e s a egy wi h he consequen socioeconomic impac .
Acknowledgemen s
This esea ch has been suppo ed by MeGUS, Minis e io de Ciencia e Inno ación (TIN2013-
46928-C3-3-R), NDTQ-F amewo k p ojec (TIC-5789), Jun a de Andalucía, Spain, and
FEDER, Eu opean Union o inancial suppo ia he p ojec “THOT. P oyec o de inno ación
de la ges ión documen al aplicada a expedien es de con a ación de se icios y ob as de
in aes uc u as de anspo e” included in o he “P og ama Ope a i o FEDER de Andalucía
2007- 2013”. We also hank all he s a and esea ches o he Public Wo ks Depa men o
Jun a de Andalucía o hei dedica ion and p o essionalism. This pape has also been suppo ed
by he Uni e sia ounda ion hanks o a g an o PhD s uden s.
Re e ences
1. AIIM: The Global Communi y o In o ma ion P o essionals (2014),
h p://www.aiim.o g/, Accessed Feb ua y 2014.
2. Al esco, h p://www.al esco.com. Accessed Decembe 14, 2013
3. B e e on, P., Ki chenham, B.A., Budgen, D., Tu ne , M., Khalil, M.: Lessons om
Applying he Sys ema ic Li e a u e Re iew P ocess Wi hin he So wa e
Enginee ing Domain. The Jou nal o Sys ems and So wa e 80(4), 571-583 (2007)
4. Da Sil a F.Q.B., San os A.L.M., Soa es S., F ança A.C.C., Mon ei o C.V.F., Maciel
F.F.: Six yea s o sys ema ic li e a u e e iews in so wa e enginee ing: An upda ed
e ia y s udy, In o ma ion and So wa e Technology 53 (9), 899-913 (2011)
5. Documen um, h p://www.emc.com/domains/documen um/index.h m, Accessed
Decembe 14, 2013
6. Domínguez-Mayo, F.J., Escalona, M. J., Mejías, M., Ross, M., S aples, G.: A Quali y
Managemen Based on he Quali y Model Li e Cycle. Compu e S anda ds & In e aces
34(4), 396-412 (2012)
7. Ga ne : Company is based in S am o d, Connec icu , USA, h p://www.ga ne .com,
Accessed July 5, 2013
8. IBM FileNe , h p://www-03.ibm.com/so wa e/p oduc s/us/en/ ilecon mana/,
Accessed Decembe 14, 2013
9. Ki chenham, B.A.: P ocedu es o Pe o ming Sys ema ic Re iews. Technical Repo
TR/SE-0401 and NICTA Technical Repo 0400011T.1, Keele Uni e si y (2004)