A data-driven approach for a project management methodology for R&D Projects.

Adie azpen G a ikoa e a Ingenia i zako P oiek uak Saila
Depa amen o de Exp esi´on G ´a ica y P oyec os de Ingenie ´ıa
A da a-d i en app oach o a p ojec managemen
me hodology o R&D P ojec s
by
Leona do Sas oque Pinilla
Supe ised by Ne ea Toledo Ganda ias and No be o L´opez de Lacalle
Disse a ion submi ed o he Depa men o G aphic Design and Enginee ing P ojec s o he
Uni e si y o he Basque Coun y (UPV/EHU) as pa ial ul ilmen o he equi emen s o he PhD
deg ee in P ojec Enginee ing
Bilbao, Decembe 2022
(cc)2023 EDWAR LEONARDO SASTOQUE PINILLA (cc by-nc-nd 4.0)
”Caminan e, son us huellas el camino, y nada m´as;
Caminan e, no hay camino: se hace camino al anda .
Al anda se hace el camino, y al ol e la is a a ´as
se e la senda que nunca se ha de ol e a pisa ,
Caminan e, no hay camino, sino es elas en la ma .”
An onio Machado
III
Acknowledgmen s.
Glo ia, I s ill emembe he nigh i all happened, he music playing in he backg ound
on one o he ew elec ical appliances we had. Si ing on he loo by he window and
hugging each o he , while wa ching how Bogo ´a was gi ing us one o i s magical nigh s,
whe e he cloud o pollu ion ga e way o a s a y nigh and a ed moon a dusk. 1
I emembe you s o y, you d eams, you ques ion; I emembe e e y single second
ha passed while I was hinking, e en hough I had al eady made up my mind om he
momen you s o y began. I emembe he emo ion, I emembe he ea , I emembe
he lo e, I will ne e o ge he lo e, and a his momen , I can’ help bu eel p oud
o you, o me and o he decision we made because o ha momen on ou li e changed
comple ely. So ha gi you ga e me changed my whole li e 2.
I wonde i i was b a e y o s upidi y, and I s ill canno answe mysel . You know
ha I don’ conside mysel a pa icula ly b a e o no o iously s upid pe son (a leas
acco ding o he de ini ion o he g ea Ca lo M. Cipolla [1]. Despi e he i eless e o
I make nigh a e nigh o p o e you o he wise; we ha e been able o walk his pa h
oge he . Ha d, and as ha d as i can be, challenging day a e day, complex as we would
ne e ha e hough , bu exci ing beyond compa e. Tha is why he sa is ac ion i gi es
me o inish his oad and o be able o mo e o wa d o se new goals, new places, new
lea ning, new pa hs, and new li es.
Tha is why I hank you, Mamo , especially you, bu also all he people who we e
he e in one way o ano he . Ta iana Pineda, my u o , iend, ad iso and a big pa
o my e e y hing. Sa a Sendino, o he company and guidance om day one. Unai
L´opez-No oa, o all he imely ad ice and guidance in he momen s when I needed i
he mos . San iago G´al is, ”Mi Pe i o”, o he closeness and ememb ance. Na giza
Mikh idino a, o cou se, he example, he dedica ion and he openness ha I needed o
lea n so much. Be ha Nge eja who is he mos posi i e human being I ha e e e had
he pleasu e o mee ing. Ca s en Wol , one o he mos b illian human beings I ha e
e e me , o his eachings, suppo and kindness. I˜nigo San is eban, he bes iend we
could ha e wished o . I zia Ga o e and Oc a io Pe ei a, o hei example, educa ion,
con e sa ions, enaci y and ime. Ai o I azabal, Jon Ande I u ioz and Ola z Pa ola,
wi hou whom, wi hou a doub , he day- o-day li e would ha e been much ha de o
bea .
Special men ion should be made o my hesis di ec o s, No be o L´opez de Lacalle
and Ne ea Toledo Ganda ias, wi hou whom none o wha I ha e w i en he e, no he
d eams I ha e buil , would ha e been ounded. Be i, o us ing me om he i s
momen , e en I was unable o do so, o suppo ing me a e e y oppo uni y, o ac ing
as a men o , and o his eachings. Immeasu able. Thank you.
1Ce a i, G., Bosio, Z. (1992). Luna Roja - Dynamo [CD]. A gen ina: Sony Music.
2Jo ge D exle (2017). Pongamos que hablo de Ma ´ınez - Sal a idas de hielo [CD]. Espa˜na: Wa ne
Music

IV
Thanks also o e e y membe o he CFAA, a place ha has been my second home
o he las i e yea s, whe e I ha e been allowed o lea n, make mis akes, pick mysel
up whene e I need o, and eel an essen ial pa o i .
Finally, I ha e always hough ha one ne e walks alone, bu ha e e y s ep we
ake wi h us, ca ying wi h us each and e e y one o he people who ha e been pa o
ou his o y. So, o cou se, my amily in Colombia, my pa en s, b o he s, sis e s, aun s,
cousins, and iends. I ca y each and e e y one o you wi h me e e y s ep o he way,
and I am wha I am because o wha you allowed me o lea n om all o you. I also
hank hose I conside my iends om he bus s op: And ´es, Nago e, Es i, Ja i, Ja ie ,
and Sil ia, who ha e allowed us o eel pa o each o he and o aise ou daugh e in
a communi y.
And you a e s ill oo young o ead and unde s and his, Mai e, bu in ime I hope
you will know ha his is also o you; o cou se, i is o you. You a e my li e, lo e, and
desi e o li e and mo e o wa d, achie e goals, and imp o e mysel daily. So THANK
YOU, wi h capi al le e s, because my hea is comple ely whole o you. Because I ha e
el hings ha I didn’ e en know exis ed and didn’ hink I dese ed, o illing me
wi h p ide e e y day o being you a he and eeling eno mously g a e ul o ha ing
he p i ilege o seeing you g ow day by day. I lo e you beyond measu e.
To all o you and hose I ha e un o una ely no men ioned he e, as he g ea es said,
”G acias, o ales”.
V
Glo ia, a´un ecue do la noche en que odo ocu i´o, la m´usica sonando de ondo en
uno de los pocos elec odom´es icos que en´ıamos, sen ados en el suelo jun o a la en-
ana y ab azados iendo como Bogo ´a nos egalaba una de sus noches m´agicas, donde al
anochece , la nube de con aminaci´on daba paso a una noche es ellada y a una luna oja
3.
Recue do u his o ia, ecue do us sue˜nos, ecue do u p egun a, ecue do cada uno
de los segundos que pasa on mien as pensaba; aunque ya me hab´ıa decidido desde el
momen o en que comenz´o u na aci´on. Recue do la emoci´on, ecue do el miedo, ecue do
el amo , nunca ol ida ´e el amo , y en es e momen o no puedo e i a sen i me o gulloso
de i, de m´ı y de la decisi´on que omamos, po que desde ese momen o nues a ida cambi´o
po comple o. Ese egalo que me dis e cambi´o mi ida en e a 4.
Me p egun o si ue alen ´ıa o es upidez, y oda ´ıa no soy capaz de esponde me. Sabes
que no me conside o una pe sona especialmen e alien e ni no o iamen e es ´upida (al
menos seg´un la de inici´on del g an Ca lo M. Cipolla [1], y a pesa del es ue zo inago able
que hago noche as noche pa a demos a e lo con a io); pe o g acias a ello hemos
podido eco e jun os es e camino. Du o, y an du o como puede se , desa ian e d´ıa
as d´ıa, complejo como nunca hubi´e amos pensado, pe o emocionan e sin compa aci´on.
Po eso, la sa is acci´on que me da e mina es e camino y pode a anza pa a plan ea
nue as me as, nue os luga es, nue os ap endizajes, nue os caminos, nue as idas.
Po eso e doy las g acias. mamo , especialmen e a ´ı, pe o ambi´en a odas las pe -
sonas que es u ie on ah´ı de una u o a mane a. Ta iana Pineda, mi u o a, mi amiga,
mi conseje a y una g an pa e de mi odo. Sa a Sendino, po su compa˜n´ıa y gu´ıa desde el
p ime d´ıa. Unai L´opez-No oa, po odos los consejos opo unos y la o ien aci´on en los
momen os en que m´as lo necesi aba. San iago G´al is, ”Mi pe i o”, po la ce can´ıa y el
ecue do. Na giza Mikh idino a, po supues o, el ejemplo, la dedicaci´on y su anqueza
que an o necesi o ap ende . Be ha Nge eja, el se humano m´as posi i o que he enido
el place de conoce en la ida. Ca s en Wol , uno de los se es humanos m´as b illan es
que he conocido en oda mi ida, po sus ense˜nanzas, su apoyo y su amabilidad. I˜nigo
San is eban, el mejo amigo que hemos podido desea . A I zia Ga o e y a Oc a io
Pe ei a, po su ejemplo, ense˜nanzas, con e saciones, es´on y iempo. A Ai o I azabal,
Jon Ande I u ioz y Ola z Pa ola, sin los cuales, sin duda, el d´ıa a d´ıa hubie a sido
mucho m´as di ´ıcil de aguan a .
Menci´on especial me ecen mis di ec o es de esis, No be o L´opez de Lacalle y Ne ea
Toledo Ganda ias, sin los cuales nada de lo que he esc i o aqu´ı, ni los sue˜nos que he
cons uido, hab ´ıan enido undamen o. Be i, po con ia en m´ı desde el p ime mo-
men o cuando ni siquie a yo e a capaz de hace lo, po apoya me en cada ocasi´on, po
eje ce de men o , po las ense˜nanzas. Inconmensu able. G acias.
G acias ambi´en a odos y cada uno de los miemb os de la CFAA, un luga que ha
sido mi segundo hoga du an e los ´ul imos 5 a˜nos, donde me han pe mi ido ap ende ,
3Ce a i, G., Bosio, Z. (1992). Luna Roja - Dynamo [CD]. A gen ina: Sony Music
4Jo ge D exle (2017). Pongamos que hablo de Ma ´ınez - Sal a idas de hielo [CD]. Espa˜na: Wa ne
Music
VI
equi oca me y le an a me cada ez que lo necesi aba, y sen i me una pieza impo an e
de all´ı.
Po ´ul imo, siemp e he pensado que uno nunca camina solo, sino que cada paso que
damos lo hacemos lle ando con noso os a odas y cada una de las pe sonas que han
o mado pa e de nues a his o ia. As´ı que, po supues o, mi amilia en Colombia, mis
pad es, mis he manos y he manas, mis ´ıas, mis p imos, mis amigos. A odos y cada
uno de us edes los lle o conmigo en cada paso del camino, y soy lo que soy g acias a lo
que me pe mi ie on ap ende de odos us edes. Tambi´en ag adezco a los que conside o
mis amigos de la pa ada del au ob´us, And ´es, Nago e, Es i, Ja i, Ja ie , Sil ia, que nos
han pe mi ido sen i nos pa e y c ia a nues a hija en comunidad.
Y ´u e es a´un demasiado jo en pa a lee y en ende es as palab as, Mai e, pe o con el
iempo espe o que sepas que es o ambi´en es pa a i, po supues o que es pa a i. T´u e es
mi ida, mi amo , mis ganas de i i y a anza , de alcanza me as y de supe a me cada
d´ıa. As´ı que GRACIAS, con may´usculas, po que po i mi co az´on es ´a comple amen e
lleno, po que he sen ido cosas que ni siquie a sab´ıa que exis ´ıan y que no c e´ıa me ece ,
po llena me de o gullo cada d´ıa po se u pad e y sen i me eno memen e ag adecido
po ene el p i ilegio de e e c ece d´ıa a d´ıa. Te quie o sin medida.
A odos us edes y a los que lamen ablemen e no he mencionado aqu´ı, como dijo el
m´as g ande, ”G acias, o ales”.
VII
Abs ac
Resea ch and De elopmen (R&D) p ojec s a e augh wi h signi ican p oblems, such
as he likelihood o ailu e, he high a e o p ojec s ending wi hou esul s, he chang-
ing p ojec scope, he p olonged p ojec li e cycle, and he clash be ween he in e es s
o academics and companies. Fu he mo e, R&D p ojec s a e also cha ac e ised by he
di icul y o bounding o de ined pe iods and planning. The non- ixed scope o hese
p ojec s can change due o in e nal and ex e nal ac o s. Besides ha , he Technology
Readiness Le el (TRL) in which he R&D p ojec is conduc ed de e mine i s cha ac e -
is ics and challenges. Fu he mo e, he quali y o he esul o an R&D p ojec is seen
only a he end o i . This esul is o med by he p og essi e and cumula i e ealisa ion
o he ac i i ies ha make i up. I also depends on se e al ea u es, cha ac e is ics and
a ibu es ha con ibu e o mee ing he needs and expec a ions o he s akeholde s.
The main goal o his hesis is o o e come some o he issues inhe en o hese ypes
o p ojec s, de eloping a p ojec managemen me hodology based on Ea ned Quali y
Me hod (EQM) and da a analysis o imp o e he e iciency o R&D p ojec s in a nea -
eal p oduc ion en i onmen in a TRL 5-7.
The hesis elies upon published pape s ha p opose measu ing and imp o ing he
managemen o Resea ch and De elopmen (R&D) p ojec s. The me hod leans on he
o mula ion and g adual and ecu en e alua ion o quali y c i e ia as a pe o mance
indica o o he wo k ca ied ou . The way o de elop he idea s ands on he concep
ha quali y is a measu able quan i y ha accumula es h oughou he p ojec .
The p oposed p ojec managemen me hodology is buil on h ee main aspec s: Col-
labo a ion be ween Uni e si y and Indus y The co ec in e p e a ion o he TRL whe e
esea ch p ojec s a e de eloped The s udy o di e en me ics o p ojec managemen ,
such as he measu emen o he success o p ojec s, he Key Pe o mance Indica o s
(KPIs) o a p ojec -based o ganisa ion, and he EQM
EQM is analysed, used and aken a s ep u he by applying i o R&D p ojec s and
p oposing new con ibu ions o he de ini ion o quali y c i e ia, a holis ic iew o he
me hod, he ein o cemen o EQM, and a se ies o ecommenda ions o i s co ec
implemen a ion.
The me hodology has been es ed wi h h ee ac ual use cases wi h di e en cha ac-
e is ics in e ms o p ojec size, unding and eam membe s; and alida ed on an R&D
Cen e in Ad anced Manu ac u ing in Ae onau ics.
The pilla s o he hesis a e ocused on he analysis o he men ioned componen s and
hei in eg a ion o he de elopmen o a me hodology o imp o e he e iciency in he
use o esou ces and quali y o ob ained esul s in he R&D p ojec s’ amewo k.
The esul s ha e been p esen ed in ou publica ions a academic con e ences and h ee
o iginal pape s submi ed o scien i ic jou nals o he JCR’ qua ile one and wo 5, and a
inal one in he inal p ocess o p epa a ion. The key indings o hese s udies demons a e
5Scopus Sou ces - h ps://www.scopus.com/sou ces.u i - Las Access: 20/12/2022

XV
Lis o abb e ia ions.
•ACWP: Ac ual Con ibu ion o he Wo k Pe o med.
•AM: Addi i e Manu ac u ing.
•CCS: C i ical Chain Scheduling.
•CFAA: Cen o de Fab icaci´on A anzada Ae on´au ica (Ad anced Manu ac u ing
Cen e o Ae onau ics: In English).
•CNC: Compu e Nume ical Con ol.
•CP: Con ol Poin s.
•CPM: C i ical Pa h Me hod.
•CSC: C i ical Success C i e ia.
•CSF: C i ical Success Fac o .
•CTQs: C i ical To Quali y.
•DEA: Da a En elopmen Analysis.
•DMAIC: De ine, Measu e, Analyse, Imp o e, Con ol.
•EC: Eu opean Commission.
•EQM: Ea ned Quali y Me hod.
•EQWP: Ea ned Quali y o he Wo k Pe o med.
•EU: Eu opean Union.
•EVM: Ea ned Value Managemen .
•EVMS: Ea ned Value Managemen Sys em.
•FoF: Fac o ies-o - he-Fu u e.
•FRED: Fi s Requi emen s Elucida o Demons a ion.
•IoT: In e ne o Things.
•KPIs: Key Pe o mance Indica o s.
•KTTOs: Knowledge and Technology T ans e O ganisa ions.
•MPCS: Mul idimensional P ojec Con ol Sys em.
•MRL: Manu ac u ing Readiness Le el.
•NASA: Na ional Adminis a ion o Space Agency.
•OEM: O iginal Equipmen Manu ac u e .
•PBEV: Pe o mance-Based Ea ned Value.
•PCTI: Science, Technology and Inno a ion Plan (in Spanish).
•PCWP: Planned Con ibu ion o he Wo k Pe o med.
•PCWS: Planned Con ibu ion o he Wo k Scheduled.
•PERT: P og am E alua ion and Re iew Technique.
•PLC: P og ammable Logic Con olle .
XVI
•PMO: P ojec Managemen O ice.
•PMs: P ojec Manage s.
•PPP: Phased P ojec Planning.
•PQWP: Planned Quali y o he Wo k Pe o med.
•PQWS: Planned Quali y o he Wo k Scheduled.
•QBS: Quali y B eakdown S uc u e.
•QC: Quali y C i e ia.
•QPI: Quali y Pe o mance Index.
•QV: Quali y Va iance.
•R&D: Resea ch and De elopmen .
•SLR: Sys ema ic Li e a u e Re iew.
•SMEs: Small and Medium En e p ises.
•TRL: Technology Readiness Le el.
•TTOs: Technology T ans e O ices.
•UIC: Uni e si y-Indus y Collabo a ion.
•VoB: Voice o he Business.
•VoC: Voice o he Cus ome .
•WBS: Wo k B eakdown S uc u e.
Con en s
Acknowledgmen s ...................................................... VI
Abs ac . ...............................................................VIII
Resumen. ...............................................................XIII
Lis o abb e ia ions ....................................................XVI
Pa I Desc ip ion o Con ibu ions.
1 In oduc ion......................................................... 3
1.1 P esen a ion. ..................................................... 3
1.2 Mo i a ion........................................................ 4
1.2.1 Pe sonal mo i a ion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Theo e ical amewo k and me hodology. ........................... 11
2.1 In oduc ion. ..................................................... 11
2.2 Theo e ical F amewo k. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.1 Uni e si y-Indus y collabo a ion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.2.2 Technology Readiness Le el. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2.3 P ojec managemen me ics and Key Pe o mance Indica o s o
R&Dp ojec s. .............................................. 28
2.2.4 R&D P ojec Managemen Me hodology. . . . . . . . . . . . . . . . . . . . . . . . 39
2.3 Resea ch me hodology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
3 Hypo hesis and Objec i es. ......................................... 51
3.1 Hypo hesis........................................................ 51
3.2 Objec i es........................................................ 53
3.2.1 Gene al Objec i es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.2.2 Seconda y Objec i es. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
XVIII Con en s
4 Summa y and discussion o he esul s. ............................. 55
4.1 Summa y and discussion.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.1.1 Fi s Publica ion CP.1 - Con e ence Pape : ”Pa e ns o
in e na ional coope a ion be ween inno a ion clus e s. Cases o
CFAA and uh alley”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.1.2 Second Publica ion CP.2 - Con e ence Pape : ”Building
Coope a ion be ween Inno a ion Clus e s Based on Compe ences
Requi emen s. Case o CFAA and uh alley”.. . . . . . . . . . . . . . . . . . . 58
4.1.3 Thi d Publica ion JP.1 - O iginal Jou nal Pape : ”TRLs 5–7
ad anced manu ac u ing cen es, p ac ical model o boos
echnology ans e in manu ac u ing”. . . . . . . . . . . . . . . . . . . . . . . . . . 59
4.1.4 Fou h Publica ion CP.3 - Con e ence Pape : ”Assessing he
success o R&D p ojec s and inno a ion p ojec s h ough p ojec
managemen li e cycle”. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.1.5 Fi h Publica ion JP.2 - O iginal Jou nal Pape : ”P ojec Success
C i e ia E alua ion o a P ojec -Based O ganiza ion and I s
S akeholde s - A Q-Me hodology App oach”. . . . . . . . . . . . . . . . . . . . . 62
4.1.6 Six h Publica ion JP.3 - O iginal Jou nal Pape : ”Iden i ica ion o
key pe o mance indica o s in p ojec -based o ganisa ions h ough
heleanapp oach”........................................... 64
4.1.7 Se en h Publica ion CP.4 - Con e ence Pape : ”Hyb id P ojec
Managemen Me hodology o R&D, Inno a ion and R&D&I
P ojec sinCFAA”........................................... 65
4.1.8 Eigh h Publica ion JP.4 - O iginal Jou nal Pape : ”A da a-d i en
app oach o a new p ojec managemen me hodology based on
quali yinc emen s”. ......................................... 66
Re e ences ..............................................................115
Pa II Conclusions
5 Conclusions and u u e esea ch. ....................................129
5.1 Conclusions.......................................................129
5.2 Fu u e esea ch....................................................130
Pa III Appendix
6 Uni e si y - Indus y Collabo a ions. ...............................137
6.1 CP. 1 - Con e ence Pape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.2 CP. 2 - Con e ence Pape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
7 Technology Readiness Le el. ........................................151
7.1 JP 1 - O iginal Jou nal Pape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
Con en s XIX
8 P ojec Managemen Me ics........................................167
8.1 CP. 3 - Con e ence Pape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
8.2 JP. 2 - O iginal Jou nal Pape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
8.3 JP. 3 - O iginal Jou nal Pape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
9 P ojec Managemen me hodology o R&D p ojec s. ...............217
9.1 CP. 4 - Con e ence Pape . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
9.2 JP. 4 - O iginal Jou nal Pape (In p ocess).. . . . . . . . . . . . . . . . . . . . . . . . . . . 227
10 Communica ions and o he Publica ions. ............................229
10.1 P ess Communica ions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
10.2O he publica ions.................................................230
11 Da a Appendix. .....................................................231
11.1CFAA’sKPIs. ....................................................231
11.2 P oposed da abases and classi ica ion scheme.. . . . . . . . . . . . . . . . . . . . . . . . . 238
11.3 C i ical Success C i e ia as in Sas oque e al. [10] . . . . . . . . . . . . . . . . . . . . . 241

Lis o Figu es
2.1 D i e s o uni e si y hi d mission [14]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 TRL and phases o de elopmen , based on [15]. . . . . . . . . . . . . . . . . . . . . . . . 24
2.3 Concep ual model o he esea ch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.4 Collec ingme hods ................................................ 45
2.5 Fi s Publica ion: ”Pa e ns o In e na ional Coope a ion be ween
Inno a ion Clus e s. Cases o CFAA and uh alley” - Resea ch
Me hodology...................................................... 46
2.6 Second Publica ion: ”Building Coope a ion be ween Inno a ion Clus e s
Based on Compe ences Requi emen s. Case o CFAA and uh alley.” -
Resea chMe hodology ............................................. 46
2.7 Fi h Publica ion: ”P ojec Success C i e ia E alua ion o a P ojec -
Based o ganisa ion and I s S akeholde s—A Q-Me hodology App oach.”
-Resea chMe hodology............................................ 48
2.8 Resea ch Me hodology JP4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.1 Resea chOnion ................................................... 56
4.2 P oposed P ojec Managemen Me hodology in phases and ac i i ies . . . . . 72
4.3 P oposed P ojec managemen me hodology including ac i i ies and
documen s........................................................ 74
4.4 QBS-WBSExample.............................................. 81
4.5 QBS-WBSP ojec 3 .............................................101
Lis o Tables
2.1 O iginal TRL model [16] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.1 CSC and KPIs chosen pe p ojec . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.2 Quali y C i e ia de ini ion - P ojec 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
4.3 Quali y C i e ia de ini ion - P ojec 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.4 Quali y C i e ia de ini ion - P ojec 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.5 Alloca ion scheme o he po en ial con ibu ion o he ac i i ies o quali y 100
4.6 Phase 1: QBS-WBS le els 2, 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.7 Phase 2: QBS-WBS le els 2, 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.8 Phase 3: QBS-WBS le els 2, 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
4.9 Phase 4: QBS-WBS le els 2, 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.10 Phase 5: QBS-WBS le els 2, 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.11 Phase 6: QBS-WBS le els 2, 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.12 Gan Cha o he p ojec . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.13 PCWS:P3Ph1....................................................106
4.14 PCWS:P3Ph2....................................................106
4.15 PCWS:P3Ph3....................................................106
4.16 PCWS:P3Ph4....................................................107
4.17 PCWS:P3Ph5....................................................107
4.18 PCWS:P3Ph6....................................................107
4.19 Planned quali y o wo k scheduled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.20 Planned con ibu ion o he wo k pe o med . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.21 Ac ual con ibu ion o wo k pe o med - Phase 1 . . . . . . . . . . . . . . . . . . . . . . 109
4.22 ACWP-P3Ph1...................................................110
4.23 ACWP-P3Ph2...................................................110
4.24 ACWP-P3Ph3...................................................110
4.25 ACWP-P3Ph4...................................................111
4.26 ACWP-P3Ph5...................................................111
4.27 ACWP-P3Ph6...................................................111
4.28 Ea ned quali y o wo k pe o med - P ojec 3 . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.29 Quali y a iance - P ojec 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
6 CHAPTER 1. INTRODUCTION.
new echnology can be challenging and haza dous. This app oach necessi a es he de el-
opmen o dis up i e p ac ices.
One o he co e bene i s o his collabo a ion can be seen by mo e esea che s and
company s a wo king wi h KTTOs, sha ing expe iences, poin s o iew, and solu ions
o p oblems encoun e ed du ing he execu ion o R&D p ojec s. Se e al ways o measu e
he success o his collabo a ion can be se [26]. Howe e , he mos impo an is no jus
how many p ojec s a e being ca ied ou bu he success a e o hose p ojec s aligned o
he o ganisa ion’s Key Pe o mance Indica o s (KPIs). Al e na i ely, i i has adequa ely
sol ed he p oblem, i wi h he gap obse ed, o ul illed he expec a ions and needs o
he di e en s akeholde s.
Limi ed access o public o p i a e unding, a long payback pe iod o some ypes o
p oduc s o w ong ma ke esea ch can lead o he loss o e o s om many yea s in
de eloping and esea ching new p oduc s. This phenomenon is in ica e o la ge compa-
nies and Small and Medium En e p ises (SMEs). As a esul , many ideas and disco e ies
de eloped a ea ly TRLs emain in uni e si ies, wi h esea che s unable o comme cially
capi alise on hei inno a ions due o a lack o en ep eneu ial skills, business know-how
and con ac s needed o gain access o he business wo ld [25].
Howe e , hese in e media e KTTOs a e undamen al and essen ial ins umen s wi hin
he inno a ion p ocess o be dismissed. Fo una ely, a new way o managing companies,
p ojec s, and uni e si ies based on da a is eme ging hanks o Indus y 4.0.
In his new e a, la ge amoun s o da a a e a ailable wi hin companies, uni e si ies,
and KTTOs; and di e en ypes o da a analysis echniques ise o maximise he bene i
om his da a, allowing o de elop o new p ocesses, ools, p ac ices, and heo ies in
p ojec managemen o imp o e he con ol o p ojec s, e en in ha sh en i onmen s
such as KTTOs o an R&D en i onmen .
Gi en hese condi ions, quali y assu ance is essen ial du ing he p ojec li e cycle ( om
he cus ome ’s and p ojec eam’s app oach). This is why, based on he p emise ha
quali y is a quan i iable quan i y and ollowing he clien ’s and p ojec eam’s expec-
a ions, his hesis is p oposed o analyse measu es o con ol and assess he p ojec ’s
quali y pe o mance h oughou i s li e cycle. I also highligh s how he use o in o -
ma ion o es ablish signi ican quali y de ia ions is in ended o help P ojec Manage s
(PMs) make imely choices o ec i y he p ojec ’s pa h.
The mos impo an concep s o he hesis a e he ollowing ones:
1. Collabo a ion be ween public and p i a e o ganisa ions o he g ow h o R&D in
local en i onmen s h ough a quick echnology ans e ;
2. he iden i ica ion and measu emen o success c i e ia o R&D p ojec s o bo h PMs
and s akeholde s;

1.2. MOTIVATION. 7
3. iden i ica ion o KPIs o p ojec -based o ganisa ions based on lean p inciples h ough
a da a-d i en app oach;
4. he iden i ica ion o quali y c i e ia o an R&D p ojec based on success ac o s,
o ganisa ional KPIs and clien -de ined c i e ia; based on a da a-d i en app oach o
con ol he quali y o he wo k pe o med and he quali y o he deli e ed p oduc ;
5. he need o b eak down quali y c i e ia in o small pieces o con ol he ou come o a
p ojec ;
6. he de elopmen o a p ojec managemen me hodology based on he concep ha
quali y is measu able and achie able h oughou he p ojec li e cycle and on gene -
a ing knowledge o imp o e p ojec pe o mance.
These a e some easons why he need o con ol R&D p ojec s is inc eased in imes
as u bulen as he ones we li e in. Looking o ways o imp o e he e iciency o R&D
in es men s gi en o public-p i a e en i ies, achie ing quali y esul s o he p ojec s and
being able o ake decisions in ime o a oid u he R&D losses a e some o he s a egies
o be ollowed o limi he impac o he cu en economic con ex . Encou aged by he
cu en socio-economic scena io, we se a gene al objec i e o c ea e a way o ca y
ou hese s a egies and hus imp o e he e iciency o R&D p ojec s in a quasi- eal
p oduc ion en i onmen in a TRL 5 - 7, which will be desc ibed in Sec ion 4.1.8.
This gene al objec i e is complemen ed h ough di e en ypes o scien i ic commu-
nica ions wi h he ollowing speci ic objec i es:
1. To de ine he collabo a ion s a egies be ween public-p i a e en i ies h ough he
s udy o ela ed inno a ion cen es whose objec i e is o p omo e scien i ic, economic,
and social g ow h in hei egions h ough collabo a ion wi h he public and p i a e
sec o s;
2. o analyse he ole o he TRL in echnology ans e and how i a ec s egions wi h
such cen es;
3. o analyse di e en me ics o R&D p ojec s, such as he success o p ojec s and
wha i means o s akeholde s, he iden i ica ion o KPIs o o ganisa ions in cha ge
o de eloping such p ojec s and he measu emen o p ojec p og ess h ough quali y
me ics;
4. o de elop an R&D p ojec managemen me hodology based on da a analysis o
imp o e he managemen o his ype o p ojec .
The scien i ic pape s p esen ed in his hesis summa ise he expe ience and knowledge
acqui ed on a heo e ical and p ac ical le el in he managemen o R&D p ojec s in
public-p i a e cen es in collabo a ion wi h indus y and loca ed in TRL 5 - 7, om he
beginning o he TRL analysis in Sep embe 2019 o he de ini ion o he me hodology
in Decembe 2022.
8 CHAPTER 1. INTRODUCTION.
In ending o desc ibe he ul ilmen o he p oposed objec i es o his hesis, we begin
wi h an analysis o he cu en s a e o esea ch cen es like he Ad anced Manu ac u ing
Cen e o Ae onau ics - CFAA (in Spanish) 3) om a p ojec managemen poin o iew.
Thanks o a bench-ma king s udy, a gene al analysis o he cu en s a e o he cen es
and some gene al ecommenda ions o he Basque Coun y scena io. The esea ch was
de eloped o unde s and and go deepe wi hin he p oduc ion chain and he TRL o he
CFAA. A esea ch pape has been published in his Sec ion [5].
A e ha , o he me hodology, i was necessa y o analyse and s udy he ole o
uni e si y-indus y collabo a ion o complemen he p e ious poin . The e o e, wo e-
sea ch p ojec s we e de eloped: i) The o me o unde s and he pa e ns o in e na ional
coope a ion be ween inno a ion clus e s [6], ii) he la e was o analyse how o c ea e
b idges o coope a ion be ween hese clus e s [7]. In his way, i was possible o o ien-
a e owa ds he pu sui o objec i es h ough p ojec managemen me hodology and
he bes way o measu e hese collabo a ions in an R&D p ojec en i onmen .
Finally, he de ini ion o he objec i es o be moni o ed o he success o an R&D
p ojec h ough P ojec Managemen Me ics and a da a-d i en analysis wi h h ee
di e en app oaches, P ojec Success, EQM and Key Pe o mance Indica o s (KPIs),
we e s udied.
The analysis o P ojec Success was ca ied ou om wo poin s o iew, he i s one
ensu ing he success o an R&D p ojec h ough he s udy o he p ojec managemen
li e cycle [8]. He e, a hyb id p ojec managemen me hodology was p oposed h ough he
analysis o he success dimensions o one R&D o ganisa ion, analysis and in e p e a ion
o in o ma ion and scien i ic li e a u e a ailable a he ime; he esul s we e published in
a pape p esen ed a a scien i ic con e ence [9]. The second componen o p ojec success
was achie ed by conduc ing a s udy using Q-Me hodology and semi-s uc u ed in e iews
o de ine he success c i e ia o he p ojec s ca ied ou in one R&D o ganisa ion om
he poin o iew o he o ganisa ion and i s s akeholde s [10]. Fo he s udy o KPIs,
esea ch was ca ied ou on iden i ying hese indica o s o p ojec -based o ganisa ions
h ough a lean app oach, and he esul s o i s de elopmen and implemen a ion we e
published in a pape included in a scien i ic jou nal [11]. Finally, o he s udy o EQM,
i s analysis and he s udy o i s use in R&D p ojec s was included in a jou nal pape
summa ising he esea ch me hodology c ea ed. 4
The me hodology was implemen ed and es ed in h ee di e en p ojec s (An ex-
panded desc ip ion o he p ojec s can be ound in Sec ion 4.1.8.4). The i s p ojec
wi h in e nal unding (UPV/EHU) in which a eal- ime machine moni o ing pla o m
was de eloped. The esul s ha e been p esen ed in a esea ch pape a an academic
con e ence [12]. A second p ojec wi h egional unding (Elka ek5), in which he aim
is o p edic he wea o cu ing ools in b oaching ope a ions. The esul s a e being
3h ps://www.ehu.eus/en/web/c aa/home - Las Access: 20/12/2022
4Pape in he p ocess o publica ion
5h ps://www.sp i.eus/en/ekoga apena/ - Las Access: 20/12/2022
1.2. MOTIVATION. 9
collec ed, and he de elopmen and conclusions will be p esen ed in a esea ch pape
unde p epa a ion.
Fu he mo e, he hi d p ojec was de eloping a Eu opean- unded p ojec ask (In e Q
P ojec - Ho izon 20206). The job was o c ea e i ual senso s o he manu ac u ing
p ocess o a machine h ough he analysis o a ailable quali y and p ocess da a. In
addi ion, one jou nal pape was published wi h he de elopmen and conclusions o he
machine assembly o da a collec ion, and analysis [13]. In addi ion, we a e de eloping
a second jou nal pape esuming he desc ip ions o he i ual senso s, da a analysis,
and conclusions o he esea ch.
We p esen he pape s published a academic con e ences and in scien i ic jou nals in
his hesis, accompanied by he pape s p esen ed.
1.2.1 Pe sonal mo i a ion.
The mo i a ion o esea ching his opic s ems om he awa eness o he need o use
be e he esou ces alloca ed o esea ch. The esea ch’s main objec i e was o ind a
way o imp o e he e iciency o he esou ces a ailable du ing esea ch. Con a y o
wha expe ience dic a ed, he esea che se ou o ind ways in which R&D p ojec s a e
mo e esea che - iendly, aluable, high-quali y, classi iable and de ensible esul s a e
ob ained, and he money in es ed bea s ui ha can bene i socie y.
The e a e many ways o do his. Fi s , da a analysis alone can help he adminis a-
ions and con ol bodies o companies and esea ch cen es o use esou ces alloca ed
o esea ch and de elopmen . Howe e , he dena u alisa ion ha comes wi h p agma ic
da a analysis dehumanises p ojec managemen .
Abo e all, p ojec managemen is based on managing people a he han de eloping
a da a pipeline o ge a job done. The e o e, a mix o he wo wo lds, da a-d i en and
p ojec managemen , is an app oach ha can be bene icial o he discipline and can help
o ul il he mo i a ion.
6h ps://in e q-p ojec .eu/ - Las Access: 20/12/2022
2
Theo e ical amewo k and me hodology.
2.1 In oduc ion.
Based on he concep s de eloped in he p e ious Sec ion (See Sec ion 1.1), his heo e ical
amewo k aims o se up he basis o o mula e a new da a-applied and knowledge-
based p ojec managemen me hodology o R&D p ojec s on KTTOs, which demands
an unde s anding o :
•Uni e si y-Indus y collabo a ion (UIC) cha ac e is ics;
•TRL unc ion be ween echnology and knowledge ans e ;
•P ojec managemen me ics o R&D p ojec s:
– P ojec Success;
– KPIs;
– quali y measu emen s.
•P ojec managemen me hodologies o R&D p ojec s.

12 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
2.2 Theo e ical F amewo k.
2.2.1 Uni e si y-Indus y collabo a ion.
D ama ic changes in he las decades ha e ans o med he o ganisa ion o mode n li e,
and uni e si ies, as a undamen al pa o mode n socie y, ha e no been immune o hose
changes. Uni e si ies se e a c i ical ole in aining he nex gene a ion o specialised,
in o med indi iduals while dissemina ing in o ma ion. On he o he hand, companies a e
swi ly adap ing o changing si ua ions, analysing and e alua ing he isks and possibil-
i ies hey ace.
2.2.1.1 Con ex ual pe spec i e.
Du ing he 1980s, Eu ope ea ed losing i s leade ship ole o eme ging coun ies due o
economic slumps. The Eu opean Commission h i es on egene a ing he echnology p o-
g am o c ea e compe i i e Eu opean indus ies wi h he ac i e collabo a ion o uni e -
si ies o inc ease in es men in applied esea ch, de elopmen , and inno a ion ac i i ies.
Go e nmen s a all le els u ged uni e si ies o make mo e ou s anding con ibu ions o
hei na ional inno a ion sys ems. As a esul , uni e si ies lou ished as essen ial playe s
in egional de elopmen ac i i ies, which eased inno a ion-based g ow h [27].
New oles ha e eme ged o uni e si ies alongside he g owing impo ance o knowledge
p oduc ion and inno a ion, c ea ing bo h he oppo uni y o and he necessi y o e hink
he meaning o Uni e si ies as he ac i e membe o he socie y hey we e. P i ileges,
oles, esou ces, asks, and du ies needed o be analysed and e o med o occupy and
ul il he ole hey we e gi en.
E en hough ha UIC is a om being a no el y [28], o e he pas decades, e o s
o enhance his collabo a ion ha e been widely suppo ed. Local companies, go e n-
men s, and socie y p essu e uni e si ies o be mo e in ol ed and ele an . Uni e si ies
esponded acco dingly by opening hemsel es up o ex e nal agencies and ac o s, en-
gaging wi h socie y and inc easing hei con ibu ions, acili a ing he eme gence o a
‘Thi d mission’ [29]. Howe e , uni e si ies we e no only asked o in ol emen and ele-
ance bu we e also cha ged wi h asks in ol ing legi imacy, go e nance, ma ke isa ion,
in e na ionalisa ion and exploi a ion o highe educa ion esul s [30].
In e na ionalisa ion o uni e si ies a i ed in he Bologna Ag eemen , which loca ed
hem in o s uc u al e o ms o hei p og ams and cu icula o enhance consensus and
alikeness among deg ees ac oss Eu ope, gi ing s uden s a mo e op ions and mo e di-
e si y in planning p og ams. Fu he mo e, he Bologna Ag eemen inc eased access
o esea ch collabo a o s and opened uni e si ies up o dispu es wi h and compa isons
agains uni e si ies in o he coun ies [14].
Comme cialising academic knowledge is one o he bes ways o gene a e he academic
impac pu sued by uni e si ies due o he easy measu emen o he ma ke accep ance o
he ou pu s o academic esea ch [31]. Simila ly, uni e si ies a e se ing up Technology
2.2. THEORETICAL FRAMEWORK. 13
T ans e O ices (TTOs) wi h in e nal suppo unc ions and p ocedu es o os e he
echnology ans e p ocess o indus ies.
The hi d mission is e e ed o Uni e si ies’ social, en ep eneu ial, and inno a i e
ac i i ies, in addi ion o hei eaching and esea ch missions ha esul in addi ional so-
cie al ad an ages [32]. The p e ious yea s ha e seen an inc easing emphasis on imp o ing
ac i i ies ela ed o his mission, con ibu ing o changing hei s akeholde expec a ions
o wha uni e si ies can achie e.
Besides he adi ional wo pilla s in academic eaching and scien i ic esea ch, he
Thi d mission is abou how consciously and s a egically uni e si ies con ibu e and de-
li e bene i s o hei socie ies in ou signi ican a eas o ac i i y: con inuous educa ion,
echnology ans e , inno a ion, and social engagemen [29]. Howe e , despi e all his,
he idea o he hi d mission eme ged om wi hin he sys em. Mo eo e , i eme ged
as a uni e si y esponse o a b oade se o d i e s mo i a ed by a di e en se o as-
pec s, such as an inc easing need o unding, scien i ic knowledge impac , knowledge
p oduc ion and compe i i eness Figu e 2.1.
Fig. 2.1: D i e s o uni e si y hi d mission [14].
Hu melinna [33] has lis ed a wide a ie y o po en ial mo i a ions o UIC, which a y
acco ding o he company’s size, cul u e o geog aphical loca ion. A he uni e si y side,
hey can be summa ised (bu no es ic ed) o he enhancemen o eaching, access
14 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
o unding/ inancial esou ces, he sou ce o knowledge and empi ical da a, poli ical
p essu e, epu a ion imp o emen , and job o e s o g adua es [34]. Fo uni e si ies,
no engaging in such collabo a ions akes hem away om hese bene i s and c ea es an
a mosphe e o isola ion which, due o an inc easingly globalised en i onmen , canno be
a o ded.
2.2.1.2 Abou inno a ion managemen .
Acco ding o Po e [35], e i o ies need o de elop inno a i e s a egies o build com-
pe i i e ad an ages based on hei exis ing esou ces, skills, capaci ies, and ends. The
ela ionship be ween he uni e si y, go e nmen and indus y ( iple helix model [36])
is conside ed an essen ial elemen o he hi d mission. Fu he mo e, i is bene icial
o classical uni e si ies because he ealisa ion o ansdisciplina y ac i i ies conduc ed
ou side he uni e si y helps hem o c ea e applied esea ch and educa ion, making hem
mo e ealis ic, applicable, and ele an o socie y and economy [37].
T adi ionally, indus ies enuncia e a p oblem si ua ion o uni e si ies and wai o a
solu ion because o he esea ch ca ied ou . Howe e , his has no been sa is ac o y and
lacks he in ol emen o c i ical s akeholde s and a clea mission, o de , and ision om
bo h sides. Nowadays, he company usually sees he impo ance o his collabo a ion o
ob ain mo e success ul and inno a i e esea ch esul s [34].
As UIC de elops, socie y and indus ies a e inc easingly mo i a ed by he added
alue o hei p oduc s and p ocesses and he oppo uni ies a ising om he in ense
collabo a ion. This has caused a d ama ic change o uni e si ies and ans o med hem
in o open ins i u ions ha became ac i e playe s wi hin he egional inno a ion sys em.
The necessi y o enhancing knowledge ans e be ween public esea ch ins i u es and
business was ecognised by he Eu opean Commission (EC) as one o he en c ucial
a eas o ac ion in he Eu opean Union’s (EU) inno a ion policy [38].
In e ms o p oduc -supplie ela ions and access o aci and explici knowledge and
labou supply, uni e si ies and local go e nmen s enhance local companies’ inno a ion
p ocesses and become a sus ainable sou ce o p ac ical knowledge and a d i ing o ce o
echnology exchange [6].
Al hough companies now ecognise knowledge as a undamen al asse and he p ima y
esou ce o boos inno a ion and inc ease p oduc i i y h ough knowledge exchange,
he acquisi ion and abso p ion o ex e nal knowledge, esou ces, and echnology a e
challenging because hei p oduce s and use s come om di e en en i onmen s [39].
This knowledge allows companies o aise se e al di e si ica ion s a egies suppo ed
by local go e nmen s pu suing hei ans o ma ion based on compe i i eness h ough
e iciency o one based on inno a ion.
Acco ding o UNE 166001:2006, inno a ion is he ”applica ion o new o signi ican ly
imp o ed me hods, echniques, o supplies in any ac i i y whose objec i e is o ob ain
new p oduc s o p ocesses o signi ican imp o emen s in exis ing ones” [40]. Despi e
2.2. THEORETICAL FRAMEWORK. 15
his clea de ini ion, in he academic li e a u e, he e is no join ag eemen abou why,
whe e, and how i occu s [41].
Inno a ion pe se is no an isola ed p ocess. I is composed o he in e ac ion be ween
s akeholde s wi h di e en knowledge, expe ience, and unde s anding, and depends on
a complex mix u e o ac o s [42] like:
•Economic ci cums ances;
•company ma u i y le el;
•go e nmen suppo ;
•uni e si y capaci y o R&D p ocesses;
•access o quali ied pe sonnel, among o he s.
Howe e , ecen inno a ion p ocesses ha e become mo e di icul due o p essu e om
he indus y o as e echnological e olu ion o compe e wi h in e na ional ma ke s
and he sho ening o he p oduc li e cycle. In compa ison, he uni e si y mus g ow in
echnical knowledge and ul il a b oade social ole. Likewise, esea ch clus e s ocused
on echnology ans e a e equi ed o inc ease unding o new esea ch ac i i ies and
equipmen o gene a e heo e ical and p ac ical knowledge.
2.2.1.3 Uni e si y-Indus y Collabo a ion’ Cha ac e is ics.
Policymake s end o ocus on sho -pe iod in es men s whe e hey can achie e esul s
as soon as possible as a boos e o hei campaigns in hei nex elec ions. Na u ally,
mo e a en ion is being paid o o ganisa ions ha ac as KTTOs in e media ies in he
inno a ion p ocess ha could accele a e he ob aining o esul s.
The e iciency o hese collabo a ions is a c ucial issue o policymake s [27] because
he esul s achie ed can be easily ans e ed o he indus y quickly and e ec i ely,
con ibu ing o he g ow h and good heal h o he economy. This e lec s on uni e si ies
by o cing hem o de elop collabo a ion s a egies wi h egional indus ies and o he
uni e si ies and wi h he indus ies o uni e si y membe s o he in e na ional inno a ion
ecosys em.
Publicly o p i a ely (o a combina ion o he wo) unded KTTOs housed in uni e -
si ies o public esea ch o ganisa ions; publicly unded egional economic de elopmen
agencies; knowledge-in ensi e business se ices o ganisa ions; p o essional associa ions;
ad iso y bodies; o knowledge wo ke s; could all be iewed as in e media ies ha acil-
i a e knowledge ans e in suppo o he inno a ion p ocess in businesses [43]. These
in e media ies a e ”o ganisa ions o bodies ha ac as agen s o b oke s in any aspec o
he inno a ion p ocess be ween wo o mo e pa ies” and a e c ucial nodes connec ing
supplie s o he use s o knowledge [44]. All ypes o inno a ion in e media ies sha e ou
unc ions [45, 46, 47]:
1. o connec ac o s be ween uni e si ies, indus ies, and go e nmen s;
22 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
Le el 5 componen and b eadboa d alida ed in he simula ed o eal-space
en i onmen .
Le el 6 Sys em adequacy alida ed in a simula ed en i onmen .
Le el 7 Sys em adequacy alida ed in space.
Table 2.1: O iginal TRL model [16]
As has been desc ibed, TRL akes a pa icula echnology om he app oach o un-
damen al p inciples, alida ion o he concep , demons a ion h ough a p o o ype, and
success ul ope a ion. The main goal o he ool is o include ex e nal skills ha can
p o ide new in o ma ion and expe ience essen ial o he g ow h and imp o emen o
he inno a i e capaci ies o businesses and ins i u ions.
Howe e , TRL is used o unde s and he ma u i y o he echnology and o communi-
ca e he ma u i y o a me hodology, concep , and de elopmen o in e nal and ex e nal
p ojec s. Companies ha e been using TRL since he mid-1990s and ha e been adap ing
he ool o ob ain go e nmen unding o R&D p ojec s. Some au ho s ha e desc ibed
ha TRL classi ica ion can se e no only as a measu e o he ma u i y o a echnology
bu also as a measu e o i s eadiness o be in eg a ed in o a mo e ex ensi e sys em. TRL
is cu en ly being used o moni o he ma u a ion p ocess wi hin a echnology de el-
opmen p ocess and o s a e expec a ions on ca ego izing esea ch p ojec s. Mo eo e ,
knowing he s a us o he esea ch se es as inpu o es ablishing long- e m pa ne ships
and esea ch ini ia i es wi hin he company.
The e is a g ea deal o empi ical esea ch indica ing he impo ance o ex e nal sou ces
o knowledge, esou ces, and echnology in he de elopmen o inno a ion in companies
[70], and no jus in NASA. E en hough much o a coun y’s capabili y o inno a ion
esides a uni e si ies, he ool is no commonly used in academia and is only used in
some collabo a i e esea ch p ojec s. Mo eo e , he ool is expanding, and hose who
ha e used i desc ibe i as a help ul ins umen o communica ing wi h he academy
ega ding he de elopmen o p ocesses and hei di e en s ages. As a esul , i s use
has become mo e popula , and hose companies who ha e used i desc ibe i as help ul
in unde s anding he uni e si y ega ding he de elopmen p ocesses and he di e en
s ages o he esea ch p ojec [42].
Depending on he sec o o he objec i e in which he TRL is applied, i is op ional
o each all le els. The echnology, sys em o p ocess may be aluable i i only eaches
he basic le els o he TRL. Howe e , he esul s a e only aluable i hey each he inal
s ages o he ool. Each s age o he TRL has associa ed isk c i e ia depending on he
le el a which i is being wo ked. The isks o pu ing a success ul p o o ype in ac ual
condi ions di e om hose o p e iously submi ing he same p o o ype o e alua ion
in o simula ion so wa e. Unde s anding, desc ibing, and con olling he e olu ion o
hese isks, depending on he case o he echnology o he p ojec o be comple ed, is
a c i ical ask o co ec managemen h ough he p ojec li e cycle and o ensu e ha
i achie es he desi ed bene i s. Howe e , when conside ing hese isks, no only hose

2.2. THEORETICAL FRAMEWORK. 23
associa ed wi h he de elopmen o he echnology (inne pic u e) mus be conside ed,
bu also he isks associa ed wi h he adop ion o he echnology (b oade pic u e), which
in many cases may de e mine he use o no o he inal esul o he p ojec .
In he 1990s, he TRL e ol ed o a nine-le el classi ica ion o b oaden he ision and
be clea e a each le el. A ha ime, he TRL was clea e as a ool o moni o and e al-
ua e he echnology de elopmen p ocess, o p o ide a c i e ion o ca ego izing esea ch
p ojec s and a scale o compa e echnologies. Fu he mo e, i p o ides s akeholde s wi h
a single language o comp ehend he unde lying echnology, he echnological demon-
s a ion o be achie ed, he p oduc p o o ype o p oduc p ojec de elopmen , and,
mos impo an ly, o assess he e icacy o echnology ans e .
Al hough i was ini ially in ended only o NASA supplie s [68], long- e m ela ion-
ships could be buil o de eloping a speci ic echnology o esea ch p ojec . Howe e ,
despi e i s spa ial ocus, i s uses soon became common because i s inal esul s could
be easily adap ed o assess he eadiness o a pa icula echnology, p oduc o sys em
equi ed o and speci ic objec i e in almos any sec o .
Each le el wi hin he classi ica ion is essen ial, as hey lay he ounda ion o p og ess
a subsequen le els and p o ide in o ma ion o make u u e decisions. The ool is based
on enginee ing assessmen s o communica e signs o p og ess and hypo heses and ca e-
go ize hem wi hin he p ocess managemen and ex e nal s akeholde s’ poin o iew.
2.2.2.1 De ini ion o Technology Readiness Le els.
A gene al ou line o each le el will be desc ibed o unde s and he ela ionship be ween
he s ages. Figu e 2.2 p o ides an o e iew o he TRL and he phases o de elopmen .
TRL 1: Basic p inciple obse ed and epo ed.
The bo om laye o TRL is whe e he basis o he e olu ion o he use o a pa icula
echnology is se . Basic scien i ic esea ch s a s a his le el by obse ing and epo ing
some o any exci ing cha ac e is ics abou a pa icula subjec (ma e ial, p og amming
language, he usabili y o echnology, and o he s.). Fo example, Addi i e Manu ac u ing
(AM) echnology has acqui ed ele ance and in e es in academia and indus y du ing
p e ious yea s because i allows he c ea ion o complex geome ies wi h cus omizable
ma e ial p ope ies.
A p ac ical example o he TRL 1 ela ed o AM would be a KTTO, uni e si y, o
company no icing he design e sa ili y o he echnology, he po en ial ligh ness o he
s uc u e, o he ma e ial p ope ies p oduced by he echnology.
TRL 2: Technology concep s and applica ion o mula ed.
Once he basic p inciples ha e been obse ed, he p ocess can be o mula ed o iden i y
some po en ial uses o he opic in ques ion (ma e ial, p og amming language, he us-
abili y o echnology, and o he s.). A his le el, he uses o applica ions a e s ill en i ely
specula i e, and suppo ing he o mula ions wi h mo e speci ic expe imen s o de ailed
analyses is s ill op ional.
24 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
Fig. 2.2: TRL and phases o de elopmen , based on [15].
Con inuing wi h ou example o AM echnology, he nex s ep ha i s a his le el is
eplacing an ac i e componen o a complex sys em, such as an engine componen , wi h
a pa designed and manu ac u ed using AM.
TRL 3: Analy ical and expe imen al c i ical unc ions and cha ac e is ic
p oo -o -concep .
A his le el, once he concei ed concep has been o mula ed, an ac i e esea ch and
de elopmen p ocess is ini ia ed o b ing he idea o ma u i y. I is hen necessa y o
include analy ical s udies o place he echnology in an app op ia e con ex and o
ca y ou labo a o y e i ica ion o alida e ha he analy ical p edic ions a e physically
co ec . In a ew wo ds, wha is sough on his le el is he p oo -o -concep ’ alida ion
h ough analy ical and expe imen al app oaches o applica ions o concep s o mula ed
a he p e ious le el.
Fo ou example o AM echnology, he nex s ep ha i s his le el is he design o
he piece o componen , alida ed h ough CAD/CAM/CAE so wa e simula ion.
TRL 4: Componen and b eadboa d alida ion in a labo a o y en i onmen .
Once he ’p oo -o -concep ’ is analy ically and expe imen ally alida ed, he p ocess o
con i ming i s unc ionali y inside he sys em s a s. This p ocess mus se e as he
bes suppo o he concep o mula ed and should accomplish he equi emen s o he
po en ial sys em applica ions. To ca y ou his ac i i y, specialized o ganiza ions o
companies should pa icipa e side-by-side in he e olu ion o he concep . No jus be-
cause o he knowledge and expe ience gained by pa icipa ing h oughou he e olu ion
o he concep bu also because, a his le el, he cos o he esea ch s a s o ise (de-
pending on he echnology), so some o mal sponso ship should be sough and a ained,
o example, h ough go e nmen s o indus y in es men s.
2.2. THEORETICAL FRAMEWORK. 25
In ou example o AM echnology, he s udy o he piece o concep ’s manu ac u ing
pa ame e s should be analyzed and comple ed. A he same ime, alida ion wi hin he
inal ensemble should be ca ied ou o ensu e ha he cha ac e is ics o he concep
mee he equi emen s o he inal sys em in which i will be ins alled.
TRL 5: Componen and b eadboa d alida ion in a ele an en i onmen .
The accu acy o he es ed concep is signi ican ly imp o ed by expe imen a ion a TRL
4, enabling in eg a ion in o he sys em wi h sui able suppo ing componen s and en-
abling es ing o he ull implemen a ion o he en i e sys em in a simula ed o ealis ic
se ing. This p ocess may include one o se e al new echnologies in he demons a ion.
This ac i i y should be ca ied ou in specialized acili ies only a ailable o o mal R&D
o ganiza ions o co po a e labo a o ies. Howe e , i should also in ol e o mal sponso -
ship needed in he p e ious TRL.
Communica ion be ween he a ious p ojec eams, bo h wi hin he o ganiza ion in
cha ge o he de elopmen o he p ojec and he o ganiza ion ha will ecei e he de-
elopmen , needs o be luid and unde s ood a di e en le els. Many o he ailu es
in echnology ans e s a a his le el, so a p ojec managemen app oach mus be
implemen ed o help sol e he p oblems ha may a ise in using he p o o ype o unde -
s anding he de elopmen and es ing p ocess o he inal applica ion. This demands
high echnology expe ise (de elopmen and implemen a ion) o he people in ol ed in
his p ocess should be high and commensu a e wi h he p ojec ’s needs.
Following he design o ou concep wi h AM echnologies, he nex s ep ha i s his
le el is he pa ’s manu ac u ing. AM p in e s a e expensi e de ices ha people wi h
speci ic knowledge can only ope a e. Fu he mo e, he necessa y acili ies o powde
handling, cleaning, and measu ing he pa a e no easy o ob ain due o hei high cos ,
so his p ocess mus be done in an R&D cen e o KTTO wi h his capabili y o in a
company’s AM lab.
TRL 6: Sys em/sub-sys em model o p o o ype demons a ion in a ele an
en i onmen .
The esul o he TRL 5 is a piece, a p o en concep , a ep esen a i e model, o a
p o o ype eady o be es ed o a echnology demons a ion in a ele an en i onmen .
This en i onmen depends en i ely on he de eloped concep s and is also aligned wi h
he p ojec ’s cos .
The demons a ion may ep esen a na u al sys em applica ion, o i may me ely be
a close ep esen a ion o he in ended applica ion while using he same echnology. This
ask demands new echnologies o be in ol ed in he demons a ion. Due o ha eason,
his ac i i y should be ca ied ou on specialized acili ies only a ailable o o mal R&D
o ganiza ions, KTTOs o co po a e labo a o ies. Howe e , i should also in ol e o mal
sponso ship needed in he p e ious TRL because o he inc easing cos s. Fo example,
acco ding o Mangkins [71], his ac i i y should be ca ied ou by app op ia e o mal
p ojec ized o ganiza ions ha can success ully manage he objec i es wi hin he ime,
cos and scope equi ed.
26 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
Fo ou concep de eloped wi h AM echnology, which a his le el is a sa is ac o-
ily manu ac u ed componen ha ul ils he manu ac u ing equi emen s, i is now he
ime o i o be u he es ed. A his le el, he piece’s po osi y, ac ion, oughness,
a igue, and ha dness analysis should be comple ed o p o e ha i is in line wi h he
equi emen s o be ins alled on he inal assembly.
TRL 7: Sys em p o o ype demons a ion in he expec ed ope a ional en i-
onmen .
The main objec i e o TRL 7 is ha he sys em, componen , p o o ype, o model com-
ple es a demons a ion in he expec ed ope a ional en i onmen . A his le el, he com-
ponen should be nea o a he scale o he planned ope a ional sys em, and he demon-
s a ion mus occu in he ac ual expec ed ope a ional en i onmen . This is done o
assu e he sys em enginee ing and de elopmen managemen con idence, one s ep u -
he han he pu pose o echnology R&D. Once again, his ac i i y can only be ca ied
ou in specialized acili ies only a ailable o o mal R&D o ganiza ions, KTTOs o co -
po a e labo a o ies, bu should also in ol e some o he o mal sponso ship needed in
he p e ious TRL. Mo eo e , i should be ca ied ou by app op ia e o mal p ojec ized
o ganiza ions ha can success ully manage he objec i es wi hin he ime, scope, quali y,
and cos equi emen s.
In ou example o he es ed p o o ype, he ac i i ies ha should be ca ied ou a
his le el o mee he objec i e a e he analysis o he componen s unde simula ed ope -
a ing condi ions, such as empe a u e changes, cu en s h ough he pa , and ib a ions
expe ienced, among o he s.
TRL 8: Ac ual sys em was comple ed and ‘quali ied’ h ough es s and
demons a ion.
Once he echnology eaches his le el, he sys em de elopmen o mos echnology i ems
is done. Once he p o o ype has been demons a ed o mee he sys em’s c i e ia, deci-
sions may be made o inco po a e i in o an exis ing sys em o o cons uc a comple ely
new sys em based on he p o o ype. Na u ally, hese specialized asks and decisions can
only be made a app op ia e specialized acili ies a ailable o o mal R&D o ganiza ions,
KTTOs o co po a e labo a o ies. Howe e , hey should in ol e some o mal sponso -
ship needed in he p e ious TRL. Fu he mo e, i should be ca ied ou by app op ia e
o mal p ojec ized o ganiza ions ha can success ully manage he objec i es wi hin he
ime, scope, and cos equi emen s.
Fo ou example o he componen manu ac u ed by AM Technologies, a his le el, ou
p o o ype has passed he a ious sa e y and pe o mance es s ca ied ou a specialized
si es close o o like he las ac o y whe e i would be p oduced.
TRL 9: Ac ual sys em ligh p o ed’ h ough success ul mission ope a ions.
All echnologies success ully being used in exis ing sys ems in any indus y ha e passed
he TRL 9. A his le el, once he p o o ype has passed all es s and p o ed ha i can
be sa ely in eg a ed wi h he sys em and will ul il he pe o mance equi emen s, i is
ins alled and es ed unde ope a ional condi ions. This ope a ion can gene a e new se s o
2.2. THEORETICAL FRAMEWORK. 27
bug- ixing p ocesses no iden i ied p e iously by he eam in cha ge o he de elopmen .
Howe e , his bug- ixing should be manageable i all p e ious TRLs we e passed.
Finally, in ou example, he componen designed and manu ac u ed u ilizing AM
echnologies can be sa ely ins alled on he inal planned sys em, and he bug- ixing
p ocess should s a .
Despi e he success o he TRL classi ica ion, some au ho s s a e some issues ega ding
he use o he ool. Sause e al. [72] s a ed ha he ool ”does no include any guidance
o he unce ain y ha may be expec ed in mo ing h ough he ma u a ion o TRL o
ha i does no compa e wi h any o he al e na i e o TRL” [73, 74]. Fu he desc ip ions
we e needed o cla i y he scope o each le el due o he o iginal desc ip ion o he ool
was conside ed ague, causing ambigui y in he unde s anding be ween he esea che s,
managemen and po en ial p og am use s.
The cen al and somewha es ic i e aspec ha has led o signi ican de elopmen s
and expansions in he use and desc ip ion o TRL is ha i is conce ned only wi h
assessing he ma u i y o indi idual echnology, lea ing aside whe e ha echnology is
loca ed wi hin a mo e ex ensi e sys em o how i in eg a es wi h o he echnologies. The
TRL scale was enhanced u he in 1995 wi h he a icula ion o he i s de ini ions o
each le el, coupled wi h examples o help wi h comp ehension [75]. Despi e i s ela i e
success, he Depa men o De ense o he Uni ed S a es in oduced he concep o Man-
u ac u ing Readiness Le el (MRL) o s eamline echnology ans e in mo e dynamic
manu ac u ing and o expand he o iginal TRL o inco po a e conce ns abou p oduc-
ion and isks associa ed wi h ime and manu ac u e o pa s, componen s, sys ems, and
echnologies.
The MRL is a me ic ha ensu es ha he de elopmen o enginee ing, design p ocess
and he ma u a ion o echnology can be associa ed wi h a manu ac u ing p ocess acil-
i a ing a quick and easy ansi ion and communica ion wi h he s akeholde s in ol ed
in he p ojec .
2.2.2.2 Abou p ojec managemen o Technology Readiness Le els 5 o 7.
De eloping new echnologies o en depends on he p e ious success o ad anced echnol-
ogy esea ch and de elopmen e o s. The co ec use o he TRL classi ica ion can also
be help ul no jus o know which skills ha e been acqui ed bu also o know i hese
skills can be used no only in a speci ic p ojec bu also in di e en p ojec s.
These de elopmen s ine i ably lead o ou signi ican challenges in each p ojec , pe -
o mance, quali y, schedule, and budge . Fi s , wi h a co ec isk analysis, ad anced
echnology de elopmen p ojec s educe he unce ain y in he so-called ’I on T iangle’.
Wi h such measu es, p ojec p og ess may be imp o ed by cos o e uns, schedule sho -
alls, and he g adual e osion o ini ial pe o mance goals [75]. The challenge o PMs is
o de e mine echnology eadiness and isk assessmen in a clea and well-documen ed
way and o be able o do so wi hin he p ecise s ages o he p ojec li e cycle.

28 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
As men ioned by Mankins [71] o TRL 5 onwa ds, a p ojec ized app oach is necessa y
o manage he p ojec ’s de elopmen e ec i ely. A his poin , a p e ious es ima ion
o he objec i es pu sued in e ms o cos , ime o scope/quali y can be s a ed. The
majo i y o p ojec de elopmen isks s a du ing hese s ages because he p ojec has
now en e ed an expe imen al phase and has le he esea ch phase behind.
Time and cos con ol wi hin R&D p ojec s has been a cons an conce n o companies,
go e nmen s, uni e si ies, KTTOS, esea ch clus e s, and esea che s. The pace wi h
which a concep de elops and he expenses associa ed wi h p ojec de elopmen is easie
o egula e in con ex s whe e ime, objec i es, and cos s can be speci ied nea ly om
he beginning. Howe e , in R&D, e en when he ime o cos cons ain s a e exceeded,
he p ojec can be conside ed a ailu e.
To inc ease he unce ain y in R&D p ojec s, e en i ime, cos o scope a e ul illed,
only some imes he quali y o he esea ch is adequa e o he esul o he p ojec , o
he di e en ac i i ies ca ied ou du ing i s li e cycle. Keeping ack o all he s ages
and ac i i ies ca ied ou in he p ojec is one way in which he quali y o he p ojec ’s
ou come can be, and again, only in some cases, conside ed a success. Howe e , a u he
p oblem a ises om his, measu es o success o R&D p ojec s a e associa ed wi h many
in e nal and ex e nal ac o s ha go beyond he i on iangle, as explained abo e.
On he o he hand, he o ganiza ional s a egy o uni e si ies and KTTOs in e ms
o de eloping esea ch lines, pa icipa ion in p ojec s o sea ching o alliances wi h
companies a e decisi e poin s o hei con inui y. The e o e, he de ini ion o KPIs o
an o ganiza ion ha ca ies ou high TRL p ojec s is i al o bo h he p ojec s’ success
and he collabo a ions’ success.
Fo hese easons, a common unde s anding o p ojec con ol me ics, luid commu-
nica ion, as well as de ined c i e ia ha de e mine he success o a p ojec and quali y
assessmen h oughou he p ojec , and he KPIs o he o ganiza ions ca ying ou hese
p ojec s (on he unde s anding ha companies ha e p e iously de ined and ac i ely
moni o hei own KPIs) is i al in o de o delinea e a sui able p ojec managemen
me hodology o R&D p ojec s and he success o he echnology ans e p ocess on
his TRLs.
2.2.3 P ojec managemen me ics and Key Pe o mance Indica o s o
R&D p ojec s.
The EU’s a emp s o accomplish he digi al ans o ma ion o he economy ha e gained
s eng h o egain he p i ileged posi ion in he wo ld ha i held yea s ago. Indus y
4.0, he solid echnological be made by Eas e n coun ies in collabo a ion wi h China,
and he inc easing pace o inno a ion in economies such as he Uni ed S a es, combined
wi h he economic c isis caused by he Co id-19 pandemic, will lea e in he coming yea s
he mos signi ican in es men in R&D in mode n his o y. 1. Funds ha will each local
1EUR 24 billion wo h o non- epayable g an s om he EU: Poland´s mos ex ensi e cohesion policy
p og amme app o ed by he Eu opean Commission - sho u l.a / ACNR
2.2. THEORETICAL FRAMEWORK. 29
go e nmen s o suppo he ans o ma ion o hei economy by suppo ing echnological
p ojec s led by indus y in collabo a ion wi h uni e si ies.
Due o he cu en en i onmen and he amoun o unding a ailable o hei im-
plemen a ion, i is essen ial o egula ly moni o how hese esou ces a e used o he
p ojec s’ de elopmen and he ou comes p oduced by UIC. Howe e , since he beginning
o he 1990s (wi h he i s s udy ca ied ou o measu e he pe o mance o p ojec s
ca ied ou in a UIC [76]), ew o he a emp s ha e been made in his a ea despi e i s
impo ance o he en i ies in ol ed and policymake s.
Fu he mo e, digi al ans o ma ion and de elopmen o highly echnical p ojec s a e
cha ac e ised by being pa o a highly in e connec ed se o subsys ems wi h high cos s,
p oduced a a low olume, ha equi e comp ehensi e and p o ound knowledge and
skills, in ol ing mul iple collabo a o s and main aining a con inuous in eg a ion be ween
he cus ome and he supplie .
Nowadays, a ully s uc u ed and widely accep ed sys em o indica o s is s ill needed
o e alua e he esul s o he UIC [77]. A i s a emp o measu e he pe o mance o
collabo a ions o implemen ing R&D p og ammes and p ojec s, oge he wi h a me hod
o measu ing i , was made by Fe nandes e al. [26]. On he o he hand, he s udy is
based on he c ea ion and de elopmen o a heo e ical echnique bu needs an ac ual
demons a ion o alida ion. Pe kmann e al. [78] iden i ied ou s ages o UIC, de eloped
a success map explaining how hese collabo a ions wo k and iden i ied cause-and-e ec
ela ionships o hei success. In addi ion, a se o pe o mance indica o s was p oposed
o each collabo a ion s age.
Cu en p ojec managemen ools and echniques ha e p o en inadequa e and insu -
icien o moni o ing he de elopmen o highly echnical and R&D p ojec s. I hese
a e no ai h ully con olled and measu ed du ing he p ojec li e cycle, hey can ac
agains hei s anda d de elopmen [79]. Addi ionally, inno a ion mus be e alua ed us-
ing a wide ange o indica o s because i is a mul idimensional and complex no ion ha
does no i wi h ypical measu emen s [80]. Addi ionally, he complexi y o inno a ion
inc eases wi h he he e ogenei y o he pa ies in ol ed in a UIC, as we ha e al eady
desc ibed in he con ex ual pe spec i e o his ype o collabo a ion (2.2.1.1).
Some esea ch sugges s ha measu emen s can be bene icial o inno a ion, a guing
ha hese measu es can help manage s o con ol asks, p ocesses and ou comes, ensu ing
ha inno a ion is well-suppo ed and ca ied ou e icien ly. Fo example, B owning &
Ramasesh [81] concluded in hei esea ch ha many exis ing models ocus mo e on he
ac i i ies pe o med han on in e ac ions o p ojec deli e ables because humans end o
pay mo e a en ion o ac i i ies ha can be measu ed, and ha can lead o a conc e e
esul .
Ano he esea ch line sugges s ha measu emen can dissuade manage s om seek-
ing o deepen inno a ion and ob ain mo e inno a i e esul s as a sho - e m ewa d.
Acco ding o se e al s udies, inno a ion measu emen hinde s inno a ion by pushing
o ganisa ion membe s o concen a e hei a en ion oo na owly and lose sigh o he
b oade ocus ha i should ha e.
30 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
Wha should we do? Which way o go? Expe ience dic a es, ega dless o he en i on-
men we a e alking abou , ha wha canno be measu ed canno be imp o ed. Howe e ,
he e is a balance o be s uck. Too much measu emen du ing he p ojec li e cycle can
be de imen al o he no mal de elopmen o he p ojec , especially when hese p ojec s
a e de eloped in SMEs o small p ojec eams o a e managed by PMs who do no ha e
he app op ia e o necessa y expe ience o manage and pe o m hese measu emen s, as
he excessi e pape wo k a ound he wo k can be o e whelming o bo h he PMs and he
p ojec eam, which is why, o example, adi ional p ojec managemen me hodologies
canno be applied o all ypes o p ojec s, and nei he do all p ojec eams.
In gene al, p ocess o pe o mance me ics o measu e he ma u i y o new echnolo-
gies and sys ems ha e ye o be ully de eloped. Cu en ly, he echniques and ools
a ailable, such as Quali y Func ion Deploymen [82], Concep Selec ion P ocess [83],
Fi s Requi emen s Elucida o Demons a ion (FRED) [84], In eg a ed Design Model
[85], Subsys em T adeo Func ional Equa ion [86], Design o Manu ac u abili y [87],
Design-Build-Tes Cycle [88]. Pe iodic P o o yping [89], cos as an Independen Va i-
able o CAIV [90], and Lean P oduc De elopmen Flow [91], a e agmen ed and no
used consis en ly h oughou he p ocess [92].
I is di icul o es ablish con ol mechanisms ha can e ec i ely a ec he de elop-
men o highly echnical p ojec s and R&D p ojec s due o he lack o p ocess me ics,
high unp edic abili y, eliance, and lack o consis ency [93].
Howe e , can he managemen o R&D p ojec s be measu ed? Unde s anding he
limi a ions and unique cha ac e is ics o his ype o p ojec and he de elopmen o his
hesis p ojec , we belie e i is possible. Some companies belie e he same and measu e
some aspec s o he p ocess. On he o he hand, esea che s ha e also discussed he
same ques ion and ha e con ibu ed o he scien i ic li e a u e by deba ing wha kind
o measu emen is bene icial. S ill, a e decades o esea ch, he conclusions a e mixed
and need cla i ica ion.
Ha ing p ocesses o planning and moni o ing p ojec s is necessa y o gua an ee he
success o he e o s made on R&D. Coope & Kleinschmid [94] concluded in hei
esea ch ha p ojec plans, ask scheduling, moni o ing, and eedback a e among he en
c i ical ac o s o he success ul de elopmen o a p ojec . D i & Lechle [95] concluded
ha e iciency (schedule, money, and scope), pe cei ed alue, and cus ome sa is ac ion
a e all posi i ely impac ed by he planning quali y. Pin o & Man el [96] ound ha
o R&D p ojec s, ine icien scheduling o asks is s ongly ela ed o ailu es in he
implemen a ion p ocesses and ha bo h moni o ing and eedback can impac cus ome
sa is ac ion. Fu he mo e, he lack o an app op ia e plan makes i di icul o con ol
he de elopmen p ocesses ha can lead o cos o e uns, delays, o ailu es in p ojec
implemen a ion as has al eady happened in some go e nmen p ojec s, R&D e o s [97]
and new p oduc de elopmen [98].
Main aining a comp ehensi e pic u e is especially c ucial du ing he ea ly phases
o de eloping an R&D p ojec when unce ain y is s ill high, bu co ec i e measu es
may s ill be add essed. PMs mus apply his app oach o he p ojec o enable hem o
2.2. THEORETICAL FRAMEWORK. 31
measu e he p ocess and con ol he de elopmen o he sys em h ough p ope planning,
scheduling, and moni o ing [79].
In his esea ch, we will ocus on h ee aspec s o unde s and his holis ic iew o
managing and con olling an R&D p ojec , he s udy o KPIs, P ojec Success and
Ea ned Quali y Me hod (EQM).
2.2.3.1 Key Pe o mance Indica o s.
A he o ganisa ional le el, he de elopmen o measu emen sys ems is necessa y o
se objec i es and moni o he e ec i eness and e iciency o he use o esou ces. Com-
monly, hese me ics ake he o m o KPIs, which p o ide an objec i e c i e ion o
o ecas ing, measu ing and planning he ac i i ies ca ied ou in he company. I should
be emphasised, hough, ha pe o mance me ics’ goals, de ini ions, and con en s di -
e . Since hey mus i he compe i i e en i onmen and s a egy, many app oaches a e
u ilised o design and choose business KPIs [11].
Wi hin he scien i ic li e a u e, i is possible o ind se e al desc ip ions o KPIs ca -
ego ies. Co es e al. [99], iden i ied i e s a egic ca ego ies o KPIs:
•Cos ;
•quali y;
• lexibili y;
•s ock;
•lead ime.
Wi h hese ca ego ies, he e is an in en ion o cap u e he o ganisa ion’s s a egic
objec i es and enable alignmen wi h ac ical, s a egic, and ope a ional pe o mance.
Fo example, Too & Ogunlana [100] ound di e en au ho s who included cus ome
and s akeholde sa is ac ion as p ojec success c i e ia in addi ion o he adi ional i on
iangle (Time, Cos , Quali y).
Wi hin p ojec s, Ke zne [101] iden i ies ime, cos , esou ces, scope, quali y and ac-
i i ies as c i ical me ics o p ojec managemen KPIs. Fo example, Too & Ogunlana
[100] enhances he p ojec eam’s capaci y o con ol p ojec isks and ind solu ions o
issues ha a ise du ing he p ojec li e cycle o e alua e he p ojec ’s success. Techni-
cal e iciency o execu ion, managemen and o ganisa ional implica ions, s a g ow h,
pa ne s’ echnological capabili ies, and o ganisa ional pe o mance is also conside ed
in measu ing p ojec success. Howe e , when using hese ancy me ics, con en ional
me ics such as cos , schedule, quali y, and secu i y should be add essed [102].
When discussing he in eg a ion o KPIs in o ganisa ions, Too & Ogunlana [100]
highligh ope a ional, li e cycle, s a egic and socio-economical aspec s. The au ho also
assu es ha he c i e ia o measu ing he success o p ojec s should be based on s a egy,
sus ainabili y and secu i y.
38 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
be buil up cumula i ely du ing p ojec implemen a ion. The EVM concep is applied
by ini ially de e mining he expec ed inal quali y and b eaking i down in o quali y
a ibu es ha a e hen associa ed wi h pa icula p ojec ac i i ies, hus c ea ing a
quan i a i e link be ween he pe o mance wi hin he p ojec and he esul ing quali y
and allowing quali y objec i es o be moni o ed.
I was he me hod o Paquin e al. [3], which in oduced he concep ha quali y is a
measu able concep ha is p og essi ely buil up o e he li e cycle o he p ojec , om
which se e al me hodologies ha e been de i ed ([125, 126, 127, 128, 129, 130]). Howe e ,
hey ha e ye o be di ec ed owa ds R&D p ojec s. Only Schuh e al. [129] used he
p inciples se ou by Paquin e al. [3] o assess he e ec s o de ia ions in ac i i ies on
he objec i es o a de elopmen p ojec .
EQM has se e al ea u es ha make i use ul o use in R&D p ojec s. Fo example, i
allows PMs o elucida e and s uc u e cus ome needs and expec a ions as i decomposes
o e all cus ome sa is ac ion in o a hie a chical s uc u e o quali y c i e ia. Fu he mo e,
acco ding o he p inciple ha quali y is achie ed p og essi ely h oughou he p ojec ,
he me hod can agg ega e lowe -le el quali y c i e ia in o highe -le el quali y objec i es.
On he o he hand, he me hod p o ides a me hod o e alua ing he planned and ea ned
quali y o he p ojec deli e able h oughou i s li e cycle. Because he quali y c i e ia,
which we e join ly es ablished by he PMs and s akeholde s and agains which he p ojec
ac i i ies a e e alua ed, a e achie ed g adually and cumula i ely as he p ojec is ca ied
ou , making i easie o es ima e he e o equi ed o comple e hem. Addi ionally, i
o e s measu es o quali y de ia ions. Finally, decisions on he ade-o be ween quali y,
ime, and cos a e made mo e accessible in his way [131].
In o de o success ully ca y ou an R&D p ojec , he PM mus ini ia e an in e ac i e,
lexible and esponsible con ol o he p ojec plan, ask scheduling, moni o ing and
e alua ion om he beginning. The plan should be ca ied ou in a manne ha is
consis en wi h he expec ed p ojec esul s, should no be unduly cons aining, and
mus be based on miles ones. Howe e , as Magnaye e al. [79] conclude, he e should
al eady be a high le el o con ol o e he p ocess du ing he de elopmen o he ea ly
s ages o he p ojec , as well as a g ea e emphasis on he applica ion o he chosen
p ocess me ics and lessons lea ned, in o de o iden i y p oblem a eas and add ess hem
quickly. So, hanks o he esea ch ha has been going on, he EQM app oach is a
sui able app oach o manage his ype o p ojec because o he cha ac e is ics we ha e
al eady men ioned.
Howe e , he e a e se e al aspec s o be aken in o accoun when applying EQM o a
p ojec :
1. I is necessa y o gua an ee ha he p ojec ’s p og ess is shown g aphically;
2. he e alua ion o he quali y o he asks is subjec i e, so i is necessa y o explo e
objec i e echniques and measu es ag eed upon be ween he PMs and he s akehold-
e s ha allow absolu e cla i y on he pa ial and inal esul s o he p ojec . Again,
communica ion plays a signi ican ole in he managemen o hese p ojec s;

2.2. THEORETICAL FRAMEWORK. 39
3. ime and cos a e no aken in o accoun in he EQM, bu knowing he p ojec ’s
quali y c i e ia in ad ance, and ha ing a comp ehensi e unde s anding o i , enable
PMs o explain wha e o s would be equi ed o ca y ou he asks and achie e he
desi ed quali y; The need o mo e cla i ica ion on de e mining he quali y c i e ia
o assessing he p ojec , acco ding o he s udy by Paquin e al. [3], is no o ious,
so app oaches aimed a sol ing his impasse should be implemen ed. The e o e, he
app oach and assessmen o quali y c i e ia om he poin o iew o PMs and s ake-
holde s is o i al impo ance o he success o he p ojec [10] and o he EVM.
2.2.4 R&D P ojec Managemen Me hodology.
As we ha e p e iously analysed, hanks o he policies implemen ed by some coun ies
o p omo e and sus ain UIC, many academic con ibu ions ha e ied o explain, un-
de s and and jus i y hese in e ac ions in economic e ms. Howe e , only a ew s udies
ha e been conduc ed o examine he ac o s ha in luence bo h he pa icipa ion o
companies and uni e si ies in R&D p ojec s, he cha ac e is ics o hese p ojec s, and
he managemen me hods used in hese p ojec s.
We mus begin wi h he ac ha he discipline o p ojec managemen is cons an ly
e ol ing in esponse o he p ojec i ica ion o socie ies and o he needs, unce ain ies
and changes ha a ise om yea o yea , which means ha he alidi y and imeliness o
he me hodologies and p ac ices de eloped a e quickly being le behind. Now, speed and
agili y a e cha ac e is ics demanded om p ojec eams and manage s as equi emen s
o implemen ing he a ious echnologies ha make up he ou h Indus ial Re olu-
ion. All his has led o con inuous changes in s anda ds, me hodologies, p ac ices and
me hods ela ed o p ojec managemen , o main ain and inc ease p ojec success a es.
Howe e , he e a e wo c ucial aspec s o ake in o accoun . Fi s ly, he managemen o
he UIC di e s om he managemen o he UIC p ojec s; while o he i s aspec , he
con ac ual clauses o managing he esul s and in ellec ual p ope y igh s, he us
be ween he pa ies o he expe ience o p e ious collabo a ions, a e some o he cha -
ac e is ics ha de e mine hese collabo a ions; o he second aspec , he managemen
o R&D p ojec s depends en i ely on he o ganisa ions, hei PMs, and he app oaches
used in he me hodologies.
Using classical, agile o hyb id me hodologies b ings di e en ad an ages and disad-
an ages o he p ojec . The e s ill needs o be a s anda d me hodology in he li e a u e
o apply o his ype o p ojec . The e m ”one size i s all” does no wo k in p ojec man-
agemen . Howe e , a i s e o was de eloped by he Eu opean Commission, which in
2016 launched he i s edi ion o he PM2[132], a hyb id p ojec managemen me hod-
ology applicable o any p ojec , and de eloped aking in o accoun he needs, cul u e
and cons ain s o he EU o deli e solu ions and bene i s o o ganisa ions e ec i ely
managing wo k h oughou he p ojec li e cycle. This could be classi ied as he i s
me hodology c ea ed and app o ed by a go e nmen al egula o y body o manage UIC
p ojec s in e na ionally. Howe e , some sho comings in e ms o he adap abili y and
ligh ness o he documen a ion o i s lack o p ac icali y when i comes o digi isa ion
40 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
and i ualisa ion ha e mean ha i s dissemina ion and use ha e no sp ead o he le el
ini ially in ended.
Na u ally, his aises he ques ion, ”And i he egula o , who is he one se ing he
es ic ions and equi emen s, does no ha e he answe o he ques ion o how o manage
p ojec s bes o mee he needs hey c ea e, hen whe e can we ind i ?” Un o una ely,
he e has ye o be a sa is ac o y esponse o his ques ion.
Al hough p ojec managemen , by i s na u e, ies o ocus on con olling and min-
imising he isk o a p ojec de ia ing om he de eloped plan and no achie ing i s
objec i es, his is no easy o achie e. P ojec managemen in ol es wo aspec s: plan-
ning a new ’en e p ise’ and execu ing i s execu ion. Fo he o me , acco ding o he
PMBOK [133], he main elemen s o con ol a e ime, cos , quali y, esou ces, commu-
nica ions, isk and p ocu emen . Fo he second aspec , in o med decision-making and
plan adjus men s acco ding o needs a e some o he mos ele an ea u es.
Acco ding o he PMBOK de ini ion [133], a p ojec managemen me hodology is a
sys em o p ac ices, echniques, p ocedu es and ules used by hose wo king in a disci-
pline, such as PRINCE2, Sc um o Kanban. In addi ion, p ojec managemen me hod-
ologies, in gene al, a e based on ca ying ou a se ies o ac i i ies in a speci ic o de
and applying knowledge, skills, ools and echniques o i , ying o make he mos o
he echnical and human esou ces a ailable o mee he p ojec ’s objec i es. Adequa e
p ojec managemen is he one ha occu s when p ojec planning allows o eseeing and
co ec ing in ime he mos signi ican numbe o un o eseen e en s ha may a ise.
Howe e , in an R&D p ojec , he esea ch esul s may di e d ama ically om wha
was an icipa ed a he beginning bu a e s ill bene icial o he o ganisa ion—because
o his, applying adi ional o exclusi ely agile me hodologies o his kind o p ojec is
complex.
The academic li e a u e s esses he impo ance ha an adequa ely cons i u ed mon-
i o ing and con ol me hodology o an R&D p ojec should ocus on he ma u i y o he
echnologies, he in eg a ion elemen s and he sys em as a whole. On he o he hand,
hey ag ee on he need o main ain a non-linea s uc u e and he idea ha de ailed
planning a he beginning o he p ojec is a challenging and un ui ul ask wi hou
he igh ools. Thus, phased li e cycle app oaches a e necessa y o isualise he p ojec
managemen p ocess. A non-linea app oach mus be de ined o allow mo e c ea i i y,
lexibili y and changes in he o e all ideas. To o e come he dynamic scope and deal
wi h changes, he ini ial planning, he de ini ion o he me hodologies and echnologies
employed, and he equi emen o an inc emen al and i e a i e esea ch phase a e all
c ucial.
An R&D p ojec managemen me hodology should be lexible, de ine miles ones based
on esea ch ma u i y o p oduc de elopmen , be in e ac i e, and be able o espond and
adap o changes in echnology and equi emen s. This can be acili a ed by an in e ac i e
p ojec managemen me hodology ha p omo es cong uence o objec i es and enhances
lea ning by c ea ing an in o ma ion in as uc u e wi h p ocess pe o mance me ics
linked o he o ganisa ion’s s a egy.
2.2. THEORETICAL FRAMEWORK. 41
Va ious me hods ha e been discussed and de eloped o manage R&D p ojec s. Phased
P ojec Planning (PPP), a con ol mechanism o new p oduc de elopmen , was in o-
duced by NASA o ensu e ha p ojec s a e execu ed acco ding o plan and deli e ed on
ime. Howe e , his enginee ing-based app oach needs o be as e and mo e bu eauc a ic
[134]. Coope hen p esen ed he s age managemen sys em app oach [135], ocusing on
quali y and equi ing ha each s age’s inpu s and ou pu s be e alua ed, es ed, and ap-
p o ed be o e going on o he nex s age o sol e hese es ic ions. Al hough his s a egy
limi s op ions o c ea i i y and inno a ion, a hyb id p ojec managemen me hodology
ha combines agile and con en ional me hods is ecommended.
The cha ac e is ics o a sui able me hodology, acco ding o Ke zne [136], include he
ecommended le el o de ail, he use o empla es, s anda dised planning, ime man-
agemen and cos con ol echniques, s anda dised epo ing, lexibili y o use ac oss
p ojec s, lexibili y o apid de elopmen , use -unde s anding, accep ance, and usabil-
i y wi hin he o ganisa ion, he use o s anda dised p ojec li e cycle phases, and he
asse ion ha i is based on guidelines.
This can be achie ed wi h agile, hyb id o classical me hodologies and app oaches
( o a lesse ex en ). Agile app oaches a e now sp eading ac oss he discipline, showing
signs o imp o ing p ojec success, and a e inc easingly being used in indus ies o he
han so wa e de elopmen [4]. P elimina y p ojec esul s a e speci ied, ini ial goals a e
es ablished, and p ojec ou comes a e con inuously assessed and imp o ed by u ilising
adap i e p ocedu es in an agile app oach. An essen ial aspec o agile me hodologies is
he dis ibu ion o esponsibili y among p ojec membe s and he inclusion o p ojec
s akeholde s in bo h o mal and in o mal communica ions a ound he p ojec [137].
On he o he hand, hyb id app oaches ha e simila e ec i eness o pu ely agile ap-
p oaches. While accomplishing he same goals in e ms o budge , ime, scope, o quali y,
analysis by Gemino e al. [67] e ealed ha hyb id and agile app oaches conside ably
boos s akeholde sa is ac ion o e adi ional ways. Fo example, a hyb id p ojec man-
agemen me hodology combines p ac ices and me hodologies om mo e han one p ojec
managemen app oach, seeking o use he bes p ac ices om each app oach o imp o -
ing he o e all esul s o he me hodology c ea ed.
Rega ding he hyb id app oaches and me hodologies de eloped, se e al p oposals ha e
been de eloped in he scien i ic li e a u e o imp o e he managemen o R&D p ojec s.
Fo example, Mikulskiene [20] de eloped an app oach in which hese p ojec s we e man-
aged in wo phases. The i s phase, planning, was associa ed wi h issues such as human
esou ces, s akeholde s, pa ne s and eams. In con as , he second phase ocused on
he p ojec ’s echnical de elopmen s.
On he o he hand, Mosb ooke [66] ep esen ed he p ojec managemen li e cycle in
ou phases, which included sepa a e concep s o p ojec planning, execu ion and com-
ple ion. The au ho u he ecommended se ing abs ac objec i es, main aining lexi-
ble planning and ocusing on cons ain s and he en i onmen . Ke zne [112] ep esen s
he new p oduc de elopmen li e cycle in i e phases: concep de elopmen , planning,
42 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
es ing, implemen a ion and closu e; i also ecommends he o e lap be ween he phases
and he di ision o long- e m p oduc de elopmen p ojec s in o smalle p ojec s.
The six-phase li e cycle, which places a s onge emphasis on managing R&D p ojec s,
is likewise consis en wi h he me hods men ioned abo e. Al hough ou -, i e- o six-
phase li e cycles a e de ined wi h de ailed in o ma ion, hei sui abili y is ques ioned as
hese app oaches main ain a non-linea s uc u e, whe eas R&D p ojec s canno con in-
ually be de eloped wi h linea app oaches.
Gu ie ez e al. [138] de eloped a me hodology in which hey combine classical p ojec
managemen heo ies and some o he bes p ac ices o Sc um. This me hodology includes
he phases o de ini ion, design, de elopmen , es ing and elease, wi h an emphasis on
apid cus ome eedback de i ed om he Sc um app oach in he de elopmen phase
and con ol o aspec s occu ing in he elease phase, as well as sa ing esou ces h ough
edesign cycles, unc ionali y and usabili y es ing o deli e ables du ing sp in s in he
de elopmen phases. Wi h a simila app oach, Coope [139] ep esen ed he T iple A
(Adap i e, Agile and Accele a ed) app oach, a hyb id s uc u e o S age-Ga e and agile
app oach, which comes om he adap a ion o he eam o he con ex o he p ojec ,
and he agili y ha occu s du ing i e a ions and spi als.
On he o he hand, du P eez & Louw [140] ep esen ed he Fugle inno a ion app oach,
which combines he s aged app oach wi h he agile app oach. They ely on he inno a ion
p ocess being ca ied ou in e nally bu connec ed o he ex e nal en i onmen and
ou sou cing, enabling o e lapping s ages and i e a i e loops. Somme e al. [141] also
in oduced indus ial Sc um, combining a s aged app oach and Sc um. In his app oach,
he o ganisa ional le el applies he S age ga e app oach, while he Sc um app oach is
used a he p ojec le el.
Howe e , when discussing he c ea ion o hyb id me hodologies, he ques ion emains
as o how he decision o combine wo o mo e app oaches is made. Fijin [142] ep esen ed
a model in which decision-making on he combina ion o app oaches is acili a ed. The
linea s uc u e and deg ee o en i onmen al con ol a e he main axes o decision-
making in his model.
Acco ding o We ne [143], he s a us o an R&D p ojec and i s imp o emen po-
en ial can be iden i ied by applying an in eg a ed planning, managemen and con ol
sys em in he R&D en i onmen . In o de o ca y ou e icien p ojec moni o ing and
o examine he cu en p ojec si ua ion, se e al ac o s sugges a con inuous assess-
men o su e p e-de ined KPIs [144]. The me hod de eloped by Paquin e al. [3] di ec ly
add esses he challenges o con olling he quali y achie ed in a p oduc de elopmen
p ojec . On he o he hand, as we ha e seen, he equi emen s o each le el change, as
well as he ecommended p ac ices o ca y ou . I is c ucial o know a wha poin o
he TRL he p ojec is ca ied ou o a oid ailing o mee he objec i es planned in he
p ojec . Ano he impo an aspec is o unde s and wha p ojec success is and how i
is e alua ed acco ding o he o ganisa ion ca ying ou he p ojec and i s s akeholde s.
Fu he mo e, i is i al o know no only how i is e alua ed bu also how his s a e o
”success” is eached. The analysis o da a h oughou he p ojec , he abili y o PMs o
2.2. THEORETICAL FRAMEWORK. 43
make imely, da a-d i en decisions, and he comple ion o deli e ables as echnology o
p oduc ma u i y le els de elop wi hou cons aining p ojec eams and manage s wi h
s ingen p ocesses and demanding and exhaus i e documen a ion will help o mee
p ojec objec i es and sa is y he a ious p ojec s akeholde s.
Finally, he esul s o he Gemino e al.[67] s udy alida e p ac i ione s’ decisions o
combine agile and adi ional p ac ices and sugges ha hyb id app oaches lead he way
in app oaches o p ojec managemen .

44 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
2.3 Resea ch me hodology.
The analysis and unde s anding o each o he i ems men ioned abo e will help o si ua e
us wi hin he concep ual model o he esea ch. The componen s and in e connec ions
be ween hese opics a e desc ibed (Figu e 2.3), wi h he inal objec i e o he esea ch:
he de elopmen o a me hodology o R&D p ojec managemen using quali y c i e ia
as a pe o mance indica o .
Fig. 2.3: Concep ual model o he esea ch
In each o he publica ions p esen ed in his hesis, he ollowing da a collec ion me h-
ods we e used (Figu e 2.4):
2.3. RESEARCH METHODOLOGY. 45
Fig. 2.4: Collec ing me hods
As seen in he g aph abo e, he esea ch me hodology ollowed in each publica ion
a ied, in he Fi s Publica ion: ”Pa e ns o In e na ional Coope a ion be-
ween Inno a ion Clus e s. Cases o CFAA and uh alley” (See Sec ion 6.1 and
Figu e 2.5) he case s udy ha ou lined how he collabo a ion be ween he wo clus e s
should p oceed was de eloped as he s a ing poin . A li e a u e e iew was equi ed o
suppo gene al ecommenda ions o coope a ion, and i was hen ollowed by a sys-
ema ic analysis o he wo clus e s o iden i y any sha ed cha ac e is ics. This analysis
hen allowed he de elopmen o c i ical ac ions o he wo en i ies’ coope a ion.
In he case o he Second Publica ion: ”Building Coope a ion be ween In-
no a ion Clus e s Based on Compe ences Requi emen s. Case o CFAA and
uh alley.” (See Sec ion 6.2 and Figu e 2.6), a se ies o in e iews on hese compe en-
cies we e conduc ed wi h h ee ocus g oups, di ided acco ding o expe ience, ole, and
echnology knowledge, in which hey we e asked o iden i y he compe ences needed o
46 CHAPTER 2. THEORETICAL FRAMEWORK AND METHODOLOGY.
Fig. 2.5: Fi s Publica ion: ”Pa e ns o In e na ional Coope a ion be ween Inno a ion
Clus e s. Cases o CFAA and uh alley” - Resea ch Me hodology
pa icipa e and manage a collabo a i e R&D p ojec . Suppo ed by a li e a u e e iew,
hey answe ed he esea ch ques ions and achie ed he publica ion’s objec i e.
Fig. 2.6: Second Publica ion: ”Building Coope a ion be ween Inno a ion Clus e s Based
on Compe ences Requi emen s. Case o CFAA and uh alley.” - Resea ch Me hodology
2.3. RESEARCH METHODOLOGY. 47
In he Thi d Publica ion: ”TRLs 5–7 ad anced manu ac u ing cen es, p ac-
ical model o boos echnology ans e in manu ac u ing.” (See Sec ion 7.1), a
benchma king s udy on compa able cen es in Eu ope was conduc ed o e alua e a ious
aspec s o scien i ic capaci y, alida ion o esul s, esea ch capaci y, equipmen sui abil-
i y, echnical quali y o he equipmen , a ailabili y o he equipmen , and capaci y o he
same in o de o comp ehend he con ex in which a esea ch clus e ocused on high
TRLs de elops. Wi h he da a collec ed, an analysis was conduc ed and, based on a
bibliog aphical e iew, a se ies o ecommenda ions we e made o he Basque Coun y
o c ea ing and s eng hening his ype o cen e.
Fo he Fou h Publica ion: ”Assessing he success o R&D p ojec s and in-
no a ion p ojec s h ough p ojec managemen li e cycle.” (See Sec ion 8.1),
in o de o assess he e ec i eness o p ojec s comple ed in a esea ch clus e om he
pe spec i e o PMs, a su ey was ca ied ou . Based on a li e a u e e iew, he success
c i e ia o R&D p ojec s we e ansla ed in o quali a i e ques ions anked on a Like
scale. The alida ion and eliabili y o he su ey we e con i med h ough an analysis
pe o med wi h Sma PLS so wa e o alida e he ques ions measu ing he con ibu ion
o each dimension o p ojec success o he success o he ealised p ojec s.
Wi h he inpu o he da a analysis, i was possible o iden i y he mos c ucial success
ac o s o he esea ch clus e p ojec s and p oduce ecommenda ions o he p ojec
me hodologies ha would be c ea ed in he u u e.
In he Fi h Publica ion: ”P ojec Success C i e ia E alua ion o a P ojec -
Based o ganisa ion and I s S akeholde s—A Q-Me hodology App oach.” (See
Sec ion 8.2 and Fig 2.7, a su ey was conduc ed using Q-Me hodology (a s a is ical semi-
quan i a i e echnique) in which he PMs o he esea ch clus e and he key s akeholde s
we e asked o ank he p e iously iden i ied success c i e ia in o de o impo ance. This
su ey was based on wo esea ch ques ions, a li e a u e e iew o he mos c i ical
success c i e ia o he ealisa ion o R&D p ojec s in he con ex o collabo a i e p ojec s
wi h public-p i a e o ganisa ions and a li e a u e e iew o he success c i e ia. Semi-
s uc u ed in e iews we e conduc ed as pa o he su ey o e i y he applicabili y o
he chosen c i e ia and ga he da a ha would be c ucial in selec ing he o de o he
componen s. A subsequen analysis o he da a allowed us o ca ego ise he pa icipan s
based on he simila i ies and di e ences in he pa icipan s’ pe spec i es on h ee ac o s
(g oups), as well as o de e mine which we e he mos c ucial success c i e ia o his
ype o p ojec o bo h s akeholde s and PMs in he inno a ion clus e .
In he Six h Publica ion: ”Iden i ica ion o key pe o mance indica o s in
p ojec -based o ganisa ions h ough he lean app oach.” (See Sec ion 8.3), a Sys-
ema ic Li e a u e Re iew (SLR) was ca ied ou wi h he p ima y pu pose o s udying
he ela ionship be ween p ojec success, lean and pe o mance indica o s in a p ojec -
based con ex . The cu en s a e-o - he-a was iden i ied and examined, da abases o
be consul ed, and keywo ds o be included in he sea ch que ies we e de ined. Subse-
quen ly, he iden i ied documen s we e selec ed acco ding o de ined exclusion c i e ia.
Once he documen s o be s udied had been de ined, he publica ions we e analysed
using hema ic analysis and syn hesising he in o ma ion collec ed.
54 CHAPTER 3. HYPOTHESIS AND OBJECTIVES.
5. o iden i y he KPIs o p ojec -based o ganisa ions and how hese KPIs can in luence
he de elopmen o a p ojec .

4
Summa y and discussion o he esul s.
4.1 Summa y and discussion.
In his sec ion, we will summa ise he conclusions and discuss he di e en con ibu ions
made du ing he de elopmen o he hesis, as seen in he Resea ch Onion o he Thesis
(Fig 4.1).
The s uc u e o his sec ion ollows he indica ions published in he Chap e XI.
Thesis by published pape s -Regula ions Go e ning he Managemen o Doc o al S udies1
1h ps://www.ehu.eus/en/web/dok o egoa/doc o al- hesis/ hesis-by-published-pape s - Las Access:
20/12/2022
56 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
Fig. 4.1: Resea ch Onion
4.1. SUMMARY AND DISCUSSION. 57
4.1.1 Fi s Publica ion CP.1 - Con e ence Pape : ”Pa e ns o in e na ional
coope a ion be ween inno a ion clus e s. Cases o CFAA and uh alley”.
The i s publica ion (See Sec ion 6.1) e e s o he sea ch o pa e ns o in e na ional
coope a ion be ween inno a ion clus e s. This publica ion explo es di e en possible
means o collabo a ion be ween he CFAA and uh alley (inno a ion clus e loca ed
in he Ruh a ea, Ge many 2). Fu he mo e, i shows how wi h he knowledge and ex-
pe ise gained om he p ojec s, he clus e s can imp o e hei capabili ies and help
hei pa ne s h ough he apid and be e -applied use o knowledge wi h a new se o
skills om his ype o collabo a ion. I is also men ioned how in e na ional join bidding
can help imp o e he ela ionship be ween he expe ise o bo h pa ies and help connec
egions and hei pa ne s in inno a ion ecosys ems.
This is consis en wi h wha is s a ed in he academic li e a u e (See Sec ion 2.2.1) and
he Eu opean Union [17] in e ms o collabo a ion be ween R&D o ganisa ions. Apa
om ecommending a ein o cemen o in e na ional scien i ic collabo a ion o inc ease
scien i ic p oduc i i y and knowledge ans e , hese collabo a ions a e men ioned as one
o he mos impo an channels o dissemina ing and aluing knowledge. They also alk
abou how his ype o collabo a ion gene a es be e aining o o ganisa ions’ human
capi al, a mo e signi ican gene a ion o pa en s and mo e excellen scien i ic p oduc ion,
among o he aspec s.
The publica ion also men ions he need o inno a ion clus e s o gain compe i i e ad-
an age, imp o e e iciency and show apid, posi i e and aluable esul s can be achie ed
h ough such collabo a i e a angemen s. We also e e o he many di icul ies ha need
o be o e come, he lack o balance in e ms o capaci ies o cope wi h aluable esea ch,
di e en ins i u ional cul u es, di e si ica ion o esea ch ac i i ies, con lic s o in e es ,
and isks gene a ed om he publica ion o esea ch esul s. Howe e , i is also men-
ioned ha hese di icul ies can be o e come wi h a conc e e de ini ion o objec i es
om he ou se and wi h clea and unde s andable go e nance s uc u es o he pa ies
in ol ed.
The main con ibu ion o his publica ion is summa ised in a desc ip ion
o he common needs o inno a ion clus e s and he app oach o suppo
ac i i ies o mee hese needs.
This publica ion is aluable o hose inno a ion ecosys ems ha seek o es ablish
ela ionships wi h simila clus e s based on he T iple Helix model and ha pu sue he
de elopmen o u ban a eas owa ds echnology and inno a ion bu wi h a ocus on
di e en indus ies.
This publica ion is aligned wi h he ul ilmen o he Thi d Main Objec i e
and Fi s Seconda y Objec i es o he Thesis.
2h ps:// uh alley. ech/en/ - Las Access: 20/12/2022
58 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
4.1.2 Second Publica ion CP.2 - Con e ence Pape : ”Building Coope a ion
be ween Inno a ion Clus e s Based on Compe ences Requi emen s. Case o
CFAA and uh alley”.
In his second publica ion (See Sec ion 6.2), we add ess he issue o how o build coop-
e a ion be ween inno a ion clus e s based on compe ence equi emen s.
As discussed in he i s publica ion, an in e na ional collabo a ion be ween inno a ion
clus e s, as well as be ween public en i ies and indus y, is sough and encou aged by he
Eu opean Union. Howe e , one o he equi emen s o such collabo a ions o wo k was
based on a conc e e de ini ion o objec i es and clea go e nmen al s uc u es. One o
he componen s o his s uc u e is he PM, which is why in his publica ion, we analyse
wha compe encies a e necessa y o a PM and mul i-skilled enginee s o implemen
R&D p ojec s success ully.
To achie e i , a b oad se o compe encies is needed. Some o hese canno be augh
in adi ional class ooms alone bu ins ead equi e a lea ning en i onmen wi h oppo -
uni ies o gain p ac ical expe ience. The s udy was conduc ed in hese wo inno a ion
clus e s because hese loca ions o e such oppo uni ies du ing he ealisa ion o R&D
p ojec s.
In o de o iden i y he se s o compe encies equi ed o collabo a e and manage a
esea ch p ojec and o ind he skills ha can be ained a he g ass oo s le el in hese
clus e s, in e iews we e conduc ed wi h di e en ocus g oups om he wo clus e s.
The main con ibu ion o his publica ion is o p o ide a lis o compe en-
cies equi ed a echnical, p o essional and global le els o R&D PMs. These
compe encies include p oblem-sol ing skills, knowledge and skills in scien i ic
and sys ema ic analysis, social skills, and c i ical and c ea i e hinking.
The publica ion s a es ha i he e is a need o speci ically desc ibe p e equisi es o
wo king on a pa icula p ojec , mo e in e iews wi h p ojec eam membe s should
be conduc ed, and assessmen measu es o e alua e equi ed compe encies should be
de eloped.
We now unde s and ha using a i icial in elligence echniques o na u al language
p ocessing migh be bene icial o de e mining people’s compe encies based on he anal-
ysis o hei CVs, publica ions, and p ojec desc ip ions, among o he [145].
This publica ion is aligned wi h he ul ilmen o he Thi d Main Objec i e
o he Thesis.
Wi h hese i s wo publica ions, we sough o analyse he main cha ac e is ics o
inno a ion clus e s, we looked o pa e ns o c ea e collabo a ion s a egies, and we
analysed which se s o compe encies PMs should ha e o pa icipa e in his ype o
collabo a ion.
We hen mo ed on o he nex s ep, which in ol ed igu ing ou how o ans e he
knowledge and indings om p ojec s ca ied ou wi hin and among inno a ion clus e s
and he ou comes o c oss-sec o collabo a ion be ween he public and p i a e sec o s.
4.1. SUMMARY AND DISCUSSION. 59
4.1.3 Thi d Publica ion JP.1 - O iginal Jou nal Pape : ”TRLs 5–7 ad anced
manu ac u ing cen es, p ac ical model o boos echnology ans e in
manu ac u ing”.
The hi d publica ion (See Sec ion 7.1) is an o iginal jou nal a icle which e e s o a
s udy ca ied ou o c ea e new ad anced manu ac u ing cen es whose ac i i ies ocus
on TRL 5 - 7. This publica ion sough o in eg a e small and medium-sized en e p ises
in o he supply chain ha can bene i om collabo a ion wi h uni e si ies and o he
esea ch ins i u ions by accessing sha ed and specialised knowledge.
Unde s anding his ype o inno a ion clus e ’s cha ac e is ics, he publica ion looked
a how o in eg a e mo e componen s in o he ae onau ical manu ac u ing alue chain
h ough a p ojec managemen app oach. The esea che s applied i o he ae onau ical
sec o in he Basque Coun y, iden i ying he necessa y p ocedu es needed o he success
o hese e o s and he knowledge domains ha can make o b eak hem.
We show h oughou he publica ion ha ad anced manu ac u ing esea ch cen es
can be a g ea solu ion o his p oblem in indus ial sec o s wi h s uc u al knowledge
and specialised skills de ici s. We ocus on he ae ospace, machine ool and o he supply
chain sec o s, as hese sec o s in es hea ily in R&D. Howe e , we we e able o de e -
mine ha mos SMEs in he Basque Coun y need mo e ixed R&D s uc u es o de elop
hei esea ch ac i i ies. Despi e he signi ican e o s made by he public adminis a-
ion o imp o e uni e si ies and aining cen es and aise s uden quali ica ions, hey
collabo a e wi h local companies o iden i y he signi ican ailu es o ecen g adua es.
One o he main conclusions o his s udy is o ecommend ha he ul-
ima e goal o hese esea ch cen es ocused on ad anced manu ac u ing
should no be concen a ed on ho izon al de elopmen s o speci ic manu ac-
u ing echnologies bu should seek o engage in a wide ange o manu ac-
u ing p ocesses; in eg a ing he de elopmen o p ocess ac i i ies such as
ool modelling, simula ion, adap i e con ol o ope a ion low, au oma ion,
among o he s, eaching a le el o ma u i y ha allows he apid and op imal
low o echnology, a oiding isks in he echnology po olio o he pa ne
companies. This way, indus ial pa ne s can unde ake applied echnological
ac i i ies wi h a high p obabili y o success.
The suppo o local, egional, na ional, and Eu opean go e nmen al bodies is a c ucial
componen o his ecosys em because sha ing public and p i a e unds can aise p o i
ma gins and pa ne s’ abili y o R&D, which imp o es he likelihood o success o hese
kinds o p ojec s.
This a icle has also been used as a s a ing poin o examining he e ec s o he
a ious ac o s impac ing p ojec managemen in acili ies like hese ( he implemen a ion
o p ojec s be ween public-p i a e en i ies, he co ec ans e o echnology, unding and
he di e en equi emen s o accessing and esponding o hese, as well as he quali y
o he esea ch, ca ied ou ). Mo eo e , how his s a egy can enhance p ojec ou comes.

60 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
This publica ion is aligned wi h he ul ilmen o he Second Main Objec i e
and Second Seconda y Objec i e o he Thesis.
The a icle men ions how he c i ical mass c ea ed h ough collabo a ion be ween
companies, esea ch cen es, and uni e si ies p omo es and imp o es he chances o
ob aining unding o R&D ac i i ies, wi h pa icipa ion in conso ia in Eu opean o
na ional p ojec s unde simila condi ions wi h Eu opean e e ence cen es, and wi h
he possibili y o choosing o pa icipa e in ac i i ies wi h a high a e o echnological
e u n, as well as access o b oade unding.
We also alked abou how, o small companies ha pa icipa e in his ype o e-
sea ch cen es, i would mean an e en mo e signi ican leap o wa d, bea ing in mind
hei limi a ions in e ms o R&D in es men and he di icul y o cons an ly de eloping
ac i i ies o he de elopmen o imp o emen o echnologies.
We also emphasise in eg a ing machine- ool manu ac u e s in o he ae onau ical man-
u ac u ing alue chain. Pa icipa ing in esea ch p ojec s wi h la ge companies imp o es
he chances o selling hei p oduc s o he la e . In addi ion o gaining access o be e
inancing schemes and imp o ing p oximi y o he end cus ome , hey also bene i om
echnological excellence inside and ou side ad anced manu ac u ing esea ch cen es.
Some o he conclusions ha can be d awn om he s udy a e:
•Resea ch cen es a ound TRLs 5-7 should consis o se e al companies and uni e si-
ies and should be s ongly suppo ed by public adminis a ions;
•supply chain collabo a ion wi h O iginal Equipmen Manu ac u e s (OEMs) o Tie
1 is highly ecommended, wi h he new hub model as a e ical concep ion;
•e o should be ocused on one pa icula indus ial sec o ;
• esea ch cen es o ad anced manu ac u ing can boos he ela i e posi ion o a uni-
e si y’s esea ch g oup wi h applied esea ch o leading posi ions;
• he ini ial lis o machines and sys ems is key o achie ing in ensi e use o he cen e’s
esou ces. Machines should be p ocu ed wi h a li espan o a leas se en yea s;
• he loca ion in a echnology pa k wi h common se ices is a key aspec o he p ojec .
The common se ices, he en i onmen and easy access by public anspo a e key;
•wi h he idea ha i should be a cen e a ailable o all pa ne s and ha managemen
o he cen e should genuinely be on behal o he en i e conso ium. The cen e should
be managed by a uni e si y o echnological agen .
We ha e al eady analysed, discussed and published a ious con ibu ions ega ding
he unc ioning o public-p i a e esea ch cen es and hei place wi hin egional and
Eu opean business ecosys ems, as well as he a ious cha ac e is ics needed o achie e
a mo e e icien ans e o esea ch esul s o end companies and o de ine pa e ns o
collabo a ion be ween esea ch cen es and how his helps in ob aining be e scien i ic
esul s, as well as boos ing local economies. Addi ionally, we we e able o emphasise he
signi icance o p ojec managemen app oaches ha enable he quick ans e o high-
4.1. SUMMARY AND DISCUSSION. 61
quali y knowledge and he ele ance o a se o PM compe encies o he pa icipa ion
and managemen o R&D p ojec s in such cen es.
The nex s ep leads us o analyse hese ypes o p ojec s om he inside, bo h h ough
he e alua ion o success c i e ia and he assessmen o his success by he esea ch
cen e and i s s akeholde s, and he iden i ica ion o KPIs o his ype o o ganisa ion
o imp o e he pe o mance and esul s o he p ojec s.
4.1.4 Fou h Publica ion CP.3 - Con e ence Pape : ”Assessing he success
o R&D p ojec s and inno a ion p ojec s h ough p ojec managemen li e
cycle”.
In his ou h publica ion (See Sec ion 8.1), we en e he wo ld o measu ing he success
o p ojec s h ough esea ch ela ed o he e alua ion o he success o R&D p ojec s
and inno a ion p ojec s du ing hei li e cycle.
This publica ion is aligned wi h he ul ilmen o he Fou h Main Objec i e
o he hesis.
In his a icle, we discuss how he success o R&D p ojec s impac s echnological ad-
ancemen and how di icul i is o manage and go e n his kind o p ojec . Focused
on imp o ing he managemen o R&D p ojec s and a oiding d ama ic ailu es, in his
esea ch, h ough a da a-d i en app oach we in es iga ed and de e mined se e al signi -
ican c i e ia o measu ing he success o p ojec s du ing he p ojec li e cycle.
The main con ibu ion o his a icle is he iden i ica ion o di e en di-
mensions and he e alua ion, in o de o impo ance o PMs, o success
c i e ia o R&D p ojec s ca ied ou a he CFAA, as inpu o imp o e
p ojec managemen .
These c i e ia we e alida ed and e alua ed by conduc ing a su ey o he PMs o
he CFAA. These esul s e ealed ha he CFAA has been ela i ely success ul in he
execu ion o R&D p ojec s and inno a ion p ojec s and has he po en ial o imp o e he
esea ch pipeline and pla o ms o new echnologies, inno a ion and c ea i i y in he
u u e. Fu he mo e, i is emphasised ha he CFAA has been ela i ely success ul in
implemen ing p ojec s on ime and wi hin budge , as well as in gene a ing new knowledge
and echnological p oduc s.
Du ing he e alua ion o he success c i e ia and which we e he mos impo an o
he manage s, i was u he concluded ha since pa ne sa is ac ion was he ac o
ha con ibu ed leas o he success o he p ojec , cus ome -o ien ed s a egies should
be implemen ed o inc ease pa ne sa is ac ion, and hus imp o e he o e all ou come
o he p ojec s.
This is mainly because he in e nal measu emen o p ojec success can and is, in ac ,
di e en om he s akeholde ’s pe cep ion o he p ojec s. While some ( he p ojec de-
elope ) may conside a p ojec inished on schedule and wi hin budge o be success ul,
he esea ch esul s may di e om wha o he s demand (s akeholde s). This is due o
a ious ac o s, including he p ojec ’s iming, he ease wi h which he esul s can be
62 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
pu in o p ac ice, he p ojec ’s echnological u gency, o e en changes in he ma ke .
One o he subjec s discussed in he ollowing a icle is mo e esea ch in o he elemen s
in luencing s akeholde s’ pe cep ions o success.
4.1.5 Fi h Publica ion JP.2 - O iginal Jou nal Pape : ”P ojec Success
C i e ia E alua ion o a P ojec -Based O ganiza ion and I s S akeholde s -
A Q-Me hodology App oach”.
In his i h publica ion (See Sec ion 8.2), we e e o he measu emen o p ojec success
e alua ed om he poin o iew o s akeholde s and PMs, who is in cha ge o de eloping
he p ojec , h ough he use o Q-Me hodology.
In he publica ion, we emphasise ha a p ojec can no longe be seen solely as a
empo a y mission unde aken o c ea e a single p oduc , se ice o esul . Howe e ,
al hough he objec i e is he same, he elemen s su ounding he p ojec ha e inc eased
in bo h numbe and complexi y. They mus be iewed om a sys ems heo y pe spec i e
[146], in which he sys em’s inpu s and componen s di ec ly de e mine he sys em’s
ou pu s. Addi ionally, a sys em whe ein he PMs a e in luenced by he beha iou s o
adi ions ha de ine an o ganisa ion may cause hem o ake ac ions ha jeopa dise
he p ojec ’s success as measu ed by he s akeholde s. Unde s anding and schema ising
his complexi y so i can be examined in de ail o inc ease p ojec success a es will be
one o he mos signi ican issues acing p ojec managemen in he u u e.
One o he majo conce ns o o ganisa ions in e ms o p ojec managemen is o
iden i y he p ojec success c i e ia when e alua ing o conside ing a p ojec success ul.
Wi h his idea in mind, he iden i ica ion o a se ies o success c i e ia di ided in o
di e en dimensions, de ined om a li e a u e e iew, and he subsequen discussion o
hei de ini ion and he exe cise de eloped using Q-Me hodology ga e us he possibili y
o c ea e a lis o hese c i e ia o R&D p ojec s and o ha pa icula o ganisa ion.
One o he main con ibu ions o his publica ion is o iden i y which
a e he mos impo an success c i e ia in collabo a i e public-p i a e R&D
p ojec s, acco ding o he poin o iew o he o ganisa ion’s PMs and in e -
nal s akeholde s.
Based on he indings, he o ganisa ion’s p ojec manage s, as well as he membe s
o he P ojec Managemen O ice, can now make plans, es ablish indica o s and de ine
me hodologies and new checkpoin s h ough which he success c i e ia iden i ied in his
s udy can be assessed. This e alua ion can be ca ied ou no only a he end o he
p ojec bu also h oughou i s li e cycle. [10].
Ano he poin o no e abou he esea ch conduc ed in his publica ion is he easy
applicabili y o o he ypes o o ganisa ions and p ojec s. Thanks o he li e a u e e iew,
we ha e shown how impo an i is o know and conside he s akeholde s’ poin o iew
in de e mining he success o p ojec s and how his is no exclusi e o R&D o ganisa ions
ca ying ou collabo a i e p ojec s bu o any p ojec o ganisa ion.
4.1. SUMMARY AND DISCUSSION. 63
Ano he signi ican con ibu ion o his publica ion is o iden i y he di -
e en subjec i e pe spec i es when assessing he success o p ojec s om he
poin o iew o s akeholde s and PMs.
Q-Me hodology o e s he possibili y o classi y people in o di e en g oups ha sha e
he same subjec i e pe spec i es, called ”Fac o s”, hanks o which h ee ac o s we e
de ined o he o ganisa ion:
•Fac o 1: High quali y-o ien ed o he ou pu ;
• ac o 2: T adi ional P ojec success o ien ed;
• ac o 3: Ex e nal iew o ien ed.
Fo example, Fac o 1, comp ising abou 50% o he pa icipan s, is cha ac e ised by
ge ing he job comple ed co ec ly, which includes a ocus on job si e sa e y, ollowing
o icial s anda ds o p ojec deli e ables o achie e g ea e cus ome sa is ac ion, bo h in
he esul and in he ac i i ies unde aken o achie e i . In addi ion o being cha ac e ised
by a desi e o mee cus ome expec a ions and enhance he o ganisa ion’s epu a ion.
In gene al, he objec i e o iden i ying hese ac o s is o cla i y wha hese poin s o
iew a e and who is included in hem, as well as o be able o g oup he pa icipan s
acco ding o hei poin s o iew. Thanks o his, i is possible o ocus e o s on mee ing
hese expec a ions, implemen p ac ices aimed a managing hese c i e ia, and imp o e
p ojec managemen om di e en poin s o iew.
This publica ion is aligned wi h he ul ilmen o he Fou h Main Objec i e
o he Thesis.
Al hough a Q-Me hodology exe cise does no equi e a la ge numbe o pa icipan s,
one o he c i ical limi a ions o his esea ch was he sample size because he componen s
did no indica e a s ong endency owa ds any o hem. As al eady men ioned, hese
esul s can be used o imp o e he app oach o R&D p ojec s in he o ganisa ion and
e alua e such p ojec s’ success.
Simila ly, he iden i ied c i e ia can ede ine new KPIs o p ojec s and he o gan-
isa ion o ee alua e exis ing me ics al eady u ilised, depending on he con ex , o
p ojec s.
Wi h he comple ion o hese wo publica ions, we we e able o co e he opic o suc-
cess measu emen o R&D p ojec s o o ganisa ions ca ying ou collabo a i e p ojec s
wi h public-p i a e unding and looking o a as echnology ans e o he esul s. The
conclusions o hese publica ions acili a e he de ini ion o quali y c i e ia o p ojec s
which can be conside ed ope a ional needs o he de ini ion o Key Pe o mance Indi-
ca o s o p ojec -based o ganisa ions.
70 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
esou ces a ailable o s uc u es such as his, as hese mus be di ec ed owa ds he
hi ing o pe sonnel, and he pu chase o machines o so wa e, p ocesses de ined in gene ic
me hodologies, such as PMBoK, a e challenging o ollow o small o medium-sized
o ganisa ions)
The de ini ion o quali y c i e ia can only some imes be ex ended o all o ganisa ions
( o some o ganisa ions, he c i e ia se ou he e may be less c i ical o may a y d as i-
cally). Gi en he complexi y in ol ed, i is a simple exe cise ha should be ca ied ou
in o ganisa ions whe e p ojec s a e ca ied ou wi h he collabo a ion o s akeholde s.
We p oposed a me hod o o e come his issue (See [10]). In addi ion, he ca ego isa ion
o KPIs helps o guide he e o s made in he p ojec s and ake ad an age o he wo k
done o imp o e he o ganisa ion’s pe o mance.
The o ganisa ion can a ange o quali y assu ance checkpoin s o be done egula ly
based on he numbe o jobs o be comple ed, he es ima ed du a ion o he p ojec , o
he c i icali y o he asks pe o med. In addi ion, pa ial p ojec epo s a e p oposed
o his con ol o help educe he equency o p ojec mee ings.
The minimum accep able quali y o he wo k comple ed, as speci ied by he PMs
and s akeholde s, is one o he c i ical success ac o s o he p oposed me hodology and
he p ojec in gene al. The gene a ion and applica ion o co ec i e ac ions o imp o e
quali y pe o mance o isk assu ance a e i al o co ec he cou se o a p ojec o o
make decisions abou he asks pe o med, he pe sonnel who pe o m hem, he p ojec ’s
objec i e, and he asks p og ammed, among o he s.
Task unce ain y implies low analyzabili y o R&D p ocesses and makes i di icul
o ho oughly plan and speci y esea ch asks in ad ance [151]. As a esul , he e is a
con inuing need o collec and dissemina e in o ma ion o speci y how o de e mine wha
is going on elsewhe e and how o deal wi h dis up ions [152].
The e o e, collabo a i e esea ch p ojec s wi h high ask unce ain y imply ha deci-
sions mus be made quickly. Consequen ly, a less cen alised decision-making p ocess is
needed, as cen alised communica ion pa e ns can cope wi h ask- ela ed unce ain y
mo e e ec i ely han hie a chical ones.
Inc eased ask unce ain y in collabo a i e esea ch leads o a g ea e decen alisa ion
o coo dina ion and con ol p ac ices. Apa om ask unce ain y, he balanced ole
o R&D pe sonnel and R&D manage s is also s imula ed by he echnology ans e
objec i es o he pa icipan s [55].
The ac ha his me hodology has been designed and based on he in o ma ion and
con ex o he CFAA does no mean ha i canno be applied ou side he o ganisa ion o
o p ojec s ha a e no in e nal. Ins ead, he au ho s belie e and ely on he scien i ic
li e a u e o s a e ha a comp ehensi e ye simple and use - iendly PM’s con ol o
he quali y o planned and execu ed ac i i ies is he p ima y inpu o maximise p ojec
esul s and achie e p ojec objec i es.
Following he ecommenda ions o Wa d & Chapman [153], he gene ic s uc u e o a
p ojec should be desc ibed in ou phases (a p ojec concep ualisa ion phase, a second

4.1. SUMMARY AND DISCUSSION. 71
planning phase, a hi d phase ou lining he execu ion o asks, and inally, a p ojec
comple ion phase). Howe e , he numbe o s ages depends on he na u e o he p ojec
(and he o ganisa ion’s p epa a ion) and migh ange om 4 o 8 o mo e [136].
Acco ding o Cha a [154], e e y p oposed me hodology should ha e p ojec phases,
which may a y depending on he size o he p ojec , he o ganisa ion, o he indus y,
bu ce ain ypical phases should be included: concep , de elopmen , implemen a ion,
and suppo . Acco ding o hese ecommenda ions, he me hodology p oposed he e is
o ganised in o ou pa s: de ini ion, planning, execu ion and con ol, and e alua ion and
elease. Based on he me hodology p oposed by Ke zne [150], Figu e 4.2 desc ibes he
ela ionship be ween he p ojec phases and he de ined ac i i ies.
72 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
Fig. 4.2: P oposed P ojec Managemen Me hodology in phases and ac i i ies
4.1. SUMMARY AND DISCUSSION. 73
The documen a ion gene a ed and a wha poin in he p ojec , depending on he
phase, is desc ibed in Figu e 4.3.
74 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
Fig. 4.3: P oposed P ojec managemen me hodology including ac i i ies and documen s
4.1. SUMMARY AND DISCUSSION. 75
The documen a ion gene a ed du ing he p ojec is o i al impo ance in o de o
be able o c ea e di e en da abases wi h p ojec in o ma ion ha will se e as inpu s
o u u e p ojec s ca ied ou by he o ganisa ion. In addi ion, we p opose collec ing
in o ma ion o c ea e he da abases desc ibed in Table 11.2.
We de ail he phases and ins umen s ha comp ise he echnique, as well as he p i-
ma y objec i es, ac ions, and ou comes o each s ep, ollowing Chin’s [155] sugges ions.
4.1.8.2 Phases and ools desc ip ion.
i Phase 1 - P ojec De ini ion.
The main objec i e o his phase is o gene a e he necessa y in o ma ion o p ojec
planning and es ablish he p ojec ’s o ganisa ional in as uc u e. The main objec-
i es a e:
•To de ine he pa ne ’s equi emen s o he p ojec ;
• o de ine, oge he wi h he pa ne , he quali y c i e ia on which he p ojec will
be e alua ed;
• o iden i y po en ial pa ne s o he ealisa ion o he p ojec ;
• o analyse, assess and documen he po en ial isks o p ojec de elopmen ;
• o de ine he main objec i es o he p ojec ;
• o de elop he i s e sion o he p ojec shee ;
• o es ablish he collabo a ion ag eemen and he app o al o s a he p ojec .
This Phase 1 includes h ee ac i i ies and wo decision s eps:
i.i Pa ne Requi emen s.
The pa ne ’s equi emen s o ealising a p ojec a e ecei ed in his i s ac-
i i y. These equi emen s mus be aligned wi h he o ganisa ion’s s a egy and
ha e he necessa y physical and human esou ces o conduc he p ojec .
i.i.i Ex e nal in o ma ion:
•Scope equi emen s.
The scope o he p ojec is de ined, oge he wi h he main and seconda y
objec i es o he p ojec .
•Technical equi emen s.
The echnical equi emen s should be de ined in as much de ail as possible.
I will be one o he c i e ia o e alua ing he p ojec ’s pe o mance.
•Quali y Requi emen s.
The pa ne should p o ide an ini ial desc ip ion o he quali y equi e-
men s o he p ojec .
•Time equi emen s.
Simila ly, an ini ial es ima e o he ime equi ed o comple e he p ojec ,

76 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
as equi ed by he pa ne . Depending on he na u e o he p ojec , his
may indica e how much ime is a ailable o comple e he p ojec .
i.i.ii In e nal in o ma ion:
•Alignmen wi h he s a egy o he o ganisa ion.
De e mine whe he he p ojec aligns wi h he o ganisa ion’s esea ch o
business s a egy.
•Resou ces a ailable, su icien , and quali ied.
The o ganisa ion should de ine whe he i has he necessa y esou ces o
comple e he p ojec equi emen s.
•P io i isa ion o he p ojec .
The p ojec will be p io i ised depending on he p ojec ype and he o -
ganisa ion’s c i e ia. This decision can be made based on he objec i e
and scope o he p ojec .
i.ii Quali y C i e ia (QC) equi emen s de ini ion.
The p ojec objec i e, as well as he in e nal and ex e nal quali y c i e ia o he
p ojec , mus be de ined in his ac i i y, as well as he p ojec manage ial da a
( echnology o he p ojec , inancing, ype o p ojec , pa ne s in ol ed, p ojec
code, p io i isa ion, C i ical Success Fac o s (CSF) es ima ion; PM Assignmen ;
Ini ial es ima ion o p ojec hou s (machines and s a , acco ding o Pa ne ’s e-
qui emen s; echnology o he p ojec and inancing). The necessa y in o ma ion
can be con e ed in o a checklis o he o ganisa ion o i s de ini ion and u he
s udy. This in o ma ion should be included in he i s e sion o a p ojec shee .
The quali y c i e ia o e alua ing he p ojec should be a mos en i ems.
i.ii.i Ex e nal in o ma ion:
•KPIs de ini ion.
As a as possible, he pa ne shall de ine he KPIs ha will go e n he
p ojec based on he echnical equi emen s, he o ganisa ion’s success c i-
e ia and o he a ailable in o ma ion.
i.ii.ii In e nal in o ma ion:
•KPI Lis .
Based on he KPIs de ined o he o ganisa ion, i is necessa y o de ine
which indica o s a e ed by he implemen a ion o he p ojec . In he
same way, based on he na u e o he p ojec o he line o esea ch o be
ollowed, de ine which indica o s can be ed by he implemen a ion o he
p ojec .
•C i ical Success Fac o s.
De ine which C i ical Success Fac o s a ec p ojec deli e y. I a ailable,
4.1. SUMMARY AND DISCUSSION. 77
selec hose CSFs de ined by he o ganisa ion and join ly by he cus ome .
[10].
•Da a Analysis abou p e ious p ojec s.
I da a om p e ious p ojec s a e a ailable, consul on simila p ojec s
ha ha e been ca ied ou , ac i i ies ca ied ou , quali y assessmen s,
p ojec esul s, and esou ces planned s used, among o he s. An example
o he da abases gene a ed using he p oposed me hodology is summa ised
in Table 11.2.
i.ii.iii Documen s:
•P ojec Shee V1.
The i s e sion o he p ojec ini ia ion documen , wi h a summa y o he
ag eemen s eached o he ealisa ion o he p ojec , will be gene a ed.
This shee will include he p ojec i le, cus ome s and con ac de ails,
inancing, pe cen age o cus ome pa icipa ion in he p ojec , an ini ial
es ima ion o machine hou s and pe sonnel, and s a and es ima ed du-
a ion. In addi ion, a summa y o he scope o he p ojec , echnical and
quali y equi emen s, and de ined quali y c i e ia.
i.iii Requi emen s De ini ion.
In his ac i i y, he pa ne mus be asked o app o e he p e iously collec ed
in o ma ion and analyse he p ojec ’s easibili y join ly. An ag eemen be ween
he pa ies mus be w i en down in P ojec Shee V1.
i.iii.i Ex e nal in o ma ion:
•Pa ne Feedback. The pa ne mus app o e he in o ma ion con ained
in P ojec Shee V1. I no , he pa ne ’s equi emen s and he o he in-
o ma ion con ained in P ojec Shee V1 mus be e-analysed.
i.i S a egical isks analysis and e alua ion.
In his ac i i y, a s a egic assessmen o he p ojec ’s isks mus be conduc ed,
including a desc ip ion o he isks, he se e i y o occu ence, an assessmen o
he p obabili y o occu ence, and he p oposal o ini ial co ec i e and p e en i e
ac ions o he occu ence o hese isks.
i.i .i In e nal in o ma ion:
•Ini ial s a egical isk e alua ion (Desc ip ion o isk; Se e i y
and Likelihood e alua ion, Co ec i e and P e en i e Ac ions).
An ini ial s udy o he p ojec isks, a desc ip ion o he isks encoun e ed,
an assessmen o he se e i y and p obabili y o occu ence, and p e en-
i e and co ec i e ac ion plans o he isks encoun e ed mus be ca ied
78 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
ou .
i. Pa ne Feedback.
Wi h he in o ma ion om he isk analysis, he pa ne will eques eedback o
check he easibili y o he p ojec . In disag eemen , a new p ojec isk assessmen
mus be ca ied ou .
i. i Risk Assessmen .
I he pa ne disag ees wi h he isk assessmen , his ac i i y should be pe o med
again, aking in o accoun he commen s ecei ed and he ecommenda ions made
by he pa ne o app o al.
i. ii P ojec S a .
I a consensus has been eached o he ealisa ion o he p ojec in e ms o ech-
nical equi emen s, ime, quali y, pe sonnel and o he componen s, he p ojec
will be o mally ini ia ed. A p ojec ini ia ion documen (P ojec Shee V1) will
be gene a ed and based on his, he con ac ual ag eemen s o he ealisa ion o
he p ojec will be gene a ed.
i. ii.i Documen s:
•Risk Managemen Plan.
Suppose a consensus has been eached on he isk assessmen and he
p oposed co ec i e and p e en i e ac ions. In ha case, his in o ma ion
shall be eco ded in a ”Risk Managemen Plan” con aining he de ini i e
desc ip ion o he isks, he se e i y and p obabili y o occu ence, and he
p oposed co ec i e and p e en i e ac ions o hei p e en ion. A pe son
shall also be designa ed o moni o and con ol hem. Once his has been
done, he p ojec can be o mally launched.
ii Phase 2 - P ojec Planning. This is he main phase o he p oposed me hodology,
as i co e s p ojec planning. The main objec i e o Phase 2 is o del e in o he
desc ip ion o he ac i i ies and con ol equi emen s necessa y o gene a e co ec
p ojec planning. The in o ma ion collec ed in Phase 1 will se e as inpu o his
phase. The main objec i es a e:
•To ca y ou he desc ip ion o he ac i i ies;
• o de ine and documen , oge he wi h he pa ne , he quali y c i e ia manage-
men plan on which he p ojec will be e alua ed;
• o plan he asks and ac i i ies necessa y o ca y ou he p ojec ;
• o gene a e he necessa y in o ma ion o p ojec con ol;
• o de elop he second e sion o he p ojec shee ;
4.1. SUMMARY AND DISCUSSION. 79
This Phase 2 includes h ee ac i i ies and one decision s ep:
ii.i P ima y Planning.
This ac i i y, oge he wi h he equi ed in o ma ion on Phase 1, will esul in he
ealisa ion o he WBS up o le el 3. The i s ac i i ies, he s a assignmen s and
he scheduling (machine schedule) o he machines acco ding o he p io i isa ion
o he p ojec will ha e o be planned. In addi ion, he p ojec miles ones mus
be de ined acco ding o he numbe and c i icali y o he asks o be pe o med.
The asks mus ha e in o ma ion on he ini ial numbe o hou s p og ammed,
classi ica ion o he ype o ask (Table 11.2) and he pe son o g oup o people
in cha ge o he ask.
ii.i.i In e nal in o ma ion:
•S a assigna ion.
The people who will o m he p ojec eam and ca y ou he ac i i ies
will be de ined.
•Resou ce alloca ion.
I equi ed, he esou ces needed o ca y ou he p ojec asks shall be
desc ibed.
•Ope a ional isks analysis and e alua ion.
This in o ma ion will be added o he p ojec ’s Risk Managemen Plan.
•Miles ones de ini ion.
P ojec miles ones will be de ined in ag eemen wi h he pa ne .
•Task Planning.
Ini ial planning o p ojec asks and ac i i ies will be ca ied ou .
ii.i.ii Documen s:
•WBS Le el 3.
A his poin , le els 1 and 2 o he WBS should ha e been de ined. Wi h
his in o ma ion, plus he in o ma ion collec ed in planning asks and ac-
i i ies, le el 3 o he WBS should be de ined.
ii.ii QC Assessmen .
Based on he in o ma ion de ined in he ac i i y (i.ii), no mo e han en qual-
i y c i e ia should be de ined o assess he p ojec . These c i e ia should g oup
he quali y c i e ia de ined by he pa ne and he o ganisa ion’s in e nal KPIs
and C i ical Success C i e ia (CSC). Subsequen ly, and ollowing he me hod de-
sc ibed by Paquin e al. [3], he WBS -QBS will be c ea ed, which will ha e he
in o ma ion o which ac i i ies con ibu e o each o he de ined quali y c i e ia,
as well as a alua ion o ha quali y con ibu ion (cj). This in o ma ion should
be included in he QC Managemen Plan, oge he wi h he pe iodici y o
86 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
•QC Managemen Plan.
The QC Managemen Plan shall be included in he p ojec documen a-
ion and documen a ion ela ing o changes and de elopmen s du ing he
p ojec .
•P ojec e alua ion.
Finally, oge he wi h he p ojec eam and he PM, an e alua ion o he
ealisa ion will be ca ied ou , aking in o accoun he achie emen o he
echnical and quali y objec i es o he p ojec , he p ojec managemen ,
p oblem sol ing and lessons lea ned.
i .ii.ii Documen s:
•P ojec Closu e Repo .
A p ojec closu e epo will be c ea ed wi h he in o ma ion collec ed
du ing he p ojec .
Ano he ad an age o using his me hodology is ha compa ing he quali y gained
wi h he planned quali y o he wo k pe o med allows PMs o de ec de ia ions in
quali y and ini ia e co ec i e ac ions ea ly, a oiding he unnecessa y expendi u e o
esou ces.
One o he main easons why he me hodology was de ined in his way lies in he
in en ion o ob ain as much da a as possible om he p ojec , wi hou his meaning
mo e pape wo k o he PM o he p ojec eam.
4.1.8.3 EQM desc ip ion.
The EQM desc ibed by Paquin e al. [3]. was bo n as a p oposal o allow he PMs o
assess and con ol he quali y o he inal p oduc h oughou he p ojec li e cycle. We
ha e discussed ha he EQM is based on wo undamen al p emises. The i s is ha
quali y is a measu able concep , and he second is ha quali y accumula es p og essi ely
h oughou he p ojec ’s li e cycle.
In hei me hod, hey s a by elucida ing he clien ’s needs. To do his, he au ho s
b eaks down he o e all quali y objec i e in o mo e de ailed, lowe -le el objec i es o
cla i y hei meaning. This esul s in he Quali y B eakdown S uc u e (QBS) ha shows
he hie a chy o he decomposi ion o he quali y objec i es. Pa ne s’ p e e ences a e
hen e alua ed and agg ega ed. In he o iginal EQM assumed no in e ac ion be ween
he gi en a ibu es, making he alue unc ion addi i e.
Since he a ibu es and quali y c i e ia a e desc ibed in a hie a chical s uc u e,
he weigh s can be conside ed condi ional e alua ions. Thus he ela i e impo ance o
he c i e ion cj o he o e all quali y objec i e wjis ma hema ically desc ibed by he
Equa ion 4.1:
wj=
K
X
k=1
aksjk o j = 1, . . . , J (4.1)

4.1. SUMMARY AND DISCUSSION. 87
Whe e:
•JThe numbe o c i e ia;
•KThe numbe o a ibu es;
• kThe a ibu e ko he QBS;
•akThe ela i e con ibu ion o a ibu e k o he o e all quali y objec i e
ak∈[0,1] and
K
X
k=1
ak= 1
•sjk The ela i e con ibu ion o c i e ion cj o a ibu e k,sjk ∈[0,1] and:
J
X
j=1
Sjk = 1, o k = 1, ..., K
•wjThe ela i e con ibu ion o c i e ion cj o he o e all quali y objec i e, and:
J
X
j=1
wj= 1.
Assuming p e e en ial independence be ween he asks, he o e all quali y Qo he
p ojec ’s inal p oduc is equal o he weigh ed sum o he u ili y alue o he esul s xj
ob ained in all c i e ia J. Ma hema ically, we can w i e (Equa ion 4.2:
Q=
J
X
j=1
wjϕj(xj) (4.2)
Es ima ing Ea ned Quali y.
Once he QBS and WBS ha e been de e mined, he PM mus de ine he c i e ia o which
each ac i i y con ibu es. The linkage be ween he WBS and he QBS allows he PM o
es ablish a ela ionship be ween he ac i i ies and he quali y a ibu es.
The PM hen deepens he analysis by es ima ing he ela i e con ibu ion o each ac i i y
o i s ela ed quali y c i e ia. This can be done by a ibu ing a condi ional weigh ij
ha measu es he es ima ed ela i e con ibu ion o ac i i y ai o c i e ion cj.
The po en ial con ibu ion o ac i i y ai’s con ibu ion qi o he o e all quali y objec-
i e can be a ge ed as ollows (Equa ion 4.3):
qi=
J
X
j=1
wj ij o i = 1, . . . , I (4.3)
Whe e:
•IThe numbe o ac i i ies.
88 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
• ij The es ima ed con ibu ion o ac i i y ai o c i e ion cj, whe e, in addi ion:
ij ∈[0,1] , and,
I
X
i=1
ij = 1, o j = 1, . . . , J.
Thus he con ibu ion o each ac i i y ai o he o e all quali y can be decomposed in o
i s con ibu ion o c i e ion cjand he con ibu ion o c i e ion cj o he o e all quali y.
Planned Quali y o Wo k Scheduled P QW S
Planned wo k e e s o he expec ed a e o comple ion o ac i i ies a ime , while
ea ned wo k e e s o he ac ual a e o comple ion o ac i i ies a ime . Planned
quali y e e s o he expec ed quali y ha should ha e been accumula ed a ime ,
while ea ned quali y measu es he ac ual quali y accumula ed a ime .
The planned quali y o scheduled wo k P QW S measu es he planned con ibu ion
o he o e all quali y a ge a ibu able o scheduled wo k o all ac i i ies a ime .
P QW S is de ined as ollows (Equa ion 4.4)
PQWS =
I
X
i=1
J
X
j=1
wjϕj(x∗
j) ∗
ij ( ) (4.4)
Whe e:
• ∗
ij ( ) The expec ed con ibu ion o he expec ed esul x∗
j, as measu ed by c i e ion
cj, a ibu able o he wo k scheduled o ac i i y aia ime , 0 ≤ ∗
ij( )≤ ij
They assume ha he scheduled wo k is such ha i s ou come will sa is y he cus-
ome ’s expec a ions x∗
jand, consequen ly, will lead o ϕjx∗
j= 1. Thus, a e comple-
ion o he p ojec P QW S = 1.
Planned Quali y o Wo k Pe o med P QW P
The Planned Quali y o Wo k Pe o med P QW P measu es he planned con ibu ion o
he o e all quali y objec i e a ibu able o he wo k pe o med in all ac i i ies a ime
.P QW P is de ined as ollows (Equa ion 4.5:
PQWP =
I
X
i=1
J
X
j=1
wjϕjx∗
j ij ( ) (4.5)
Whe e:
• ij ( ) is he expec ed con ibu ion o he expec ed esul x∗
jmeasu ed by c i e ion
cja ibu able o he wo k done in ac i i y aia ime , 0 ≤ ij( )≤ ij
Consequen ly, he p ojec is comple ed a P QW P = 1
4.1. SUMMARY AND DISCUSSION. 89
Ea ned Quali y o Wo k Pe o med EQW P
The Ea ned Quali y o Wo k Pe o med EQW P measu es he cus ome ’s o e all sa -
is ac ion wi h he esul s ob ained o he quali y gained, a ibu able o he wo k pe -
o med on all ac i i ies a ime , Equa ion 4.6 gi es he EQW P a ime .
EQWP =
I
X
i=1
J
X
j=1
wjϕj(ˆxj) ˆ ij ( ) (4.6)
Whe e:
•ˆxj( ) The ac ual esul ob ained wi h espec o he c i e ion cjo he wo k done a
ime
•ˆ ij ( ) The es ima ed con ibu ion o he ac ual esul ˆxj( ) acco ding o c i e ion cj
a ibu able o he wo k done in ac i i y aia ime .
Assessmen o quali y de ia ions and ini ia ion o co ec i e measu es.
Compa ing he Ea ned Quali y o Wo k Pe o med EQW P wi h he Planned Quali y
o Wo k Pe o med P QW P we ob ain he quali y a iance (QV ) a ime (Equa ion
4.7:
QV = EQWP −PQWP (4.7)
The Quali y Pe o mance Index (QPI) a ime is calcula ed as ollows (Equa ion
4.8):
QPI =EQWP
PQWP
×100 (4.8)
4.1.8.4 P ac ical examples o he use o he me hodology.
The me hodology has been es ed in he ealisa ion o h ee p ojec s, as men ioned a
he beginning o his sec ion. (See Sec ion 4.1.8):
iP ojec 1: P ojec c ea ed o he de elopmen o a eal- ime machine moni o ing
pla o m.
ii P ojec 2: P ojec c ea ed o p edic he wea o cu ing ools in b oaching machin-
ing.
iii P ojec 3: P ojec c ea ed o he de elopmen o i ual senso s o manu ac u ing
p ocesses h ough he analysis o machine da a and pa quali y.
These p ojec s we e chosen o se e al easons:
iSou ces o unding:
90 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
•Fo P ojec 1, he unding was in e nal, i.e. i came om he CFAA unds.
•Fo P ojec 2, unds we e used om a p ojec submi ed, app o ed by he Basque
Go e nmen and inanced by he ”Elka ek” call, an ins umen wi hin he Basque
Science, Technology and Inno a ion Plan (PCTI 2030), which aims o suppo
collabo a i e esea ch in s a egic a eas o undamen al and indus ial esea ch.
4. Mo eo e , hese a e non- e undable g an s.
•The P ojec 3 was de eloped wi hin he amewo k o a Eu opean esea ch
p ojec unded by he Eu opean Union unde he EU esea ch and inno a ion
unding p og amme ”Ho izon 2020” 5, which was ope a ional om 2014 o 2020,
wi h a budge o a ound 80 billion eu os.
ii P ojec go e nance en i ies:
•Fo P ojec 1, he main go e ning en i y o he p ojec was he CFAA, which
was he Resea ch Cen e whe e he p ojec was ca ied ou . A epo jus i ying
he wo k ca ied ou , expe imen al and p elimina y es s a he CFAA, and doc-
umen a ion ela ed o he de elopmen o he p ojec and he in as uc u e, as
well as publica ions ei he in con e ences o in specialised scien i ic jou nals, a e
he e o e eques ed.
•Fo P ojec 2, he main go e ning en i y o he p ojec was he Basque Go e n-
men , which has es ablished minimum equi emen s o he accep ance o p ojec s
ca ied ou wi h i s unds [156].
– ”A pe o mance epo jus i ying he compliance wi h he condi ions imposed
in he awa d o he g an , indica ing he ac i i ies ca ied ou and he esul s
ob ained”.
– ”Economic epo jus i ying he cos o he ac i i ies ca ied ou ”.
– ”Signed audi epo d awn up by a pe son egis e ed in he O icial Regis-
e o S a u o y Audi o s unde he Ins i u e o Accoun ing and Audi ing o
Accoun s.”
– ”Expendi u e Ce i ica ion Documen o he p ojec , wi h he exp ess Decla-
a ion o Concu en G an s”.
Mo eo e , in his p ojec , he UPV/EHU wo ked wi h se en o he egional com-
panies ha o med he esea ch conso ium.
•Fo P ojec 3, he main go e ning en i y o he p ojec was he EC, wi h he EC
con olling and supe ising he esul s o he p ojec s and demanding minimum
equi emen s o hei accep ance om hose who ha e been g an ed unding,
among hem:
4h ps://www.sp i.eus/es/ayudas/elka ek/ - Las Access: 20/12/2022
5h ps:// esea ch-and-inno a ion.ec.eu opa.eu/ unding/ unding-oppo uni ies/ unding-p og ammes-
and-open-calls/ho izon-2020en−Las Access : 20/12/2022
4.1. SUMMARY AND DISCUSSION. 91
– Publica ion o esea ch esul s in Open Access scien i ic jou nals and con e -
ences.
– Open access o esea ch da a.
– A epo jus i ying compliance wi h he echnical and scien i ic condi ions
imposed in he awa d o he g an , indica ing he ac i i ies ca ied ou and
he esul s ob ained.
– Economic epo jus i ying he cos o he ac i i ies ca ied ou .
– Ce i ica ion documen o p ojec expendi u e, and o he s.
The UPV/EHU is pa o a conso ium o 25 Eu opean companies and esea ch
cen es in his p ojec . The wo k is ocused on he coo dina ion o wo o he wo k
packages and he implemen a ion and pa icipa ion in se e al o he ac i i ies o
he o he wo k packages.
iii Technical and scien i ic equi emen s o p ojec s:
•Fo P ojec 1, he echnical and scien i ic equi emen s we e o mula ed and
ag eed upon by he CFAA, he P ojec Manage and he P incipal In es iga o ;
and inally app o ed by he CFAA.
•Fo P ojec 2, he echnical and scien i ic equi emen s we e o mula ed and
ag eed upon be ween he conso ium’s eigh Resea ch Cen es and Uni e si ies
membe s, aligned wi h he lines o esea ch p oposed by he Basque Go e nmen
o he implemen a ion o hese p ojec s. In his case, he Basque Go e nmen
app o ed he echnical and scien i ic p oposals o he p ojec unde i s c i e ia.
•Fo P ojec 3, he echnical and scien i ic equi emen s we e o mula ed and
ag eed upon among he conso ium’s 25 membe companies and esea ch cen es.
These equi emen s we e o mula ed acco ding o he guidelines o he ”Fac o ies-
o - he-Fu u e (FoF) Public P i a e Pa ne ship” call 6, EU esea ch and inno a-
ion unding p og amme ”Ho izon 2020”. In his case, he Eu opean Commission
inally app o ed he equi ed unding aligned o he echnical and scien i ic p o-
posals o he p ojec .
A. Desc ip ion o p ojec s, p ojec objec i es and equi emen s - Ac i i y i.i
o he p oposed Me hodology (See sec ion 4.1.8.2.)
iP ojec 1 - De elopmen o a eal- ime machine moni o ing pla o m.
Fo his p ojec , he echnical and quali y equi emen s o p ojec pe o mance, ob-
jec i es o be achie ed, and ime and cos s we e ag eed upon be ween he CFAA and
he p ojec ’s lead esea che .
6h ps://www.e a.eu/ ac o ies- u u e - Las Access: 20/12/2022

92 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
i.i The p ojec eam consis ed o ou people:
•Lead esea che .
•P ojec manage .
•P ojec supe iso .
•Expe ad iso .
i.ii The ag eed objec i es o he p ojec we e o:
•C ea e a digi al win on a CFAA machining cen e.
•Pe o m an analysis o he a iables (P og ammable Logic Con olle (PLC)
and Compu e Nume ical Con ol (CNC)) necessa y o moni o he machine
in ques ion.
•To achie e a scalable, high-pe o mance implemen a ion o he digi al win
ha enables nea eal- ime da a p ocessing wi hou loss o in o ma ion.
•C ea e a moni o ing dashboa d ha shows he da a o he digi al win and
he s a us o he a iables being p ocessed in s eaming.
•Make use o 5G echnologies o he i ualisa ion o se ices.
i.iii The echnical equi emen s app o ed o he p ojec we e:
•Use open sou ce echnologies o da a connec i i y, p ocessing, analysis and
isualisa ion.
•Make use o indus ial p o ocols o da a inges ion.
•De elopmen o a digi al win p o o ype using Spa k S uc u ed S eaming 7,
unning on a single node.
•The p og am mus il e and p ocess he de ined a iables.
•The deployed in as uc u e shall be able o de ec possible anomalies in he
da a h ough he de ec ion o ou lie s.
•Use o da abase o he s o age o da a so ed h ough imes amps.
•Rep esen da a h ough simple and in ui i e g aphs on a dashboa d so CFAA
membe s can unde s and wha is happening on he machine.
•Remo e ope a ion o he in as uc u e hos ed on i ual machines in a p i a e
5G ne wo k.
•Ensu ing sa e connec ion o he in as uc u e.
i.i The quali y equi emen s o he p ojec we e:
•De ec ion o signal p ocessing inciden s and ensu ing da a pe sis ence.
•Machine signal p ocessing wi h high equency.
7Wha is Apache Spa k S uc u ed S eaming? - h ps://docs.da ab icks.com/s uc u ed-
s eaming/index.h ml - Las Access: 20/12/2022
4.1. SUMMARY AND DISCUSSION. 93
•Analysis o a ailable open-sou ce ools.
•Pla o m pe o mance.
•Compliance wi h he ini ial schedule.
•Scien i ic publica ions gene a ed om wo k done.
ii P ojec 2 - P edic ion o cu ing ool wea in b oaching machining.
Fo his p ojec , he p ojec ’s echnical and pe o mance quali y equi emen s, objec-
i es o be achie ed, and imes and cos s we e ag eed upon be ween he conso ium
o med by he Resea ch Cen es and he Uni e si ies. The Basque Go e nmen sub-
sequen ly app o ed hese.
In his p ojec , he wo k o he CFAA was educed o ca ying ou one o he p ojec
asks. The ask was o use da a analysis and he implemen a ion o AI algo i hms
o p edic he wea o cu ing ools used in b oaching machining. Fo his pu pose,
machine a iables we e used as ea u es and wea as a a ge . Da ase s om wo
machine es s ha sha e simila i ies so ha a single model is alid o bo h we e
used o he analysis. In addi ion, di e en machine lea ning models we e examined
o ob ain he one ha bes i s he ac ual da a.
ii.i The p ojec eam consis ed o six people:
•Lead esea che .
•Task coo dina o .
•Expe ad iso .
•Th ee machine echnicians esea che s.
ii.ii The ag eed objec i es o he p ojec we e:
•De elopmen o a beha iou al model o manu ac u ing p ocesses aimed a
quali y p edic ion (su ace and geome ic) o b oaching.
•De elopmen o unc ionali ies o machine ool condi ion moni o ing.
•De ine and de elop a da a a chi ec u e ha in eg a es in o ma ion ela ed o
he manu ac u ing p ocess and quali y cha ac e isa ion.
ii.iii The echnical equi emen s app o ed o he p ojec we e:
•De e mine ela ionships be ween p ocess and pa quali y a iables, allowing
o es ablish p ocess con ol ac ions a he machine le el ( eal- ime) and ac o y
le el (ea ly de ec de ec ion).
•C ea ion o a moni o ing sys em o cha ac e ise p oduc quali y.
•Cu ing ool wea p edic ion.
ii.i The quali y equi emen s o he p ojec we e:
•Analysis o machine da a on edge compu ing de ices in eal- ime.
94 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
•P edic cu ing ool wea wi h high accu acy.
•Compliance wi h he ini ial schedule.
•Model pe o mance.
•Publica ions gene a ed om wo k done.
iii P ojec 3 - De elopmen o i ual senso s o manu ac u ing p ocesses
h ough analysis o machine da a and pa quali y.
Fo his p ojec , he echnical and quali y equi emen s o p ojec pe o mance,
objec i es o be achie ed, ime and cos s we e ag eed upon by he 25 Eu opean
companies, esea ch cen es and uni e si ies o ming he conso ium. The Eu opean
Commission subsequen ly app o ed hese.
Fo his p ojec , he UPV/EHU was esponsible o managing wo wo k packages
and pa icipa ing in se e al asks in six o he nine wo k packages. We will show
he in o ma ion ela ed o comple ing one o hese asks. Speci ically, his ask con-
sis ed o de eloping i ual senso s (a i ual senso is a ”pu e so wa e senso which
au onomously p oduces signals by combining and agg ega ing signals ha i ecei es
(synch onously o asynch onously) om physical, o o he i ual senso s” [157]),
o p ocess con ol, which was able o de e mine he s a e and he di e en phenom-
ena (b eaks, wea , and o he s.) occu ing in he cu ing ool, based on he eal- ime
analysis o he machine da a and he da a ob ained om he su ace quali y o he
pa . Di e en a i icial ision echniques we e used o isola e, measu e, and iden i y
b eaks and wea on he cu ing ools o de e mine he su ace quali y o he pa .
iii.i The p ojec eam consis ed o nine people:
•Fou esea che s.
•Task coo dina o .
•Expe ad iso .
•Th ee machine echnicians esea che s.
iii.ii The ag eed objec i es o he p ojec we e:
•De elopmen o a i ual senso o he b oaching p ocess.
•De ine and de elop a da a a chi ec u e ha in eg a es in o ma ion ela ed o
he manu ac u ing p ocess and quali y cha ac e isa ion.
iii.iii The echnical equi emen s app o ed o he p ojec we e:
•De e mine ela ionships be ween p ocess and ool quali y a iables.
•To de e mine by c ea ing a i ual senso he di e en phenomena occu ing
in he cu ing ool du ing he b oaching p ocess.
•Cu ing ool wea p edic ion.
4.1. SUMMARY AND DISCUSSION. 95
iii.i The quali y equi emen s o he p ojec we e:
•Analysis o a ailable machine and senso da a.
•Wea da a analysis.
•Co ela ions be ween machine and wea da a.
•Compliance wi h he ini ial schedule.
•Model pe o mance.
•Publica ions gene a ed om wo k done.
i De ini ion o Quali y C i e ia Requi emen s o p ojec s.
Based on he in o ma ion ga he ed om he e iew o KPIs (Table 11.1) and he CSC
(Table 11.3) om CFAA, he quali y equi emen s de ined o each o he p ojec s
we e analysed o de e mine which o hose men ioned abo e we e a ec ed by he
quali y equi emen s de ined o each o he p ojec s.
This in o ma ion can be ound in Table 4.1.
ID Ca ego y Desc ip ion P ojec
1
P ojec
2
P ojec
3
CSC - 1 CSC - Cos
Managemen Re u n on In es men o he p ojec X X X
CSC - 2 CSC - Knowledge
Managemen
Knowledge gene a ion ega ding p ojec
ac i i ies (e.g., ools, echniques, ap-
p oaches, p ocesses)
XXX
CSC - 3
CSC - Quali y
Managemen
Cus ome sa is ac ion ega ding he deli -
e able. XXX
CSC - 4
Cus ome sa is ac ion ega ding he qual-
i y o deli e y ac i i ies o he speci ic
p ojec
XXX
CSC - 5 Deg ee o which he deli e able mee s i s
in ended pu pose. XXX
CSC - 6 The deli e able mee he de ined quali y
c i e ia. XXX
CSC - 7 CSC - Risk
Managemen Wo kplace Sa e y X X X
CSC - 8 CSC - Scope
Managemen P ojec goal was achie ed X X X
CSC - 9 CSC - S ake-
holde
Managemen
Deli e y ac i i ies ha e a good epu a ion X X X
CSC - 10 Repu a ion o he o ganiza ion has in-
c eased XXX
KPI - 1 KPI - Financial Inc ease he e u n o in es men o he
p ojec s XXX
102 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
i Task Desc ip ion de ini ion.
Following he me hodology, he asks o be ca ied ou in he p ojec a e de ined.
The con ibu ions o each o he sub asks ij o he ac i i ies ai o each o he c i e ia
cj o be assessed o he p ojec , a le el 3 and 4 o he WBS and QBS, a e desc ibed:
•Phase 1: ”Managemen and planning.”
WBS Le el 2 P3Ph1 - Managemen and Planning
WBS Le el 3 P3Ph1 -
1.1
P3Ph1 -
1.2
P3Ph1 -
1.3
P3Ph1 -
1.4
cj0.250 0.250 0.250 0.250
Task WBS Le el
4 ij
P3Ph1 - 1.1.1 0.167
P3Ph1 - 1.1.2 0.167
P3Ph1 - 1.1.3 0.167
P3Ph1 - 1.1.4 0.167 0.111
P3Ph1 - 1.1.5 0.167
P3Ph1 - 1.1.6 0.167 0.111
P3Ph1 - 1.2.1 0.111
P3Ph1 - 1.2.2 0.111 0.250
P3Ph1 - 1.2.3 0.111
P3Ph1 - 1.2.4 0.111
P3Ph1 - 1.2.5 0.111
P3Ph1 - 1.2.6 0.111
P3Ph1 - 1.2.7 0.111
P3Ph1 - 1.3.1 0.250
P3Ph1 - 1.3.2 0.250
P3Ph1 - 1.3.3 0.250
P3Ph1 - 1.4.1 1.000
Table 4.6: Phase 1: QBS-WBS le els 2, 3 and 4.

4.1. SUMMARY AND DISCUSSION. 103
•Phase 2: ”Da a analysis ac i i ies.”
WBS Le el 2 P3Ph2 - Da a Analysis Ac i i ies
WBS Le el 3 P3Ph2 - 2.1 P3Ph2 - 2.2
cj0.500 0.500
Task WBS Le el 4 ij
P3Ph2 - 2.1.1 0.167
P3Ph2 - 2.1.2 0.167
P3Ph2 - 2.1.3 0.167
P3Ph2 - 2.1.4 0.167
P3Ph2 - 2.1.5 0.167
P3Ph2 - 2.1.6 0.167
P3Ph2 - 2.2.1 0.333
P3Ph2 - 2.2.2 0.333
P3Ph2 - 2.2.3 0.333
Table 4.7: Phase 2: QBS-WBS le els 2, 3 and 4.
•Phase 3: ”Wea analysis ac i i ies.”
WBS Le el 2 P3Ph3 - Wea Analysis Ac i i ies
WBS Le el 3 P3Ph3 - 3.1 P3Ph3 - 3.2
cj0.500 0.500
Task WBS Le el 4 ij
P3Ph3 - 3.1.1 0.143
P3Ph3 - 3.1.2 0.143
P3Ph3 - 3.1.3 0.143
P3Ph3 - 3.1.4 0.143
P3Ph3 - 3.1.5 0.143
P3Ph3 - 3.1.6 0.143
P3Ph3 - 3.1.7 0.143
P3Ph3 - 3.2.1 0.200
P3Ph3 - 3.2.2 0.200
P3Ph3 - 3.2.3 0.200
P3Ph3 - 3.2.4 0.200
P3Ph3 - 3.2.5 0.200
Table 4.8: Phase 3: QBS-WBS le els 2, 3 and 4.
104 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
•Phase 4: ”Co ela ion and modelling.”
WBS Le el 2 P3Ph4 - Co ela ion and modelling
WBS Le el 3 P3Ph4 - 4.1 P3Ph4 - 4.2
cj0.500 0.500
Task WBS Le el 4 ij
P3Ph4 - 4.1.1 0.333
P3Ph4 - 4.1.2 0.333
P3Ph4 - 4.1.3 0.333
P3Ph4 - 4.2.1 0.500
P3Ph4 - 4.2.2 0.500
Table 4.9: Phase 4: QBS-WBS le els 2, 3 and 4.
•Phase 5: ”E alua ion and elease.”
WBS Le el 2 P3Ph5 - E alua ion and elease
WBS Le el 3 P3Ph5 -
5.1
P3Ph5 -
5.2
P3Ph5 -
5.3
P3Ph5 -
5.4
cj0.250 0.250 0.250 0.250
Task WBS Le el
4 ij
P3Ph5 - 5.1.1 1.000
P3Ph5 - 5.2.1 1.000
P3Ph5 - 5.3.1 0.250
P3Ph5 - 5.3.2 0.250
P3Ph5 - 5.3.3 0.250
P3Ph5 - 5.3.4 0.250
P3Ph5 - 5.4.1 0.500
P3Ph5 - 5.4.2 0.500
Table 4.10: Phase 5: QBS-WBS le els 2, 3 and 4.
4.1. SUMMARY AND DISCUSSION. 105
•Phase 6: ”Pos -p ojec e alua ion.”
WBS Le el 2 P3Ph6 - Pos p ojec e alua ion
WBS Le el 3 P3Ph6 -
6.1
P3Ph6 -
6.2
P3Ph6 -
6.3
P3Ph6 -
6.4
P3Ph6 -
6.5
cj0.200 0.200 0.200 0.200 0.200
Task WBS
Le el 4 ij
P3Ph6 - 6.1.1 1.000
P3Ph6 - 6.2.1 1.000
P3Ph6 - 6.3.1 1.000
P3Ph6 - 6.4.1 1.000
P3Ph6 - 6.5.1 1.000
Table 4.11: Phase 6: QBS-WBS le els 2, 3 and 4.
Gan Cha o he p ojec .
Wi h he in o ma ion collec ed, we de ined he Gan cha o P ojec 3 and he
con ol poin s (cp) o he p ojec .
Time (days) 15 30 45 60 75 90 105 120 135 150 165 180 195 210
P3Ph1 Managemen and
planning cp cp cp cp cp cp
P3Ph2 Da a analysis cp cp cp cp
P3Ph3 Wea analysis cp cp cp cp
P3Ph4 Co ela ion and
modelling cp cp
P3Ph5 E alua ion and e-
lease cp cp cp cp
P3Ph6 Pos -p ojec e alu-
a ion cp cp
Table 4.12: Gan Cha o he p ojec
106 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
Wi h he in o ma ion collec ed om he con ol poin s, plus he inpu s om each
o he sub- asks o he ac i i ies, he ables o he Planned Con ibu ion o Wo k
Scheduled (PCWS) (See sec ion 4.1.8.3) can be c ea ed:
PCWS: P3Ph1 - Managemen and planning.
P3QC2
(0,050)
11 = 1,0
cp
11 ( )
Weigh ed sum
Pjwjϕjxcp
j cp
1j( )
15 0,167 0,008
30 0,333 0,017
45 0,500 0,025
60 0,667 0,033
75 0,833 0,042
90 1,000 0,050
Table 4.13: PCWS: P3Ph1
PCWS: P3Ph2 Da a analysis ac i i ies.
P3QC1
(0,150)
22 = 0,700
cp
22 ( )
P3QC6
(0,150)
23 = 0,200
cp
23 ( )
P3QC4
(0,200)
25 = 0,150
cp
25 ( )
P3QC5
(0,300)
26 = 0,200
cp
26 ( )
Weigh ed sum
Pjwjϕjxcp
j cp
1j( )
75 0,175 0,050 0,038 0,050 0,056
90 0,350 0,100 0,075 0,100 0,113
105 0,525 0,150 0,113 0,150 0,169
120 0,700 0,200 0,150 0,200 0,225
Table 4.14: PCWS: P3Ph2
PCWS: P3Ph3 - Wea analysis ac i i ies.
P3QC6
(0,150)
33 = 0,500
cp
33 ( )
P3QC4
(0,200)
35 = 0,150
cp
35 ( )
P3QC5
(0,300)
36 = 0,200
cp
36 ( )
Weigh ed sum
Pjwjϕjxcp
j cp
1j( )
105 0,125 0,038 0,050 0,041
120 0,250 0,075 0,100 0,083
135 0,375 0,113 0,150 0,124
150 0,500 0,150 0,200 0,165
Table 4.15: PCWS: P3Ph3
4.1. SUMMARY AND DISCUSSION. 107
PCWS: P3Ph4 - Co ela ion and modelling.
P3QC6
(0,150)
43 = 0,200
cp
43 ( )
P3QC3
(0,150)
44 = 0,500
cp
44 ( )
P3QC4
(0,200)
45 = 0,500
cp
45 ( )
P3QC5
(0,300)
46 = 0,200
cp
46 ( )
Weigh ed sum
Pjwjϕjxcp
j cp
1j( )
135 0,100 0,250 0,250 0,100 0,133
150 0,200 0,500 0,500 0,200 0,265
Table 4.16: PCWS: P3Ph4
PCWS: P3Ph5 - E alua ion and elease.
P3QC1
(0,150)
52 = 0,300
cp
52 ( )
P3QC6
(0,150)
53 = 0,100
cp
53 ( )
P3QC3
(0,150)
54 = 0,500
cp
54 ( )
P3QC4
(0,200)
55 = 0,200
cp
55 ( )
P3QC5
(0,300)
56 = 0,200
cp
56 ( )
Weigh ed sum
Pjwjϕjxcp
j cp
1j( )
135 0,075 0,025 0,125 0,050 0,050 0,059
150 0,150 0,050 0,250 0,100 0,100 0,118
165 0,225 0,075 0,375 0,150 0,150 0,176
180 0,300 0,100 0,500 0,200 0,200 0,235
Table 4.17: PCWS: P3Ph5
PCWS: P3Ph6 - Pos -p ojec e alua ion.
P3QC5
(0,300)
66 = 0,200
cp
66 ( )
Weigh ed sum
Pjwjϕjxcp
j cp
1j( )
195 0,100 0,030
210 0,200 0,060
Table 4.18: PCWS: P3Ph6

108 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
In he same way, we can de ine he able o he Planned Quali y o he Wo k Scheduled
(PQWS) (See Sec ion 4.1.8.3).
Ac i i y,
ai
15 30 45 60 75 90 105 120 135 150 165 180 195 210
P3Ph1 0,008 0,017 0,025 0,033 0,042 0,05 0,05 0,05 0,05 0,05 0,05 0,05 0,05 0,05
P3Ph2 - - - - 0,056 0,113 0,169 0,225 0,225 0,225 0,225 0,225 0,225 0,225
P3Ph3 - - - - - - 0,041 0,083 0,124 0,165 0,165 0,165 0,165 0,165
P3Ph4 - - - - - - - - 0,133 0,265 0,265 0,265 0,265 0,265
P3Ph5 - - - - - - - - 0,059 0,118 0,176 0,235 0,235 0,235
P3Ph6 - - - - - - - - - - - - 0,03 0,06
P QW S 0,008 0,017 0,025 0,033 0,098 0,163 0,26 0,358 0,59 0,823 0,881 0,94 0,97 1
Table 4.19: Planned quali y o wo k scheduled
Now, assuming ha he p ojec s a s and inishes wi hin he planned ime, i can
be said ha he Planned Con ibu ion o he Wo k Pe o med (PCWP) would be:
Ac i i y,
ai
15 30 45 60 75 90 105 120 135 150 165 180 195 210
P3Ph1 0,008 0,017 0,025 0,033 0,042 0,05 0,05 0,05 0,05 0,05 0,05 0,05 0,050 0,05
P3Ph2 - - - - 0,056 0,113 0,169 0,225 0,225 0,225 0,225 0,225 0,225 0,225
P3Ph3 - - - - - - 0,041 0,083 0,124 0,165 0,165 0,165 0,165 0,165
P3Ph4 - - - - - - - - 0,133 0,265 0,265 0,265 0,265 0,265
P3Ph5 - - - - - - - - 0,059 0,118 0,176 0,235 0,235 0,235
P3Ph6 - - - - - - - - - - - - 0,03 0,06
P QW P 0,008 0,017 0,025 0,033 0,098 0,163 0,26 0,358 0,59 0,823 0,881 0,94 0,97 1
Table 4.20: Planned con ibu ion o he wo k pe o med
ii Quali y Me ics Re iew (See sec ion iii.ii)
Fo P ojec 3, 3 con ol poin s we e de ined o e iew he p ojec quali y me ics. The
ac i i ies we e ca ied ou wi hin he scheduled ime, and he e we e no de ia ions
om he plan. The e o e, we will show he e alua ion o each p ojec ask in he
Ac ual Con ibu ion o Wo k Pe o med (ACWP) ables (See sec ion 4.1.8.3).
The e alua ion was ca ied ou om 0 o 1, wi h 1 being he op imal esul o he
ask in e ms o quali y.
4.1. SUMMARY AND DISCUSSION. 109
The ollowing a ings we e ob ained o he i s phase o he p ojec :
P3P h1.1ϕj
P3P h1.2ϕj
P3P h1.3ϕj
P3P h1.4ϕjWeigh ed sum
cj0,25 0,25 0,25 0,25
15
P3Ph1 -
1.1.1 0,167 0,9 - - - - - - 0,038
P3Ph1 -
1.1.2 0,167 0,8 - - - - - - 0,033
P3Ph1 -
1.1.3 0,167 0,8 - - - - - - 0,033
30
P3Ph1 -
1.1.4 0,167 0,9 0,111 0,7 - - - - 0,057
P3Ph1 -
1.1.5 0,167 1 - - - - - - 0,042
45
P3Ph1 -
1.1.6 0,167 0,9 0,111 1 - - - - 0,065
P3Ph1 -
1.2.1 - - 0,111 0,8 - - - - 0,022
IQT2.2
- 1.2.2 - - 0,111 1 0,25 0,5 - - 0,059
60
IQT2.2
- 1.2.3 - - 0,111 0,8 - - - - 0,022
P3Ph1 -
1.2.4 - - 0,111 0,9 - - - - 0,025
P3Ph1 -
1.2.5 - - 0,111 0,8 - - - - 0,022
P3Ph1 -
1.2.6 - - 0,111 1 - - - - 0,028
75
IQT2.2
- 1.2.7 - - 0,111 0,9 - - - - 0,025
P3Ph1 -
1.3.1 - - - - 0,25 1 - - 0,063
90
IQT2.2
- 1.3.2 - - - - 0,25 1 - - 0,063
IQT2.2
- 1.3.3 - - - - 0,25 0,8 - - 0,050
P3Ph1 -
1.4.1 - - - - - - 1 0,9 0,225
Table 4.21: Ac ual con ibu ion o wo k pe o med - Phase 1
110 CHAPTER 4. SUMMARY AND DISCUSSION OF THE RESULTS.
The in o ma ion collec ed in Table 4.21, can be summa ised as ollows:
P3QC2
(0,050) Weigh ed sum
Pjwjϕj(ˆxj)ˆ 1j( )
11 = 1,0
cp
11( )ϕ1(ˆx1)
15 0,167 0,1042 0,001
30 0,333 0,2028 0,003
45 0,5 0,3493 0,009
60 0,667 0,4465 0,015
75 0,833 0,534 0,022
90 1 0,8715 0,044
Table 4.22: ACWP - P3Ph1
Following he same e alua ion me hod, assessmen s we e made o he ac i i ies ca -
ied ou in he subsequen phases o he p ojec .
Ac ual con ibu ion o wo k pe o med - P3Ph2
P3QC1
(0,150)
22 = 0,700
P3QC6
(0,150)
23 = 0,200
P3QC4
(0,200)
25 = 0,150
P3QC5
(0,300)
26 = 0,200
Weigh ed sum
Pjwjϕj(ˆxj) ˆ 2j( )
cp
22( )ϕ2(ˆx2) cp
23( )ϕ3(ˆx3) cp
25( )ϕ5(ˆx5) cp
26( )ϕ6(ˆx6)
75 0,175 0,225 0,050 0,225 0,038 0,225 0,050 0,225 0,013
90 0,350 0,442 0,100 0,442 0,075 0,442 0,100 0,442 0,050
105 0,525 0,742 0,150 0,742 0,113 0,742 0,150 0,742 0,125
120 0,700 0,892 0,200 0,892 0,150 0,892 0,200 0,892 0,201
Table 4.23: ACWP - P3Ph2
Ac ual con ibu ion o wo k pe o med - P3Ph3
P3QC6
(0,150)
33 = 0,500
P3QC4
(0,200)
35 = 0,150
P3QC5
(0,300)
36 = 0,200
Weigh ed sum
Pjwjϕj(ˆxj) ˆ 3j( )
cp
33 ( )ϕ3(ˆx3) cp
35 ( )ϕ5(ˆx5) cp
36 ( )ϕ6(ˆx6)
105 0,125 0,136 0,038 0,136 0,050 0,136 0,006
120 0,250 0,314 0,075 0,314 0,100 0,314 0,026
135 0,375 0,633 0,113 0,633 0,150 0,633 0,078
150 0,500 0,893 0,150 0,893 0,200 0,893 0,147
Table 4.24: ACWP - P3Ph3
4.1. SUMMARY AND DISCUSSION. 111
Ac ual con ibu ion o wo k pe o med - P3Ph4
P3QC6
(0,150)
43 = 0,200
P3QC3
(0,150)
44 = 0,500
P3QC4
(0,200)
45 = 0,500
P3QC5
(0,300)
46 = 0,200
Weigh ed sum
Pjwjϕj(ˆxj) ˆ 4j( )
cp
43 ( )ϕ3(ˆx3) cp
44 ( )ϕ4(ˆx4) cp
45 ( )ϕ5(ˆx5) cp
46 ( )ϕ6(ˆx6)
135 0,100 0,467 0,250 0,467 0,250 0,467 0,100 0,467 0,062
150 0,200 0,917 0,500 0,917 0,500 0,917 0,200 0,917 0,243
Table 4.25: ACWP - P3Ph4
Ac ual con ibu ion o wo k pe o med - P3Ph5
P3QC1
(0,150)
52 = 0,300
P3QC6
(0,150)
53 = 0,100
P3QC3
(0,150)
54 = 0,500
P3QC4
(0,200)
55 = 0,200
P3QC5
(0,300)
56 = 0,200
Weigh ed sum
Pjwjϕj(ˆxj)ˆ 5j(
)
cp
52( )ϕ2(ˆx2) cp
53( )ϕ3(ˆx3) cp
54( )ϕ4(ˆx4) cp
55( )ϕ5(ˆx5) cp
56( )ϕ6(ˆx6)
135 0,075 0,425 0,025 0,425 0,125 0,425 0,050 0,425 0,050 0,425 0,025
150 0,150 0,531 0,050 0,531 0,250 0,531 0,100 0,531 0,100 0,531 0,062
165 0,225 0,750 0,075 0,750 0,375 0,750 0,150 0,750 0,150 0,750 0,132
180 0,300 0,850 0,100 0,850 0,500 0,850 0,200 0,850 0,200 0,850 0,200
Table 4.26: ACWP - P3Ph5
Ac ual con ibu ion o wo k pe o med - P3Ph6
P3QC5
0,300
66 = 0,200
Weigh ed sum
Pjwjϕj(ˆxj)ˆ 6j(
)
cp
66( )ϕ6(ˆx6)
195 0,100 0,52 0,016
210 0,200 0,86 0,052
Table 4.27: ACWP - P3Ph6
118 Re e ences
[36] H. E zkowi z and L. Leydesdo , “The dynamics o inno a ion: om na ional sys-
ems and “mode 2” o a iple helix o uni e si y–indus y–go e nmen ela ions,”
Resea ch Policy, ol. 29, no. 2, pp. 109–123, 2000.
[37] M. Gibbons, The new p oduc ion o knowledge: The dynamics o science and e-
sea ch in con empo a y socie ies. Sage, 1994.
[38] E. Commission, “Imp o ing knowledge ans e be ween esea ch ins i u ions and
indus y ac oss eu ope,” ech. ep., Eu opean Commission, 2007.
[39] A. Spi ho en, B. Cla ysse, and M. Knockae , “Building abso p i e capaci y o
o ganise inbound open inno a ion in adi ional indus ies,” Techno a ion, ol. 30,
no. 2, pp. 130–141, 2010.
[40] U. AENOR, “Une 166001: 2006: Ges i´on de la i+ d+ i: Requisi os de un p oyec o
de i+d+i,” No ma espa˜nola, 2006.
[41] R. S e nbe g, “Reasons o he genesis o high- ech egions— heo e ical explana-
ion and empi ical e idence,” Geo o um, ol. 27, no. 2, pp. 205–223, 1996.
[42] J. WALLIN, O. ISAKSSON, A. LARSSON, and B.-O. ELFSTR¨
OM, “B idging he
gap be ween uni e si y and indus y: Th ee mechanisms o inno a ion e iciency,”
In e na ional Jou nal o Inno a ion and Technology Managemen , ol. 11, no. 01,
p. 1440005, 2014.
[43] R. Land y, N. Ama a, J.-S. Clou ie , and N. Halilem, “Technology ans e o gani-
za ions: Se ices and business models,” Techno a ion, ol. 33, no. 12, pp. 431–449,
2013.
[44] J. Howells, “In e media ion and he ole o in e media ies in inno a ion,” Resea ch
Policy, ol. 35, no. 5, pp. 715–728, 2006.
[45] M. Agogu´e, E. Be he , T. F edbe g, P. Le Masson, B. Seg es in, M. S oe zel,
M. Wiene , and A. Ys ¨om, “Explica ing he ole o inno a ion in e media ies in
he “unknown”: a con ingency app oach,” Jou nal o S a egy and Managemen ,
2017.
[46] H. Chesb ough, W. Vanha e beke, T. Bakici, and H. Lopez-Vega, “Open inno a-
ion and public policy in eu ope,” 2011.
[47] A. A undel and A. Geuna, “P oximi y and he use o public science by inno a i e
eu opean i ms,” Economics o Inno a ion and New Technology, ol. 13, no. 6,
pp. 559–580, 2004.
[48] G. Schiuma, D. Ca lucci, A. Le o, R. Land y, and N. Ama a, “Elucida ion and
enhancemen o knowledge and echnology ans e business models,” Vine, 2012.
[49] Y. S. Lee, “The sus ainabili y o uni e si y-indus y esea ch collabo a ion: An
empi ical assessmen ,” The jou nal o Technology ans e , ol. 25, no. 2, pp. 111–
133, 2000.
[50] P. D’es e and M. Pe kmann, “Why do academics engage wi h indus y? he en-
ep eneu ial uni e si y and indi idual mo i a ions,” The Jou nal o Technology
T ans e , ol. 36, no. 3, pp. 316–339, 2011.

Re e ences 119
[51] R. S ones, “You can’ always ge wha you wan ,” Le I Bleed, 1969.
[52] M. San o o and P. Bie ly, “Facili a o s o knowledge ans e in uni e si y-indus y
collabo a ions: A knowledge-based pe spec i e,” IEEE T ansac ions on Enginee -
ing Managemen , ol. 53, no. 4, pp. 495–507, 2006.
[53] T. Nomakuchi and M. Takahashi, “A s udy abou p ojec managemen o
indus y-uni e si y coope a ion dilemma,” P ocedia Compu e Science, ol. 64,
pp. 47–54, 2015. Con e ence on ENTERp ise In o ma ion Sys ems/In e na ional
Con e ence on P ojec MANagemen /Con e ence on Heal h and Social Ca e In o -
ma ion Sys ems and Technologies, CENTERIS/P ojMAN / HCis 2015 Oc obe
7-9, 2015.
[54] O. Haup man and K. K. Hi ji, “Managing in eg a ion and coo dina ion in c oss-
unc ional eams: an in e na ional s udy o concu en enginee ing p oduc de el-
opmen ,” R&D Managemen , ol. 29, no. 2, pp. 179–192, 1999.
[55] V. Mo andi, “The managemen o indus y–uni e si y join esea ch p ojec s: how
do pa ne s coo dina e and con ol &d ac i i ies?,” The Jou nal o Technology
T ans e , ol. 38, no. 2, pp. 69–92, 2013.
[56] A. H. V. D. Ven, A. L. Delbecq, and R. Koenig, “De e minan s o coo dina ion
modes wi hin o ganiza ions,” Ame ican Sociological Re iew, ol. 41, no. 2, pp. 322–
338, 1976.
[57] W. H. A. Johnson and D. A. Johns on, “O ganisa ional knowledge c ea ing p o-
cesses and he pe o mance o uni e si y-indus y collabo a i e &d p ojec s,” In .
J. Technol. Manag., ol. 27, pp. 93–114, 2004.
[58] S. Koba g, J. S ump -Wolle sheim, and I. M. Welpe, “Uni e si y-indus y collab-
o a ions and p oduc inno a ion pe o mance: The mode a ing e ec s o abso p-
i e capaci y and inno a ion compe encies,” The Jou nal o Technology T ans e ,
ol. 43, no. 6, pp. 1696–1724, 2018.
[59] S. U. Nsanzumuhi e and W. G oo , “Con ex pe spec i e on uni e si y-indus y
collabo a ion p ocesses: A sys ema ic e iew o li e a u e,” Jou nal o Cleane
P oduc ion, ol. 258, p. 120861, 2020.
[60] A. L. Oli e , K. Mon gome y, and S. Ba da, “The mul i-le el p ocess o us and
lea ning in uni e si y–indus y inno a ion collabo a ions,” The Jou nal o Tech-
nology T ans e , ol. 45, no. 3, pp. 758–779, 2020.
[61] E. Bellini, G. Pi oli, and L. Pennacchio, “Collabo a i e know-how and us in
uni e si y–indus y collabo a ions: Empi ical e idence om ic i ms,” The Jou nal
o Technology T ans e , ol. 44, no. 6, pp. 1939–1963, 2019.
[62] E. Alba s, I. Fiegenbaum, and J. A. Cunningham, “A mic o le el s udy o uni e -
si y indus y collabo a i e li ecycle key pe o mance indica o s,” The Jou nal o
Technology T ans e , ol. 43, no. 2, pp. 389–431, 2018.
[63] C. Wol and A. Nuseibah, “A p ojec ized pa h owa ds an e ec i e indus y-
uni e si y-clus e : Ruh alley,” in 2017 12 h In e na ional Scien i ic and Technical
Con e ence on Compu e Sciences and In o ma ion Technologies (CSIT), ol. 2,
120 Re e ences
pp. 123–131, 2017.
[64] J. om B ocke and S. Lippe, “Managing collabo a i e esea ch p ojec s: A syn hesis
o p ojec managemen li e a u e and di ec i es o u u e esea ch,” In e na ional
Jou nal o P ojec Managemen , ol. 33, no. 5, pp. 1022–1039, 2015.
[65] S. Ank ah and O. AL-Tabbaa, “Uni e si ies–indus y collabo a ion: A sys ema ic
e iew,” Scandina ian Jou nal o Managemen , ol. 31, no. 3, pp. 387–408, 2015.
[66] M. L. Mosb ooke , The implemen a ion o p ojec managemen : he p o essional’s
handbook. Basic Books, 1981.
[67] A. Gemino, B. H. Reich, and P. M. Se ado , “Agile, adi ional, and hyb id ap-
p oaches o p ojec success: Is hyb id a poo second choice?,” P ojec Managemen
Jou nal, ol. 52, no. 2, pp. 161–175, 2021.
[68] A. L. Olechowski, S. D. Eppinge , N. Jogleka , and K. Tomaschek, “Technology
eadiness le els: Sho comings and imp o emen oppo uni ies,” Sys ems Enginee -
ing, ol. 23, no. 4, pp. 395–408, 2020.
[69] ˚
Asa Fas -Be glund, L.-O. Blig˚a d, M. ˚
Ake man, and M. Ka lsson, “Using he l-
me hodology o design suppo ing ic - ools o p oduc ion ope a o s,” P ocedia
CIRP, ol. 17, pp. 726–731, 2014. Va ie y Managemen in Manu ac u ing.
[70] N. Becheikh, R. Land y, and N. Ama a, “Lessons om inno a ion empi ical s udies
in he manu ac u ing sec o : A sys ema ic e iew o he li e a u e om 1993–2003,”
Techno a ion, ol. 26, no. 5, pp. 644–664, 2006.
[71] J. C. Mankins, “Technology eadiness assessmen s: A e ospec i e,” Ac a As o-
nau ica, ol. 65, no. 9, pp. 1216–1223, 2009.
[72] B. J. Sause , J. E. Rami ez-Ma quez, D. Hen y, and D. DiMa zio, “A sys em
ma u i y index o he sys ems enginee ing li e cycle,” In e na ional Jou nal o
Indus ial and Sys ems Enginee ing, ol. 3, no. 6, pp. 673–691, 2008.
[73] J. C. Mankins, “App oaches o s a egic esea ch and echnology ( & ) analysis
and oad mapping,” Ac a As onau ica, ol. 51, no. 1, pp. 3–21, 2002.
[74] J. Smi h, “An al e na i e o echnology eadiness le els o non-de elopmen al i em
(ndi) so wa e,” in P oceedings o he 38 h Annual Hawaii In e na ional Con e ence
on Sys em Sciences, pp. 315a–315a, 2005.
[75] J. C. Mankins, “Technology eadiness le els,” Whi e Pape , Ap il, ol. 6, p. 1995,
1995.
[76] R. Van Die donck, K. Debacke e, and B. Engelen, “Uni e si y-indus y ela ion-
ships: How does he belgian academic communi y eel abou i ?,” Resea ch Policy,
ol. 19, no. 6, pp. 551–566, 1990.
[77] E. Pi a and C. Rossi-Lamas a, “Sys ems o indica o s o e alua e he pe o mance
o uni e si y-indus y alliances: a e iew o he li e a u e and di ec ions o u u e
esea ch,” Measu ing Business Excellence, 2013.
[78] M. Pe kmann, A. Neely, and K. Walsh, “How should i ms e alua e success in
uni e si y–indus y alliances? a pe o mance measu emen sys em,” R&D Man-
Re e ences 121
agemen , ol. 41, no. 2, pp. 202–216, 2011.
[79] R. Magnaye, B. Sause , P. Pa anakul, D. Nowicki, and W. Randall, “Ea ned eadi-
ness managemen o scheduling, moni o ing and e alua ing he de elopmen o
complex p oduc sys ems,” In e na ional Jou nal o P ojec Managemen , ol. 32,
no. 7, pp. 1246–1259, 2014.
[80] K. Smi h, “Measu ing Inno a ion,” in The Ox o d Handbook o Inno a ion, Ox o d
Uni e si y P ess, 01 2006.
[81] T. R. B owning and R. V. Ramasesh, “A su ey o ac i i y ne wo k-based p ocess
models o managing p oduc de elopmen p ojec s,” P oduc ion and Ope a ions
Managemen , ol. 16, no. 2, pp. 217–240, 2007.
[82] J. R. Hause , D. Clausing, e al., “The house o quali y,” Ha a d Business Re iew,
1988.
[83] S. Pugh, To al design: in eg a ed me hods o success ul p oduc enginee ing.
Addison-Wesley, 1991.
[84] J. E. Kasse , “The i s equi emen s elucida o demons a ion ( ed) ool,” Sys-
ems Enginee ing, ol. 7, no. 3, pp. 243–256, 2004.
[85] A. Volle hun, “Design- o-ma ke in eg a ing concep ual design and ma ke ing,”
Sys ems Enginee ing, ol. 5, no. 4, pp. 315–326, 2002.
[86] T. Shell, “The syn hesis o op imal sys ems design solu ions,” Sys ems Enginee ing,
ol. 6, no. 2, pp. 92–105, 2003.
[87] D. E. Whi ney e al., “Manu ac u ing by design,” Ha a d Business Re iew, ol. 66,
no. 4, pp. 83–91, 1988.
[88] K. B. Cla k, “P oduc de elopmen pe o mance: S a egy,” O ganiza ion, and
Managemen in he Wo ld Au o Indus y, 1991.
[89] S. C. Wheelw igh and K. B. Cla k, Re olu ionizing p oduc de elopmen : quan um
leaps in speed, e iciency, and quali y. Simon and Schus e , 1992.
[90] J. B ady, “Sys ems enginee ing and cos as an independen a iable,” Sys ems
Enginee ing, ol. 4, no. 4, pp. 233–241, 2001.
[91] B. W. Oppenheim, “Lean p oduc de elopmen low,” Sys ems Enginee ing, ol. 7,
no. 4, 2004.
[92] K. S. Pawa and H. D i a, “Pe o mance measu emen o p oduc design and de-
elopmen in a manu ac u ing en i onmen ,” In e na ional Jou nal o P oduc ion
Economics, ol. 60-61, pp. 61–68, 1999.
[93] A. Tiwana and M. Keil, “Con ol in in e nal and ou sou ced so wa e p ojec s,”
Jou nal o Managemen In o ma ion Sys ems, ol. 26, no. 3, pp. 9–44, 2009.
[94] R. G. Coope and E. J. Kleinschmid , “Winning businesses in p oduc de el-
opmen : The c i ical success ac o s,” Resea ch-Technology Managemen , ol. 50,
no. 3, pp. 52–66, 2007.
122 Re e ences
[95] D. D i and T. Lechle , “Plans a e no hing, changing plans is e e y hing: he
impac o changes on p ojec success,” Resea ch Policy, ol. 33, no. 1, pp. 1–15,
2004.
[96] J. Pin o and S. Man el, “The causes o p ojec ailu e,” IEEE T ansac ions on
Enginee ing Managemen , ol. 37, no. 4, pp. 269–276, 1990.
[97] J. K. Pin o and D. P. Sle in, “C i ical ac o s in success ul p ojec implemen a-
ion,” IEEE T ansac ions on Enginee ing Managemen , ol. EM-34, no. 1, pp. 22–
27, 1987.
[98] C. K. Ba , “Con olling new p oduc s: a con ingency app oach,” In e na ional
Jou nal o Technology Managemen , ol. 18, no. 5-8, pp. 395–413, 1999.
[99] H. Co es, J. Daaboul, J. Le Duigou, and B. Eyna d, “S a egic lean managemen :
In eg a ion o ope a ional pe o mance indica o s o s a egic lean managemen ,”
IFAC-Pape sOnLine, ol. 49, no. 12, pp. 65–70, 2016. 8 h IFAC Con e ence on
Manu ac u ing Modelling, Managemen and Con ol MIM 2016.
[100] S. u Rehman Too and S. O. Ogunlana, “Beyond he ‘i on iangle’: S akeholde
pe cep ion o key pe o mance indica o s (kpis) o la ge-scale public sec o de-
elopmen p ojec s,” In e na ional Jou nal o P ojec Managemen , ol. 28, no. 3,
pp. 228–236, 2010.
[101] H. Ke zne , P ojec managemen me ics, KPIs, and dashboa ds: a guide o mea-
su ing and moni o ing p ojec pe o mance. John Wiley & Sons, 2022.
[102] F. Espa˜na, C. C. Tsao, and M. Hause , “D i ing con inuous imp o emen by de-
eloping and le e aging lean key pe o mance indica o s,” in annual con e ence o
he in e na ional g oup o lean cons uc ion, ol. 20, 2012.
[103] U. Domb owski, K. Schmid chen, and D. Eben eich, “Balanced key pe o mance
indica o s in p oduc de elopmen ,” In e na ional Jou nal o Ma e ials, Mechanics
and Manu ac u ing, ol. 1, no. 1, pp. 27–31, 2013.
[104] F. A. Mi and A. H. Pinning on, “Explo ing he alue o p ojec managemen : Link-
ing p ojec managemen pe o mance and p ojec success,” In e na ional Jou nal
o P ojec Managemen , ol. 32, no. 2, pp. 202–217, 2014.
[105] O. Bizan, “The de e minan s o success o &d p ojec s: e idence om ame i-
can–is aeli esea ch alliances,” Resea ch Policy, ol. 32, no. 9, pp. 1619–1640, 2003.
[106] A. J. Shenha , A. Tishle , D. D i , S. Lipo e sky, and T. Lechle , “Re ining he
sea ch o p ojec success ac o s: a mul i a ia e, ypological app oach,” R&D Man-
agemen , ol. 32, no. 2, pp. 111–126, 2002.
[107] A. de Wi , “Measu emen o p ojec success,” In e na ional Jou nal o P ojec
Managemen , ol. 6, no. 3, pp. 164–170, 1988.
[108] T. Cooke-Da ies, “The “ eal” success ac o s on p ojec s,” In e na ional Jou nal
o P ojec Managemen , ol. 20, no. 3, pp. 185–190, 2002.
[109] Axelos, A Guide o AgileSHIFT. The S a ione y O ice, London, UK, 2018.
Re e ences 123
[110] A. Benne , Managing Success ul P ojec s wi h PRINCE2. The S a ione y O ice,
London, UK, 2017.
[111] A. ul Musawi , C. E. M. Se a, O. Zwikael, and I. Ali, “P ojec go e nance, bene i
managemen , and p ojec success: Towa ds a amewo k o suppo ing o ganiza-
ional s a egy implemen a ion,” In e na ional Jou nal o P ojec Managemen ,
ol. 35, no. 8, pp. 1658–1672, 2017.
[112] H. Ke zne , P ojec managemen : a sys ems app oach o planning, scheduling, and
con olling. John Wiley & Sons, 2017.
[113] P. L. Banne man, “De ining p ojec success: a mul ile el amewo k,” in De in-
ing he Fu u e o P ojec Managemen , (Wa saw, Poland), P ojec Managemen
Ins i u e, 2008.
[114] G. Thomas and W. Fe n´andez, “Success in i p ojec s: A ma e o de ini ion?,”
In e na ional Jou nal o P ojec Managemen , ol. 26, no. 7, pp. 733–742, 2008.
Special Issue: Achie ing IT P ojec Success.
[115] Y. Xue, J. R. Tu ne , L. Lecoeu e, and F. Anba i, “Using esul s-based moni o ing
and e alua ion o deli e esul s on key in as uc u e p ojec s in china,” Global
Business Pe spec i es, ol. 1, no. 2, pp. 85–105, 2013.
[116] P. M. Ins i u e, “uccess in dis up i e imes — pulse o he p o ession 2018,” ech.
ep., P ojec Managemen Ins i u e, 2018.
[117] O. Pank a z and D. Bas en, “Opening he black box: Manage s’ pe cep ions o is
p ojec success mechanisms,” In o ma ion & Managemen , ol. 55, no. 3, pp. 381–
395, 2018.
[118] W. Abba, “How ea ned alue go o p ime ime: A sho look back and a glance
ahead,” in P ojec managemen ins i u e semina s and symposium in Hous on,
TX, 2000.
[119] S. Rozenes, G. Vi ne , and S. Sp agge , “Mpcs: Mul idimensional p ojec con ol
sys em,” In e na ional Jou nal o P ojec Managemen , ol. 22, no. 2, pp. 109–118,
2004.
[120] M. Lau as, G. Ma ques, and D. Gou c, “Towa ds a mul i-dimensional p ojec pe -
o mance measu emen sys em,” Decision Suppo Sys ems, ol. 48, no. 2, pp. 342–
353, 2010.
[121] ¨
Onc¨u Hazı , “A e iew o analy ical models, app oaches and decision suppo ools
in p ojec moni o ing and con ol,” In e na ional Jou nal o P ojec Managemen ,
ol. 33, no. 4, pp. 808–815, 2015.
[122] P. J. Solomon and R. R. Young, Pe o mance-based ea ned alue. Ci esee , 2007.
[123] B. Pollack-Johnson and M. Libe a o e, “Inco po a ing quali y conside a ions in o
p ojec ime/cos adeo analysis and decision making,” IEEE T ansac ions on
Enginee ing Managemen , ol. 53, no. 4, pp. 534–542, 2006.
[124] M. J. Libe a o e and B. Pollack-Johnson, “Imp o ing p ojec managemen decision
making by modeling quali y, ime, and cos con inuously,” IEEE T ansac ions on

124 Re e ences
Enginee ing Managemen , ol. 60, no. 3, pp. 518–528, 2013.
[125] M. A. B agadin and K. K¨ahk¨onen, “Schedule heal h assessmen o cons uc ion
p ojec s,” Cons uc ion Managemen and Economics, ol. 34, no. 12, pp. 875–897,
2016.
[126] J. Pi k¨anen e al., “P ojec moni o ing in indus ial mechanical ins alla ions,”
Mas e ’s hesis, Aal o Uni e si y, 2017.
[127] E. Akg¨un, “Quali y in eg a ed ea ned alue managemen o cons uc ion
p ojec s,” Mas e ’s hesis, Middle Eas Technical Uni e si y, 2019.
[128] H. Jing, O. Hisa ciklila , and V. Thomson, “Ensu ing co ec p oduc cha ac e is-
ics du ing he design p ocess,” in P oceedings o he 2014 In e na ional Con e ence
on Inno a i e Design and Manu ac u ing (ICIDM), pp. 193–198, IEEE, 2014.
[129] G. Schuh, M. Riesene , C. Doelle, and S. B ockmann, “E alua ion o he e ec s
o ac i i y de ia ions on o a de elopmen p ojec ’s a ge dimensions,” in 2017
Po land In e na ional Con e ence on Managemen o Enginee ing and Technology
(PICMET), pp. 1–6, IEEE, 2017.
[130] T. Ca bone, “In eg a ing ope a ions and p oduc de elopmen me hodologies
o imp o ed p oduc success using ad anced p oduc quali y planning,” in
IEEE/SEMI Con e ence and Wo kshop on Ad anced Semiconduc o Manu ac u -
ing 2005., pp. 228–233, 2005.
[131] J. Kim, C. Kang, and I. Hwang, “A p ac ical app oach o p ojec scheduling:
conside ing he po en ial quali y loss cos in he ime–cos adeo p oblem,” In-
e na ional Jou nal o P ojec Managemen , ol. 30, no. 2, pp. 264–272, 2012.
[132] E. Commission and D.-G. o In o ma ics, PM², P ojec managemen me hodology
guide : open edi ion. Publica ions O ice o he Eu opean Union, 2016.
[133] P. M. Ins i u e e al.,A Guide o he P ojec Managemen Body o Knowledge:
PMBOK Guide. P ojec Managemen Ins i u e, Inc., 2021.
[134] R. G. Coope , “Thi d-gene a ion new p oduc p ocesses,” Jou nal o P oduc In-
no a ion Managemen , ol. 11, no. 1, pp. 3–14, 1994.
[135] R. G. Coope , “S age-ga e sys ems: a new ool o managing new p oduc s,” Busi-
ness ho izons, ol. 33, no. 3, pp. 44–54, 1990.
[136] H. Ke zne , S a egic planning o p ojec managemen using a p ojec managemen
ma u i y model. John Wiley & Sons, 2002.
[137] M. L. D u y-G ogan, “Pe o mance on agile eams: Rela ing i e a ion objec i es
and c i ical decisions o p ojec managemen success ac o s,” In o ma ion and
So wa e Technology, ol. 56, no. 5, pp. 506–515, 2014. Pe o mance in So wa e
De elopmen .
[138] R. Gu i´e ez, J. Canela, T. Femen´ıas, and F. A ´es, “Expe iences in agile &d
p ojec managemen o new p oduc design and de elopmen in he au omo i e
indus y,” J. T ends De . Mach. Assoc. Technol, ol. 16, pp. 83–86, 2012.
Re e ences 125
[139] R. G. Coope , “Wha ’s nex ?: A e s age-ga e,” Resea ch-Technology Manage-
men , ol. 57, no. 1, pp. 20–31, 2014.
[140] N. D. du P eez and L. Louw, “A amewo k o managing he inno a ion p o-
cess,” in PICMET ’08 - 2008 Po land In e na ional Con e ence on Managemen
o Enginee ing & Technology, pp. 546–558, 2008.
[141] A. F. Somme , C. Hedegaa d, I. Duko ska-Popo ska, and K. S ege -Jensen, “Im-
p o ed p oduc de elopmen pe o mance h ough agile/s age-ga e hyb ids: The
nex -gene a ion s age-ga e p ocess?,” Resea ch-Technology Managemen , ol. 58,
no. 1, pp. 34–45, 2015.
[142] E. Fijn, “Inno a ion p ojec s and hei p omising p ojec managemen app oach,”
Mas e ’s hesis, TU Del , 2016.
[143] H. We ne , S a egisches Fo schungs-und En wicklungs-Con olling. Sp inge -
Ve lag, 2013.
[144] J. E. Riedl, P ojek -Con olling in Fo schung und En wicklung: G unds¨a ze, Me h-
oden, Ve ah en, Anwendungsbeispiele aus de Nach ich en echnik. Sp inge -
Ve lag, 2013.
[145] L. Sas oque Pinilla, B. J. Nge eja, N. Mikh idino a, C. Wol , and N. Toledo Gan-
da ias, “Knowledge disco e y p ocess applied o building compe ence p o iles
desc ip ion,” in Do mund In e na ional Resea ch Con e ence 2022 P oceedings,
Fachhochschule Do mund; Do mund, 2022.
[146] D. H. Meadows, Thinking in sys ems: A p ime . chelsea g een publishing, 2008.
[147] S. Ma celino-S´adaba, A. P´e ez-Ezcu dia, A. M. Eche e ´ıa Lazcano, and P. Vil-
lanue a, “P ojec isk managemen me hodology o small i ms,” In e na ional
Jou nal o P ojec Managemen , ol. 32, no. 2, pp. 327–340, 2014.
[148] M. Fowle , J. Highsmi h, e al., “The agile mani es o,” So wa e de elopmen ,
ol. 9, no. 8, pp. 28–35, 2001.
[149] S. A el , “Mi iga e he isk o p ojec schedule o e uns and p ojec delay chains
in a l 5-7 esea ch cen e,” Mas e ’s hesis, Uni e si y o he Basque Coun y,
2020.
[150] H. Ke zne , Inno a ion p ojec managemen : Me hods, case s udies, and ools o
managing inno a ion p ojec s. John Wiley & Sons, 2022.
[151] L. A go e, “Inpu unce ain y and o ganiza ional coo dina ion in hospi al eme -
gency uni s,” Adminis a i e Science Qua e ly, ol. 27, no. 3, pp. 420–434, 1982.
[152] H. Sico e and A. Langley, “In eg a ion mechanisms and &d p ojec pe o mance,”
Jou nal o Enginee ing and Technology Managemen , ol. 17, no. 1, pp. 1–37, 2000.
[153] S. Wa d and C. Chapman, “S akeholde s and unce ain y managemen in
p ojec s,” Cons uc ion Managemen and Economics, ol. 26, no. 6, pp. 563–577,
2008.
[154] J. Cha a , P ojec managemen me hodologies: selec ing, implemen ing, and sup-
po ing me hodologies and p ocesses o p ojec s. John Wiley & Sons, 2003.
126 Re e ences
[155] C. Chin, E. Yap, A. Spowage, e al., “P ojec managemen me hodology o
uni e si y-indus y collabo a i e p ojec s,” Re iew o In e na ional Compa a i e
Managemen , ol. 12, no. 5, pp. 901–918, 2011.
[156] “Bole ´ın o icial del pa´ıs asco - bop n.º30, ie nes 11 de eb e o de 2022,” 2022.
[157] S. Kabadayi, A. P idgen, and C. Julien, “Vi ual senso s: abs ac ing da a om
physical senso s,” in 2006 In e na ional Symposium on a Wo ld o Wi eless, Mobile
and Mul imedia Ne wo ks(WoWMoM’06), pp. 6 pp.–592, 2006.
[158] T. L. Saa y, “How o make a decision: The analy ic hie a chy p ocess,” Eu opean
Jou nal o Ope a ional Resea ch, ol. 48, no. 1, pp. 9–26, 1990. Desicion making
by he analy ic hie a chy p ocess: Theo y and applica ions.
[159] A. Rod ´ıguez, A. Fe n´andez, L. N. L´opez de Lacalle, and L. Sas oque Pinilla,
“Flexible ab asi e ools o he debu ing and inishing o holes in supe alloys,”
Jou nal o Manu ac u ing and Ma e ials P ocessing, ol. 2, no. 4, 2018.
[160] G. G. Escude o, H. Gonz´alez, A. Calleja, L. S. Pinilla, and I. A. Remen e ia, “Tec-
nolog´ıas cla e pa a la nue a ´ab ica in eligen e,” Eu o ach elec onica: Ac ualidad
y ecnolog´ıa de la indus ia elec ´onica, no. 475, pp. 28–36, 2020.
[161] L. L´opez de Lacalle, A. Fe n´andez Valdi ielso, F. Amigo, and L. Sas oque, “Milling
wi h ce amic inse s o aus empe ed duc ile i on (adi): p ocess condi ions and
pe o mance,” The In e na ional Jou nal o Ad anced Manu ac u ing Technology,
ol. 110, no. 3, pp. 899–907, 2020.
[162] A. D. Olmo, G. M. de Piss´on, L. Sas oque, A. Fe n´andez, A. Calleja, and L. N.
L. D. Lacalle, “Me ging complex in o ma ion in high speed b oaching ope a ions
in o de o ob ain a obus machining p ocess,” IOP Con e ence Se ies: Ma e ials
Science and Enginee ing, ol. 1193, oc 2021.
[163] N. Mikh idino a, B. J. Nge eja, L. Sas oque Pinilla, C. Wol , and W. Van Pe egem,
“De eloping and imp o ing compe ence p o iles o p ojec eams in enginee ing
educa ion,” in SEFI Annual Con e ence, 2022.
Pa II
Conclusions
230 CHAPTER 10. COMMUNICATIONS AND OTHER PUBLICATIONS.
10.2 O he publica ions.
•O iginal Jou nal Pape .
1. Rod ´ıguez, A., Fe n´andez, A., L´opez de Lacalle, L. N., & Sas oque Pinilla, L.
(2018). Flexible ab asi e ools o he debu ing and inishing o holes
in supe alloys. Jou nal o Manu ac u ing and Ma e ials P ocessing, 2(4), 82
[159].
2. Escude o, G. G., Gonz´alez, H., Calleja, A., Pinilla, L. S., & Remen e ia, I. A.
(2020). Tecnolog´ıas cla e pa a la nue a ´ab ica in eligen e. Eu o ach elec-
onica: Ac ualidad y ecnolog´ıa de la indus ia elec ´onica, (475), 28-36 [160].
3. L´opez de Lacalle, L. N., Fe n´andez Valdi ielso, A., Amigo, F. J., & Sas oque, L.
(2020). Milling wi h ce amic inse s o aus empe ed duc ile i on (ADI):
p ocess condi ions and pe o mance. The In e na ional Jou nal o Ad anced
Manu ac u ing Technology, 110(3), 899-907 [161].
4. Del Olmo, A., de Lacalle, L. L., de Piss´on, G. M., P´e ez-Salinas, C., Ealo, J. A.,
Sas oque, L., & Fe nandes, M. H. (2022). Tool wea moni o ing o high-speed
b oaching p ocess wi h ca bide ools o educe p oduc ion e o s. Me-
chanical Sys ems and Signal P ocessing, 172, 109003 [13].
•Con e ence Pape .
1. Del Olmo, A., de Piss´on, G. M., Sas oque, L., Fe n´andez, A., Calleja, A., & De
Lacalle, L. L. (2021, Oc obe ). Me ging complex in o ma ion in high-speed
b oaching ope a ions in o de o ob ain a obus machining p ocess. In
IOP Con e ence Se ies: Ma e ials Science and Enginee ing (Vol. 1193, No. 1, p.
012079). IOP Publishing [162].
2. Sas oque-Pinilla, L., Mikh idino a, N., Nge eja, B. J., Wol , C., & Toledo Gan-
da ias, N. (2022). Knowledge Disco e y P ocess Applied o Building
Compe ence P o iles Desc ip ion. In Do mund In e na ional Resea ch Con-
e ence 2022 P oceedings. Fachhochschule Do mund; Do mund [145].
3. Mikh idino a, N., Nge eja, B. J., Sas oque Pinilla, L., Wol , C., & Van Pe egem,
W. (2022). De eloping and imp o ing compe ence p o iles o p ojec
eams in enginee ing educa ion. In SEFI Annual Con e ence [163].

11
Da a Appendix.
11.1 CFAA’s KPIs.
232 CHAPTER 11. DATA APPENDIX.
1. Scope - 1.1. S a egical Need -
1.1.1. Ope a ional Need / Ca ego y Financial Li e cycle Ope a ional P ojec S a egic Sus ainabili y Technical To al
1. Inno a ion — — 7 — 5 12 10 34
1.1. C ea ing and ma ke ing new se -
ices, p oduc lines, and echnological
capabili ies.
— — 7 — 2 12 3 24
1.1.1. Con ibu e o sus ainabili y in
he c ea ion, design and esul o de-
li e ables om he p ojec s
— — — — — 12 — 12
1.1.2. De elop and implemen ma -
ke ing o he R&D se ice po olio — — — — — — 1 1
1.1.3. Gene a e new usable knowledge
and enginee ing solu ions — — 7 — — — — 7
1.1.4. Inc ease demand o ien a ion in
ans e — — — — 2 — — 2
1.1.5. In oduce con inuous inno a-
ion managemen — — — — — — 2 2
1.2. Ins i u ional suppo o SMEs
wi h inno a ion impulses in de elop-
ing new business models.
— — — — 3 — 7 10
1.2.1. Enable and encou age alen
ea ly on — — — — — — 3 3
1.2.2. Expand exis ing coope a ion
in o s a egic inno a ion pa ne ships — — — — 3 — — 3
1.2.3. P omo e he ounda ion, es ab-
lishmen and accompanimen o spin-
o s
— — — — — — 4 4
2. Region 6 — 5 5 18 — 4 38
11.1. CFAA’S KPIS. 233
2.1. CFAA is a place o in e es o lo-
cal and in e na ional pa ne s o de-
elop p ojec s.
6 — 4 5 3 — 1 19
2.1.1 Assu e by any possible means
ha he p ojec ’s esul s a e help ul
o he p ojec pa ne
— — 2 — — — — 2
2.1.2 Assu e luid communica ion
wi h he pa ne be o e, du ing and
a e he de elopmen o he p ojec
— — — — 2 — 1 3
2.1.4 Find pa ne s eage o sha e and
collabo a e wi h knowledge, comme -
cial and echnical in e es s.
— — — — 1 — — 1
2.1.5 Inc ease he epu a ion o PMO — — — 5 — — — 5
2.1.6 Inc ease he e u n o in es -
men o he p ojec s 6 — — — — — — 6
2.1.7 Inc easing awa eness o CFAA
as a cen e o excellence — — 1 — — — — 1
2.1.8 P ojec de elopmen acco ding
o he pa ne s’ expec a ions — — 1 — — — — 1
2.2. Inc ease he a ac i eness o he
Basque Coun y o esea ch, inno a-
ion, employmen and s a -ups
— — — — 9 — — 9
2.2.1 Con ibu e o he c ea ion o
weal h in he Basque Coun y, sup-
po ing and p omo ing sus ainable
compe i i eness o Basque Coun y
Companies
— — — — 4 — — 4
2.2.2 In ol e local companies o
s a egical sec o s in R&D p ojec s — — — — 1 — — 1
234 CHAPTER 11. DATA APPENDIX.
2.2.3 P omo e he in e na ionalisa-
ion and echnological inno a ion o
pa ne s companies
— — — — 4 — — 4
2.3. In ensi y knowledge ans e — — 1 — 6 — — 7
2.3.1 Con ibu e o he ad ancemen
o knowledge and social de elopmen
h ough R&D and inno a ion
— — — — 4 — — 4
2.3.2 Inc ease g adua e e en ion in
SMEs in he egion — — — — 1 — — 1
2.3.3 In ol e pe sonnel om pa -
ne companies as ac i e ac o s in he
implemen a ion and de elopmen o
p ojec s
— — 1 — — — — 1
2.3.4 P omo e and coo dina e he
echnology ans e o R&D and In-
no a ion ac i i ies
— — — — 1 — — 1
2.4. In e ac wi h socie y, s eng hen
ac o s and ci il socie y — — — — — — 3 3
2.4.1 P esence a popula science
e en s — — — — — — 1 1
2.4.2 P esence in egional and na-
ional p ess — — — — — — 1 1
2.4.3 P esence on ele an po als and
social media — — — — — — 1 1
3. Resea ch — — — — 6 — 4 10
11.1. CFAA’S KPIS. 235
3.1. Ga he and gene a e expe ience,
new knowledge and unde s anding
om ac i i ies and managemen o
he p ojec (e.g. ools, echniques, ap-
p oaches o p ocesses)
— — — — 4 — 2 6
3.1.1 De elop, es , es ablish and
ans e heo y and me hodology. — — — — 4 — — 4
3.1.2 Issuing cen al e e ence publi-
ca ions — — — — — — 2 2
3.2. Inc ease he impac o R&D in
he scien i ic communi y — — — — 2 — 2 4
3.2.1 Inc ease publica ion ou pu and
con e ence a endances — — — — — — 1 1
3.2.2 Ini ia ion o join p ojec s wi h
enowned esea ch pa ne s — — — — 2 — — 2
3.2.3 Scien i ic publica ions om
p ojec esul s in e e eed jou nals — — — — — — 1 1
4. Uni e si y 1 5 2 6 12 — 16 42
4.1. P o essionalizing p ojec man-
agemen — — — 6 3 — 1 10
4.1.1 Implemen oubleshoo ing
me hods and p ocess — — — — — — 1 1
4.1.2 Pe o mance moni o ing and
e alua ion o p ojec s de elopmen
and esul s
— — — 5 — — — 5
4.1.3 Risk e alua ion o ealised op-
po uni ies (known unknowns) and
su e ed h ea s
— — — 1 — — — 1

236 CHAPTER 11. DATA APPENDIX.
4.1.4 Se up s a egic and ope a ional
mul i-p ojec managemen — — — — 3 — — 3
4.2. P omo e compliance wi h he
goals and objec i es o which CFAA
was c ea ed.
1 — — — 1 — 10 12
4.2.1 Ensu e he inancial, scien-
i ic and in e na ional success o he
CFAA and he con inui y o he ac-
i i ies
1 — — — — — 7 8
4.2.2 Mee he echnical and manage-
men objec i es o he Cen e — — — — — — 2 2
4.2.3 S a egic po en ial o he
p ojec s de eloped — — — — 1 — — 1
4.2.4 Top managemen suppo o as-
sis he complimen o he mission,
policies and s a egies o CFAA
— — — — — — 1 1
4.3. Secu ing he eedom, inancial
and pe sonnel basis o esea ch and
ans e ac i i ies in he long e m
— 5 1 — 3 — 5 14
4.3.1 C ea ing scope o esea ch — — — — 2 — 4 6
4.3.2 De ec , analyse and esol e non
con o mi ies a ime — 2 — — — — — 2
4.3.3 Keep he echnical capaci ies
needed o he p ojec ’s de elopmen . — 3 — — — — — 3
4.3.4 Mobilisa ion o esou ces o
p ojec s as needed — — 1 — — — — 1
4.3.5 Pa icipa ion in p og ammes
ini ia i es o he EU Resea ch F ame-
wo k P og amme
— — — — 1 — — 1
11.1. CFAA’S KPIS. 237
4.3.6 Secu ing unding o manage-
men capaci y — — — — — — 1 1
4.4. S eng hen collabo a ion wi h
he Uni e si y in he implemen a ion
and de elopmen o p ojec s
— — 1 — 5 — — 6
4.4.1 De i e eaching con en om
RDI p ojec s’ esul s. — — — — 1 — — 1
4.4.2 Encou age he pa icipa ion
o s uden s in he de elopmen o
p ojec s
— — — — 1 — — 1
4.4.3 O ganize, coo dina e and di ec
he ac i i ies in he Cen e — — 1 — — — — 1
4.4.4 P epa e s uden s, company and
uni e si y s a h ough specialized
aining
— — — — 1 — — 1
4.4.5 S eng hening esea ch-based
and p ac ice-o ien ed eaching — — — — 2 — — 2
To al gene al 7 5 14 11 41 12 34 124
Table 11.1: CFAA’s KPIs
238 CHAPTER 11. DATA APPENDIX.
11.2 P oposed da abases and classi ica ion scheme.
Da abase Minimum in o ma ion
o be included No es
C i ical Suc-
cess Fac o s
Fu u e po en ial
Pa ne sa is ac ion
P ojec goals and mission
(knowledge gene a ion)
P ojec managemen
success ac o s
KPIs
His o ically used KPIs
KPIs pe pa ne
KPIs pe echnology
KPIs pe ype o p ojec
Lessons Lea ned
Lessons lea n epo o
he p ojec
Lessons lea ned pe
pa ne
Lessons lea ned pe
echnology
Lessons lea ned pe ype
o p ojec
Pa ne s’
e alua ion
Pa ne s’ e alua ion o
li e cycle o he p ojec
Pa ne s’ e alua ion o
he deli e able
Pa ne s’ e alua ion pe
echnology
Pa ne s’ e alua ion pe
ype o p ojec
Pa ne s’ eedback abou
he u u e
implemen a ion o he
p ojec ’s deli e able.
P ojec
E alua ion
Miles ones complimen
P ojec success e alua ion
Quali a i e e alua ion
Quan i a i e e alua ion
P ojec Team
E alua ion
Quali a i e e alua ion
Quan i a i e e alua ion
based on da a
accumula ed h oughou
he p ojec .
11.2. PROPOSED DATABASES AND CLASSIFICATION SCHEME. 239
P ojec s
in o ma ion
Ac i a ion da e
Deli e able lis
Due Da e
Financing en i y
Pa ne s in ol ed
Pa ne s pa icipa ion
Planned cos
Planned hou s
Planned esou ces
P io i iza ion
P ojec ’ echnology
Scope
To al cos
ype o p ojec
We p oposed o classi y he p ojec s in he ollowing
ca ego ies:
i) C ea ion o de elopmen o new p oduc echnol-
ogy pla o m p ojec s;
ii) Inno a ion p ojec s,
iii) P oduc o echnology enhancemen p ojec s,
i ) P ojec o ien ed o new p oduc o p ocess.
Quali y
E alua ion
KPIs e alua ion
Quali y assessmen o he
asks
We p oposed o o ganise he in o ma ion abou he
quali y assessmen on he ollowing ca ego ies: i)
Type o he p ojec , ii) Classi ica ion o he Ac i -
i y, iii) Used esou ces
Quali y Con ol Ac i i ies
Quali y e alua ion pe
pa ne
Quali y e alua ion pe
echnology
Quali y e alua ion pe
ype o p ojec
Risk Assessmen
Ongoing p ojec isks
iden i ica ion
P e iously iden i ied isks
(e alua ion in e ms o
p obabili y and impac )
Risk mi iga ion ac i i ies