1
Me ics in Agile and Rapid So wa e
De elopmen
López, L., Bu gués, X., Ma ínez-Fe nández, S., Vollme , A.M., Behu iye, W., Ka hapää, P., F anch, X.,
Rod íguez, P., & Oi o, M. Quali y Measu emen in Agile and Rapid So wa e De elopmen : A Sys ema ic
Mapping [Da ase ], .1. Uni e si a Poli ècnica de Ca alunya.
Quali y Requi emen s Me ics (ISO/IEC 25010)
QR (#me ics)
Me ics as epo ed in p ima y s udies
Reliabili y
(41)
De ec modi ica ions; de ec densi y; numbe o de ec s ca ied o e he nex i e a ion; lis o open
de ec s; mean ime o ailu e; mean ime be o e ailu e; passed es s; as es s; es co e age ( es ing
s a us); bug densi y; non-bug densi y; e o s a un ime; a ailabili y up ime; pos poned issues a io;
c i ical issues a io; build s abili y; uni es co e age o he de eloped code; open de ec se e i y
index; allowed e o coun pe ime; Se iceIns ances. ailO e Time; E o s.se e e; componen
ailu e; e o de ec ion; s a e co e age; ansi ion co e age; pa h co e age; numbe o de ec s; ype
o de ec s; numbe o bugs pe ime uni ; de ec densi y ( eleased de ec s/KLOEC o code ou
mon hs a e elease) ound by he cus ome a e elease; de ec densi y es de ec s/KLOEC o code;
numbe o es cases; es co e age; es densi y pe class; numbe o bugs; aul con e gence a e; C1
co e age; numbe o es ed classes; mean alue unc ion o de ec ed aul s; aul con en unc ion;
ailu e de ec ion a e unc ion. [No e: KLOEC = Thousands o lines o execu able code].
Main ainabili
y (27)
Complexi y; commen s; duplica ion; non-blocking iles; well de ined issues; o al LOC; change a e
o he sou ce code; main ainabili y; highly changed iles; weigh ed me hods o a class; coupling
be ween objec s; esponse o a class; lack o cohesion in me hods; dep h o inhe i ance ee; numbe
o child en; echnical deb ; access o o eign da a; class-le el cycloma ic complexi y; igh class
cohesion; e e en coupling; McCabe’s cycloma ic complexi y o a unc ion; e ec i e complexi y o
a ile; e ec i e cycloma ic complexi y pe cen age o a ile; code inco ec ness de ec abili y; lines o
code; uni es s; File.size.
Pe o mance
e iciency (22)
Sys em pe o mance; maximum ime pe mi ed o deploymen o new ins ances; execu ion;
Se iceIns ances.coun ; ope a ional pe o mance; a e age CPU usage; ins an aneous CPU usage;
memo y usage; la ency; pe o mance esponse ime; numbe o ansac ions pe second; esponse
ime; ansac ion p ocessing esponse ime; page ende ing esponse ime; que y p ocessing esponse
ime; numbe o pages ende ed pe second; numbe o messages pe second; scene g aph size;
milliseconds/ ame; maximum polygon & objec coun s a X ps ( ame pe second); ex e nal
g aphics ca d usage i he ha dwa e al eady ha e one and i 's ge ing used; h oughpu .
Secu i y (9)
Coun o baseline ulne abili ies; coun o eg essi e ulne abili ies; coun o new ulne abili ies;
en o cemen a io; amewo k misuse; numbe o open secu i y de ec s; p obabili y o success ul
a ack; impac o a ack; numbe o ulne abili ies.
Usabili y (5)
Sc een swi ch ime; isual/ins umen a ion ailu e; numbe o usabili y issues me by use s;
impo ance o usabili y issues me by use s; Use sa is ac ion.
Func ional
sui abili y (2)
End use eedback; ea u e usage (so wa e usage).
No classi ied
(1)
Lines o code [because he p ima y s udy epo ed i as a eliabili y me ic bu we would map i o
main ainabili y].
2
Quali y Managemen Indica o s
QMI (# epo ed
QMI)
QMIs as epo ed in p ima y s udies
P oduc Quali y (35)
A chi ec u al in eg i y, A chi ec u e s abili y, Blocking, Blocking code, Code Quali y,
Con inuous Code Quali y, De ec a e, De ec s deli e ed o he cus ome , De elopmen p og ess,
De elopmen Quali y, Ex e nal Quali y, Field Quali y, In e nal Quali y, Pos - elease quali y,
P e- elease quali y, P oduc quali y (based on secu i y), P oduc Quali y, P oduc Readiness,
P oduc Reliabili y, Pull Reques Release Readiness, Quali y, Quali y eedback loop, Realized
quali y equi emen s, Release Quali y, Release Readiness, So wa e design s abili y, So wa e
quali y, So wa e Reliabili y, So wa e s abili y (bugs), So wa e usage, Sys em complexi y,
Team's main enance quali y, Technical Deb , Tes ing S a us, Value
P oduc i i y (23)
Coding ac i i y, De ec ixing e iciency, De ec ixing e o , Deli e y speed, De elopmen
p oduc i i y, De elopmen p og ess, De elopmen speed, De elopmen s a us, Es ima ed e o ,
Fea u e h oughpu , Fixing speed, issues' eloci y, P oduc i i y, P oduc i i y a e, P og amme
P oduc i i y, P ojec cos and ime, P ojec p og ess, Release equency, Resou ce P oduc i i y,
Sp in p og ess, Team P oduc i i y, Tes ing ac i i ies e iciency, T end de elopmen
Risk (7)
Co ec ed isks, Pos poned isks, Rela i e isk, Risk, Risk (secu i y isk), Risk alue, Unhandled
isks
Cos (6)
Cos , De elopmen Cos , P ojec Cos Change, Rela i e Cos De ia ion, Re u n on In es men
(ROI), Technical Deb Cos
Schedule (6)
On-Time Deli e y, O e ime, Rela i e Schedule De ia ion, Release Delay, Time o elease,
Time- o-ma ke
P ocess
Pe o mance (3)
Implemen a ion phase pe o mance, P ocess pe o mance, Sys em es phase pe o mance
De elope
Sa is ac ion (2)
De elope sa is ac ion, Team mo ale
Agili y (2)
NERV Agili y Index, Ac i i y Agili y deg ee
P ojec Success (1)
P ojec Success
Cus ome
Sa is ac ion (1)
Cus ome sa is ac ion
Quali y Managemen Indica o s Me ics
QR (#me ics)
Me ics as epo ed in p ima y s udies
P oduc Quali y
(122)
#de ec s = numbe o open de ec s a he momen ; #de ec s injec ed pe sp in ; #de ec s/#bugs;
(de ec _ emo al_ a e) = a e age numbe o de ec s emo ed du ing las 4 weeks; (E o spen o
bug ixing / E o ) x 100; ( es _execu ion_ a e) = a e age numbe o es cases execu ed du ing he
las 4 weeks; ( es _pass_ a e) = a e age numbe o es cases passed du ing he las 4 weeks; Amoun
o p og amming e o ; a ailabili y up ime; Bad code smells; Blocking iles; B anch
Co e age/SLOC; b ead h o unc ionali y (mos o he unc ionali y, low- unc ionali y); Bug
densi y; Build s abili y; Code Chu n/ReleaseTime ( elease da e and s a da e o he de elopmen
phase); code is well s uc u ed and commen ed; Code Quali y Checking Ra e (Numbe o builds
subjec o a code quali y check di ided by he o al numbe o builds); Code e iews; Coding
Viola ions Densi y/SLOC; Commen Densi y/SLOC; commen ed code; Commen s wo ds;
Complexi y; C i ical Issues Ra io; C i ical iola ions; Cycloma ic Complexi y; Cycloma ic
Complexi y/SLOC; De ec densi y (De ec s / KLOC) ; de ec densi y (numbe o de ec s pe lines
o code); "de ec modi ica ions (numbe o iden i ied modi ica ions o each de ec pe unc ion);
de ec s ca ied o e nex i e a ion; De elopmen ask comple ion; duplica e code; E ec i eness
3
0.2*Abs ac ion + 0.2*Encapsula ion + 0.2*Composi ion + 0.2*Inhe i ance + 0.2*Polymo phism);
Elapsed F ame be ween Checks(A e age numbe o builds be ween wo builds subjec o a code
quali y check); Elapsed Time be ween Checks (A e age numbe o days be ween wo builds subjec
o a code quali y check); en i onmen a ailabili y; E o s a un ime; Ex endibili y
(0.5*Abs ac ion–0.5*Coupling+0.5*Inhe i ance+0.5* Polymo phism); Ex e nally-Visible Quali y
( eleased de ec s/KLOEC o code ou mon hs a e elease) ound by he cus ome a e elease;
Fas es builds; Fea u e usage; Flexibili y (0.25*Encapsula ion-0.25*Coupling+0.5*Composi ion
+0.5*Polymo phism); F equency o p og am speci ica ion change; unc ional co e age (no me ic);
Func ionali y (0.12*Cohesion+0.22*Polymo phism+0.22*Messaging + 0.22*Design
Size+0.22*Hie a chies; Hea map e isions pe model pe week (numbe o new checked iles);
Highly changed iles: Fo each commi : iles changed, lines o code added/modi ied/dele ed, au ho ,
and e ision ; In e nally-Visible Quali y as in e nal (p e- elease) de ec densi y es de ec s / KLOEC
o code KLOEC = Thousands o lines o execu able code; Le el o p og amming echnologies;
Li e ime (open o close pull eques ); LOC; Median Daily C ash Coun ; Median Daily C ash Coun :
he median o he numbe o c ashes pe day o a pa icula e sion (lowe is be e ); Median
Up ime; Median Up ime: he median ac oss he up ime alues o all he c ashes ha a e epo ed o
a e sion (highe is be e ).; Me ge ime (open o me ge pull eques ); Mos iola ed ules; Non-bug
densi y; Numbe o bugs epo s; Numbe o code changes; Numbe o commi commen s; Numbe
o commi s; Numbe o de ec s de ec ed (o e all); Numbe o de ec s de ec ed (pe pull eques s);
Numbe o de ec s pe i e a ion; numbe o ailed es cases; Numbe o iles added; Numbe o iles
dele ed; Numbe o iles modi ied; Numbe o unc ion pa ame e s; Numbe o issue commen s;
numbe o open de ec s; Numbe o pa icipan s; numbe o passed accep ance es s; numbe o
planned/no planned/cancelled es cases; Numbe o sou ce chu n; Numbe o sou ce iles; Numbe
o es chu n; Numbe o ouble epo s; Open De ec Se e i y Index; Pa e o dis ibu ion o de ec s
pe o igin; Pass/ all es cases; Passed es s (uni es densi y, passed/ o al); Pe cen age o Checked
B anches (Numbe o b anches con aining a leas one build subjec o a code quali y check di ided
by he numbe o o al b anches scheduled o build); Pe cen age o eused modules; Pos poned
issues a io; Pos -Release Bugs; Pos -Release Bugs: he numbe o bugs epo ed a e he elease
da e o a e sion (lowe is be e ); P og am complexi y; P og am wo kload (s ess); Rela ionship o
de ailed design o equi emen ; eliabili y; Requi emen analysis; Requi emen s elici a ion and
b eakdown; esponse imes; Reusabili y (-0.25*Coupling+0.25*Cohesion+0.5*Messaging
+0.5*Design Size); RR= #de ec s / ( (de ec _ emo al _ a e) − (( es _ execu ion _ a e) - ( es _
pass a e)) ) ; ules compliance (showing he size); SDI (Sys em Design Ins abili y): pe cen age o
classes whose names change om one i e a ion o he nex + pe cen age o class which we e added
+ pe cen age o classes which we e dele ed; SDIe (Sys em Design Ins abili y wi h en opy): using
he numbe classes added, dele ed, changed and unchanged om he p e ious i e a ion; Se e i y o
de ec s de ec ed; so wa e ix quali y (no me ic); Sou ce code commen s; Speci ica ion ask
comple ion; s a us o hei code, ela ed o se e al aspec s. E.g. ha he code is well s uc u ed, ha
i is easy o ead, and ha i has low complexi y; sys em esou ce u iliza ion (such as memo y, heap,
CPU, disk space, ne wo k); echnical pe o mance (e.g. speed); Tes co e age; Tes co e age
pe cen age (o e all); Tes co e age pe cen age (pe pull eques ); Tes de ec densi y pe line o code
(de ec #/LoC); Tes de ec densi y pe es scena io (de ec #/TS#); Tes success (Uni es success
densi y); Tes ing co e age; Tes ing e o ; h oughpu a e; TQI (To al quali y index) =
eusabili y+Flexibili y + Unde s andabili y + Func ionali y + Ex endibili y + E ec i eness; Types
o de ec s de ec ed; Unde s andabili y (-0.33*Abs ac ion+0.33*Encapsula ion-
0.33*Coupling+0.33*Cohesion-0.33*Polymo phism–0.33*Complexi y–0.33* Design Size); Uni
Tes Co e age;
P oduc i i y
(36)
lines o new code de eloped pe pe son-day and adjus ed o di e ing le els o p oduc complexi y;
%e o dis ibu ion pe p ojec ; (E o spen o bug ixing / E o ) x 100; Bug co ec ion ime;
Bu n-up/down (es ima ed/done s o y poin s); Check in pace; Check-in end; Code Chu n (# iles
modi ied in a commi ); Deli e y on ime; De elope Con ibu ion (#commi s); de elope s’ e o
spen on ixing de ec s.The numbe o de ec s di ided by he de elope s’ e o (measu ed by pe son
hou s); De elopmen Time; Fix Time: he du a ion o he ixing pe iod o a FIXED pos - elease bug,
i.e., he di e ence be ween he bug open ime and he las modi ica ion ime (lowe alue is be e );
4
Fix Time: he du a ion o he ixing pe iod o he bug (i.e., he di e ence be ween he bug open ime
and he las modi ica ion ime). This me ic is compu ed only o bugs wi h he s a us FIXED (lowe
is be e ).; Fixed Bugs: he numbe o pos - elease bugs ha a e closed wi h he s a us ield se o
FIXED (highe is be e ); Fixed Bugs: he numbe o p e- elease and pos - elease bugs ha a e closed
wi h he s a us ield se o FIXED (highe numbe s a e be e ).; in eg a ion speed (a g. ime o deli e
ea u es); Issues comple ely speci ied; Mean Complexi y; Mon hly Ac i e De elope s; Numbe o
check-ins cu en week; Numbe o check-ins las week; Numbe o open de ec s; passed/execu ed
es s; P ojec M/M (Man * Mon h); pull eques s me ged pe mon h (agains es co e age and usage
o CI o hei hypo hesis); Ra e o New Code (new lines o code); Resol ed issues assigned o a
da e; Schedule Pe o mance Index (SPI): ea ned alue di ided by he planned cos measu ed in
pe son hou s; Size/ E o ; S aleness: he numbe o days he e sion i is s ill in use a e a newe
e sion i+1 has been eleased; S o ies / Pe son Mon h S o ies in KLOEC (Thousands o lines o
execu able code); Tes p og ess; To al Lines o Code; Uncon i med Bugs: he numbe o pos -
elease bugs wi h he s a us ield se o UNCONFIRMED; Veloci y /E o ;
P ocess
Pe o mance
(30)
A e age au oma ion du a ion; Bug Leakage; Bugs a io; Build pe o mance; CI eedback ime;
Commi esponse ime; Commi e iew du a ion; Commi e iew i e a ions; Co e componen
commi s; De ec Remo al E o Ra io (pe son-hou /pe son-hou ); De elope - es e
communica ion; E o Es ima ion Capabili y (pe son-hou /pe son-hou ); End use eedback; E o
co ec ion; E o iden i ica ion; Fas es s; issue- ype in a ime ame; Long es s; Non-issue
componen commi s; Old issues; P oduc i i y (LoC/pe son-hou ); Resol ed issues h oughpu ;
Team h oughpu ; Tes E ec i eness (de ec #/pe son-hou ); Tes Execu ion V&V E ec i eness
(de ec #/de ec #); Tes Speed (TS#/pe son-hou ); Tes s pe p oduc ; Timely ea u e deli e y; Timely
ea u e speci ica ion deli e y; Uni es du a ion
Schedule (13)
% O e ime = (Ac ual Hou s/Expec ed Hou s) -1; ((Real ime - Planned ime) / Planned ime) x 100;
Abili y o esol e emaining alloca ed e o ; Abili y o esol e emaining unalloca ed e o ;
Accu acy o planning issues due da e: pe cen age o he di e ence be ween he planned due da e o
pas closed issues and he ac ual acked issue closing da e; ac ual - planned elease da e; Co e
componen commi s; Non-issue componen commi s; Pe cen age o issues la ge han he size
h eshold; Planning accu acy: a pe cen age o he di e ence be ween he planned e o o pas
closed issues and he ac ual acked e o ; Ra io o he a e age pas eloci y and he heo e ical
eloci y o he a ailable uni s; Timely ea u e speci ica ion deli e y; Timely elease deli e y
Agili y (19)
Agile P ocess Index; Agile P ojec Risk Fac o ; Change Tole a e; Cus ome In e ac ion; Flexibili y;
F ee o Modeling and Documen a ion; In o mali y; I e a i e; People O ien a ion; Requi emen s
Ambigui y Fac o ; Requi emen s Risk Fac o ; Requi emen s Vola ili y Fac o ; Simplici y; Speed o
Execu ion; Team Communica ions Index; Team Ma u i y Index; Team Technical Compe ency
Index; Team Veloci y Index; Team-Cus ome Collabo a ion Index
Cos (6)
Money; Time; E o ; (1-Cos _pp/Cos _bp) x 100; ((Real cos s - Planned cos s) / Planned cos s) x
100; Value (in pe cen age e ms) deli e ed pe pound spen on de elopmen
Risk (5)
Rela i e_Risk=E ec i e_M%*NR; E ec i e MacCabe's cycloma ic Complexi y pe cen age
(E ec i e_M%); Numbe o e isions o a ile (NR); Risk p io i y numbe (RPN)=se e i y anking,
occu ence anking, de ec ion anking; p obabili y o success ul a ack, impac o a ack
P ojec Success
(4)
De ec a e (numbe o e o s/complexi y); on Budge ; on Time; So wa e size FP = FPUI + FPOU
+ FPI whe e FP = Func ion Poin , FPUI = FP based on he numbe o inpu / ou pu documen s,
FPOU = FP based on he numbe o o ganiza ional uni s, FPI = FP based on he numbe o in e aces
wi h o he modules The esul is ounded o he i s g ea e in ege alue. FPUI = NoUI * WFUI,
FPOU= NoOU * WFOU, FPI = NoI * WFI, W = weigh
Cus ome
Sa is ac ion (3)
Cus ome sa is ac ion a ed be ween 0-100%, Quali a i e da a, ia in e iew and cus ome eedback
De elope
Sa is ac ion (2)
De elope sa is ac ion a ed be ween 0-100%; Su ey ques ion on he Shodan Adhe ence Su ey.
The ques ion ead, “How o en can you say you a e enjoying you wo k?”