Tes ing Resea ch So wa e
Nasi Eis y, Uni e si y o Tennessee, Knox ille, TN
Je ey Ca e , Uni e si y o Alabama, Tuscaloosa, AL
Upulee Kanewala, Uni e si y o No h Flo ida, Jackson ille, FL
Con ex
Re e ence
Conclusion
Recommenda ions
Objec i es
Key Resul s
❖Challenges:
➢Tes case design
➢E alua ing es quali y
➢Co ec ness o ou pu s
❖App oaches:
➢62% manual inpu design
➢Ou pu s o en lack o acle
➢Non-de e minism
complica es alida ion
❖Me ics:
➢S a emen co e age mos
used
➢Many don’ ack quali y
me ics
❖Execu ion:
➢Mos ly on push o end o
cycle
➢Few use p io i iza ion
❖Tools:
➢Gi Hub Ac ions, Py es ,
Jenkins, Codeco
➢No s anda d se
❖Explo e es ing p ac ices o
esea ch so wa e
❖Unde s and challenges: es
design, ou pu s, me ics,
execu ion, ools
❖In es iga e demog aphic
in luences (domain, eam
size, oles)
❖Resea ch so wa e is c i ical
o science and inno a ion
❖Ensu ing co ec ness is
essen ial bu challenging
❖Indus ial es ing ools a e
o en inadequa e
❖Resea ch so wa e es ing is
less sys ema ic and au oma ed
han indus y.
❖O acle p oblem and es case
design a e bigges hu dles.
❖Mo e esou ces, specialized
ools, and educa ion needed.
❖Funde s/manage s: alloca e
mo e es ing esou ces
❖De elope s: adop
documen a ion + sys ema ic
es ing
❖Tool builde s: c ea e
domain-awa e and easy o use
ools
❖Educa o s: in eg a e es ing
in o esea ch cu icula
Me hod
❖Su ey o 131 esea ch
so wa e de elope s
❖Topics: es inpu design,
expec ed ou pu s, me ics,
ools, limi a ions
❖Rec ui men ia US-RSE,
UK-RSE, IDEAS-ECP, BSSW
mailing lis s
Eis y, N.U., Kanewala, U. & Ca e , J.C. Tes ing
esea ch so wa e: an in-dep h su ey o p ac ices,
me hods, and ools. Empi So wa e Eng 30, 81
(2025). h ps://doi.o g/10.1007/s10664-025-10620-6
Demog aphics
Cha ac e is ics o he Tes ing P ocess
RSE? Deg ee ype? Domain
Sou ce o es ing
knowledge #FTEs % ime on es ing
Ranking o Tes ing Challenges
In luence o Demog aphics (Sys ema ic s FTEs)
Scan o ead he pape