This is a sel -a chi ed e sion o an o iginal a icle. This e sion
may di e om he o iginal in pagina ion and ypog aphic de ails.
Au ho (s):
Ti le:
Yea :
Ve sion:
Copy igh :
Righ s:
Righ s u l:
Please ci e he o iginal e sion:
CC BY 4.0
h ps://c ea i ecommons.o g/licenses/by/4.0/
Con inuous design con ol o machine lea ning in ce i ied medical sys ems
© The Au ho (s) 2022
Published e sion
S i bu, Vlad; G anlund, Tuomas; Mikkonen, Tommi
S i bu, V., G anlund, T., & Mikkonen, T. (2023). Con inuous design con ol o machine lea ning
in ce i ied medical sys ems. So wa e Quali y Jou nal, 31, 307-333.
h ps://doi.o g/10.1007/s11219-022-09601-5
2023
Vol.:(0123456789)
So wa e Quali y Jou nal
h ps://doi.o g/10.1007/s11219-022-09601-5
1 3
Con inuous design con ol o machine lea ning ince i ied
medical sys ems
VladS i bu1,5 · TuomasG anlund2,3· TommiMikkonen4,5
Accep ed: 1 Sep embe 2022
© The Au ho (s) 2022
Abs ac
Con inuous so wa e enginee ing has become commonplace in nume ous ields. Howe e ,
in egula ing in ensi e sec o s, whe e addi ional conce ns need o be aken in o accoun , i
is o en conside ed di icul o apply con inuous de elopmen app oaches, such as de ops.
In his pape , we p esen an app oach o using pull eques s as design con ols, and apply
his app oach o machine lea ning in ce i ied medical sys ems le e aging model ca ds, a
no el echnique de eloped o add explainabili y o machine lea ning sys ems, as a egula-
o y audi ail. The app oach is demons a ed wi h an indus ial sys em ha we ha e used
p e iously o show how medical sys ems can be de eloped in a con inuous ashion.
Keywo ds Machine lea ning· ML· MLOps· CD4ML· Design con ol· Medical
so wa e· Regula ed so wa e· Con inuous enginee ing
1 In oduc ion
Du ing he la es decade, he Web has silen ly become he dominan pla o m o so wa e
applica ions. E ec i ely, his p ocess has made eleasing so wa e so simple and cheap
ha o a deg ee, de elopmen and deploymen ac i i ies a e en angled. New pa s o so -
wa e a e expe imen ally deployed, and eedback om eleased so wa e is used o assis in
Tuomas G anlund and Tommi Mikkonen ha e con ibu ed equally o his wo k
* Vlad S i bu
[email p o ec ed]
Tuomas G anlund
[email p o ec ed]
Tommi Mikkonen
ommi.j.mikk[email p o ec ed]; [email p o ec ed]
1 CompliancePal, Tampe e, Finland
2 Soli a, Tampe e, Finland
3 Tampe e Uni e si y, Tampe e, Finland
4 Uni e si y o Jy äskylä, Jy äskylä, Finland
5 Uni e si y o Helsinki, Helsinki, Finland
So wa e Quali y Jou nal
1 3
de elopmen . As poin ed ou in Tai alsaa i e al. (2008), his leads o a new ype o de el-
opmen app oach, ad ancing in e olu iona y ashion, whe e so wa e is always on, and
upda es a e iny changes in he code. Conc e e models o such con inuous so wa e engi-
nee ing (Fi zge ald & S ol,2017) include con inuous deli e y (Humble & Fa ley,2010)
and De Ops (Ebe e al.,2016).
Howe e , no all so wa e li es on he Web, whe e applica ions can cons an ly e ol e
behind he cu ains. Ins ead, nume ous applica ions a e used o powe ac o ies, con ol
elec onics, p o ide guidance o d ones, and so on. Fo hese, i is common ha addi-
ional conce ns a e added in he de elopmen p ocess. These can be added o he con inu-
ous de elopmen p ocess as add-ons o amalgama ions, some imes also e lec ed in hei
espec i e names, such as De SecOps (My bakken & Colomo-Palacios,2017) o De Ops
used o de elop secu e sys ems, RegOps (D a e al.,2020) o digi alizing he egula o y
alue chain, o MLOps (T e eil e al.,2020) o con inuous deli e y o Machine Lea ning
(ML) ea u es.
Un o una ely, hese app oaches ocus on one pa icula aspec ha is added o he con-
inuous so wa e enginee ing pipeline, and do no conside how o in eg a e hem o a big-
ge whole. Hence, hei in e ope abili y emains weak. Consequen ly, ela ing egula o y
compliance and ML, o ins ance, equi es addi ional conside a ions which a e no a pa
o any o - he-shel app oach. As an example, conside he MLOps pipeline isualized in
Fig. 1, consis ing o da a ope a ions, execu ed by da a enginee s, da a analysis and ML
ope a ions, un by da a scien is s, and de elope s who implemen and deploy he inal
applica ion. In con as , RegOps ocuses only on so wa e de elopmen — he inal pa ,
pe o med by so wa e de elope s — bu o e looks he es (Toi akka e al.,2021; S i bu
& Mikkonen,2018). Hence, while MLOps helps in o ming a pipeline o he whole de el-
opmen e o , RegOps only suppo s he inal pa s wi h egula o y conside a ions and
design con ols.
In his pape , we in oduce con inuous design con ols o ML in ce i ied medical sys-
ems, co e ing he MLOps pipeline. The p oposed app oach s a s wi h con inuous so -
wa e enginee ing p ac ices, which is hen expanded wi h ML and da a p ocessing acili ies.
Then, we in oduce he necessa y egula o y p ocesses, which co e bo h so wa e and ML
pa s o he de elopmen . To simpli y p esen a ion, de ails o da a ope a ions, which in any
case a e o en speci ic o ce ain o ganiza ion (Aho e al.,2020), a e la gely o e looked,
al hough p ac ical echniques ha bind hem o con inuous so wa e enginee ing p ac ices
aining
code
Buildlabeled
da a
model Deploy
app
code
da a enginee s da a scien is s de elope s and ops
Fig. 1 Simpli ied MLOps pipeline. Figu e adap ed om G anlund e al. (2021)
So wa e Quali y Jou nal
1 3
a e included in he pape . The esul ing model is hen demons a ed wi h an indus ial case
s udy ha we ha e used in ou p e ious pape (G anlund e al.,2021), wi h an ex ended
discussion ega ding he p oposed imp o emen s.
The es o his pape is s uc u ed as ollows. In Sec .2, we in oduce he necessa y
backg ound o he pape . In Sec .3, we p opose a solu ion o con inuous design con ols.
In Sec .4, we demons a e he solu ion wi h an indus y example. In Sec .5, we discuss he
implica ions o he p oposed solu ion. In Sec .6, we d aw some inal conclusions.
2 Backg ound
Because o mul i- ace ed na u e o his wo k, i combines se e al di e en esea ch ields,
including con inuous so wa e de elopmen , ML, and he landscape o medical egula ions.
In he ollowing, we p esen ecen ad ances in hese ields, so ha we can in oduce he
p oposed p ac ical pipeline o egula ed MLOps.
2.1 Con inuous so wa e enginee ing p ac ices
The co e o con inuous so wa e enginee ing p ac ices is wo old. On one hand, i con-
sis s o a mindse whe e he de elope s ake esponsibili y o he whole so wa e as a
whole, and, while a single de elope wo ks on a pa icula ea u e, he bigge whole is
cons uc ed in e ms o de elope s’ collabo a i e e o . On he o he hand, i includes a
oolse ha allows deploying new ea u es o use as soon as hey a e a ailable (Fi zge ald
& S ol,2017). The goal is o p oduce con inuous low o alue adding so wa e a i ac s
om he de elopmen o he ac ual p oduc ion use, wi h quali y assu ance also happening
con inuously as a pa o he low.
A pa icula la o o con inuous so wa e enginee ing is called De Ops (Debois,2011;
Rajkuma e al.,2016). I can be desc ibed as a se o p ac ices whose goal is o sho en he
commi eedback cycle wi hou comp omising quali y (Bass e al.,2015), and o expand
he de elopmen eam wi h he ope a o s. In addi ion, he con inuous so wa e enginee -
ing oolse is expanded wi h moni o ing capabili ies, ensu ing ha each s akeholde ge s a
imely access o wha hey need.
Finally, i is impo an o no ice ha he au oma ed pipeline is no abou so wa e going
in o p oduc ion wi hou any ope a o supe ision, bu a he he pipeline p o ides a eed-
back loop o each o he s akeholde s om all s ages o he deli e y p ocess. Mo eo e , as
he so wa e p og esses h ough he pipeline, di e en s ages can be igge ed o example
by ope a ions and es eams by he click o a bu on.
2.2 ML li e cycle challenges
The ypical p ocess o de eloping a ML applica ion s a s wi h ans o ming inpu da a
om a a ie y o sou ces in o a collec ion o labeled da a, a p ocedu e pe o med by da a
enginee s. This collec ion is hen used by da a scien is s o pe o m a se ies o expe imen s
ha allows hen o build a se o candida e models. The model ha ul ills bes he desi ed
c i e ia cap u ed in he unc ional and non- unc ional equi emen s is selec ed o deploy-
men . Finally, he selec ed model is inco po a ed in o he so wa e sys em and deployed
o he p oduc ion en i onmen , a p ocedu e ha is pe o med by so wa e and ope a ions
enginee s.
So wa e Quali y Jou nal
1 3
Using con inuous so wa e enginee ing p ac ices h oughou ML applica ion li e cycle
is no s aigh o wa d. In Fig.1, we can see ha he de elopmen is pe o med ac oss h ee
compe ence clus e s: da a enginee s, da a scien is and so wa e de elope s. The le el o
expe ise and skills ela ed o con inuous enginee ing p ac ices a ies among hese special-
ies: while so wa e de elope s use ooling o achie e a high deg ee o au oma ion in hei
daily wo k, da a scien is and, o a lesse ex en , da a enginee s ha e a less s uc u ed way
o wo king. Many da a scien is s a e no awa e o ools like e sion con ol sys ems ha
can ack e icien ly changes in he aining code and he da a used o pe o m he expe i-
men s, o icke ing sys ems ha enable hem o ack p og ess om he ea u e implemen a-
ion o equi emen s. Ins ead, hey ely on bespoke solu ions ha migh no be app op ia e
when p ac iced in he de elopmen p ocess o sa e y c i ical sys ems.
To inco po a e ML ea u es in so wa e de elopmen , se e al echniques ha e been
p oposed. In his wo k, we build on Con inuous Deli e y o ML (Sa o e al.,2019) and
ML Model Ca ds (Mi chell e  al.,2019). These echniques a e b ie ly in oduced in he
ollowing.
2.2.1 Con inuous deli e y o ML
P obably he bes -known MLOps implemen a ion, Con inuous Deli e y o Machine
Lea ning (CD4ML) (Sa o e  al., 2019) by Though Wo ks, aims a au oma ing he ML
applica ion li e cycle in an end- o-end ashion (Fig.2). In CD4ML, a c oss- unc ional eam
p oduces applica ions based on code, da a, and models in small and sa e inc emen s ha
can be ep oduced and eliably eleased a any ime, in sho adap a ion cycles. Th ee dis-
inc s eps a e included: (i) iden i y sui able da a sou ces and p epa e he da a o aining,
(ii) expe imen wi h di e en models o ind he bes pe o ming candida e, and (iii) deploy
and use he selec ed model in p oduc ion as a pa o a bigge so wa e sys em.
egula o y "locked"
bounda y
Building
E alua ion and
Expe imen a ion
Packaging
Tes ing
Deploymen
Moni o ing
aining
da a
candida e
model
selec ed
model
packaged
model
packaged
model
aining
code
applica ion
code
alida ion
da a
es
code
me ics
applica ion
bundle
machine lea ning pipeline deploymen pipeline
da a scien is sde elope s and opsda a enginee s
da a
sou ce
da a labeling pipeline
es
da a
Fig. 2 CD4ML pipelines and a i ac s. Figu e adap ed om G anlund e al. (2021)

So wa e Quali y Jou nal
1 3
CD4ML is no he only solu ion ha aims o p e en he accumula ion o hidden ech-
nical dep in machine lea ning applica ions (Sculley e al., 2015). While o he solu ions
a e hea ily op imized o a pa icula cloud in as uc u e (AWS Solu ions,2021; Google
Cloud Solu ions,2021), an ML so wa e s ack implemen a ion (Baylo e al.,2017), o con-
cep ual cha ac e is ics o he app oach like MLOps (John e al.,2021), he CD4ML model
has he ad an age ha i can be used as a e e ence model in any applica ion domain. Fu -
he mo e, i has he necessa y phases documen ed a he app op ia e le el, and he imple-
men a ion is based on open sou ce componen s. Hence, i se es as a solid ounda ion o
expe imen a ion, and o ansla e he esul s o o he MLOps pipelines.
2.2.2 ML model ca ds
ML model ca ds is a amewo k o anspa en communica ion o in o ma ion ha acili-
a es he co ec u iliza ion o machine lea ning models (Mi chell e al.,2019). In ended as
documen a ion ha accompanies a ained machine lea ning model, he model ca d con-
ains in o ma ion abou in ended use case, benchma k e alua ion in a a ie y o ele an
condi ions, such as demog aphics o geog aphic loca ion, o any o he in o ma ion ha he
c ea o s conside sui able o he p ope use o he ained model. Al hough he o iginal
p oposal p esen ed he model ca d as a isual ep esen a ion, ecen de elopmen s1 p opose
a p og amma ic mechanism o gene a e he model ca ds, and a machine- eadable se ializa-
ion2 ha acili a es he consump ion o model ca ds by sc ip s in ML pipelines.
While he machine- eadable se ializa ion o he model ca d is a s ep in he good di ec-
ion, he in o ma ion included in he ca d is limi ed o speci ic audiences, like da a scien-
is s o machine lea ning enginee s. To ul ill he model ca ds po en ial, he in o ma ion
should be ex ended o include he needs o o he s akeholde s. The model ca d me ada a
becomes a model om which a ious iews a ge ed a di e en s akeholde s can be
de i ed, in a simila ashion as a ious so wa e a chi ec u e iews can be de i ed om
a single so wa e model. The app oach will u n he model ca d in o a mus ha e a i ac
ha con eys no only he in o ma ion abou he model’s in ended use o pe o mance, bu
also wha o he ac i i ies ha e been conduc ed in ela ion wi h he model (e.g., egula o y
isk managemen ), o me ada a abou da a se s ha acili a e downs eam es ing (G anlund
e al.,2021).
2.3 Medical egula o y landscape
The manu ac u ing o medical de ices is s ic ly con olled by au ho i ies, and manu ac-
u e s mus con o m o he egion’s egula o y equi emen s in which a medical de ice is
being ma ke ed o use. Fo his eason, medical so wa e sys ems mus also be de eloped
acco ding o he equi emen s o he a ge a ea. Fo example, he de elopmen is egula ed
by Medical De ice Regula ion (MDR) (Eu opean Pa liamen and he Council,2017) and
InVi o Diagnos ics Regula ion (IVDR) (Eu opean Pa liamen and he Council,2017) in
he EU egion and by Fede al Food, D ug, and Cosme ic Ac (FD&C Ac ) (U.S. Depa -
men o Heal h and Human Se ices,2021) in he USA.
1 h ps:// gi hub. com/ enso low/ model- ca d- oolk i
2 Tenso Flow model ca d schema
So wa e Quali y Jou nal
1 3
2.3.1 Design con ol
The egula o y amewo k aims o ensu e ha a medical de ice is sa e o use and clinically
e ec i e o i s in ended medical pu pose. In p ac ice, he e a e ce ain manda o y p o-
cesses ha include con ol mechanisms o he whole so wa e li e cycle, including design,
de elopmen , and manu ac u ing o he p oduc . The egula o y equi emen s ela ed o
hese p ocess phases a e gene ally e e ed o as he Design Con ols (FDA - Cen e o
De ices and Radiological Heal h,1997).
The pu pose o he Design Con ol p ocess (depic ed in Fig.3), is o p omo e a well-
designed de elopmen p ocess ha includes aceabili y be ween p ocess inpu s and ou -
pu s a di e en s ages o he p ocess. S a ing om use needs con e ed in o design
inpu s, con inuing wi h he design p ocess ha ans o ms he inpu s in o design ou pu s,
and inally o ming he esul ing medical de ice. In addi ion, he e iews du ing each s ep
o he p ocess e i ies and alida es ha he equi emen s a e me by he implemen a ion,
educing he possibili y o design and implemen a ion de ec s. In gene al, hese egula-
o y bounda ies a e add essed in so wa e de elopmen wi h dedica ed medical so wa e
li e cycle managemen ools, such as Pola ion3.
Fo so wa e medical de ices he design con ol is implemen ed in wo laye s, depic ed
in Fig.4: he p oduc and sys em de elopmen ac i i ies (IEC 82304 (In e na ional Elec-
o echnical Commission, 2016)), and he so wa e de elopmen ac i i ies (IEC 62304
(In e na ional Elec o echnical Commission, 2015)). A he p oduc le el, he iden i-
ied use needs a e con e ed o sys em equi emen s ha se e as design inpu s o he
so wa e de elopmen p ocess. Du ing so wa e de elopmen , he sys em equi emen s
a e ans o med in o high le el so wa e equi emen s ha co e he so wa e sys em and
Re iew
Use Needs
Re iew
Design Inpu
Re iew
Design P ocess
Ve i ica ion Re iew
Design Ou pu
Valida ion Medical De ice
Fig. 3 Design con ol p ocess o medical de ices. Figu e adap ed om FDA - Cen e o De ices and
Radiological Heal h (1997)
3 h ps:// pola ion. plm. au om a ion. sieme ns. com/ p odu c s/ pola ion- alm
So wa e Quali y Jou nal
1 3
a chi ec u al conce ns. La e on, he high le el so wa e equi emen s a e u he dis illed
in o low le el so wa e equi emen s ha se e as design inpu o implemen a ion.
The a chi ec u al design ac i i y de ines he majo s uc u al componen s o he so -
wa e, known as so wa e i ems. I iden i ies hei key esponsibili ies, hei ex e nally
isible p ope ies, and he ela ionship among hem. The esul ing so wa e a chi ec u e
a i ac ensu es he co ec implemen a ion o he so wa e equi emen s, and is comple e
when all so wa e equi emen s can be implemen ed by he iden i ied so wa e i ems. The
a chi ec u al decisions a e ex emely impo an o implemen ing e ec i e isk con ol
measu es. The p ope unde s anding and accu a e documen a ion o so wa e i ems beha -
io a e essen ial o ensu ing ha he so wa e sys em is sa e. De ailed design ac i i ies
e ine he iden i ied so wa e i ems du ing a chi ec u e design in o smalle so wa e i ems.
When a so wa e i em is no decomposed u he i is called so wa e uni . In he end, he
manu ac u e is esponsible o he g anula i y o he so wa e decomposi ion, and should
ensu e ha he ac i i y pe o med o he app op ia e de ail o allow a sa e and e ec i e
implemen a ion.
The esul ing code, es cases and a ious o he a i ac s, such as a chi ec u e and
de ailed module design documen a ion, c ea ed du ing he so wa e de elopmen ac i i-
ies, se e as he design ou pu s. The e iew o he a i ac s and he es esul p o ide an
e ec i e e i ica ion p ocedu e a uni , in eg a ion and sys em le el. The accep ance es s
oge he wi h he esul epo s o clinical ials se e as he alida ion p ocedu e. All hese
p ocedu es ensu e ha he p ope design con ols ha e been applied du ing de elopmen ,
esul ing in a medical p oduc ha mee s he use needs
2.3.2 Design con ol o ML: hemissing pa s
Al hough he design con ol p ocess o medical so wa e is o he wise well-de ined, he
egula o y amewo ks o he majo ma ke a eas, such as he EU and USA, do no cu en ly
explici ly add ess he equi emen s ela ed o AI/ML echnologies. While AI/ML-based
High le el SW
equi emen s/
A chi ec u al design
Low le el SW
equi emen s/
De ailed design
Uni es
In eg a ion es
Sys em es
Accep ance es
Sys em
equi emen s
Use equi emen s
Use needs Use needs me
Sys em de elopmen ac i i ies
So wa e de elopmen ac i i ies
Implemen a ion
IEC 82304
IEC 62304
Ve i ica ion
Ve i ica ion
Ve i ica ion
Valida ion
Fig. 4 Sys em and so wa e de elopmen design con ol ac i i ies
So wa e Quali y Jou nal
1 3
medical de ices ha e g ea po en ial o imp o e ca e o indi idual pa ien s, hey ca y
some unique isks. Fo example, ce ain ML sys ems may be designed o lea n and op imize
hei unc ionali y in eal ime. As a esul , he use - acing isk p o ile o hese sys ems may
change o e ime which is an incompa ible idea compa ed o gene al design con ol p ac-
ices. E iden ly, he cu en lack o p ecise equi emen s o AI/ML is a se e e sho coming
in he legisla ion as i c ea es pa icula challenges and unce ain ies o he manu ac u e s
on how o p o e AI/ML de ice sa e y and e iciency when seeking de ice app o als.
In gene al, he AI/ML de ices mus comply wi h equi emen s applicable o all medi-
cal de ice so wa e. The e o e, manu ac u e s a e equi ed o manage he isks ela ed
o hei p oduc s h oughou he de ices’ whole li e cycle (Eu opean Pa liamen and he
Council, 2017; G anlund e  al., 2021). In p ac ice, medical so wa e isk managemen
ac i i ies a e implemen ed acco ding o he equi emen s o ISO 14971 (In e na ional
O ganiza ion o S anda diza ion, 2019) and IEC 62304 (In e na ional Elec o echnical
Commission, 2015). Howe e , he isk managemen ac i i ies desc ibed in de ail in he
s anda ds men ioned assume ha he de ice’s unc ionali y emains he same as du ing he
p oduc de elopmen phase, e en a e deploymen , and does no change o e ime. As a
esul , i may be di icul o he AI/ML manu ac u e o p o e he e ec i eness o he
implemen ed isk managemen ac i i ies agains he cu en equi emen s.
Fu he mo e, speci ic AI/ML models can be complex and da a-in ensi e, so hey may
be complica ed o unde s and ully. As a esul , i may no be easy o assess how he model
has eached a decision. In addi ion, he quali y o da a plays a signi ican ole and may con-
ain biases no isible o a human audi o . Also, speci ic challenges ela ed o model ain-
ing, such as o e i ing and unde i ing, mus be conside ed when alida ing he sys em’s
clinical e iciency and sa e y.
The e a e cu en ly ce ain ongoing e o s o add ess he p oblem o missing AI/ML
egula o y guidance. Fo example, in he EU, he heme “A i icial In elligence unde
MDR/IVDR amewo k” will be add essed wi hin he o hcoming guidance documen
by he Medical De ice Coo dina ion G oup (MDCG) (Medical De ice Coo dina ion
G oup, 2021). Fu he mo e, in he USA, he US Food and D ug Adminis a ion (FDA)
has eleased an ac ion plan documen “A i icial In elligence/Machine Lea ning (AI/ML)-
Based So wa e as a Medical De ice (SaMD) Ac ion Plan” (U.S. Food and D ug Admin-
is a ion (FDA),2021). While speci ic and binding egula o y equi emen s a e s ill unde
de elopmen , he In e es G oup o he No i ied Bodies o Medical De ices in Ge many
(IG-NB) has c ea ed pe haps he mos de ailed and conc e e guideline a ailable cu en ly,
“Guideline o A i icial In elligence in Medical De ices” (de Benann en S ellen ü
Medizinp oduk ein Deu schland(IG-NB),2021). The IG-NB guideline can be used as a
basis o gain an unde s anding o he expec a ion le el o he no i ied bodies ela ed o AI/
ML p oduc s.
3 P oposed solu ion
The p oposed design con ol p ocess o CD4ML pipelines aims a o malizing so wa e
de elopmen so ha i be used o QMS pu poses. This is achie ed by using pull eques s
as basis o e iews ha o ms he design con ol, and using model ca ds me ada a as he
design ou pu a i ac ha se es also as an audi ail o egula o y ac i i ies such as clini-
cal alida ion and isk managemen .
So wa e Quali y Jou nal
1 3
been implemen ed in line wi h he equi emen s o IEC 62304, which go e ns he de el-
opmen o so wa e used in medical p oduc s.
In p ac ical e ms, he pull eques cons i u es he design con ol mechanism ha ensu e
ha he e olu ion o he sys em is sys ema ically e iewed and ha he code baseline is
always up o da e om a egula o y pe spec i e. Besides aligning he de elopmen and
he egula o y ac i i ies, we iden i ied he model ca d me ada a as an ideal candida e o
documen ing no only he model bu also he egula o y speci ic ac i i ies pe o med
du ing model’s de elopmen , such as da ase jus i ica ion o clinical pe o mance e alu-
a ion, among o he p e-ma ke isk managemen ac i i ies. The model ca d me ada a docu-
men , oge he wi h he model code, se es e ec i ely as he design ou pu a i ac . Being
machine- eadable, he model ca d me ada a can be used in pipelines o gene a e au oma i-
cally addi ional documen s in ended o end use s (e.g., he model ca d), and egula o y
au ho i ies (e.g., clinical alida ion epo ). Pos -ma ke moni o ing and main enance ac i -
i ies ha iden i y de ia ions om he expec ed model beha io a e iden i ied and cap u ed
as bug epo s o eedback and ed in o he eam backlog as equi emen s. The e minol-
ogy and de elopmen phase ha moniza ion oge he wi h he design ou pu a i ac s a e
desc ibed in Fig.9.
4 Case s udy: O a izio p ocess e ised
In ou p e ious wo k, we ha e in oduced O a izio6, CE ce i ied medical so wa e o
assessing he isks o join eplacemen su ge ies (G anlund e al.,2021). We use his sys-
em o demons a e how o apply con inuous design con ol o ML in ce i ied medical
sys ems. The wo k is a concep p o o ype in i s na u e; i builds on expe iences om an
indus y sys em, bu he p oposed implemen a ion has no been deployed o indus ial use.
Ou p e ious wo k wi h O a izio has in oduced a con inuous aining pipeline. The
pipeline allows o o e come egula o y cons ain s associa ed wi h O a izio ML model
aining and o simul aneously achie e au oma ion goals associa ed wi h MLOps (G anlund
Pe o mance
e alua ion
Iden i y
de ia ions
Iden i y isks and
mi iga ions
De ailed design
So wa e uni implemen a ion and e i ica ion
So wa e
in eg a ion and
in eg a ion es ing
So wa e sys em
es ing So wa e elease
Da a p epa a ion Building E alua ion and
expe imen a ion Packaging Tes ing Deploymen Moni o ing
CD4ML
IEC 62304
Main enance
Ma ke
p e pos
So wa e isk managemen (ML)
Da ase
jus i ica ion
Gene a e alida ion
epo
Fig. 9 Design con ol p ocess
6 h ps:// o a iz. io/

So wa e Quali y Jou nal
1 3
e al.,2021). In addi ion, he pipeline add esses he medical de ice so wa e design con ol
equi emen s by design.
Fo his pape , we ha e e ised he pipeline by ex ending i wi h a ca e ully selec ed se
o model ca d documen s, o demons a e he p oposed solu ion. Figu e 10 p esen s he
pipeline o G anlund e al. (2021), wi h he p oposed model ca d ex ensions ma ked wi h a
da ke colo .
4.1 Con inuous aining wi hMLOps pipeline
The da a used o ain and e- ain he O a izio ML models is gene a ed wi hin he clinical
p ocesses o a collabo a ing pa ne hospi al, and, by design, O a izio does no gene a e
co esponding da a in p oduc ion use. Because o he sensi i e na u e o da a, access o
he hospi al’s compu a ional en i onmen is s ic ly es ic ed. As a esul , he con inuous
aining pipeline was designed o ope a e inside he clinical pa ne ’s con olled en i on-
men and e ch new da a om he da a s o e based on p e-de ined igge s. Fu he mo e,
as he models a e de e minis ic by na u e, hei echnical pe o mance can be alida ed
acco ding o he p inciples o clinical e alua ion wi h es da a in a es ic ed en i onmen
Da a S o e
- scheme
- alues
Da a Ex ac
- con igu able pe
ins alla ion
Da a Valida ion
Da a
snapsho
Ve i ica ion
T aining &
Building
Packaging &
alida ion suppo
T igge
Pe o mance
& alida ion
epo
Packaged model
Audi ail
Manual esolu ion
Anomaly de ec ed
Pe o mance
dec eased
Isola ed & es ic ed en i onmen
(e.g. hospi al's compu a ional en i onmen )
Deploymen pipeline
Design and
de elopmen ou pu
e iew & alida ion
Da a enginee s and da a scien is s
So wa e de elope s and ops
Risk manage s and compliance o ice s
Da a Quali y
Con ol
Risk
Managemen
Pe o mance
equi emen s
Da a P epa a ion
- anonymous da a
Managemen o
da a se s
Fig. 10 Con inuous aining pipeline wi h model ca ds. A ows indica e da a lows
So wa e Quali y Jou nal
1 3
wi hou he need o do he in ended medical use speci ic alida ion in he inal p oduc ion
en i onmen . Finally, O a izio was designed o be deployed in a p oduc ion en i onmen
wi h i s ML model in a “locked” s a e o egula o y easons. In p ac ice, O a izio’s models
a e ained du ing he de elopmen phase, and hei abili y o imp o e he ou come on he
ly is disabled in p oduc ion use. Despi e his limi a ion, he pipeline enables he labo ious
ask o e- aining o be au oma ed.
In addi ion, he ac ha he de elopmen eam does no need access o he es ic ed
en i onmen beyond he pipeline’s ins alla ion and main enance is a aluable design ea-
u e. Fu he mo e, he pipeline can au oma ically gene a e he equi ed documen a ion
needed o assess he model pe o mance in he clinical pe o mance e alua ion. All c ea ed
a i ac s a e deli e ed o he de elopmen eam om he isola ed en i onmen .
4.2 Design con ol documen a ion wi hmodel ca ds
E en i he o iginal e sion o he pipeline in G anlund e al. (2021) con ains he equi ed
design con ol documen a ion, which can be gene a ed au oma ically, he documen a ion
o ma has no been based on any gene ally known s anda d. The eason is ha a he ime
o implemen ing he pipeline, no such o ma was a ailable (Mi chell e al.,2019). In addi-
ion, he selec ed documen empla es ha e been simila o mo e adi ional quali y man-
agemen sys em ypes o eco ds a ge ed o egula o y s akeholde s and wi hou he ech-
nical abili y o be se ialized. To add ess hese challenges and u he suppo con inuous
design con ol o ML, we expanded he pipeline wi h he model ca ds ailo ed o add ess
he design con ol documen a ion equi emen s.
4.2.1 Da a se managemen
To ensu e he equi ed pe o mance o he models, he selec ed da a se mus be ep esen a-
i e o he a ge popula ion. In O a izio’s case, da a sou ces could di e pe ins alla ion,
and, as a esul , he p ocedu e o da a ex ac ion mus be con igu able. In addi ion, he e
a e many ela ed documen a ion equi emen s, which can be documen ed wi h a model
ca d. Fi s ly, he allowed da a sou ces mus be lis ed wi h he speci ic equi emen s and
desc ip ions o a da a sou ce. Secondly, he da a inclusion and exclusion c i e ia mus be
de ined wi h he p ocedu e o in alid da a managemen . Thi dly, da a p o ec ion policy
needs o be de ined wi h clea ins uc ions on how o ensu e da a p o ec ion a la e s ages
o da a p ocessing. Finally, po en ial biases in he da a mus be e lec ed and he selec ions
made jus i ied acco dingly.
4.2.2 Da a quali y con ol
As pa o da a quali y con ol o a sys em using supe ised lea ning, such as O a izio, he
labeling and label e i ica ion p ocedu es a e essen ial. Mo eo e , when u ilizing au oma-
ion in e-lea ning, he co ec ness o labeling needs o be cons an ly moni o ed. I is pos-
sible o use di e en p e-de ined schemas and bounda ies o alida e he da a quali y, bo h
in e ms o o ma and con en . As model ca ds a e al eady in a machine- eadable o ma ,
hey can be used simul aneously as a documen a y and a alida ing a i ac . I he a e o
e o in he da a alida ion ises abo e he accep able limi , he e-lea ning canno con inue
au oma ically, and he anomaly mus be esol ed manually.
So wa e Quali y Jou nal
1 3
4.2.3 Risk managemen
In conjunc ion wi h clinical e alua ion, e ec i e isk managemen p o ides a p ac ical
ool o he manu ac u e o p o e he sa e y and clinical e iciency o he de ice. As
he ML models a e a cen al pa o O a izio, he models ha e a conside able impac on
he de ice’s isk p o ile. As a esul , he po en ial isks ela ed o he models need o be
ca e ully add essed.
When u ilizing model ca ds, he iden i ied model- ela ed haza ds and co espond-
ing isk mi iga ions a e documen ed on he model ca d documen . I is wo h no icing
ha ce ain isks, bo h in e ms o pa ien sa e y and da a secu i y, a e unique o AI/ML
sys ems. These isks include, o ins ance, model d i , he d i in da a dis ibu ion, and
isks ela ed o con inuous lea ning sys ems. In addi ion, also isks ela ed o ad e sa ial
a acks need o be conside ed.
4.2.4 Pe o mance equi emen s
Acco ding o he egula o y equi emen s, manu ac u e s o medical de ices mus docu-
men he in ended pu pose o hei de ice, including speci ica ion o indica ions, con-
aindica ions, pa ien a ge g oups, and ope a ing pa ame e s and limi a ions (Eu opean
Pa liamen and he Council,2017). In addi ion, pe o mance cha ac e is ics, accu acy,
and he limi s o accu acy, p ecision, and analy ical pe o mance mus be add essed i
applicable.
Many o he abo e equi emen s apply di ec ly o O a izio’s ML models. Mo eo e ,
hey can be con enien ly documen ed wi hin a model ca d.
5 Discussion
In his sec ion, we p o ide an ex ended discussion ega ding he p oposed app oach,
highligh ing how i b ings bene i s o he in ol ed s akeholde s. In addi ion, we discuss
limi a ions o his wo k.
5.1 Aligning ML de elopmen and egula o y p ac ices
The use o ML echnologies in ce i ied medical de ices is an eme ging end in a no o-
iously conse a i e indus y. Consequen ly, he egula o y p ac ice is no as es ablished
as he p ac ices in he de elopmen o adi ional so wa e medical de ices.
Ou p oposal b ings oge he he ML applica ion li e cycle, demons a ed using
CD4ML, and he medical de ice so wa e li e cycle p ocess s anda d IEC 62304. The
esul lowe s he cogni i e ba ie s be ween he machine lea ning model de elope s,
such as da a enginee s and da a scien is s, and egula o y p ac i ione s, allowing hem
o wo k oge he o e ec i ely de elop medical de ices ha include machine lea ning
echnologies.
So wa e Quali y Jou nal
1 3
5.1.1 A oiding common ML sys em design p oblems
In gene al, ML models cons i u e only a subse o he inal sys em ha inco po a es he
espec i e ML echnology and makes i a ailable o he end use s (Sculley e al.,2015).
Wi h his in mind, he con inuous design con ol app oach o handling ML in ce i ied
medical sys ems de elopmen i s unde he p ocess managemen and ools ca ego y.
Table1 desc ibes how he ac i i ies de eloped as pa o he con inuous design con-
ol help mi iga e design p oblems, such as he accumula ion o echnical dep , and
he ole played by he model ca ds documen s as he audi ail o pe o ming hese
ac i i ies.
5.1.2 Answe ing egula o s’ conce ns
Al hough he use o ML echnology wi hin he medical de ices is ela i ely new, he egu-
la o s a e ac i ely engaged in a dialog wi h he indus y o guide i s adop ion (Food and
D ug Adminis a ion,2021). This indica es ha he egula o s a e awa e o he new ech-
nologies, and a e conside ing how o bes egula e he de elopmen o medical de ices
ha include ML ea u es. Table 2 desc ibes how ou app oach add esses he egula o s’
conce ns.
5.2 Model ca d me ada a asaudi ail
The eme ging model ca d ecosys em inc eases he enginee ing ma u i y o machine
lea ning model de elopmen . The model ca d me ada a documen p o ides an ex ensible
machine- eadable medium in which conce ns ela ed o he model de elopmen can be cap-
u ed. In ou wo k, we le e aged he model ca d me ada a documen o cap u e he egu-
la o y aspec s, such as in ended use, he sou ces o da a used o aining, o he clinical
pe o mance e alua ion, ele an when he machine lea ning model is used in a ce i ied
medical de ice.
In doing so, we e ined exis ing p ope ies de ined in he Tenso Flow’s model ca d
me ada a schema and added new p ope ies when needed (e.g., model_pa ame e s.
da a[].x_sou ces). We ound ha he desc ip ion p ope ies de ined in he
schema documen as s ing a e no s uc u ed enough and we used Ma kdown o ha e a
empla e d i en ep esen a ion o he in ended use o he model (e.g., model_de ails.
documen a ion), and o he desc ip ion o he da ase s used in model aining (e.g.,
model_pa ame e s.da a[].desc ip ion). The app oach allowed us o i e a e
as , enabling he eam membe s o ocus on adding con en . The expe ience, backed wi h
eedback om o he implemen e s, will allow us o iden i y he ele an in o ma ion ha
can e en ually be ex ac ed and o malized in o model ca d schema ex ensions.
The combina ion o using a s uc u ed documen wi h he semi-s uc u ed ma kdown
desc ip ion is app op ia e o he a ge audience o med by enginee s and egula o y p o-
essionals, each ca ego y con ibu ing using speci ic modali ies. Al hough he app oach is
e ec i e a collec ing he needed in o ma ion ha se es as an audi ail o he ac i i ies
pe o med by he eam membe s, edi ing he me ada a documen using ex edi o s does no
p o ide he bes use expe ience o all use s.
So wa e Quali y Jou nal
1 3
Table 1 Mi iga e common ML sys em design p oblems (Sculley e al.,2015) wi h con inuous design con ol
ML sys em design p oblems Mi iga ion s a egy Model ca d documen ole
Con igu a ion: Do we know all con igu a ion op ions
and hei e ec s? Con igu a ion managemen is a p ime conce n in he
SDLC o medical sys ems (In e na ional Elec o echnical
Commission,2015). The exis ing p ac ices es ablished
by he medical de ice manu ac u e o con igu a ion
and so wa e isk managemen can be expanded o co e
he con igu a ions o he ML model in eg a ed in o he
medical sys em.
Con ains he ML model pa ame e s and hei alid anges
o he in ended use.
Da a collec ion & Fea u e ex ac ion: Do we know i he
inpu da a and ea u es de eloped a e enough o he
in ended use?
The e iew includes an analysis o he selec ed da a
sou ces and he me hods used o ea u e ex ac ion.
The model ca d is ex ended o cap u e he sou ce o each
da a se . Toge he wi h he jus i ica ion sec ion included
desc ip ion, i p o ides he e idence o needed o ul ill
he clinical e alua ion.
Da a e i ica ion The ele an da a se is used o e i y he in eg a ion o
each change eques .
The e i ica ion da a se is included in he model ca d,
and should be used by he ele an MLOps in eg a ion
s ages.
Resou ce managemen : Do we know ha he needed
ha dwa e and so wa e esou ces a e a ailable o
ensu e he model pe o ms co ec ly?
The quali y managemen sys em, implemen ed by
he manu ac u e o ul ill egula o y equi emen s
(In e na ional O ganiza ion o S anda diza ion,2016),
ensu es ha he needed esou ces o he p ope
unc ioning o he medical p oduc a e alloca ed.
The model ca d con ains he in o ma ion abou he
special esou ces ha a e needed o un he model. The
in o ma ion should be used downs eam when planning
he esou ce alloca ion.
Se ing in as uc u e: Do we know ha he model is
in eg a ed and wo ks p ope ly? ML models a e packaged and in eg a ed in o medical
sys ems as lib a ies (see Sec .3.2). As such, hei
unc ionali y is exposed ia an API and changes o he
ML model a e con ained.
-
Moni o ing: Can we de ec de ia ions while he model is
used wi h p oduc ion da a? Medical de ice manu ac u e s mus es ablish pos -ma ke
su eillance p og am (In e na ional O ganiza ion o
S anda diza ion,2016; In e na ional Elec o echnical
Commission,2015). The p og am should co e he
moni o ing he ML componen s in use.
The quan i a i e da a included in he model ca d documen
se es as inpu o moni o ing componen s ha de ec
de ia ions.

So wa e Quali y Jou nal
1 3
In he u u e, we plan o explo e wi h ha ing dedica ed edi o s o egula o y p o ession-
als so ha hey can in oduce hei con en using mo e amilia app oaches, such as wha
you see is wha you ge edi o s.
5.3 Pull eques ascon inuous design con ol
The pull eques is he p ac ice ypically used by so wa e de elopmen eams o man-
age changes. Ou p oposal ex ends he use o pull eques h oughou he machine lea n-
ing de elopmen li e cycle. Besides so wa e enginee s, da a enginee s and da a scien is
use he pull eques o manage he e olu ion o he so wa e p oduc s wi hin hei a ea o
Table 2 Suppo ing he good ML p ac ice guiding p inciples (Food and D ug Adminis a ion,2021) wi h
con inuous design con ol
Guiding p inciples Implemen a ion
Mul i-disciplina y expe ise is le e aged h oughou
he o al p oduc li e cycle
The pull eques is he enue o pe o m mul i-
disciplina y e iews du ing all de elopmen s ages
Good so wa e enginee ing and secu i y p ac ices
a e implemen ed
ML de elopmen is in eg a ed in o he p oduc and
so wa e de elopmen le e aging bes p ac ices and
ools
Clinical s udy pa icipan s and da a se s a e
ep esen a i e o he in ended pa ien popula ion
Al hough clinical s udies can be seen as pa ly
ou side he scope o p oduc de elopmen , he
model ca d documen can be used o documen
da a collec ion p o ocols and da a cha ac e is ics
ha a e ele an o he in ended pa ien popula ion.
In addi ion, con inuous design con ol p omo es
aceabili y om clinical s udy da a se s o he inal
model
T aining da a se s a e independen o es se s Following he con inuous design p ac ice ensu es ha
he es da a se is e iewed, e sioned and i is no
used du ing model de elopmen
Selec ed e e ence da a se s a e based upon bes
a ailable me hods
I accep ed e e ence da a se s a e a ailable, hei use
in he model de elopmen should be p omo ed and
documen ed in he model ca d documen
Model design is ailo ed o he a ailable da a and
e lec s he in ended use o he de ice
Con inuous design con ol ensu es, by including he
jus i ica ion in he model ca d documen , ha he
da a se s a e enough o sa is y he in ended use
Focus is placed on he pe o mance o he human-ai
eam
Tes ing a di e en de elopmen s ages (e.g., aining,
in eg a ion, sys em) ensu es ha he di e en
s akeholde s’ in e es s a e cap u ed
Tes ing demons a es de ice pe o mance du ing
clinically ele an condi ions
Con inuous es ing in s aging en i onmen s ha
mimic he clinically ele an condi ions
Use s a e p o ided clea , essen ial in o ma ion The egula o y equi ed documen a ion con ains
in o ma ion ha can be collec ed in he model ca d
documen s du ing de elopmen . The con inuous
design and e iew ac i i ies ensu es ha he end
use documen a ion i s hei needs
Deployed models a e moni o ed o pe o mance
and e- aining isks a e managed
The moni o ing s age o he MLOps pipeline,
implemen ing common echniques o de ec ing model
pe o mance anomalies, oge he wi h he egula o y
equi ed pos -ma ke moni o ing p ac ices enable he
manu ac u e o iden i y de ia ions and pe o m he
necessa y co ec i e ac ions
So wa e Quali y Jou nal
1 3
esponsibili y. As he pull eques is linked wi h equi emen s (e.g., design inpu s), and
he in oduced changes consis mainly o he machine lea ning model and he model ca d
me ada a (e.g., design ou pu s), we ha e an e ec i e quali y ga e ha ensu es ha design
e iews a e pe o med sys ema ically and he app op ia e audi ails a e build a e e y i e -
a ion h oughou he de elopmen li e cycle.
5.4 Handling model anomalies
One o he c i ical ad an ages o u ilizing in e p e able machine lea ning models is ha
hey allow o mo e e icien anomaly de ec ion and analysis. The ML model can be in e -
p e ed as consis ing o di e en componen s, such as inpu s, ea u es, pa ame e s, and
weigh s, and he unde s andabili y o he model inc eases i i can be decomposed in o
di e en explainable pa s (Lip on,2018). Ou p oposed app oach p o ides a solid ounda-
ion o documen di e en model aspec s o suppo explainabili y, which can, in u n, help
he enginee ing eam o pe o m anomaly and oo cause analysis ac i i ies. Fu he mo e,
e en i ini ially designed o p omo e egula o y ac i i ies in he o m o an audi ail, he
ine-g ained aceabili y p o ides addi ional suppo o he anomaly analysis. Based on he
esul s, he eam can de e mine he app op ia e co ec i e ac ions needed o be pe o med
and included in subsequen model eleases.
5.5 Sa e con inuous sel ‑lea ning
The abili y o lea n a e being deployed o eal-wo ld use is undoub edly one o he c i ical
di e ences be ween an AI/ML-enabled sys em and a mo e adi ional ule-based sys em.
Howe e , as discussed p e iously, due o he cu en egula o y unce ain ies, manu ac u -
e s o medical de ice AI/ML-enabled sys ems may p e e such AI/ML models ha can be
deployed in a locked s a e. I is e iden ha such a design app oach can se iously educe
he bene i s o AI/ML-enabled echnology. The e o e, al e na i e ye pa ien sa e y ensu -
ing design and de elopmen me hods a e needed.
A obus and e ec i e isk managemen p ocess is he basis o sa e medical de ice so -
wa e de elopmen . As he p ocess s a s wi h isk iden i ica ion (In e na ional O ganiza-
ion o S anda diza ion,2019), he de elopmen eam mus be compe en in assessing he
p oduc ’s speci ic ML change- ela ed aspec s, pa icula ly when he chosen echnology’s
complexi y and opaqueness inc ease. In p ac ice, a c oss- unc ional de elopmen eam
should include knowledgeable and expe ienced da a scien is s, in addi ion o he ypical se
o clinical and p oduc de elopmen specialis s.
Acco ding o he egula ions, medical de ice manu ac u e s mus seek app o al o
changes o he app o ed design o a de ice p io o making he change, whe e he change
has a subs an ial impac o can a ec he de ice’s con o mi y wi h he gene al sa e y and
pe o mance equi emen s (Eu opean Pa liamen and he Council,2017). The e o e, i is
clea ha i enabled, sel -lea ning can only occu wi hin a p e-de e mined ole ance and
change con ol plan. In addi ion, he manu ac u e is esponsible o demons a ing ha he
change ole ance complies wi h he de ice’s in ended use, use en i onmen , use g oups,
and o he medical claims p io o placing he de ice on he ma ke .
Finally, an essen ial aspec o sel -lea ning and sa e y is he abili y o moni o he
de ice’s pe o mance as a pa o he de ice’s pos -ma ke su eillance ac i i ies. Con a y
o he i s hough , i can be a gued ha sel -lea ning AI/ML sys ems a e, in ac , mo e
ole an agains model d i han he locked sys ems as hey a e cons an ly imp o ing hei
So wa e Quali y Jou nal
1 3
pe o mance wi h he new da a. Howe e , moni o ing he cons an ly changing sys em can
be mo e di icul as he e a e addi ional aspec s o conside and measu e. The mos impo -
an hing is o ensu e ha he de ice’s pe o mance canno dec ease due o an upg ade.
5.6 C oss domain e minology challenges
A con lic ing e minology is a common p oblem when se e al domains — such as medical
de ice egula o y concep s, da a science, and so wa e enginee ing — a e combined wi hin
a single p ojec . This p oblem can lead o miscommunica ion, misunde s andings, and, a
wo s , poo decision-making (Vogel,2011). The e minology con lic s we e also eme gen
wi hin his pape ’s con ex , pa icula ly ega ding he e m alida ion. Wi hin he ield o
ML alone, he e m has been used wi h wo di e en meanings: o da a cu a ion (i.e., da a
alida ion) o ML model uning. To make ma e s e en mo e complica ed, in he con ex
o medical de ice de elopmen , alida ion means con i ma ion ha he pa icula equi e-
men s o speci ic in ended use can be consis en ly ul illed (In e na ional Medical De ice
Regula o s Fo um,2022). As a p ac ical solu ion, we p opose a o ing egula o y e mi-
nology in he documen s ha demons a e con o mi y, and, in gene al en o cing explici
communica ion o a oid con usion.
5.7 In o ma ion secu i y conside a ions
The model ca d me ada a documen se es as an e ec i e audi ail o he model de el-
opmen . As such, i con ains a ple ho a o in o ma ion ha should be conside ed p i a e,
as i migh con ain pe sonal da a, in o ma ion ha is no open o public, o e en c i ical
ade sec e s. While he documen should se e as inpu o gene a ing he public echnical
documen a ion o he medical de ice, as expec ed by egula ion and applicable s anda ds,
manu ac u e s should employ he necessa y in o ma ion managemen p ac ices o ensu e
ha he p ope ies classi ied as p i a e a e no included in he model ca d ep esen a ions
in ended o public consump ion.
5.8 Limi a ions
Ou implemen a ion o he p oposed app oach le e ages exis ing ools and p ocesses
widely used by so wa e de elopmen eams, such as Gi o e sion con ol, issues o
acking equi emen s and wo k i ems, o pull eques s o e iews and change manage-
men . Howe e , we wish o poin ou ha ou implemen a ion has no been exposed o
a wide ange o eal li e medical p oduc s, excep O a izio. Fo example, managing he
e olu ion o he medical p oduc has been implemen ed using he ea u e-b anch app oach,
in which a new b anch is c ea ed om he mainline, o each equi emen , and me ged
ollowing a success ul e iew. O he de elopmen models such as unk-based de elop-
men (Jø gensen,2001) ha e no been in es iga ed ho oughly, al hough equi alen e iew
acili ies a e suppo ed by ools used o his de elopmen s a egy. The e o e, he p o-
posed app oach is no in ended o be a model ha sui s any medical p oduc o si ua ion,
which one mus ollow in a e ba im ashion. Ra he , we wan o emphasize ha sys ema ic
e iews and using he model ca d as he audi ail o egula o y ac i i ies ep esen an
e ec i e o m o design con ol ha is compa ible wi h he egula o y equi emen s ha
So wa e Quali y Jou nal
1 3
go e n medical de ices ha con ain so wa e. Simila app oaches o ack he model ca d
me ada a and pe o ming equi alen ac i i ies will mos likely esul in a sa is ac o y solu-
ion om a egula o y pe spec i e.
Ope a ions ela ed o da a ha e been o e looked in he pape , because much o he wo k
happens in da a enginee s’ own en i onmen , ollowing hei own ways o wo king (Aho
e al.,2020). Howe e , we p oposed model ca ds as a mechanism o eco d he ail o p o -
enance om da a ope a ions o he model, so ha his pa can be included in he MLOps
pipeline as well. The e o e, explo ing he da a ope a ions and hei ela ion o model ca ds
is a pa o u u e wo k we plan o ca y ou .
6 Conclusions
Theso wa e enginee ing indus y has widely adop ed con inuous de elopmen and deploy-
men o new ea u es. These ea u es may include AI/ML unc ions, which ha e become
commonplace in nume ous applica ions, calling o deploymen pipelines whe e such unc-
ions can be included in mains eam de elopmen ac i i ies. Such con inuous se up o ms
a sha p con as o he de elopmen o medical sys ems, whe e design con ols a e o en
in e p e ed o equi e wa e all-like de elopmen app oach.
In his pape , we p opose using an app oach whe e con inuous design con ol o ML
is enabled while de eloping medical sys ems. The p oposed app oach builds on ou ea -
lie wo k on MLOps, bu ex ends i wi h he design con ols ha a e explici ly included in
he MLOps pipeline. The app oach was demons a ed wi h an indus y sys em, which is in
ac i e use. As u u e wo k, we plan o in es iga e da a ope a ions, ela ed o building ML
models, in mo e dep h.
Acknowledgemen s The au ho s wish o hank p ojec AHMED and associa ed conso ium, unded by
Business Finland, o suppo ing his esea ch.
Funding Open Access unding p o ided by Uni e si y o Helsinki.
Da a a ailabili y Da a sha ing no applicable o his a icle as no da ase s we e gene a ed o analyzed du ing
he cu en s udy.
Decla a ions
Con lic o in e es The au ho s decla e no compe ing in e es .
Open Access This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License,
which pe mi s use, sha ing, adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as long
as you gi e app op ia e c edi o he o iginal au ho (s) and he sou ce, p o ide a link o he C ea i e Com-
mons licence, and indica e i changes we e made. The images o o he hi d pa y ma e ial in his a icle
a e included in he a icle’s C ea i e Commons licence, unless indica ed o he wise in a c edi line o he
ma e ial. I ma e ial is no included in he a icle’s C ea i e Commons licence and you in ended use is no
pe mi ed by s a u o y egula ion o exceeds he pe mi ed use, you will need o ob ain pe mission di ec ly
om he copy igh holde . To iew a copy o his licence, isi h p:// c ea i eco mmons. o g/ licen ses/ by/4. 0/.