Critical privacy issues on medical data

id192780


CRITICAL PRIVACY ISSUES ON MEDICAL DATA
ISABEL PIEDRAHÍTA VÉLEZ
Thesis supe iso
ULISESCORTÉSGARCÍA(Depa men o Compu e Science)
Thesis co-supe iso
ATIACORTÉSMARTÍNEZ(Ba celonaSupe compu ingCen e )
Deg ee
Mas e 'sDeg eeinA i icialIn elligence
Mas e 's hesis
School o Enginee ing
Uni e si a Ro i a i Vi gili (URV)
Facul y o Ma hema ics
Uni e si a de Ba celona (UB)
Ba celona School o In o ma ics (FIB)
Uni e si a Poli ècnica de Ca alunya (UPC) - Ba celonaTech
27/01/2025

Acknowledgmen s
I wish o exp ess my g a i ude o all he indi iduals who con ibu ed o he comple ion o his
mas e ’s hesis. I would i s like o hank my supe iso , P o . Ulises Co ´es, o his in aluable
pa ience, ad ice, and eedback. I would also like o hank my co-supe iso , A ia Co ´es, o he
insigh and guidance. I was a pleasu e o collabo a e wi h and lea n om you.
I also ex end my since e g a i ude o he s a o he Pablo Tob´on U ibe Hospi al in Medell´ın,
Colombia, whose expe ise and knowledge we e in aluable o he execu ion o his p ojec , as
well as o all he o he esea che s and medical p o essionals who gene ously len me hei ime
and skills. This p ojec would no ha e been wha i is wi hou you help.
Las ly, I would like o hank my amily and lo ed ones o hei unwa e ing suppo , la e-
nigh calls, and expe ad ice. Thei belie in me has mo i a ed me beyond measu e.
I am uly g a e ul o each o you.
i
Con en s
Acknowledgmen s i
Abs ac ix
1 In oduc ion 1
2 Me hodology 5
2.1 Documen S uc u e .................................. 5
2.2 Resea chDesign .................................... 6
2.3 Resea ch Plan pe Resea ch Ques ion . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3.1 Which echnical con ols exis o AI-based sys ems in heal hca e? . . . . 7
2.3.2 How should a la ge-scale AI-based sys em in heal hca e be planned o
ensu e he p i acy o indi iduals? . . . . . . . . . . . . . . . . . . . . . . . 7
2.3.3 How can we gua an ee ha he bene i s o AI-based sys ems in heal hca e
a e balanced wi h igo ous da a p o ec ion s anda ds? . . . . . . . . . . . 8
2.3.4 A e he manda ed s anda ds o da a p o ec ion in AI-based heal hca e
sys ems in Ca alunya su icien o be conside ed a e e en in he ield? . . 8
2.3.5 A e he manda ed s anda ds o da a p o ec ion in AI-based heal hca e
sys ems in Colombia su icien ? . . . . . . . . . . . . . . . . . . . . . . . . 8
ii
Con en s iii
2.3.6 How could Colombia implemen key indings om he case o Ca alunya
in o de o pu sue he de elopmen o AI-based medical applica ions in an
e hical and p i acy-conscious way? . . . . . . . . . . . . . . . . . . . . . . 9
3 Li e a u e Re iew 10
3.1 Pe sonal and Medical Da a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Designing AI Sys ems wi h P i acy in Mind . . . . . . . . . . . . . . . . . . . . . 12
3.2.1 P i acy by Design P inciples . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.2.2 Da aMinimiza ion............................... 15
3.3 P i acy-Enhancing Technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.3.1 End- o-End App oaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.3.2 Rela ed o Da ase Cu a ion . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.3.3 Rela ed o De elopmen and Model T aining . . . . . . . . . . . . . . . . 24
3.3.4 Rela ed o Deploymen . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.4 Da aGo e nance.................................... 28
3.4.1 Documen a ion................................. 29
3.4.2 Ex e nalAudi ing ............................... 34
3.4.3 In e nalAudi ing................................ 37
3.4.4 Da aLi ecycle ................................. 39
3.4.5 Da a Li ecycle Managemen . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4 Regula o y Landscape on Pe sonal Da a P o ec ion in Ca alunya and Colom-
bia 47
4.1 Regula o y Landscape o Ca alunya . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.1.1 Pe sonal Da a Regula ion in Ca alunya . . . . . . . . . . . . . . . . . . . 48
Con en s i
4.1.2 Regula ion Rega ding Medical Reco ds in Ca alunya . . . . . . . . . . . . 49
4.1.3 Regula ion o A i icial In elligence in Ca alunya . . . . . . . . . . . . . . 52
4.2 Regula o y Landscape o Colombia . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2.1 Pe sonal Da a Regula ion in Colombia . . . . . . . . . . . . . . . . . . . . 53
4.2.2 Regula ion Rega ding Medical Reco ds in Colombia . . . . . . . . . . . . 61
4.2.3 Regula ion o A i icial In elligence in Colombia . . . . . . . . . . . . . . . 65
4.3 Legal F amewo k Compa ison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5 P oposal o a P i acy-Awa e Da a Li ecycle and Managemen Plan o Med-
ical Da a, MDLC 74
5.1 TheMDLC ....................................... 74
5.1.1 S age 1: Plan and Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
5.1.2 S age 2: Selec and Collec . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.1.3 S age 3: Analyze, Assu e, and P epa e . . . . . . . . . . . . . . . . . . . . 86
5.1.4 S age 4: Model and Compa e . . . . . . . . . . . . . . . . . . . . . . . . . 90
5.1.5 S age 5: Assess and Sha e . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.1.6 S age6:Deploy................................. 97
5.1.7 S age 7: Dele e o A chi e . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.1.8 In e media e Risk Assessmen (IRA) . . . . . . . . . . . . . . . . . . . . . 99
6 P oposal o an Audi ing F amewo k o MLDC, UMAPER 103
6.1 UMAPER: An Audi ing F amewo k o MDLC P ojec s . . . . . . . . . . . . . . 103
6.1.1 Unde s and: Why is he Audi Necessa y? . . . . . . . . . . . . . . . . . . 105
6.1.2 Map: Wha is he Sys em Being Audi ed? . . . . . . . . . . . . . . . . . . 107
6.1.3 Assessmen : Wha a e he Risks? . . . . . . . . . . . . . . . . . . . . . . . 108

Con en s
6.1.4 Plan: Wha Tes s Will be Conduc ed? When Will hey be Conduc ed? . 110
6.1.5 Execu e: Ca ying Ou he Plan . . . . . . . . . . . . . . . . . . . . . . . 111
6.1.6 Re lec : Audi Re ospec i e . . . . . . . . . . . . . . . . . . . . . . . . . 112
7 Conclusions 113
7.1 Limi a ions and Fu u e Wo k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
Lis o Figu es
3.1 GDPR obliga ions o di e en ypes o pseudonymised and anonymised da a [19] 21
3.2 Fac shee me hodology s eps [50] . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3 O e iew o he SMACTR F amewo k [57] . . . . . . . . . . . . . . . . . . . . . . 38
3.4 Compa ison o da a li ecycles and me hodologies. . . . . . . . . . . . . . . . . . . 42
5.1 P oposal o an I e a i e, Non-Lineal Medical Da a Li ecycle . . . . . . . . . . . 76
5.2 Plan and Design S age Summa y . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.3 Selec and Collec S age Summa y . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.4 Analyze, Assu e, and P epa e S age Summa y . . . . . . . . . . . . . . . . . . . . 87
5.5 Model and Compa e Summa y . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.6 Assess and Sha e Summa y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.7 DeploySumma y.................................... 97
5.8 Dele e o A chi e Summa y . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.9 In e media e Risk Assessmen Summa y . . . . . . . . . . . . . . . . . . . . . . . 101
6.1 O e iew o he UMAPER F amewo k. Double-lined boxes ep esen p ocesses,
egula boxes ep esen asks, blue ep esen s inpu documen s, and o ange ep-
esen sou pu documen s................................ 105
i
Lis o Tables
1.1 Resea chQues ions................................... 3
3.1 Rele ance o P i acy by Design on he GDPR . . . . . . . . . . . . . . . . . . . . 15
3.2 Rele ance o Documen a ion on he GDPR . . . . . . . . . . . . . . . . . . . . . 33
3.3 Rele ance o Documen a ion on he AI Ac . . . . . . . . . . . . . . . . . . . . . 33
4.1 Legisla ion o Medical Reco ds . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.2 Compa ison Be ween Colombia and Ca alunya . . . . . . . . . . . . . . . . . . . 73
5.1 P ojec De ini ion and Scope Deli e ables . . . . . . . . . . . . . . . . . . . . . . 79
5.2 O ganiza ional S uc u e and Policies Deli e ables . . . . . . . . . . . . . . . . . 80
5.3 Founda ions o Responsible Da a Use Deli e ables . . . . . . . . . . . . . . . . . 81
5.4 Ex e nal Communica ion Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . 82
5.5 Da a Selec ion Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
5.6 Da a Collec ion Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.7 Know he Da a Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.8 P epa e he Da a Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
5.9 Compa e Modeling Techniques Deli e ables . . . . . . . . . . . . . . . . . . . . . 91
5.10 Model and Tes Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
ii
Lis o igu es iii
5.11 Assess Compliance Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.12 Assess P ojec Resul s Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.13 Assess Risk Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.14Sha eDeli e ables ................................... 96
5.15DecideDeli e ables................................... 96
5.16 Plan Deploymen Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.17 Conduc Final Assessmen Deli e ables . . . . . . . . . . . . . . . . . . . . . . . 98
5.18 inalize Documen a ion Deli e ables . . . . . . . . . . . . . . . . . . . . . . . . . 99
5.19A chi eDeli e ables .................................. 100
5.20Dele eDeli e ables................................... 101
5.21IRASumma y ..................................... 102
2.2. Resea ch Design 6
Unde s anding hese egula ions is necessa y o p opose managemen amewo ks ha com-
ply wi h in e na ional and local legal equi emen s, which will be ele an o RQ2 and RQ3.
This chap e co e s all he necessa y legal in o ma ion o incula e he egula o y aspec s o
da a p osessing and p o ec ion wi h esponsible sys em design. Al hough special a en ion is
gi en o he legal speci ics o he medical ield, he insigh s o his chap e can be ex ended o
o he domains.
Chap e 5: Based on indings om he li e a u e e iew, his chap e p oposes a p i acy-
cen ic da a li ecycle o medical AI sys ems, add essing RQ2 and RQ3.
Chap e 6: Sugges s a me hodology o in e nal audi ing ha is ailo ed o he con ols
inco po a ed in o he da a li ecycle p oposal. This discussion ies in o RQ2 and RQ3.
Chap e 7: P esen s conclusions, limi a ions, u u e wo k and discussion. Summa izes he
main indings o his mas e s hesis, e isi s he esea ch ques ions (RQ1-RQ6), and p oposes
di ec ions o u u e esea ch. Iden i ies challenges and cons ain s o he p oposals made.
2.2 Resea ch Design
To achie e he goals o his p ojec , we will conduc in-dep h esea ch on cu en da a p o ec ion
egula ions, as well as on echnical, design, and go e nance p o ocols o such sys ems. We will
ake a quali a i e esea ch app oach o acqui e a deepe unde s anding o he s udy a ea, cap u e
con ex ual nuance, and p o ide de ailed insigh s in o hese complex issues.
Du ing he quali a i e esea ch conduc ed, we will ocus on ex ual in o ma ion, in he o m
o academic pape s, o icial guides, and legal ins umen s. We will also p io i ize, when possible,
gi ing domain expe s a oice h ough he inclusion o in e iews. In hese cases, he ole o
he in e iewee and hei ele ance o he opic a hand will be made clea . Howe e , hei
a ilia ions will only be e ealed wi h hei explici consen .

2.3. Resea ch Plan pe Resea ch Ques ion 7
2.3 Resea ch Plan pe Resea ch Ques ion
2.3.1 Which echnical con ols exis o AI-based sys ems in heal hca e?
We will explo e his ques ion du ing ou li e a u e e iew by esea ching a ious echnical con-
ols cu en ly a ailable o AI-based sys ems and o he da a-in ensi e sys ems. We will p i-
ma ily s udy a wide a ay o academic pape s and echnical guides. Combining hese sou ces
will o e a comp ehensi e iew o he cu en echnologies a ailable. This esea ch will in o m
ac i i ies and con ols wi hin he me hodologies ha will be p oposed in his hesis.
2.3.2 How should a la ge-scale AI-based sys em in heal hca e be planned o
ensu e he p i acy o indi iduals?
I is ou in en ion o ga he a wide pe spec i e o he necessa y p ocesses, echniques, and
policies ha would enable he de elopmen o such a sys em. Wi h ha in mind, we will
s udy he amewo ks, me hodologies, and bes p ac ices ha would p o ide he ools needed
o achie e p i acy-conscious medical AI sys ems.
We will app oach his issue om h ee main pe spec i es: Design, echnology and go e -
nance. Toge he , hese h ee aspec s p o ide a holis ic iew o a sys em. Design ocuses on
how sys ems a e concep ualized, s uc u ed and implemen ed o mee equi emen s and gi es
us insigh in o how sys ems may be shaped o espec p i acy inhe en ly. Technology ocuses on
he me hods and ools ha enable sys ems o p ocess da a secu ely while s ill being unc ional,
and i is he backbone o p i acy-p ese ing AI applica ions. Finally, Go e nance add esses he
egula o y and e hical amewo ks ha guide de elopmen and ensu e compliance wi h laws and
in e nal policies. Th ough he s udy o hese h ee dimensions, we s i e o ind in e ac ions
be ween hem ha may p o ide balance ega ding inno a ion, compliance and da a p o ec ion.
Based on he indings o he esea ch we will p opose a da a li ecycle model ha e lec s
on he exis ing amewo ks, managemen models, and o he bes p ac ices. This medical da a
li ecycle could po en ially guide he de elopmen o AI-based applica ions wi h longe li espans
and be e policy compliance in he medical ield.
2.3. Resea ch Plan pe Resea ch Ques ion 8
2.3.3 How can we gua an ee ha he bene i s o AI-based sys ems in heal h-
ca e a e balanced wi h igo ous da a p o ec ion s anda ds?
To add ess his ques ion we will le e age he esea ch conduc ed o o he esea ch ques ions
and del e deepe in o go e nance concep s ega ding isk managemen and impac assessmen .
We will also look in o ex e nal and in e nal audi ing p ac ices ha can se e as ools o iden i y,
moni o and mi iga e isk.
The esea ch conduc ed in his sec ion will be e lec ed on an audi ing and isk managemen
amewo k p oposal ha can be easily in eg a ed wi h he p oposed me hodology esul ing om
he wo k conduc ed o RQ2 (see §2.3.2).
2.3.4 A e he manda ed s anda ds o da a p o ec ion in AI-based heal hca e
sys ems in Ca alunya su icien o be conside ed a e e en in he ield?
In o de o explo e his esea ch ques ion, we will begin by iden i ying he speci ic se o laws
ha go e n pe sonal da a, a i icial in elligence and medical eco ds in Ca alunya. Once he
necessa y legal ins umen s ha e been iden i ied we will ely on expe ’s opinions o asce ain
he comple eness and co e age o hese egula ions, allowing us o conclude abou i s i ness as
a e e en o Colombia.
We will hen go in o de ail ega ding he co e enan s o he egula o y en i onmen o
Ca alunya o each o hese h ee main a eas o in e es . This is done so ha we may hen d aw
compa ison be ween i and he Colombian egula ion.
2.3.5 A e he manda ed s anda ds o da a p o ec ion in AI-based heal hca e
sys ems in Colombia su icien ?
We will p esen a summa y o he cu en legal landscape in he egion and explo e i s su iciency
in e ms o co e age, comple eness and en o ceabili y. The au ho will suppo he analysis wi h
legal esea ch indings and p o ide any di e ences iden i ied by he au ho a e s udying bo h
legal amewo ks.
In addi ion, his sec ion will ea u e in e iews wi h expe s in di e en domains o Colombian
2.3. Resea ch Plan pe Resea ch Ques ion 9
medicine. These in e iews a e conduc ed in o de o iden i y in e es ing in e ac ions o he law
wi h he daily unning o hospi als and how hey a ec managemen ac i i ies and medical
p ac ice. The in e iew also in o ms on gene al eelings owa ds a i icial in elligence in he
medical ield.
In e iews we e conduc ed wi h a Compliance A o ney o he Gene al Sec e a ia and Legal
A ai s o he Pablo Tob´on U ibe Hospi al in Medell´ın, wi h he Head o he Medical Di ision
o he Pablo Tob´on U ibe Hospi al in Medell´ın, and wi h a Colombian Gene al P ac i ione
who holds a mas e ’s deg ee on Da a Science and who is cu en ly a PhD candida e pu suing a
deg ee in a compu a ional biology p og am sha ed by he Uni e si y o Pi sbu gh and Ca negie
Mellon Uni e si y.
2.3.6 How could Colombia implemen key indings om he case o Ca alunya
in o de o pu sue he de elopmen o AI-based medical applica ions
in an e hical and p i acy-conscious way?
F om he insigh s acqui ed du ing he esea ch p oposed o RQ4 and RQ5 (see §2.3.4 and §2.3.5
espec i ely), we will speci y pa icula insigh s om he Eu opean egula o y body ha may
be o immedia e use o Colombia. The conclusions we each on his sec ion will g ea ly depend
on he di e ences ound be ween bo h egula o y amewo ks.
Chap e 3
Li e a u e Re iew
Ox o d Languages de ines bes p ac ices as comme cial o p o essional p ocedu es accep ed o
p esc ibed as being co ec o mos e ec i e. In he case o so wa e de elopmen , bes p ac ices
a e usually he esul o he combined expe ience o skilled so wa e de elope s. These p ac ices
will, on a e age and o e ime, p omo e cha ac e is ics (o non- unc ional equi emen s) in he
de eloped applica ions, such as esilience, usabili y, po abili y, and many o he s.
E hical p inciples o AI-based applica ions a e e y simila o non- unc ional equi emen s.
We can a emp o explo e hese p inciples h ough he same lens so wa e enginee ing has used
o s udy a sys em’s cha ac e is ics ha ela e o i s o e all conduc a he han i s indi idual
ea u es. Fo medical AI-based applica ions, he desi ed ou come o good p ac ices goes beyond
he pu sui o s ong pe o mance, and s i es o achie e adhe ence o e hical AI p inciples.
This Mas e Thesis will ocus mainly on p i acy; howe e , o he cha ac e is ics o esponsible
AI, such as anspa ency o ai ness, could po en ially be explo ed h ough his lens.
Simply de ining p inciples canno gua an ee e hical AI. While i would be ideal i me ely
s a ing ou in en ion o de elop anspa en ,explainable, o ai AI-based applica ions, we e
enough o ensu e ha he de eloped sys em complies wi h hese p inciples, i is ce ainly no he
case. I is necessa y o ind and implemen de elopmen p ac ices ha se ou o gua an ee hese
cha ac e is ics in ou applica ions. This is a lesson we lea n om adi ional so wa e enginee ing,
as he e has been a consis en e o o u n a so wa e p ojec ’s e he eal quali ies—such as
eliabili y, main ainabili y, o compa ibili y—in o conc e e ac ions wi h measu able ou comes.
10
3.1. Pe sonal and Medical Da a 11
In his sec ion, we will de ine pe sonal and medical da a, and s udy how p i acy-p ese ing
AI sys ems can be achie ed om he design, echnical, and da a go e nance pe spec i es. We
will explo e he cu en s a e o P i acy-P ese ing AI (PPAI) and he echniques i p oposes
o p o ec AI sys ems agains model a acks and leakages ha ex ac in o ma ion abou he
aining da a o model pa ame e s [6]. We will desc ibe he design, de elopmen , and go e -
nance p ac ices cu en ly used o p omo e mo e p i acy-o ien ed AI sys ems. We will emphasize
echniques ha ha e been p o en help ul in medical AI-based sys ems and o he high-secu i y
ields, and men ion ele an d awbacks when possible.
3.1 Pe sonal and Medical Da a
Any discussion abou p o ec ing pe sonal and medical da a mus s a wi h a clea de ini ion
o hese e ms. The Eu opean Commission de ines pe sonal da a as “(...) any in o ma ion ha
ela es o an iden i ied o iden i iable li ing indi idual”, o al e na i ely “Di e en pieces o
in o ma ion, which oge he can lead o he iden i ica ion o a pa icula pe son.” [7] As such,
da a ha has been enc yp ed, de-iden i ied, o pseudonymised bu is s ill e-iden i iable emains
pe sonal da a and alls wi hin ou scope o in e es . Howe e , any da a ha has been anonymised
in such a way ha i can no longe be e-iden i ied o de-anonymised is no longe conside ed
pe sonal da a[7].
Some examples o pe sonal da a include:
•Names and su names.
•Home add esses.
•IP add esses.
•Emails and phone numbe s.
•Da a held by a hospi al o doc o , which could uniquely iden i y a pe son.
Medical da a is a pa icula ly sensi i e ype o pe sonal da a ela ed o he pas , p esen ,
and u u e men al o physical heal h o an indi idual, as de ined by Spain’s o ganic law 15/1999,

3.2. Designing AI Sys ems wi h P i acy in Mind 12
on he p o ec ion o pe sonal da a [Ley O g´anica 15/1999, de p o ecci´on de da os de ca ´ac e
pe sonal (Real Dec e o 1720/2007)] [8]. In Reci al 35 o he Gene al Da a P o ec ion Regula-
ion (GDPR) g ea e de ail is p o ided as o which pieces o in o ma ion gene a ed du ing he
p o ision o medical ca e a e conside ed medical da a: [9]
•In o ma ion abou a pe son collec ed du ing egis a ion o medical ca e o du ing medical
ca e.
•Any numbe o symbol assigned o a pe son o uniquely iden i y hem o heal h pu poses.
•Resul s om clinical es s o examina ions o a body pa o bodily subs ance.
•Any in o ma ion de i ed om es ing o examining gene ic da a o biological samples.
•Any in o ma ion on a disease, disabili y, disease isk, medical his o y, clinical ea men s,
o physiological o biomedical s a e o a pa ien , ega dless o sou ce.
3.2 Designing AI Sys ems wi h P i acy in Mind
In his sec ion, we will ocus on how p i acy p ese a ion should be inco po a ed in o he plan-
ning and design s ages, which aim o esponsibly c ea e AI-based applica ions The goal is o
design sys ems ha posi i ely impac socie y wi hou comp omising he undamen al igh o
p i acy. He e, we will g oup design p inciples and ac i i ies ha occu be o e de elopmen
begins, which a e usually mo e abs ac and ypically equi e s akeholde and adminis a i e
in ol emen .
3.2.1 P i acy by Design P inciples
The ollowing sec ion summa izes he concep s ound in he “Guide o P i acy by Design” [10]
w i en by he Spanish Agency o Da a P o ec ion (AEPD, by i s ini ials in Spanish), “P i acy
by Design: The De ini i e Wo kshop. A Fo ewo d by Ann Ca oukian, Ph.D.” [11], and he
Gene al Da a P o ec ion Regula ion.
3.2. Designing AI Sys ems wi h P i acy in Mind 13
The In o ma ion and P i acy Commissione o On a io p oposed he concep o p i acy by
design o e wen y yea s ago. I was i s p esen ed a he 31s In e na ional Con e ence o
Da a P o ec ion and P i acy Commissione s, in e na ionally accep ed a he 32nd In e na ional
Con e ence o Da a P o ec ion and P i acy Commissione s, and subsequen ly adop ed by he
GDPR as a necessa y p ac ice. The GDPR goes in o de ail ega ding p i acy by design in
Reci al 78 and A icle 25. A icle 83 is pa icula ly in e es ing as i s a es ha no adhe ing o
da a p o ec ion by design is a sanc ionable o ence. A lis o he ins ances o he GDPR ha
p o ide a legal basis o p i acy design p inciples can be ound in Table 3.1.
The e m p i acy by design e e s o p o ec ing da a h ough echnology design and aims o
p o ide he use wi h s a e-o - he-a da a p o ec ion. The desi e o p i acy by design eme ges
om he idea ha da a p o ec ion s anda ds a e easie o adhe e o when hey a e in eg a ed
wi h he echnology om i s c ea ion. Howe e , he e is no s ic de ini ion o p i acy by design,
which is e lec ed in he legisla ion, as he GDPR lea es he exac p o ec i e measu es ha
should be aken comple ely open. Al hough pa icula echniques o p i acy p ese a ion a e
men ioned in he law, i does no go in o a le el o de ail ha dic a es he exac measu es a
da a-based p ojec should ake [10]. Ne e heless, se en p i acy-by-design p inciples can be
used o guide he de elopmen o p i acy-conscious AI-based applica ions and o he da a-based
applica ions om a mo e abs ac le el [11], a ac we will examine he e. The pa icula p i acy-
p ese ing echniques men ioned in he GDPR will be explo ed in §3.3; in his sec ion, we will
e iew each o he se en p i acy-by-design p inciples.
P oac i e no Reac i e; P e en a i e no Remedial: Possible e en s ha may a ec
p i acy should be an icipa ed. Risks ega ding people’s igh s and eedoms should be iden i ied
and minimized du ing he design s age o mi iga e possible damages. Addi ionally, a sys ema ic
app oach o ea ly de ec ion o po en ial issues du ing and a e deploymen , as well as a clea ly
de ined esponsibili y dis ibu ion, should ensu e ha e e y indi idual wo king on he p ojec
is awa e o hei obliga ions ega ding use p i acy.
P i acy as he De aul Se ing: The de aul con igu a ion o any applica ion, sys em,
p oduc , o se ice shall be de ined om he design s age o be as espec ul o people’s p i acy
as possible. This ensu es ha e en i an indi idual akes no ac ion o con igu e he sys em, hei
3.2. Designing AI Sys ems wi h P i acy in Mind 14
p i acy will be gua an eed. This implies he need o da a minimiza ion h ough all s ages o
da a handling. Fo example, by de aul he e should be well-de ined and limi ed c i e ia o da a
collec ion, he use o pe sonal da a should be limi ed o he pu poses o which i was collec ed,
hese pu poses should ha e a legi ima e basis, access should be es ic ed o a need- o-know
basis, and so on.
P i acy Embedded in o Design: P i acy should no be an add-on o a se ice o sys em,
i should be an essen ial componen o he co e unc ionali y being cons uc ed and deli e ed.
P i acy is no mean o be acked on a he end o p oduc ion, i mus be a non- unc ional
equi emen om he p ojec ’s concep ion.
Full Func ionali y; Posi i e-Sum, no Ze o-Sum: Unnecessa y ade-o s should be
a oided. P i acy by design seeks o accommoda e, no hinde , legi ima e in e es s and objec-
i es while p ese ing p i acy. Ins ead o adop ing a p i acy s. secu i y o p i acy s. u ili y
men al amewo k, p i acy by design aims o demons a e ha all pa ies in ol ed can achie e
hei goals, eaching a balance ha enables e e yone o “win”. This equi es de elope s, o -
ganiza ions, and esea che s o acknowledge ha mul iple legi ima e in e es s may coexis and
should be balanced acco dingly.
End- o-End Secu i y – Full Li ecycle P o ec ion: P i acy is a conce n a all da a
li ecycle s ages. I mus be planned e en be o e he i s piece o in o ma ion is collec ed and
should ex end un il he da a is ul ima ely dele ed. To adequa ely p ese e p i acy du ing each
s age o he da a li ecycle, i is necessa y o analyse e e y single ope a ion equi ed o each
s age o da a p ocessing, and implemen he mos app op ia e p o ec i e measu es a each s ep.
Visibili y and T anspa ency – Keep i Open: This p inciple aims o allow independen
e i ica ion o he ul ilmen o a gi en sys em’s p omised da a sa e y gua an ees. T anspa ency
in da a ea men is a undamen al pilla in demons a ing he diligence wi h which da a p o-
ec ion measu es a e unde aken o bo h he compe en au ho i ies and he indi iduals o whom
he da a pe ains. This p inciple aligns wi h Reci al 39 o he GDPR, which s a es ha na -
u al pe sons should be in o med abou he collec ion, use, consul a ion, and handling o hei
pe sonal da a.
3.2. Designing AI Sys ems wi h P i acy in Mind 15
GDPR Sec ion Summa y o Rele an Aspec s
Reci al 78 S a es ha in o de o demons a e compliance wi h he
egula ion ha dic a es he igh s and eedoms o na u al pe sons
wi h ega d o hei pe sonal da a a e being me , he da a
con olle should be able o demons a e ha hey ha e adap ed
in e nal policies and echnological and o ganiza ional measu es
ha adhe e o he p inciples o da a p o ec ion by design.
Reci al 108 S a es ha when ans e ing da a o a coun y ha does no
ha e he same p i acy s anda ds as he Eu opean Union, he
con olle o p ocesso o he da a will be obliga ed o compensa e
o ha lack o p o ec ion by implemen ing he app op ia e
sa egua ds, among which is he adhe ence o he p i acy by design
and de aul p inciples.
A icle 25 S a es ha da a con olle s should implemen app op ia e
echnical and o ganiza ional measu es o p omo e da a p o ec ion
p inciples. By de aul , con olle s mus also implemen
app op ia e echnical and o ganiza ional measu es o ensu e
p i acy. Con olle s may use an app o ed ce i ica ion mechanism
as an elemen o demons a e hei compliance wi h he
obliga ions de ailed in his a icle.
A icle 47 S a es ha he compe en supe iso y au ho i y will app o e
legally binding co po a e ules, and hese ha e minimum equi ed
speci ica ions o he sys em, among which is he applica ion o
gene al da a p o ec ion p inciples, including da a p o ec ion by
design and by de aul , and da a minimiza ion.
Table 3.1: Rele ance o P i acy by Design on he GDPR
Respec o Use P i acy – Keep i Use -Cen ic: P i acy by design equi es ha a -
chi ec s, ope a o s, and de elope s p io i ize he igh s and eedoms o he indi iduals o whom
he da a pe ains. This includes o e ing measu es such as p i acy- espec ing de aul s, app o-
p ia e no ice o da a ea men , and o he use - iendly op ions ha empowe use s ega ding
hei in o ma ional sel -de e mina ion.
3.2.2 Da a Minimiza ion
Da a minimiza ion is a design s a egy o p i acy ha aims o educe he amoun o da a
collec ed and p ocessed o he smalles possible amoun . In his way, i in ends o a oid he
p ocessing o unnecessa y da a o he p ocessing o da a o unjus i ied pu poses. Da a mini-
miza ion is one o he concep s om p i acy by design ha needs o be aken in o accoun a
3.3. P i acy-Enhancing Technologies 22
public. Local di e en ial p i acy, on he o he hand, add esses scena ios whe e he collec o is
no us wo hy, allowing use s o pe u b hei da a locally on hei own de ices [23].
The e a e many me hods o in oduce di e en ial noise in o a da ase . Some o he mos
common include he Laplace mechanism and he Gaussian mechanism, which a e ypically used
in cen alized di e en ial p i acy. O he mechanisms like gene al andomized esponse a e mos
commonly applied in local di e en ial p i acy. [23]
Di e en ial p i acy coun s among i s s eng hs i s s ong ma hema ical de ini ion, and i s
academic backing, as i has been well-s udied o e he yea s. The ma hema ical de ini ion o DP
also p o ides a quan i a i e no ion o he pe o mance o a gi en model in ega ds o i s p i acy
p o ec ion [24]. Addi ionally, compa ed o o he p i acy-p ese ing echniques like homomo phic
enc yp ion, i has a ela i ely small impac on pe o mance and can be implemen ed ela i ely
quickly.
Howe e , di e en ial p i acy has d awbacks. Fo ins ance, he p o ec ion gua an ees o e ed
by di e en ial p i acy o g oups o use s diminish exponen ially as he g oup size inc eases.
Ne e heless, a la ge g oup o people sha ing he same in o ma ion does no make i any less
sensi i e [25]. This d awback is pa icula ly p onounced in he case o medical da a, as e en
i a la ge g oup o pa ien s sha e he same diagnosis, he da a s ill needs o be highly secu e.
Addi ionally, i is essen ial o conside how adding noise may lead o a loss o accu acy. This is a
common phenomenon o p i acy-p ese ing echniques, usually e e ed o as he p i acy-u ili y
ade-o .
3.3.2.3 Da ase Sani iza ion
Da a sani iza ion is a p ocess h ough which da a is delibe a ely and pe manen ly emo ed om
a sou ce. I is designed o be an i e e sible p ocess, a e which no usable esidual in o ma ion
ega ding he sani ized da a emains, making i un eco e able [26]. The e m da ase sani iza ion
is used mo e o less in e changeably o e e o wo di e en ac i i ies: emo ing in o ma ion
om a physical de ice ha will be disposed o o eused, and emo ing da a om a da ase o
make i mo e sui able o aining, bo h o which a e ele an o p i acy p ese a ion.

3.3. P i acy-Enhancing Technologies 23
When pe o ming da a sani iza ion on a memo y de ice o euse o disposal, h ee main
me hods exis : physical des uc ion o he memo y de ice, c yp og aphic e asu e, and da a
e asu e [26]. On he o he hand, o da ase sani iza ion on a da a sou ce in ended o model
aining o da a sha ing, a common app oach in ol es using a pa se -based algo i hm ollowed
by a classi ie model ha ags each uni o in o ma ion based on de ined pa e ns o ules lea ned
om da a ha has been manually agged as sensi i e o no sensi i e [6].
The main ad an age o da a sani iza ion is ha da a is no longe being s o ed in any capaci y
and canno be misused o leaked. I a physical memo y de ice is o be disposed o o epu posed
o ano he use, all in o ma ion should be en i ely emo ed, pa icula ly i he da a is sensi i e.
This would undoub edly help a oid si ua ions such as he Maine 2021 da a b each, in which
Heal hReach Communi y Heal h Cen e s disco e ed ha a wo ke a a hi d-pa y da a s o age
acili y had imp ope ly disposed o ha d d i es con aining pa ien da a, esul ing in he medical
da a o o e 116,898 pa ien s being leaked [27].
The ole o da a sani iza ion in p i acy-p ese ing AI is o pe manen ly dele e sensi i e
in o ma ion om a da ase used o aining, so ha i may no be misused o leaked. The e
has been academic in e es in de ining me ics o he le el o sani iza ion and he u ili y o a
sani ized da ase , which helps quan i y he p i acy gene a ed by he p ocess o sani iza ion [28].
This is c ucial, as p i acy canno be ea ed as an unquan i iable quali y o a sys em.
Un o una elly, using da a sani a ion o da ase s in model aining has signi ican d awbacks.
Fo s a e s, i in ol es a subs an ial p i acy-u ili y ade-o . This disp opo iona ely a ec s
he medical ield and o he high-secu i y sec o s, as he use ul in o ma ion o model aining,
such as diagnoses o medical his o ies, is o en highly sensi i e. Fu he mo e, a cu en issue
wi h medical da a, a leas in Spain, as s a ed by D . Joan Guanyabens, is he lack o s uc u e
in he da a. This lack o s uc u e makes i ha de o s anda d da a sani iza ion me hods o
wo k e ec i ely, as hey a e be e sui ed o well- o ma ed da a [6]. Fu he mo e, iden i ying
sensi i e in o ma ion in a da a co pus is bo h challenging and con ex -dependen . This issue
becomes e en mo e ele an wi h he ad en o LLM-powe ed medical applica ions [29]. These
models a e ained on ex -based examples whe e i is nea ly impossible o de e mine which
sec ions o ex should be conside ed sensi i e, as his ask is con ex -dependen and sensi i e
3.3. P i acy-Enhancing Technologies 24
sec ions o en lack clea ly de ined bounda ies, making da a sani iza ion less e ec i e.
I is impo an o no e wo addi ional issues. The i s is ela ed o he gua an ee o ull
dele ion o all da a ela ed o an indi idual, which is echnically challenging. Mal e Schwa zkop
e al. add ess his subjec in hei pape i led Posi ion: GDPR Compliance by Cons uc ion
[30]. The second issue a ises when an indi idual eques s o dele e hei da a a e a model has
been ained. The ac ha he indi idual’s in o ma ion was used du ing aining causes hei
da a o pe sis in some ways e en a e i has been dele ed om he aining da ase , as well as
making hem ulne able o MIA a acks. In heo y, he model would need o be e ained o
elimina e he in luence o his da a poin . Howe e , in some cases, i is no easible o e ain
he en i e model due o compu a ional cos and ime cons ain s. This has led o an eme ging
b anch o esea ch called “machine unlea ning,” which aims o add ess his issue by enabling a
model o emo e he in luence a pa icula da a poin has had on i [31]. This ield is apidly
gaining ele ance, pa icula ly due o he GDPR’s policy ega ding he igh o be o go en.
3.3.3 Rela ed o De elopmen and Model T aining
3.3.3.1 Fede a ed Lea ning
Fede a ed lea ning is a collabo a i e, p i acy-p ese ing pa adigm [32] ha enables mul iple
de ices o join ly ain a model wi hou sha ing hei da a wi h a cen al se e [33]. Fede a ed
lea ning ypically in ol es he dis ibu ed aining o deep neu al ne wo ks (DNNs), al hough
o he models, such as decision ees [34] [35] and suppo ec o machines [36], ha e also been
success ully implemen ed using his echnique.
The main ad an age o ede a ed lea ning is ha i elimina es he need o da a sha ing. This
is bene icial no only because i p ese es p i acy bu also because i acili a es esea ch and dis-
co e y by bypassing he usual bu eauc a ic p ocesses associa ed wi h da a sha ing, pa icula ly
in scena ios whe e hospi als sha e pa ien da a o model aining and esea ch. Addi ionally,
keeping he da a wi hin he hospi al helps pa ien s main ain con ol o e hei in o ma ion, as
he da a is no exposed o ex e nal pa ies. This also makes i mo e challenging o ex e nal
en i ies o in inge upon pa ien s’ igh s conce ning hei da a.
3.3. P i acy-Enhancing Technologies 25
Howe e , i is impo an o no e ha echniques like ede a ed lea ning (FL) may lead o
liabili y de lec ion. Decen alizing da a p ocessing complica es use s’ awa eness o how hei
da a is being used and could esul in use s being classi ied as con olle s o p ocesso s o hei
own da a unde laws like he GDPR [33]. While hospi als gene ally ha e he in as uc u e and
esou ces o manage hese esponsibili ies e ec i ely, he si ua ion is di e en when he hospi al
is no p esen o ac as an in e media y. Fo ins ance, his issue a ises when wea able de ices a e
used as a da a sou ce o ede a ed lea ning, which is a g owing end [37]. In such cases, pa ien s
may ind hemsel es a a disad an age, as hey migh inad e en ly assume accoun abili y o
da a p ocessing wi hou he expe ise o esou ces o ad oca e o hemsel es success ully.
3.3.3.2 Enc yp ion
The GDPR de ines enc yp ion as a p ocess ha “con e s clea ex in o a hashed code using a
key.” This p ocess makes he da a accessible only o pa ies ha possess a key. The p ocess can
be e y simple, such as shi ing e e y cha ac e o i s immedia e p edecesso in ASCII code, o
e y complex, as in he commonly used 3DES o AES enc yp ion s anda ds.
Da a enc yp ion helps p o ec p i acy and de ends agains cybe secu i y h ea s. I is a ool
o p ese ing he con iden iali y and in eg i y o digi al in o ma ion [38]. HIPAA egula ions
explici ly equi e enc yp ion o medical da a, and he GDPR ad ises i s use o pe sonal da a,
aking in o accoun he “ isk o a ying likelihood and se e i y o he igh s and eedoms o
na u al pe sons” (A . 32), which, in he case o medical da a, a e consis en ly high.
Enc yp ion is usually incompa ible wi h model aining, as mos adi ional enc yp ion s an-
da ds lack ope abili y [39]. This is why da a mus be dec yp ed be o e i can be p ocessed. This
lea es he in o ma ion ulne able o de elope s, as hey will handle unenc yp ed da a du ing
some s ages o he de elopmen p ocess. Homomo phic enc yp ion eme ges as a po en ial solu-
ion o his issue.
Homomo phic enc yp ion was i s p oposed in 1978 by Ri es , Adleman, and De ouzos.
Howe e , i was no implemen ed un il 2009, when C aig Gen y’s ully homomo phic en-
c yp ion (FHE) scheme was published. The e a e di e en le els o homomo phic enc yp ion
3.3. P i acy-Enhancing Technologies 26
which a y in s ic ness and e sa ili y. FHE is he mos e sa ile, allowing compu a ion o e
ully enc yp ed da a and e u ning ully enc yp ed ou pu s so ha only he owne o he de-
c yp ion key may e eal he esul s o he compu a ion. This echnique p ese es p i acy and
secu i y h oughou he aining p ocess. Addi ionally, i educes he po en ial damage caused
by a da a b each, as all in o ma ion emains unin elligible.
Pa ially homomo phic enc yp ion (PHE) allows ei he addi ion o mul iplica ion
while enc yp ed. Fo example, RSA suppo s mul iplica ion, and Paillie suppo s addi ion.
Meanwhile, somewha homomo phic enc yp ion (SHE) suppo s a limi ed numbe o op-
e a ions wi hou c ea ing e o s bu does no suppo all ope a ions as FHE does. Fo example,
BGN is a ype o SHE ha allows bo h addi ion and mul iplica ion [40].
Enc yp ion is a powe ul ool o main aining con iden iali y and p o ec ing p i acy; how-
e e , i has i s pi alls. To begin, i is impo an o no e ha di e en enc yp ion s anda ds
a y in how challenging hey a e o b eak. Fo ins ance, 3DES eme ged as a successo o DES,
which, wi h only a 56-bi key leng h, has become easily b eakable in ou cu en echnological
en i onmen [38], and cu en ad ances on quan um compu ing ha e caused some un es ega d-
ing he sa e y o cu en enc yp ion s anda ds. Addi ionally, he e is he p e iously men ioned
issue o adi ional enc yp ion’s lack o compa ibili y wi h compu a ion.
Al hough FHE sol es he issue o ope abili y and would be an excellen ool o p ese e p i-
acy h oughou he da a li e cycle, i has a signi ican compu a ional o e head, which g ea ly
limi s i s p ac ical use [24]. While esea che s a e ac i ely wo king o make i less compu a-
ionally axing [40], he o e head emains subs an ial enough o p e en i om being widely
adop ed.
Fu he mo e, enc yp ion as a whole elies hea ily on adequa e key managemen o ul ill i s
pu pose o p o ec ing da a. I i is ela i ely easy o acqui e he dec yp ion key h ough illegi -
ima e means, enc yp ion becomes a less e ec i e. In addi ion o he possibili y o malicious
in en leading o he he o a key, he e is also he need o conside he consequences o losing a
key. Un o eseen si ua ions, such as se e damage, could esul in he loss o he key, ende ing
he enc yp ed da a inaccessible. In he con ex o medical da a, his is especially conce ning,
3.3. P i acy-Enhancing Technologies 27
as being unable o access clinical eco ds would immedia ely hinde pa ien ca e. Al hough he
loss o medical da a o esea ch and disco e y would no ha e such a de as a ing impac , i
would s ill be a om ideal. [38]
3.3.3.3 Di e en ial P i acy
In he con ex o model aining, di e en ial p i acy in ol es adding a small amoun o noise
du ing model aining [24]. In he case o ede a ed lea ning, in e media e model upda es a e
pe o med locally, and noise is added du ing he agg ega ion o hese upda es [33] o conceal he
con ibu ions o indi idual pa ies o he da ase used o aining.
Di e en ial p i acy is pa icula ly e ec i e agains Membe ship In e ence A acks (MIA).
The mos common MIA o ma in ol es de e mining whe he a da a eco d was included in
he model’s aining da ase , gi en a da a eco d and black-box access o a model. This can
be achie ed by ad e sa ially using a machine lea ning model ained o ecognize di e ences
in he a ge model’s p edic ions on inpu s ha we e pa o he aining da ase and hose
ha we e no [41]. Mo e sophis ica ed and accu a e al e na i es, such as he Likelihood Ra io
A ack, also exis [42]. Al hough he e a e no con i med la ge-scale ins ances o MIA, ei he
because hey ha e no ye occu ed o ha e gone un epo ed, p oo -o -concep a acks as hose
men ioned abo e ha e been e y success ul. Thus, he e is eason o accoun o hem when
planning p i acy s a egies.
The ad an ages and disad an ages o DP du ing model aining a e he same as hose dis-
cussed o local and cen alized di e en ial p i acy du ing da ase cu a ion. Howe e , he e is
one mo e impo an de ail: di e en ial p i acy impac s model pe o mance, and i s impac is no
uni o m ac oss all classes and g oups. When DP is used du ing model aining accu acy d ops
signi ican ly mo e o unde ep esen ed classes and subg oups [43]. This is pa icula ly ouble-
some in heal hca e, whe e a ailable da ase s a e disp opo iona ely whi e and male [44]. This
aises immedia e conce ns ega ding ai ness, as any un ai ness in he o iginal model would only
be exace ba ed by di e en ial p i acy. Ongoing esea ch aiming o mi iga e his cha ac e is ic
o DP has sugges ed ha local di e en ial p i acy is less p one o his beha io [45].

3.4. Da a Go e nance 28
3.3.4 Rela ed o Deploymen
3.3.4.1 Secu e and A ailable In e aces
I is impo an o emembe ha s anda d da a secu i y p ac ices should no be neglec ed.
The p e iously men ioned bes p ac ices o p i acy in a i icial in elligence complemen any
ac i i ies al eady in place o inc ease a ailabili y, in eg i y, and con iden iali y.
I is necessa y o ensu e ha he in e aces h ough which medical p o essionals, esea che s,
o e en pa ien s in e ac wi h any po en ial models mee da a sa e y s anda ds. Addi ionally,
he high a ailabili y o any sys em deployed as a diagnos ic o a en ion aid is a c i ical ea-
u e, as p olonged down imes (and, depending on he sys em’s c i icali y, e en sho ones) a e
inadmissible.
Nume ous ac i i ies can be pe o med du ing all p ojec s ages o achie e his goal, bu hese
all ou side he scope o his mas e ’s hesis and will no be discussed. Howe e , guidance can
be ound in he ISO guidelines, pa icula ly ISO 27001 on In o ma ion Secu i y Managemen
Sys ems (ISMS), ISO 25010 on Sys em and So wa e Quali y Requi emen s and E alua ion
(SQuaRE), and ISO 15408 on Common C i e ia o IT Secu i y E alua ion.
3.4 Da a Go e nance
The ISO 3700 S anda d de ines go e nance as “ he sys em by which he whole o ganiza ion is
di ec ed, con olled, and held accoun able o achie e i s co e pu pose o e he long e m.” Da a
go e nance e e s o he exe cise o au ho i y and con ol o e he managemen o da a. Ab aham
e al. [46] expand on his de ini ion a e e iewing 145 scien i ic a icles on he opic, de ining
i as “a c oss- unc ional amewo k o managing da a as a s a egic en e p ise asse . In doing
so, da a go e nance speci ies decision igh s and accoun abili ies o an o ganiza ion’s decision-
making abou i s da a. Fu he mo e, da a go e nance o malizes da a policies, s anda ds, and
p ocedu es and moni o s compliance.”
Da a go e nance is a ans e sal conce n ha p o ides a amewo k o managing a da a-
based applica ion h oughou he en i e da a/model li ecycle. This sec ion will co e he a ail-
3.4. Da a Go e nance 29
able accoun abili y mechanisms o uphold policies, p ocedu es, and egula ions.
3.4.1 Documen a ion
Documen a ion o da ase s, models, and sys ems has been a he o e on o discussions ega d-
ing accoun abili y and anspa ency in he ield o a i icial in elligence. Addi ionally, i aligns
wi h he Visibili y and T anspa ency P i acy by Design p inciple. App op ia e eco d-keeping
has also become essen ial in ligh o egula ions like he GDPR and he AI Ac , as shown in
Tables 3.2 and 3.3, espec i ely. These egula ions, howe e , do no p opose o en o ce any
speci ic empla e o sys em documen a ion.
The e ha e been a ious p oposals ega ding which aspec s o an AI-based sys em should be
documen ed and how hey should be documen ed. Among hese p oposals, h ee majo con i-
bu ions s and ou : Da ashee s o Da ase s, Google’s Model Ca ds o Model Repo ing, and
IBM’s Fac Shee s o AI Sys ems. The medical ield has also made s ides owa d anspa en
epo ing o s a is ical models, which can se e as a aluable esou ce o adap ing comme cial
AI p oposals o he medical domain.
3.4.1.1 Da ashee s o Da ase s
Da ashee s o Da ase s we e p oposed in an a icle by he same name in 2018 [47]. They
eme ged as a s anda dized p ocess o documen ing machine lea ning da ase s, inspi ed by he
elec onics indus y’s da ashee s. A da ashee is mean o documen a da ase ’s mo i a ion,
composi ion, collec ion p ac ices, and ecommended uses, among o he aspec s, in an e o o
inc ease anspa ency and accoun abili y wi hin he machine lea ning communi y. [48]
The au ho s p opose a lis o ques ions ela ed o each s age in he da a li ecycle o b ing
o h he necessa y in o ma ion o ill ou a da ashee . These ques ions also encou age da ase
c ea o s o e lec on he p ocesses o c ea ing, p ocessing, and dis ibu ing da ase s, while
helping da ase consume s make in o med decisions abou he p ac ical use o a gi en da ase .
[47]
Da ashee s also p o ide alue o policymake s, indi iduals whose da a is included in a
3.4. Da a Go e nance 30
da ase , esea che s, consume ad oca es, and mo e. In he medical ield, i is pa icula ly
in e es ing o conside he impac a da ashee can ha e on indi iduals whose da a has been
added o a da ase , as i would allow hem o make an in o med choice ega ding whe he hey
wan o be pa o i o i hey would p e e o e oke hei consen . Ano he no able ad an age
o da ashee s in he con ex o heal hca e, whe e sha ing da ase s is challenging, is ha hey
would allow di e en hospi als o ec ea e da ase s wi h simila cha ac e is ics wi hin hei own
ins i u ions, enabling hem o po en ially eplica e he esul s o AI-based solu ions ha ha e
shown p omising ou comes elsewhe e. [48]
One o he main challenges o he Da ashee s o Da ase s p oposal is he need o da ase
c ea o s o adap he ques ions and wo k low p oposed o be e i hei ield, o ganiza ional
s uc u e, and wo k lows. Mo eo e , he p ocess o c ea ing a da ashee imposes o e head on
da ase c ea o s. Al hough i is clea ha da ashee s alone will no su ice o comple ely sol e
o mi iga e all bias o isks, i is widely accep ed ha he bene i s o he machine lea ning
communi y ou weigh he cos s o de eloping da ashee s. Among o he ad an ages, he use o
da ashee s can help da ase c ea o s and da ase use s de ec po en ial biases in he aining
da a o inconsis encies be ween he aining da a and he expec ed eal-wo ld da a, which could
hinde model pe o mance. [48]
3.4.1.2 Model Ca ds o Model Repo ing
Model ca ds a e Google’s p oposal o documen a ion ha communica es he pe o mance cha -
ac e is ics o a ained ML o AI model. Model ca ds can in o m use s abou wha a machine
lea ning sys em is capable o doing, i s limi a ions, he ypes o e o s i migh make, and any
addi ional s eps ha could c ea e ai e ou comes. They a e designed wi h human-cen ic mod-
els in mind and aim o inc ease ai ness by encou aging popula ion-based epo ing o esul s
h ough in e sec ional analysis. This analysis con as s he esul s o di e en aces, gende s,
ages, and demog aphics in o de o p o ide an o e iew o how he sys em’s beha io migh
di e depending on cul u al, demog aphic, o gene a ional g oups. Slicing he e alua ion ac oss
hese di e en g oups helps highligh e o s ha disp opo iona ely a ec a speci ic subse o
people. [49]
3.4. Da a Go e nance 31
3.4.1.3 Fac Shee s
Fac Shee s a e IBM’s p oposal o AI sys em documen a ion [50]. They ake a b oade app oach
han Da ashee s and Model Ca ds and a emp o documen he en i e sys em, which may be
composed o many models ained on many da ase s. They aim o inc ease us by deli e ing
clea eco ds o he de elopmen and deploymen o models wi hin sys ems. This solu ion cen es
on business applica ions o AI; howe e , i is gene al enough o be applied o AI sys ems ou side
o his se ing.
The gene ali y o Fac Shee s is one o i s co e ea u es. Fac Shee s om di e en domains
and model ypes will ha e common elemen s bu will mos likely con ain di e en in o ma ion
and le els o de ail. Fac Shee s a e no p oposed as a one-size- i s-all solu ion. They will look
di e en depending on he ype o AI sys em hey pe ain o, who will consume he Fac Shee ,
and o wha pu pose. The in o ma ion p esen ed in hem will a y acco dingly.
In he same a icle in which Fac Shee s a e p oposed, a me hodology o c ea ing hem is
also in oduced. I is an i e a i e app oach wi h se en s eps, shown in Figu e 3.2. In s ep one,
he Fac Shee s eam in e iews he expec ed consume s o he Fac Shee . S ep wo in ol es
in e iewing he Fac Shee p oduce s. In s ep h ee, using he in o ma ion acqui ed in s eps
one and wo, he Fac Shee s eam p oduces a Fac Shee Templa e o he speci ic sys em being
documen ed. In s ep ou , he Fac Shee s eam ills ou he Fac Shee empla e o assess i .
Al hough coope a ion om hose di ec ly in ol ed in he sys em’s de elopmen is bene icial o
s ep i e, i is no necessa y. This is because, in s ep six, he p oduce s o he sys em will ill
ou he Fac Shee . In s ep se en, he Fac Shee s eam epea s he p ocess o inc ease co e age
and alue by de ising empla es o o he audiences and pu poses.
3.4.1.4 The TRIPOD S a emen and TRIPOD + AI
The epo ing and documen a ion s a egies we ha e p e iously men ioned o igina e om he
so wa e indus y. The TRIPOD s a emen [51], in oduced by he medical ield in 2015, aimed
o p omo e p ope epo ing in s udies de eloping o alida ing mul i a iable p edic ion models
o indi idual p ognosis o diagnosis. Howe e , wi h ad ances in a i icial in elligence (AI)
3.4. Da a Go e nance 38
Figu e 3.3: O e iew o he SMACTR F amewo k [57]
3.4.3.2 The SMACTR F amewo k
The Google Responsible AI Impac Lab has also p oposed an end- o-end amewo k o in e nal
algo i hmic audi ing, he SMACTR amewo k. This amewo k inco po a es lessons om au-
di ing p ac ices in o he sa e y-c i ical and egula ed indus ies, such as ae ospace and medical
de ice design and manu ac u ing. These indus ies ha e a long his o y o audi able p ocesses
and design con ols ha ha e helped imp o e sa e y.
The SMACTR amewo k comp ises i e s ages plus a pos -audi s age, as shown in Figu e
3.3, aken om he o iginal pape [57]. I aims o c ea e in e nal audi s ha p oduce a ail
o documen a ion a each s age o de elopmen and enable c i ical e lec ion on he sys em’s
impac , which can also se e as in e nal educa ional ma e ial on e hical awa eness.
The goal o he scoping s age cla i ies he audi ’s objec i e. Audi o s e iew he sys em’s
mo i a ions and in ended impac o an icipa e po en ial sou ces o ha m and social impac .
The mapping s age aims o e iew he cu en s a e o he sys em. This includes sea ching
o in e nal s akeholde s and ele an collabo a o s. Ano he impo an aspec o his s age is
ha Failu e Modes and E ec s Analysis (FMEA) will be s a ed a his s age and upda ed and
expanded h ough he ollowing s ages.
In he a i ac collec ion s age, audi o s se ou o iden i y and collec all he necessa y
documen a ion om he de elopmen p ocess o p io i ize es ing. I necessa y, he audi o may
en o ce e oac i e documen a ion o documen some p ocesses hemsel es, checking he esul s
wi h he de elopmen eam. The ou come o his s age includes he da ashee s and model ca ds,

3.4. Da a Go e nance 39
as well as a design checklis ha keeps ack o he a ailabili y o all he expec ed documen a ion.
Du ing he es ing s age he audi ing eam execu es a se ies o es s o assess he sys em’s
le el o compliance wi h he o ganiza ion’s e hical alues. In his s age, he e will be a iabili y
in he app oach aken by he audi ing eam o conduc es s, as hey will depend on he o gani-
za ional con ex . This pape ’s au ho s p opose he use o ad e sa ial es ing by a emp ing o
simula e a hos ile ac o ; howe e , hey do no p o ide mo e de ail. The main ou pu o his s age
is he e hical isk analysis cha , in which he impo ance o each isk is de ined in p opo ion
o i s se e i y and likelihood.
The e lec ion s age is ese ed o analyzing he esul s o he es s in he con ex o he
e hical expec a ions de ined in he scoping s age. The FMEA is inalized in his s age, and
a i ic s such as a mi iga ion plan can be p oposed, oge he wi h a design his o y ile ha
con ains all he documen a ion ela ed o p e ious s ages, and inally, a summa y o he audi .
The pos -audi s age is conce ned wi h keeping ack o he implemen a ion o ac ion
plans p oposed by he audi ing eam, as well as making inal decisions ega ding he p oposed
AI-based applica ion, e en o he ex en o deciding o ully sc ap he p ojec .
This amewo k is e y in o ma i e as an ou line o in e nal audi ing o AI, bu i is o e all
a he gene al. This is de ini ely an in eded ea u e o he amewo k, as i is mean o be able
o pe o m in mul iple domains.
3.4.4 Da a Li ecycle
Da a li ecycles a e a ool o da a managemen , as hey p o ide a s uc u ed amewo k ha
can guide da a managemen p ac ices. O acle de ines da a managemen as collec ing, keeping,
and using da a secu ely, e icien ly and cos -e ec i ely [58]. In many ways, a da a managemen
s a egy es ablishes he ounda ion upon which a da a p ojec , such as a medical AI applica ion,
will be buil . P ope da a managemen has a wide a ie y o bene i s, bu in sho , i s unda-
men al ole is ensu ing ha da a is indable, accessible, in e ope able, and eusable [59]. These
ou cha ac e is ics a e usually e e ed o as he FAIR p inciples, o hei ini ials [60].
P ope da a managemen can also enhance collabo a ion by making da a clea and accessible
3.4. Da a Go e nance 40
while p e en ing he da a en opy o which digi al in o ma ion is o en p one. Mo eo e , jou -
nals and unding bodies inc easingly equi e open da a in academic esea ch, making adequa e
managemen essen ial o ensu e i can be unde s ood and eused. In essence, e ec i e da a
managemen ensu es ha da a emains accessible and meaning ul o yea s o come. In he case
o mos p ojec s by “yea s o come”, we e e o a decade o wo, bu in heal hca e, i can span
an en i e li e ime and mo e. Fo each pa ien , he e is an en i e li e’s wo h o da a ha needs
o be accessible, unde s andable, and secu e o p ope ly suppo hem as a pa ien a any gi en
poin in hei li e. This makes adequa e da a managemen a co e conce n in he de elopmen o
da a-based medical applica ions. [59]
Da a li ecycles a e abs ac ions o he a ious s ages da a unde goes, om i s c ea ion o
cap u e o i s a chi al o dele ion. As men ioned, hey p o ide a s uc u ed amewo k o man-
aging da a h oughou i s exis ence, ensu ing desi ed cha ac e is ics such as e iciency, quali y,
and esponsible use. The s ages commonly iden i ied in gene al da a li ecycle models include
da a collec ion, s o age, p ocessing, analysis, and isualizing o in e p e ing. Each s age p esen s
unique challenges and oppo uni ies, pa icula ly conce ning da a quali y, secu i y, and accessi-
bili y. [61]
The speci ic s ages and cha ac e is ics emphasized in a da a li ecycle o en depend on he
ype o da a and i s in ended pu pose. Fo ins ance, esea ch-o ien ed da a li ecycles di e
signi ican ly om hose designed o business applica ions. In he case o medical da a, he e
is a clea need o heigh ened anspa ency, e hics, and accoun abili y h oughou he da a’s
li espan.
To he bes o he au ho ’s knowledge, as o Decembe 2024, no da a li ecycle model explici ly
ailo ed o heal hca e pu poses exis . To e i y his, a e iew o se e al academic da abases,
including PubMed, Else ie , IEEE, and he ACM Digi al Lib a y, was conduc ed using keywo ds
such as medical,da a li ecycle,medical da a li ecycle,da a managemen ,heal hca e, and abs ac
amewo k medical da a. The same keywo ds we e also used in he Google Schola academic
sea ch engine, yielding simila esul s.
The lack o a domain-speci ic da a li ecycle o medicine migh be p oblema ic, as common-
3.4. Da a Go e nance 41
pu pose li ecycles do no accoun o he unique equi emen s o medical da a. Fo example,
many gene al da a li ecycle models omi a dele ion o a chi al s age. Howe e , in he case o
elec onic medical eco ds, such s ages a e c i ical since, by law in coun ies like Colombia and
Spain, medical eco ds mus be e ained o i een yea s a e he las heal hca e in e ac ion and
hen secu ely dele ed h ough a p ede ined p ocess. In Colombia, eco ds ela ed o he a med
con lic o c imes agains humani y mus be p ese ed inde ini ely, unde sco ing he necessi y o
an a chi al s age. These legal and e hical conside a ions highligh he need o a medical-speci ic
da a li ecycle ha adequa ely add esses his ield’s unique demands.
In he ollowing sec ions, we will s udy da a li ecycles p oposed by Ha a d Business School
[62], he CRISP-DM me hodology [63], Da aONE [59], and Ha a d Longwood Medical School
[64]. We do his o gene a e a gene al unde s anding o he mos common da a li ecycles and
hose ha migh closely esemble wha a medical da a li ecycle should be. The i s wo a e
con en ional business and indus y s anda ds, while he second wo a e explici ly designed o
scien i ic da a managemen , wi h he las one being pa icula ly in e es ed in biomedical da a.
3.4.4.1 Ha a d Business School Da a Li ecycle
The Ha a d Business School da a li ecycle is a ela i ely simple model consis ing o eigh
s ages: gene a ion, collec ion, p ocessing, s o age, managemen , analysis, isualiza ion, and
in e p e a ion, which can be seen in sec ion (a) o Figu e 3.4. These s eps a e desc ibed as a
cycle because he lessons lea ned and insigh s gained om one da a p ojec ypically in o m
subsequen p ojec s. Howe e , as wi h all he da a li ecycles we will s udy in his sec ion, i is
unde s ood ha he e can be jumps be ween di e en s ages h oughou a single p ojec . [62]
Fo example, in one i e a ion, a p ojec migh aim o iden i y clus e s o use s. The insigh s
om his i s i e a ion could hen in o m he nex , whe e a subsequen p ojec seeks o classi y
new use s in o he p e iously iden i ied clus e s. This i e a i e p ocess allows o build upon
he insigh s o he p e ious i e a ions, compounding knowledge o e ime. Ne e heless, i is
possible ha while isualizing he use clus e s, i becomes clea ha he ou come is no ideal
and mo e in o ma ion needs o be collec ed, o he a ailable in o ma ion migh need o be
p ocessed di e en ly, so he e can be jumps be ween he di e en s ages. [62]
3.4. Da a Go e nance 42
(a) Ha a d Business School Da a Li ecycle [62] (b) CRISP-DM Me hodology [63] [65]
(c) Da a ONE Da a Li ecycle [59] (d) Ha a d Medical School Da a Li ecycle [64]
Figu e 3.4: Compa ison o da a li ecycles and me hodologies.
3.4. Da a Go e nance 43
I is clea ha his da a li ecycle in ends o keep ex ac ing alue om all he a ailable da a.
Hence, i does no include a dele ion o a chi al s age. This li ecycle also neglec s o include
a planning s age. This migh be ela ed o he ac ha being a business- ocused da a model,
i does no ha e o communica e in o ma ion such as wha da a is needed o , how i will be
s o ed, o how long i will be accessible, and how i will be used o any go e ning bodies. As we
disco e ed by speaking wi h di e en esea che s in Ca alonia, his is a necessa y s ep usually
led by he e hics commi ee in any medical esea ch p ojec .
Ano he cha ac e is ic o his model is ha i emphasizes p esen ing da a in an easily un-
de s andable o ma , such as g aphs o cha s. This is an in e es ing ea u e, as u ning da a
in o diag ams o cha s p esen s in o ma ion while obscu ing he iden i y o he humans om
whom his in o ma ion was ga he ed. This could be an in e es ing insigh when p oposing a
da a li ecycle model o medical da a.
This is a s anda d de ini ion o he da a li ecycle, and o he sou ces p opose e y simila
models. The Columbia Uni e si y’s Da a Science Depa men is one o hose sou ces [66].
3.4.4.2 CRISP-DM Da a Li ecycle
The C oss-Indus y S anda d P ocess o Da a Mining (CRISP-DM) was o iginally published in
1999 o s anda dize da a mining p ac ices. Al hough i is no echnically a da a li ecycle model
bu a da a mining e e ence model, i also p o ides a s uc u ed app oach o da a p ojec s wi h
compa able s ages and ac i i ies o a da a li ecycle[65]. Because o his and i s wide use and
a ailabili y, we chose o include i among he da a li ecycle models s udied.
Ins ead o being a ci cula o a linea model, i p oposes a di e en eedback s uc u e, as
seen in sec ion (b) o Figu e 3.4. This aims o allow o as e e inemen o he p ojec as new
insigh s a e gained, as he e is no need o comple e an en i e cycle o eadjus . The a ows in
he p ocess diag am illus a e he mos impo an and equen eedback loops be ween phases,
while he ou e ci cle ep esen s he comple e cycle h ough which he inal s age is connec ed
o he nex i s s age. [63]
Al hough i is mo e a ge ed owa ds indus y p ojec s, i makes some p oposals ha a e

3.4. Da a Go e nance 44
also in e es ing in he medical ield. Fo ins ance, he da a unde s anding s age includes asks
such as desc ibing he da a and e i ying da a quali y, which would be essen ial o medical
da a applica ions. CRISP-DM also emphasizes documen a ion in all i s deli e able objec s,
acili a ing audi ing and p omo ing anspa ency. Addi ionally, i has di e en e ospec i e
s ages in which bo h he p ocess and he esul s a e e alua ed, making iden i ying and co ec ing
e o s easie . [63]
Unlike he Ha a d Business School da a li ecycle, he inal s age o he CRISP-DM me hod-
ology is deploymen . This s age is pa icula ly ele an in he con ex o medical AI-based
applica ions, as he ul ima e goal is o make hese solu ions accessible o clinicians and o he
medical p o essionals. Howe e , deploymen equi es me iculous planning o ensu e ha da a
secu i y and pa ien p i acy a e upheld. Robus measu es mus be in place, such as imple-
men ing a secu e login sys em ha es ic s da a access o au ho ized use s. Addi ionally,
o he deploymen -speci ic conside a ions mus be add essed o align wi h he sensi i e na u e o
heal hca e da a and he demand o high a ailabili y.
3.4.4.3 Da aONE Da a Li ecycle
The Da a Obse a ion Ne wo k o Ea h p oposes he Da aONE li ecycle. This communi y-
d i en p og am aims o acili a e access o Ea h and en i onmen al da a while educa ing he
communi y on good p ac ices. I is an eigh -s age, i e a i e li ecycle, as seen in sec ion (c)
o Figu e 3.4. This model emphasizes p ac ices ha enhance da a sha ing and euse wi hin
he scien i ic communi y, as i is mean o scien i ic da a managemen . As such, i has a
s uc u e ha di e s om he p e iously desc ibed models, wi h a mo e conside able emphasis
on planning and quali y assu ance. One o i s main goals is o c ea e sel -desc ibing da ase s
ha a e unde s andable and usable e en a e a signi ican pe iod o ime has passed. [59]
Like mos o he da a li ecycle models, he Da a ONE p oposal is i e a i e and includes co e
s ages such as planning, da a collec ion, and analysis. This model emphasises me ada a as a
ool o da a unde s anding and euse. I also p oposes some unique s ages: assu e, desc ibe,
and in eg a e. [59]
3.4. Da a Go e nance 45
The assu ance s age is o quali y con ol and quali y assu ance. I includes asks such as
desc ibing any condi ions du ing collec ion ha migh a ec da a quali y, iden i ying es ima ed
alues, and checking hand-en ied da a. Meanwhile, he nex s age, desc ibe, is conce ned wi h
he gene a ion o me ada a and documen a ion by desc ibing he digi al con ex o he da a,
s akeholde s and pe sonnel, scien i ic con ex , and in o ma ion ega ding uni s o measu emen ,
o ma s, p ecision, unce ain y, and o he in o ma ion abou he da a’s di e en alues. Finally,
he in eg a ion s age ocuses on he combina ion o da a om di e en sou ces. This can be
pa icula ly in e es ing o ou case s udy, as medical da a will come om elec onic heal h
eco ds, lab esul s, pha macies, and o he membe s o he heal hca e ecosys em. [59]
3.4.4.4 Ha a d Longwood Medical School Da a Li ecycle
As seen in sec ion (d) o Figu e 3.4, his is a six-s age, i e a i e li ecycle. I ocuses pa icula ly
on biomedical da a o esea ch p ojec s. Because o his, i p io i izes he p esen a ion o
esea ch indings in academic publica ions ins ead o he deploymen s age ound in o he da a
li ecycles. This model pu s special emphasis on he planning and design s age and ad oca es o
a de ailed managemen plan ha co e s aspec s such as s a egies o managing clinical da a,
policies o da a secu i y, compliance wi h egula ion, and he en o cemen o di ec o y and
naming con en ions ha ensu e da a o ganiza ion and accessibili y. [64]
This model is especially conce ned wi h da a secu i y. Al hough da a secu i y is an un-
de lying conce n o o he models, i seldom ge s any mo e han a men ion in he oo no es.
By con as , his model highligh s bes p ac ices o secu ing ac i e da a, choosing app op ia e
s o age loca ions, and c ea ing backups. This model conside s he impo ance o collabo a ion
and da a sha ing o his ield and, as such, p oposes he c ea ion o da a use ag eemen s and
adequa e pe mission managemen . All o hese indi idual conside a ions a e made a ailable o
he esea che h ough he Resea ch Da a Managemen Li ecycle Checklis documen [67].
3.4.5 Da a Li ecycle Managemen
De ining a heo e ical amewo k o he da a being handled has many ad an ages. By de ining a
amewo k, we can p o ide s uc u e and o maliza ion o da a managemen . This is whe e Da a
3.4. Da a Go e nance 46
Li ecycle Managemen (DLM) eme ges as a comp ehensi e policy-based app oach o managing
he low o da a [68] h ough he da a li ecycle, which we explo ed in dep h in Sec ion §3.4.4.
A common i s s ep owa d DLM is c ea ing a da a managemen plan, which de ails how
da a will be desc ibed, managed, and s o ed. The plan can also de ine any s anda ds ha will
be used ega ding da a p o ec ion and o he ma e s.
The de ini ion o a da a li ecycle managemen plan usually in ol es asks like iden i ying he
da a ha is being gene a ed and collec ed and how i will be o ganized in e ms o wha ools
will be needed o manage i , which s o age ype will be used, and whe he i should be s o ed
on-si e o locally. Documen ing a da a s o age and p ese a ion s a egy is ano he easonable
inco po a ion in o a DLM plan, s a ing in o ma ion abou he li espan o da a, who has access
o i and unde wha ci cums ances, as well as documen ing measu es aken o p o ec he da a
h oughou i s li ecycle [68].
Common addi ions o a DLM plan also include he de ini ion o da a policies, oles, and
esponsibili ies. By da a policies, we e e o documen ing how da a will be managed and
sha ed, emphasising he legal and e hical es ic ions on access o use o he da a. The oles and
esponsibili ies sec ion aims o de ine he necessa y playe s o ensu e adequa e da a managemen
and hei espec i e esponsibili ies. Fo example, i is necessa y o de ine who will ensu e ha
policies a e ollowed and he ex en o hei esponsibili ies [68].
Chap e 4
Regula o y Landscape on Pe sonal
Da a P o ec ion in Ca alunya and
Colombia
The apid ad ancemen in AI and he inc easing eliance on digi al sys ems o manage sensi i e
in o ma ion, such as medical eco ds, unde sco es he impo ance o o obus egula o y ame-
wo ks o pe sonal da a p o ec ion. In his con ex , he Ca alan egula o y landscape s ands
ou due o i s comp ehensi e app oach o da a go e nance. This sec ion explo es he Ca alan
egula ion go e ning pe sonal da a, medical eco ds and AI de elopmen . By doing so, we in end
o lay he g oundwo k o a compa a i e analysis wi h he Colombian egula o y landscape o
iden i y any po en ial gaps.
This sec ion’s goal is no o go in o de ail ega ding he speci ics o he legal amewo k,
as no in-dep h legal analysis will be conduc ed. Ou p incipal goal ough his s udy o he
egula o y landscape is o elici insigh s ha enable us o accu a ely o mula e s a egies wi hin
da a managemen ha acili a e compliance, guiding he design o he amewo ks which will
be p oposed la e on.
As a seconda y goal we would like o analyse he cu en s a e o egula ion in Colombia, o
unde s and whe e i s ands in compa ison o he Eu opean example. Since i is ou in en ion
o p opose amewo ks ha could wo k bo h wi hin he Ca alan and Colombian con ex i is
necessa y o e i y ha he e a e no majo egula o y di e ences ha would make his impossi-
47
4.2. Regula o y Landscape o Colombia 54
We will begin by ci ing a ansla ion o A icle 15 o he Colombian Na ional Cons i u ion,
p o ided by Ox o d Uni e si y’s Compa a i e Cons i u ions P ojec [79], as i is he undamen al
igh o Colombian ci izens ha p o ides he legal basis o all o he de elopmen s on his opic.
ARTICLE 15 - All indi iduals ha e he igh o pe sonal and amily p i acy and o
hei good epu a ion, and he S a e has o espec hem and o make o he s espec
hem. Simila ly, indi iduals ha e he igh o know, upda e, and ec i y in o ma ion
collec ed abou hem in da a banks and in he eco ds o public and p i a e en i ies.
F eedom and he o he gua an ees app o ed by he Cons i u ion shall be espec ed
in he collec ion, p ocessing, and ci cula ion o da a. Co espondence and o he o ms
o p i a e communica ion may no be iola ed. They may only be in e cep ed o eco ded based
on a cou o de in cases and ollowing he o mali ies es ablished by s a u e. Fo ax o legal
pu poses and cases o inspec ion, he o e sigh and in e en ion o he S a e may equi e making
a ailable accoun ing eco ds and o he p i a e documen s wi hin he limi s p o ided by s a u e.
[79]
We will now ou line he guidelines o Colombian legisla ion on he p ocessing o pe sonal
da a. These guidelines apply o all pe sonal da a ha exis in any da abase belonging o any
en i y wi hin he Colombian ju isdic ion, ega dless o whe he i is o a public o p i a e na u e.
This egula ion also applies o all o Colombia’s Na ional Te i o ies and ou side o hem when
Colombian legisla ion is applicable due o di e en in e na ional ea ies. I only conside s he
ollowing excep ions:
•Da a o exclusi ely pe sonal and domes ic use. Should his da a be p o ided o any hi d
pa ies, i will become subjec o he legal disposi ions de ined by he Colombian legisla ion
o pe sonal da a.
•Da abases and iles ela ed o na ional secu i y and de ence, con ol o money launde ing
and unding o e o ism.
•Da abases ha con ain in o ma ion ela ed o in elligence and coun e in elligence.
•Da abases and iles con aining jou nalis ic in o ma ion and o he edi o ial con en s.
The de ini ion o pe sonal da a gi en by he Colombian law is consis en wi h he de ini ion

4.2. Regula o y Landscape o Colombia 55
p o ided in he GDPR, as shown in sec ion 3.1. Colombian legisla ion also de ines sensi i e
da a as in o ma ion which may a ec he da a subjec ’s p i acy, and which can, i misused,
gene a e disc imina ion. Fo ins ance, in o ma ion pe aining o hei ace, poli ical o ien a-
ion, eligious con ic ions, and in o ma ion ela i e o hei heal h, and biome ic da a (among
o he s).
4.2.1.2 P inciples o he T ea men o Pe sonal Da a
The ollowing p inciples a e es ablished in Law 1581 o 2012 as necessa y o handling pe sonal
da a.
The p inciple o legali y in ma e s o pe sonal da a ea men dec ees pe sonal
da a handling is a egula ed ac i i y which mus adhe e o he legal amewo k o Colombia.
The p inciples o pu pose es ablishes he need o a legi ima e pu pose o p ocessing
in acco dance wi h he cons i u ion and he law, and said da a p ocessing mus be p e iously
in o med o he da a subjec .
The p inciple o eedom s a es ha he ea men o pe sonal da a may only occu wi h
p io in o med and explici consen . Addi ionally, pe sonal da a may no be di ulged wi hou
p e ious au ho iza ion o legal basis ha elie es he need o consen .
The p inciple o e aci y and quali y calls o co ec , comple e, exac , upda ed, asce -
ainable, and comp ehensible da a o he pu poses o da a ea men . I explici ly o bids he
handling o da a ha does no ul il hese cha ac e is ics o da a ha may induce e o s.
The p inciple o anspa ency g an s he i ula o he da a he igh o ob ain a any
momen , and wi hou es ic ions, in o ma ion ega ding he exis ence o hei pe sonal da a
om he esponsible o p ocesso o hei pe sonal da a.
The p inciple o access and es ic ed ci cula ion ei e a es ha he p ocessing o
pe sonal da a may only be ca ied ou by hose wi h he da a subjec ’s consen . Addi ionally,
i o bids displaying pe sonal da a on he in e ne o o he massi e di ulga ion channels unless
hey a e con ollable h ough echnology o p o ide es ic ed access only o he da a i ula
4.2. Regula o y Landscape o Colombia 56
and o hi d pa ies o which consen o access da a has been g an ed.
The p inciple o secu i y s a es ha he da a subjec o p ocessing mus be handled
wi h adequa e echnical, human, and adminis a i e con ols o ensu e he sa e y o he da a,
a oiding adul e a ion, loss, unau ho ized consul , unau ho ized use, aud o misuse.
The p inciple o con iden iali y exp esses ha all pa ies in ol ed in p ocessing pe sonal
da a a e obliga ed o sa egua d he sec ecy o pe sonal da a, e en a e hey no longe ha e any
ela ionship o he asks in ol ed in p ocessing.
4.2.1.3 Righ s and Obliga ions
The p inciples s a ed in Sec ion 4.2.1.2 a e used as he base o de ine he igh s o da a subjec s.
These p inciples also pe ain o he obliga ions o da a con olle s and p ocesso s.
Righ s o Da a Subjec s The pe son o which pe sonal da a pe ains, known as he Da a
Subjec in he GDPR o he Ti ula de los Da os in Colombian laws, has he ollowing igh s in
Colombia:
•To know, upda e, and ec i y one’s pe sonal da a be o e he Da a Con olle s o da a
owne s.
•To eques p oo o he au ho iza ion g an ed o he Da a Con olle , excep when i is
explici ly exemp ed as a equi emen o P ocessing.
•To be in o med by he Da a Con olle o Da a P ocesso , upon eques , o how one’s
pe sonal da a has been used.
•To ile complain s wi h he Supe in endence o Indus y and Comme ce o iola ions o
he p o isions o he law.
•To e oke he au ho iza ion and/o eques he dele ion o da a when he p inciples, con-
s i u ional and legal igh s, and gua an ees a e no espec ed du ing he p ocessing.
•To access, ee o cha ge, one’s pe sonal da a ha has been subjec o P ocessing and o
consul he da a pe aining o hemsel es on any da abase.
4.2. Regula o y Landscape o Colombia 57
As s a ed by Law 1581 o 2012, he p ocessing o pe sonal da a is only allowed unde he
in o med and asce ainable au ho iza ion on behal o he da a subjec . Au ho iza ion is con-
side ed in o med i he Da a Con olle makes i clea and unde s andable o he da a owne
he p ocessing hei pe sonal da a will be subjec ed o, he pu pose o his p ocessing, he pos-
sibili y o e using o answe ques ions ela ed o sensi i e da a o ega ding he pe sonal da a
o child en and eenage s, as well as he igh s o he da a subjec and he da a con olle ’s
physical add ess, elec onical add ess, and phone numbe . In o de o he au ho iza ion o be
asce ainable, he da a con olle mus keep p oo o au ho iza ion.
The ins ances in which au ho iza ion on behal o he da a subjec is unnecessa y a e ew
and well-de ined by Law 1581 o 2012. These a e limi ed o ins ances in which he da a is
equi ed by a public o adminis a i e en i y in he exe cise o hei legal unc ions. The da a
is conside ed public da a in cases o medical o sani a y eme gencies, i au ho ized by law o
his o ical, s a is ical o scien i ic ends, o i he da a pe ains o he ci il egis y o people
( om he Spanish Regis o Ci il de Pe sonas, a legal and adminis a i e ins umen by which
he S a e ecognizes he igh s and obliga ions o Colombian ci izens). Howe e , e en in hese
cases, hose who access he da a mus s ill ollow he obliga ions ha will be discussed in he
ollowing wo sec ions.
Obliga ions o Da a Con olle s The Da a Con olle , o Responsable del T a amien o
as hey a e called in Colombian law, mus ul il ce ain du ies in addi ion o he addi ional
egula ions o hei ield. The e a e 16 obliga ions in o al, which we will g oup in o ou
di e en ca ego ies.
•Obliga ions owa ds he Da a Subjec : Gua an ee he igh o habeas da a. Ask hem
o consen o au ho ize da a p ocessing. In o m hem abou he pu pose o he collec ion
and he igh s con e ed by he au ho iza ion. P ocess any inqui ies o claims submi ed
by he da a subjec . And upon eques , in o m he owne o any in o ma ion ega ding
he use o hei da a.
•Obliga ions owa ds he Da a P ocesso : Ensu e ha he da a gi en o he p ocesso
adhe es o he p inciple o e aci y and quali y. In o m he p ocesso in a imely manne
4.2. Regula o y Landscape o Colombia 58
o any upda es o he in o ma ion so ha he p ocesso may upda e i as well. Rec i y
any inco ec in o ma ion and communica e i o he p ocesso . Supply he p ocesso only
wi h he da a ha has been p e iously au ho ized o p ocessing in acco dance wi h he
law. Demand om he da a p ocesso o adhe ence o sa e y and p i acy egula ions.
In o m he p ocesso when he da a subjec is dispu ing a gi en piece o in o ma ion.
•Obliga ions owa ds he Compe en Au ho i y: In o m he compe en au ho i y
abou any sa e y b eaches and abou he exis ence o any isks ega ding he adminis a ion
o he da a owne s’ in o ma ion. Follow he ins uc ions and equi emen s s a ed by he
Compe en Au ho i y.
•Obliga ions ega ding Go e nance: Keep a copy o he au ho iza ion g an ed by he
owne o he da a. Ensu e adequa e condi ions o sa egua d he p inciple o secu i y. Adop
in e nal policies and p ocedu es o gua an ee he ul ilmen o he legal equi emen s and
o suppo he solu ion o consul s and eclama ions.
On op o hese obliga ions ou lines in Law 1581 o 2012, Dec ee No. 090 o 2018 es ablishes
ha Da a Con olle s ha a e non-p o i en i ies wi h o e 100.000 UVTs (equi alen o 1044763
eu os as o 2024) in asse s mus egis e o he na ional da abase egis y (RNBD, o i s ini ials
in Spanish) all o he da abases unde hei con ol ha con ain pe sonal in o ma ion subjec ed
o manual o au oma ed p ocessing. This is done wi h he pu pose o keeping ack o he numbe
o da a owne s, he ypes o pe sonal da a being p ocessed, he Da a Con olle s ha exis wi hin
he sys em, he objec i e o he p ocessing and he policies o pe sonal da a handling being
used.
Obliga ions o Da a P ocesso s The Da a P ocesso , o Enca gado del T a amien o, also
ha e a lis o 12 du ies hey mus ul il, which we will simila ly ca ego ize in o ou di e en
g oups.
•Obliga ions owa ds he Da a Subjec : Gua an ee he igh o habeas da a. P ocess
any inqui ies o claims submi ed by he da a subjec .
4.2. Regula o y Landscape o Colombia 59
•Obliga ions owa ds he Da a Con olle : Upda e any in o ma ion epo ed by he
Da a Con olle in a ime span o 5 wo k days.
•Obliga ions owa ds he Compe en Au ho i y: Re ain om ci cula ing in o ma-
ion ha is being dispu ed by he da a subjec , and he Compe en Au ho i y has o de ed
o i o be blocked. Regis e any eclama ions in he designa ed da abase. Regis e he
da a ega ding in o ma ion ha is unde ju idical discussion in he da abase es ablished
o i once he compe en au ho i y makes he P ocesso awa e o any such cases. In o m
he compe en au ho i y abou any sa e y b eaches and abou he exis ence o any isks
ega ding he adminis a ion o he da a subjec s’ in o ma ion. Follow he ins uc ions
and equi emen s s a ed by he Compe en Au ho i y.
•Obliga ions ega ding Go e nance: Ensu e adequa e condi ions o sa egua d he secu-
i y p inciple. Ca y ou any upda es, co ec ions o dele ions in a imely manne . Adop
in e nal policies and p ocedu es o gua an ee he ul ilmen o he legal equi emen s and
o suppo he solu ion o consul a ions and eclama ions. Allow access o he in o ma ion
only o hose who ha e a legal basis o access i .
4.2.1.4 Mechanism o Vigilance and Sanc ions
The au ho i y o da a p o ec ion in Colombia is he Supe in endence o Indus y and Comme ce
h ough he Delega ion o he P o ec ion o Pe sonal Da a. I is esponsible o exe cising
su eillance o ensu e espec o he p inciples, igh s, gua an ees and p ocedu es desc ibed by
he law du ing da a p ocessing. Wi hin he Delega ion o he P o ec ion o Pe sonal Da a,
he e is a Di ec ion o In es iga ions Rega ding he P o ec ion o Pe sonal Da a, inside o which
he e is a second Di ec ion o habeas da a.
The Supe in endence o Indus y and Comme ce exe cises he ollowing co e unc ions wi h
espec o he p o ec ion o pe sonal da a:
•Ensu e compliance wi h exis ing egula ions o he p o ec ion o pe sonal da a.
•In es iga e, on i s own ini ia i e o a he eques o a hi d pa y, and o de he necessa y

4.2. Regula o y Landscape o Colombia 60
measu es o en o ce he igh o habeas da a.
•O de he empo a y blocking o da a when, based on he eques and he e idence p o-
ided by he da a subjec , he e is a clea isk o iola ing hei undamen al igh s.
•Ca y ou p omo ion, dissemina ion, and educa ional ac i i ies o in o m and ain ci izens
abou exe cising and gua an eeing he undamen al igh o da a p o ec ion.
•Issue ins uc ions on he measu es and p ocedu es ha exis ing da a con olle s and p o-
cesso s mus adop o comply wi h he p o isions o he law.
•Collec om da a con olle s and p ocesso s any in o ma ion equi ed o ca y ou i s
unc ions.
•Issue decla a ions o con o mi y ega ding in e na ional da a ans e s.
•Adminis e he Na ional Public Regis y o Da abases and issue any o de s and ac s nec-
essa y o i s adminis a ion and ope a ion.
•Sanc ion da a con olle s and p ocesso s who ha e ailed o comply wi h he law.
Rega ding sanc ions, he Supe in endence o Indus y and Comme ce may issue successi e
ines while he egula ion iola ion pe sis s. The amoun o hese ines will depend on he
damage, isk, o economic bene i o he in ac o and o he s caused by non-compliance wi h
he law. The Supe in endence may also call o he suspension o ac i i ies ela ed o da a
p ocessing o up o six mon hs, empo a y closing o ope a ions ela ed o he p ocessing, and
immedia e and de ini i e cease o ac i i ies ha equi e he p ocessing o sensi i e da a i he
necessa y co ec i e measu es ha e no been a he ime he suspension ends. Howe e , hese
measu es only apply o p i a e ins i u ions, in he case o a public au ho i y being in b each o he
egula ion he Supe in endence o Indus y and Comme ce will pass he case o he P ocu adu ´ıa
Gene al de la Naci´on (O ice o he Inspec o Gene al) o ca y ou he in es iga ion.
4.2.1.5 T ans e o Da a o O he Coun ies
The ans e o pe sonal da a o o he coun ies is o bidden i he o he coun y does no ha e
adequa e le els o da a p o ec ion egula ions, which in all cases mus no be in e io o hose
4.2. Regula o y Landscape o Colombia 61
manda ed by Colombian law. The Supe in endence o Indus y and Comme ce es ablishes which
coun ies a e conside ed o ul il his equi emen .
The e a e some excep ions o his p ohibi ion, mainly when he da a subjec gi es hei
consen o ans e he da a o when i comes o he exchange o medical da a equi ed o he
da a subjec ’s ea men o easons ela ed o hei own heal h o public heal h conce ns. I
will no apply ei he o da a ans e s ha a e ag eed upon in he amewo k o an in e na ional
ea y based on mu ual ecip oci y o i he ans e s a e legally equi ed o sa egua d public
in e es o o exe cise o de end a legal igh in a judicial p ocess. One inal excep ion is da a
ans e s ela ed o s ock ading o bank ansac ions, which a e egula ed by hei own se o
laws ha a e no conside ed ele an o he opic o his p ojec .
Suppose an in e na ional ans e o pe sonal da a is needed in a di e en con ex han
hose desc ibed be o e. In ha case, he Supe in endence o Indus y and Comme ce issues a
decla a ion o con o mi y on said ans e (excep , o cou se, i he ans e alls wi hin he lis
o excep ions p esen ed abo e). To do his, he eques e will be allowed o eques in o ma ion
and ca y ou he necessa y s eps o es ablish compliance wi h he equi emen s ha would
enable he eques e o ans e he da a.
4.2.2 Regula ion Rega ding Medical Reco ds in Colombia
Medical eco ds a e medical da a sou ces ha ecei e pa icula ca e du ing legisla ion. Al hough
medical da a is conside ed sensi i e da a acco ding o Colombian law, his law ex ends addi ional
gua an ees and equi emen s o medical eco ds. We will explo e hese special cha ac e is ics in
his sec ion.
In gene al, medical eco ds a e egula ed in a di e en way han medical da a. In any o
he laws conce ning medical eco ds, i is no explici ly men ioned ha o he medical da a
should be ea ed wi h he same le el o ca e. This esul s in o he ypes o medical da a
being co e ed as pe sonal and sensi i e da a by he egula ions ela ed o Law 1581 o 2012
o pe sonal da a p o ec ion. A s an example o his, we can see how Su ame icana (one o
he la ges companies in he heal h sec o in Colombia) buil he p i acy policy o i s gene al
4.2. Regula o y Landscape o Colombia 62
Type Full Name Rele an Sec ions Topics
Resolu ion Resoluci´on 1995 de
1999
– Dic a es he no ms o
he managemen o he
medical eco d.
Law Ley 23 de 1981 Chap e III Law by which ules on
medical e hics a e
es ablished. Chap e
III on Medical
P esc ip ion, Medical
Reco ds, Medical
Con iden iali y, and
Some P ocedu es.
Law Ley 2015 de 2020 – By means o which he
in e ope able digi al
medical eco d is
c ea ed and o he
disposi ions.
Table 4.1: Legisla ion o Medical Reco ds
insu ance, EPS (En idad P omo o a de Salud: ins i u ion esponsible o ensu ing access o
medical a en ion), IPS (Ins i ucion p es ado a del Se icio de Salud: Heal hca e p o ie ), and
diagnosis and medical assis ance b anches c ea ed hei p i acy policy a ound Law 1581 o 2012
[80].
A icle 34 o Law 23 o 1981 de ines he medical eco d as he obliga o y egis e o a pa ien ’s
heal h condi ions. The law conside s his eco d a p i a e documen ha may only be accessed
by hi d pa ies wi h p io consen om he pa ien o in cases co e ed by he law.
The medical eco d de ails a pe son’s physical, psychological, and social p o ile. I ch onolog-
ically eco ds any in o ma ion ega ding a pa ien ’s condi ion, ea men , and o he p ocedu es.
Pe sonal and amilia in o ma ion is de ailed in his ile. As such, i is a highly sensi i e eposi-
o y o da a ha equi es special p o ec ion.
The body o law ha egula es medical eco ds in Colombia is as and can be ound in
Annex II. Howe e , Table 4.1 lis s some o he mos ele an egula o y documen s. We will
ocus on he main cha ac e is ics o he Colombian medical eco d and how i di e s om egula
pe sonal da a.
4.2. Regula o y Landscape o Colombia 63
4.2.2.1 Da a Subjec and Da a Con olle
Law 2015 o 2020 s a es in i s six h a icle ha each pe son owns hei medical eco ds; in i s
i h a icle, i s a es ha heal hca e p o ide s a e he cus odians o hese medical eco ds. This
pu s hospi als and o he sys em p o ide s in a posi ion whe e hey mus adop he necessa y
echniques and p ocedu es o p o ec his in o ma ion.
4.2.2.2 S o age
Resolu ion No. 1995 o 1999 s a es in A icle 18 ha heal hca e p o ide s may s o e medical
eco ds using physical o echnological means. Whiche e me hod is selec ed mus comply wi h
Dec ee No. 1080 o 2015 (Ti le II, chap e 5 on Documen Managemen , A icles 2.8.2.5.1 o
2.8.2.8.3), Law 527 o 1999 by means o which he access and use o da a messages, elec onic
comme ce, and digi al signa u es a e de ended and egula ed and any pos e io egula ions ha
modi y o add on o hem.
Any elec onic sys em used o adminis e medical eco ds needs o ul il a se o equi emen s,
which a e de ailed in Ci cula No. 2 o 1997 o he Gene al A chi e o he Na ion (A chi o
Gene al de la Naci´on). Among hese equi emen s, i is s a ed ha he e mus be p io echnical
s udies o be o e implemen ing any new s o age sys ems, and he choice o mo e o a new s o age
sys em mus be jus i ied. The documen s s o ed in he sys em mus ha e he same alidi y and
e icacy as he o iginals. The sys em mus main ain he same le el o sa e y, du abili y, and
ep oduc ion o he in o ma ion as he p io sys em and gua an ee he adequa e unc ioning o
he se ices ha depend on i .
4.2.2.3 A chi al and Dele ion
In acco dance wi h A icle 3 o Resolu ion No. 839 o 2017, medical eco ds in Colombia mus
be s o ed o a minimum o i een yea s, which a e coun ed s a ing on he da e o he las
medical se ice p o ided. Du ing he i s i e yea s, he medical eco d mus be s o ed in a
da a managemen sys em o a chi o de ges i´on, and o he ollowing 10 yea s, i can be s o ed in
a cen al a chi e o a chi o cen al. This ime span will be doubled o ic ims o human igh s
4.3. Legal F amewo k Compa ison 70
4.3 Legal F amewo k Compa ison
O e all, Colombian law implemen s simila enan s o he GDPR in ega d o pe sonal da a
p o ec ion, wi h simila igh s o da a subjec s and obliga ions o da a con olle s and p oces-
so s. Bo h egional legisla ions also include a special classi ica ion o sensi i e da a wi h simila
de ini ions, howe e , he Ca alan legal ins umen s p o ide mo e special gua an ees o his ype
o da a.
As o medical eco ds, bo h p oposes a simila s uc u e in which he medical ins i u ion
is he cus odian o he da a, and he pa ien is i s owne . I di e s in one aspec , he lack
o cen alized go e nmen al medical eco d sys em. Due o he Law 2015 o 2020 his migh
change in he u u e, as he de elopmen o an in e ope able scheme o medical eco ds has been
manda ed. This di ec ly con lic s wi h he minimiza ion o cen aliza ion da a minimiza ion
s a egy, howe e , so a only pilo es s ha e been conduc ed and due o he lack o in o ma ion
on he inal design and implemen a ion de ails o he p oposed sys em i is no possible o
p o ide any u he conclusions ega ding i .
On his a ea, one o he key insigh s om he Ca alan example ha Colombia migh wan
o look in o is he c ea ion o a PADRIS-equi alen . As a as we asce ained om in e iews
conduc ed wi h Colombian medical and compu a ional esea che he access o medical in o -
ma ion is s ic ly go e ned by he hospi al and i s e hical boa d. Addi ionally, we did no ind
any legal ins umen ha manda ed he c ea ion o such an en i y. The in oduc ion o a simila
o ganiza ion migh acili a e inno a ion by empowe ing esea che in di e en ins i u ions o
access medical da a. Addi ionally, he in oduc ion o a hi d pa y which explici ly ad oca es
o pa ien ’s da a igh s could be e y in e es ing in he Colombian con ex .
As o a i icial in elligence egula ion, al hough Colombia has no passed any laws on he
opic ye , he p oposals ha a e cu en ly being made a e pa ly based on he AI Ac and
p opose simila ideas. Howe e , one ecommenda ion ha can be made om he example o he
AI Ac is he in oduc ion o a Scien i ic Panel o Independen Expe s (AI Ac A . 68), which
does no make pa o any o he law p oposals cu en ly unde conside a ion.
In conclusion, he Colombian legisla ion does no di e om he Ca alan legisla ion in any

4.3. Legal F amewo k Compa ison 71
way ha would make i impossible o c ea e de elopmen and audi ing amewo ks ha can
po en ially be use ul in bo h con ex s. I is impo an o no e ha al hough he Colombian laws
a e less de ailed han he Eu opean legisla ion, which is pa icula ly no iceable when s udying
he GDPR, he law in Colombia is complemen ed by a conside able amoun o esolu ions,
dec ees and ci cula s which a e all en o ceable by law. This makes he compa ison challenging,
due o he shee amoun o documen s ha should be s udied in o de o gain ull unde s anding
o he Colombian egula o y con ex .
Table 4.2 summa izes some o he impo an aspec s o he cu en s a e o egula ion o
bo h egions ega ding pe sonal da a, medical eco ds, and a i icial in elligence applica ions.
Colombia Ca alunya
Compe en
Au ho i y o
Pe sonal Da a
Delega ion o he P o ec ion o
Pe sonal Da a o he
Supe in endence o Indus y and
Comme ce (Delega u a pa a la
P o ecci´on de Da os Pe sonales
de la Supe in endecia de
Indus ia y Come cio)
Spanish Da a P o ec ion Agency
(Agencia Espa˜nola de P o ecci´on
de Da os) a a na ional le el.
Posi ion o he
Compe en
Au ho i y o
Pe sonal Da a
Wi hin he
Go e nmen
The Delega ion o he
P o ec ion o Pe sonal Da a
exis s wi hin he execu i e
b anch, hie a chically below he
p esiden , he ice p esiden , he
Minis y o Indus y and
Comme ce, and he
Supe in endence o Indus y and
Comme ce.
Fully independen adminis a i e
au ho i y.
4.3. Legal F amewo k Compa ison 72
Special Ca ego ies
o Da a
Da a e ealing acial o e hnic
o igin, poli ical o ien a ion,
eligious o philosophical belie s,
membe ship in unions, social
o ganiza ions, human igh s
o ganiza ions, o en i ies
p omo ing he in e es s o any
poli ical pa y o gua an eeing
he igh s and gua an ees o
opposi ion poli ical pa ies. I
also encompasses da a ela ed
o heal h, sexual li e, and
biome ic da a. (Ley 1581 de
2012)
Da a e ealing acial o e hnic
o igin, poli ical opinions,
eligious o philosophical belie s,
o ade union membe ship, and
he p ocessing o gene ic da a,
biome ic da a o he pu pose
o uniquely iden i ying a na u al
pe son, da a conce ning
heal h o da a conce ning a
na u al pe son’s sex li e o sexual
o ien a ion (GDPR A . 9).
Righ o Da a
Subjec s
Know, upda e, and ec i y
pe sonal da a. Reques p oo o
consen o p ocessing. Be
in o med upon eques o how
one’s pe sonal da a is used by
he con olle o p ocesso . File
complain s i da a p o ec ion
laws a e iola ed. Re oke
au ho iza ion, eques dele ion o
da a, and access he pe sonal
da a s o ed by a con olle o
p ocesso . (Ley 1581 de 2012)
T anspa en in o ma ion,
communica ion, and modali ies
o he exe cise o he igh s o
he da a subjec . Be in o med o
whe e hei da a is being
collec ed di ec ly om hem, be
in o med o whe e hei da a is
being collec ed om ano he
sou ce, igh o access hei da a,
igh o ec i y, o e ase, o
es ic p ocessing, o objec , and
o e use au oma ed indi idual
decision-making (GDPR Ch. 3).
4.3. Legal F amewo k Compa ison 73
B each Repo ing Obliga ed o in o m he
compe en au ho i y. (Ley 1581
de 2012)
Obliga ed o in o m he
compe en au ho i y and da a
subjec s (GDPR A . 33).
Sys em o he
S o age o Medical
Reco ds
De ined by each medical cen e
acco ding o he disposi ion o
he compe en au ho i y.
Cen alized sys em de eloped by
pa liamen a y manda e. [87]
Pe sis ence o
Medical Reco ds
15-yea minimum legally
equi ed, and in p ac ice
pe pe ually.
15-yea maximum o pa ien
iden i ica ion da a, in o med
consen o ms, discha ge epo s,
su gical epo s and deli e y
eco ds, anes hesia- ela ed da a,
epo s om complemen a y
examina ions, au opsy epo s,
and pa hology epo s; 5-yea
maximum o e e y hing else.
Legisla i e
Adop ion o AI
Regula ion
No laws accep ed ye . AI Ac accep ed and in o ce.
Table 4.2: Compa ison Be ween Colombia and Ca alunya
Chap e 5
P oposal o a P i acy-Awa e Da a
Li ecycle and Managemen Plan o
Medical Da a, MDLC
5.1 The MDLC
Based on he esea ch conduc ed, i has become clea ha a da a li ecycle amewo k can be
a undamen al ool o p ese ing p i acy. This is because i can be used o embed p i acy-
p ese ing measu es a e e y s age o da a p ocessing. Pa icula ly in he medical ield, which
is known o be highly egula ed, i can also aid in inco po a ing in e nal con ols in o p ojec
de elopmen and educe he need o e oac i e documen a ion du ing a po en ial audi . In ligh
o some o he in e iews conduc ed wi h medical p o essionals and compliance specialis s, i is
my iew ha , o AI-based sys ems o exis wi hin hospi als and he wide assis i e amewo k
o heal hca e, he le el o in e nal con ol will need o ise o he s anda d o o he p ocesses in
he ield.
Regula ion is also an impo an conce n when i comes o medical da a, which a domain-
speci ic da a li ecycle amewo k can help add ess. Mos da a li ecycles aim o be gene al and
co e a wide ange o di e en p ojec s; howe e , his comp omises he le el o de ail hey can
p o ide ega ding he necessa y asks o egula o y compliance. In he case o sys ems ha
p ocess medical da a, he e a e a a ie y o legal equi emen s, such as conduc ing a p i acy
74
5.1. The MDLC 75
impac assessmen . O he da a li ecycles do no men ion he need o a p i acy impac assess-
men o he de ini ion o a da a access plan; ne e heless, hese a e necessa y when dealing wi h
medical da a (acco ding o he GDPR). A domain-speci ic da a li ecycle can b ing such asks o
a en ion and help speci y he app op ia e poin in ime a which hey should be pe o med o
achie e be e esul s.
P i acy-by-design p inciples place ull li e cycle p o ec ion, o end- o-end secu i y, as one o
he co e pilla s o da a secu i y. Howe e , he bene i s o a s uc u ed app oach o in eg a ing
p i acy conce ns in de elopmen go e en u he , as i can also help main ain anspa ency
and isibili y, p omo e p oac i eness, and gi e designe s and de elope s ample oppo uni y o
conside p i acy h ough each s age o de elopmen . Howe e , e en i his is ue, mos da a
li ecycles s udied do no emphasize p i acy conce ns in any signi ican o de ailed way.
As a inal ema k, a ecen su ey [88] conduc ed in he so wa e de elopmen indus y
has b ough o ligh a consis en pa e n o small de elopmen eams inding hemsel es in a
posi ion whe e hey mus make p i acy decisions by hemsel es a a ious s ages o he so wa e
de elopmen li ecycle. This su ey iden i ied inconsis en no ions o p i acy held by indi iduals
a di e en s ages o he de elopmen p ocess and he need o a s anda dized app oach ha
co e s he di e en aspec s o p i acy a a ious s ages o a p ojec . Clea ly de ining he ac i i ies
and policies ha exis wi hin each s age o da a handling o he pu pose o p o ec ing p i acy
is a necessa y i s s ep owa d achie ing a s anda d amewo k o p i acy-awa e de elopmen ,
which could po en ially aid esponsible de elopmen .
The in oduc ion o a medical da a li ecycle is pe inen because medical da a is uniquely
sensi i e. As such, he common ac i i ies p oposed in b oade da a li ecycles a e no likely
su icien o ul ill legal, anspa ency, and p i acy equi emen s. Addi ionally, he p oposal o
a domain-speci ic li ecycle can aid in making explici ac i i ies ha a e no ele an o common
in o he ields, such as acqui ing an e hical boa d’s consen .
Audi abili y, egula o y compliance, he cha ac e is ics and secu i y equi emen s o medical
da a, and he need o s anda d p ac ices mo i a e he p oposal o a domain-speci ic medical
da a li ecycle o ill he gap in he li e a u e o an audi able, p i acy- and egula ion-awa e

5.1. The MDLC 76
abs ac amewo k o suppo he de elopmen o AI-based sys ems ha pe o m p ocessing
asks on medical da a.
The p oposed medical da a li ecycle, called MDLC o he ini ials o he Medical Da a Li e
cycle, consis s o se en s ages, plus an addi ional in e media e s age, he In e media e Risk
Assessmen (IRA). Figu e §5.1 illus a es he comple e da a li ecycle p oposed. The p oposed
solu ion is i e a i e, and al hough he asks a e p esen ed in a pa icula o de consis en wi h
mos common da a li ecycles, i is no mean o be lineal, as we acknowledge ha he e can be
unexpec ed eedback loops ha do no pe ec ly adhe e o he p oposed o de .
Figu e 5.1: P oposal o an I e a i e, Non-Lineal Medical Da a Li ecycle
This da a li ecycle is speci ically aimed a AI-based p ojec s conduc ed wi hin he con ex o
a heal hca e acili y, de eloped by in e nal o ex e nal eams, and elian on pa ien s’ pe sonal
da a. I is pa icula ly ocused on p ojec s expec ed o ha e a medium o long li espan, such
as esea ch p ojec s es ed o e p olonged pe iods o a emp s o adap he indings o an
academic publica ion o he acili y’s en i onmen . I may also se e as a ele an guide o
public o p i a e en i ies ha a e no heal hca e p o ide s bu a e in ol ed in he b oade
5.1. The MDLC 77
medical a en ion en i onmen . Howe e , i is no well-sui ed as a e e ence o p ojec s in he
ield o medicine ha do no u ilize pa ien da a, such as he de elopmen o models ela ed o
d ug disco e y, d ug in e ac ions, equipmen , o s a , among o he s, as i places special emphasis
on da a subjec s’ p i acy and igh s.
This da a li ecycle is designed wi h p i acy-by-design in mind. I p omo es p oac i i y
h ough asks aimed a p edic ing and de ec ing isks ea ly and places special emphasis on he
de elopmen o adequa e documen a ion o ensu e anspa ency and isibili y. Addi ionally,
conduc ing asks h oughou he da a li ecycle ha p omp he de elopmen eam o conside
how p i acy can be impac ed o p o ec ed du ing he p ojec ’s li espan suppo s embedding
p i acy in o he sys em’s design om i s incep ion h ough o i s conclusion.
In his chap e , we will go h ough he se en s ages o he p oposed medical da a li ecycle and
he p oposed IRA. We will explo e he pu pose, ac i i ies, and ou pu s o each s age. Al hough
e o was made o c ea e ac i i ies ha a e as conc e e as possible, some le el o abs ac ion
is necessa y due o he lack o speci ic knowledge abou he pa icula p ojec being conduc ed.
This need o abs ac ion will be pa icula ly no iceable in p ocessing, modelling, e alua ion,
and deploymen sec ions.
5.1.1 S age 1: Plan and Design
The Plan and Design s age se s he ounda ion o he esponsible de elopmen o he p ojec .
This s age o he da a li ecycle ocuses on de ining p ojec objec i es, scope, and guiding p in-
ciples while add essing he unique challenges o medical da a and i s applica ions, such as he
medical con ex , he mo i a ion o de elopmen , and he ele an popula ion.
This s age ensu es ha key conside a ions, such as p i acy-by-design, egula o y compliance,
and alignmen wi h p inciples, a e in eg a ed om he ou se . I also p omo es he mapping o
use cases and he iden i ica ion o he a ge popula ion, which will be help ul la e o selec ing
he da a o be used. Addi ionally, i in ol es ope a ional aspec s, such as de ining oles and
es ablishing a ime ame o access o he da a.
By he end o he plan and design s age, he p ojec should ha e a clea oadmap ha aligns
5.1. The MDLC 78
wi h e hical, legal, and p ac ical equi emen s, se ing he s age o esponsible and e ec i e
de elopmen . The main asks and ou pu s o he Plan and Design s age a e shown in Figu e
5.2, and a de ailed explana ion o each ask and i s deli e ables can be ound in Tables 5.1, 5.2,
5.3, and 5.4, espec i ely.
As a gene al ema k, i is impo an o keep in mind ha he speci ic de ails will change om
p ojec o p ojec . In o de o e lec his we do no p o ide documen s such as he p inciples
and iola ions checklis , bu in i e he de elopmen eam o cons uc i hemsel es so ha i
will ma ch hei con ex , p ojec p oposal, and scope o execu ion. Fo his same eason we
don’ speci y a esponsible o each ask, we a e awa e ha de elopmen g oups will a y in
size and con igu a ion, and hey will be be e equipped o de ine which eam membe has he
equi ed expe ise o ca y ou each ask.
Figu e 5.2: Plan and Design S age Summa y
5.1. The MDLC 79
Task Ou line P ojec De ini ion and Scope
The eam’s i s objec i e in his ask is o unde s and he
objec i es and in ended each o he p ojec om bo h a clinical
and a echnical s andpoin . O en, he e will be many compe ing
objec i es and cons ain s ha mus be balanced, and h ough
his ask, he eam may begin o cla i y hese ade-o s and
unco e po en ial ac o s ha migh a ec he p ojec ’s ou come.
Neglec ing his s ep may lead o pu ing e o in o making a
co ec analysis o he w ong p oposal.
Ou pu s P ojec Backg ound Documen
Reco d he in o ma ion ha is known abou he p ojec ’s
si ua ion a he beginning o he p ojec .
Key Ac i i ies: Resea ching analogous de elopmen s made in o he
ields o o he medical ins i u ions, explo ing he clinical con ex o
he p oposal, and de ining ele an egula o y amewo ks.
Requi emen Speci ica ion Documen
Lis s all he equi emen s o he p ojec , bo h unc ional and
non- unc ional. Lis s any assump ions made abou he p ojec o
he da a ha may ha e o be echecked once access has been
g an ed, and lis p ojec cons ain s.
P ojec P oposal Documen
Desc ibes he in ended plan o achie ing he goals desc ibed in
he P ojec De ini ion and Scope documen . Lis s he asks ha
will be ca ied ou du ing p ojec de elopmen , hei equi ed
inpu s, ou pu s, dependencies, and es ima ed comple ion ime. I
is also pe inen beginning a p elimina y assessmen o ools and
echniques, which migh be o ced o change du ing he ollowing
s ages o de elopmen .
Table 5.1: P ojec De ini ion and Scope Deli e ables
5.1. The MDLC 86
5.1.3 S age 3: Analyze, Assu e, and P epa e
The main objec i es o his s age a e o shed ligh on possible quali y issues o he da a, highligh
in e es ing ela ionships be ween ea u es, and c ea e he inal da a sou ces o model aining
and e alua ion.
This sec ion is ela i ely consis en wi h o he da a li ecycles. Ne e heless, his does no
mean ha da a p o ec ion is neglec ed du ing his s age. I is impo an o ensu e ha de elop-
men is guided by he p oposals made du ing he Plan and Design s age and ha he p omise o
secu e s o age and con olled access made in he plans is adhe ed o. I would be in e es ing o
conside he use o ze o- us sys ems o us ed execu ion en i onmen s s a ing a his s age
in o de o p o ide a highe le el o secu i y o he da a.
This s age places sligh ly mo e emphasis on da a quali y assu ance han o he p oposed da a
li ecycles, simila ly o CRISP-DM and he Da aONE li ecycle. This is pa ly due o ecommen-
da ions made by medical p o essionals du ing he in e iews, bu i is also a common conce n
when de eloping AI models; as he saying goes, “ga bage in, ga bage ou .”
This s age is also conce ned wi h p oducing su icien me ada a. This can be used o guide
a emp s o eplica e a success ul p ojec o e ec i ely euse he da a on a u u e p ojec ,
educing he amoun o new p ocessing equi ed.
Figu e 5.4 p o ides an o e iew o he asks and deli e ables o his s age, which a e ex-
panded upon on Tables 5.7 and 5.8.

5.1. The MDLC 87
Figu e 5.4: Analyze, Assu e, and P epa e S age Summa y
5.1. The MDLC 88
Task Know he Da a
Examine he da a o gain an unde s anding o i s supe icial
p ope ies and hose undamen al, unde lying, o inhe en
a ibu es o da a ha a e no immedia ely obse able. This ask
can gi e he de elopmen eam insigh s as o wha ype o
ques ions can be answe ed wi h he da a, which g aphs o plo s
migh be illus a i e, and epo ing needs.
Ou pu s Da a Desc ip ion
Desc ibes he su ace p ope ies o he da a, including i s o ma ,
he quan i y o da a, he names and desc ip ions o ields, and
o he g oss ea u es o he da a disco e ed.
Key Ac i i ies: Elici ield desc ip ions i he names a e no clea ,
conduc basic coun unc ions on he da a, and documen he
names, sizes and opics o he a ailable da ase s.
Da a Explo a ion Repo
De ail he ac i i ies ca ied ou du ing da a explo a ion and hei
ou comes.
Key Ac i i ies: Calcula e he dis ibu ion o ele an a ibu es,
s udy ela ionships be ween a ibu es, conduc basic s a is ical
ope a ions, iden i y ele an sub-popula ions o isk g oups,
ha ness domain knowledge o guide da a explo a ion.
Da a Quali y Repo
De ail he da a-quali y e i ica ion ac i i ies conduc ed and lis
hei esul s. I issues in da a quali y a e ound, lis possible
solu ions o implica ions o subsequen s ages o p ojec
de elopmen .
Key Ac i i ies: Ve i y he compliance o he da a p o ided wi h he
pa icipan accep ance c i e ia, con i m ha he amoun o da a is
consis en wi h he numbe o pa icipan s selec ed, e iew ha he
a ailable ields a e consis en wi h he ields eques ed on S age 2:
Selec and Collec , check o e o s paying special a en ion o
ields ha a e illed by hand, iden i y missing alues, use s a is ical
ope a ions o ind impossible alues in minimums, maximums o
a e ages, and iden i y ou lie s. Conside using quali y lags o
indica e he pe cei ed da a quali y [89].
Table 5.7: Know he Da a Deli e ables
5.1. The MDLC 89
Task P epa e he Da a
In his ask, he de elopmen eam akes ac ion on he insigh s
gained du ing he Know he Da a ask by p ocessing he da a o
clean i , handle missing alues, gene a e new ea u es and
in eg a e i wi h o he da a sou ces.
Ou pu s Da a Cleaning Repo
Desc ibes he ac ions aken o imp o e he quali y issues
disco e ed du ing he p e ious ask (such as missing da a), as well
as explaining he easoning behind he cou se o ac ion selec ed.
I de ails how he da a was ans o med o cleaning and any
possible impac s hese ac i i ies may ha e on subsequen s ages o
p ojec de elopmen .
Cons uc ed Da a Repo
I any new ea u es we e p oduced by combining exis ing
a ibu es o by inco po a ing ex e nal da a, hey should be lis ed
and explained in his documen . Addi ionally, i any syn he ic
egis e s a e gene a ed o he pu pose o modeling, hey should
be disclosed in his documen and he easoning behind hem
should be made explici , as well as he possible impac s his
decision may ha e on subsequen s ages.
Da a In eg a ion Repo
I any o he da a sou ces we e me ged, i should be disclosed and
jus i ied in his sec ion, as da a segmen a ion is p e e ed o
p i acy p ese a ion. Addi ionally, i any ex e nal da ase s o
esou ces a e inco po a ed in o he inal da ase , hey should be
lis ed and jus i ied as well.
Final Da ashee
A his poin , he eam should be able o answe all he ques ions
in he Da ashee s o Da ase s ques ionnai e and ill ou a
comple e da ashee o he da ase s ha will be used o aining
and e alua ion.
Final Da ase s
This da a will be used du ing he Model and Compa e s age o
ain and es he p oposed models.
Table 5.8: P epa e he Da a Deli e ables
5.1. The MDLC 90
5.1.4 S age 4: Model and Compa e
This s age ocuses on he e alua ion o a ious model e sions, implemen a ions, and hype -
pa ame e s. Du ing his phase, he de elopmen eam ansi ions om he p elimina y assess-
men o ools and echniques conduc ed in S age 1: Plan and Design o he conc e e implemen-
a ion o a speci ic model o se o models om which esul s will be de i ed.
This p ocess ollows a s anda d app oach bu inco po a es a highe le el o documen a ion.
Among he p oposed documen a ion o his s age is he Design His o y File (DHF), a concep
no co e ed by any o he da a li ecycles explo ed in he li e a u e e iew. O igina ing om he
medical de ice indus y, he DHF is no uni e sally manda ed; howe e , i is a equi ed a e ac
o medical de ice de elopmen in he Uni ed S a es unde he Code o Fede al Regula ions
(CFR) Ti le 21, Sec ion 820.30(j). The CFR de ines he DHF as “a compila ion o eco ds
which desc ibes he design his o y o a inished de ice.”
The adap a ion o he DHF as a ool o audi ing AI sys ems was p oposed in [57]. I s
applica ion appea s pa icula ly app op ia e o AI-based heal hca e solu ions, whe e igo ous
documen a ion and aceabili y a e c i ical o compliance, sa e y, and e hical conside a ions.
The p oposed asks and deli e ables o s age 4 a e shown in Figu e 5.5 and explained in
mo e de ail on Tables 5.9 and 5.10
5.1. The MDLC 91
Figu e 5.5: Model and Compa e Summa y
Task Compa e Modeling Techniques
The goal o his ask is o selec he bes implemen a ions and
con igu a ions o he possible models.
Ou pu s E alua ion Me ics
Lis o he e alua ion me ics selec ed o compa e he di e en
models and con igu a ions. Conside inco po a ing pe o mance
me ics as well as me ics ha a ge ai ness and bias,
explainabili y and o he non- unc ional equi emen s.
Model Selec ion Repo
This documen will desc ibe all he ac i i ies ca ied ou o selec
any speci ic implemen a ion and con igu a ion o he algo i hm o
model. I will a a minimum lis he es s ca ied ou , an
explana ion o why hese es s we e conside ed ele an and
su icien , and desc ibe he ain/ es spli used. I will also
display he esul s o he es s and he inal decision ega ding
model selec ion. I will ou line he ade-o s o each op ion being
conside ed.
Key Ac i i ies: De ine candida e models, de ine hype -pa ame e s
o conside , de ine es s, un es s, p oduce g aphs and me ics o
show he esul s o he es s, and de ine he models ha will be
used om his poin onwa d.
Table 5.9: Compa e Modeling Techniques Deli e ables

5.1. The MDLC 92
Task Model and Tes
In his ask, he de elopmen eam will ocus on he selec ed
model o models o gain a be e unde s anding o how hey
beha e and he esul s i p o ides.
Ou pu s Model Tes and Insigh Repo
This documen should de ail he esul s o model aining and
es ing, which migh di e om hose in Compa e Modeling
Techniques i hose es s we e conduc ed wi h subse s o da a. I
should desc ibe any o he es s conduc ed o ex ac insigh s om
he model’s beha iou , such as es s ega ding isk g oups o
mino i y popula ions wi hin he da ase .
Key Ac i i ies: Do he ull aining necessa y o achie e he
p ojec ’s goals, conduc addi ional in-dep h es s as necessa y ( o
ins ance es s ela ed o isk g oups), speci y conc e e use cases,
de ine assump ions he model makes abou he da a o a oid a
po en ial misma ch wi h p oduc ion da a.
Model Ca ds o Model Repo ing
Fill ou a Model Ca d [49] o each o he ained models.
Comple e DHF
Mas e documen e e s o all he design and implemen a ion
documen a ion iles o he algo i hm’s de elopmen and any no es
made by he de elopmen eam o o he s akeholde s du ing IRAs.
Table 5.10: Model and Tes Deli e ables
5.1. The MDLC 93
5.1.5 S age 5: Assess and Sha e
This s age seeks o go e en u he han he e alua ion conduc ed in S age 4: Model and Compa e
and o assess o wha ex en he p oposed model o models ul ill he p ojec ’s objec i es wi hou
comp omising on he ag eed ounda ions o esponsible da a use. This sec ion emphasizes he
adhe ence o he plans made on s age 1, en o ces accoun abili y, and p o ides space o e lec ion
on he p ocess so a .
This s age also p o ides mechanisms o communica e wi h and ga he eedback om s ake-
holde s. This will be undamen al o in o m he decisions abou he p ojec ’s nex s eps, which
will be made based on wha he assessmen and eedback e eal abou he p ojec so a .
Al hough some o he s udied da a li ecycles p o ide simila s uc u es o his s age, hey a e
no as expansi e in hei p ojec assessmen and do no emphasize isk assessmen o adhe ence
o o iginal plans. A summa y o he asks and deli e ables o his s age is shown in Figu e 5.6,
wi h mo e de ails a ailable o each ask on Tables 5.11, 5.11, 5.11, 5.11, and 5.11.
5.1. The MDLC 94
Figu e 5.6: Assess and Sha e Summa y
5.1. The MDLC 95
Task Assess Compliance
In his ask, he goal is o assess he de elopmen eam’s
compliance wi h he plans and policies made du ing S age 1: Plan
and Design.
Ou pu s Lis o De ia ions om Da a Access Plan
Lis o changes, de ia ions, o addi ions o he da a access plan,
oge he wi h hei jus i ica ion and a sho assessmen o i s
possible impac s.
Lis o De ia ions om P ojec P oposal
Lis o changes, de ia ions, o addi ions o he p ojec p oposal,
oge he wi h hei jus i ica ion and a sho assessmen o i s
possible impac s.
Table 5.11: Assess Compliance Deli e ables
Task Assess P ojec Resul s
Assess he le el o ul ilmen o he objec i es o he p ojec , as
well as de e mine i all he impo an asks ha e been ca ied ou
and all he ele an ac s ha e been conside ed du ing he
de elopmen p ocess. This ask will se e as a e ospec i e on he
p ojec so a .
Ou pu s Re iew o P ocess
Summa izes he p ocess o e iewing i all he asks ha e been
comple ed o an accep able deg ee and lis any asks ha migh
need o be edone o ha we e, o some eason, o go en. Lis s
any ac s ha we e no aken in o accoun du ing planning o
de elopmen bu ha , in e ospec , migh cause issues la e down
he oad.
Re iew o P ojec Goal Ful illmen
Lis s all he goals s a ed du ing he p ojec p oposal and goes in o
de ail ega ding he le el o comple ion o each o hem.
Table 5.12: Assess P ojec Resul s Deli e ables
Task Assess Risk
This ask is aimed owa ds c ea ing a gene al o e iew o he
possible isks o he implemen ed sys em.
Ou pu s Upda ed P elimina y P i acy Impac Assessmen
P i acy Impac Assessmen upda ed o e lec he de eloped
sys em.
Sel Assessmen Checklis
Re iew o he Fu u e AI Assessmen Checklis o us wo hy
medical AI ools [90].
Table 5.13: Assess Risk Deli e ables
5.1. The MDLC 102
Task Assess
In o de o p omo e he ea ly de ec ion o po en ial o
ma e ialized isks, his ask p omp s he de elopmen eam o
upda e hei ans e sal documen a ion o e lec he cu en s a e
o he sys em and o e iew hei adhe ence o he ini ial
commi men s made o he guiding p inciples.
Ou pu s Upda ed PPIA
Buil upon he documen s a ed in S age 1: Plan and Design,
and on op o any upda es made on p io IRAs, he upda ed
PPIA will e iew he p e iously iden i ied isks and conside any
new po en ial ones. I may include, emo e, o modi y any isks in
ligh o he sys em’s cu en s a e.
Upda ed His o y File
This documen is a e iewed sys em his o y, upda ed o e lec
any new de elopmen s. Pas en ies may no be edi ed, bu in
case o e o , hey may be co ec ed by adding a no e including
he co ec ion and he sou ce o he e o . I is impo an o no e
ha unlike he PPIA, which is i s c ea ed in S age 1: Plan and
Design, he e y i s his o y ile will be c ea ed du ing an IRA.
Documen Requi emen Checklis Assessmen
A s uc u ed checkpoin o ensu e ha all necessa y
documen a ion o he s age being exi ed is in place. The
de elopmen eam e iews he equi ed documen s o e i y hei
comple ion, accu acy, and ele ance o he p ojec ’s cu en s a e.
I any equi ed documen s a e missing o incomple e, he eam
mus c ea e a plan o add ess hese gaps e oac i ely.
P inciples and Viola ions Checklis Assessmen
Re iew o adhe ence o guiding p inciples es ablished du ing he
p ojec ’s planning. The eam ca e ully examines he checklis o
ensu e no iola ions o hese p inciples ha e occu ed up o he
cu en s age. This assessmen helps iden i y de ia ions om
ag eed-upon s anda ds and e hical commi men s, p omo ing
accoun abili y and allowing o ea ly co ec i e ac ions i
iola ions a e de ec ed.
Table 5.21: IRA Summa y

Chap e 6
P oposal o an Audi ing F amewo k
o MLDC, UMAPER
6.1 UMAPER: An Audi ing F amewo k o MDLC P ojec s
An audi is he ac i i y o e i ying p ocesses and quali y sys ems o ensu e ha “ ex i he
o ganiza ion’s con ol p ocesses a e adequa e o mi iga e i s isks, go e nance p ocesses a e
e ec i e and e icien , and o ganiza ional goals and objec i es a e me ” [91]. Audi ing ga he s
e idence o con i m he p ope unc ioning o es ablished in e nal con ols. I is a aluable
ool o iden i ying e o s, ensu ing egula o y compliance, and highligh ing a eas wi h ou da ed
con ols.
In e nal con ols a e policies, p ocesses, asks, and beha iou s ha acili a e e ec i e ope -
a ions, hey ensu e quali y epo ing, and main ain compliance wi h applicable laws and eg-
ula ions [56]. Th oughou he s ages o he MDLC, a ious in e nal con ol mechanisms we e
in oduced o aise awa eness o isks and challenging si ua ions, p omo e and suppo in o med
decision-making, and minimize e o s.
Heal hca e sys ems a e al eady subjec o high le els o sc u iny and equen audi s due o
he c i ical na u e o hei ope a ions and he need o main ain public us . Audi ing in his
domain enhances anspa ency and accoun abili y, educes e o s, and s eng hens compliance
wi h egula ions. Howe e , he in oduc ion o AI-based sys ems in oduces unique isks, pa -
103
6.1. UMAPER: An Audi ing F amewo k o MDLC P ojec s 104
icula ly in ega d o p i acy. As add essed in he jus i ica ion o he MDLC, i is likely ha
i AI-based sys ems e e become widesp ead in he medical ield, hey will need o be audi ed.
This is why we p opose an audi ing me hodology based on he adi ional audi ing cycle and he
SMACTR me hodology [57] o conduc end- o-end audi s o p ojec s modeled a e he MDLC.
A ailo ed audi ing amewo k o he MDLC o e s se e al key ad an ages. Fi s , i p o ides
a s uc u ed app oach o e i ying compliance wi h in e nal con ols while assessing he pe o -
mance and sa e y o AI-based medical sys ems. By ocusing on bo h he p ocess and p oduc
le els, his amewo k ensu es ha wo k lows adhe e o he MDLC-de ined s anda ds and ha
he esul ing sys ems mee quali y benchma ks. Fu he mo e, adap ing he exis ing audi ing cy-
cle o i he MDLC model educes he ini ial cos s and esou ce demands o conduc ing audi s,
making i a mo e easible op ion o o ganiza ions.
The bene i s o such a amewo k ex end beyond ope a ional e iciency. A well-designed
audi me hodology p omo es con inuous imp o emen by c ea ing a eedback loop be ween audi
indings and sys em de elopmen . This i e a i e p ocess no only imp o es he quali y o he
audi ed p ojec , as i migh kicks a a new i e a ion o he MDLC, bu i can also imp o e he
quali y o u u e p ojec s h ough he insigh s i p oduces.
S akeholde us is a c ucial conside a ion in he adop ion o AI-based medical sys ems.
Audi ing ou comes a e o en me wi h scep icism due o hei eliance on human judgmen ,
which can lead o he insigh s p o ided by he audi being dis ega ded. We can gene a e a sense
o legi imacy and c edibili y a ound he audi ing p ocess by p oposing a speci ic amewo k wi h
p ede ined s ages and ac i i ies. In pa icula , he de elopmen o an audi ing amewo k seeks
o es ablishing p ocedu al jus ice in he audi ing p ocess, as es ablishing a amewo k can help
audi o s demons a e he adhe ence o accep ed s anda ds and demons a e he in eg i y o he
audi .
The p oposed audi ing amewo k has six s ages: Unde s and, Map, Assess, Plan, Execu e,
and Re lec . Each o hese s ages is based on he key audi ing ac i i ies, de ined by e e encing
he SMACTR amewo k, he s anda d audi ing cycle, and he ISO/IEC 27001:2022 equi e-
men s o in o ma ion secu i y, cybe secu i y and p i acy p o ec ion. An o e iew o he audi
6.1. UMAPER: An Audi ing F amewo k o MDLC P ojec s 105
Figu e 6.1: O e iew o he UMAPER F amewo k. Double-lined boxes ep esen p ocesses,
egula boxes ep esen asks, blue ep esen s inpu documen s, and o ange ep esen s ou pu
documen s.
amewo k is shown in Figu e 6.1
6.1.1 Unde s and: Why is he Audi Necessa y?
This s age aims o e alua e he audi ’s ele ance and speci y i s objec i es by an icipa ing
sou ces o isk and a eas o in es iga e. In o de o do his, he audi ing p o ide ( he en i y
ha p o ides he audi ing se ice, whe he i is in e nal o ex e nal) will collabo a e wi h he
audi equi e (indi idual, g oup, o o ganiza ion ha eques s he audi o be conduc ed) o
ga he in o ma ion such as:
•Why was he audi eques ed?
•Is he audi eques ed as pa o ou ine con ols, o is i a special eques ?
6.1. UMAPER: An Audi ing F amewo k o MDLC P ojec s 106
•Wha a e he main conce ns pe cei ed by he equi e ?
•Wha is he ele ance o his sys em in he con ex o he equi e ’s ope a ions?
•Wha a e he main cha ac e is ics and unc ionali ies o he sys em?
•Has he sys em al eady been audi ed in he pas ? I so, wha we e he ecommenda ions
made? Whe e hey ollowed h ough on?
•Ha e he e been any signi ican changes made o he sys em?
•Wha is he expec ed ime ame o he audi ?
•Wha would be he expec ed deli e ables o he audi p ocess in ega d o he equi e ’s
in e nal policies?
•Wha is he expec ed scope o he audi ?
•Who is he sys em owne ?
•Who a e he key collabo a o s he audi ing eam can ely on o ga he in o ma ion abou
he sys em?
The audi o can complemen his in o ma ion wi h he documen a ion p oduced by he
MDLC a di e en s ages:
•F om he Plan and Design s age: The p ojec backg ound documen , he equi emen
speci ica ion documen , and he p inciples and iola ions checklis .
•F om he Deploy s age: The inal p i acy impac assessmen , and he audi igge s.
In sho , his s age’s main asks a e ga he ing in o ma ion on he audi ’s mo i a ion and
he sys em’s o ganiza ional, e hical, and egula o y con ex . The eason o ca ying ou his
p elimina y in es iga ion is wo old. Fi s , i will help he audi ing eam ga he ini ial da a o
assess he easibili y o he audi . In second place, i will help he audi ing eam iden i y he
a eas on which he audi will ocus o he audi scope. The audi scope could po en ially be
6.1. UMAPER: An Audi ing F amewo k o MDLC P ojec s 107
p i acy isk assessmen , bu his amewo k is lexible enough o p o ide a base o o he ypes
o audi s. These indings o his s age should be e lec ed in a epo ha is o be p esen ed
o he equi e once i is inished and o he sys em owne ( he indi idual esponsible o he
sys em) p io o he ini ial mee ing.
The Audi Scope Repo : This epo should include no es on he equi e in e iew, he
easons o he conduc ion o he audi , he a eas o in e es o he audi , he no ma i e e e -
ences o he audi , he p oposed audi ing eam, and he immedia e ac i i ies, which will include
he no i ica ion o he audi o he sys em owne and he se ing o a da e o an ini ial mee ing
wi h he sys em owne so ha hey may p epa e.
6.1.2 Map: Wha is he Sys em Being Audi ed?
The goal o his s age is o he audi ing eam o lea n as much as possible om he sys em’s
a ailable documen a ion. This s age akes place be o e he i s mee ing wi h he sys em owne ,
as i s goal is o minimize he imposi ion on hei ime by becoming as amilia wi h he sys em as
possible independen ly. The knowledge acqui ed du ing his s age will la e need o be e i ied
wi h he sys em owne eam, as al hough he documen a ion s i es o cap u e he sys em’s
eali y, i is p one o e o s and inaccu acies, especially i he sys em has had any majo changes
ecen ly.
In his s age, he audi ing eam will e iew he sys ems documen a ion o amilia ize i -
sel wi h he de elopmen p ocess and iden i y in e nal s akeholde s and collabo a o s. The
documen a ion e iew aims o iden i y oppo uni ies o es ing by e iewing he de elopmen
p ocess and assessing he comple eness o he documen a ion by in e ac ing wi h i . Meanwhile,
he in e nal s akeholde and collabo a o iden i ica ions is done o help es ablish ele an con-
ac s which migh be necessa y du ing ollowing s ages, and o iden i y indi idual pa icipa ion
owa ds he inal ou come, allowing audi o s o assess pe sonal accoun abili y owa ds each s age
o he de elopmen p ocess.
In o de o ca y ou he asks o his s age, he audi ing eam may ely on documen s om
he MDLC, such as:

6.1. UMAPER: An Audi ing F amewo k o MDLC P ojec s 108
•F om he Plan and Design s age: The ou line o oles and esponsibili ies.
•F om he Analyse, Assu e, and P ocess s age: The inal da ashee .
•F om he Model and Compa e s age: The model ca ds and he design his o y ile.
•F om he Deploy s age: The sys em ac shee and p ojec e ospec i e.
Once hese asks ha e been comple ed, he audi ing eam may mo e on o he ini ial mee ing
wi h he sys em owne . Du ing his mee ing, he audi ing eam will sha e and con i m he in o -
ma ion hey ha e acqui ed abou he sys em, and he sys em owne will p o ide any addi ional
esou ces. A e his mee ing, he audi ing eam will c ea e a inal sys em map wi h he mos
up- o-da e in o ma ion on he sys em. I may hen mo e on o he ollowing s ages, in which
hey will begin p epa ing ques ionnai es and es s o conduc he audi . I is impo an o no e
ha i is common p ac ice o ha e an opening mee ing wi h he sys em owne be o e going in o
de ailed planning, as i se s up he s age o a be e in o med and p epa ed audi .
The Sys em Map: This documen p o ides a checklis o he expec ed documen a ion, ma k-
ing i as a ailable o una ailable and poin ing ou i any e o s o inaccu acies came o ligh
du ing he ini ial mee ing. he sys em map also p o ides a summa y o he sys em’s s uc u e,
a desc ip ion o each ele an sys em uni and and he ele an s akeholde s and con ac s.
6.1.3 Assessmen : Wha a e he Risks?
A necessa y p ecu so o he planning s age is he isk assessmen s age. The po en ial isks
incu ed by he sys em a e necessa y o in o m es selec ion, as hey a e he key o de ining
ele an es s. This is essen ial in o de o gua an ee ha he audi is e icien and e ec i e,
ocusing on a eas o g ea es isk.
Al hough he MDLC inna ely conside s p i acy impac assessmen , he expe ise o audi o s
goes a long way owa ds e iewing and complemen ing his documen . Addi ionally, as an
audi ’s scope is no es ic ed o p i acy, i migh be ele an o conduc a human igh s impac
assessmen , sys em bias isk assessmen , o ailu e modes and e ec s, among o he s.
6.1. UMAPER: An Audi ing F amewo k o MDLC P ojec s 109
This s age is in o med by he sys em map, howe e , his in o ma ion can and should be
complemen ing by consul ing o he s. Rele an con ac s ound du ing he p e ious s age, domain
expe s (medical p o essionals o hospi al adminis a i e pe sonnel in his con ex ), ex e nal
expe s, o legal ad iso s should be consul ed in o de o ha e wide co e age o he po en ial
isks ha a e inhe en o he sys em being audi ed and sys ems simila o i .
Addi ionally, documen a ion p oduced du ing he MDLC ha is no conside ed du ing he
sys em mapping can also p o ide insigh in o po en ial isks and mi iga o y measu es al eady
in place. These documen s will be pa icula ly ele an o a p i acy audi , bu hey can also
be ele an owa ds in o ming audi s wi h di e en objec i es as hey co e a wide a ie y o
ele an aspec s ega ding he p ojec ’s de elopmen . Rele an documen s include:
•F om he Plan and Design s age: he da a access plan and he p inciples and iola ions
checklis .
•F om he Selec and Collec s age: The da a selec ion epo , he da a o igin epo and
he da a deiden i ica ion epo .
•F om he Analyze, Assu e, and P opose s age: The da a quali y epo .
•F om he Model and Compa e s age: The model es and insigh epo .
•F om he Assess and Sha e s age: The lis o da a access plan de ia ions and he sel
assessmen checklis .
•F om he Deploy s age: The p oposal o moni o ing and main enance, he inal p i acy
impac assessmen , he p ojec e ospec i e and he audi igge s.
The audi ing eam’s wo k should be egis e ed in a epo . This documen will be sha ed
wi h he sys em owne and he audi equi e , so ha hey may e iew i and make necessa y
ecommenda ions be o e mo ing on o he ollowing s age.
The Risk Assessmen Repo : Lis s he isks ound, hei likelihood and hei se e i y, as
well as possible mi iga o y measu es o hem. This documen will also disclose any communica-
6.1. UMAPER: An Audi ing F amewo k o MDLC P ojec s 110
ion wi h ele an con ac s ha led o he disco e y o his po en ial isks o hei jus i ica ion
based on he sys em map.
6.1.4 Plan: Wha Tes s Will be Conduc ed? When Will hey be Conduc ed?
Du ing his s age, audi o s will use all he in o ma ion hey ha e ga he ed so a ega ding he
sys em, i s con ex , and he isks i en ails o p oduce a conc e e se o es s o be execu ed. Tes s
a e one o he main ools audi o s use o assess a sys em’s compliance wi h in e nal company
policies, legal egula ions, and e en–as p oposed in he SMACTR F amewo k–e hical alues.
These es s should ideally help assess he sys em’s mos conce ning o mos likely isks, which is
why he planning is conduc ed once he audi o s ha e a ho ough unde s anding o he sys em.
I his is a ou ine con ol and no majo changes o he sys em ha e occu ed be ween his
audi and he las , a lis o ele an es s migh al eady exis s. I his is he case, hen audi o s
should ake i as a base, upda e he ime ame, e iew he es s o ensu e hey a e all s ill
ele an , and ei he emo e hose ha no longe a e o upda e hem as necessa y.
The esul ing plan will be deli e ed o he audi equi e and he sys em owne , so ha hey
may be in o med o he p oposed es s. I he e a e any ques ions o cla i ica ions ha he audi
equi es o he sys em owne wishes o make, hey can be aken in o accoun by he audi ing
eam. Wi h his in o ma ion, he sys em owne can also begin o p epa e o he upcoming
es s by designing which human, echnical and o he esou ces will be assigned o aid du ing
es execu ion.
Tes selec ion and he sa is ac ion c i e ia o each es a e highly dependen on he sys em’s
speci ic pu pose, implemen a ion de ails and con ex . This is why planning will ha e o be
conduc ed o each indi idual sys em, and esul s may a y widely. Howe e , audi o s may
e e ence es ing plans made o simila sys ems du ing his p ocess. Fo ins ance, o a medical
image classi ie sys em, audi o s may wan o ake inspi a ion om clinical audi guides o assess
he medical aspec o he sys em, audi guides o classi ica ion sys ems o assess he algo i hmic
aspec s o he sys em, GDPR compliance guides o assess he egula o y aspec s o he sys em,
and so on.
6.1. UMAPER: An Audi ing F amewo k o MDLC P ojec s 111
The Tes ing Plan: This documen will include he es s o be ca ied ou du ing he execu ion
s age, hei jus i ica ion, he isk hey a ge , he equi ed collabo a o s o ca y hem ou , hei
expec ed s a and end da e, he membe o he audi ing eam esponsible o hem, and any
addi ional conside a ions ha a e deemed ele an by he audi ing eam. I his is an upda e
o a p e iously exis ing plan, i should highligh he added, emo ed o edi ed es s. Each es
will also be accompanied by a jus i ied g ading ub ic, which will de ail he equi emen s o a
es esul o be sa is ac o y. In addi ion o he indi idual es design, he es ing plan will also
include how sys em sa is ac ion will be calcula ed based on he indi idual es esul s.
6.1.5 Execu e: Ca ying Ou he Plan
Du ing his s age he audi eam, oge he wi h he sys em eam, will ca y ou he es ablished
es s in acco dance wi h he es plan. This is whe e mos o he audi ime is likely o be
consumed and whe e he audi eam will need o wo k mos closely wi h he sys em eam.
T ough his s age audi o s will engage wi h he sys em o ca y ou each es de ined in he
plan, g ade i acco ding o he ub ic p oposed in he es plan, and poin ou any specially
ele an esul s.
The es esul s will be p o ided o he sys em owne and he audi equi e , who may eques
a e-do es i he esul s seem inconsis en o unexpec ed. These e- es s need o be highligh ed
in he es esul epo , as hey can highligh aspec s o he sys em ha a e o special in e es
o u u e audi s.
The Tes Resul Repo : This documen will lis he es s conduc ed, i i was no possible
o any eason o execu e all he es s p oposed in he es plan i will ha e o be jus i ied in
his documen o. Fo each es conduc ed, he audi o s esponsible will p o ide he ime ame
o execu ion, a esul acco ding o he ub ic, and any no es made by hemsel es o he sys em
eam membe s hey collabo a ed wi h.
Bibliog aphy ii
[14] Naeem Fi dous Syed, Syed W. Shah, A ash Shaghaghi, Adnan Anwa , Zubai Baig, and
Robin Doss. Ze o us a chi ec u e (z a): A comp ehensi e su ey. IEEE Access, 10:57143–
57179, 2022.
[15] Ch is oph Buck, Ch is ian Olenbe ge , And ´e Schweize , Fabiane V¨ol e , and To s en Ey-
mann. Ne e us , always e i y: A mul i ocal li e a u e e iew on cu en knowledge and
esea ch gaps o ze o- us . Compu e s Secu i y, 110:102436, 2021.
[16] Cybe secu i y and In as uc u e Secu i y Agency Cybe secu i y Di ision. Ze o T us Ma-
u i y Model, P e-decisional D a . Technical epo , June 2021. Accessed on Jan. 17,
2024.
[17] Da id Loshin. 8 - dimensions o da a quali y. In Da id Loshin, edi o , The P ac i ione ’s
Guide o Da a Quali y Imp o emen , MK Se ies on Business In elligence, pages 129–146.
Mo gan Kau mann, Bos on, 2011.
[18] Sun he as a an Mu hy, Asmida Abu Baka , Fiza Abdul Rahim, and Ramona Ramli. A
compa a i e s udy o da a anonymiza ion echniques. In 2019 IEEE 5 h In l Con e ence on
Big Da a Secu i y on Cloud (BigDa aSecu i y), IEEE In l Con e ence on High Pe o mance
and Sma Compu ing, (HPSC) and IEEE In l Con e ence on In elligen Da a and Secu i y
(IDS), pages 306–309, 2019.
[19] Mike Hin ze and Khaled El Emam. Compa ing he bene i s o pseudonymisa ion and
anonymisa ion unde he gdp . Jou nal o Da a P o ec ion & P i acy, 2(2):145–158, 2018.
[20] APDCAT. Guia de p o ecci´
On de da os pa a pacien es y pe sonas usua ias de los se icios
de salud. h ps://apdca .genca .ca /web/.con en /03-documen acio/documen s/
Guia-p o eccio-de-dades-pacien s- 14-CAST.pd . Accessed: 2024-12-26.
[21] Thomas Neubaue and Johannes Heu ix. A me hodology o he pseudonymiza ion o
medical da a. In e na ional Jou nal o Medical In o ma ics, 80(3):190–204, 2011.
[22] Helge Aamo , Ch is ophe D Kohl, Daniel Rich e , e al. Pseudonymiza ion o pa ien iden-
i ie s o ansla ional esea ch. BMC Medical In o ma ics and Decision Making, 13(1):75,
2013.
[23] Mengmeng Yang, Taolin Guo, Tianqing Zhu, I an Tjuawina a, Jun Zhao, and Kwok-Yan
Lam. Local di e en ial p i acy and i s applica ions: A comp ehensi e su ey. Compu e
S anda ds In e aces, 89:103827, 2024.
[24] Mic oso Resea ch. P i acy-p ese ing machine lea ning: Main aining con iden iali y and
p ese ing us , 2022. Accessed: Decembe 29, 2024.
[25] Hannah B own, Ka he ine Lee, Fa emehsada Mi eshghallah, Reza Shok i, and Flo ian
T am`e . Wha does i mean o a language model o p ese e p i acy?, 2022.
[26] Da a Sani iza ion O ganiza ion. Da a sani iza ion e minology.
[27] S e e Alde . Heal h each communi y heal h cen e s epo s imp ope disposal
inciden a ec ing almos 117,000 pa ien s. h ps://www.hipaajou nal.com/
heal h each-communi y-heal h-cen e s- epo s-imp ope -disposal-inciden -a ec ing-almos -117000-pa ien s/.
Accessed: 2024-12-28.

Bibliog aphy iii
[28] Mina Aska i, Reihaneh Sa a i-Naini, and Ken Ba ke . An in o ma ion heo e ic p i acy
and u ili y measu e o da a sani iza ion mechanisms. In P oceedings o he Second ACM
Con e ence on Da a and Applica ion Secu i y and P i acy, CODASPY ’12, page 283–294,
New Yo k, NY, USA, 2012. Associa ion o Compu ing Machine y.
[29] James C. L. Chow, Vale ie Wong, and Kay Li. Gene a i e p e- ained ans o me -
empowe ed heal hca e con e sa ions: Cu en ends, challenges, and u u e di ec ions in
la ge language model-enabled medical cha bo s. BioMedIn o ma ics, 4(1):837–852, 2024.
[30] Mal e Schwa zkop , Eddie Kohle , M. F ans Kaashoek, and Robe Mo is. Posi ion: Gdp
compliance by cons uc ion. In He e ogeneous Da a Managemen , Polys o es, and Analy ics
o Heal hca e, page 39–53, Be lin, Heidelbe g, 2022. Sp inge -Ve lag.
[31] Jie Xu, Zihan Wu, Cong Wang, and Xiaohua Jia. Machine unlea ning: Solu ions
and challenges. IEEE T ansac ions on Eme ging Topics in Compu a ional In elligence,
8(3):2150–2168, June 2024.
[32] Bingyan Liu, Nuoyan L , Yuanchun Guo, and Yawen Li. Recen ad ances on ede a ed
lea ning: A sys ema ic su ey. Neu ocompu ing, 597:128019, 2024.
[33] Rui-Jie Yew, Lucy Qin, and Su esh Venka asub amanian. You s ill see me: How da a
p o ec ion suppo s he a chi ec u e o ai su eillance, 2024.
[34] William Lindskog and Ch is ian P eho e . A ede a ed lea ning benchma k on abula
da a: Compa ing ee-based models and neu al ne wo ks. In 2023 Eigh h In e na ional
Con e ence on Fog and Mobile Edge Compu ing (FMEC), page 239–246. IEEE, Sep embe
2023.
[35] Zijun Wang and Keke Gai. Decision ee-based ede a ed lea ning: A su ey. Blockchains,
2(1):40–60, 2024.
[36] Yu-Chi Chen, Song-Yi Hsu, Xin Xie, Sa u Kuma i, Sachin Kuma , Joel Rod igues, and
Bande A. Alzah ani. P i acy p ese ing suppo ec o machine based on ede a ed lea n-
ing o dis ibu ed io -enabled da a analysis. Compu a ional In elligence, 40(2):e12636,
2024.
[37] A shad Fa had, Sand a Woolley, and Pe e And as. Fede a ed lea ning o ai o imp o e
pa ien ca e using wea able and iom senso s. In 2021 IEEE 9 h In e na ional Con e ence
on Heal hca e In o ma ics (ICHI), pages 434–434, 2021.
[38] Google Cloud. Wha is enc yp ion?, 2025. Accessed: 2025-01-11.
[39] Kha Dinh Duy, Taehyun Noh, Siwon Huh, and Hojoon Lee. Con iden ial machine lea ning
compu a ion in un us ed en i onmen s: A sys ems secu i y pe spec i e. IEEE Access,
9:168656–168677, 2021.
[40] Abbas Aca , Hidaye Aksu, A. Selcuk Uluagac, and Mau o Con i. A su ey on homomo phic
enc yp ion schemes: Theo y and implemen a ion. ACM Compu . Su ., 51(4), July 2018.
[41] Reza Shok i, Ma co S ona i, Cong Song, and Vi aly Shma iko . Membe ship in e ence
a acks agains machine lea ning models. 2017 IEEE Symposium on Secu i y and P i acy
(SP), pages 3–18, 2017.
Bibliog aphy i
[42] Milad Nas , Reza Shok i, Ami Houmansad , James Nieh, and Dan Song. Membe ship
in e ence a acks om i s p inciples. In P oceedings o he 31s USENIX Secu i y Sympo-
sium, pages 937–954, 2018.
[43] Eugene Bagdasa yan, Omid Pou saeed, and Vi aly Shma iko . Di e en ial p i acy has dis-
pa a e impac on model accu acy. In Ad ances in Neu al In o ma ion P ocessing Sys ems,
olume 32, pages 15453–15462, 2019.
[44] Vic o ia Ruiz-Se a, Na aly Busl´on, Oli ie R. Philippe, Diego Saby, Ma ´ıa Mo ales, Camila
Pon es, Alejand o Mu˜noz Andi k´o, Gemma L. Holliday, Aina Jen´e, Mau icio Moldes, Jo di
Rambla, Al onso Valencia, Ma ´ıa Jos´e Remen e ia, A ia Co ´es, and Da ide Ci illo. Ana-
lyzing sex imbalance in ega and dbgap biological da abases: Recommenda ions o be e
p ac ices. iScience, 27(10):110831, 2024.
[45] H´ebe Hwang A colezi, Ka ima Makhlou , and Ca uscia Palamidessi. Local di e en ial
p i acy has no dispa a e impac on ai ness. In Vijay A lu i and Anna Lisa Fe a a, edi o s,
DBSec 2023 - 37 h IFIP Annual Con e ence on Da a and Applica ions Secu i y and P i acy,
pages 3–21, SOPHIA ANTIPOLIS, F ance, Jul 2023.
[46] Rene Ab aham, Johannes Schneide , and Jan om B ocke. Da a go e nance: A concep ual
amewo k, s uc u ed e iew, and esea ch agenda. In e na ional Jou nal o In o ma ion
Managemen , 49:424–438, 2019.
[47] Timni Geb u, Jamie Mo gens e n, B iana Vecchione, Jenni e Wo man Vaughan, Hanna
Wallach, Hal Daum´e III, and Ka e C aw o d. Da ashee s o da ase s, 2021.
[48] Timni Geb u, Jamie Mo gens e n, B iana Vecchione, Jenni e Wo man Vaughan, Hanna
Wallach, Hal Daum´e III, and Ka e C aw o d. Da ashee s o da ase s. Commun. ACM,
64(12):86–92, No embe 2021.
[49] Ma ga e Mi chell, Simone Wu, And ew Zaldi a , Pa ke Ba nes, Lucy Vasse man, Ben
Hu chinson, Elena Spi ze , Inioluwa Debo ah Raji, and Timni Geb u. Model ca ds o
model epo ing. In P oceedings o he Con e ence on Fai ness, Accoun abili y, and T ans-
pa ency, FAT* ’19, page 220–229, New Yo k, NY, USA, 2019. Associa ion o Compu ing
Machine y.
[50] John T. Richa ds, Da id Pio kowski, Michael Hind, S ephanie Houde, Aleksand a Mo-
jsilo ic, and Kush R. Va shney. A human-cen e ed me hodology o c ea ing ai ac shee s.
IEEE Da a Eng. Bull., 44:47–58, 2021.
[51] Ga y S. Collins, Johannes B. Rei sma, Douglas G. Al man, and Ka el G. M. Moons. T ans-
pa en epo ing o a mul i a iable p edic ion model o indi idual p ognosis o diagnosis
( ipod): he ipod s a emen . BMC Medicine, 13:1, 2015.
[52] Ga y S Collins, Ka el G M Moons, Paula Dhiman, Richa d D Riley, And ew L Beam,
Ben Van Cals e , Ma zyeh Ghassemi, Xiaoxuan Liu, Johannes B Rei sma, Maa en an
Smeden, Anne-Lau e Boules eix, Jenni e Ca he ine Cama adou, Leo An hony Celi, Spi os
Denaxas, Alas ai K Dennis on, Ben Glocke , Robe M Golub, Hugh Ha ey, Geo g Heinze,
Michael M Ho man, And ´e Pascal Kengne, Emily Lam, Naomi Lee, Elizabe h W Lode ,
Lena Maie -Hein, Bilal A Ma een, Melissa D McC adden, Lau en Oakden-Rayne , Jo-
han O dish, Richa d Pa nell, She i Rose, Ka andeep Singh, Lau e Wynan s, and Pa icia
Logullo. T ipod+ai s a emen : upda ed guidance o epo ing clinical p edic ion models
ha use eg ession o machine lea ning me hods. BMJ, 385, 2024.
Bibliog aphy
[53] Eu opean Da a P o ec ion Boa d. Ai audi ing, 2024.
[54] Banco In e ame icano de Desa ollo. Quan ilai: Algo i hmic audi , 2023.
[55] ACCA. A b ie guide o in e nal audi ing. Accessed: Decembe 31, 2024.
[56] Tu nbull Re iew G oup. In e nal con ol: Guidance o di ec o s on he combined code,
1999. Also known as he Tu nbull Repo .
[57] Closing he ai accoun abili y gap: de ining an end- o-end amewo k o in e nal algo i hmic
audi in. 2020.
[58] O acle. Wha is da a managemen ?
[59] Ca ly S asse , Robe Cook, William Michene , and Ambe Budden. Da aone bes p ac-
ices p ime : p ime on da a managemen : wha you always wan ed o know, 2012. 11pp.
DOI: h p://dx.doi.o g/10.25607/OBP-84.
[60] FAIR Da a and Se ices Wo king G oup. Fai p inciples.
[61] J. M. Wing. The da a li e cycle. Ha a d Da a Science Re iew, 1(1), 2019.
[62] Tim S obie ski. The da a li e cycle: P ocessing.
[63] P. e al. Chapman. C isp-dm1.0 s ep-by-s ep da a mining guide, 2009. Tech. Rep.
[64] Ha a d Medical School. The biomedical da a li ecycle.
[65] X. Wang and P. S. Yu. Da a li ecycle managemen : An o e iew. IEEE T ansac ions on
Knowledge and Da a Enginee ing, 32(12):2339–2357, 2019.
[66] Jeanne e M. Wing. The da a li e cycle.
[67] Julie Goldman. Resea ch da a managemen li ecycle checklis (pd ), 2024. Las upda ed on
Oc obe 2, 2024. Pa o Resea ch Managemen Resou ces on OSF [h ps://os .io/pw7ed].
[68] E en E yu ek, U i Gilad, Valliappa Lakshmanan, Ani a Kibunguchy-G an , and Jessi
Ashdown. Chap e 4: [da a go e nance o e a da a li e cycle]. In Da a Go e nance: The
De ini i e Guide. O’Reilly, 2021.
[69] Eu opean Pa liamen and Council o he Eu opean Union. Regula ion (EU) 2016/679 o
he Eu opean Pa liamen and o he Council.
[70] Eu opean Pa liamen and Council o he Eu opean Union. Regula ion (EU) 2023/1234 o
he Eu opean Pa liamen and o he Council.
[71] Gene al da a p o ec ion egula ion (gdp ). Online, n.d. Accessed: 2025-01-17.
[72] Bloombe g Law. Gdp p og am checklis , ND. Accessed: 2025-01-11.
[73] Da id E dos. Eu opean Da a P o ec ion Regula ion, Jou nalism, and T adi ional Publish-
e s: Balancing on a Tigh ope? Ox o d Uni e si y P ess, 12 2019.
[74] Pad is - p og ama d’anal´ı ica de dades pe a la ece ca i la inno aci´o en salu . Online.
Accessed: 2025-01-17.
Bibliog aphy i
[75] Ai ac en e s in o o ce. Online, 2024. Accessed: 2025-01-17.
[76] A i icial in elligence ac : Council gi es inal g een ligh o he i s wo ldwide ules on ai.
Online P ess Release, 2024. Accessed: 2025-01-17.
[77] A i icial in elligence - eu opean council policies. Online, n.d. Accessed: 2025-01-17.
[78] High-le el summa y o he a i icial in elligence ac . Online, n.d. Accessed: 2025-01-17.
[79] Zacha y Elkins, Tom Ginsbu g, and James Mel on. Cons i u e: The wo ld’s cons i u ions
o ead, sea ch, and compa e. Online a h ps://www.cons i u ep ojec .o g. Accessed
Janua y 5, 2025.
[80] EPS SURA. A icle on eps su a, n.d. Accessed: 2025-01-19.
[81] P oyec o de ley 154 de 2024 de la ep´ublica de colombia.
[82] P oyec o de ley 005 de 2024 de la ep´ublica de colombia.
[83] P oyec o de ley 130 de 2023 de la ep´ublica de colombia.
[84] P oyec o de ley 91 de 2023 de la ep´ublica de colombia.
[85] P oyec o de ley 59 de 2023 de la ep´ublica de colombia.
[86] P oyec o de ley 447 de 2024 de la ep´ublica de colombia.
[87] San iago Ma imon-Su˜nol, Ma ´ıa Ro i a-Ba be `a, Ma eo Acedo-An a, Mon se a A. Nozal-
Baldajos, and Joan Guanyabens-Cal e . His o ia cl´ınica compa ida en ca alu˜na. Medicina
Cl´ınica, 134:45–48, 2010.
[88] Maxwell P ybylo, Sa a Haghighi, Sai Teja Peddin i, and Sepideh Ghana a i. E alua ing
p i acy pe cep ions, expe ience, and beha io o so wa e de elopmen eams, 2024.
[89] NASA. Wha a e quali y lags (qa), wha do hey mean, and whe e can i ind hem?, n.d.
Accessed: 2025-01-19.
[90] Fu u e AI. The Fu u e AI E hical Checklis , n.d. Accessed: 2025-01-19.
[91] The Ins i u e o In e nal Audi o s (IIA). Abou in e nal audi , 2025. Accessed: 2025-01-15.
Annex I: Ca alan Regula ion on
Medical Reco ds
Taken om [20] he ollowing is a lis o ele an laws ega ding medical eco ds in Ca alunya.
•Reglamen o (UE) 2016/679 del Pa lamen o y del Consejo, de 27 de ab il de 2016,
ela i o a la p o ecci´on de las pe sonas ´ısicas en lo que espec a al a amien o de da os
pe sonales y a la lib e ci culaci´on de dichos da os, y po el que se de oga la Di ec i a
95/46/CE (Reglamen o Gene al de P o ecci´on de Da os).
•Ley O g´anica 3/2018, de 5 de diciemb e, de P o ecci´on de Da os de Ca ´ac e Pe sonal
y ga an ´ıa de los de echos digi ales.
•Ley O g´anica 1/1982, de 5 de mayo, de p o ecci´on ci il del de echo al hono , a la
in imidad pe sonal y amilia y a la p opia imagen.
•Ley O g´anica 10/1995, de 23 de no iemb e, del C´odigo Penal.
•Ley O g´anica 3/1986, de 14 de ab il, de Medidas Especiales en el ´
Ambi o de la Salud
P´ublica.
•Ley 21/2000, de 29 de diciemb e, sob e los de echos de in o maci´on conce nien e a la
salud y la au onom´ıa del pacien e, y a la documen aci´on cl´ınica.
•Ley 41/2002, de 14 de no iemb e, b´asica egulado a de la au onom´ıa del pacien e y de
de echos y obligaciones en ma e ia de in o maci´on y documen aci´on cl´ınica.
•Ley 25/2010, de 29 de julio, del segundo lib o del C´odigo ci il de Ca alu˜na, ela i o a la
pe sona y la amilia.
•Ley 14/2010, de 27 de mayo, de los de echos y las opo unidades en la in ancia y la
adolescencia.
•Ley 14/2007, de 3 de julio, de In es igaci´on Biom´edica.
•Ley 31/1995, de 8 de no iemb e, de p e enci´on de Riesgos Labo ales.
•Ley 19/2015, de 13 de julio, de medidas de e o ma adminis a i a en el ´ambi o de la
Adminis aci´on de Jus icia y del Regis o Ci il.
•Ley 14/2006, de 26 de mayo, sob e ´ecnicas de ep oducci´on humana asis ida.
•Ley 2/1974, de 13 de eb e o, sob e Colegios P o esionales.
ii

Bibliog aphy iii
•Ley 7/2006, de 31 de mayo, del eje cicio de p o esiones i uladas y de los colegios p o e-
sionales.
•Ley 14/1986, de 25 de ab il, Gene al de Sanidad.
•Ley 33/2011, de 4 de oc ub e, Gene al de Salud P´ublica.
•Ley 18/2009, del 22 de oc ub e, de salud p´ublica.
•Ley 15/1990, de 9 de julio, de o denaci´on sani a ia en Ca alu˜na.
•Real Dec e o Legisla i o 2/2015, de 23 de oc ub e, po el que se ap ueba el ex o
e undido de la Ley del Es a u o de los T abajado es.
•Real Dec e o Legisla i o 5/2015, de 30 de oc ub e, po el que se ap ueba el ex o
e undido de la Ley del Es a u o B´asico del Empleado P´ublico.
•Real Dec e o 190/1996, de 9 de eb e o, po el que se ap ueba el Reglamen o Peni en-
cia io.
•Dec e o 203/2015, de 15 de sep iemb e, po el que se c ea la Red de Vigilancia Epi-
demiol´ogica y se egulan los sis emas de no i icaci´on de en e medades de decla aci´on obli-
ga o ia y b o es epid´emicos.
•Dec e o 169/2015, de 21 de julio, po el que se es ablece el p ocedimien o pa a acili a
el conocimien o de los o ´ıgenes biol´ogicos.
•O den SSI/81/2017, de 19 de ene o, po la que se publica el Acue do de la Comisi´on
de Recu sos Humanos del Sis ema Nacional de Salud, po la que se ap ueba el p o ocolo
median e el que se de e minan pau as b´asicas des inadas a asegu a y p o ege el de echo
a la in imidad del pacien e po los alumnos y esiden es en Ciencias de la Salud.
Annex II: Colombian Regula ion on
Medical Reco ds
Lis o ele an laws ega ding medical eco ds in Colombia by hei i les in Spanish.
Legal
Ins umen
Numbe Yea A icle Ti le
Cons i uci´on
Pol´ı ica de
Colombia
N.A 1991 15 T´ı ulo 2. De los de echos, ga an ´ıas y
debe es. Cap´ı ulo 1: De los de echos
undamen ales, a ´ıculo 15: De echo a
la in imidad
Ley 23 1981 ´
E ica m´edica. Cap´ı ulo III: De la
p esc ipci´on m´edica, la his o ia
cl´ınica, el sec e o p o esional y
algunas conduc as
Ley 527 1999 Sob e come cio elec ´onico
Resoluci´on 1995 1999 No ma pa a el manejo de his o ia
cl´ınica, los anexos a ella y el
consen imien o in o mado
Ley 594 2000 Inco po a odos los acue dos
p omulgados po el A chi o Gene al
de la Naci´on
Resoluci´on 3374 2000 Minsalud es ablece los da os b´asicos
ele an es que deben gene a los
p es ado es de salud
Ley 594 2000 Se es ablecen las eglas y p incipios
gene ales que egulan la unci´on
a chi ´ıs ica del Es ado
Dec e o 2200 2005 En su cap´ı ulo IV: P esc ipci´on de
medicamen os
Resoluci´on 2346 2007 Minsalud egula la p ´ac ica de
e aluaciones m´edicas ocupacionales y
el con enido de sus his o ias cl´ınicas
ix
Bibliog aphy x
Legal
Ins umen
Numbe Yea A icle Ti le
Resoluci´on 1918 2009 Minsalud modi ica los a ´ıculos 11 y
17 de la esoluci´on 2346 de 2007 en
cuan o a la con a aci´on de se icios
de e aluaci´on m´edica ocupacional y la
cus odia y en ega de las e aluaciones
m´edicas e his o ias cl´ınicas
ocupacionales
Ley 1581 2012 No mas gene ales pa a p o ecci´on de
da os pe sonales
Dec e o 2364 2012 Reglamen a el a ´ıculo 7 de la ley 527
de 1999 e e ido a la i ma elec ´onica
Ley 1712 2014 Ley de anspa encia y del de echo de
acceso a la in o maci´on p´ublica
nacional
Ley 1955 2019 246 Plan Nacional de Desa ollo
2019-2022. In e ope a i idad his o ia
cl´ınica
Resoluci´on 2003 2014 Minsalud es ablece los p ocedimien os
y condiciones de insc ipci´on y
habili aci´on de los p es ado es de
se icios de salud
Dec e o 1074 2015 ´
Unica eglamen a ia del sec o
come cio, indus ia y u ismo,
cap´ı ulo 25 del ´ı ulo 2 y la pa e 2
Dec e o 1080 2015 2.8.2.5.1
al
2.8.2.8.3
T´ı ulo 2, cap´ı ulo V, ges i´on de
documen os. Obliga a las en idades
p i adas que p es en se icios de
ca ´ac e p´ublico y a las en idades
p´ublicas
Resoluci´on 839 2017 Es ablece el manejo, cus odia, iempo
de e enci´on, conse aci´on y
disposici´on inal de las his o ias
cl´ınicas, as´ı como los manejos que se
deben da en el Sis ema Gene al de
Segu idad Social en Salud, en caso de
liquidaci´on de alguna de sus en idades
Bibliog aphy xi
Legal
Ins umen
Numbe Yea A icle Ti le
Ley 2015 2020 Po la cual se c ea la his o ia cl´ınica
elec ´onica in e ope able y se obliga a
los p es ado es de se icios de salud a
diligencia y dispone los da os,
documen os y expedien es de la
his o ia cl´ınica en la pla a o ma de
in e ope a i idad que disponga el
Gobie no Nacional
Resoluci´on 866 2021 Minsalud eglamen a el conjun o de
elemen os de da os cl´ınicos ele an es
pa a la in e ope a i idad de las
his o ias cl´ınicas elec ´onicas
Plan de abajo N.A 2023 Minsalud es ablece el p oceso de
despliegue del plan de
in e ope a i idad de las his o ias
cl´ınicas de mane a escala , modula y
po ases
Lineamien o
diagn´os ico TIC
N.A 2023 Diagn´os ico de las capacidades TIC
en ecu sos ecnol´ogicos, humanos y
de p ocesos en los p es ado es de
se icios de salud
Plan de abajo N.A 2023 Plan de adopci´on e i o ial de la
in e ope a i idad de la his o ia
cl´ınica elec ´onica (IHCE)
Plan de abajo
e i o ial
N.A 2023 Lineamien os pa a la o mulaci´on del
plan e i o ial de in e ope a i idad
de his o ias cl´ınicas elec ´onicas
(IHCE)
Plan de abajo N.A 2023 Lineamien os ´ecnicos pa a la
ope aci´on
Consul a p´ublica N.A 2023 Recibo de p opues as y obse aciones
de la comunidad