Con inuous Assessmen -D i en Requi emen
Elici a ion o T us wo hy AI Sys ems
Anna Fedo o a1(
), Wol gang S e ani1, Ch is oph Hei z1, and Rica do
Cha a iaga1
Zu ich Uni e si y o Applied Science, Cen e o A i icial In elligence, Win e hu ,
Swi ze land [email p o ec ed]
Abs ac . As AI sys ems inc easingly impac c i ical decisions, e hi-
cal p inciples should be in eg a ed h oughou he en i e de elopmen
li ecycle. This is pa icula ly ele an when hese echnologies in luence
high-s akes decisions and c i ical in as uc u e. Regula o y e o s em-
phasize he need o adop an e hics-by-design app oach, i.e., embedding
e hics om he ea lies s ages o de elopmen . Howe e , mos assessmen
amewo ks a e in ended o use a e de elopmen is comple e, which
con lic s wi h his app oach. Relying solely on ex-pos e alua ions isks
o e looking e hical issues ha should be add essed ea lie . This high-
ligh s he need o assessmen me hods sui able o ea ly and con inuous
e alua ion h oughou he sys em li ecycle. We p opose an assessmen
app oach ha di e en ia es be ween e hical conce ns and isks ha a e:
(1) wi hin he cu en scope, (2) ele an a la e s ages, and (3) no
ele an . This di e en ia ion enables s age-app op ia e, e ec i e appli-
ca ion o e hics-by-design p inciples. We demons a e his in a p ac ical
use case, whe e we adap he EU Assessmen Lis o T us wo hy A i i-
cial In elligence (ALTAI) o elici posi i e unc ional and non- unc ional
equi emen s du ing he design phase o an AI-assis ed decision-making
sys em o ailway managemen . We show how he me hod elici s a lis
o Technology Readiness Le els (TRL) sensi i e equi emen s. We also
p o ide sugges ions o imp o e us wo hiness assessmen ools sui able
o ea ly-s age e alua ion.
Keywo ds: us wo hy AI ·e hical amewo ks ·e hics-by-design
1 In oduc ion
As a i icial in elligence (AI) sys ems become inc easingly embedded in high-
s akes and sa e y-c i ical applica ions, ensu ing hei e hical and us wo hy de-
sign has become a main conce n. T us wo hiness assessmen ools ha e eme ged
as p ac ical ins umen s o guide de elope s and s akeholde s in aligning AI sys-
ems wi h e hical p inciples (c. ., [1], [2]). Re iews o exis ing amewo ks can
be ound in [3], [4].
One such ool is he Assessmen Lis o T us wo hy A i icial In elligence
(ALTAI), de eloped by he Eu opean Commission as a amewo k o ex-pos
2 Au ho s Supp essed Due o Excessi e Leng h
sel -assessmen o AI sys ems’ us wo hiness [5]. Al hough ALTAI was o igi-
nally in ended o use a e a sys em’s de elopmen , i is inc easingly applied in
ex-an e con ex s [6], [7].
When applied du ing ea ly de elopmen s ages - such as du ing p oo -o -
concep o low Technology Readiness Le el (TRL) phases - he use o assessmen
ools designed o ex-pos assessmen s may ail o cap u e isks ha only mani es
in la e s ages. This can lead o incomple e o misleading assessmen s and may
p e en he e ec i e in eg a ion o e hical p inciples ac oss he ull AI li ecycle.
The Eu opean Commission, among o he s, emphasizes he impo ance o e hics-
by-design [8], which in ol es embedding e hical conside a ions om he ea lies
s ages o de elopmen , a he han add essing hem ex-pos , i.e., once he sys em
is ully implemen ed.
We a gue ha sel -assessmen ools mus be designed o be applied h ough-
ou he en i e design cycle, ins ead o ocusing on ex-pos assessmen s. Conse-
quen ly, hese assessmen ools mus be designed o explici ly acknowledge he
p ojec scope and in ended TRL, as well as suppo he an icipa ion o isks ha
may mani es a la e s ages o he cycle o highe TRL and p o ide guidance
ha is app op ia e o ha s age. By doing so, hey should p o ide guidance ha
is app op ia e o he cu en s age o he li ecycle, and guide po en ial ac ions
o be aken a la e s ages. Fo his pu pose, hey mus di e en ia e be ween
elemen s and cha ac e is ics ha a e (1) ele an and ac ionable a he cu en
s age, (2) likely o eme ge in u u e phases, which should be an icipa ed, and (3)
cu en ly ou o scope.
We illus a e his by iden i ying limi a ions on he use o ALTAI o ea ly-
s age assessmen s and p opose a me hodology o suppo s age-app op ia e anal-
ysis o he us wo hiness o AI sys ems. This pape p esen s a me hodology o
ansla ing us wo hy assessmen ques ions in o posi i e unc ional and non-
unc ional equi emen s sui able o ea ly s ages o he de elopmen li ecycle,
hus enabling T us wo hiness-by-design. We illus a e i s use in a use case o
AI-assis ed decision-making in ailway scheduling.
1.1 The Assessmen Lis o T us wo hy AI (ALTAI)
The High-Le el Expe G oup on A i icial In elligence, appoin ed by he Eu o-
pean Commission, de eloped E hics Guidelines o T us wo hy A i icial In el-
ligence [9]. These guidelines a e suppo ed by he Assessmen Lis o T us wo -
hy AI (ALTAI), a comp ehensi e sel -assessmen ool based on his amewo k
o achie ing T us wo hy AI (TAI). ALTAI is s uc u ed a ound se en key e-
qui emen s: Human Agency and O e sigh : Ensu ing AI sys ems suppo
human decision-making and au onomy, wi h adequa e human o e sigh mech-
anisms. Technical Robus ness and Sa e y: Add essing esilience, secu i y,
accu acy, eliabili y, and all-back plans. P i acy and Da a Go e nance:
Ensu ing da a p o ec ion and go e nance align wi h GDPR. T anspa ency:
Emphasizing aceabili y, explainabili y, and communica ion. Di e si y, Non-
disc imina ion, and Fai ness: A oiding bias, ensu ing accessibili y, and in-
ol ing s akeholde s. Socie al and En i onmen al Well-being: Conside ing
Ti le Supp essed Due o Excessi e Leng h 3
he impac on socie y, en i onmen , democ acy, and he wo king en i onmen ,
including changes in wo king a angemen s and skills o people in e ac ing wi h
an AI sys em. Accoun abili y: Ensu ing audi abili y and isk managemen . To
a oid con usion wi h he echnical equi emen s o he use cases, we will e e o
he ALTAI equi emen s u he in his pape as dimensions o us wo hiness.
The ALTAI-based assessmen equi es going h ough a se o yes/no ques ions
on each o he dimensions and hei subsec ions. The ALTAI ques ionnai e is
a ailable as a checklis and an online in e ac i e ool [5]. The ool is in ended
o lexible use, o ganiza ions a e expec ed o adap i o hei speci ic needs
and sec o s. I s guidelines speci ically men ion ha he assesso s migh need
addi ional ques ions o complemen he assessmen depending on he applica ion.
I encou ages a mul idisciplina y app oach, in ol ing a ious s akeholde s such
as AI designe s, da a scien is s, legal expe s, and managemen .
2 Requi emen elici a ion based on ALTAI
We ex end he ALTAI ool o enable assessmen s applicable a di e en s ages
o he li ecycle. The ex ended me hodology allows no only o assess he us -
wo hiness dimensions, bu also elici a ion o unc ional and non- unc ional e-
qui emen s explici ly ela ed o hese dimensions.
We ollowed an i e a i e app oach, s a ing wi h an ini ial desc ip ion o
he use case. The use case desc ip ion mus be de ined by a mul idisciplina y
se o s akeholde s o include domain-speci ic knowledge om ele an a eas.
I desc ibes he AI sys ems’ asks and p o ides an o e iew o he necessa y
unc ions and asks o he sys em.
As a s a ing poin , we use a use case desc ip ion ollowing s anda ds IEC
PAS 62559 [10] and ISO/IEC 24030:2024 [11]. The de ailed desc ip ion o he use
case me hodology is ou o scope o his pape and can be ound in [12]. Al e -
na i e me hods can be used o de ine he use case and i s con ex o subsequen
equi emen elici a ion.
The use case is hen analyzed ollowing he ALTAI s uc u e o iden i y
ele an us wo hiness dimensions ha need o be add essed a he cu en o
la e li ecycle s ages (See Fig. 1).
Nex , based on he desc ip ion o he use cases, he g oup o s akeholde s
iden i y equi emen s o he unc ions based on he business needs, wi hou ex-
plici ly iden i ying any speci ic echnologies o p oduc s. The same documen
can also co e ’non- unc ional’ equi emen s, such as cons ain s, pe o mance,
secu i y, and da a in e ac ions wi h o he applica ions o sys ems [10].
The nex s age o he p ocess is o pe o m he us wo hiness assessmen
based on he cu en use case desc ip ion using ALTAI. This allows o iden-
i y new unc ional and non- unc ional equi emen s in an i e a i e p ocess by
analyzing esponses o he ALTAI ques ionnai e. The p ocess s a s wi h he
de ini ion o he scope o analysis, including he s age o he li e-cycle and in-
ended TRL. Fo each o hese ques ions, s akeholde s mus de e mine whe he
i is "Rele an o he scope", "Rele an o he ull sys em" o "No ele an ".
4 Au ho s Supp essed Due o Excessi e Leng h
Fig. 1. Me hodology o elici ing equi emen s based on us wo hy assessmen .
This dis inc ion is ou ex ension o he me hod desc ibed in [7], and i allows
di e en ia ion du ing he assessmen be ween he elemen s o cha ac e is ics o
he sys em, which a e no conside ed in he ull use case, hose ha mus be
conside ed a he cu en li ecycle s age o scope, and hose ele an o a ully
implemen ed sys em.
In an i e a i e p ocess, o each ques ion, he i s s ep is o assess whe he is
ele an o he scope o analysis (i.e., li ecycle s age and TRL). In he a i ma i e
case, i is ma ked as "Rele an o he scope". I i is no ele an o he scope,
he g oup should assess i s ele ance o he ull scope o he use case. I he
ques ion becomes ele an a a la e phase o o sys ems wi h highe TRL, i
is ma ked as "Rele an o he ull sys em". This p ocess is epea ed o all
assessmen ques ions.
The nex s ep ocuses on elici ing he echnical and non- echnical equi e-
men s. Fi s , i mus be checked i exis ing equi emen s al eady add ess he
ele an ques ions. In his case, no new equi emen will be gene a ed. O h-
e wise, a new equi emen is de ined. A e his, he use case documen a ion
is upda ed o explici ly link each equi emen o he ele an us wo hiness
ques ions.
Ti le Supp essed Due o Excessi e Leng h 5
Since he ALTAI ques ionnai e was designed o ex-pos analysis, when new
equi emen s a e gene a ed, i s ques ions we e eph ased in he o m o posi-
i e equi emen s. This me hodology allows us o keep accoun o he necessa y
unc ionali y and ensu e ha he AI sys em is aligned wi h he us wo hiness
dimensions. I s applica ion h oughou he li ecycle suppo s he equi emen o
a con inuous isk managemen included in he EU AI Ac [13].
3 Applica ion o a use case
We es ed he me hodology in a use case o AI-based assis ed decision making
o ailway ne wo k con ol. Since i co esponds o an applica ion in c i ical
in as uc u e, i has highe equi emen s o us wo hiness. This wo k was
pe o med as pa o he EU- unded p ojec AI4REALNET. Below, we p esen
a b ie desc ip ion o he use case. The ull desc ip ion is a ailable in [12].
Business p oblem: The demand o ailway ne wo k capaci y in e ms o
e iciency, ne wo k capaci y, a ic densi y and a ic managemen esilience is
g owing apidly. Simul aneously, en i onmen al awa eness esul s in changing
mobili y egula ions and he need o no el dispa ching echnologies.
Objec i e: AI-based solu ion mus enhance dispa che s’ capabili ies by as-
sis ing hem in escheduling ains o au oma ing decision-making p ocesses and
p o iding suppo o human decision-making in complex ope a ion scena ios.
The AI sys em should de ec de ia ions in eal ime, assess hei impac on he
o e all schedule, and p edic issues ha may a ise in he u u e.
The AI4REALNET p ojec aims a de eloping p oo o concep (POC) sys-
ems wi h a maximum TRL o 5 (i.e., limi ed unc ionali y compa ed o he ull
sys em). Tes s will be pe o med in a simula ed en i onmen ocusing on he
echnical easibili y.
3.1 Requi emen elici a ion
The use case desc ip ion was o mula ed by he domain expe s, indus y ep-
esen a i es and ele an s akeholde s. Nex , hey pe o med he ALTAI-based
assessmen s. The ques ionnai e was sha ed as a collabo a i e documen , which
enabled wo kshop pa icipan s, s akeholde s, de elope s, and o he con ibu-
o s o p o ide answe s o indi idual ques ions. A designa ed domain expe
managed he p ocess, ensu ing e ec i e coo dina ion. The collec ed inpu was
ca e ully e alua ed o assess he ele ance o each ques ion o he use case. The
assessmen esul s we e documen ed in a epo including ques ions, decisions
and suppo ing a gumen s. This documen summa izes he ou come in a able,
which included a ull lis o he ALTAI ques ions and co esponding columns:
Rele ance - ele ancy o he ques ion o he UC (Rele an o he sys em -
R ull, ele an o scope - R scope, No ele an - NR); Conside a ion – a
summa y o commen s ha s akeholde s had on he opic, including a ionale
o he decision; Measu e – i he ques ion is ele an , lis o equi emen s ha
add ess i . An exce p om his able is p esen ed in Table 1.
6 Au ho s Supp essed Due o Excessi e Leng h
Table 1. Exce p om he able wi h he T us wo hiness Assessmen .
Dimension: 3 - P i acy and Da a Go e nance
Ques ion: Is you AI sys em being ained, o was i de eloped, by using o p ocessing
pe sonal da a (including special ca ego ies o pe sonal da a)?
Rele ance Conside a ion Measu e
NR Simula ion en i onmen s a e used
o aining.
None
Dimension: 5 - Di e si y, Non-disc imina ion and Fai ness
Ques ion: Did you es ablish a s a egy o a se o p ocedu es o a oid c ea ing o
ein o cing un ai bias in he AI sys em, bo h ega ding he use o inpu da a as well
as o he algo i hm design?
Rele ance Conside a ion Measu e
R ull Bias a oidance mus be consid-
e ed du ing de elopmen and mon-
i o ed a e implemen a ion.
In oduce Fai ness equi emen s
(Fa-1 and Fa-2) o ensu e ha he
sys em ai ly dis ibu es una oid-
able delays h oughou he sys em
and does no un ai ly a o spe-
ci ic Railway Unde aking Ope a -
ing Manage s (RUOMs)
Dimension: 7 - Accoun abili y
Ques ion: Did you es ablish a p ocess o discuss and con inuously moni o and assess
he AI sys em’s adhe ence o his Assessmen Lis o T us wo hy AI (ALTAI)?
Rele ance Conside a ion Measu e
R scope Moni o ing p ocesses a e ou -o -
scope o he POC. They mus be
conside ed o he ull scope sys-
em.
No measu e o his scope
This s uc u e is simila o [7]. The in oduced no el y is he dis inc ion
be ween " ele an o ull sys em" and " ele an o scope". I allows keeping
ack o unc ionali ies planned o di e en e sions o he sys em and e alua ing
hei TRL acco ding o he ma u i y and le el o de ail o equi emen s.
The same s akeholde g oup, which de ined he use case desc ip ion, hen
analyzed exis ing equi emen s and iden i ied gaps, p oposing new equi emen s
whe e necessa y. All pe inen equi emen s we e inco po a ed in o he o icial
use case documen a ion. This sys ema ic e alua ion and documen a ion o e hi-
cal conside a ions se e o jus i y he decisions made wi hin he AI4REALNET
p ojec . An exce p om his lis is shown in Table 2.
The inal ou come o he p ocess is (i) an assessmen Table, whe e each
ele an ques ion is assigned a us wo hiness- ela ed equi emen , and (ii) an
upda ed lis o Use Case equi emen s.
3.2 Resul s o he assessmen
To e alua e he ole o us wo hy dimensions o di e en scopes, we show he
p opo ion o ques ions ma ked as ele an o he ull scope and o he in ended
Ti le Supp essed Due o Excessi e Leng h 7
Table 2. Example o elici ed equi emen s. The ull lis is a ailable in [14].
ALTAI dimension Subdimension AI4REALNET equi emen s
Di e si y, Non-
disc imina ion and
Fai ness
A oidance o Un-
ai Bias
The sys em should no un ai ly a o spe-
ci ic RUOMs. Rescheduling in ailway ope a-
ions mus impac he RUOMs ai ly. Measu es
should be pu in place o ensu e ha hese con-
s ain s a e obse ed.
Technical obus ness
and sa e y
Gene al sa e y Iden i y he sys em’s le els o exposu e o such
h ea s, bo h in e ms o quan i y and du a ion.
Accoun abili y Risk manage-
men
Pu in place by design mechanism in case o ap-
plica ions ha can ad e sely a ec indi iduals
in e ms no only o haza d bu also exposu e
and ulne abili y.
scope (POC) in Figu e 2. No ice ha we don’ epo he spide diag ams p o-
duced by he online ALTAI assessmen ool, as he e is no in o ma ion on how
exac ly he in o ma ion om esponses is used o de ine he isualiza ion.
We can see ha assessmen o he POC AI sys em yields a ela i ely small
numbe o ALTAI ques ions as ele an . Since he sys ems will be de eloped
and es ed in a simula ed en i onmen , us wo hiness cha ac e is ics, such as
secu i y, a e ma ked as no ele an o his scope as hey will depend on o he
elemen s o he whole sys em. Likewise, in his scope, sa e y in ex eme si ua ions
will no be es ed as he simula ed da a only includes he mos common scena ios
and es s o hese cases will be planned o la e s ages o sys ems wi h highe
TRL. O e all, he mos ele an dimensions o his scope a e Human Agency
and O e sigh , T anspa ency, Socie al and En i onmen al Well-being.
When analyzing he us wo hiness o he ull scope sys em, he ele ance o
all dimensions inc eases. 4 dimensions ha e o e 80% ele ance: Accoun abili y,
Technical Robus ness and Sa e y, P i acy and Da a Go e nance, T anspa ency.
In he ollowing pa ag aphs, we summa ize he esponses and conside a ions
ga he ed om he s akeholde s o each o he e hical dimensions making clea
he di e ence in he unc ionali ies o POC and ull use case.
Human Agency and O e sigh . The AI sys em’s in e ac ion wi h end-
use s in luences hei au onomy and decision-making p ocesses, in oducing he
isk o o e dependence. Ta ge ed aining o ensu e use s unde s and he sys-
em’s unc ionali y and limi a ions can mi iga e his isk. Al hough he AI does
no simula e human-like in e ac ion, i may s ill encou age compulsi e usage
pa e ns. In ailway ope a ions, a ying o e sigh models - such as Human-in-
he-Loop and Human-in-Command - equi e well-de ined p o ocols o allow sa e
human in e en ion when he AI is in con ol. O e sigh mechanisms should also
de ec undesi able ou comes and enable he managemen o he sys em’s adap-
i e lea ning beha io . These mechanisms mus al eady be es ed in he POC o
es ima e he impac on he ope a o s. These es should in o m decisions abou
ull-scale deploymen .
8 Au ho s Supp essed Due o Excessi e Leng h
Fig. 2. Compa ison o he numbe o ele an ques ions in he us wo hiness assess-
men o ull scope and he in ended scope (POC).
Technical Robus ness and Sa e y. While he p ojec add esses some o-
bus ness and sa e y dimensions, u he de ail is needed du ing ull sys em de-
ploymen (e.g., sa e y mechanisms like collision a oidance a e managed by a
sepa a e componen ). Resilience agains ad e sa ial h ea s is conside ed wi hin
he POC, while aspec s like ce i ica ion and long- e m secu i y s a egies a e
beyond he p ojec ’s scope. Ensu ing aul ole ance and ongoing human in ol e-
men in echnical e alua ions is c i ical. Sys em pe o mance mus be closely
moni o ed, and unce ain y in AI ou pu s should igge human e iew and con-
ol ans e mechanisms. Con inuous lea ning unc ions mus be documen ed
and in e p e able o p ese e bo h sys em eliabili y and human au ho i y.
P i acy and Da a Go e nance. Since he sys em does no p ocess pe -
sonal da a, immedia e p i acy issues a e limi ed. Howe e , as he sys em e ol es,
u u e s ages should include mechanisms o add ess po en ial p i acy isks. While
cu en da a go e nance complies wi h egula o y expec a ions, speci ic da a p i-
acy equi emen s a e deemed unnecessa y o he POC due o he non-sensi i e
na u e o he da a in use.
T anspa ency. Ensu ing aceabili y h ough eco d-keeping o sys em de-
cisions - including inpu s and ou pu s - is undamen al o anspa ency. The
explainabili y o AI beha io is p io i ized o help ope a o s g asp i s objec i es,
decision logic, and lea ning pa e ns. E ec i e communica ion be ween he AI
and human use s is essen ial o minimize misuse and es ablish us . Sys em ou -
pu s a e clea ly dis inguishable om human ac ions, wi h use s kep in o med
abou he sys em’s capabili ies and limi a ions a all imes.
Ti le Supp essed Due o Excessi e Leng h 9
Di e si y, Non-disc imina ion, and Fai ness. The AI sys em mus a oid
in oducing o ein o cing biases du ing i s de elopmen , pa icula ly in ensu ing
equi able ou comes such as neu al delay dis ibu ion ac oss ailway ope a o s.
Inclusi e s akeholde engagemen , including public and p o essional wo kshops,
ensu es ha de elopmen aligns wi h socie al expec a ions and use needs.
Socie al and En i onmen al Well-being. The sys em can po en ially
b ing en i onmen al bene i s by imp o ing ope a ional e iciency. I s deploymen
will also in luence wo k o ce oles and skill equi emen s, which mus be consid-
e ed in he design. Collabo a ion wi h end-use s and human ac o s specialis s
is encou aged o add ess hese impac s. While he sys em will equi e new skills,
he de elopmen o ela ed aining p og ams alls ou side he POC scope.
Accoun abili y. Sys em accoun abili y is suppo ed h ough exhaus i e doc-
umen a ion and logging p ac ices, which a e key o e ospec i e analysis and
pe o mance acking. Al hough comp ehensi e isk managemen planning is no
pa o POC scope, exis ing logging in as uc u e p o ides a s ong basis o u-
u e e hical o e sigh and accoun abili y mechanisms.
4 Recommenda ions o imp o ing he us wo hy
assessmen ools
C ea e ques ionnai es ailo ed o di e en de elopmen s ages o he
AI li ecycle. Conduc ing a us wo hiness assessmen om he ea ly s ages
and h oughou he en i e li e cycle o de elopmen is bene icial o en isioning
measu es o ensu e us wo hiness-by-design. Howe e , ALTAI was de eloped
as a pos -hoc assessmen , and he o mula ion o ques ions e lec s i .
Ou app oach o con inuous us wo hiness assessmen is compa ible wi h
ele an egula o y equi emen s. The EU AI ac , A 9. s a es ha isk man-
agemen "shall be unde s ood as a con inuous i e a i e p ocess planned and
un h oughou he en i e li ecycle o a high- isk AI sys em, equi ing egu-
la sys ema ic e iew and upda ing" [13]. Likewise, he Gene al Da a P o ec ion
Regula ion (GDPR), A . 35, equi es ha Da a P o ec ion Impac Assessmen s
mus be conduc ed "p io o p ocessing" and con inuous egula assessmen s a e
equi ed du ing he li ecycle as means o add ess any po en ial isks [15].
ALTAI-like ques ionnai es o checklis s shall be adap ed o become ac ion-
able ins umen s o adop hese equi emen s in o he de elopmen p ocedu e.
Fo his pu pose, he assessmen should be designed so as o add ess speci ic
challenges a each s age o he li ecycle [16]. These assessmen s should be con-
duc ed as ea ly as possible and epea ed a e any signi ican change he sys em
unde goes. The iden i ica ion and mi iga ion o isks secu ed in his way should
also be consis en wi h es ablished s anda ds (e.g., ISO/IEC 23894 [17]).
Dis inguish be ween non- ele an and ou -o -scope unc ionali y.
As we expe ienced in AI4REALNET p ojec , some ea u es may no be planned
o he cu en de elopmen scope bu a e o eseen o la e implemen a ions.
I he us wo hiness assessmen is pe o med be o e he AI sys em is inalized
his unc ionali y will no be co e ed by ALTAI. Ano he ac o ha in luences
