The NORMAN Association and the European Partnership for Chemicals Risk Assessment (PARC): let’s cooperate!

Dulioe al. En i on Sci Eu (2020) 32:100
h ps://doi.o g/10.1186/s12302-020-00375-w
COMMENTARY
The NORMAN Associa ion and heEu opean
Pa ne ship o Chemicals Risk Assessmen
(PARC): le ’s coope a e!
Vale ia Dulio1* , Jan Koscho eck2*, Be an Ba el3, Paul an den B ink4, Juliane Hollende 5,
John Mun he6, Ma in Schlabach7, Reza Aalizadeh11, Ma lene Age s and9, Lu z Ah ens10, Ian Allan3,
Niki o os Alygizakis8,11, Damia’ Ba celo’12, Pe nilla Bohlin‑Nizze o7, Susanne Bou oup13, We ne B ack14,15,
Adèle B essy16, Jan H. Ch is ensen17, Lubos Ci ka8, Ad ian Co aci18, Anja De ksen19, Gene iè e De ille 20,
Milou M. L. Dingemans21,22, Magnus Engwall23,24, Despo Fa a‑Kassinos25, Pablo Gago‑Fe e o26,
Félix He nández27, Do e He zke7, Klá a Hilsche o á28, Henne Holle 15, Ma ion Junghans29,
Ba ba a Kasp zyk‑Ho de n30, S e en Kei e 24, S e an A. E. Kools21, Anneli K u e9, Dimi a Lamb opoulou31,
Ma ja Lamo ee32, Pim Leona ds32, Benjamin Lopez33, Mi en López de Alda12, Lian Lundy34,50,
Ja mila Mako inská35, Ionan Ma igómez36, Jona han W. Ma in9, B endan McHugh37, Cécile Miège38,
Simon O’Toole39, Noo a Pe kola40, S e ano Polesello41, Leo Pos huma42,43, Sa a Rod iguez‑Mozaz26,
I o Roessink4, Pawel Ros kowski7, Heinz Ruedel44, Sae Samanipou 45, Tobias Schulze14, Emma L. Schymanski46,
Man ed Sengl47, Pe e Ta ábek35, Do ien Ten Hulsche 48, Nikolaos Thomaidis11, Anne Togola33, Sa a Valsecchi41,
S e an an Leeuwen4, Pe e on de Ohe2, Ka in Vo kamp49, B anisla V ana28 and Ja osla Slobodnik8*†
Abs ac
The Pa ne ship o Chemicals Risk Assessmen (PARC) is cu en ly unde de elopmen as a join esea ch and inno a‑
ion p og amme o s eng hen he scien i ic basis o chemical isk assessmen in he EU. The plan is o b ing chemi‑
cal isk assesso s and manage s oge he wi h scien is s o accele a e me hod de elopmen and he p oduc ion o
necessa y da a and knowledge, and o acili a e he ansi ion o nex ‑gene a ion e idence‑based isk assessmen , a
non‑ oxic en i onmen and he Eu opean G een Deal. The NORMAN Ne wo k is an independen , well‑es ablished and
compe en ne wo k o mo e han 80 o ganisa ions in he ield o eme ging subs ances and has eno mous po en ial
o con ibu e o he implemen a ion o he PARC pa ne ship. NORMAN s ands eady o p o ide expe ad ice o
PARC, d awing on i s long expe ience in he de elopmen , ha monisa ion and es ing o ad anced ools in ela ion o
chemicals o eme ging conce n and in suppo o a Eu opean Ea ly Wa ning Sys em o un a el he isks o con ami‑
nan s o eme ging conce n (CECs) and close he gap be ween esea ch and inno a ion and egula o y p ocesses. In
his commen a y we highligh he ools de eloped by NORMAN ha we conside mos ele an o suppo ing he
© The Au ho (s) 2020. This a icle is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License, which pe mi s use, sha ing,
adap a ion, dis ibu ion and ep oduc ion in any medium o o ma , as long as you gi e app op ia e c edi o he o iginal au ho (s) and
he sou ce, p o ide a link o he C ea i e Commons licence, and indica e i changes we e made. The images o o he hi d pa y ma e ial
in his a icle a e included in he a icle’s C ea i e Commons licence, unless indica ed o he wise in a c edi line o he ma e ial. I ma e ial
is no included in he a icle’s C ea i e Commons licence and you in ended use is no pe mi ed by s a u o y egula ion o exceeds he
pe mi ed use, you will need o ob ain pe mission di ec ly om he copy igh holde . To iew a copy o his licence, isi h p://c ea i eco
mmons .o g/licen ses/by/4.0/.
Open Access
*Co espondence: [email p o ec ed]; jan.koscho [email p o ec ed];
[email p o ec ed]†
1 INERIS, Na ional Ins i u e o En i onmen and Indus ial Risks, Ve neuil
en Hala e, F ance
2 UBA, Fede al En i onmen Agency, Dessau‑Roßlau, Ge many
8 En i onmen al Ins i u e, Koš, Slo akia
Full lis o au ho in o ma ion is a ailable a he end o he a icle
Page 2 o 11
Dulioe al. En i on Sci Eu (2020) 32:100
Backg ound
The PARC pa ne ship is cu en ly unde de elopmen
as a join esea ch and inno a ion p og amme o sup-
po he Eu opean Commission (EC) and na ional
chemical isk assessmen and managemen au ho i-
ies by p o iding new e idence and me hodologies and
p omo ing hei up ake in egula o y p ocesses. This
applies o cu en ly ecognised as well as po en ial
u u e con aminan s o eme ging conce n (CECs).
O e he pas 15yea s, he NORMAN Associa ion
has de eloped a ne wo k o expe o ganisa ions in he
ield o CECs in he en i onmen . This e o i s well
wi h he aims and s uc u e o he PARC ini ia i e [1]
o a Eu opean Pa ne ship o Chemicals Risk Assess-
men . I is he e o e no su p ise ha NORMAN ully
suppo s he “e olu ion, no e olu ion” p inciple o
he PARC ini ia i e, and ag ees ha exis ing da a and
me hods should be in eg a ed and u he de eloped
wi hou “ ein en ing he wheel”, as we explain in mo e
de ail in his pape .
In he PARC pa ne ship, he plan is o join o ces
wi h scien is s o c ea e he nex gene a ion o chemical
isk assessmen and o acili a e ‘The Eu opean G een
Deal’ [2], which includes he sus ainable managemen
o chemicals o a non- oxic en i onmen . An essen ial
ole o his pa ne ship is o os e be e use o exis -
ing knowledge and da a, and be e coope a ion and
coo dina ion o esea ch on he egula o y needs—all
in o de o imp o e isk assessmen and managemen ,
including he de elopmen o an EU ea ly wa ning sys-
em o eme ging isks o chemicals in he en i onmen .
The Eu opean Commission acknowledges he impo -
ance o con inuously imp o ing knowledge abou he
(eco) oxici y o chemicals and o adequa ely add essing
unce ain ies ega ding exposu e o chemicals [3, 4].
Mo eo e , cu en egula ions a e no su icien ly e ec-
i e o ackle CECs and chemical isks in gene al, since
a holis ic iew is missing and he e a e o en incon-
sis encies be ween di e en use sec o s [5]. The ision
o u u e chemicals policy is ha chemicals should be
deal wi h in an in eg a ed manne in an o e a ching
chemicals policy amewo k co e ing all ypes o chem-
icals and all uses, beyond he cu en sec o -speci ic
egula ions.
I is in his con ex ha he NORMAN ne wo k came
in o exis ence in 2005 as a p ojec , ollowing a call by
he EC (DG Resea ch) aimed a c ea ing a pe manen
pla o m o educe knowledge gaps and be e mee he
equi emen s o isk assesso s and isk manage s con-
ce ning CECs [6]. NORMAN is oday an independen ,
sel - unded, non-p o i , mul idisciplina y and mul ina-
ional associa ion in he ield o CECs in he en i on-
men , which b ings oge he mo e han 80 o ganisa ions
ep esen ing a ious s akeholde s, including compe en
au ho i ies, na ional e e ence labo a o ies, esea ch cen-
es, academia and indus y—mos ly in Eu ope, bu also
in No h Ame ica and Asia [7].
The missions o NORMAN a e o: (i) acili a e a mo e
apid and wide exchange o da a on he iden i y, occu -
ence and e ec s o CECs in wa e , bio a, ai , soil and
indoo en i onmen ; (ii) imp o e da a quali y and com-
pa abili y ia alida ion and ha monisa ion o common
sampling and measu emen me hods (chemical and
biological), and (iii) p o ide ools o he isk and haz-
a d assessmen o CECs [6]. Since he p ima y objec i e
o he NORMAN Associa ion is o ac as a science- o-
policy in e ace, he ou comes o he ne wo k’s ac i i-
ies a e egula ly sha ed wi h he EC’s se ices including
DG En i onmen , Eu opean Chemicals Agency (ECHA),
Eu opean En i onmen Agency (EEA), EC Join Resea ch
Cen e (JRC), in e na ional i e commissions, egional
sea con en ions and na ional egula o y bodies. Cu -
en ly, nine na ional egula o y agencies a e membe s o
he NORMAN Associa ion.
The NORMAN Associa ion has conside able—and
con inuously de eloping—expe ience o es ablishing: (i) a
consolida ed ne wo k o closely coope a ing labo a o ies
ac i e in esea ch o suppo chemical isk assessmen
and managemen ; (ii) a join , use - iendly and open-
access da a space o sha e knowledge on CECs in he
en i onmen and p omo e ha monised p o ocols o da a
collec ion and epo ing; (iii) a collabo a i e amewo k
PARC ini ia i e: (i) join da a space and cu ing‑edge esea ch ools o isk assessmen o con aminan s o eme ging
conce n; (ii) collabo a i e Eu opean amewo k o imp o e da a quali y and compa abili y; (iii) ad anced da a analysis
ools o a Eu opean ea ly wa ning sys em and (i ) suppo o na ional and Eu opean chemical isk assessmen hanks
o ha nessing, combining and sha ing e idence and expe ise on CECs. By combining he ex ensi e knowledge and
expe ience o he NORMAN ne wo k wi h he inancial and policy‑ ela ed s eng hs o he PARC ini ia i e, a la ge s ep
owa ds he goal o a non‑ oxic en i onmen can be aken.
Keywo ds: NORMAN ne wo k, Suspec sc eening, Non‑ a ge sc eening, Con aminan s o eme ging conce n,
En i onmen al moni o ing, High‑ esolu ion mass spec ome y, E ec ‑based me hods, Chemical isk assessmen and
p io i isa ion
Page 3 o 11
Dulioe al. En i on Sci Eu (2020) 32:100
o os e alida ion and ha monisa ion o measu e-
men me hods and moni o ing ools; (i ) ad anced da a
analysis ools o deal wi h less-in es iga ed subs ances
in suppo o a Eu opean ea ly wa ning sys em o de ec
eme ging chemical isks o he en i onmen , and ( ) a
sys em o ha nessing, combining and sha ing expe ise
among esea ch eams, na ional e e ence labo a o ies
and en i onmen al agencies in inno a i e me hods in
suppo o chemical isk assessmen . So a , NORMAN
has been s ongly in ol ed in CECs in he esh wa e
aqua ic en i onmen and he associa ed EU policies. The
ocus has ecen ly ex ended beyond esh wa e o he
indoo , ma ine, soil and e es ial en i onmen s and
wa e euse, while he scope o CECs is also expanding
o include addi ional pa ame e s such as an ibio ic esis -
ance de e minan s and mic oplas ics.
In his opinion pape we would like o highligh he
ools de eloped by NORMAN ha we conside mos el-
e an o suppo he PARC ini ia i e.
Join da a space andcu ing‑edge esea ch ools
o  isk assessmen o con aminan s o eme ging
conce n
NORMAN Da abase Sys em (NDS): da a ga he ing andda a
managemen
Pe spec i e and ecommenda ions
Con inue o de elop he NORMAN Da abase Sys em
(NDS) as a e e ence da abase ha b ings oge he , in a
single pla o m, widely di e ing chemical moni o ing
da a acqui ed using a ious echniques and in di e -
en ma ices, he eby ensu ing a ha monised app oach
o da a collec ion, s o age, quali y con ol, cu a ion and
exchange among NORMAN membe s and mo e widely.
Fu u e pla o m de elopmen will be guided by he FAIR
p inciples (Findabili y, Accessibili y, In e ope abili y, and
Reuse o da a).
The NDS is complemen a y o he EC In o ma ion
Pla o m o Chemical Moni o ing (IPCHEM) [8, 9] in
ha es ing chemical a ge moni o ing da a, while a he
same ime pa ing he way o he de elopmen o a new
Eu opean in as uc u e o handling da a coming om
inno a i e me hods, such as non- a ge sc eening (NTS)
and e ec -based me hods (EBM). I should con inue in
ha ole.
Ra ionale
The c ucial ask o ga he ing and managing en i on-
men al CEC exposu e da a o suppo chemical isk
assessmen has been he co e ac i i y o he NORMAN
Associa ion om i s s a in 2005.
The cu en NDS [10] is an open-access pla o m o
in e connec ed da abases able o assis e ec i e and
apid sc eening and isk assessmen o con aminan s in
he en i onmen .
The unique ea u e o he NDS is ha i p o ides a com-
p ehensi e se o da a on CECs oge he wi h a ange o
inno a i e applica ions o hei haza d and isk assess-
men . These ools ange om physico-chemical p ope -
ies, use cha ac e is ics, mass spec al in o ma ion, and
exposu e da a om a ge and non- a ge sc eening in
all en i onmen al compa men s, o eco oxici y da a
and insi u bioassay signals e lec ing mix u e oxici y.
The NDS cu en ly consis s o 12 modules (Fig.1), o
which ele en (Subs ance Da abase (SusDa ); Suspec Lis
Exchange (SLE); Chemical Occu ence Da a (EMPO-
DAT); Eco oxicology; Bioassays Moni o ing Da a; Mass-
Bank Eu ope; Digi al Sample F eezing Pla o m (DSFP);
Indoo En i onmen ; Passi e Sampling; Subs ance
Fac shee s; P io i isa ion) a e accessible, in e linked and
popula ed wi h da a. The 12 h is an an ibio ic- esis -
an bac e ia and genes module (ARB&ARG) ha is s ill
unde de elopmen , while a new module hos ing da a on
mic oplas ics is cu en ly being designed.
A selec ion o he NDS modules mos ele an o PARC
is p esen ed below.
NORMAN Subs ance Da abase: acommon lis
o subs ances o ha monised chemical isk assessmen
Pe spec i es and ecommenda ions
Fu he de elop he Subs ance Da abase (SusDa ) as
he co ne s one o a common Eu opean pla o m whe e
in o ma ion on highly ele an and newly disco e ed
en i onmen al pollu an s can be sha ed in a ha monised
o ma [11].
Ra ionale
A common, ha monised lis o chemical compounds
sha ed among all pa ies in esea ch and egula ion is one
c i ical equi emen o enhanced coope a ion among
exis ing egula o y amewo ks and shi ing owa ds
a “one chemical, one assessmen ” pa adigm. Howe e ,
cu en chemicals lis s a e agmen ed collec ions, wi h
esea che s and egula o s all using hei own lis s.
We belie e ha he combina ion o NORMAN Sus-
pec Lis Exchange (SLE) [12] and he me ged NORMAN
Subs ances Da abase [13] o he NDS could be a globally
leading model o collabo a i ely wo king owa ds such a
lis . Nume ous o ganisa ions, na ional and in e na ional
egula o y agencies and esea ch g oups om Eu ope
and No h Ame ica al eady con ibu e o his ini ia i e.
NORMAN SLE is a pla o m o sha e lis s o subs ances
po en ially esponsible o eme ging isks o ecosys ems
and human heal h. The submi ed lis s a e sha ed wi h
US EPA CompTox Chemicals Dashboa d [14, 15] and
PubChem [16, 17] and a e published on Zenodo [18]. By
Page 4 o 11
Dulioe al. En i on Sci Eu (2020) 32:100
ac ing as a da a collec o , he NORMAN SLE has become
an impo an sou ce o specialised esea ch in o ma-
ion o majo chemical da abases such as PubChem and
CompTox, beyond he ealms and means o indi idual
esea che s. In e u n, he in eg a ion o he NORMAN
SLE in o majo chemical da abases adds eno mous alue
o he o iginal con ibu ions, o e ing up new unc ional-
i y o all pa ies.
The me ged lis (wi hou duplica es) is known as
NORMAN SusDa [13]—a cu a ed compound da abase
(65,697 compounds as o Ap il 2020), whe e subs ances
a e me ged by he S anda d InChIKey, which ac s as he
unique iden i ie . This is accompanied by o he s uc-
u al in o ma ion such as CAS numbe s and SMILES,
as well as physico-chemical p ope ies. SusDa also
con ains mappings o he equi alen “MS Ready” o ms
[19], as well as o he mass spec ome ic in o ma ion o
he iden i ica ion o compounds wi h NTS echniques,
es ima ed (in silico) P edic ed No-E ec Concen a-
ions (PNECs), and o he in o ma ion equi ed o he
p io i isa ion and isk assessmen o subs ances. Since
2016, SusDa has been used o in e link all NORMAN
da abases among hemsel es, as well as he NDS wi h
majo ex e nal da abases.
NORMAN Eco oxicology Da abase: acommon pla o m
o eco oxici y assessmen
Pe spec i es and ecommenda ions
Es ablish a co e eam o eco oxicology expe s, om
EU Membe S a es and globally, using he Eco oxicol-
ogy Da abase as a basis o e alua e he eliabili y and
ele ance o eco oxici y s udies and each consensus on
Quali y S anda ds (i.e. PNEC alues) o a mo e ha mo-
nised isk assessmen o chemicals.
Ra ionale
We p opose o sha e he NORMAN Eco oxicology Da a-
base [20] o ha monised eco oxici y assessmen wi hin
he PARC pa ne ship. The da abase p o ides a anspa -
en ool o guide expe s in: (i) he iden i ica ion o he
Fig. 1 The NORMAN Da abase Sys em (NDS)
Page 5 o 11
Dulioe al. En i on Sci Eu (2020) 32:100
eliable eco oxici y s udies, based on he CRED (C i e ia
o Repo ing and E alua ing eco oxici y Da a) classi ica-
ion sys em [21]; (ii) he online de i a ion o a se o qual-
i y s anda ds o each ma ix and egula o y amewo k
based on selec ed ‘ eliable’ eco oxici y s udies, using a
buil -in so wa e ool implemen ing he equi emen s
o he EC guidelines [22], and (iii) he inal selec ion o a
single, common PNEC alue, ag eed upon as a esul o
Eu ope-wide expe consul a ions.
A p esen he da abase comp ises, o almos all Sus-
Da subs ances (i.e. > 65,000), a leas one in silico PNEC
[23] based on p edic ed acu e e ec s o each o he h ee
basic ophic le els o he esh wa e compa men ( ish,
daphnia, and algae), which a e used when expe imen al
oxici y da a a e insu icien o no a ailable. In 2019, a
semi-au oma ed ool o e ie ing expe imen al (eco)
oxici y da a om he US EPA ECOTOX Knowledge-
base allowed he impo o > 125,000 expe imen al da a
on s anda d (eco) oxici y endpoin s o abou 5000 Sus-
Da subs ances in a o ma compa ible wi h he me ada a
equi emen s o he NORMAN Eco oxicology Da abase.
Addi ional expe imen al (eco) oxici y da a and h eshold
alues will be e ie ed om o he da abases such as he
REACH po al, he ETOX da abase o he Ge man Fed-
e al En i onmen Agency, as well as exis ing PNECs and
Quali y S anda ds (EQS) om a ious egula o y sou ces.
The (eco) oxici y h eshold alues used o chemicals p i-
o i isa ion a e ag eed by expe s and e e ed o as ‘Low-
es PNECs’. These alues a e gene ally calcula ed o he
esh wa e ma ix and hen con e ed o an equi alen
PNEC alue o ma ine wa e , sedimen and bio a ma i-
ces ( o example, bioconcen a ion ac o s (BCF) a e used
o con e sion o equi alen PNECs o bio a).
EMPODAT: ada abase o  a ge moni o ing da a
Pe spec i es and ecommenda ions
P o ide a Eu ope-wide s anda d o essen ial quali y
in o ma ion (me ada a) accompanying chemical analysis
esul s and commonly ag eed minimum equi emen s o
allow in e ope abili y o a chi ed moni o ing da a.
Ra ionale
A game change o nex gene a ion chemical isk assess-
men is a sys em able o p o ide comp ehensi e in o ma-
ion on he exposu e o humans and he en i onmen o
la ge numbe s o chemicals du ing he en i e li e cycle o
p oduc s, including was e and ecycled p oduc s.
Wi h he EMPODAT da abase module [24] o he
NDS, he NORMAN Associa ion has al eady es ablished
a collabo a ion wi h IPCHEM, he o icial Eu opean
eposi o y o moni o ing da a p oduced by na ional mon-
i o ing p og ammes and EU- unded esea ch p ojec s
in all ma ices and compa men s. EMPODAT oday
hos s app oxima ely 10.3 million geo- e e enced a ge
moni o ing da a o mo e han 3100 subs ances in wa e
(su ace, g ound, and was e wa e ), sedimen , bio a,
soil, sewage sludge and ai ma ices. The da a a e pub-
licly accessible and p o ide an o e iew o benchma k
alues on he occu ence o con aminan s o eme ging
conce n ac oss Eu ope. F om he s a , NORMAN has
made a g ea e o o ensu e ha he da a a e ga he ed
in a s anda d o ma in o de o acili a e da a compa a-
bili y and exploi a ion ac oss Eu ope and beyond. These
sp eadshee -based Da a Collec ion Templa es (DCTs)
we e de eloped o each o he ma ices, and con ain
in o ma ion allowing o au oma ed assessmen o da a
quali y.
Non‑ a ge sc eening (NTS) ools andDigi al Sample
F eezing Pla o m (DSFP) o  e ospec i e suspec
sc eening o en i onmen al con aminan s
Pe spec i es and ecommenda ions
Es ablish a ede a ed Eu opean in as uc u e s o ing
aw non- a ge sc eening da a con e ed in o a common
(open) o ma , designed o e ospec i e sc eening.
Es ablish a cen al pla o m/da abase s o ing egula ly
upda ed in o ma ion on a ailable da a se s Eu ope-wide
and, e en ually, a a global scale.
Apply commonly ag eed wo k low(s) o e ospec i e
analysis o iden i y and p io i ise pollu an s equen ly
de ec ed in en i onmen al samples.
Ra ionale
Thanks o NTS echniques i is possible o ob ain an
o e iew o human and en i onmen al exposu e o hou-
sands o chemicals simul aneously, wi h a high le el o
sensi i i y and selec i i y, including chemicals ha ha e
no been iden i ied p e iously [25]. The NTS wo k lows
(comp ising wide-scope a ge , suspec and non- a ge
sc eening) based on ull scan, high- esolu ion mass spec-
ome y (HRMS), de eloped by NORMAN membe s,
ep esen he s a e-o - he-a me hods o deal wi h eal-
wo ld con aminan mix u es in a mo e holis ic way.
Ac i e since 2013, he NORMAN NTS Wo king G oup
has buil a s ong collabo a i e in as uc u e and de el-
oped inno a i e ools o acili a e exploi a ion and in e -
p e a ion o complex da a p oduced by ull scan, HRMS
me hods. NORMAN membe s ha e also de eloped p o-
ocols o implemen NTS in ou ine, egula o y applica-
ions. Suspec sc eening o p e-de ined lis s o ens o
ens-o - housands o known subs ances in each sample
(suppo ed by NORMAN SLE and NORMAN SusDa ) is
p esen ly he ecommended way o wa d.
In his con ex he Digi al Sample F eezing Pla o m
[26] is a key ool de eloped by NORMAN o suppo
suspec and non- a ge sc eening. This no el echnology

Page 6 o 11
Dulioe al. En i on Sci Eu (2020) 32:100
allows he s o age o housands o high- esolu ion mass
spec a ( inge p in s) o all chemicals, me aboli es and
ans o ma ion p oduc s de ec ed in each o he analysed
samples. Thanks o his pla o m, i is possible o use s
o sea ch e ospec i ely o a la ge numbe o com-
pounds (e.g. hose in SusDa ; see abo e) in all he “digi-
ally ozen” samples s o ed in he da abase and ob ain
eliable quali a i e and semi-quan i a i e da a on hei
occu ence in he in es iga ed samples.
Fu he key ools, suppo ed by NORMAN and embed-
ded in he NDS, o assis non- a ge sc eening, a e:
• MassBank Eu ope, an open-sou ce, open-access
da abase o mass spec a o suppo highe con i-
dence iden i ica ion o suspec s and non- a ge s [27,
28]. Based on MassBank Japan, MassBank Eu ope
was ounded in 2011, a ising om a NORMAN
ini ia i e. Today MassBank con ains o e 80,000
unique mass spec a o > 14,300 compounds (da a-
base elease 2020.05 [29]), including mass spec a o
en a i ely iden i ied compounds. MassBank Eu ope
is a co e se ice o NORMAN as well as o o he
ini ia i es such as HBM4EU (Human Biomoni o -
ing o Eu ope) ini ia i e [30], ELIXIR [31], he Ge -
man Ne wo k o Bioin o ma ics In as uc u e (de.
NBI) [32] and he Ge man Na ional Resea ch Da a
In as uc u e Ini ia i e o Chemis y (NFDI4Chem)
[33];
• A Re en ion Time Index (RTI) p edic ion model [34,
35] allowing o en a i e iden i ica ion o each com-
pound in SusDa as a combina ion o i s exac mass,
MS/MS agmen s and he p edic ed RTI alue,
educes he numbe o alse posi i es in suspec
sc eening.
Thanks o all he abo e-men ioned in e connec ed
ools, DSFP can p o ide eliable quali a i e and semi-
quan i a i e da a on he occu ence o al eady iden i-
ied as well as no el CECs, he eby p o iding exhaus i e
insigh in o he spa ial and empo al dis ibu ion o con-
aminan mix u es in he en i onmen , making NOR-
MAN DSFP a i ual en i onmen al obse a o y on
chemical con amina ion. Ex ensions o DSFP o addi-
ional chemicals cap u ed in SusDa (e.g. highly pola
molecules and gas ch oma og aphy-only amenable sub-
s ances) a e unde way.
Collabo a i e Eu opean amewo k oimp o e da a quali y
andcompa abili y: de elopmen andha monisa ion
o me hods
Pe spec i es and ecommenda ions
Build he capaci y o labo a o ies in Eu ope and globally
by sys ema ic o ganisa ion o in e na ional Collabo a i e
T ials add essing analysis o CECs in a ious ma ices by
no el analy ical echnologies.
Pu sue p og essi e es ing and implemen a ion o no el
sampling and analy ical me hodologies o help design
sma (e ) moni o ing s a egies ha can be applied in
egula o y moni o ing ac i i ies.
Ra ionale
NORMAN b ings oge he he leading Eu opean ins i u-
ions in he de elopmen and ha monisa ion o measu e-
men me hods o he de ec ion o eme ging chemicals in
he en i onmen . The s udies o ganised by he ne wo k
ep esen a c ucial s ep o he scien i ic communi y and
o en i onmen al agencies o alida ion and ha monisa-
ion o inno a i e sampling and moni o ing ools be o e
hei possible u u e implemen a ion in egula ions.
NORMAN is he au ho o he i s common ame-
wo k o alida ion o chemical and biological moni-
o ing me hods—a p o ocol which is now adop ed as a
Technical Speci ica ion (TS) o he Eu opean Commi ee
o S anda diza ion (CEN) (CEN TS 16800:2015) [36, 37].
Mo e han 15 collabo a i e ials ha e been o ganised
by NORMAN since 2006 on a wide ange o me hods,
including non- a ge sc eening in wa e [38], sedimen
[39], indoo dus [40] and bio a [41], in i o and in i o
bioassays [42] and passi e sampling [43, 44]. They ha e
ackled aspec s ele an o moni o ing and ea ly wa n-
ing o CECs in he en i onmen and app oaches o haz-
a d assessmen , including in eg a ion o e ec -based
me hods wi h chemical analysis o imp o e in e p e a-
ion o cause–e ec links. These ials included no only
he assessmen o sample p epa a ion and ins umen al
pe o mance, bu also he e alua ion o he impac ha
compu a ional and da a p ocessing ools ha e on in e -
p e a ion o esul s.
Ad anced da a analysis ools: owa dsaEu opean
Ea ly Wa ning Sys em
P io i isa ion o subs ances andp io i y se ing
Pe spec i es and ecommenda ions
Sys ema ically collec wide-scope a ge , suspec and
non- a ge sc eening da a a Eu opean scale o imp o e
he spa ial and empo al co e age and ange o ma ices
a ailable o isk assessmen .
Iden i y compounds o which obus (eco) oxici y
s udies a e needed as a p io i y.
P io i ise chemicals o which s anda ds o mass spec-
a will be equi ed om indus y, o enable hei de ec-
ion in he en i onmen .
De elop dynamic open-access links o spa ially de ailed
in o ma ion abou p oduc ion, uses, exposu e o and
consump ion o chemicals.
Page 7 o 11
Dulioe al. En i on Sci Eu (2020) 32:100
De elop a common Eu opean scheme o g ouping o
chemicals and indica o subs ances, based on a ious
c i e ia including sec o o use, chemical s uc u e and
mode o ac ion.
In eg a e mo e s ongly chemical analy ical and e ec -
based me hods in o de o iden i y e ec and (mix u e-)
isk d i e s, i.e. subs ances o g oups o subs ances ha
should be selec ed o u he isk assessmen .
Ra ionale
In he pas decade, NORMAN has de eloped an in e-
g a ed s a egy o deal wi h less-in es iga ed subs ances
o which knowledge gaps a e iden i ied (e.g. insu icien
in o ma ion on he exposu e le els and/o ad e se e ec s,
o inadequa e pe o mance o he analy ical me hods o
hei measu emen in he en i onmen ) [45]. The con-
cep in ol es he applica ion o a decision ee which
allows he alloca ion o subs ances in o six main ac ion
ca ego ies, based on he iden i ied knowledge gaps and
ac ions needed o add ess hem. The p io i y wi hin each
ca ego y is hen e alua ed on he basis o speci ic occu -
ence, haza d (pe sis ence, bioaccumula ion, mobili y,
endoc ine dis up ion po en ial, e c.) and isk indica o s
such as he F equency o Exceedance (FoE) and Ex en o
Exceedance (EoE) o he Lowes PNECs.
Va ious aspec s o a ca ego isa ion/ anking sys em ha e
been scaled up and es ed in nume ous la ge-scale Eu o-
pean p ojec s and na ional p io i isa ion p ocesses such
as de ining Wa e F amewo k Di ec i e (WFD) Ri e
Basin Speci ic Pollu an s (RBSP) in he Danube Ri e
Basin [46] o selec ing na ional Wa ch Lis subs ances
and RBSP in F ance [47, 48] and in The Ne he lands [49].
In his way, NORMAN aims o p o ide a scheme o ha -
monised RBSP assessmen ac oss he EU. On a egula
basis, NORMAN also makes ecommenda ions o he
Commission ega ding subs ances o be added o he lis
o WFD P io i y Subs ances and EU Wa ch Lis [50].
This wo k low, o iginally designed o wo k wi h a ge
moni o ing da a, now in eg a es he au oma ic que y o
NTS mass spec al in o ma ion a chi ed in DSFP (see
abo e). Thanks o DSFP and he se o ully in eg a ed
ools and da abases de eloped by NORMAN, i is now
possible o ob ain an o e iew o he s a e o knowledge
(spa ial dis ibu ion o con aminan s, deg ee o exceed-
ance o h eshold alues based on semi-quan i ied da a,
e c.) o a dynamically upda ed lis o > 60,000 chemicals,
including many ne e s udied be o e, and o iden i y p i-
o i y subs ances/g oups o subs ances o which u he
ac ions need o be aken.
This app oach i s well wi h he equi emen s o an
Ea ly Wa ning Sys em, whe e he da a o co ec ly iden-
i y an eme ging isk a an ea ly s age a e ypically limi ed
o o poo quali y. In his con ex , i is impo an o use
a anspa en and a ional app oach o signal iden i i-
ca ion and cha ac e isa ion ha is able o deal wi h he
knowledge gaps ha s ill p e en p ope isk assessmen
and isk anking o mos eme ging subs ances. Indi idual
componen s o he Ea ly Wa ning Sys em concep , such
as No maNEWS, ha e al eady been ialled [51].
E ec ‑based me hods (EBM) o moni o ing o chemical
mix u es in heen i onmen
Pe spec i es and ecommenda ions
Sys ema ically include NTS and EBM in in es iga-
i e moni o ing p og ammes o suppo chemicals isk
assessmen .
Fu he de elop and implemen e ec -based me hods
in a wide ange o en i onmen al compa men s, includ-
ing he ma ine and e es ial en i onmen s.
Ha monise, and p o ide aining on, he use o e ec -
based me hods.
Ra ionale
Bioassays a e he only cu en ly a ailable me hods able
o espond o he ecen ly ecognised need o add ess
unknown mix u e isks p esen in he en i onmen ,
which can hen be linked o speci ic chemical compounds
ia chemical analysis [52, 53].
NORMAN is ac i ely con ibu ing o he cons uc ion
o a common posi ion o he Eu opean expe s on he use
o bioassays in he egula o y amewo k o he WFD, in
pa icula wi h he de ini ion o a ba e y o bioassays o
chemical wa e quali y assessmen [54]. Besides an in e -
labo a o y s udy o ganised in 2009 o assess he compa-
abili y o esul s ob ained wi h a ba e y o bioassays [42]
and a comp ehensi e li e a u e e iew on he de elop-
men o an eco oxicological pe spec i e on neu o oxici y
assessmen [55], NORMAN con ibu ed o he Science
o Policy In e ace (SPI) Es ogen moni o ing p ojec (a
olun a y ini ia i e o 12 coun ies and 24 o ganisa ions
in Eu ope), which has ecen ly p o ided conc e e demon-
s a ion da a abou he pe o mance o he es ed EBM
[56].
In e ms o p ac ical implemen a ion o EBMs in he
egula ion, ano he c ucial s ep is he de e mina ion
o e ec -based igge alues (EBT), which de ine he
accep able le el o e ec o each oxicological endpoin
o conce n and hus allow en i onmen al manage s o
in e p e EBM da a and dis inguish be ween mo e and
less pollu ed si es. In collabo a ion wi h he SOLUTIONS
p ojec (FP7/603437), NORMAN has con ibu ed o he
d a ing o a p oposal o a ha monised me hodology o
he de ini ion o e ec -based igge (EBT) alues [57]
and he way o p oceed when an EBT is exceeded [58,
59].
Page 8 o 11
Dulioe al. En i on Sci Eu (2020) 32:100
In con as o EQSs, EBTs conside all chemicals in a
mix u e con ibu ing o a measu ed e ec in a gi en
sample. Explaining he obse ed ac i i y de ec ed by he
applied bioassays and add essing he combined e ec
o chemicals can be done (using mass balances/‘icebe g
modelling’) by calcula ion o Toxic Uni s (TU) o each
o he quan i ied pollu an s o Bioanaly ical Equi alen
concen a ions (BEQ), depending on he bioassay. This
should be ollowed by a compa ison o he es ima ed
∑TU o BEQ om he componen -based assessmen
wi h he TU and BEQ de i ed om he bioassay es ing.
I EBTs a e exceeded and he componen -based assess-
men canno explain he ac i i y de ec ed in he bioas-
say, an E ec -Di ec ed Analysis (EDA) p o ocol should
be pe o med in o de o iden i y he isk d i e s [54, 60,
61].
In his con ex , NTS-based app oaches a e key o
imp o ing he iden i ica ion o isk d i e s and acili a -
ing compound/mix u e p io i isa ion in di e en ma i-
ces. As a ma e o ac , la ge da ase s om non- a ge
sc eening and e ec -based me hods can be explo ed
using mul i a ia e s a is ics and pa e n ecogni ion
me hods o iden i y peaks ha co- a y wi h de ec ed
e ec s ( i ual e ec -di ec ed analysis). The NORMAN
Join P og amme o Ac i i ies p omo es his ype o
s udy as a way o iden i y candida e compounds o u -
he in es iga ion [41].
Finally, as pa o i s la es Join P og amme o Ac i i-
ies, NORMAN will de elop a bioac i i y da abase. This
p ojec aims o suppo he in e p e a ion o e ec -based
moni o ing da a o mix u e oxici y modelling. A iche
se o bioac i i y da a will be c ucial o unde s anding
he con ibu ion o de ec ed chemicals o he obse ed
e ec in he di e en assays. Cu en ly, he lack o e ec
da a o he de ec ed chemicals in di e en assays is a
majo limi a ion and mo e da a is needed o a signi ican
imp o emen o mix u e modelling and elucida ion o
d i e s o oxici y. This da abase will be essen ial o e eal
CEC-induced bioassay ac i i y ha canno be explained
by he measu ed concen a ions o he ew indi idual
chemicals o which e ec da a a e al eady known [41].
O he needs and pu poses o his da abase, e.g. selec ion
o EBMs, a e cu en ly being explo ed wi hin he NOR-
MAN ne wo k.
Suppo ona ional andEu opean chemical isk
assessmen : ha nessing, combining andsha ing
e idence andexpe ise onCECs
C oss‑bo de coope a ion andin o ma ion exchange—
moni o ing supe ‑si es inEu ope
Pe spec i es and ecommenda ions
O ganise Eu ope-wide collabo a i e en i onmen al mon-
i o ing p og ammes using no el analy ical me hodologies
in a b oad ange o ma ices and on selec ed supe -si es
p o iding ep esen a i e geog aphical co e age and
esul s di ec ly suppo ing egula ions.
Imp o e he sha ing and use o local, egional, na ional
and EU-le el moni o ing da a be ween coun ies and
policy a eas (e.g. legisla ion o en i onmen , chemicals,
ood, p oduc s, was e, e c.) and ele an ins i u ions.
Ra ionale
All s a e-o - he-a ools p esen ed he e ha e been
de eloped and es ed wi hin la ge-scale Eu opean
p ojec s (e.g. FP7 SOLUTIONS, EDA-EMERGE
FP7-PEOPLE-2011-ITN/290100, ANSWER H2020-
MSCA-ITN-2015/675530, NEREUS COST Ac ion
ES1403, APEX LIFE17 ENV/SK/000355).
NORMAN wo ks in close coope a ion wi h in e na-
ional i e basin o ganisa ions (e.g. he In e na ional
Commission o he P o ec ion o he Danube Ri e
(ICPDR) ± 14 Eu opean coun ies and he EU; o ganis-
ing Join Danube Su eys e e y 6yea s), sea con en ions
(e.g. Black Sea Commission; OSPAR), en i onmen al
specimen banks and en i onmen al au ho i ies in a ious
Membe S a es (e.g. F ance, Ge many, No dic coun ies,
The Ne he lands).
In 2019 he NORMAN Associa ion ecei ed unding
om he ICPDR as a con ibu ion in suppo o i s pa -
icipa ion in he expe imen al ac i i ies o he 4 h Join
Danube Su ey (JDS4). The added alue o his ype o
collabo a ion is he oppo uni y o in es iga e and dem-
ons a e he capabili ies and limi s o new en i onmen al
assessmen amewo ks wi h a clea link o hei applica-
ion in a egula o y amewo k.
So a , NORMAN has been s ongly in ol ed in issues
ela ed o CECs in he esh wa e cycle and he associ-
a ed EU policies. In he ligh o NORMAN’s missions and
he need o ensu e a holis ic iew o eme ging isks asso-
cia ed wi h chemicals in he en i onmen , he ac i i ies
a e p og essi ely being ex ended o he indoo , ma ine
and e es ial en i onmen and wa e euse, he eby
building on expe ience gained in he wa e compa men
o acili a e he ans e o o he en i onmen al ma ices.
Conclusions
Scien i ic knowledge con inues o p og ess, and no el
ools a e cons an ly being de eloped. This helps com-
pe en au ho i ies and indus y in he ull alue chain o
chemicals o p o ide answe s o unanswe ed o newly
a ising ques ions ega ding isks o chemicals o he
en i onmen and human heal h, wi h a pa icula ocus
on ea ly wa ning, an icipa ion and p e en ion o u u e
isks.
In his pape , we ha e sough o p o ide a clea and
anspa en message abou how NORMAN as an
Page 9 o 11
Dulioe al. En i on Sci Eu (2020) 32:100
independen , well-es ablished and compe en ne wo k
o expe o ganisa ions in he ield o eme ging sub-
s ances has eno mous po en ial o con ibu e o he
implemen a ion o he PARC pa ne ship by sha ing
se e al o i s exis ing key ools ha we belie e a e pa -
icula ly ele an o he success o he ini ia i e.
An impo an ole o he PARC pa ne ship will be
o os e coope a ion and be e use o exis ing knowl-
edge, o be e coo dina ion o esea ch and up ake o
scien i ic indings in egula ion.
NORMAN s ands eady o p o ide expe ad ice o
PARC’s s akeholde o um, d awing on i s 15yea s o
expe ience in he de elopmen , ha monisa ion and
es ing o ad anced ools in ela ion o CECs and in
suppo o a Eu opean Ea ly Wa ning Sys em o un a el
he isks o CECs and close he gap be ween esea ch
and inno a ion and egula o y p ocesses. NORMAN
is a pla o m o scien i ic coope a ion building upon
olun a y membe con ibu ions o ad ance ou knowl-
edge and unde s anding o CECs in he en i onmen .
By combining he ex ensi e knowledge and expe i-
ence o he NORMAN ne wo k wi h he inancial and
policy- ela ed s eng hs o he PARC ini ia i e, a la ge
s ep owa ds he goal o a non- oxic en i onmen can
be aken.
Abb e ia ions
ARB: an ibio ic‑ esis an bac e ia; ARG : an ibio ic‑ esis an genes; BCF: bio‑
concen a ion ac o ; CAS: chemical Abs ac s Se ice; CEC: con aminan s o
eme ging conce n; CEN: Eu opean Commi ee o S anda diza ion; CEN TS:
CEN Technical Speci ica ions; CIS: Common Implemen a ion S a egy o he
WFD; DG ENV: Di ec o a e‑Gene al o En i onmen o he Eu opean Com‑
mission; DG Resea ch: Di ec o a e Gene al o Resea ch and Inno a ion o he
Eu opean Commission; EBM: e ec ‑based me hods; EBT: e ec ‑based igge
alues; EC: Eu opean Commission; ECHA: Eu opean Chemical Agency; EC
JRC: Join Resea ch Cen e o he Eu opean Commission; EDA: e ec ‑di ec ed
analysis; EEA: Eu opean En i onmen Agency; EQS: en i onmen al quali y
s anda d; ICPDR: In e na ional Commission o he P o ec ion o he Danube
Ri e ; IPCHEM: Eu opean In o ma ion Pla o m o Chemical Moni o ing; JDS:
Join Danube Su ey; NTS: non‑ a ge sc eening; PNEC: p edic ed no‑e ec
concen a ion; RBSP: Ri e Basin Speci ic Pollu an s; WFD: Wa e F amewo k
Di ec i e.
Acknowledgemen s
The NORMAN Associa ion g a e ully acknowledge he suppo o all hei
membe s and collabo a o s o e he 15 yea s o he associa ion.
Au ho s’ con ibu ions
VD, JK and JS ha e closely collabo a ed o w i e he i s d a manusc ip .
P dO, ES and TS p o ided de ailed eedback/ma e ial on speci ic ac i i ies. All
au ho s ha e ead, made commen s and app o ed he inal manusc ip .
Funding
No applicable.
A ailabili y o da a and ma e ials
No applicable.
E hics app o al and consen o pa icipa e
No applicable.
Consen o publica ion
No applicable.
Compe ing in e es s
The au ho s decla e ha hey ha e no compe ing in e es s.
Au ho de ails
1 INERIS, Na ional Ins i u e o En i onmen and Indus ial Risks, Ve neuil en
Hala e, F ance. 2 UBA, Fede al En i onmen Agency, Dessau‑Roßlau, Ge many.
3 NIVA, No wegian Ins i u e o Wa e Resea ch, Oslo, No way. 4 Wageningen
Uni e si y & Resea ch, Wageningen, The Ne he lands. 5 Eawag, Dübendo ,
Swi ze land. 6 IVL, Swedish En i onmen al Resea ch Ins i u e, Go ebo g,
Sweden. 7 NILU, No wegian Ins i u e o Ai Resea ch, Kjelle , No way. 8 En i‑
onmen al Ins i u e, Koš, Slo akia. 9 Depa men o En i onmen al Science,
S ockholm Uni e si y, S ockholm, Sweden. 10 Depa men o Aqua ic Sciences
and Assessmen , SLU, Swedish Uni e si y o Ag icul u al Sciences, Uppsala,
Sweden. 11 Na ional and Kapodis ian Uni e si y o A hens, A hens, G eece.
12 CSIC, Spanish Council o Scien i ic Resea ch, Ba celona, Spain. 13 DCE,
Aa hus Uni e si y, Aa hus, Denma k. 14 UFZ, Helmhol z Cen e o En i onmen‑
al Resea ch, Leipzig, Ge many. 15 Goe he Uni e si y, F ank u am Main, Ge ‑
many. 16 Leesu Ecole des Pon s, Uni Pa is Es C e eil, Ma ne‑la‑Vallee, F ance.
17 Depa men o Plan and En i onmen al Sciences, Uni e si y o Copenha‑
gen, Copenhagen, Denma k. 18 Toxicological Cen e, Uni e si y o An we p,
Wil ijk, Belgium. 19 AD eco ad ies, Ams e dam, The Ne he lands. 20 DERAC‑
En i onmen al isk assessmen o chemicals, Ma seille, F ance. 21 KWR Wa e
Resea ch Ins i u e, Nieuwegein, The Ne he lands. 22 Ins i u e o Risk Assess‑
men Sciences, U ech Uni e si y, Nieuwegein, The Ne he lands. 23 SWACCS,
Swedish Academic Conso ia o Chemical Sa e y, Ö eb o, Sweden. 24 MTM
Resea ch Cen e, Ö eb o Uni e si y, Ö eb o, Sweden. 25 Depa men o Ci il
and En i onmen al Enginee ing and Ni eas‑In e na ional Resea ch Cen e ,
Uni e si y o Cyp us, Nicosia, Cyp us. 26 ICRA, Ca alan Ins i u e o Wa e
Resea ch, Gi ona, Spain. 27 Uni e si y Jaume I, Cas ellón, Spain. 28 Facul y o Sci‑
ence, Cen e RECETOX, Masa yk Uni e si y, B no, Czech Republic. 29 Eco ox
Cen e Eawag, Dübendo , Swi ze land. 30 Depa men o Chemis y, Uni e si y
o Ba h, London, UK. 31 Depa men o Chemis y, A is o le Uni e si y o Thes‑
saloniki, Thessaloniki, G eece. 32 V ije Uni e si ei Ams e dam, Ams e dam,
The Ne he lands. 33 BRGM, O léans, F ance. 34 Lulea Technical Uni e si y, Lulea,
Sweden. 35 VÚVH, Wa e Resea ch Ins i u e, B a isla a, Slo akia. 36 Plen zia
Ma ine S a ion–Uni Basque Coun y (PiE‑UPV/EHU), Plen zia, Basque Coun y,
Spain. 37 Ma ine Ins i u e, Galway, I eland. 38 INRAE, U Ri e ly, Lyon, F ance.
39 The En i onmen al P o ec ion Agency I eland, Dublin, I eland. 40 SYKE,
Finnish En i onmen Ins i u e, Helsinki, Finland. 41 IRSA‑CNR, Wa e Resea ch
Ins i u e, B ughe io, I aly. 42 RIVM, Cen e o Sus ainabili y, En i onmen
and Heal h, Bil ho en, The Ne he lands. 43 Depa men o En i onmen al Sci‑
ence, Radboud Uni e si y Nijmegen, Nijmegen, The Ne he lands. 44 F aun‑
ho e Ins i u e o Molecula Biology and Applied Ecology IME‑AE, Schmal‑
lenbe g, Ge many. 45 Uni e si y o Ams e dam, Ams e dam, The Ne he lands.
46 LCSB, Uni e si y o Luxembou g, Bel aux, Luxembou g. 47 L U, Ba a ian
En i onmen Agency, Augsbu g, Ge many. 48 RWS, Minis y o In as uc u e
and he En i onmen , Nijmegen, The Ne he lands. 49 Depa men o En i on‑
men al Science, Aa hus Uni e si y, Aa hus, Denma k. 50 Middlesex Uni e si y,
London, UK.
Recei ed: 31 May 2020 Accep ed: 6 July 2020
Re e ences
1. Eu opean Commission: Candida es o Eu opean Pa ne ships, Eu opean
Pa ne ship o Chemicals Risk Assessmen . h ps ://ec.eu op a.eu/in o/ iles
/eu op ean‑pa n e shi p‑chemi cals‑ isk‑asses smen _en. Accessed 20 June
2020
2. Eu opean Commission: The Eu opean G een Deal‑COM (2019) 640 inal.
2019
3. Council o he Eu opean Union: Towa ds a Sus ainable Chemicals Policy
S a egy o he Union—Council conclusions. (2019) h p://da a.consi lium.
eu op a.eu/doc/docum en /ST‑10713 ‑2019‑INIT/en/pd . Accessed 20 May
2020
4. Council o he Eu opean Union: The 8 h En i onmen Ac ion P o‑
g amme—Tu ning he T ends Toge he —Council conclusions (2019)