Evaluating the effectiveness of large language models in converting clinical data to FHIR format

Recei ed: 17 Feb ua y 2025
Re ised: 8 Ma ch 2025
Accep ed: 12 Ma ch 2025
Published: 19 Ma ch 2025
Ci a ion: Delaunay, J.; Gi bes, D.;
Cusido, J. E alua ing he E ec i eness
o La ge Language Models in
Con e ing Clinical Da a o FHIR
Fo ma . Appl. Sci. 2025,15, 3379.
h ps://doi.o g/10.3390/
app15063379
Copy igh : © 2025 by he au ho s.
Licensee MDPI, Basel, Swi ze land.
This a icle is an open access a icle
dis ibu ed unde he e ms and
condi ions o he C ea i e Commons
A ibu ion (CC BY) license
(h ps://c ea i ecommons.o g/
licenses/by/4.0/).
A icle
E alua ing he E ec i eness o La ge Language Models in
Con e ing Clinical Da a o FHIR Fo ma
Julien Delaunay 1,* , Daniel Gi bes 1and Jo di Cusido 1,2,*
1Top Heal h Tech, 08035 Ba celona, Spain; [email p o ec ed]
2Depa amen de P ojec es i Cons uccio, Uni e si a Poli ecnica de Ca alunya, 08034 Ba celona, Spain
*Co espondence: [email p o ec ed] (J.D.); [email p o ec ed] (J.C.)
Abs ac : The con e sion o uns uc u ed clinical da a in o s uc u ed o ma s, such as
Fas Heal hca e In e ope abili y Resou ces (FHIR), is a c i ical challenge in heal hca e in-
o ma ics. This s udy explo es he po en ial o la ge language models (LLMs) o au oma e
his con e sion p ocess, aiming o enhance da a in e ope abili y and imp o e heal hca e
ou comes. The e ec i eness o a ious LLMs in con e ing clinical epo s in o FHIR
bundles was e alua ed using di e en p omp ing echniques, including i e a i e co ec-
ion and example-based p omp ing. The indings demons a e he c i ical ole o p omp
enginee ing, wi h he wo-s ep app oach shown o signi ican ly imp o e accu acy and
comple eness. While ew-sho lea ning enhanced pe o mance, i also in oduced a isk o
o e eliance on examples. The pe o mance o he LLMs is assessed based on he p ecision,
hallucina ion a e, and esou ce mapping accu acy ac oss mammog aphy and de ma ologi-
cal epo s om wo clinics, p o iding insigh s in o e ec i e s a egies o eliable FHIR
da a con e sion and highligh ing he impo ance o ailo ed p omp ing s a egies.
Keywo ds: la ge language models; Fas Heal hca e In e ope abili y Resou ces; uns uc-
u ed clinical da a; heal hca e in o ma ics; na u al language p ocessing; p omp enginee ing
1. In oduc ion
The ad en o mode n na u al language p ocessing (NLP) echniques, d i en by
ad ances in machine lea ning (ML), has signi ican ly enhanced ou abili y o analyze and
in e p e complex ex da a [
1
–
4
]. These imp o emen s a e pa ly a ibu ed o me hods ha
encode and manipula e ex da a using la en ep esen a ions. These me hods embed ex
in o high-dimensional ec o spaces ha cap u e he unde lying seman ics and s uc u e
o language. The medical ield has wi nessed a signi ican ans o ma ion in ecen yea s
wi h he eme gence and popula iza ion o a ious ad anced echniques, including la ge
language models (LLMs) [5,6].
One o he mos c i ical a eas in which hese ad ances ha e made a subs an ial impac
is he in eg a ion and analysis o elec onic heal h eco ds (EHR). Wi h ich and a ied in-
o ma ion, EHR da a o e unique oppo uni ies o de eloping and alida ing ML models
ha can imp o e diagnos ic accu acy, p edic pa ien ou comes, and pe sonalize ea men
plans [
7
–
9
]. Fo ins ance, Hashi and Sawhney [
9
] demons a ed he po en ial o uns uc-
u ed clinical no es o enhance mo ali y p edic ion, while Alghamdi and Mos a a [
8
]
showed ha in eg a ing ad anced NLP echniques wi h de ailed EHR da a signi ican ly
imp o ed medica ion managemen p edic ions. In heal hca e in o ma ics, con e ing
uns uc u ed clinical da a in o s uc u ed o ma s like Fas Heal hca e In e ope abili y
Resou ces (FHIR) is c ucial o enhancing in e ope abili y and imp o ing pa ien ca e.
Appl. Sci. 2025,15, 3379 h ps://doi.o g/10.3390/app15063379
Appl. Sci. 2025,15, 3379 2 o 28
FHIR is a s anda d o exchanging heal hca e in o ma ion elec onically, acili a ing he
in eg a ion o dispa a e heal hca e sys ems. Howe e , a gap emains be ween FHIR imple-
men a ion guides (IGs) and he p ocess o building ac ual se ices, as IGs p o ide ules
wi hou speci ying conc e e me hods, p ocedu es, o ools [
10
]. Thus, s akeholde s may
eel i non i ial o pa icipa e in he ecosys em, highligh ing he need o mo e ac ionable
p ac ice guidelines o p omo e FHIR’s as adop ion.
Se e al s udies ha e highligh ed he complexi ies and impo ance o his ask. Fo
ins ance, Shah [
11
] unde sco ed he signi icance o FHIR in achie ing in e ope abili y
be ween di e en heal hca e sys ems, demons a ing i s po en ial o imp o e heal hca e
deli e y and pa ien ou comes. Hong e al. [
12
] discussed he in icacies o mapping
uns uc u ed da a o he highly s uc u ed and s anda dized FHIR o ma , emphasizing
he need o sophis ica ed echniques and ools. Thei wo k demons a ed he easibili y o
a scalable clinical da a no maliza ion pipeline known as NLP2FHIR, which in eg a es NLP
wi h FHIR o model uns uc u ed EHR da a. Howe e , he NLP2FHIR echnique cu en ly
elies on ans o ma ion sc ip s o mapping and con en no maliza ion, which limi s i s
s anda diza ion and au oma ion capabili ies.
Despi e ad ances in NLP, con e ing uns uc u ed clinical da a, such as clinical e-
po s, in o s uc u ed o ma s like FHIR emains a challenging ask. The impo ance o his
con e sion lies in i s po en ial o enable seamless da a sha ing, enhance clinical decision
suppo , and acili a e esea ch and public heal h ini ia i es [
13
,
14
]. Fo ins ance, conside
he a i icial example shown in Figu e 1, which illus a es he con e sion o in o ma ion
om a clinical epo (on he le ) o an FHIR bundle (on he igh ). This example un-
de sco es he well-de ined s uc u e o FHIR and highligh s he complexi y in ol ed in
con e ing e en simple elemen s om a clinical epo in o his s uc u ed o ma . This
challenge unde sco es he need o ad anced echniques, such as hose explo ed in ou
s udy, o acili a e his con e sion p ocess e ec i ely.
Appl.Sci.2025,15,33792o 28

Mos a a[8]showed ha in eg a ingad ancedNLP echniqueswi hde ailedEHRda a
signi ican lyimp o edmedica ionmanagemen p edic ions.Inheal hca ein o ma ics,
con e inguns uc u edclinicalda ain os uc u ed o ma slikeFas Heal hca eIn e op‐
e abili yResou ces(FHIR)isc ucial o enhancingin e ope abili yandimp o ingpa ien 
ca e.FHIRisas anda d o exchangingheal hca ein o ma ionelec onically, acili a ing
hein eg a iono dispa a eheal hca esys ems.Howe e ,agap emainsbe weenFHIR
implemen a ionguides(IGs)and hep ocesso buildingac ualse ices,asIGsp o ide
uleswi hou speci yingconc e eme hods,p ocedu es,o  ools[10].Thus,s akeholde s
may eeli non i ial opa icipa ein heecosys em,highligh ing heneed o mo eac‐
ionablep ac iceguidelines op omo eFHIR’s as adop ion.
Se e als udiesha ehighligh ed hecomplexi iesandimpo anceo  his ask.Fo 
ins ance,Shah[11]unde sco ed hesigni icanceo FHIRinachie ingin e ope abili ybe‐
weendi e en heal hca esys ems,demons a ingi spo en ial oimp o eheal hca ede‐
li e yandpa ien ou comes.Honge al.[12]discussed hein icacieso mappinguns uc‐
u edda a o hehighlys uc u edands anda dizedFHIR o ma ,emphasizing heneed
o sophis ica ed echniquesand ools.Thei wo kdemons a ed he easibili yo ascala‐
bleclinicalda ano maliza ionpipelineknownasNLP2FHIR,whichin eg a esNLPwi h
FHIR omodeluns uc u edEHRda a.Howe e , heNLP2FHIR echniquecu en ly e‐
lieson ans o ma ionsc ip s o mappingandcon en no maliza ion,whichlimi si s
s anda diza ionandau oma ioncapabili ies.
Despi ead ancesinNLP,con e inguns uc u edclinicalda a,suchasclinical e‐
po s,in os uc u ed o ma slikeFHIR emainsachallenging ask.Theimpo anceo 
hiscon e sionliesini spo en ial oenableseamlessda asha ing,enhanceclinicaldeci‐
sionsuppo ,and acili a e esea chandpublicheal hini ia i es[13,14].Fo ins ance,con‐
side  hea i icialexampleshowninFigu e1,whichillus a es hecon e siono in o ‐
ma ion omaclinical epo (on hele ) oanFHIRbundle(on he igh ).Thisexample
unde sco es hewell‐de ineds uc u eo FHIRandhighligh s hecomplexi yin ol ed
incon e inge ensimpleelemen s omaclinical epo in o hiss uc u ed o ma .This
challengeunde sco es heneed o ad anced echniques,suchas hoseexplo edinou 
s udy, o acili a e hiscon e sionp ocesse ec i ely.

Figu e1.A i icialexampleo somein o ma ion omaclinical epo (on hele )con e ed oanFHIR
bundle(on he igh ).Some esou ces, hei ID,and e e ence o hepa ien ha ebeen emo ed o 
place.
Figu e 1. A i icial example o some in o ma ion om a clinical epo (on he le ) con e ed o an
FHIR bundle (on he igh ). Some esou ces, hei ID, and e e ence o he pa ien ha e been emo ed
o place.
The p ima y objec i e o his esea ch is o e alua e he e ec i eness o a ious
LLMs in con e ing uns uc u ed clinical epo s in Spanish in o s uc u ed FHIR bundles.
Di e en p omp ing echniques a e employed, including p o iding examples o clinical
epo s and hei co esponding FHIR bundles. The impac o i e a i e co ec ion on he
Appl. Sci. 2025,15, 3379 3 o 28
quali y o gene a ed FHIR bundles is also explo ed, wi h he hypo hesis ha mul iple
co ec ion cycles may yield mo e accu a e esul s compa ed o a single gene a ion a emp .
The me hodology includes ga he ing di e se clinical epo s, designing p omp ing
echniques, in es iga ing he impac o empe a u e and i e a i e co ec ion, and e alua ing
he p ecision, hallucina ion a e, and esou ce mapping accu acy o he gene a ed FHIR
bundles. The esul s indica e ha p o iding well-de ined examples wi hin p omp s and
employing a wo-s ep con e sion app oach a e pa icula ly e ec i e s a egies o accu a e
and eliable FHIR con e sion. This s udy aims o demons a e he po en ial o LLMs in
au oma ing he con e sion o uns uc u ed clinical da a in o s uc u ed o ma s, he eby
enhancing da a in e ope abili y and imp o ing heal hca e ou comes.
The emainde o his pape is o ganized as ollows. Sec ion 2p esen s he p elimina -
ies, de ining la ge language models and FHIR esou ces. Sec ion 3p o ides an o e iew o
ela ed wo k on he use o LLMs in medical da a and hei pe o mance in da a con e sion.
Sec ion 4desc ibes he da ase s, LLMs, p omp echniques, and empe a u e le els used in
ou s udy. Sec ion 5p esen s ou me hodology o e alua ing he con e sion made by he
LLMs. Sec ion 6 epo s he esul s o ou s udy, while Sec ion 7discusses he implica ions
o ou indings o he use o LLMs in heal hca e. Finally, Sec ion 8concludes he pape
and ou lines di ec ions o u u e esea ch.
2. P elimina ies
This sec ion p o ides an o e iew o he key concep s and echnologies used in his
s udy. This ensu es ha eade s om bo h heal hca e and na u al language p ocessing
(NLP) domains a e amilia wi h he main concep s om bo h ields. We discuss okeniza-
ion and embeddings, which a e undamen al o NLP. Then, we p esen la ge language
models (LLMs), which a e cen al o ou app oach and co e ad anced echniques such as
mix u e o expe s (MoE) and he ine uning o LLMs o speci ic domains. We also discuss
he impac o p omp s and empe a u e se ings on hei pe o mance. Addi ionally, we
in oduce Fas Heal hca e In e ope abili y Resou ces (FHIR), a s anda d o exchanging
heal hca e in o ma ion elec onically.
2.1. Tokeniza ion and Embeddings
Tokeniza ion is a c ucial p ep ocessing s ep in NLP ha con e s ex in o okens,
which a e smalle uni s o meaning. In he con ex o LLMs, okeniza ion is essen ial o
ans o ming clinical epo s in o a o ma ha he model can unde s and and p ocess.
E ec i e okeniza ion ensu es ha he con ex and seman ic meaning o he clinical da a
a e p ese ed. This is pa icula ly impo an in heal hca e, whe e he accu acy o he
con e ed da a can di ec ly impac pa ien ou comes. P ope okeniza ion allows he LLMs
o p ocess he ex mo e e icien ly, educing compu a ional esou ces and ime. Consis en
okeniza ion ensu es ha simila clinical e ms a e ep esen ed uni o mly, which is c ucial
o main aining he in eg i y o he con e ed da a. The p ocess in ol es b eaking down
he ex in o indi idual wo ds o subwo ds, which a e hen con e ed in o nume ical
ep esen a ions called embeddings.
Embeddings cap u e he seman ic meaning o he ex , allowing he LLMs o un-
de s and he con ex and ela ionships be ween di e en pa s o he clinical epo . In
LLMs, embeddings a e gene a ed h ough a p ocess ha maps okens o high-dimensional
ec o s. These ec o s a e used by he model o unde s and and gene a e ex . The e a e
se e al ypes o embeddings, including wo d embeddings, subwo d embeddings, and
con ex ual embeddings. Wo d embeddings ep esen indi idual wo ds in a con inuous
ec o space, cap u ing seman ic and syn ac ic simila i ies. Techniques like Wo d2Vec [
15
]
and GloVe [
16
] a e commonly used o c ea e wo d embeddings. Subwo d embeddings
Appl. Sci. 2025,15, 3379 4 o 28
b eak wo ds in o smalle uni s, such as mo phemes, o handle ou -o - ocabula y wo ds and
a e e ms mo e e ec i ely. By e pai encoding (BPE) and Wo dPiece a e popula me hods
o subwo d okeniza ion. Con ex ual embeddings conside he con ex in which a wo d
appea s, allowing he same wo d o ha e di e en embeddings based on i s usage. Models
like BERT [1] and RoBERTa [2] use con ex ual embeddings o cap u e nuanced meanings.
2.2. La ge Language Models
La ge language models a e a class o a i icial in elligence models designed o un-
de s and and gene a e human language. These models a e ained on as amoun s o
ex da a and can pe o m a wide ange o na u al language p ocessing asks, such as ex
gene a ion, ansla ion, summa iza ion, and ques ion answe ing [
17
–
20
]. LLMs le e age
deep lea ning echniques, pa icula ly ans o me a chi ec u es [
4
], o cap u e complex
linguis ic pa e ns and con ex ual in o ma ion.
Key cha ac e is ics o LLMs include scalabili y, e sa ili y, and he abili y o imp o e
pe o mance h ough p omp enginee ing. Scalabili y allows he LLMs o handle la ge
olumes o ex da a and gene a e cohe en and con ex ually ele an esponses. Ve sa ili y
enables hei applica ion ac oss a ious domains, including heal hca e, inance, and cus-
ome se ice, o au oma e and enhance ex -based asks. P omp enginee ing in ol es
c a ing speci ic inpu p omp s o guide he model’s ou pu , signi ican ly imp o ing pe o -
mance. Ad anced LLMs a e designed o be compu a ionally e icien , allowing hem o
handle complex asks wi h high pe o mance and minimal esou ce usage.
Mix u e o expe s (MoE) is an ad anced echnique ha is used o enhance he capa-
bili ies o LLMs by le e aging a combina ion o specialized sub-models, o “expe s,” each
ained o handle speci ic aspec s o a ask. In an MoE a chi ec u e, he model dynamically
selec s he mos app op ia e expe s o p ocess di e en pa s o he inpu , allowing o
mo e e icien and e ec i e handling o complex asks. This app oach has been shown
o imp o e he pe o mance and scalabili y o LLMs, especially in scena ios equi ing
specialized knowledge o ine-g ained p ocessing [
21
]. The MoE app oach allows o he
in eg a ion o a la ge numbe o pa ame e s, making i well sui ed o handling complex
and di e se da ase s.
Fine uning is a p ocess ha is used o adap a p e- ained LLM o speci ic domains
o asks by u he aining he model on domain-speci ic da a. This app oach allows
he model o lea n he unique linguis ic pa e ns, e minology, and con ex ual nuances
o he a ge domain, enhancing i s pe o mance and accu acy. Fine uning has been suc-
cess ully applied in a ious domains, including heal hca e, whe e models a e ine uned
using specialized clinical da a o imp o e hei unde s anding and gene a ion o medical
in o ma ion [
22
,
23
]. By adap ing he model o he unique cha ac e is ics o he a ge
domain, ine uning can signi ican ly imp o e i s accu acy and eliabili y. Fine uned models
a e be e equipped o unde s and and gene a e con ex ually ele an esponses, making
hem mo e e ec i e in specialized applica ions.
In he con ex o heal hca e, LLMs ha e shown p omise in asks such as clinical no e
summa iza ion, medical in o ma ion ex ac ion, and diagnos ic suppo [
23
–
25
]. Howe e ,
hei e ec i eness in con e ing uns uc u ed clinical da a in o s uc u ed o ma s, such as
FHIR, emains an ac i e a ea o esea ch.
2.3. Impac o P omp s and Tempe a u e
LLMs o e p omising a enues o au oma ing many di e se complex asks, including
con e ing uns uc u ed clinical epo s in o s anda dized FHIR bundles. Howe e , he
e ec i eness o hese models is signi ican ly in luenced by wo key ac o s: he design o
inpu p omp s and he empe a u e se ing.
Appl. Sci. 2025,15, 3379 5 o 28
P omp s: The way in o ma ion is p esen ed o an LLM h ough p omp s plays a c ucial
ole in shaping i s ou pu . Di e en p omp ing s a egies can guide he model’s easoning
and in luence i s abili y o accu a ely ex ac in o ma ion and map i o FHIR esou ces.
B oadly, hese s a egies can be ca ego ized by he le el o guidance and examples p o ided.
A basic p omp ing app oach in ol es p o iding a di ec ins uc ion o he LLMs, such
as “Con e his clinical epo o an FHIR bundle”. While simple o implemen , basic
p omp ing o en elies hea ily on he model’s p e-exis ing knowledge and may no p o ide
su icien con ex o complex con e sion asks like FHIR.
Mo e sophis ica ed o ad anced p omp ing echniques aim o enhance he model’s
pe o mance by p o iding addi ional con ex , s uc u e, o examples. These echniques
can include p o iding explici ins uc ions on he desi ed ou pu o ma , which helps he
model o unde s and he speci ic equi emen s o FHIR bundles. They can also in ol e
inco po a ing examples o co ec ly o ma ed FHIR esou ces, which allows he model
o lea n om examples and imp o e i s abili y o gene a e alid bundles. Finally, hese
ad anced echniques can guide he model’s easoning p ocess h ough s ep-by-s ep in-
s uc ions, which helps he model o b eak down he complex con e sion ask in o smalle ,
mo e manageable s eps.
Tempe a u e Con ol: Tempe a u e is a hype pa ame e ha con ols he andomness
o “c ea i i y” o he LLMs’ ou pu . A lowe empe a u e se ing esul s in mo e de e -
minis ic and ocused esponses, p io i izing accu acy and consis ency. This is pa icula ly
impo an o asks like FHIR con e sion, whe e adhe ence o s ic s anda ds is c ucial.
Con e sely, a highe empe a u e se ing in oduces mo e andomness, leading o mo e
di e se and po en ially c ea i e ou pu s, bu a he cos o consis ency and po en ially
impac ing he alidi y o he gene a ed FHIR bundles. The e o e, inding an app op ia e
empe a u e is essen ial o balancing he need o accu acy wi h he po en ial bene i s o
explo ing di e se con e sion s a egies.
The choice o p omp ing s a egy and empe a u e alue has a di ec impac on
he quali y, alidi y, and comple eness o he gene a ed FHIR bundles. By analyzing he
in e ac ion be ween p omp s and empe a u e se ings, his s udy sheds ligh on he op imal
con igu a ions o le e aging LLMs in con e ing uns uc u ed clinical da a o s uc u ed
FHIR o ma s, pa icula ly o Spanish clinical cases om di e en special ies such as skin
ca e and mammog aphy.
2.4. Fas Heal hca e In e ope abili y Resou ces
Fas Heal hca e In e ope abili y Resou ces (FHIR) is a s anda d o exchanging heal h-
ca e in o ma ion elec onically. I aims o acili a e he in e ope abili y o heal hca e sys ems
by p o iding a consis en and lexible amewo k o ep esen ing clinical da a.
The key componen s o FHIR include:
•
Resou ces: FHIR de ines a se o esou ces ha ep esen speci ic heal hca e concep s,
such as pa ien s, obse a ions, medica ions, and diagnoses. Each esou ce has a
well-de ined s uc u e and can be combined o o m complex clinical eco ds. In ou
expe imen s, we es ic ed he esou ces ypes ha he LLMs migh use o con e
inpu ex o Condi ion, Obse a ion, Medica ionS a emen , P ocedu e, Alle gyIn ole -
ance, Encoun e , Immuniza ion, and EpisodeO Ca e. We de ail why we selec ed his
cu a ed se in Appendix ASec ion Resou ce Selec ion o FHIR Con e sion.
•
In e ope abili y: FHIR suppo s a ious da a exchange o ma s, including JSON and
XML, making i compa ible wi h a wide ange o heal hca e sys ems and echnologies.
•
Ex ensibili y: FHIR is designed o be ex ensible, allowing o he addi ion o new
esou ces and he cus omiza ion o exis ing ones o mee speci ic heal hca e needs.

Appl. Sci. 2025,15, 3379 6 o 28
•
S anda diza ion: FHIR adhe es o s ic s anda ds and p o ocols, ensu ing consis ency
and eliabili y in da a exchange ac oss di e en heal hca e sys ems.
The adop ion o FHIR has he po en ial o signi ican ly imp o e heal hca e ou comes
by enabling seamless da a sha ing, enhancing clinical decision suppo , and acili a ing
esea ch and public heal h ini ia i es. Howe e , con e ing uns uc u ed clinical da a
in o he s uc u ed FHIR o ma is a complex ask ha equi es sophis ica ed echniques
and ools.
In ou implemen a ion, we u ilized he HAPI FHIR lib a y, an open-sou ce Ja a imple-
men a ion o he FHIR speci ica ion. HAPI FHIR de ines model classes o e e y esou ce
ype and da a ype speci ied by FHIR, ensu ing obus da a alida ion and acili a ing he
se ializa ion o clinical documen elemen s in o s anda d FHIR JSON ep esen a ions.
In his s udy, we explo e he use o LLMs o au oma e he con e sion o uns uc u ed
clinical da a in o he FHIR o ma , wi h a ocus on imp o ing he accu acy and eliabili y
o he gene a ed FHIR bundles.
3. Rela ed Wo ks
This sec ion p o ides an o e iew o he cu en esea ch landscape ela ed o he
applica ion o LLMs in heal hca e, he ole o elec onic heal h eco ds (EHRs) and he
FHIR s anda d, and me hods o con e ing uns uc u ed da a in o s uc u ed o ma s. We
highligh key s udies and indings ha unde sco e he po en ial and challenges o hese
echnologies in heal hca e se ings.
3.1. La ge Language Models in Heal hca e
LLMs ha e been inc easingly applied in heal hca e o a ious asks, including ex
summa iza ion and in o ma ion ex ac ion [
25
]. Recen s udies ha e demons a ed hei
po en ial in clinical ex summa iza ion, adiology epo analysis, and ex ac ing s uc u ed
in o ma ion om elec onic heal h eco ds [
23
–
25
]. P omp enginee ing has eme ged as
a c i ical echnique o enhance he pe o mance o LLMs in hese asks. These s udies
a e pa icula ly ele an o ou esea ch as hey highligh he e ec i eness o p omp
enginee ing and he po en ial o ine uned LLMs in heal hca e applica ions.
Fo ins ance, Wei e al. [
26
] in oduced chain-o - hough p omp ing, which imp o ed
LLM pe o mance on mul i-s ep easoning p oblems. Addi ionally, Kojima e al. [
27
]
demons a ed ha ze o-sho chain-o - hough p omp ing can enhance LLM capabili ies
wi hou ask-speci ic examples. These indings unde sco e he impo ance o p omp
enginee ing in maximizing he po en ial o LLMs o di e se applica ions. Building on
hese insigh s, ou s udy e alua es di e en p omp ing echniques o imp o e he accu acy
and e iciency o con e ing uns uc u ed clinical da a in o s uc u ed o ma s.
The aim o his adap ed Delphi s udy [
28
] om 2024 was o collec esea che s’ opin-
ions on how LLMs migh in luence heal hca e and on he s eng hs, weaknesses, oppo uni-
ies, and h ea s o LLM use in heal hca e. Pa icipan s o e ed se e al use cases, including
suppo ing clinical asks, documen a ion asks, and medical esea ch and educa ion, and
ag eed ha LLM-based sys ems can ac as heal h assis an s o pa ien educa ion. The
ag eed-upon bene i s included inc eased e iciency in da a handling and ex ac ion, im-
p o ed au oma ion o p ocesses, imp o ed quali y o heal h ca e se ices and o e all heal h
ou comes, p o ision o pe sonalized ca e, accele a ed diagnosis and ea men p ocesses,
and imp o ed in e ac ion be ween pa ien s and heal hca e p o essionals.
In hei s udy, Dalmaz e al. [
29
] le e aged clinician eedback o lea n om hei
p e e ences o ailo he model’s ou pu s, ensu ing ha hey aligned wi h he needs o
clinical p ac ice. Thei expe imen s demons a ed ha his app oach imp o ed upon he
pe o mance o he model pos -supe ised ine uning. Thei indings unde sco ed he
Appl. Sci. 2025,15, 3379 7 o 28
po en ial o in eg a ing di ec clinician inpu in o LLM aining p ocesses, pa ing he way
o mo e accu a e, ele an , and accessible ools o clinical ex summa iza ion. These
insigh s a e pa icula ly ele an o ou s udy as we e alua e he pe o mance o a ine uned
LLM in heal hca e con ex s.
3.2. Elec onic Heal h Reco ds and FHIR
EHR sys ems a e essen ial o mode n heal hca e, and he FHIR s anda d plays a
c ucial ole in ensu ing in e ope abili y. S udies ha e shown ha EHRs imp o e pa ien
ca e by enhancing communica ion be ween clinicians, s eamlining clinical ac i i ies, and
p o iding comp ehensi e pa ien in o ma ion a he poin o ca e. A s udy by Koo e al. [
30
]
showed ha EHR-based sign-ou s educed medical e o s by 30% compa ed o e bal hand-
o s be ween hospi al p o ide s, while a e iew ound ha EHR enhanced communica ion
be ween p o ide s educed hospi al eadmissions by 20–30% [31].
The FHIR s anda d has eme ged as a c i ical componen in achie ing in e ope abili y
be ween di e en heal hca e sys ems. Bende and Sa ipi [
13
] in oduced FHIR as a solu ion
o add ess he limi a ions o p e ious s anda ds, highligh ing i s po en ial o acili a e
seamless da a exchange. Mandel e al. [
14
] u he demons a ed FHIR’s e ec i eness
in enabling hi d-pa y apps o in eg a e wi h EHRs, enhancing unc ionali y and da a
accessibili y. Implemen a ion o EHR sys ems wi h FHIR capabili ies has shown p omising
esul s in imp o ing heal hca e deli e y [
32
]. Fo ins ance, Ayaz e al. [
32
] de eloped a
da a analy ic amewo k ha suppo ed clinical s a is ics and analysis by le e aging FHIR,
showcasing he p ac ical applica ions and bene i s o FHIR in eal-wo ld heal hca e se ings.
These s udies collec i ely emphasize he c ucial ole o EHR sys ems and he FHIR s anda d
in mode nizing heal hca e deli e y, imp o ing pa ien ou comes, and ensu ing e icien ,
in e ope able heal h in o ma ion exchange.
3.3. Da a Con e sion and S uc u ing
Se e al s udies ha e demons a ed he e ec i eness o machine lea ning and NLP
echniques in ex ac ing s uc u ed in o ma ion om scien i ic ex . Fo example, a ecen
pape [
33
] demons a ed he e ec i eness o ine uned LLMs in ex ac ing s uc u ed
in o ma ion om scien i ic ex . Thei indings included ha ine uning LLMs can ex ac
complex scien i ic knowledge and o ma i as JSON objec s. The me hod wo ked well wi h
only a ew hund ed aining examples. Models showed he abili y o au oma ically co ec
e o s and no malize common en i y pa e ns. In e . [
34
], he au ho s es ed a ious LLMs,
including GPT-3, Ins uc GPT, and PaLM, on asks like ques ion answe ing, summa iza ion,
and code gene a ion. Fo each ask, LLMs we e asked o gene a e esponses in h ee
o ma s: ee- o m ex , s uc u ed o ma s (e.g., ables, lis s, code), and a mix u e o bo h.
They ound ha LLMs consis en ly pe o med be e on ee- o m ex gene a ion compa ed
o s uc u ed o ma s. Mo eo e , his s udy e ealed ha LLMs ha e inhe en biases owa d
p oducing na u al language ex and s uggle mo e wi h adhe ing o he cons ain s o
s uc u ed da a o ma s. Finally, s ic e o ma cons ain s esul ed in g ea e pe o mance
deg ada ion in he LLMs’ easoning abili ies.
Despi e hese ad ancemen s, he e a e ela i ely ew ools and me hods speci ically
designed o con e ing uns uc u ed clinical da a in o s uc u ed o ma s, pa icula ly in
heal hca e. Ou s udy add esses his gap by being he i s o igo ously es he capabili ies
o LLMs in con e ing uns uc u ed clinical da a in o he highly s uc u ed and complex
FHIR o ma . This app oach no only highligh s he po en ial o LLMs in heal hca e da a
con e sion bu also aims o guide o he esea che s by p o iding insigh s in o which
LLMs o use, how o p omp models e ec i ely, and which pa ame e alues o choose o
op imal pe o mance.
Appl. Sci. 2025,15, 3379 8 o 28
4. Me hod
In his sec ion, we i s p esen he documen s we used o e alua e he capaci ies
o di e se LLMs in con e ing uns uc u ed in o ma ion in Spanish o FHIR. Then, we
ou line he a ious models we used o con e he in o ma ion and e alua e he capaci ies
o each models. Finally, we p esen echnical de ails, including p omp echniques and
empe a u e alues.
4.1. Da a Collec ion om Clinical Cen e s
In his s udy, we ex ac ed clinical cases om wo di e en cen e s, he Mammog aphy
Clinic, specializing in plas ic su ge y, and he De ma ology Clinic, ocusing on skin ca e,
p o iding a compelling and complemen a y da ase o his s udy. Bo h clinics gene a e a
signi ican olume o uns uc u ed clinical da a ha o e s a ich oppo uni y o explo e he
applica ion o LLMs and FHIR in heal hca e.
4.1.1. Mammog aphy Clinic
B eas su ge y epo s, which cons i u e he majo i y o he Mammog aphy Clinic’s
uns uc u ed da a, ep esen a aluable esou ce o se e al easons. Fi s , mammog aphy
is a ou ine sc eening p ocedu e wi h well-es ablished epo ing s anda ds, p o iding a
consis en o ma o da a ex ac ion. Second, he de ailed desc ip ions o indings wi hin
mammog aphy epo s o e a complex and nuanced language ha can challenge he
capabili ies o LLMs o accu a ely iden i y and classi y en i ies. By ocusing on mammog a-
phy epo s, we can e alua e he pe o mance o LLMs in handling medical e minology,
handling nega ions, and ecognizing sub le nuances in clinical language.
4.1.2. De ma ology Clinic
The de ma ological clinical epo s complemen ed he mammog aphy da a by p o id-
ing a b oade ange o clinical condi ions and p esen ing di e en challenges o na u al
language p ocessing. While bo h he mammog aphy and de ma ological epo s con ain
de ailed desc ip ions o indings, he de ma ological epo s o en in ol e mo e subjec i e
assessmen s and may include desc ip ions o complex lesions and medica ions. This di e -
si y in clinical p esen a ion allows us o assess he deg ee o gene aliza ion o ou indings
and o iden i y any limi a ions o he model when applied o di e en ypes o clinical da a.
The use o bo h mammog aphy and de ma ological epo s o e s se e al ad an ages.
While he speci ic condi ions and indings di e , bo h ypes o epo s employ simila
linguis ic s uc u es and con en ions. By compa ing he pe o mance o LLMs on hese
wo da ase s, we can gain insigh s in o he ac o s ha in luence he accu acy and e iciency
o na u al language p ocessing in heal hca e.
The combina ion o mammog aphy and de ma ological epo s p o ides a di e se
and challenging da ase ha can be used o e alua e he po en ial o LLMs and FHIR in
ans o ming uns uc u ed clinical da a in o s uc u ed, machine- eadable o ma s. By
add essing he speci ic challenges posed by hese da ase s, we can ad ance he s a e o he
a in na u al language p ocessing o heal hca e applica ions. This compa a i e analysis
allows us o iden i y commonali ies and di e ences in how LLMs handle a ious ypes
o clinical language, ul ima ely con ibu ing o mo e obus and e sa ile NLP models
o heal hca e.
4.2. Models
In his s udy, we selec ed h ee cu ing-edge LLMs o assess hei p o iciency in con e ing
uns uc u ed documen s o FHIR. Each model possesses dis inc ea u es and capabili ies
ha ende hem well sui ed o his ask and me i u he explo a ion. To acili a e he
Appl. Sci. 2025,15, 3379 9 o 28
ep oducibili y o ou expe imen s, we le e age he Langchain lib a y (h ps://www.langchain.
com/), which se es as he ounda ion o he LLMs employed in ou e alua ion. By using
he Langchain lib a y, esea che s can eplica e ou expe imen s and build upon ou
indings o ad ance he s a e o he a in LLM-based con e sion quali y. Table 1p esen s
he di e en LLMs e alua ed in his s udy ha we e accessed on 12 Feb ua y 2025. This
able p esen s hei numbe o pa ame e s, ine uning, open sou ce, and companies, and
hey a e p esen ed in o de o inc easing anspa ency, anging om he mo e opaque o
he mos open-sou ce model.
Table 1. This able includes in o ma ion on he LLMs e alua ed in his s udy, such as he numbe o
pa ame e s, whe he he model was ine uned and open sou ce, and he company ha de eloped i .
Model Name Pa ame e s FineTuned Open Sou ce Company
Gemini 1.5 P o 540B * No No
Google (Moun ain View,
CA, USA)
Mix al 8×22b 141B ** No Yes Mis al AI (Pa is,
F ance)
Llama3-Med42 70B Yes Yes M42 (Abu Dhabi,
Uni ed A ab Emi a es)
* Es ima ed coun , as he exac numbe o pa ame e s is no publicly disclosed by Google. ** Mix al 8
×
22b is a
SMoE model wi h 22 dis inc models a 7B, using 39B ac i e pa ame e s ou o 141B.
4.2.1. Gemini P o
Gemini P o is a highly compu e-e icien mul imodal model capable o ecalling and
easoning o e ine-g ained in o ma ion om millions o okens o con ex [
20
]. I achie es
nea -pe ec ecall on long-con ex e ie al asks ac oss modali ies and imp o es he s a e
o he a in long-documen QA, long- ideo QA, and long-con ex ASR. Gemini P o has
been shown o ma ch o su pass he pe o mance o o he s a e-o - he-a models on a
b oad se o benchma ks.
The key ea u es o Gemini P o include mul imodal capabili ies, which allows Gemini
P o o p ocess and gene a e esponses ac oss di e en modali ies, such as ex and images,
making i e sa ile o a wide ange o applica ions. Mo eo e , he model’s abili y o handle
long con ex s makes i ideal o p ocessing de ailed clinical epo s, which o en con ain
ex ensi e and complex in o ma ion.
By le e aging Gemini P o’s ad anced capabili ies, we aimed o achie e high accu acy
and e iciency in con e ing uns uc u ed clinical da a in o he s uc u ed FHIR o ma s.
4.2.2. Mix al 8×22b
In his s udy, we ha e selec ed Mix al 8
×
22b, a powe ul a ian o Mis al AI’s
p e ious model [
19
], Mix al 8
×
7b. Mix al 8
×
22b is a spa se mix u e o expe s (SMoE)
model [
21
] wi h 39B ac i e pa ame e s ou o 141B, making i a signi ican ly la ge model
wi h 22 imes mo e pa ame e s han i s p edecesso . This inc eased pa ame e coun aims
o enhance Mix al’s capabili ies in handling complex asks and unde s anding nuanced
language pa e ns.
Mix al 8
×
22b o e s high pe o mance and e iciency and is luen in mul iple lan-
guages, including Spanish. I s s ong ma hema ics and coding capabili ies make i well
sui ed o p ocessing complex medical in o ma ion. Mix al 8
×
22b is known o i s inno-
a i e a chi ec u e and e icien p ocessing. By including Mix al 8
×
22b in ou s udy, we
aimed o gain aluable insigh s in o how i s unique design in luences i s abili y o g asp
in o ma ion based on clinical cases and con e i o FHIR.
Appl. Sci. 2025,15, 3379 16 o 28
6.3. P ecision
Table 3indica es he a iabili y in he pe o mance o he LLMs depending on he
p omp echnique used, highligh ing he impo ance o p omp enginee ing in op imizing
he con e sion o uns uc u ed clinical da a o he s uc u ed FHIR o ma .
Table 3. Mean and s anda d de ia ion o p ecision sco es o di e en clinics, models, p omp
echniques, and example p esence a a ious empe a u es. Colo coding indica es alue anges:
1–2.5 ( ed), 2.5–5 (o ange), 5–7.5 (oli e g een), and 7.5–10 (g een). S anda d de ia ions a e shown in
pa en heses. Highe sco es a e a o able while ×deno es no alid con e sion.
P ecision
Clinic Model Name P omp Tech. Ze o-Sho Few-Sho
Low High Low High
Skin Ca e
Gemini
Basic 2.09 (1.87) 1.73 (1.62) 4.45 (3.33) 6.45 (2.73)
Two-S ep 7.64 (0.5) 7.64 (0.5) 7.82 (0.4) 7.45 (0.52)
CoT 5.18 (2.96) 5.27 (2.94) 6.27 (2.65) 7.64 (0.5)
Mis al
Basic ×1.82 (2.71) 5.56 (3.2) 7.64 (0.5)
Two-S ep 7.18 (0.4) 6.7 (2.06) 5.06 (3.11) 7.44 (0.53)
CoT 3.91 (3.36) 3.27 (3.17) 7.55 (0.52) 7.55 (0.52)
Mammog aphy
Gemini
Basic 1.44 (1.33) 1.56 (1.33) 7.78 (0.97) 7.44 (0.53)
Two-S ep 7.78 (0.44) 7.67 (0.5) 7.67 (0.5) 7.89 (0.33)
CoT 3.78 (3.42) 4.33 (3.28) 8.0 (0.87) 7.67 (0.5)
Mis al
Basic × × 5.86 (3.21) 7.44 (0.53)
Two-S ep 7.89 (0.93) 7.44 (0.53) 6.0 (3.35) 7.56 (0.53)
CoT 4.78 (3.6) 4.9 (3.38) 7.62 (0.52) 7.89 (0.33)
Fi s , we obse e highe p ecision wi h ew-sho p omp ing compa ed o he ze o-sho
echnique. Second, highe empe a u es inc ease his e ec compa ed o lowe empe a u es.
Addi ionally, asking he model o s uc u e he in o ma ion be o e con e ing i o FHIR
( wo-s ep con e sion) yields he bes esul s, e en wi hou he p esence o examples in
he p omp .
Al hough bo h models exhibi simila ends, Gemini gene ally ou pe o ms Mis al
ac oss bo h clinics and a ious p omp echniques. Bo h models show he lowes p ecision
wi h he basic con e sion echnique.
6.4. Hallucina ion Ra e
We depic in Table 4 he mean and s anda d de ia ion o he hallucina ion a e o each
model’s con e sion om uns uc u ed da a o FHIR. The able highligh s he e ec i eness
o di e en p omp ing echniques and empe a u e le els in mi iga ing hallucina ions.

Appl. Sci. 2025,15, 3379 17 o 28
Table 4. A e age and s anda d de ia ion o hallucina ion a es o di e en clinics, models, p omp
echniques, and example p esence a a ious empe a u es. Colo coding e lec s alue anges: 1–2.5
(g een), 2.5–5 (oli e g een), 5–7.5 (o ange), and 7.5–10 ( ed). S anda d de ia ions a e shown in
pa en heses. Lowe alues a e ideal and ×indica es no alid con e sion.
Hallucina ion
Clinic Model Name P omp Tech. Ze o-Sho Few-Sho
Low High Low High
Skin Ca e
Gemini
Basic 9.45 (0.93) 9.82 (0.6) 7.45 (2.5) 5.55 (2.34)
Two-S ep 3.64 (0.92) 3.55 (0.82) 3.36 (1.21) 4.18 (1.66)
CoT 6.73 (3.2) 6.55 (3.05) 5.27 (2.57) 4.27 (1.68)
Mis al
Basic ×10.0 (0.0) 6.25 (2.89) 3.73 (1.27)
Two-S ep 4.0 (1.48) 4.7 (2.16) 5.88 (3.33) 4.11 (1.05)
CoT 7.36 (3.38) 8.27 (2.65) 3.73 (1.35) 3.64 (1.36)
Mammog aphy
Gemini
Basic 9.78 (0.67) 9.78 (0.67) 5.56 (2.24) 6.44 (2.51)
Two-S ep 3.44 (1.24) 3.67 (1.12) 3.33 (1.41) 3.11 (1.27)
CoT 7.44 (3.54) 7.22 (2.82) 4.33 (1.41) 5.11 (2.03)
Mis al
Basic × × 5.86 (2.93) 4.56 (2.3)
Two-S ep 3.78 (1.56) 4.56 (1.74) 4.86 (3.61) 4.33 (1.66)
CoT 6.78 (3.15) 6.1 (3.48) 4.25 (1.04) 3.89 (1.45)
Key obse a ions include consis en ly lowes hallucina ion a es wi h wo-s ep con e -
sion ac oss all models and p omp ing echniques. By i s s uc u ing he in o ma ion and
hen con e ing i o FHIR, he model can be e unde s and he con ex and ele ance o he
da a, leading o mo e accu a e and less hallucinogenic ou pu s. Then, p o iding examples
in he p omp sligh ly inc eases he hallucina ion a e compa ed o ze o-sho p omp ing.
This sugges s ha while examples can guide he model, hey may also in oduce biases o
i ele an in o ma ion, leading o highe hallucina ion a es. While highe empe a u es
gene ally inc ease he hallucina ion a e, especially wi h ze o-sho p omp ing, we do no
obse e di e ence in hese expe imen s.
Finally, Mis al ends o ha e sligh ly highe hallucina ion a es compa ed o Gemini
wi h ze o-sho p omp ing, while he opposi e is obse ed wi h ew-sho p omp ing.
6.5. Resou ce Mapping
Table 5shows ha p o iding examples in he p omp inc eases he esou ce mapping
accu acy, especially wi h basic and CoT p omp ing echniques. We also obse e mo e
di e si y in he esul s wi h ew-sho p omp ing se ings, pa icula ly a lowe empe a u es.
No ably, in some si ua ions (e.g., Mis al wi h ew-sho p omp ing), e y high accu acy
esul s a e obse ed wi h he CoT p omp ing echnique (abo e 7.5). Addi ionally, Gemini
and Mis al end o ha e simila esou ce mapping accu acy ac oss all p omp echniques
and empe a u e le els.
Appl. Sci. 2025,15, 3379 18 o 28
Table 5. A e age and s anda d de ia ion o esou ce mapping accu acy o di e en clinics, models,
p omp echniques, and example p esence a a ious empe a u es. Colo coding ep esen s alue
anges: 1–2.5 ( ed), 2.5–5 (o ange), 5–7.5 (oli e g een), and 7.5–10 (g een). S anda d de ia ions a e
shown in pa en heses. Lowe alues a e desi able and × ep esen s no success ul con e sion.
Resou ce Mapping
Clinic Model Name P omp Tech. Ze o-Sho Few-Sho
Low High Low High
Skin Ca e
Gemini
Basic 1.91 (1.87) 1.55 (1.29) 4.73 (3.58) 6.09 (2.59)
Two-S ep 7.36 (0.5) 7.27 (0.65) 7.55 (0.82) 7.55 (0.52)
CoT 4.91 (2.98) 5.09 (2.95) 6.27 (2.69) 7.18 (1.08)
Mis al
Basic ×1.82 (2.71) 5.25 (3.0) 7.27 (0.65)
Two-S ep 7.82 (0.4) 7.0 (2.16) 5.59 (3.52) 7.44 (0.73)
CoT 3.73 (3.23) 3.36 (3.32) 7.73 (0.9) 7.64 (0.67)
Mammog aphy
Gemini
Basic 1.44 (1.33) 1.44 (1.33) 6.78 (1.64) 6.89 (1.45)
Two-S ep 7.67 (0.87) 7.33 (1.0) 7.67 (1.0) 7.44 (1.01)
CoT 3.56 (3.5) 4.22 (3.49) 7.44 (1.13) 7.33 (0.5)
Mis al
Basic × × 5.71 (3.17) 7.44 (1.01)
Two-S ep 8.11 (1.05) 7.89 (0.78) 6.07 (3.45) 7.33 (0.71)
CoT 4.44 (3.28) 4.9 (3.38) 7.25 (0.71) 7.89 (1.17)
These obse a ions unde sco e he impo ance o ca e ul p omp enginee ing and
empe a u e con ol in op imizing he con e sion o uns uc u ed clinical da a o s uc u ed
FHIR o ma s. By unde s anding hese ac o s, we can be e design p omp s and adjus
pa ame e s o enhance he accu acy and eliabili y o he con e ed da a.
6.6. Quali a i e Analysis
The quali a i e analysis in ol ed compa ing 480 gene a ed FHIR bundles wi h hei
o iginal clinical epo s. We examined en con e sions p oduced by each LLM (Gemini and
Mis al) ac oss e e y combina ion o p omp echniques (basic, wo-s ep, CoT), empe a u e
se ings (low s. high), p esence o examples, and clinics. Ou analysis encompassed he
aw LLM ou pu s, HAPI FHIR- alida ed bundles, and o iginal clinical epo s. A e an
ini ial o e iew o pe o mance ends o each con igu a ion, we le e aged GPT-4o o
p o ide a quali a i e assessmen o he con e sions, iden i ying bo h posi i e and nega i e
aspec s. We hen used hese e alua ions as he basis and e i ied he elemen s indica ed as
posi i e and nega i e by compa ing hem wi h he clinical epo s.
6.6.1. Impac o P omp Techniques
Wi h basic p omp ing, he LLMs p oduce s uc u ed epo s con aining impo an
in o ma ion. Howe e , hese epo s do no ollow FHIR s anda ds. Fo ins ance, hey
o en use non-s anda d e ms (e.g., ‘Doc o ’ ins ead o ‘P ac i ione ’ o ‘S udy’ ins ead
o ‘Obse a ion’), making au oma ic con e sion o FHIR impossible. The basic app oach,
while s aigh o wa d, lacks he necessa y guidance o ensu e compliance wi h FHIR
s anda ds, esul ing in epo s ha a e in o ma i e bu no s uc u ally alid.
Using chain-o - hough (CoT) p omp ing, we ound ha he models accu a ely iden-
i y he ele an in o ma ion om he sou ce documen s. Implemen ing a second CoT
ound, in ended o allow he models o co ec any ini ial e o s, inc eases he numbe
o alid FHIR bundles gene a ed. Howe e , his app oach does no consis en ly p oduce
Appl. Sci. 2025,15, 3379 19 o 28
ully alid bundles and equen ly esul s in a educ ion in he numbe o in o ma ion
elemen s included.
The wo-s ep con e sion app oach pe o med well. This me hod in ol es i s s uc-
u ing he in o ma ion and hen con e ing i o FHIR. I co ec ly iden i ies he in o ma ion
and success ully con e s i o FHIR, demons a ing a mo e obus and eliable me hod o
his ask. The wo-s ep app oach ensu es ha he in o ma ion is i s o ganized in a way
ha he models can unde s and and is hen accu a ely mapped o FHIR esou ces, esul ing
in mo e comple e and alid bundles.
6.6.2. O he Fac o s In luencing Bundle Gene a ion
The e a e no able di e ences in he comple eness and complexi y o he bundles
gene a ed o he skin ca e and mammog aphy clinics. The bundles gene a ed o he
skin ca e clinic a e gene ally mo e comple e and complex, o en including de ailed pa ien
in o ma ion, diagnos ic no es, and ea men plans. By con as , he bundles gene a ed o
he mammog aphy clinic some imes include only pa ien and p ac i ione names, lacking
he comp ehensi e de ails equi ed o a ull FHIR ep esen a ion. This dispa i y may be
a ibu ed o he na u e o he clinical epo s om each clinic as obse ed in Sec ion 6.2,
wi h he skin ca e epo s con aining mo e s uc u ed and de ailed in o ma ion.
Few-sho p omp ing helps he models o iden i y mo e in o ma ion and gene a e mo e
comple e bundles. By p o iding examples, he models a e be e guided in unde s anding
he expec ed o ma and con en o he FHIR bundle. Howe e , a no able issue wi h ew-
sho p omp ing is ha he models some imes ollow he example oo closely, ep oducing
he alues as hey appea in he example wi hou upda ing hem wi h he ac ual alues
om he clinical epo . Fo ins ance, he models migh e ain placeholde alues like “Da e-
Time: YYYY-MM-DD” ins ead o inse ing he co ec da e and ime om he documen .
This o e eliance on he example can lead o e o s and inaccu acies in he gene a ed
bundles. The impac o empe a u e on he LLMs’ pe o mance appea s o be minimal.
Tempe a u e se ings, which con ol he andomness o he models’ ou pu , do no seem o
signi ican ly a ec he esou ce mapping accu acy o he comple eness o he bundles. This
sugges s ha he models’ pe o mance is mo e in luenced by he p omp ing echnique and
he p esence o examples a he han empe a u e a ia ions.
O e all, he e a e ew signi ican di e ences be ween Mis al and Gemini, excep ha
Mis al ends o e ain mo e o he o iginal ex in Spanish. This cha ac e is ic o Mis al
may be bene icial in p ese ing he nuances o he o iginal clinical epo s bu can also
in oduce challenges in s anda dizing he in o ma ion o FHIR con e sion. Gemini, on he
o he hand, appea s o be mo e adep a ansla ing and s uc u ing he in o ma ion in a
way ha aligns wi h FHIR s anda ds.
7. Discussion
This s udy in es iga ed he e ec i eness o LLMs in con e ing uns uc u ed clinical
epo s in o s anda dized FHIR bundles, explo ing he impac o p omp ing s a egies,
ew-sho lea ning, empe a u e se ings, and LLM selec ion. Ou indings o e aluable
insigh s in o op imizing his p ocess o heal hca e da a in e ope abili y and allow us o
alida e ou ini ial hypo heses.
7.1. P omp ing S a egies and Con e sion Success (H1.1 and H1.2)
Ou esul s s ongly suppo H1.1, demons a ing ha s uc u ed p omp ing ech-
niques like CoT and wo-s ep con e sion signi ican ly ou pe o m basic p omp ing. The
high ailu e a es obse ed wi h basic p omp ing (Table 2) highligh i s inadequacy o
complex FHIR con e sion due o a lack o guidance in adhe ing o FHIR s anda ds. As
Appl. Sci. 2025,15, 3379 20 o 28
hypo hesized in H1.2, he wo-s ep con e sion me hod p o ed mos e ec i e, achie ing
nea -pe ec con e sion a es wi h Gemini and signi ican ly educing he ailu e a e wi h
Mis al. This unde sco es he impo ance o explici ly s uc u ing he con e sion p ocess
o LLMs, con i ming ha o ganizing in o ma ion p io o FHIR mapping is c ucial o
accu a e esou ce mapping and bundle cons uc ion. While CoT o e ed an imp o emen
o e basic p omp ing, i did no ma ch he pe o mance o he wo-s ep app oach. This
disc epancy sugges s ha he CoT app oach, while e ec i e in b eaking down complex
asks, may no be ideally sui ed o he p ecise and s uc u ed equi emen s o FHIR con-
e sion. The model’s abili y o iden i y ele an in o ma ion is no ully ansla ed in o
accu a e FHIR esou ce mapping, leading o incomple e o inco ec bundles. This inding
aligns wi h he obse a ions o Wu e al. [
42
], which demons a ed ha LLMs s uggle o
sel -co ec easoning wi hou ex e nal eedback and may e en expe ience pe o mance
deg ada ion a e sel -co ec ion a emp s. In ou con ex , he lack o explici s uc u ing in
CoT p omp s can be seen as a o m o limi ed eedback, hinde ing he model’s abili y o
consis en ly gene a e alid FHIR.
7.2. The Role o Examples (Few-Sho Lea ning) (H2.1 and H2.2)
Ou indings pa ially suppo H2.1, indica ing ha ew-sho p omp ing gene ally
imp o es pe o mance, pa icula ly in p ecision and esou ce mapping accu acy (Tables 3
and 5). This can be a ibu ed o se e al ac o s. Fi s ly, p o iding an example helps
he LLMs o unde s and he desi ed ou pu o ma and s uc u e mo e clea ly, educing
ambigui y and guiding he models o gene a e mo e accu a e FHIR bundles. Secondly,
examples se e as a o m o con ex ual lea ning, allowing he LLMs o be e in e p e he
nuances o he clinical da a and map hem co ec ly o he co esponding FHIR elemen s.
Las ly, examples may help o mi iga e hallucina ions by ancho ing he models’ esponses
o a conc e e e e ence, he eby educing he likelihood o gene a ing i ele an o inco ec
in o ma ion. Howe e , we also con i med H2.2, iden i ying he isk o o e eliance on
examples, whe e he models ep oduced placeholde alues ins ead o ex ac ing in o -
ma ion om he inpu epo s. This highligh s he delica e balance be ween p o iding
su icien guidance and a oiding o e i ing. In e es ingly, he ailu e a e was educed
wi h ew-sho p omp ing a highe empe a u es compa ed o lowe empe a u es. This
could be due o he ac ha highe empe a u es allow he models o de ia e mo e om he
examples, hus mi iga ing he o e - eliance issue. This inding unde sco es he impo ance
o inco po a ing examples in p omp s when using LLMs o s uc u ed da a con e sion
asks in heal hca e, pa icula ly when dealing wi h sophis ica ed and s anda dized o ma s
like FHIR.
7.3. In luence o Tempe a u e (H3.1 and H3.2)
Ou esul s pa ially suppo ou hypo heses ega ding empe a u e. While we did
obse e ha highe empe a u es some imes led o he gene a ion o mo e esou ces, we did
no ind a consis en and signi ican inc ease as p edic ed by H3.1. Fu he mo e, while lowe
empe a u es gene ally con ibu ed o mo e consis en ou pu s, he impac o empe a u e
was less p onounced han ha o p omp ing s a egies o he use o examples. This
sugges s ha while empe a u e plays a ole in con olling ou pu a iabili y, i s in luence
on FHIR con e sion is seconda y o p omp design. Howe e , highe empe a u es did
inc ease he isk o hallucina ions, especially wi h ze o-sho p omp ing (Table 4), pa ially
suppo ing H3.2, which s a ed ha lowe empe a u e would yield mo e p ecise bundles.
This aligns wi h he gene al unde s anding ha lowe empe a u es p omo e mo e ocused
and de e minis ic ou pu s, which is c ucial o asks equi ing adhe ence o s ic s anda ds.
Appl. Sci. 2025,15, 3379 21 o 28
7.4. Model Pe o mance (H4)
Ou esul s s ongly e u e H4, which hypo hesized ha he specialized, ine uned
Llama3-Med42 model would ou pe o m he gene al-pu pose models. Llama3-Med42
p o ed en i ely unsui able o he FHIR con e sion ask, consis en ly p io i izing diagnos ic
sugges ions o e accu a e da a mapping, esul ing in no alid FHIR bundles. This high-
ligh s he impo ance o selec ing models app op ia e o he speci ic ask. Addi ionally,
Gemini gene ally ou pe o med Mis al in e ms o p ecision and hallucina ion a e, pa -
icula ly wi h ze o-sho p omp s. While Mis al did e ain mo e Spanish ex , his did no
ansla e o imp o ed FHIR con e sion and o en in oduced s anda diza ion challenges.
S anda dizing elec onic heal h eco d (EHR) da a using o ma s like FHIR o e s
signi ican po en ial o enhancing clinical da a in e ope abili y, enabling high- h oughpu
compu a ion, and p omo ing meaning ul use. To ad ance FHIR-based EHR da a modeling,
his s udy e alua ed he capaci y o LLMs o ex ac , s anda dize, and in eg a e in o ma ion
om uns uc u ed clinical na a i es in o FHIR esou ces. We belie e ha LLM-d i en
modeling o uns uc u ed EHR da a can play a c ucial ole in achie ing ad anced seman ic
in e ope abili y ac oss dispa a e EHR sys ems. I is impo an o emphasize ha he
FHIR o ma is highly es ic ed and complex, wi h clea ly de ined s anda ds. Ou esul s
indica e ha LLMs, when used wi h app op ia e p omp ing s a egies, can con ibu e
signi ican ly o his goal. This indica es ha he main challenge lies in he con e sion o
he ex ac ed in o ma ion in o a alid FHIR s uc u e, a he han he ini ial in o ma ion
iden i ica ion i sel .
7.5. Limi a ions and Fu u e Wo k
This s udy has some limi a ions. The e alua ion was conduc ed using epo s in Span-
ish om only wo clinics (skin ca e and mammog aphy). This limi s he gene alizabili y o
he indings o o he languages, clinical domains, and epo ypes. Fu u e wo k should
include a mo e di e se da ase o clinical epo s o assess he obus ness o he p oposed
me hods ac oss di e en clinical con ex s.
While we e alua ed se e al LLMs, including he specialized, heal hca e- ine uned
Llama3-Med42, esou ce cons ain s limi ed ou in-dep h analysis o his single specialized
model. This model p o ed unsui able o he FHIR con e sion ask due o i s ocus on
diagnosis. This highligh s he impo ance o selec ing models app op ia e o he speci ic
ask. Fu u e wo k should explo e LLMs explici ly ine uned o FHIR con e sion asks.
Fu he mo e, a key limi a ion is he lack o con ol o e he aining da a used o all LLMs.
These models a e ained on massi e ex co po a, making i impossible o know p ecisely
wha in o ma ion hey ha e lea ned and how his migh in luence hei pe o mance on
FHIR con e sion.
The quali a i e analysis was conduc ed by he au ho s, wi h GPT-4o used as a comple-
men a y ool. While GPT-4o p o ided addi ional insigh s, i exhibi ed limi a ions such as
occasional hallucina ions (iden i ying non-exis en e o s) and p o iding non-subs an ial
eedback, such as “The hallucina ion a e was ela i ely low bu could be imp o ed” o
“The esou ce mapping could be imp o ed o be e accu acy.” While he o e all ends
obse ed by GPT-4o we e in line wi h ou own obse a ions, he limi a ions o LLMs as
e alua o s o p ecision, hallucina ion, and esou ce mapping should be acknowledged.
This aligns wi h he indings om e . [
41
], which sugges ed e alua ing single me ics a a
ime o mo e eliable esul s, al hough a he cos o inc eased compu a ional esou ces.
Ou e alua ion p ima ily ocused on he s uc u al alidi y and in o ma ion con en
o he gene a ed FHIR bundles. We did no explici ly e alua e he accu acy o coded alues
mapped o s anda d e minologies and on ologies (e.g., LOINC, RxNo m, SNOMED CT)

Appl. Sci. 2025,15, 3379 22 o 28
o he use o locally main ained dic iona ies and look-up ables wi hin FHIR p o iles. This
is an impo an aspec o FHIR compliance ha should be add essed in u u e wo k.
Addi ionally, u u e esea ch could in es iga e mo e ad anced p omp ing echniques,
such as p omp chaining o ine uning on FHIR-speci ic da a, o u he imp o e con e -
sion accu acy and educe he isk o o e eliance on examples. Explo ing di e en LLM
a chi ec u es and sizes could also p o ide aluable insigh s. Addi ionally, de eloping
au oma ed me ics o e alua ing FHIR bundle quali y would enhance he objec i i y and
scalabili y o u u e s udies. Fu u e wo k should also e alua e he accu acy o mapping
clinical concep s o s anda d e minologies and on ologies wi hin FHIR esou ces. Finally,
u u e wo k could in es iga e he use o FHIR p o iles o u he cons ain and guide he
LLMs’ ou pu , ensu ing adhe ence o speci ic implemen a ion equi emen s.
8. Conclusions
This s udy explo ed using LLMs o con e uns uc u ed clinical epo s o FHIR bun-
dles, a key s ep o heal hca e da a in e ope abili y. We in es iga ed p omp ing s a egies
(basic, CoT, wo-s ep), ew-sho lea ning, empe a u e, and LLM selec ion (Gemini, Mis al,
Llama3-Med42).
P omp enginee ing p o ed pi o al in achie ing e ec i e FHIR con e sion. Basic
p omp ing yielded he lowes p ecision (4.71) and highes hallucina ion a es (7.25), high-
ligh ing he necessi y o de ailed guidance o LLMs. The wo-s ep con e sion me hod
achie ed he highes p ecision (7.30) and esou ce mapping accu acy (7.32), signi ican ly
ou pe o ming all o he app oaches. Chain-o - hough (CoT) p omp ing o e ed a mode a e
imp o emen o e basic p omp ing, wi h a p ecision o 5.98, bu did no ma ch he wo-s ep
me hod’s e ec i eness. Few-sho lea ning enhanced p ecision (7.07) and esou ce mapping
(6.90), while educing hallucina ions (4.71), bu i also in oduced a po en ial o e eliance on
examples. Tempe a u e had a less signi ican impac ; while highe empe a u es sligh ly in-
c eased p ecision (6.19 s. 5.97), hey also co ela ed wi h a mino inc ease in hallucina ion
isk (5.53 s. 5.57). Llama3-Med42 demons a ed unsui abili y o his ask, p io i izing
diagnos ic sugges ions o e accu a e FHIR mapping. Gemini gene ally ou pe o med
Mis al ac oss all me ics.
The key con ibu ions o his wo k include: (1) demons a ing he e ec i eness o he
wo-s ep con e sion app oach; (2) highligh ing he ade-o s inhe en in ew-sho lea ning;
(3) e alua ing he impac o empe a u e and LLM selec ion; (4) p o iding a comp ehensi e
e alua ion amewo k o FHIR con e sion using LLMs; and (5) add essing he challenge
o con e ing uns uc u ed EHR da a o FHIR.
In conclusion, LLMs demons a e signi ican po en ial o au oma ing FHIR con e -
sion, bu ca e ul p omp enginee ing, app op ia e use o ew-sho lea ning, and hough ul
pa ame e selec ion a e essen ial o op imal esul s. The wo-s ep con e sion s a egy
o e s a pa icula ly p omising pa h owa d gene a ing accu a e and alid FHIR bundles,
he eby ad ancing heal hca e da a in e ope abili y.
Au ho Con ibu ions: Concep ualiza ion, J.D. and J.C.; Me hodology, J.D. and D.G.; So wa e, J.D.;
Valida ion, D.G.; Fo mal analysis, D.G.; In es iga ion, J.D. and J.C.; W i ing— e iew & edi ing,
J.D. and J.C.; Supe ision, J.C.; Funding acquisi ion, J.C. All au ho s ha e ead and ag eed o he
published e sion o he manusc ip .
Funding: J.C. wo k was suppo ed by he To es Que edo g an “Asis en e Vi ual pa a la mejo a del
sis ema de salud—Vi ual Assis an o Be e Heal hCa e-VA4BHC” ( e . PTQ2021-012147).
Ins i u ional Re iew Boa d S a emen : No applicable.
In o med Consen S a emen : No applicable.
Appl. Sci. 2025,15, 3379 23 o 28
Da a A ailabili y S a emen : The o iginal con ibu ions p esen ed in his s udy a e included in he
a icle. Fu he inqui ies can be di ec ed o he co esponding au ho s.
Con lic s o In e es : The au ho s Julien Delaunay, Daniel Gi bes and Jo di Cusido we e employed
by he company Top Heal h Tech. The emaining au ho s decla e ha he esea ch was conduc ed
in he absence o any comme cial o inancial ela ionships ha could be cons ued as a po en ial
con lic o in e es .
Abb e ia ions
The ollowing abb e ia ions a e used in his manusc ip :
MDPI Mul idisciplina y Digi al Publishing Ins i u e
DOAJ Di ec o y o open access jou nals
LLM La ge Language Model
NLP Na u al Language P ocessing
ML Machine Lea ning
RAG Re ie al Augmen ed Gene a ion
CoT Chain-o -Though
Appendix A. S udy Design Choices
Resou ce Selec ion o FHIR Con e sion
We ca e ully selec ed a subse o FHIR esou ce ypes o he LLMs o use in con e ing
inpu ex . The chosen esou ce ypes we e Condi ion, Obse a ion, Medica ionS a emen ,
P ocedu e, Alle gyIn ole ance, Encoun e , Immuniza ion, and EpisodeO Ca e.
This decision was based on hei collec i e abili y o comp ehensi ely ep esen he
clinical complexi ies inhe en in mos possible scena ios. This cu a ed se encompassed a
b oad spec um o heal hca e ac i i ies, om diagnosis and ea men o pa ien his o y
and ollow-up. By ocusing on a small se , we es ablished a obus ounda ion o ensu e
ha he LLMs had he co ec ools no only o accu a ely model he majo i y o clinical
scena ios in hese special ies bu also o cap u e all de ails due o hei in insic complexi y.
The absence o esou ces speci ically o imaging is jus i ied by he ac ha he p ima y
ocus o his s udy was on ex ac ing in o ma ion om ex ual epo s. Fu he mo e, he
selec ed se o esou ces was su icien o ep esen ing he key elemen s o mammog aphy
and de ma ology epo s. The indings wi hin de ma ology and mammog aphy epo s
we e cap u ed using he Condi ion esou ce, while ins ances ela ed o imaging obse a-
ions we e eco ded using he Obse a ion esou ce. P ocedu es such as biopsies o su gical
in e en ions we e documen ed using he P ocedu e esou ce. Medica ions p esc ibed o
ea men we e cap u ed using he Medica ionS a emen esou ce, and pa ien alle gies and
in ole ances we e eco ded using he Alle gyIn ole ance esou ce. Encoun e s was used o
documen pa ien isi s and he se ices p o ided, while immuniza ions we e eco ded
using he Immuniza ion esou ce.
This selec ion allowed us o ensu e ha he LLMs ollowed ou guidelines, using he
mos ele an and comp ehensi e se o FHIR esou ces o he gi en clinical con ex s.
Appendix B. P omp Templa e
In his sec ion, we de ail he di e se p omp s we used o ask he la ge language models
(LLMs) o gene a e a ask. We i s p esen in Appendix B.1 he p omp s we used o make
he models con e a clinical case o FHIR. Then, in Appendix B.2, we ou line he p omp s
we used o e alua e he p ecision and hallucina ion a e o he con e sion gene a ed by
he LLMs. Tex be ween ‘<>’ ep esen s subs i u ions ha we e made o he p omp a he
in e ence ime, o example, p o iding he ex o he clinical epo .
Appl. Sci. 2025,15, 3379 24 o 28
Appendix B.1. Con e sion P omp s
To ensu e he accu a e con e sion o uns uc u ed clinical da a in o s uc u ed FHIR
o ma , we designed a se ies o p omp s o guide he LLMs h ough he p ocess. In his
sec ion, we p esen he p omp s used in ou s udy.
Fo he sys em p omp , we es ed se e al a ia ions o de e mine which one would
yield he bes esul s. We ound ha p o iding he model wi h a b ie explana ion o he
ask helped o imp o e he model’s pe o mance. We also expe imen ed wi h di e en
le els o de ail in he ins uc ions, anging om e y speci ic o mo e gene al. Fo example,
we ini ially p o ided de ailed ins uc ions on how o s uc u e each FHIR esou ce bu
ound ha his some imes led o o e ly complex ou pu s. Simpli ying he ins uc ions o
ocus on he key elemen s o each esou ce ype esul ed in mo e accu a e and consis en
ou pu s.
The p omp speci ied ha he LLMs should ac as an FHIR specialis and emphasized
he impo ance o accu a ely ep esen ing he o iginal ex . Tex in ed is he emo ional
s imulus we added o he p omp diagnosis, in line wi h Li e al. [
43
]. We eplaced he
<in o ma ion p o ided> and < ask> depending on whe he we used ze o-sho , ew-sho ,
wo-s ep easoning, o chain-o - hough (CoT) app oaches.
Appendix B.1.1. Ze o-Sho s. Few-Sho P omp ing
In he case o ze o-show and ew-sho p omp ing, we eplaced <in o ma ion p o ided>
wi h “I will p o ide you wi h he plain ex o a ile con aining uns uc u ed in o ma ion
abou a pa ien and you will e u n a alid JSON con aining he app op ia e esou ces o
ep esen he in o ma ion con ained in ha plain ex ” and we eplaced < ask> by “Display
he o iginal ex om he clinical case wi hou making any changes o adding any in o ma-
ion ha does no exis .” Fo he example-based p omp echnique, we illed in he de ails
o he clinical case and he associa ed FHIR esou ce. We also es ed di e en a ia ions o
he empla e, such as changing he o de o he symp oms o using di e en ph asing, o
see how hese changes would a ec he models’ pe o mance. We ound ha p o iding
clea and concise examples helped he models o unde s and he expec ed o ma and
s uc u e o he FHIR esou ces. This example was chosen o p o ide a comp ehensi e
and ep esen a i e se o FHIR esou ce ypes, ensu ing he obus ness and eliabili y o
ou indings.
Appl.Sci.2025,15,337924o 28

Thep omp speci ied ha  heLLMsshouldac asanFHIRspecialis andemphasized
heimpo anceo accu a ely ep esen ing heo iginal ex .Tex in edis heemo ional
s imulusweadded o hep omp diagnosis,inlinewi hLie al.[43].We eplaced he<in‐
o ma ionp o ided>and< ask>dependingonwhe he weusedze o‐sho , ew‐sho , wo‐
s ep easoning,o chain‐o ‐ hough (CoT)app oaches.

AppendixB.1.1.Ze o‐Sho  s.Few‐Sho P omp ing

In hecaseo ze o‐showand ew‐sho p omp ing,we eplaced<in o ma ionp o‐
ided>wi h“Iwillp o ideyouwi h heplain ex o a ilecon aininguns uc u edin o ‐
ma ionabou apa ien andyouwill e u na alidJSONcon aining heapp op ia e e‐
sou ces o ep esen  hein o ma ioncon ainedin ha plain ex ”andwe eplaced< ask>by
“Display heo iginal ex  om heclinicalcasewi hou makinganychangeso addingany
in o ma ion ha doesno exis .”Fo  heexample‐basedp omp  echnique,we illedin he
de ailso  heclinicalcaseand heassocia edFHIR esou ce.Wealso es eddi e en  a i‐
a ionso  he empla e,suchaschanging heo de o  hesymp omso usingdi e en 
ph asing, oseehow hesechangeswoulda ec  hemodels’pe o mance.We ound ha 
p o idingclea andconciseexampleshelped hemodels ounde s and heexpec ed o ‐
ma ands uc u eo  heFHIR esou ces.Thisexamplewaschosen op o ideacomp e‐
hensi eand ep esen a i ese o FHIR esou ce ypes,ensu ing he obus nessand elia‐
bili yo ou  indings.

Figu eA1.Sys emp omp used oins uc  heLLMoncon e inguns uc u edclinicalda ain o
s uc u edFHIR o ma .We eplaced<in o ma ionp o ided>and< ask>by heco esponding ex 
dependingon hep omp  echniqueused.Finally,<FHIRs uc u e>was eplacedwi hde ailedin‐
s uc iono  heFHIRs anda ds obe ollowed.
AppendixB.1.2.Basic s.Two‐S ep s.Chain‐o ‐Though Reasoning

Fo  heCoTsys emp omp ,wedesignedap omp  ha ins uc ed hemodel o
co ec  hegene a edFHIRbundlebasedon heo iginalclinical epo .Thus,we i s 
eplaced<in o ma ionp o ided>by“Iwillp o ideyouwi h heplain ex o a ilecon‐
aininguns uc u edin o ma ionabou apa ien andaJSON ha isaFHIRbundlewi h
hesein o ma ions uc u ed.Youwill e iew hep o idedFHIRbundleandco ec any
e o so inconsis encies.”Wealso eplaced< ask>wi h“Ensu e ha  hein o ma ionis
accu a ely ep esen edand ha  hes uc u eadhe es oFHIRs anda ds.”
Finally, hesys emp omp con ainedin o ma ionabou  heFHIRguideliness uc‐
u e ha wewan ed hemodel o ollow o eacho  heaccep ed esou ce ypes.Figu e
A2p o idesanexampleo how heins uc ion o  heLLMswasp esen edin hesys em
p omp  o  he esou ce ype“Condi ion.”Thisexampledemons a es hespeci ic o ma 
ands uc u e ha  heLLMsshould ollowwhengene a ing heCondi ion esou cein he
FHIRbundle.

Inou s udy,wele e aged hecapabili ieso Langchainand hecha s uc u eo  he
model o acili a e hecon e siono uns uc u edclinicalda ain o hes uc u edFHIR o ‐
ma .Once hesys emp omp wasse up,wep o ided hemodelwi h heplain ex o 
heclinical epo .Fo  hei e a i eco ec ionp ocess,weused hep omp de ailedin
Figu e A1. Sys em p omp used o ins uc he LLM on con e ing uns uc u ed clinical da a in o
s uc u ed FHIR o ma . We eplaced <in o ma ion p o ided> and < ask> by he co esponding ex
depending on he p omp echnique used. Finally, <FHIR s uc u e> was eplaced wi h de ailed
ins uc ion o he FHIR s anda ds o be ollowed.
Appendix B.1.2. Basic s. Two-S ep s. Chain-o -Though Reasoning
Fo he CoT sys em p omp , we designed a p omp ha ins uc ed he model o co ec
he gene a ed FHIR bundle based on he o iginal clinical epo . Thus, we i s eplaced
<in o ma ion p o ided> by “I will p o ide you wi h he plain ex o a ile con aining
uns uc u ed in o ma ion abou a pa ien and a JSON ha is a FHIR bundle wi h hese
in o ma ion s uc u ed. You will e iew he p o ided FHIR bundle and co ec any e o s
Appl. Sci. 2025,15, 3379 25 o 28
o inconsis encies.” We also eplaced < ask> wi h “Ensu e ha he in o ma ion is accu a ely
ep esen ed and ha he s uc u e adhe es o FHIR s anda ds.”
Finally, he sys em p omp con ained in o ma ion abou he FHIR guidelines s uc u e
ha we wan ed he model o ollow o each o he accep ed esou ce ypes. Figu e A2
p o ides an example o how he ins uc ion o he LLMs was p esen ed in he sys em
p omp o he esou ce ype “Condi ion.” This example demons a es he speci ic o ma
and s uc u e ha he LLMs should ollow when gene a ing he Condi ion esou ce in he
FHIR bundle.
In ou s udy, we le e aged he capabili ies o Langchain and he cha s uc u e o he
model o acili a e he con e sion o uns uc u ed clinical da a in o he s uc u ed FHIR
o ma . Once he sys em p omp was se up, we p o ided he model wi h he plain ex
o he clinical epo . Fo he i e a i e co ec ion p ocess, we used he p omp de ailed in
Figu e A3 and we eplaced <FHIR gene a ed> wi h he bundle gene a ed in he p e ious
ound and <clinical case> wi h he plain ex o he clinical epo . This i e a i e app oach
allowed us o e ine he con e sion p ocess, ensu ing ha he gene a ed FHIR bundles
we e accu a e and consis en wi h he o iginal clinical da a.
Appl.Sci.2025,15,337925o 28

Figu eA3andwe eplaced<FHIRgene a ed>wi h hebundlegene a edin hep e ious
oundand<clinicalcase>wi h heplain ex o  heclinical epo .Thisi e a i eapp oach
allowedus o e ine hecon e sionp ocess,ensu ing ha  hegene a edFHIRbundles
we eaccu a eandconsis en wi h heo iginalclinicalda a.

Figu eA2.Exampleo how heins uc ion o  heLLMisp esen edin hesys emp omp  o  he
esou ce ypeCondi ion.Thisexampledemons a es hespeci ic o ma ands uc u e ha  heLLMs
should ollowwhengene a ing heCondi ion esou cein heFHIRbundle.

Figu eA3.P omp used o  hei e a i ep ocesso con e inguns uc u edda a oFHIR.The
p omp ins uc s heLLM oco ec  hegene a edFHIRbundlebasedon heo iginalclinical epo .
I speci ies ha  heLLMshould e u nonly heco ec edFHIRbundlewi hou addinganyaddi‐
ionalin o ma ionandshoulduseonly hespeci ied esou ce ypes os o e hein o ma ion.
AppendixB.2.E alua o P omp s

Fo  hee alua ionp omp ,weusedas uc u edapp oach oassess hep ecisionandhalluci‐
na ion a eo  hegene a edFHIRbundles.Weins uc ed hee alua o  oe alua e hep ecision
andhallucina ion a eo  hegene a edbundle.Wep o ideinFigu eA4 heguidelines o assessing
heaccu acyand heex en o hallucina ionin hegene a edbundle.We es eddi e en  a ia ions
o  hee alua ionp omp ,suchaschanging hescaleused o sco ingo p o idingaddi ionalcon‐
ex  o  hee alua ion.We ound ha aclea ands uc u ede alua ionp omp helped oensu e
consis en and eliablee alua ion esul s.

Figu eA4.P omp usedby hee alua o  oe alua e hep ecisionandhallucina ionsco eo  he
con e sion.Thep omp ins uc s hee alua o  oe alua e hep ecisionandhallucina ion a eo 
Figu e A2. Example o how he ins uc ion o he LLM is p esen ed in he sys em p omp o he
esou ce ype Condi ion. This example demons a es he speci ic o ma and s uc u e ha he LLMs
should ollow when gene a ing he Condi ion esou ce in he FHIR bundle.
Appl.Sci.2025,15,337925o 28

Figu eA3andwe eplaced<FHIRgene a ed>wi h hebundlegene a edin hep e ious
oundand<clinicalcase>wi h heplain ex o  heclinical epo .Thisi e a i eapp oach
allowedus o e ine hecon e sionp ocess,ensu ing ha  hegene a edFHIRbundles
we eaccu a eandconsis en wi h heo iginalclinicalda a.

Figu eA2.Exampleo how heins uc ion o  heLLMisp esen edin hesys emp omp  o  he
esou ce ypeCondi ion.Thisexampledemons a es hespeci ic o ma ands uc u e ha  heLLMs
should ollowwhengene a ing heCondi ion esou cein heFHIRbundle.

Figu eA3.P omp used o  hei e a i ep ocesso con e inguns uc u edda a oFHIR.The
p omp ins uc s heLLM oco ec  hegene a edFHIRbundlebasedon heo iginalclinical epo .
I speci ies ha  heLLMshould e u nonly heco ec edFHIRbundlewi hou addinganyaddi‐
ionalin o ma ionandshoulduseonly hespeci ied esou ce ypes os o e hein o ma ion.
AppendixB.2.E alua o P omp s

Fo  hee alua ionp omp ,weusedas uc u edapp oach oassess hep ecisionandhalluci‐
na ion a eo  hegene a edFHIRbundles.Weins uc ed hee alua o  oe alua e hep ecision
andhallucina ion a eo  hegene a edbundle.Wep o ideinFigu eA4 heguidelines o assessing
heaccu acyand heex en o hallucina ionin hegene a edbundle.We es eddi e en  a ia ions
o  hee alua ionp omp ,suchaschanging hescaleused o sco ingo p o idingaddi ionalcon‐
ex  o  hee alua ion.We ound ha aclea ands uc u ede alua ionp omp helped oensu e
consis en and eliablee alua ion esul s.

Figu eA4.P omp usedby hee alua o  oe alua e hep ecisionandhallucina ionsco eo  he
con e sion.Thep omp ins uc s hee alua o  oe alua e hep ecisionandhallucina ion a eo 
Figu e A3. P omp used o he i e a i e p ocess o con e ing uns uc u ed da a o FHIR. The
p omp ins uc s he LLM o co ec he gene a ed FHIR bundle based on he o iginal clinical epo .
I speci ies ha he LLM should e u n only he co ec ed FHIR bundle wi hou adding any addi ional
in o ma ion and should use only he speci ied esou ce ypes o s o e he in o ma ion.
Appendix B.2. E alua o P omp s
Fo he e alua ion p omp , we used a s uc u ed app oach o assess he p ecision and
hallucina ion a e o he gene a ed FHIR bundles. We ins uc ed he e alua o o e alua e
he p ecision and hallucina ion a e o he gene a ed bundle. We p o ide in Figu e A4 he
guidelines o assessing he accu acy and he ex en o hallucina ion in he gene a ed bundle.
We es ed di e en a ia ions o he e alua ion p omp , such as changing he scale used
o sco ing o p o iding addi ional con ex o he e alua ion. We ound ha a clea and
s uc u ed e alua ion p omp helped o ensu e consis en and eliable e alua ion esul s.