La ge Language Models: A Su ey o Su eys
MAX HORT,Simula Resea ch Labo a o y, No way
FERNANDO VALLECILLOS-RUIZ,Simula Resea ch Labo a o y, No way
LEON MOONEN∗,Simula Resea ch Labo a o y, No way
No only did he g owing in e es in La ge Language Models (LLMs) lead o a mul i ude o applica ions,
news a icles, social media pos s, and new p oduc s, bu i also esul ed in a signi ican inc ease in esea ch
publica ions. To gain a be e unde s anding o his as numbe o publica ions, su eys help p o ide
p ac i ione s and esea che s wi h much-needed o e iews.
Howe e , we ha e eached a poin wi h ens o housands o LLM publica ions, and he numbe o su eys
on LLM publica ions has g own in o hund eds. I onically, he same su eys ha se ou o b ing o de and
s uc u e now con ibu e o he con olu ion o he space. Fo example, someone in e es ed in he use o LLMs
o he heal h sec o has mo e han 80 po en ial su eys o choose om. To add ess his challenge, we ca y
ou a e ia y li e a u e e iew o ga he and analyze LLM- ela ed su eys, e iews, and mapping s udies. By
doing so, we aim o help p ac i ione s and esea che s na iga e he as a ay o exis ing su eys.
In o al, we ound 424 LLM su eys ha ha e been published up o Sep embe 2024 ha a e included in his
s udy. We de ise a axonomy and ca ego ise su eys acco ding o hei main ocus (e.g., ine- uning o LLMs,
applica ion o so wa e enginee ing asks). To u he suppo he na iga ion o LLM su eys and keep up o
da e, we c ea ed a Gi Hub eposi o y ha ex ends ou scope o a o al o 984 publica ions published up o
Augus 2025, which is a ailable om h ps://gi hub.com/da aSED-condenSE/LLM-Su ey-Su ey.
CCS Concep s: •Gene al and e e ence
→
Su eys and o e iews;•Compu ing me hodologies
→
Na u al language p ocessing.
Addi ional Key Wo ds and Ph ases: la ge language model, li e a u e su ey, e ia y s udy
1 In oduc ion
Summa ies a e use ul ools o p o iding o e iews ha help acili a e he unde s anding o di e se
opics. In esea ch, summa ies a e ypically conduc ed h ough seconda y s udies (e.g., su ey,
mapping s udy, li e a u e e iew) [
83
,
126
]. Going one s ep u he , e ia y e iews sys ema ically
analyze and summa ize seconda y s udies [
148
]. Such e ia y e iews ha e p o en use ul ac oss
a ious domains, such as so wa e enginee ing [
43
,
83
], medical deep lea ning [
62
], economics [
46
],
machine lea ning [150], equi emen s enginee ing [12], and sen imen analysis [181].
The su ging popula i y o La ge Language Models (LLMs) o e he ecen yea s has led o
housands o esea ch a icles and, in u n, hund eds o seconda y s udies. This olume o seconda y
s udies c ea es a need o a s uc u ed o e iew, and we a gue ha i is ime o conduc a e ia y
s udy o he ield o LLMs. By c ea ing an o e iew, we suppo esea che s and p ac i ione s in
na iga ing he ield o su eys, inding ele an ones when lea ning abou LLMs, and unde s anding
which aspec s ha e been s udied when designing new su eys.
∗Co esponding au ho .
Au ho s’ Con ac In o ma ion: Max Ho , [email p o ec ed], Simula Resea ch Labo a o y, Oslo, No way; Fe nando Vallecillos-
Ruiz, [email p o ec ed], Simula Resea ch Labo a o y, Oslo, No way; Leon Moonen, leon.moonen@compu e .o g, Simula
Resea ch Labo a o y, Oslo, No way.
This wo k is licensed unde a C ea i e Commons A ibu ion 4.0 In e na ional License.
2 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
871
18
15
80
28
11
33
This Su ey
Big Su ey
Awesome Su ey
Fig. 1. Venn diag am illus a ing he o e lap be ween su eys om ou sys ema ic sea ch and wo eposi o ies.
To he bes o ou knowledge, no e ia y s udy on LLMs has been published. Howe e , we a e
awa e o wo Gi Hub eposi o ies ha p o ide a lis o LLMs su eys. These a e: ABigSu ey-
O LLMs
1
and Awesome-LLM-Su ey,
2
wi h 159 and 139 lis ed su eys on LLMs espec i ely. While
aluable esou ces, hese su eys did no ollow a sys ema ic sea ch me hodology. We ex end a
beyond hei scope by ca ying ou a sys ema ic li e a u e sea ch and collec ing a o al o 424
su eys ha a e included in his a icle, as well as an addi ional 560 ha a e included in ou Gi Hub
eposi o y (see Sec ion 2 o mo e de ails). The o e lap o hese wo eposi o ies and ou collec ed
su eys is shown in Figu e 1, highligh ing he addi ional s udies collec ed.
Figu e 2shows he s uc u e o ou su ey and how we ca ego ize he exis ing su eys.
2 Su ey Me hodology and Sea ch Resul s
2.1 Sea ch P ocedu e
To sea ch he li e a u e o seconda y s udies abou LLMs, we make use o he publica ion da abase
dblp.
3
dblp con ains publica ions om mo e han 1,800 jou nals and 6,000 con e ences om he
compu e science domain, as well as non-pee - e iewed pape s om a Xi . In pa icula , we use
dblp o ca y ou a sea ch o ele an publica ions based on il e ing hei i les.
To ensu e ha we ob ain ele an sea ch esul s, we de ine wo se s o keywo ds. The i s se
con ains e ms ela ed o language models, while he second con ains e ms ela ed o li e a u e
collec ion (inspi ed by Ko i e al. [150]):
•LLM keywo ds: LLM, Language Model.
•
Su ey keywo ds: Su ey, O e iew, Li e a u e, Re iew, Backg ound, Resea ch, Taxonomy,
Sys ema ic.
In addi ion o equi ing ha publica ion i les con ain bo h keywo d ypes, we ea hem as
inclusion c i e ia o ou pape collec ion:
(1) LLMs: The pape ocuses on language models.
(2)
Li e a u e o e iew: The pape ep esen s a seconda y s udy by collec ing and p esen ing
o he wo ks.
We exclude all s udies ha do no ma ch hese c i e ia, and omi s udies ha a e no w i en in
English. We check inclusion in wo s ages. Fi s , we de e mine ele ance o he sea ch esul s based
on hei i le. Fo ins ance, his emo es li e a u e on he s udy o “language modeling”. Second, we
ead each pape wi h a sui able i le and make a inal inclusion decision based on i s con en .
1h ps://gi hub.com/NiuT ans/ABigSu eyO LLMs, las upda ed on 19 h Feb ua y 2025
2h ps://gi hub.com/HqWu-HITCS/Awesome-LLM-Su ey, las upda ed on 25 h o May 2025
3h ps://dblp.o g
La ge Language Models: A Su ey o Su eys 3
LLM Su eys
Risks and Th ea s
(Sec ion 8)
...
Secu i y and P i acy
Hallucina ion
Fai ness and Bias
Mul imodali y
(Sec ion 7)...
G aph
Visual
Applica ions
(Sec ion 6)...
So wa e and Code
Medical and Heal h
Capabili ies
(Sec ion 5)
Augmen ed
Eme gen
Basic
Componen s o
LLMs (Sec ion 4)
E alua ion
In e ence
Agen s
P omp ing
Alignmen
T aining and Lea ning
Da a
A chi ec u e
Ha dwa e and Se ing
Comp ehensi e
Su eys (Sec ion 3)His o y
Bibliome ics
Gene al
Fig. 2. S uc u e o his s udy.
2.2 Selec ion and Sea ch Resul s
Table 1summa izes he esul s o ou sea ch, which we ca ied ou on 10 h o Sep embe 2024.
We s a wi h a o al o 1,173 unique publica ions om dblp, which i a leas one o he keywo d
combina ions. 461 o hese ag ee wi h ou inclusion c i e ia acco ding o hei i les. A e examining
he 461 pape s, we exclude 37 and end up wi h a o al o 424 s udies ha a e included in ou su ey
and p esen ed in he ollowing sec ions. To ensu e he imeliness o ou wo k, we ca ied ou an
iden ical sea ch on he 15 h o Augus 2025, o ind su eys ha ha e been published in he las
yea . This esul ed in an addi ional 560 su eys. Due o he la ge numbe o ecen publica ions, we
4 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
Table 1. Summa y o sea ch esul s. The sea ch was ca ied ou on he 10 h o Sep embe 2024 on dblp.
Resul s o he upda ed sea ch ca ied ou on he 15 h o Augus 2025 a e shown in (blue). The addi ional 560
s udies can be ound in ou Gi Hub eposi o y.
LLM Keywo d # Pape s
Su ey Keywo d “LLM” “Language Model” Unique Ti le Con en
“Su ey” 63 (+181) 427 (+476)
“O e iew” 6 (+12) 43 (+35)
“Li e a u e” 18 (+54) 65 (+91)
“Re iew” 43 (+146) 231 (+291) 1,173 461 424
“Backg ound” 0 (+2) 8(+9) (+1,672) (+560)
“Resea ch” 33 (+146) 201 (+241)
“Taxonomy” 14 (+40) 48 (+51)
“Sys ema ic” 25 (+87) 134 (+169)
2020 2021 2022 2023 2024 2025
Yea
0
100
200
300
400
500
Numbe o publica ions
3416
92
435 434
Fig. 3. Numbe o publica ions pe yea . The coun o 2025 is based on a cu -o da e o 15 h o Augus 2025.
decided o only include hem in ou supplemen a y online eposi o y.4The empo al dis ibu ion
o hese publica ions is shown in Figu e 3.
3 O e iews and Comp ehensi e Su eys
We s a he p esen a ion o exis ing su eys by p esen ing gene al o e iews ha a e help ul as
an in oduc ion o lea n abou LLMs. These su eys s and ou by hei comp ehensi eness and
he conside a ion o se e al aspec s o LLMs. In addi ion o comp ehensi e su eys, we include
su eys abou bibliome ics and he his o y o LLMs o p o ide ex ensi e backg ound de ails.
Table 1shows he included su eys o his sec ion. Fo each, we lis basic in o ma ion (Au ho s,
enue, yea o publica ion), as well as he numbe o e e ences hey ha e and how o en hey ha e
been ci ed. These can be use ul indica o s o hei comp ehensi eness and popula i y. Las ly, we
gi e a Unique Selling Poin (USP), a poin o ocus which di e en ia es hem om o he su eys.
3.1 Comp ehensi e
Zhao e al
. [417]
c ea ed he mos comp ehensi e su ey on LLMs o da e. This can no only be seen
by he high numbe o e e ences included (946), bu also i s popula i y (o e 5000 ci a ions). This
scale allows he su ey o co e all impo an aspec s o LLMs and is he only su ey o conside
aspec s such as “scaling laws”, which ha e no been co e ed by he o he su eys.
4h ps://gi hub.com/da aSED-condenSE/LLM-Su ey-Su ey
La ge Language Models: A Su ey o Su eys 5
Table 2. O e iew o comp ehensi e su eys and hei unique selling poin . The numbe o ci a ions was
collec ed om Google Schola on he 21s o Augus 2025. The # s udies shows how many publica ions a e
co e ed by he espec i e su eys.
Au ho s Venue Yea # S udies Ci a ions Focus USP
Mo a e al. [222] NAACL 2024 59 24 Bibliome ics Indus y and Academia ( oles)
Fan e al. [65] a Xi 2023 86 199 Bibliome ics Resea ch opics
Na eed e al. [223] a Xi 2023 487 1496 Comp ehensi e A chi ec u e de ails
Raiaan e al. [254] IEEE Access 2024 187 667 Comp ehensi e Da ase s pe model
Zhao e al. [417] a Xi 2023 946 5658 Comp ehensi e De ailed se ings
Yang e al. [368] ACM TKDD 2024 143 1212 Comp ehensi e NLP asks
Minaee e al. [218] a Xi 2024 243 1263 Comp ehensi e Capabili ies
Liu e al. [196] a Xi 2024 175 144 Comp ehensi e T aining & In e ence
Ling e al. [185] a Xi 2024 297 57 Comp ehensi e Specializa ion
Guo and Yu [93] a Xi 2022 175 34 Comp ehensi e Domain Adap a ion
Wang e al. [317] a Xi 2024 305 35 Comp ehensi e Challenges and Oppo uni ies
Miao e al. [216] a Xi 2023 375 103 Comp ehensi e Sys ems and Se ing
Wei e al. [330] a Xi 2023 223 74 His o y Con en ional models and linguis ic uni s
Chu e al. [38] a Xi 2024 88 93 His o y Ad ancemen o LLMs
Kuma [153] A i . In ell. Re . 2024 249 138 His o y Wo d embeddings, Deep Lea ning
The e a e se e al o he su eys ha p o ide a comp ehensi e o e iew o LLMs. While some o
hei con en s na u ally o e lap, we ou line hei unique iewpoin s. Raiaan e al
. [254]
p o ided
an o e iew o he di e en sou ces o da ase s (e.g., webpages, books, code). Na eed e al
. [223]
lis ed de ails on he a chi ec u e o LLMs. This includes in o ma ion such as aining objec i e,
ocabula y size, ype o a en ion, numbe o laye s, a en ion heads, and hidden s a es. Minaee
e al. [
218
] p o ided an o e iew o he capabili ies o language models. Mo eo e , hey su ey
he componen s necessa y o building LLMs. Miao e al
. [216]
co e ed he se ing o LLMs and
op imiza ion o as e in e ence ime ia modi ying he models hemsel es o he hos ing sys em.
Yang e al. [
368
] include he mos comp ehensi e desc ip ion o NLP asks o LLMs. The su ey
by Liu e al
. [196]
ocused on aining and in e ence, anging om he da a p ocessing s age
o di e en ine- uning pa adigms and me hods o speeding up he in e ence. Ling e al
. [185]
add essed he adap a ion o LLMs o di e en domains in hei su ey. These echniques ange om
augmen a ion wi h ex e nal knowledge o ine- uning. Simila ly, Guo and Yu
[93]
desc ibed domain
adap a ion ia da a augmen a ion, model op imiza ion ( aining) and model pe sonaliza ion.
3.2 Bibliome ic
Fan e al. [
65
] ca ied ou a bibliome ic s udy co e ing 5752 publica ions om he Web o Science
(WoS) Co e Collec ion, collec ed om 2017 o ea ly 2023. They in es iga ed opics add essed by
hese publica ions and di ided hem in o i e ca ego ies: algo i hm and NLP asks, medical and
enginee ing applica ions, social and humani a ian applica ions, c i ical s udies, and in as uc u e.
Among hese, “Algo i hm and NLP asks” span he majo i y o publica ions (54%), while “In as-
uc u e” and “C i ical s udies” co e less han 2% each. The coun ies which p oduced he highes
numbe o esea ch in his pe iod a e China and he USA. In e ms o he collabo a ion among
ins i u es, USA and UK ha e he highes cen ali y sco e.
Mo a e al
. [222]
pe o med a s udy o e eal he in luence o LLMs on AI esea ch, and analyzed
16,979 LLM- ela ed pape s om a Xi du ing he pe iod o Janua y 2018 o Sep embe o 2023. They
obse ed ha many au ho s ha e no p e iously published NLP- ela ed esea ch, and a g owing
in e es on he socie al impac o LLMs. Simila o he indings by Fan e al. [
65
], US and China-based
ins i u es con ibu ed he highes numbe o publica ions. O e all, Mo a e al
. [222]
obse ed ew
collabo a ions ac oss coun ies.
6 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
3.3 His o y
Ano he h ee su eys ou line cu en ad ances in language models while p o iding in o ma ion
on he his o y and ea ly app oaches [
38
,
153
,
330
]. Fo ins ance, Wei e al. [
330
] s a ed hei su ey
wi h an o e iew on con en ional language models (e.g., s uc u al and bidi ec ional language
models), while also desc ibing a ious linguis ic uni s (i.e., cha ac e s, wo ds, subwo ds, ph ases,
sen ences). The ole o wo d embeddings and deep lea ning o language models is add essed by
Kuma [
153
]. Chu e al
. [38]
conside ed app oaches anging om 1990 (s a is ical language models)
o 2023 (la ge language models).
4 Componen s o LLMs
This sec ion ou lines su eys ha add essed he di e en componen s equi ed o he aining
and use o LLMs. We s uc u e ou e iew a ound nine key componen s as shown in Figu e 4. This
axonomy is inspi ed by he wo k o Na eed e al. [223] and Minaee e al. [218].
4.1 Ha dwa e and Se ing
LLMs a e compu e-in ensi e machine lea ning models and he e o e equi e a ce ain deg ee o
compu e powe and ha dwa e in as uc u e o be used. Fo ins ance, he unning o LLMs can
bene i om he use o GPUs [
305
] and high-pe o mance compu ing [
31
]. The e iciency o aining
and applying LLMs has been imp o ed om a di e se ange o componen s [
138
,
308
], such as
p ocessing uni s, s o age sys ems, scheduling, and memo y managemen [
55
,
61
,
163
,
305
,
356
,
386
,
428
]. In addi ion, he e a e wo dedica ed su eys on imp o ing e iciency ia he key- alue (KV)
cache [
274
] (du ing aining and in e ence) and compu e-in-memo y (i.e., educes o e head o
memo y access by pe o ming compu a ions in memo y) [334].
A equen ly men ioned app oach o accele a ing he aining p ocess is pa alleliza ion [
9
,
18
,
55
,
61
,
305
]. He e, Duan e al
. [61]
men ioned h ee di e en ypes (Hyb id, Au o, He e ogeneous) and
desc ip ions on op imizing communica ion. Ano he ha dwa e conside a ion is he de ice on which
LLMs a e un. These can be edge de ices [14,250,356] o in he cloud [9,163,260,356,386,428].
4.2 A chi ec u e
In his sec ion, we p esen su eys ha desc ibe exis ing model ypes and in o ma ion on hei
a chi ec u es. Fo ins ance, Gao e al. [
79
] lis ed models and p o ided de ails, such as hei numbe
o pa ame e s and unde lying base models. In addi ion, hey e alua ed 32 o hem in a ious se ings
(e.g., ze o-sho , ew-sho , mul i-modal) and p esen ed ools ha suppo he de elopmen wi h and
o LLMs. Pahune and Chand asekha an
[229]
showed he di e en a ailable e sions o each o
he models and ha dwa e de ails o hei implemen a ion.
O he su eys ocus on speci ic model amilies. Kuk eja e al
. [152]
conside ed open-sou ce mod-
els, wi h pa icula ocus on FALCON, BLOOM, and Llama2, o which da a collec ion, a chi ec u e,
and aining s ages a e desc ibed. Kalyan
[139]
ocused on GPT language models, in pa icula
models anging om GPT-3 o GPT-4, and collec ed hei applica ion o downs eam asks (e.g., ex
classi ica ion, in o ma ion ex ac ion, coding). Alipou e al
. [5]
ocused on Cha GPT and OpenAI
(e.g., he OpenAI playg ound). O he models we e in oduced as al e na i es o Cha GPT. Lu e al
.
[201]
conside ed di e en me hods o LLM collabo a ion. Fo ins ance, LLM esponses can be
me ged, o one can c ea e an ensemble o mul iple LLMs.
La ge Language Models: A Su ey o Su eys 7
Componen s
E alua ion
(Sec ion 4.9)[
27
,
94
,
159
,
235
,
432
]
In e ence
(Sec ion 4.8)Dynamic Ac-
cele a ion
[
9
,
145
,
305
,
320
,
344
,
351
,
356
,
386
,
391
,
428
]
Model Comp ession
[
9
,
28
,
55
,
134
,
232
,
260
,
305
,
320
,
351
,
356
,
359
,
365
,
386
,
428
,
430
]
Agen s (Sec ion 4.7)
[
13
,
23
,
77
,
92
,
98
,
101
,
111
,
118
,
171
,
177
,
207
,
255
,
313
,
343
,
409
,
414
]
P omp ing
(Sec ion 4.6)
[
17
,
25
,
29
,
66
,
80
,
90
,
112
,
120
,
136
,
166
,
193
,
205
,
243
,
264
,
302
,
346
,
427
]
Alignmen
(Sec ion 4.5)
[
22
,
32
,
84
,
98
,
128
,
147
,
197
,
271
,
278
,
296
,
325
,
326
,
340
]
T aining and Lea n-
ing (Sec ion 4.4)
Unlea ning [
16
,
251
,
360
]
Inc emen al
Lea ning [
137
,
272
,
341
,
374
,
419
]
Fine-Tuning [
9
,
55
,
213
,
251
,
265
,
305
,
333
,
355
,
356
,
401
]
P e-T aining [
9
,
55
,
61
,
68
,
149
,
305
,
356
]
Da a (Sec ion 4.3)
Con amina ion [
51
,
230
,
256
,
350
]
Anno a ion and
Gene a ion [
199
,
291
]
Selec ion [
4
,
9
,
55
,
305
,
312
,
328
,
356
,
370
]
Da ase s [
58
,
195
,
236
,
261
,
282
,
371
]
A chi ec u e
(Sec ion 4.2)
[
5
,
79
,
139
,
152
,
201
,
229
]
Ha dwa e and
Se ing (Sec ion 4.1)
[
9
,
14
,
18
,
31
,
55
,
61
,
138
,
163
,
250
,
260
,
274
,
305
,
308
,
334
,
356
,
386
,
428
]
Fig. 4. Taxonomy o su eys on LLM componen s.
4.3 Da a
The cha ac e is ics o an LLM a e undamen ally de e mined by he da a used in i s c ea ion and
e alua ion. Consequen ly, su eys in his ield explo e he en i e da a li ecycle, om he composi ion
o da ase s o he e alua ion o hei quali y.
Da ase s: Liu e al
. [195]
p esen ed an exhaus i e o e iew o da ase s o la ge language models.
They conside ed a o al o 444 da ase s om i e ca ego ies: p e- aining, ins uc ion ine- uning,
p e e ence, e alua ion, and NLP. S i as a a and Memon
[282]
p esen ed 52 da ase s o he open-
domain ques ion-answe ing asks, and he s udy by Yang e al
. [371]
e iewed da ase s o causal
easoning benchma ks. Rö ge e al. [261] p esen ed 102 da ase s o sa e y e alua ion.
8 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
While la ge da ase s can be bene icial o LLM pe o mance, one needs o be ca e ul when ob-
aining da a om public sou ces. Challenges aced when using web-mined co po a o p e- aining
LLMs ha e been e iewed by Pe elkiewicz and Poswia a
[236]
. Among o he s, hey p esen ed
challenges based on sensi i e in o ma ion, bias o he low quali y o da a. Du e al
. [58]
ga he ed 32
da ase s (16 o p e- aining and 16 o ine- uning) while ocusing on hei quali y and quan i y.
Da a selec ion: Da ase s o aining LLMs con ain an eno mous amoun o samples wi h a ying
cha ac e is ics. While all da a samples can be used o aining, one can also selec he ones mos
sui able o one’s goal. Fo his pu pose, he amoun o aining da a can be educed ia deduplica ion,
sampling o selec ion [
9
,
55
,
305
,
356
]. Albalak e al. [
4
] su eyed da a selec ion o LLMs. Me hods
a e o ganized based on he ype o da a (e.g., da a selec ion o p e- aining, in-con ex lea ning).
Addi ionally, hey p o ided an o e iew o he main objec i es o da a selec ion a each s age o
he aining p ocess (e.g., he main objec i e o da a selec ion o ine- uning is bias educ ion and
model pe o mance). Wang e al. [312] specialized in selec ing da a o ins uc ion uning.
Wang e al. [
328
] conside ed da a collec ion om a da a managemen pe spec i e. This includes
conce ns ega ding da a quali y and quan i y (e.g., il e ing s a egies) o p e- aining and ine-
uning da ase s. Las ly, [
370
] su eyed he impac o adding sou ce code o he aining da a o
LLMs, and ound ha i can imp o e downs eam pe o mance.
Da a anno a ion and gene a ion: While da ase s o aining LLMs a e o en ob ained om
human-c ea ed sou ces, LLMs hemsel es can be used o augmen o enhance exis ing da a, be
i by gene a ing new da a om sc a ch (da a gene a ion) o p o iding addi ional in o ma ion o
exis ing da a (da a anno a ion). Fo ins ance, Long e al. [
199
] su eyed syn he ic da a gene a ion
wi h LLMs o ou line he wo k low o da a gene a ion, consis ing o gene a ion, cu a ion, and
e alua ion o syn he ic da a. Tan e al. [
291
] conside ed di e en ace s o he da a anno a ion
p ocess wi h LLMs (gene a ion, assessmen and u iliza ion).
Da a con amina ion: Da a con amina ion is a p oblem ha a ises when he aining da a o
LLMs o e lap wi h he e alua ion benchma ks. We ound ou su eys summa izing app oaches
o de ec ion and mi iga ion o da a con amina ion. Pala alli e al
. [230]
conside ed wo se e i ies
o da a con amina ion (ins ance le el, da ase le el) and examined hem in wo case s udies (i.e.,
summa iza ion, ques ion answe ing). Xu e al. [
350
] conside ed he se e i y o da a con amina ion
(i.e., seman ic, in o ma ion, da a, label le el) and p esen ed se e al asks whe e con amina ion has
been obse ed (e.g., code gene a ion, sen imen analysis). Deng e al. [
51
] conside ed language
model ypes (whi e-box, g ay-box, black-box LLMs) when i comes o da a con amina ion as
well as se e al me hods o de ec ing da a con amina ion. Las ly, Ra au e al
. [256]
o ganized
con amina ion de ec ion app oaches based on open-da a (da ase is known) and closed-da a (da ase
is no known).
4.4 T aining and Lea ning
By le e aging la ge amoun s o da a, LLMs lea n pa e ns ha shape hei pe o mance ac oss
di e en s ages. F om ini ial p e- aining o con inuous adap a ion, hese s ages allow hem o
acqui e and e ine hei capabili ies.
P e- ain: P e- aining desc ibes he ini ial aining s age o LLMs, in which models lea n a
gene al unde s anding o ex s and language. Ko ei and Thi una uka asu
[149]
su eyed di e en
p e- aining echniques ( om sc a ch, incessan p e aining, based on knowledge inhe i ance,
mul i- ask p e- aining). A e wa ds, hey discussed how his knowledge can be ans e ed o
downs eam asks ia ine- uning. Fang e al
. [68]
e iewed me ics o conside o he aining
p ocess and moni o ing o he aining success. While we ound no o he dedica ed s udies, se e al
p e- aining echniques ha e been co e ed by comp ehensi e su eys. Fo ins ance, he mos
La ge Language Models: A Su ey o Su eys 9
equen ly conside ed me hod o imp o ing he e iciency o he p e- aining p ocess is mixed-
p ecision aining [9,55,61,305,356].
Fine-Tune: A e p e- aining, LLMs can be ine- uned o speci ic asks, which usually in ol es
smalle da ase s o highe quali y. Weng
[333]
conside ed se e al ine- uning pa adigms, such as
mul i- ask lea ning, knowledge dis illa ion, ans e lea ning, and ew-sho lea ning. O he su eys
conside ed speci ic lea ning pa adigms, such as ede a ed lea ning [
251
], mul i- ask lea ning [
265
],
o ins uc ion- uning [
401
]. A la ge subse o su eys add essed he e iciency o he ine- uning
p ocess ia Pa ame e E icien Fine-Tuning (PEFT) [9,55,305,355,356].
Xu e al. [
355
] co e ed he e iciency o he aining o LLMs by PEFT me hods. Ra he han uning
he en i e model (all pa ame e s), a limi ed subse is ine- uned o sa e ime and memo y. They
ca ego ized PEFT me hods in o 5 ypes: addi i e ine- uning, pa ial ine- uning, epa ame e ized
ine- uning, hyb id ine- uning, and uni ied ine- uning. In addi ion o he collec ion and desc ip ion
o a mul i ude o PEFT me hods, Xu e al. ca ied ou an empi ical compa ison o ine- uning a
RoBERTa model and 11 PEFT me hods. Ano he PEFT me hod ha ecei ed a su ey o i s own is
LoRA (Low-Rank Adap a ion) [213].
Inc emen al lea ning: To make su e ha LLMs keep up wi h an e ol ing knowledge base, i is
o en no enough o ain hem once, bu upda e hem o e ime. Jo ano ic e al. [
137
] conside ed
di e en s a egies o an inc emen al lea ning o LLMs. These include con inual lea ning (CL),
me a-lea ning, pa ame e -e icien lea ning, and mix u e-o -expe s lea ning.
Shi e al. [
272
] conduc ed a comp ehensi e su ey on CL. He e, app oaches a e di ided in
wo ca ego ies: e ical and ho izon al con inui y. Ve ical con inui y add esses app oaches ha
specialize capabili ies om a gene al se o knowledge. Ho izon al con inui y desc ibes app oaches
ha adap capabili ies ac oss ime and domains. In addi ion o ou lining CL app oaches, hey
included backg ound in o ma ion on CL, aining objec i es, as well as an o e iew o benchma ks.
Wu e al. [
341
] showed ha CL can be used o upda e se e al dimensions: ac s, domains, language,
asks, skills, alues, p e e ences. Yang e al
. [374]
ook p e- ained, ine- uned, and ision-language
models in accoun and CL me hods a e spli in o o line and online me hods. In addi ion o in e nal
me hods o CL, such as he upda ing o pa ame e s, Zheng e al. [
419
] included ex e nal app oaches
in hei su ey. Ex e nal knowledge can ei he be inco po a ed by e ie ing in o ma ion om
websi es (e.g., Wikipedia), o he use o ools o allow LLMs o ca y ou addi ional asks.
Unlea ning: Lea ning can help LLMs a ain aluable capabili ies bu no all he in o ma ion
migh be use ul o lea n. Among o he s, LLMs migh lea n biases o access p i a e in o ma ion
o indi iduals in he aining da a, which should no be eplica ed. Unlea ning app oaches a e
p oposed o help LLMs o ge abou undesi ed in o ma ion. The su ey by Blanco-Jus icia e al
. [16]
p esen ed di e en ypes o unlea ning app oaches wi h ega d o global weigh modi ica ion, local
weigh , a chi ec u e modi ica ion, and inpu o ou pu modi ica ion. They also showed da ase s,
models, and me ics used o e alua ion. Xu
[360]
conside ed unlea ning adi ional ML models
and LLMs, while Qu [251] su eyed unlea ning app oaches o ede a ed lea ning.
4.5 Alignmen
Via p e- aining and ine- uning, LLMs a e capable o lea ning om da a and gene a ing sensible
esponses o a a ie y o asks. Howe e , such esponses can be ac ually inco ec o ha m ul
due o undesi ed biases in he aining da a [
84
,
326
]. To comba his, alignmen app oaches a e
p oposed no only o align LLM esponses wi h human alues bu also es ic hei misuse in
sensi i e o po en ially ha m ul con ex s.
Wang e al. [
326
] ocused on alignmen echniques, such as ein o cemen lea ning om human
eedback. They su eyed di e en s ages o he ein o cemen lea ning p ocess and included
equa ions o explain he espec i e echniques. Shen e al. [
271
] di ided alignmen app oaches in o
16 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
echniques o ex ending con ex leng h. Howe e , Zeng e al
. [390]
unde sco ed ha con ex leng h
is only one o he h ee con lic ing goals, he o he wo being accu acy and pe o mance.
5.6 In e ac ing wi h Use s
LLMs a e inc easingly deployed in applica ions ha equi e di ec use in e ac ion. This in e ac ion
equi es abili ies o engage in dynamic con e sa ion, unde s and he in en ions om he use , and
gene a e help ul esponses [
309
,
380
]. Zaib e al
. [387]
pe o med a gene al su ey on dialogue
sys ems wi h LLMs, explo ing how hey can be le e aged o con e sa ional agen s. Simila ly, Dam
e al. [45] analyzed LLM-based cha bo s and hei impac on di e se ields. Gao e al. [78] de ised
ou s ages o he in e ac ion be ween humans and LLMs. They consis o : planning, acili a ing,
i e a ing, and es ing. Focusing on he p og ession o hese sys ems, Wang e al
. [309]
p o ided a
deepe assessmen on he e olu ion and ends o LLM-based dialogue sys ems. One key aspec
in his e olu ion, mul i- u n dialogues, was su eyed by Yi e al
. [380]
. Howe e , Dong e al
. [57]
s a ed ha as hese models become mo e in eg a ed in o use - acing applica ions, ensu ing hei
sa e y and obus ness becomes essen ial.
Beyond he ocus on dialogue, LLMs a e used o unde s and use needs and cha ac e is ics. This
a ea o esea ch is known as use modeling. Tan and Jiang
[292]
desc ibed how LLMs a e used o
model and unde s and use -gene a ed con en . Jin e al
. [135]
su eyed how LLMs can in e he use
backg ound based on cues in he p omp s and ailo hei esponses acco dingly. Ano he a enue
o in e ac ion be ween LLMs and use s is ecommenda ion sys ems. In his con ex , he su ey by
Li e al
. [175]
p o ided backg ound de ails on ML and DL-based ecommenda ion, compa ing hem
o LLM-based app oaches. Lin e al. [182] and Va s e al. [300] p esen ed how and wha pa s o a
ecommenda ion sys em can be suppo ed by LLMs. Wu e al
. [339]
ca ego ized LLM app oaches o
ecommenda ion in wo pa adigms: disc imina i e (gene a ing embeddings o use s and i ems) and
gene a i e. O he han compu ing sco es o i ems, gene a i e ecommenda ion di ec ly gene a es
ecommenda ions. This can be achie ed by ep esen ing use and i em IDs ia okens. Gene a i e
ecommenda ion was examined in mo e de ail by wo mo e su eys [169,175].
O he su eys ha e ocused on p ac ical aspec s o building hese LLM-based ecommende
sys ems. Fo ins ance, Liu e al
. [191]
examined he aining s a egies o LLMs in ecommenda ion
asks, desc ibing lea ning objec i es, da a ypes, and da ase s used in each publica ion. Las ly, Chen
[30] su eyed how o gene a e explana ions o ecommenda ions and accompanying challenges.
5.7 Sel -Imp o emen
Sel -Imp o emen encompasses he capabili y o LLMs o lea n om eedback o au onomously
enhance hei esul s. Two su eys o e a b oad iew: Pan e al
. [231]
classi ied sel -co ec ion
s a egies acco ding o when he co ec ion occu s ( aining, gene a ion, pos -hoc). Tao e al
. [294]
used he concep o “sel -e olu ion” and b oke i down in o a ou -phase i e a i e cycle: expe ience
acquisi ion, expe ience e inemen , upda ing, and e alua ion.
O he su eys go in o a deepe analysis o he sel -imp o emen p ocess. Kamoi e al
. [140]
claimed sel -co ec ion esul s a e being o e s a ed due o un ai e alua ion. They concluded ha
eliable eedback is o en he bo leneck, indica ing ha sel -co ec ion wi hou ex e nal ools
gene ally ails excep o sui able asks. The un eliabili y o sel - eedback is u he discussed by
Liang e al
. [179]
, who connec ed he success o sel -imp o emen o in e nal consis ency. They
concluded ha since LLMs a e ained on mos ly co ec da a, imp o ing he consis ency o hei
ou pu s ends o inc ease he p obabili y o a co ec ou pu mo e han an inco ec one. Las ly, Wu
e al
. [342]
conside ed how e olu iona y algo i hms can be used o enhance LLMs by suppo ing
hem wi h sea ch capabili ies, while LLMs can be used o enhance e olu iona y algo i hms by
guiding he sea ch p ocess wi h domain knowledge.
La ge Language Models: A Su ey o Su eys 17
5.8 Tool U iliza ion
LLMs a e capable o using ools wi h he goal o in e ac ing and le e aging ex e nal p og ams, such
as ex e nal so wa e and APIs, o o e come limi a ions and pe o m addi ional unc ionali ies. This
capabili y allows he LLMs o sol e mo e complex p oblems and in e ac wi h he en i onmen .
Wang e al
. [327]
pe o med a gene al su ey and p oposed a axonomy o ools based on hei
unc ionali y. Qu e al
. [249]
su eyed ool u iliza ion and p oposed a ou -s age wo k low o
ool lea ning. O he au ho s su ey speci ic applica ions in his ield. Fo example, Shi e al. [273]
examined he use o ools a e he con en is gene a ed by ocusing on Tex - o-SQL asks, and
Mialon e al. [215] s udied he use o o he models, sea ch engines, and he web as ools.
6 Applica ions
LLMs ha e shown p omise in a ious applica ions and indus ies [
299
], anging om c i ical ields
(e.g., inance, heal h, law) [
35
], o niche opics such as i ness o clima e modeling [
142
]. This sec ion
ou lines he main applica ion domains in which LLMs ha e been used, and hei espec i e su eys.
6.1 Medical and Heal h
Medical and heal h applica ions a e he mos popula domain o LLM su eys we encoun e ed,
wi h a o al o 32 su eys ca ied ou up o Sep embe ’24.
Xiao e al
. [347]
and Zhou e al
. [422]
c ea ed su eys con aining in o ma ion abou he aining,
da a and applica ions o LLMs in he medical domain, as well as challenges and a eas o u u e
esea ch. Bo h su eys p o ided help ul o e iews o da ase s and models, wi h in o ma ion such
as he base model and da a sou ce, whe e Xiao e al
. [347]
also ook mul imodal LLMs in o accoun .
In o al, Xiao e al
. [347]
conside ed six applica ions: medical diagnosis, clinical epo gene a ion,
medical educa ion, men al heal h se ices, medical language ansla ion, and su gical assis ance.
The se o applica ions s udied by Zhou e al
. [422]
shows some o e lap; howe e , he ields o
medical obo ics, clinical coding, medical inqui y, and esponse a e no el.
Simila ly, Wang e al. [
306
] conside ed ision and s anda d LLMs o p e- ained models and
ine- uning o downs eam asks. Luo e al
. [206]
ocused hei su ey on p e- ained LLMs o
NLP asks. Thei o e iew included English and Chinese LLMs used o a ious asks, such as
ques ion-answe ing, machine ansla ion, sen imen analysis, and named en i y ecogni ion. Fo
each ask, hey p o ided de ails on da ase s and me ics used.
He e al. [
102
] ansi ioned om PLMs o LLMs. This included de ails on aining and da ase s.
Simila ly, Wang e al. [311] co e ed he da a acquisi ion p ocess and di e en aining pa adigms
o adap gene al LLMs o he medical domain. Thei su ey also included conce ns abou ai ness,
accoun abili y, anspa ency and e hics. Pa k e al. [
233
] conside ed e hical implica ions in hei
e iew, as well as legal and socioeconomic conce ns. In addi ion o a comp ehensi e o e iew, Liu
e al. [
190
] pu emphasis on us wo hiness and sa e y o LLMs, which includes a discussion o
hei ai ness, accoun abili y, p i acy, and obus ness. Se e al o he su eys conside ed p i acy
and e hical conce ns in he medical domain [96,224,252,420].
Huang e al. [
121
] ocused on he e alua ion o medical LLMs. This included e alua ion ap-
p oaches and me ics o di e en applica ions: depa men s and speci ic diseases, medical esea ch,
medical educa ion and public awa eness, and medical ex p ocessing. LLMs in he medical domain
ha e been e alua ed by h ee di e en e alua o s: human expe s, au oma ed me ics, and AI-d i en
assessmen s. Au oma ed me ics can be ca ego ized in ou g oups: co ec ness, comple eness,
usabili y, and consis ency. AI-d i en assessmen s a e in he mino i y. Chen e al.[
33
] also conside ed
he e alua ion o LLMs o medical asks such as image p ocessing and in o ma ion ex ac ion.
18 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
Applica ions
O he s (Sec ion 6.9)
Ag icul u e [
429
]
Ha dwa e [
210
]
Robo ics [
146
,
276
,
389
]
Blockchain [
85
,
105
]
Telecommunica ion [
423
]
Games [
76
,
111
,
289
]
Spo s [
345
]
T anspo a ion and
D i ing (Sec ion 6.8)
[
41
,
173
,
277
,
375
,
392
,
412
]
Science (Sec ion 6.7)
[
106
,
141
,
158
,
180
,
192
,
255
,
398
,
404
,
408
]
Educa ion
(Sec ion 6.6)
[
36
,
73
,
82
,
160
,
183
,
238
,
240
,
316
,
353
,
364
]
Finance (Sec-
ion 6.5)[
161
,
176
,
228
,
257
]
Law (Sec ion 6.4)[
8
,
156
,
246
,
288
,
373
]
Cybe secu i y
(Sec ion 6.3)
[
34
,
49
,
100
,
198
,
354
,
394
]
So wa e and
Code (Sec ion 6.2)
In eg a ion [
88
,
267
,
329
]
Tes ing and Repai [
117
,
310
,
399
,
425
]
Code gene a ion [
31
,
74
,
107
,
123
,
127
,
283
,
388
]
[
17
,
64
,
109
,
240
,
270
,
362
,
397
,
400
,
410
,
421
]
Medical and
Heal h (Sec ion 6.1)
[
33
,
72
,
89
,
95
,
96
,
99
,
102
,
103
,
113
,
121
,
143
,
170
,
190
,
206
,
214
,
220
,
224
,
226
,
233
,
252
,
259
,
275
,
285
,
306
,
311
,
335
,
347
,
348
,
384
,
385
,
420
,
422
]
[
35
,
142
,
299
]
Fig. 6. Taxonomy o su eys on applica ions.
While a lo o he su eys desc ibed asks based on ex s, he ield o medicine is mul imodal and
se e al ypes o da a ha e been su eyed [
99
,
306
,
347
,
385
]. Fe a a
[72]
s udied da a collec ed by
wea able senso s and he su ey by Ne ella e al
. [226]
co e ed da a ypes such as NLP, medical
imaging, s uc u ed Elec onic Heal h Reco ds (EHR), social media, biophysiological signals, and
biomolecula sequences. Pa icula ly, elec onic heal h eco ds ha e been o in e es o su eys [
170
,
335
,
348
]. Li e al
. [170]
su eyed LLMs wo king wi h Elec onic Heal h Reco ds, in pa icula wi h
ega ds o se en asks: named en i y ecogni ion, in o ma ion ex ac ion, ex summa iza ion,
ex simila i y, ex classi ica ion, dialogue sys em, diagnosis, and p edic ion. Xie e al
. [348]
only
conside ed he ask o ex summa iza ion, which has been applied o EHR and biomedical li e a u e,
medical con e sa ion, and ques ions.
Ano he se o su eys included bibliome ic analysis. Fo ins ance, Res epo e al. [
259
] analyzed
me ada a such as au ho a ilia ions, coun ies, and unding sou ce o assess di e si y. Yu e al
. [384]
La ge Language Models: A Su ey o Su eys 19
conside ed in o ma ion such as collabo a ion ne wo ks. The emaining su eys co e ed a eas ang-
ing om only conside ing Spanish language models [285] o LLMs in medical examina ions[220],
psychology [103,143], men al heal h [89,95,113,214], and c i ical ca e medicine [275].
6.2 So wa e and Code
In he so wa e enginee ing domain, we ound se e al su eys which p o ided comp ehensi e
o e iews. The ea lies su ey is by Xu and Zhu [
362
], om 2022. They su eyed da ase s, asks,
and a chi ec u es o p e- ained LLMs as well as hei aining p ocedu es.
Subsequen su eys inc eased in comp ehensi eness, wi h he su ey by Ziyin Zhang e al. [
410
]
co e ing mo e han 900 wo ks. They c ea ed bo h a axonomy o code LLMs as well as a axonomy
o mo e han 40 asks acco ding o he so wa e de elopmen s ages. The su ey by Quanjun
Zhang e al. [
400
], which also en ails mo e han 900 e e ences, p o ided ano he comp ehensi e
o e iew. In e es ing aspec s hey conside ed included an o e iew o p e- aining asks as well as
he in eg a ion o LLMs o SE ac i i ies (e.g., hei secu i y o size).
Zheng e al
. [421]
ga e in o ma ion abou o ganiza ions which de eloped he LLMs (e.g.,
Company-led, Uni e si y-led, Resea ch eams & Open-sou ce communi y-led). Also, hei su -
ey pu emphasis on he pe o mance o LLMs. One esea ch ques ion was aimed a inding
whe he code LLMs pe o m be e han gene al LLMs o SE asks. Mo eo e , hey p esen ed he
pe o mance epo ed in collec ed wo ks o se e al asks, o ind which LLM is mos sui able. Hou
e al
. [109]
p o ided aluable insigh s on he da ase s used o SE asks, including da a collec ion,
selec ion, and p ocessing s eps. She e al
. [270]
su eyed pi alls which could hinde he pe o -
mance o LLMs in p ac ice. These a e di ided in o i e ca ego ies: da a collec ion and labeling,
sys em design and lea ning, pe o mance e alua ion, deploymen , and main enance. Fo each o
hese pi alls, implica ions and solu ions a e ou lined. Simila ly, Fan e al. [
64
] lis ed open p oblems
o each s age o he so wa e de elopmen li ecycle.
O he su eys in es iga ed how LLMs ha e been p omp ed o a ious SE asks [
17
], how
LLMs can be used in an educa ional se ing o help wi h code ela ed asks (e.g., explaining e o
messages) [
240
], o suppo ailu e managemen o A i icial In elligence o IT Ope a ions [
397
].
Code gene a ion: Jiang e al
. [127]
c ea ed a comp ehensi e su ey on he gene a ion o code
om na u al language desc ip ions. Collec ed wo ks a e s uc u ed gi en a axonomy in: da a
cu a ion, ecen ad ances (e.g., aining and p omp ing), e alua ion, and applica ion (e.g., Gi Hub
Copilo ). They also p o ided an o e iew o exis ing LLMs and a pe o mance compa ison o se e al
LLMs on wo popula benchma king da ase s: HumanE al and MBPP. Zan e al. [
388
] also p o ided
a compa ison o LLMs on he HumanE al benchma k, whe e hey included a la ge quan i y o
small LLMs (smalle han 1 billion pa ame e s). Addi ionally, hey p esen ed 17 benchma ks wi h
s a is ics, such as he numbe o es s a ailable. In con as , Hong e al
. [107]
su eyed app oaches
o gene a ing SQL que ies om na u al language.
Husein e al. [
123
] su eyed he comple ion o code a he han gene a ing code om na u al
language desc ip ions. They conside ed di e en g anula i ies ( oken, line, API calls, Block le el)
and pe o mance me ics o e alua ion. O he han gene a ing code i sel , LLMs ha e been used o
gene a e p og amming exe cises [74], in as uc u e con igu a ions [283], and suppo HPC [31].
Tes ing and Repai : The su ey by Wang e al. [
310
] discussed he ield o so wa e es ing and
di e en associa ed asks. The mos commonly add essed asks include p og am epai as well
as he gene a ion o es s (e.g., uni es s, sys em es s). Fo hese, Wang e al. ex ac ed he mos
common p omp s (e.g., ze o-sho ) and he LLMs used o hese asks.
The su ey by Zhang e al
. [399]
ocused on APR and ound 127 APR pape s co e ing 18 bug
ypes ha used LLMs. Zhou e al
. [425]
conside ed bo h ulne abili y de ec ion and epai . They
20 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
in es iga ed how LLMs ha e been adap ed o hese asks and ound ha he majo i y o app oaches
pe o m ine- uning. Huang e al. [117] co e ed he use o LLMs o uzzing as a es ing ac i i y.
In eg a ion: While p e iously ou lined su eys co e ed he use o LLMs o so wa e enginee ing
ac i i ies, hey can also be ea ed as componen s o so wa e i sel . In his ega d, Webe
[329]
c ea ed a axonomy o LLM-in eg a ed so wa e sys ems, and Se geyuk e al
. [267]
s udied he use
o LLMs in In eg a ed De elopmen En i onmen s (IDEs). Go issen e al
. [88]
conside ed he use o
LLMs in Low-Code De elopmen Pla o ms.
6.3 Cybe secu i y
Fou s udies c ea ed comp ehensi e o e iews o he ield o cybe secu i y [
34
,
100
,
354
,
394
]. Has-
sanin and Mous a a
[100]
co e ed di e se cybe de ense s a egies such as ulne abili y assessmen ,
in usion de ec ion, o anonymiza ion, while o he s pu emphasis on ulne abili y assessmen s [
49
]
and h ea de ec ion [
34
]. Zhang e al
. [394]
no only ou lined de ense ac i i ies bu also indica ed
ways o use LLMs o a acks. Xu e al
. [354]
p o ided insigh s on how o cons uc LLMs o he
secu i y domain ia means o ine- uning, p omp ing, o augmen a ion wi h ex e nal ools. Las ly,
Liu
[198]
ga e an o e iew o a ailable p e- ained models o cybe secu i y, and Xu e al
. [354]
add essed he da a collec ion p ocess and a ailable da ase s.
6.4 Law
LLMs ha e been used o au oma e a ious legal asks, bu hei adop ion also aised challenges [
288
].
Anh e al
. [8]
esea ched he impac o LLMs on NLP, ocusing on legal ex p ocessing. They
explained how NLP add esses di e en challenges in he ield, such as ambigui y and sen ence
complexi y. They pe o med an empi ical analysis ha sugges s ha encode -decode models
ou pe o m encode -only a chi ec u es, ad oca ing o hei use in legal NLP asks.
Lai e al
. [156]
p o ided a gene al su ey on he applica ions o LLMs wi hin he judicial sys ems.
They included he impac on common use s as well as expe s (e.g., judges and lawye s). The
au ho s indica ed limi a ions and issues o LLMs ha can a ec judicial p ac ices. They ga e
p ac ical ecommenda ions o imp o ing he use o LLMs in he legal sys em and highligh ed
he impo ance o unde s anding he socie al impac s o hese echnologies. Simila ly, Qin and
Sun
[246]
co e ed he p ac ical applica ion o LLMs in he legal sys em, such as case e ie al
and legal analysis. They indica ed po en ial challenges such as biases, in e p e abili y issues, and
da a p i acy conce ns. This s udy emphasized he need o ine- uned models and p esen ed an
o e iew o da ase s o hei aining in di e en languages. Las ly, Yang e al
. [373]
p esen ed
a sys ema ic e iew o legal LLMs ocusing on ine- uning o ques ion-answe ing asks. They
p o ided a p ac ical iew ocusing on he implemen a ion o hese sys ems and he echniques
ha hey could use (e.g., Low-Rank Adap a ion). They used a bo om-up app oach o examine how
exis ing models can be adap ed o he legal domain.
6.5 Finance
Nie e al
. [228]
p o ided a comp ehensi e su ey on LLMs o inance. They i s ca ego ized
exis ing wo ks acco ding o applica ion a eas in he inancial domain, including, among o he s, ime
se ies o ecas ing, easoning, and sen imen analysis. Fu he in o ma ion on da ase s, benchma ks,
and challenges is p esen ed. In addi ion o p o iding an o e iew o inance applica ions, Li e al
.
[176]
de eloped a decision amewo k o help p ac i ione s selec an LLM based on hei ask. Fo
his, hey also p o ided a compa ison wi h es ima ed cos s o di e en LLM op ions (e.g., ze o-sho ,
ine- uning, aining om sc a ch). Lee e al
. [161]
pu emphasis on p esen ing benchma k asks and
da ase s. Mo eo e , hey showed a imeline o LLMs and inancial LLMs. Ren e al
. [257]
add essed
La ge Language Models: A Su ey o Su eys 21
he use o LLMs in an e-comme ce se ing. In his con ex , LLMs ha e been used o asks such as
p oduc ecommenda ions, ques ion answe ing and analysis o cus ome eedback.
6.6 Educa ion
Wang e al. [
316
] c ea ed a comp ehensi e su ey on how LLMs can assis eache s, s uden s,
and di e en ools ha a e a ailable. Addi ionally, hey p o ided an o e iew o da ase s and
benchma ks, as well as discussed isks and challenges o LLMs in educa ion. Pes e e al
. [238]
add essed he use o LLMs o imme si e lea ning ac i i ies. The su ey by Xu e al. [
353
] p o ided
mo e backg ound in o ma ion on educa ion, as well as how o in eg a e LLMs in he p ocess, while
Ga cía-Méndez e al
. [82]
conside ed LLMs used o di e en educa ion ac i i ies. This ocus on
in eg a ion also ex ends o speci ic disciplines, wi h dedica ed su eys explo ing he use o LLMs
in subjec s such as compu e science [240] o enginee ing [73].
Yan e al
. [364]
co e ed a o al o 53 educa ional asks om nine ca ego ies (e.g., g ading, con en
gene a ion) and pu emphasis on p ac ical and e hical challenges. In a simila ashion, Chhina
e al
. [36]
looked a bo h he challenges and bene i s o LLMs in educa ion. Lee e al
. [160]
ocused
hei su ey on di e en ypes o biases when using LLMs in an educa ional se ing. Biases we e
in es iga ed a di e en s ages o he LLM li ecycle (e.g., da a collec ion, aining, and deploymen ).
Lin e al. [183] lis ed a ailable open-sou ce LLMs o use in educa ion ac i i ies.
6.7 Science
Ho e al. [
106
] p o ided an o e iew o scien i ic LLMs applied o ex , and p esen ed di e en
asks, da ase s, and exis ing models. In addi ion o scien i ic LLMs o ex , Zhang e al. [
404
]
su eyed mo e han 260 LLMs, no only aking di e en scien i ic ields bu also di e en modali ies
in o accoun . Complemen ing his b oad o e iew, o he su eys ocus on LLM applica ions in
speci ic ields such as chemis y [
180
,
255
,
398
], biology [
398
], and ma hema ics [
192
], as well as
o specialized sub-domains like single-cell biology [158] and compu a ional neu oscience [141].
A ai o scien i ic ex s is he p esence o use o ci a ions, o gi e c edi o ele an sou ces.
He e, Zhang e al. [
408
] c ea ed a su ey o show he ela ion be ween LLMs and ci a ions. Thei
su ey p o ided an o e iew o ou di e en ci a ion asks LLMs can be applied o: ci a ion classi-
ica ion, ci a ion-based summa iza ion, ci a ion sen ence gene a ion, and ci a ion ecommenda ion.
Addi ionally, hey discussed how ci a ions can be inco po a ed in he aining o LLMs.
6.8 T anspo a ion and D i ing
In he ealm o In elligen T anspo a ion Sys ems (ITS), LLMs ha e been used o ad ance ans-
po a ion in elligence and a ic managemen . The su eys by Shoaib e al
. [277]
co e ed asks
such as a ic p edic ion and anspo a ion managemen , while Zhang e al
. [392]
conside ed
a ic managemen , anspo a ion sa e y, and au onomous d i ing. Mo eo e , hey p o ided a lis
o da ase s o he ITS domain. Zhang e al
. [412]
ocused on a el beha io p edic ion as a ime
se ies o ecas ing p oblem and p o ided an o e iew o LLM-based app oaches.
Au onomous d i ing was co e ed by h ee dedica ed su eys [
41
,
173
,
375
]. Cui e al. [
41
]
add essed he use o LLMs o au onomous d i ing om a mul imodal pe spec i e ( ision and lan-
guage). They p o ided a holis ic o e iew, conside ing he use o mul imodal LLMs o au onomous
d i ing, anspo a ion, and maps. Fu he mo e, hey p esen ed da ase s o au onomous d i ing
and a ic scene unde s anding, and ex ac ed in o ma ion om exis ing app oaches, such as he
LLMs used. In con as , Yang e al
. [375]
p o ided a mo e ine-g ained iew on asks and me ics
used o e alua ion. They dis inguish ou ca ego ies, based on he espec i e asks: planning, pe -
cep ion, ques ion answe ing, and gene a ion. Li e al. [
173
] co e ed he use o LLMs in au onomous
d i ing ei he as pa o he pipeline, o suppo exis ing sys ems, o as end- o-end sys ems.
22 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
6.9 O he s
Spo s: Xia e al. [
345
] in es iga ed da ase s and applica ions o LLMs in a spo s se ing. He e,
LLMs can be applied o di e en inpu ypes ( ex , ideo, audio) and ha e add essed a di e se ange
o asks (e.g., ha e speech de ec ion, an engagemen , game summa iza ion).
Games: Swee se
[289]
ca ied ou a scoping e iew on 76 pape s on LLMs o ideo games, o
p o ide an o e iew and suppo u u e esea ch. In he gameplaying con ex , LLMs ha e been
used as pa s o he game (e.g., agen s, dialogue gene a ion) o pa o he de elopmen and analysis
p ocess (e.g., con en gene a ion, analysis o e iews).
Gallo a e al. [
76
] add essed he di e en oles o LLMs in games in hei su ey. In o al, hey
iden i ied nine oles ha LLMs ha e aken: Playe , NPC (non-playe cha ac e ), playe assis an ,
commen a o , analys , game mas e , game mechanic, au oma ed designe , and design assis an .
Addi ionally, hey p esen ed a oadmap o u u e applica ions o LLMs o games, as well as
limi a ions and e hical implica ions o hei use.
Hu e al. [
111
] ocused hei su ey on LLM-based game agen s, o which hey ound 62 ap-
p oaches. These we e ca ego ized based on game ype ( ex , ideo) and gen e (e.g., ad en u e,
coope a ion, simula ion). Mo eo e , agen s we e discussed om six pe spec i es: pe cep ion, mem-
o y, hinking, ole-playing, ac ion, and lea ning.
Telecommunica ion: Zhou e al
. [423]
p esen ed a comp ehensi e o e iew o LLMs in he
ield o elecommunica ions. In pa icula , LLM ac i i ies (gene a ion, classi ica ion, op imiza ion,
p edic ion) we e mapped o elecommunica ion applica ions. Such applica ions include ne wo k
issue oubleshoo ing, ne wo k de ec de ec ion, and a ic load le el p edic ion.
Blockchain: Ge en e al. [
85
] su eyed how blockchain echniques can suppo he secu i y and
sa e y o LLMs, o example by e i ying aining da a au hen ici y and p i acy p ese a ion. On
he o he hand, He e al. [
105
] su eyed LLMs o suppo ing blockchain secu i y. They showed
ha LLMs can suppo he blockchain communi y by de ec ing ulne abili ies in he sou ce code
o sma con ac s, de ec ing i egula ansac ion pa e ns o gene a ing o sma con ac s.
Robo ics: Kim e al. [
146
] explo ed he use o LLMs in obo ics. Thei ocus is on ecen LLMs (a e
GPT-3.5) and ex -based LLMs, while s ill allowing he inclusion o ele an mul imodal app oaches.
They dis inguish ou main ca ego ies o LLM use: communica ion, pe cep ion, planning, and
con ol. Addi ionally, hey p o ided guidelines o p omp ing LLMs o ou obo ic asks: in e ac i e
g ounding, scene-g aph gene a ion, ew-sho planning, ewa d unc ion gene a ion.
Simila ly, he su ey by Zeng e al. [
389
] p esen ed LLM applica ions in obo ics wi h ega ds
o con ol, pe cep ion, decision-making and pa h planning. Di e en om Kim e al. [
146
], hey
pu mo e emphasis on LLMs and ans o me a chi ec u es, as well as challenges. Las ly, Shi e
al. [
276
] add essed he use o LLMs in socially assis i e obo s (SARs) wi h a sho su ey. He ein,
challenges and oppo uni ies o using LLMs in SAR we e discussed.
Ha dwa e: Simila o hei use in de ec ing so wa e ulne abili ies (Sec ion 6.2), LLMs can suppo
he secu i y o ha dwa e componen s. Makhzan and Kamali [210] compa ed 10 such s udies.
Ag icul u e: Zhu e al. [
429
] e iewed how LLMs and ision models can be applied in ag icul u e.
7 Mul imodali y
Gene ally, LLMs a e applied o ex ual da a and excel a language-based asks. Howe e , hei
applica ion has been ex ended beyond ex s o u he modali ies, o which we discuss ele an
su eys. Wu e al
. [337]
ou lined he his o y o mul imodal app oaches, om single modali y o
ecen la ge-scale mul imodal sys ems. Yin e al. [
382
] p esen ed in o ma ion on he a chi ec u e
o Mul imodal La ge Language Models (MLLMs) as well as hei aining and e alua ion. Song e
La ge Language Models: A Su ey o Su eys 23
Mul imodali y
O he s (Sec ion 7.3)
3D [
208
]
Geospa ial [
297
,
426
]
S uc u ed [
69
,
202
,
403
]
Audio [
336
]
Time-se ies [
130
,
284
,
378
,
402
]
G aph (Sec ion 7.2)[
3
,
131
,
165
,
174
,
212
,
258
,
268
]
Visual (Sec ion 7.1)
[
1
,
19
,
24
,
60
,
67
,
86
,
90
,
97
,
99
,
162
,
187
,
189
,
200
,
209
,
219
,
227
,
293
,
301
,
349
,
395
,
406
,
424
,
429
]
[
10
,
104
,
247
,
281
,
337
,
367
,
382
]
Fig. 7. Taxonomy o su eys on mul imodali y.
al. [
281
] desc ibed how di e en modali ies can be aligned. O he su eys s udied he gene a ion
and edi ing ac oss modali ies [104], aining da a [10,247], o analysis o sen imen s [367].
7.1 Visual
The mos equen applica ion o mul imodal language models we ound is o isual asks, wi h
mul iple su eys p esen ing comp ehensi e o e iews [
19
,
395
]. Fo example, Zhang e al
. [395]
ga e backg ound in o ma ion on he isual pa adigm as well as a summa y o cha ac e is ics such
as downs eam asks o Vision Language Models (VLMs) and hei a chi ec u e. Among o he s,
hey ou lined da ase s and p e- aining me hods. The e a e se e al o he comp ehensi e su eys,
which pu di e en oci, such as da ase s [97], models [86], o de ails on egula LLMs [24].
The su eys by Du e al. [
60
] and Long e al. [
200
] ocused on p e- ained ision-language
models. Fi s , da a is ans o med in o desi ed ep esen a ions. A e wa ds, an a chi ec u e is
designed o model he in e ac ion be ween ex and image. Fu he su eys ook p omp ing [
90
],
ine- uning [
349
], and he de ec ion o ou -o -dis ibu ion samples and anomalies [
219
] in o accoun .
These ad ancemen s enabled he applica ion o VLMs in di e se domains wi h su eys desc ibing
hei applica ions in ag icul u e [
429
], medicine [
99
], au onomous na iga ion [
209
,
406
], documen
unde s anding [1], and ideo analysis [227,293,424]
While VLMs o e ad an ages in se e al asks, hey can be ulne able o a acks, which a ec s
hei usabili y in eal-wo ld applica ions [
67
]. He e, Liu e al
. [187]
su eyed ou ypes o a ack
me hods (ad e sa ial a acks, jailb eak, p omp injec ion, and da a poisoning) as well as po en ial
de ense me hods. Fan e al
. [67]
conside ed di e en a ack scena ios based on he ype o model
access (i.e., whi e-box, g ay-box, black-box). E hical AI has been u he aken in o accoun by
Va sa e al. [
301
] who su eyed bias, obus ness, and in e p e abili y o VLMs. Lee e al.[
162
] solely
ocused on biases and hei mi iga ion. Ano he sho coming o VLMs a e hallucina ions, which was
su eyed by Liu e al. [
189
]. They collec ed me hods and benchma ks o e alua ing hallucina ions
and mi iga e hem. In o al, he e a e i e a eas ha ha e been add essed o mi iga ion: da a, ision
encode , connec ion module, LLM, pos -p ocessing.
7.2 G aph
Jin e al.[
131
] c ea ed a comp ehensi e su ey on di e en ways LLMs can in e ac wi h he
s uc u ed in o ma ion p o ided in g aphs. He eby, he e a e h ee ypes o g aphs o conside : pu e
24 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
g aphs, ex -a ibu ed g aphs (e.g., nodes ha e ex s), ex -pai ed g aphs (a comple e g aph is pai ed
wi h ex ). Addi ionally, LLMs can be used in h ee di e en manne s o g aph asks: as p edic o s,
encode s (e.g., encoding node ex s as ec o s), o o aligning ex encoding wi h G aph Neu al
Ne wo ks (GNNs). Thei axonomy conside ed he in e sec ion o wo dimensions: he applica ion
scena io (g aph ype) and he LLM echnique. Mo eo e , hey c ea ed an o e iew o da ase s om
di e en domains (e.g., academia, e-comme ce, books, Wikipedia) and g aph p oblems s udied (e.g.,
sho es pa h, neighbo de ec ion).
O he axonomies a e included in he wo ks by Ren e al
. [258]
and Li e al
. [174]
. The axonomy
by Ren e al. [
258
] conside ed ou aspec s: GNNs as P e ix, LLMs as P e ix, LLMs-G aphs In eg a ion,
LLMs-Only. Fo GNN as p e ix, da a is i s p ocessed by GNNs and hen ed in o LLMs. LLMs as
P e ix does he opposi e, p ocessing da a wi h LLMs o imp o e GNNs. LLMs-G aphs In eg a ion
en ails me hods ha imp o e he abili y o LLMs o handle g aph da a, while LLMs-only desc ibes
wo k ha applies LLMs o g aph asks ia p omp ing. Li e al. [
174
] de ised a axonomy wi h h ee
ca ego ies: enhance (enhancing quali y o node embeddings), p edic o (using LLMs o p edic ion
in g aph- asks), and alignmen (aligning embedding spaces o LLMs and G aph Neu al Ne wo ks).
Mao e al. [
212
] s udied he in eg a ion o LLMs o G aph Rep esen a ion Lea ning (GRL). They
ou lined exis ing app oaches o using LLMs o imp o e GRL asks. App oaches a e in es iga ed
wi h ega ds o ou componen s: knowledge ex ac ion, knowledge o ganiza ion, in eg a ion
s a egies, and aining s a egies. Shang and Huang [
268
] su eyed he use o LLMs o g aph
analy ics asks. Thei su ey conside ed h ee aspec s, he p ocessing o g aph que ies wi h LLMs,
in e ence and lea ning o e g aphs, and applica ions. Ample isual examples a e p o ided o he
asks (g aph unde s anding, g aph lea ning, g aph- o med easoning) and p omp s.
While he ou lined su eys add essed g aphs in gene al, we ound wo su eys ocused on
knowledge g aphs. Such g aphs a e used o model and s uc u e knowledge bases. On one hand,
Ag awal e al. [
3
] su eyed how knowledge g aphs ha e been used o comba hallucina ions in
LLMs. Fo his, hey de ined h ee g oups: in e ence (e.g., RAG), aining (e.g., p e- aining, ine-
uning), and alida ion (e.g., ac -checking LLMs). On he o he hand, Li and Xu [
165
] add essed
bo h, how LLMs can enhance knowledge g aphs and how knowledge g aphs can enhance LLMs.
7.3 O he s
Beyond he ex ensi ely esea ched domains o ision and g aphs, MLLMs a e expanding o a
b oade ange o o ma s. The ollowing su eys co e hese eme ging modali ies, each p esen ing
unique challenges and oppo uni ies when in eg a ed wi h LLMs.
Time-se ies: Jiang e al. [
130
] c ea ed a su ey on ime-se ies analysis wi h LLMs. LLMs can model
ime-se ies ia que ying, okeniza ion, p omp ing, ine- uning, o he in eg a ion o LLM ou pu
in exis ing models. O e all, his su ey includes 21 s udies, o e a ious applica ions (e.g., CV,
mobili y, heal hca e, inance), o which he modeling app oaches, asks, and unde lying LLMs a e
ex ac ed. The su ey by Ye e al
. [378]
con ains ime-se ies s udies o simila applica ion domains,
howe e , hei analysis ocused on h ee dimensions: e ec i eness, e iciency, and explainabili y.
Su e al. [284] included a discussion o anomaly de ec ion o ime se ies.
Beyond his scope, Zhang e al
. [402]
conside ed isual ep esen a ions o ime se ies as well as
LLM-based ools o suppo he p ocessing o ime-se ies, o example by c ea ing code.
Audio: By con e ing audio in o disc e e codes, hey can be p ocessed by language models. Wu e
al. [336] p o ided an o e iew o six neu al models and 11 language models o p ocessing audio.
Fo each language model, hey p esen ed he add essed asks as well as inpu and ou pu o ma .
S uc u ed: Tables ep esen da a in a s uc u ed, wo-dimensional manne and can be p ocessed
wi h LLMs. Fang e al. [
69
] e iewed echniques, me ics, da ase s, and models o ou echniques
o applying LLMs o ables: se ializa ion, able manipula ion, p omp enginee ing, and end- o-end
La ge Language Models: A Su ey o Su eys 25
sys ems. Emphasis is also pu on he use o LLMs o gene a e abula da a. In addi ion o discussing
aining app oaches o LLMs and Visual language models, Lu e al. [
202
] desc ibed p omp ing
echniques and he use o agen s.
Zhang e al. [
403
] ocused hei su ey on echniques o imp o e he pe o mance o LLMs o
di e en able p ocessing asks (QA, ac e i ica ion, able o ex , ex o SQL). Fo i e popula
imp o emen echniques, hey showed a pe o mance compa ison o e ou da ase s.
Geospa ial: Zhou e al. [
426
] su eyed LLMs wi h geo-pe cep i e capabili ies o handle mul iple
modali ies o geospa ial da a. They ocused on a speci ic amily o language models, Vision-language
geo- ounda ion models (VLGFM). These VLGFM inco po a e di e se da a modali ies (sa elli e
images, geo- agged ex , emo e sensing images) o add ess a wide ange o geospa ial asks (e.g,
image cap ioning, isual g ounding) The su ey includes an o e iew o asks, da ase s and me ics
o e alua ion as well as a desc ip ion o model a chi ec u es. Tucke [
297
] e iewed LLMs o
Geospa ial Loca ion Embeddings (GLE) o ep esen and exp ess space.
3D: LLMs ha e seen use in spa ial asks, which equi e he conside a ion o h ee dimensions. In
pa icula , Ma e al. [
208
] in es iga ed how LLMs can unde s and and in e ac wi h 3D da a. Thei
su ey p o ided in o ma ion on di e en 3D da a ep esen a ions (e.g., poin cloud, g id, mesh),
asks (cap ioning, g ounding, con e sa ion (ques ion answe ing), agen , gene a ion), and da ase s.
Addi ionally, he LLMs and 3D componen s o 37 publica ions a e ex ac ed and desc ibed.
8 Risks and Mi iga ion
While p io sec ions ou lined he bene i s in a ious domains, LLMs can be suscep ible o bias and
sa e y issues o sha e p i a e in o ma ion [
42
,
197
]. These conce ns p opaga e o a ious ields [
47
],
such as heal hca e [
96
] o educa ion [
364
], and a e majo challenges ha need o be o e come o
achie e us [
71
,
119
,
184
,
197
,
301
] and anspa ency (e.g., by explaining esponses) [
20
,
203
,
253
,
413]. Resea che s showed in e es in he di e en ypes o isks and hei mi iga ion [266].
In he ollowing, we discuss he main conce ns poin ed ou and co e ed by exis ing su eys:
ai ness, hallucina ions, secu i y, and p i acy [40,56,81,119,144,154].
8.1 Fai ness and Bias
LLMs can p opaga e social biases om he aining da a, causing ai ness and bias issues, which
has been co e ed by se e al s udies [
39
,
75
,
172
]. The su ey by Gallegos e al. [
75
] is he mos
comp ehensi e wi h h ee axonomies, one o me ics, da ase s, and bias mi iga ion me hods
each. Chu e al
. [39]
p esen ed oolki s in addi ion o da ase s, while Li e al
. [172]
ook model size
in o accoun . They dis inguished ai ness s udies based on LLM size, as smalle models allow o
ine- uning, while la ge models a e p omp ed ins ead.
Su eys ha e also ocused on a speci ic aspec , such as me ics [
50
] o he debiasing o LLMs [
184
].
Ano he se o wo ks su eyed speci ic ields o biases, such as educa ion [
160
], e-comme ce [
257
],
in o ma ion e ie al [44], ision-language models [162], o ecommende sys ems [263].
Las ly, Wang e al. [
314
] collec ed human pe spec i es on LLM bias om se e al s udies and
summa ized hei pe spec i es. Among o he hings, people pe cei ed bias mo e when hey ailed
o ecei e desi ed esponses.
8.2 Hallucina ion
A imes, he ou pu s gene a ed by LLMs a e inconsis en wi h he ac ual answe o he use
inpu i sel , which is called “hallucina ion”. The e a e h ee comp ehensi e su eys add essing his
issue [
116
,
377
,
405
]. They con ain de ails on causes, benchma ks, and mi iga ion app oaches. We
ha e also ound wo su eys add essing hallucina ions o ision-language models [11,189].
32 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
[64]
Angela Fan, Beliz Gokkaya, Ma k Ha man, Mi ya Lyuba skiy, Shubho Sengup a, Shin Yoo, and Jie M. Zhang. 2023.
La ge Language Models o So wa e Enginee ing: Su ey and Open P oblems. In IEEE/ACM In e na ional Con e ence
on So wa e Enginee ing: Fu u e o So wa e Enginee ing, ICSE-FoSE 2023, Melbou ne, Aus alia, May 14-20, 2023. IEEE,
Melbou ne, Aus alia, 31–53. h ps://doi.o g/10.1109/ICSE-FOSE59343.2023.00008
[65]
Lizhou Fan, Lingyao Li, Zihui Ma, Sanggyu Lee, Huizi Yu, and Libby Hemphill. 2023. A Bibliome ic Re iew o La ge
Language Models Resea ch om 2017 o 2023. h ps://doi.o g/10.48550/ARXIV.2304.02020 a Xi :2304.02020
[66]
Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Ta -Seng Chua, and Qing Li. 2024. A
Su ey on RAG Mee ing Llms: Towa ds Re ie al-Augmen ed La ge Language Models. In P oceedings o he 30 h
ACM SIGKDD Con e ence on Knowledge Disco e y and Da a Mining, KDD 2024, Ba celona, Spain, Augus 25-29, 2024,
Rica do Baeza-Ya es and F ancesco Bonchi (Eds.). ACM, 6491–6501. h ps://doi.o g/10.1145/3637528.3671470
[67]
Yihe Fan, Yuxin Cao, Ziyu Zhao, Ziyao Liu, and Shao eng Li. 2024. Unb idled Ica us: A Su ey o he Po en ial
Pe ils o Image Inpu s in Mul imodal La ge Language Model Secu i y. h ps://doi.o g/10.48550/ARXIV.2404.05264
a Xi :2404.05264
[68]
Wenyi Fang, Hao Zhang, Ziyu Gong, Longbin Zeng, Xuhui Lu, Biao Liu, Xiaoyu Wu, Yang Zheng, Zheng Hu, and
Xun Zhang. 2023. A Su ey o Me ics o Enhance T aining Dependabili y in La ge Language Models. In 34 h IEEE
In e na ional Symposium on So wa e Reliabili y Enginee ing, ISSRE 2023 - Wo kshops, Flo ence, I aly, Oc obe 9-12,
2023. IEEE, Flo ence, I aly, 180–185. h ps://doi.o g/10.1109/ISSREW60843.2023.00071
[69]
Xi Fang, Weijie Xu, Fiona An ing Tan, Jiani Zhang, Ziqing Hu, Yanjun Qi, Sco Nickleach, Diego Socolinsky,
S ini asan H. Sengamedu, and Ch is os Falou sos. 2024. La ge Language Models(Llms) on Tabula Da a: P edic ion,
Gene a ion, and Unde s anding - A Su ey. h ps://doi.o g/10.48550/ARXIV.2402.17944 a Xi :2402.17944
[70]
Zhangyin Feng, Wei ao Ma, Weijiang Yu, Lei Huang, Hao ian Wang, Qianglong Chen, Weihua Peng, Xiaocheng
Feng, Bing Qin, and Ting Liu. 2023. T ends in In eg a ion o Knowledge and La ge Language Models: A Su ey and
Taxonomy o Me hods, Benchma ks, and Applica ions. h ps://doi.o g/10.48550/ARXIV.2311.05876 a Xi :2311.05876
[71]
Md Me ahul Fe daus, Mahdi Abdelgue i, Elias Ioup, Kendall N. Niles, Ken Pa hak, and S e en Sloan. 2024. Towa ds
T us wo hy AI: A Re iew o E hical and Robus La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2407.13934
a Xi :2407.13934
[72]
Emilio Fe a a. 2024. La ge Language Models o Wea able Senso -Based Human Ac i i y Recogni ion, Heal h
Moni o ing, and Beha io al Modeling: A Su ey o Ea ly T ends, Da ase s, and Challenges. Senso s 24, 15 (2024),
5045. h ps://doi.o g/10.3390/S24155045
[73]
S e ano Filippi and Ba ba a Mo yl. 2024. La ge Language Models (Llms) in Enginee ing Educa ion: A Sys ema ic
Re iew and Sugges ions o P ac ical Adop ion. In . 15, 6 (2024), 345. h ps://doi.o g/10.3390/INFO15060345
[74]
Edua d F ank o d, Ingo Höhn, Clemens Saue wein, and Ru h B eu. 2024. A Su ey S udy on he S a e o he A
o P og amming Exe cise Gene a ion Using La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2405.20183
a Xi :2405.20183
[75]
Isabel O. Gallegos, Ryan A. Rossi, Joe Ba ow, Md. Meh ab Tanjim, Sungchul Kim, F anck De noncou , Tong
Yu, Ruiyi Zhang, and Nes een K. Ahmed. 2023. Bias and Fai ness in La ge Language Models: A Su ey. h ps:
//doi.o g/10.48550/ARXIV.2309.00770 a Xi :2309.00770
[76]
Robe o Gallo a, G aham Todd, Ma in Zammi , Sam Ea le, An onios Liapis, Julian Togelius, and Geo gios N.
Yannakakis. 2024. La ge Language Models and Games: A Su ey and Roadmap. h ps://doi.o g/10.48550/ARXIV.
2402.18659 a Xi :2402.18659
[77]
Chen Gao, Xiaochong Lan, Nian Li, Yuan Yuan, Jing ao Ding, Zhilun Zhou, Fengli Xu, and Yong Li. 2023. La ge
Language Models Empowe ed Agen -Based Modeling and Simula ion: A Su ey and Pe spec i es. h ps://doi.o g/10.
48550/ARXIV.2312.11970 a Xi :2312.11970
[78]
Jie Gao, Sim e A aya Geb eegziabhe , Kenny Tsu Wei Choo, Toby Jia-Jun Li, Simon Tangi Pe aul , and Thomas W.
Malone. 2024. A Taxonomy o Human-LLM In e ac ion Modes: An Ini ial Explo a ion. In Ex ended Abs ac s o
he CHI Con e ence on Human Fac o s in Compu ing Sys ems, CHI EA 2024, Honolulu, HI, USA, May 11-16, 2024,
Flo ian ’Floyd’ Muelle , Penny Kybu z, Julie R. Williamson, and Co ina Sas (Eds.). ACM, Honolulu HI USA, 24:1–24:11.
h ps://doi.o g/10.1145/3613905.3650786
[79]
Kaiyuan Gao, Sunan He, Zhenyu He, Jiacheng Lin, Qizhi Pei, Jie Shao, and Wei Zhang. 2023. Examining Use -
F iendly and Open-Sou ced La ge GPT Models: A Su ey on Language, Mul imodal, and Scien i ic GPT Models.
h ps://doi.o g/10.48550/ARXIV.2308.14149 a Xi :2308.14149
[80]
Yun an Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng
Wang, and Hao en Wang. 2023. Re ie al-Augmen ed Gene a ion o La ge Language Models: A Su ey. h ps:
//doi.o g/10.48550/ARXIV.2312.10997 a Xi :2312.10997
[81]
Zhengjie Gao, Xuanzi Liu, Yuanshuai Lan, and Zheng Yang. 2024. A B ie Su ey on Sa e y o La ge Language Models.
Jou nal o Compu ing and In o ma ion Technology 32, 1 (2024), 47–64. h ps://doi.o g/10.20532/ci .2024.1005778
La ge Language Models: A Su ey o Su eys 33
[82]
Sil ia Ga cía-Méndez, F ancisco de A iba Pé ez, and Ma ia del Ca men Lopez-Pe ez. 2024. A Re iew on he Use o
La ge Language Models as Vi ual Tu o s. h ps://doi.o g/10.48550/ARXIV.2405.11983 a Xi :2405.11983
[83]
Vahid Ga ousi and Mika V. Män ylä. 2016. A Sys ema ic Li e a u e Re iew o Li e a u e Re iews in So wa e Tes ing.
In o ma ion and So wa e Technology 80 (Dec. 2016), 195–216. h ps://doi.o g/10.1016/j.in so .2016.09.002
[84]
Jiahui Geng, Fengyu Cai, Yuxia Wang, Heinz Koeppl, P esla Nako , and I yna Gu e ych. 2024. A Su ey o Con idence
Es ima ion and Calib a ion in La ge Language Models. In P oceedings o he 2024 Con e ence o he No h Ame ican
Chap e o he Associa ion o Compu a ional Linguis ics: Human Language Technologies (Volume 1: Long Pape s), NAACL
2024, Mexico Ci y, Mexico, June 16-21, 2024, Ke in Duh, Helena Gómez-Ado no, and S e en Be ha d (Eds.). Associa ion
o Compu a ional Linguis ics, Mexico Ci y, Mexico, 6577–6595. h ps://doi.o g/10.18653/V1/2024.NAACL-LONG.366
[85]
Caleb Ge en, Amanda Boa d, Gaby G. Daghe , Tim Ande sen, and Jun Zhuang. 2024. Blockchain o La ge Language
Model Secu i y and Sa e y: A Holis ic Su ey. h ps://doi.o g/10.48550/ARXIV.2407.20181 a Xi :2407.20181
[86]
Akash Ghosh, A kadeep Acha ya, S ipa na Saha, Vinija Jain, and Aman Chadha. 2024. Explo ing he F on ie o
Vision-Language Models: A Su ey o Cu en Me hodologies and Fu u e Di ec ions. h ps://doi.o g/10.48550/
ARXIV.2404.07214 a Xi :2404.07214
[87]
Panagio is Giadikia oglou, Ma ia Lympe aiou, Gio gos Filand ianos, and Gio gos S amou. 2024. Puzzle Sol ing Using
Reasoning o La ge Language Models: A Su ey. h ps://doi.o g/10.48550/ARXIV.2402.11291 a Xi :2402.11291
[88]
Simon Co nelius Go issen, S e an Saue , and Wol G. Beckmann. 2024. A Su ey o Na u al Language-Based Edi ing
o Low-Code Applica ions Using La ge Language Models. In Human-Cen e ed So wa e Enginee ing - 10 h IFIP WG
13.2 In e na ional Wo king Con e ence, HCSE 2024, Reykja ik, Iceland, July 8-10, 2024, P oceedings (Lec u e No es in
Compu e Science, Vol. 14793), Ma a K is ín Lá usdó i , Bilal Naq i, Regina Be nhaup , Ca melo A di o, and S e an
Saue (Eds.). Sp inge , Cham, 243–254. h ps://doi.o g/10.1007/978-3-031-64576-1_15
[89]
Candida Ma ia G eco, And ea Sime i, And ea Taga elli, and Es e Zumpano. 2023. T ans o me -Based Language
Models o Men al Heal h Issues: A Su ey. Pa e n Recogni ion Le e s 167 (2023), 204–211. h ps://doi.o g/10.1016/J.
PATREC.2023.02.016
[90]
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Bei ami, Bailan He, Gengyuan Zhang, Ruo ong Liao, Yao Qin, Volke
T esp, and Philip H. S. To . 2023. A Sys ema ic Su ey o P omp Enginee ing on Vision-Language Founda ion
Models. h ps://doi.o g/10.48550/ARXIV.2307.12980 a Xi :2307.12980
[91]
Shangwei Guo, Chunlong Xie, Jiwei Li, Lingjuan Lyu, and Tianwei Zhang. 2022. Th ea s o P e-T ained Language
Models: Su ey and Taxonomy. a Xi :2202.06862
[92]
Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Ni esh V. Chawla, Ola Wies , and Xiangliang
Zhang. 2024. La ge Language Model Based Mul i-Agen s: A Su ey o P og ess and Challenges. h ps://doi.o g/10.
48550/ARXIV.2402.01680 a Xi :2402.01680
[93]
Xu Guo and Han Yu. 2022. On he Domain Adap a ion and Gene aliza ion o P e ained Language Models: A Su ey.
h ps://doi.o g/10.48550/ARXIV.2211.03154 a Xi :2211.03154
[94]
Zishan Guo, Ren en Jin, Chuang Liu, Yu ei Huang, Dan Shi, Sup yadi, Linhao Yu, Yan Liu, Jiaxuan Li, Bojian Xiong,
and Deyi Xiong. 2023. E alua ing La ge Language Models: A Comp ehensi e Su ey. h ps://doi.o g/10.48550/
ARXIV.2310.19736 a Xi :2310.19736
[95]
Zhijun Guo, Al ina Lai, Johan Hilge Thygesen, Joseph Fa ing on, Thomas Keen, and Kezhi Li. 2024. La ge Language
Model o Men al Heal h: A Sys ema ic Re iew. h ps://doi.o g/10.48550/ARXIV.2403.15401 a Xi :2403.15401
[96]
Joschka Hal au de heide and Robe Ranisch. 2024. The E hics o Cha GPT in Medicine and Heal hca e: A Sys ema ic
Re iew on La ge Language Models (LLMs). npj Digi . Medicine 7, 1 (2024), 183. h ps://doi.o g/10.1038/S41746-024-
01157-X
[97]
Raby Hamadi. 2023. La ge Language Models Mee Compu e Vision: A B ie Su ey. h ps://doi.o g/10.48550/ARXIV.
2311.16673 a Xi :2311.16673
[98]
Tho s en Händle . 2023. Balancing Au onomy and Alignmen : A Mul i-Dimensional Taxonomy o Au onomous
LLM-powe ed Mul i-Agen A chi ec u es. h ps://doi.o g/10.48550/ARXIV.2310.03659 a Xi :2310.03659
[99]
I yna Ha sock and Ghulam Rasool. 2024. Vision-Language Models o Medical Repo Gene a ion and Visual
Ques ion Answe ing: A Re iew. h ps://doi.o g/10.48550/ARXIV.2403.02469 a Xi :2403.02469
[100]
Mohammed Hassanin and Nou Mous a a. 2024. A Comp ehensi e O e iew o La ge Language Models (Llms) o
Cybe De ences: Oppo uni ies and Di ec ions. h ps://doi.o g/10.48550/ARXIV.2405.14487 a Xi :2405.14487
[101]
Feng He, Tianqing Zhu, Dayong Ye, Bo Liu, Wanlei Zhou, and Philip S. Yu. 2024. The Eme ged Secu i y and P i acy
o LLM Agen : A Su ey wi h Case S udies. h ps://doi.o g/10.48550/ARXIV.2407.19354 a Xi :2407.19354
[102]
Kai He, Rui Mao, Qika Lin, Yucheng Ruan, Xiang Lan, Mengling Feng, and E ik Camb ia. 2023. A Su ey o
La ge Language Models o Heal hca e: F om Da a, Technology, and Applica ions o Accoun abili y and E hics.
h ps://doi.o g/10.48550/ARXIV.2310.05694 a Xi :2310.05694
[103]
Tianyu He, Guanghui Fu, Yijing Yu, Fan Wang, Jianqiang Li, Qing Zhao, Changwei Song, Hongzhi Qi, Dan Luo,
Huijing Zou, and Bing Xiang Yang. 2023. Towa ds a Psychological Gene alis AI: A Su ey o Cu en Applica ions
34 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
o La ge Language Models and Fu u e P ospec s. h ps://doi.o g/10.48550/ARXIV.2312.04578 a Xi :2312.04578
[104]
Yingqing He, Zhaoyang Liu, Jingye Chen, Zeyue Tian, Hongyu Liu, Xiaowei Chi, Run ao Liu, Ruibin Yuan, Yazhou
Xing, Wenhai Wang, Ji eng Dai, Yong Zhang, Wei Xue, Qi eng Liu, Yike Guo, and Qi eng Chen. 2024. LLMs Mee
Mul imodal Gene a ion and Edi ing: A Su ey. h ps://doi.o g/10.48550/ARXIV.2405.19334 a Xi :2405.19334
[105]
Zheyuan He, Zihao Li, and Sen Yang. 2024. La ge Language Models o Blockchain Secu i y: A Sys ema ic Li e a u e
Re iew. h ps://doi.o g/10.48550/ARXIV.2403.14280 a Xi :2403.14280
[106]
Xanh Ho, Anh-Khoa Duong Nguyen, Tuan-An Dao, Jun eng Jiang, Yuki Chida, Kai o Sugimo o, Huy Quoc To,
Flo ian Boudin, and Akiko Aizawa. 2024. A Su ey o P e-T ained Language Models o P ocessing Scien i ic Tex .
h ps://doi.o g/10.48550/ARXIV.2401.17824 a Xi :2401.17824
[107]
Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Fei an Huang, and Xiao Huang. 2024. Nex -
Gene a ion Da abase In e aces: A Su ey o LLM-based Tex - o-SQL. h ps://doi.o g/10.48550/ARXIV.2406.08426
a Xi :2406.08426
[108]
Max Ho , Anas asiia G ishina, and Leon Moonen. 2023. An Explo a o y Li e a u e S udy on Sha ing and Ene gy
Use o Language Models o Sou ce Code. In ACM/IEEE In e na ional Symposium on Empi ical So wa e Enginee ing
and Measu emen , ESEM 2023, New O leans, LA, USA, Oc obe 26-27, 2023. IEEE, New O leans, LA, USA, 1–12.
h ps://doi.o g/10.1109/ESEM56168.2023.10304803
[109]
Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, Da id Lo, John C. G undy, and
Haoyu Wang. 2023. La ge Language Models o So wa e Enginee ing: A Sys ema ic Li e a u e Re iew. h ps:
//doi.o g/10.48550/ARXIV.2308.10620 a Xi :2308.10620
[110]
Linmei Hu, Zeyi Liu, Ziwang Zhao, Lei Hou, Liqiang Nie, and Juanzi Li. 2024. A Su ey o Knowledge Enhanced
P e-T ained Language Models. IEEE T ansac ions on Knowledge and Da a Enginee ing 36, 4 (2024), 1413–1430.
h ps://doi.o g/10.1109/TKDE.2023.3310002
[111]
Sihao Hu, Tiansheng Huang, Fa ih Ilhan, Selim F. Tekin, Gaowen Liu, Ramana Kompella, and Ling Liu. 2024. A
Su ey on La ge Language Model-Based Game Agen s. h ps://doi.o g/10.48550/ARXIV.2404.02039 a Xi :2404.02039
[112]
Yucheng Hu and Yuxing Lu. 2024. RAG and RAU: A Su ey on Re ie al-Augmen ed Language Model in Na u al
Language P ocessing. h ps://doi.o g/10.48550/ARXIV.2404.19543 a Xi :2404.19543
[113]
Yining Hua, Fenglin Liu, Kailai Yang, Zehan Li, Yi-han Sheu, Peilin Zhou, Lau en V. Mo an, Sophia Ananiadou, and
And ew Beam. 2024. La ge Language Models in Men al Heal h Ca e: A Scoping Re iew. h ps://doi.o g/10.48550/
ARXIV.2401.02984 a Xi :2401.02984
[114]
Jie Huang and Ke in Chen-Chuan Chang. 2023. Towa ds Reasoning in La ge Language Models: A Su ey. In Findings
o he Associa ion o Compu a ional Linguis ics: ACL 2023, To on o, Canada, July 9-14, 2023, Anna Roge s, Jo dan L.
Boyd-G abe , and Naoaki Okazaki (Eds.). Associa ion o Compu a ional Linguis ics, To on o, Canada, 1049–1065.
h ps://doi.o g/10.18653/V1/2023.FINDINGS-ACL.67
[115]
Kaiyu Huang, Feng an Mo, Hongliang Li, You Li, Yuanchi Zhang, Weijian Yi, Yulong Mao, Jinchen Liu, Yuzhuang
Xu, Jinan Xu, Jian-Yun Nie, and Yang Liu. 2024. A Su ey on La ge Language Models wi h Mul ilingualism: Recen
Ad ances and New F on ie s. h ps://doi.o g/10.48550/ARXIV.2405.10936 a Xi :2405.10936
[116]
Lei Huang, Weijiang Yu, Wei ao Ma, Weihong Zhong, Zhangyin Feng, Hao ian Wang, Qianglong Chen, Weihua Peng,
Xiaocheng Feng, Bing Qin, and Ting Liu. 2023. A Su ey on Hallucina ion in La ge Language Models: P inciples,
Taxonomy, Challenges, and Open Ques ions. h ps://doi.o g/10.48550/ARXIV.2311.05232 a Xi :2311.05232
[117]
Linghan Huang, Peizhou Zhao, Huaming Chen, and Lei Ma. 2024. La ge Language Models Based Fuzzing Techniques:
A Su ey. h ps://doi.o g/10.48550/ARXIV.2402.00350 a Xi :2402.00350
[118]
Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, De u Lian, Yasheng Wang, Ruiming Tang, and
Enhong Chen. 2024. Unde s anding he Planning o LLM Agen s: A Su ey. h ps://doi.o g/10.48550/ARXIV.2402.
02716 a Xi :2402.02716
[119]
Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, Changshun Wu, Saddek Bensalem, Ronghui Mu, Yi
Qi, Xingyu Zhao, Kaiwen Cai, Yanghao Zhang, Sihao Wu, Peipei Xu, Dengyu Wu, And é F ei as, and Mus a a A.
Mus a a. 2024. A Su ey o Sa e y and T us wo hiness o La ge Language Models h ough he Lens o Ve i ica ion
and Valida ion. A i icial In elligence Re iew 57, 7 (2024), 175. h ps://doi.o g/10.1007/S10462-024-10824-0
[120]
Yizheng Huang and Jimmy Huang. 2024. A Su ey on Re ie al-Augmen ed Tex Gene a ion o La ge Language
Models. h ps://doi.o g/10.48550/ARXIV.2404.10981 a Xi :2404.10981
[121]
Yining Huang, Keke Tang, and Meilian Chen. 2024. A Comp ehensi e Su ey on E alua ing La ge Language Model
Applica ions in he Medical Indus y. h ps://doi.o g/10.48550/ARXIV.2404.15777 a Xi :2404.15777
[122]
Yunpeng Huang, Jingwei Xu, Zixu Jiang, Junyu Lai, Zenan Li, Yuan Yao, Taolue Chen, Lijuan Yang, Zhou Xin, and
Xiaoxing Ma. 2023. Ad ancing T ans o me A chi ec u e in Long-Con ex La ge Language Models: A Comp ehensi e
Su ey. h ps://doi.o g/10.48550/ARXIV.2311.12351 a Xi :2311.12351
[123]
Rasha Ahmad Husein, Hala Abu ajouh, and Caga ay Ca al. 2025. La ge Language Models o Code Comple ion: A
Sys ema ic Li e a u e Re iew. Compu e S anda ds & In e aces 92 (2025), 103917. h ps://doi.o g/10.1016/J.CSI.2024.
La ge Language Models: A Su ey o Su eys 35
103917
[124]
A ab Hussain, Md. Ra iqul Islam Rabin, Tou ique Ahmed, Bowen Xu, P emkuma T. De anbu, and Mohammad Amin
Alipou . 2024. T ojans in La ge Language Models o Code: A C i ical Re iew h ough a T igge -Based Taxonomy.
h ps://doi.o g/10.48550/ARXIV.2405.02828 a Xi :2405.02828
[125]
Sho a o Ishiha a. 2023. T aining Da a Ex ac ion om P e-T ained Language Models: A Su ey. h ps://doi.o g/10.
48550/ARXIV.2305.16157 a Xi :2305.16157
[126]
J. Jesson, L. Ma heson, and F.M. Lacey. 2011. Doing You Li e a u e Re iew: T adi ional and Sys ema ic Techniques.
SAGE Publica ions. h ps://books.google.no/books?id=NAY Lb8qsd4C
[127]
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A Su ey on La ge Language Models o
Code Gene a ion. h ps://doi.o g/10.48550/ARXIV.2406.00515 a Xi :2406.00515
[128]
Ruili Jiang, Kehai Chen, Xue eng Bai, Zhixuan He, Jun ao Li, Muyun Yang, Tiejun Zhao, Liqiang Nie, and Min Zhang.
2024. A Su ey on Human P e e ence Lea ning o La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2406.
11191 a Xi :2406.11191
[129]
Xuhui Jiang, Yuxing Tian, Feng ui Hua, Chengjin Xu, Yuanzhuo Wang, and Jian Guo. 2024. A Su ey on La ge Lan-
guage Model Hallucina ion ia a C ea i i y Pe spec i e. h ps://doi.o g/10.48550/ARXIV.2402.06647 a Xi :2402.06647
[130]
Yushan Jiang, Zijie Pan, Xikun Zhang, Sahil Ga g, Ande son Schneide , Yu iy Ne my aka, and Dongjin Song. 2024.
Empowe ing Time Se ies Analysis wi h La ge Language Models: A Su ey. h ps://doi.o g/10.48550/ARXIV.2402.03182
a Xi :2402.03182
[131]
Bowen Jin, Gang Liu, Chi Han, Meng Jiang, Heng Ji, and Jiawei Han. 2023. La ge Language Models on G aphs: A
Comp ehensi e Su ey. h ps://doi.o g/10.48550/ARXIV.2312.02783 a Xi :2312.02783
[132]
Haibo Jin, Leyang Hu, Xinuo Li, Peiyan Zhang, Chonghan Chen, Jun Zhuang, and Haohan Wang. 2024. Jailb eakZoo:
Su ey, Landscapes, and Ho izons in Jailb eaking La ge Language and Vision-Language Models. h ps://doi.o g/10.
48550/ARXIV.2407.01599 a Xi :2407.01599
[133]
Hanlei Jin, Yang Zhang, Dan Meng, Jun Wang, and Jinghua Tan. 2024. A Comp ehensi e Su ey on P ocess-O ien ed
Au oma ic Tex Summa iza ion wi h Explo a ion o LLM-based Me hods. h ps://doi.o g/10.48550/ARXIV.2403.02901
a Xi :2403.02901
[134]
Yizhang Jin, Jian Li, Yexin Liu, Tianjun Gu, Kai Wu, Zhengkai Jiang, Muyang He, Bo Zhao, Xin Tan, Zhenye Gan,
Yabiao Wang, Chengjie Wang, and Lizhuang Ma. 2024. E icien Mul imodal La ge Language Models: A Su ey.
h ps://doi.o g/10.48550/ARXIV.2405.10739 a Xi :2405.10739
[135]
Zhijing Jin, Nils Heil, Jia ui Liu, Shehzaad Dhuliawala, Yahang Qi, Be nha d Schölkop , Rada Mihalcea, and M inmaya
Sachan. 2024. Implici Pe sonaliza ion in Language Models: A Sys ema ic S udy. h ps://doi.o g/10.48550/ARXIV.
2405.14808 a Xi :2405.14808
[136]
Zhi Jing, Yongye Su, Yikun Han, Bo Yuan, Haiyun Xu, Chunjiang Liu, Kehai Chen, and Min Zhang. 2024. When La ge
Language Models Mee Vec o Da abases: A Su ey. h ps://doi.o g/10.48550/ARXIV.2402.01763 a Xi :2402.01763
[137]
Mladjan Jo ano ic and Pe e Voss. 2024. Towa ds Inc emen al Lea ning in La ge Language Models: A C i ical Re iew.
h ps://doi.o g/10.48550/ARXIV.2404.18311 a Xi :2404.18311
[138]
Ch is o o os Kach is. 2024. A Su ey on Ha dwa e Accele a o s o La ge Language Models. h ps://doi.o g/10.
48550/ARXIV.2401.09890 a Xi :2401.09890
[139]
Ka ikapalli Sub amanyam Kalyan. 2024. A Su ey o GPT-3 Family La ge Language Models Including Cha GPT and
GPT-4. Na u al Language P ocessing Jou nal 6 (2024), 100048. h ps://doi.o g/10.1016/J.NLP.2023.100048
[140]
Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. 2024. When Can Llms Ac ually Co ec Thei Own
Mis akes? A C i ical Su ey o Sel -Co ec ion o Llms. h ps://doi.o g/10.48550/ARXIV.2406.01297 a Xi :2406.01297
[141]
An onia Ka amolegkou, Mos a a Abdou, and Ande s Søgaa d. 2023. Mapping B ains wi h Language Models: A Su ey.
In Findings o he Associa ion o Compu a ional Linguis ics: ACL 2023, To on o, Canada, July 9-14, 2023, Anna Roge s,
Jo dan L. Boyd-G abe , and Naoaki Okazaki (Eds.). Associa ion o Compu a ional Linguis ics, To on o, Canada,
9748–9762. h ps://doi.o g/10.18653/V1/2023.FINDINGS-ACL.618
[142]
P a nee Kau , Gau am Siddha h Kashyap, Anki Kuma , Md. Tab ez Na is, Sandeep Kuma , and Vik an Shokeen.
2024. F om Tex o T ans o ma ion: A Comp ehensi e Re iew o La ge Language Models’ Ve sa ili y. h ps:
//doi.o g/10.48550/ARXIV.2402.16142 a Xi :2402.16142
[143]
Luoma Ke, Song Tong, Peng Cheng, and Kaiping Peng. 2024. Explo ing he F on ie s o Llms in Psychological
Applica ions: A Comp ehensi e Re iew. h ps://doi.o g/10.48550/ARXIV.2401.01519 a Xi :2401.01519
[144]
K ishna am Ken hapadi, Meh noosh Sameki, and Anku Taly. 2024. G ounding and E alua ion o La ge Language
Models: P ac ical Challenges and Lessons Lea ned (Su ey). In P oceedings o he 30 h ACM SIGKDD Con e ence on
Knowledge Disco e y and Da a Mining, KDD 2024, Ba celona, Spain, Augus 25-29, 2024, Rica do Baeza-Ya es and
F ancesco Bonchi (Eds.). ACM, Ba celona Spain, 6523–6533. h ps://doi.o g/10.1145/3637528.3671467
[145]
Mahsa Khoshnoodi, Vinija Jain, Mingye Gao, Mala ika S ikan h, and Aman Chadha. 2024. A Comp ehensi e Su ey
o Accele a ed Gene a ion Techniques in La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2405.13019
36 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
a Xi :2405.13019
[146]
Yeseung Kim, Dohyun Kim, Jieun Choi, Jisang Pa k, Nayoung Oh, and Daehyung Pa k. 2024. A Su ey on In eg a ion
o La ge Language Models wi h In elligen Robo s. h ps://doi.o g/10.48550/ARXIV.2404.09228 a Xi :2404.09228
[147]
Hannah Rose Ki k, Be ie Vidgen, Paul Rö ge , and Sco A. Hale. 2023. Pe sonalisa ion wi hin Bounds: A Risk
Taxonomy and Policy F amewo k o he Alignmen o La ge Language Models wi h Pe sonalised Feedback. h ps:
//doi.o g/10.48550/ARXIV.2303.05453 a Xi :2303.05453
[148]
Ba ba a Ann Ki chenham and S ua Cha e s. 2007. Guidelines o Pe o ming Sys ema ic Li e a u e Re iews in
So wa e Enginee ing. Technical Repo EBSE 2007-001. Keele Uni e si y and Du ham Uni e si y Join Repo / Keele
Uni e si y. h ps://www.else ie .com/__da a/p omis_misc/525444sys ema ic e iewsguide.pd
[149]
E ans Ko ei and Ramkuma Thi una uka asu. 2023. A Sys ema ic Re iew o T ans o me -Based P e-T ained
Language Models h ough Sel -Supe ised Lea ning. In . 14, 3 (2023), 187. h ps://doi.o g/10.3390/INFO14030187
[150]
Zoe Ko i, Ra aila Galanopoulou, and Diomidis Spinellis. 2023. Machine Lea ning o So wa e Enginee ing: A
Te ia y S udy. Compu . Su eys 55, 12 (Dec. 2023), 1–39. h ps://doi.o g/10.1145/3572905 a Xi :2211.09425 [cs]
[151]
Labeha K yeziu and Visa Shehu. 2022. A Su ey o Using Unsupe ised Lea ning Techniques in Building Masked
Language Models o Low Resou ce Languages. In 11 h Medi e anean Con e ence on Embedded Compu ing, MECO 2022,
Bud a, Mon eneg o, June 7-10, 2022. IEEE, Bud a, Mon eneg o, 1–6. h ps://doi.o g/10.1109/MECO55406.2022.9797081
[152]
Sanjay Kuk eja, Ta un Kuma , Ami Pu ohi , Abhiji Dasgup a, and Debashis Guha. 2024. A Li e a u e Su ey on
Open Sou ce La ge Language Models. In P oceedings o he 7 h In e na ional Con e ence on Compu e s in Managemen
and Business, ICCMB 2024, Singapo e, Janua y 12-14, 2024. ACM, Singapo e Singapo e, 133–143. h ps://doi.o g/10.
1145/3647782.3647803
[153]
P anjal Kuma . 2024. La ge Language Models (LLMs): Su ey, Technical F amewo ks, and Fu u e Challenges. A i icial
In elligence Re iew 57, 9 (2024), 260. h ps://doi.o g/10.1007/S10462-024-10888-Y
[154]
Sachin Kuma , Vidhisha Balachand an, Lucille Njoo, An onios Anas asopoulos, and Yulia Ts e ko . 2023. Language
Gene a ion Models Can Cause Ha m: So Wha Can We Do abou I ? An Ac ionable Su ey. In P oceedings o he 17 h
Con e ence o he Eu opean Chap e o he Associa ion o Compu a ional Linguis ics, EACL 2023, Dub o nik, C oa ia,
May 2-6, 2023, And eas Vlachos and Isabelle Augens ein (Eds.). Associa ion o Compu a ional Linguis ics, Dub o nik,
C oa ia, 3291–3313. h ps://doi.o g/10.18653/V1/2023.EACL-MAIN.241
[155]
Su ende Su esh Kuma , Ma y L. Cummings, and Alexande J. S impson. 2024. S eng hening LLM T us Bounda ies:
A Su ey o P omp Injec ion A acks Su ende Su esh Kuma D . M.L. Cummings D . Alexande S impson. In 4 h
IEEE In e na ional Con e ence on Human-Machine Sys ems, ICHMS 2024, To on o, on, Canada, May 15-17, 2024. IEEE,
To on o, ON, Canada, 1–6. h ps://doi.o g/10.1109/ICHMS59971.2024.10555871
[156]
Jinqi Lai, Wensheng Gan, Jiayang Wu, Zhenlian Qi, and Philip S. Yu. 2023. La ge Language Models in Law: A Su ey.
h ps://doi.o g/10.48550/ARXIV.2312.03718 a Xi :2312.03718
[157]
Ha sh Nishan Lalai, Aashish Anan ha Ramak ishnan, Raj Sanjay Shah, and Dongwon Lee. 2024. F om In en ions
o Techniques: A Comp ehensi e Taxonomy and Challenges in Tex Wa e ma king o La ge Language Models.
h ps://doi.o g/10.48550/ARXIV.2406.11106 a Xi :2406.11106
[158]
Wei Lan, Guohang He, Mingyang Liu, Qing eng Chen, Junyue Cao, and Wei Peng. 2024. T ans o me -Based Single-Cell
Language Model: A Su ey. h ps://doi.o g/10.48550/ARXIV.2407.13205 a Xi :2407.13205
[159]
Md. Tahmid Rahman Laska , Sawsan Alqah ani, M. Sai ul Ba i, Mizanu Rahman, Mohammad Abdullah Ma in
Khan, Haida Khan, Is a Jahan, Am an Bhuiyan, Chee Wei Tan, Md. Rizwan Pa ez, Enamul Hoque, Sha iq Jo y,
and Jimmy Xiangji Huang. 2024. A Sys ema ic Su ey and C i ical Re iew on E alua ing La ge Language Models:
Challenges, Limi a ions, and Recommenda ions. h ps://doi.o g/10.48550/ARXIV.2407.04069 a Xi :2407.04069
[160]
Jinsook Lee, Yann Hicke, Renzhe Yu, Ch is ophe B ooks, and René F. Kizilcec. 2024. The Li e Cycle o La ge Language
Models: A Re iew o Biases in Educa ion. h ps://doi.o g/10.48550/ARXIV.2407.11203 a Xi :2407.11203
[161]
Jean Lee, Nicholas S e ens, Soyeon Ca en Han, and Minseok Song. 2024. A Su ey o La ge Language Models in
Finance (FinLLMs). h ps://doi.o g/10.48550/ARXIV.2402.02315 a Xi :2402.02315
[162]
Nayeon Lee, Yejin Bang, Holy Lo enia, Samuel Cahyawijaya, Wenliang Dai, and Pascale Fung. 2023. Su ey o Social
Bias in Vision-Language Models. h ps://doi.o g/10.48550/ARXIV.2309.14381 a Xi :2309.14381
[163]
Baolin Li, Yankai Jiang, Vijay Gadepally, and De esh Tiwa i. 2024. LLM In e ence Se ing: Su ey o Recen Ad ances
and Oppo uni ies. h ps://doi.o g/10.48550/ARXIV.2407.12391 a Xi :2407.12391
[164]
Dong ang Li, Ze ian Sun, Xinshuo Hu, Zhenyu Liu, Ziyang Chen, Bao ian Hu, Aiguo Wu, and Min Zhang. 2023. A
Su ey o La ge Language Models A ibu ion. h ps://doi.o g/10.48550/ARXIV.2311.03731 a Xi :2311.03731
[165]
DaiFeng Li and Fan Xu. 2024. Syne gizing Knowledge G aphs wi h La ge Language Models: A Comp ehensi e
Re iew and Fu u e P ospec s. h ps://doi.o g/10.48550/ARXIV.2407.18470 a Xi :2407.18470
[166]
Haochen Li, Jona han Leung, and Zhiqi Shen. 2024. Towa ds Goal-O ien ed La ge Language Model P omp ing: A
Su ey. h ps://doi.o g/10.48550/ARXIV.2401.14043 a Xi :2401.14043
La ge Language Models: A Su ey o Su eys 37
[167]
Junyi Li, Tianyi Tang, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2024. P e-T ained Language Models o Tex
Gene a ion: A Su ey. Acm Compu ing Su eys 56, 9 (2024), 230:1–230:39. h ps://doi.o g/10.1145/3649449
[168]
Jiawei Li, Yizhe Yang, Yu Bai, Xiao eng Zhou, Yinghao Li, Huashan Sun, Yuhang Liu, Xingpeng Si, Yuhao Ye, Yixiao
Wu, Yiguan Lin, Bin Xu, Ren Bowen, Chong Feng, Yang Gao, and Heyan Huang. 2024. Fundamen al Capabili ies o
La ge Language Models and Thei Applica ions in Domain Scena ios: A Su ey. In P oceedings o he 62nd Annual
Mee ing o he Associa ion o Compu a ional Linguis ics (Volume 1: Long Pape s), ACL 2024, Bangkok, Thailand, Augus
11-16, 2024, Lun-Wei Ku, And e Ma ins, and Vi ek S ikuma (Eds.). Associa ion o Compu a ional Linguis ics,
Bangkok, Thailand, 11116–11141. h ps://doi.o g/10.18653/ 1/2024.acl-long.599
[169]
Lei Li, Yong eng Zhang, Dugang Liu, and Li Chen. 2024. La ge Language Models o Gene a i e Recommenda ion:
A Su ey and Visiona y Discussions. In P oceedings o he 2024 Join In e na ional Con e ence on Compu a ional
Linguis ics, Language Resou ces and E alua ion (LREC-COLING 2024), Nicole a Calzola i, Min-Yen Kan, Vé onique
Hos e, Alessand o Lenci, Sak iani Sak i, and Nianwen Xue (Eds.). ELRA and ICCL, To ino, I alia, 10146–10159.
[170]
Lingyao Li, Jiayan Zhou, Zhenxiang Gao, Wenyue Hua, Lizhou Fan, Huizi Yu, Loni Hagen, Yong eng Zhang, Themis-
ocles L. Assimes, Libby Hemphill, and Siyuan Ma. 2024. A Scoping Re iew o Using La ge Language Models (Llms)
o In es iga e Elec onic Heal h Reco ds (Eh s). h ps://doi.o g/10.48550/ARXIV.2405.03066 a Xi :2405.03066
[171]
Xinzhe Li. 2024. A Su ey on LLM-based Agen s: Common Wo k lows and Reusable LLM-p o iled Componen s.
h ps://doi.o g/10.48550/ARXIV.2406.05804 a Xi :2406.05804
[172]
Yingji Li, Mengnan Du, Rui Song, Xin Wang, and Ying Wang. 2023. A Su ey on Fai ness in La ge Language Models.
h ps://doi.o g/10.48550/ARXIV.2308.10149 a Xi :2308.10149
[173]
Yun Li, Kai Ka suma a, Ehsan Ja anma di, and Manabu Tsukada. 2024. La ge Language Models o Human-like
Au onomous D i ing: A Su ey. h ps://doi.o g/10.48550/ARXIV.2407.19280 a Xi :2407.19280
[174]
Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Je ey Xu Yu. 2023. A Su ey o
G aph Mee s La ge Language Model: P og ess and Fu u e Di ec ions. h ps://doi.o g/10.48550/ARXIV.2311.12399
a Xi :2311.12399
[175]
Yongqi Li, Xinyu Lin, Wenjie Wang, Fuli Feng, Liang Pang, Wenjie Li, Liqiang Nie, Xiangnan He, and Ta -Seng
Chua. 2024. A Su ey o Gene a i e Sea ch and Recommenda ion in he E a o La ge Language Models. h ps:
//doi.o g/10.48550/ARXIV.2404.16924 a Xi :2404.16924
[176]
Yinheng Li, Shao ei Wang, Han Ding, and Hang Chen. 2023. La ge Language Models in Finance: A Su ey. In 4 h
ACM In e na ional Con e ence on AI in Finance, ICAIF 2023, B ooklyn, NY, USA, No embe 27-29, 2023. ACM, B ooklyn
NY USA, 374–382. h ps://doi.o g/10.1145/3604237.3626869
[177]
Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang
Wang, Yi Sun, Rui Kong, Yile Wang, Han ei Geng, Jian Luan, Xue eng Jin, Zilong Ye, Guanjing Xiong, Fan Zhang,
Xiang Li, Mengwei Xu, Zhijun Li, Peng Li, Yang Liu, Ya-Qin Zhang, and Yunxin Liu. 2024. Pe sonal LLM Agen s:
Insigh s and Su ey abou he Capabili y, E iciency and Secu i y. h ps://doi.o g/10.48550/ARXIV.2401.05459
a Xi :2401.05459
[178]
Zhen Li, Xiaohan Xu, Tao Shen, Can Xu, Jia-Chen Gu, and Chongyang Tao. 2024. Le e aging La ge Language Models
o NLG E alua ion: A Su ey. h ps://doi.o g/10.48550/ARXIV.2401.07103 a Xi :2401.07103
[179]
Xun Liang, Shichao Song, Zi an Zheng, Hanyu Wang, Qingchen Yu, Xunkai Li, Rong-Hua Li, Feiyu Xiong, and Zhiyu
Li. 2024. In e nal Consis ency and Sel -Feedback in La ge Language Models: A Su ey. h ps://doi.o g/10.48550/
ARXIV.2407.14507 a Xi :2407.14507
[180]
Chang Liao, Yemin Yu, Yu Mei, and Ying Wei. 2024. F om Wo ds o Molecules: A Su ey o La ge Language Models
in Chemis y. h ps://doi.o g/10.48550/ARXIV.2402.01439 a Xi :2402.01439
[181]
Alexande Lig ha , Caga ay Ca al, and Bedi Tekine dogan. 2021. Sys ema ic Re iews in Sen imen Analysis: A
Te ia y S udy. A i icial In elligence Re iew 54, 7 (Oc . 2021), 4997–5053. h ps://doi.o g/10.1007/s10462-021-09973-3
[182]
Jianghao Lin, Xinyi Dai, Yunjia Xi, Weiwen Liu, Bo Chen, Xiangyang Li, Chenxu Zhu, Hui eng Guo, Yong Yu, Ruiming
Tang, and Weinan Zhang. 2023. How Can Recommende Sys ems Bene i om La ge Language Models: A Su ey.
h ps://doi.o g/10.48550/ARXIV.2306.05817 a Xi :2306.05817
[183]
Michael Pin-Chuan Lin, Daniel Chang, Sa ah Hall, and Gaganp ee Jhajj. 2024. P elimina y Sys ema ic Re iew o
Open-Sou ce La ge Language Models in Educa ion. In Gene a i e In elligence and In elligen Tu o ing Sys ems - 20 h
In e na ional Con e ence, ITS 2024, Thessaloniki, G eece, June 10-13, 2024, P oceedings, Pa I (Lec u e No es in Compu e
Science, Vol. 14798), Angelo Si ale as and Fuhua Lin (Eds.). Sp inge , Cham, 68–77. h ps://doi.o g/10.1007/978-3-031-
63028-6_6
[184]
Zichao Lin, Shuyan Guan, Wending Zhang, Huiyan Zhang, Yugang Li, and Huaping Zhang. 2024. Towa ds T us wo hy
LLMs: A Re iew on Debiasing and Dehallucina ing in La ge Language Models. A i icial In elligence Re iew 57, 9
(2024), 243. h ps://doi.o g/10.1007/S10462-024-10896-Y
[185]
Chen Ling, Xujiang Zhao, Jiaying Lu, Chengyuan Deng, Can Zheng, Junxiang Wang, Tanmoy Chowdhu y, Yun Li,
Hejie Cui, Xuchao Zhang, Tianjiao Zhao, Ami Panalka , Wei Cheng, Haoyu Wang, Yanchi Liu, Zhengzhang Chen,
38 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
Hai eng Chen, Ch is Whi e, Quanquan Gu, Ca l Yang, and Liang Zhao. 2023. Beyond One-Model-Fi s-All: A Su ey
o Domain Specializa ion o La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2305.18703 a Xi :2305.18703
[186]
Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Lijie Wen, I win King, and Philip S. Yu. 2023. A Su ey o Tex
Wa e ma king in he E a o La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2312.07913 a Xi :2312.07913
[187]
Daizong Liu, Mingyu Yang, Xiaoye Qu, Pan Zhou, Yu Cheng, and Wei Hu. 2024. A Su ey o A acks on La ge
Vision-Language Models: Resou ces, Ad ances, and Fu u e T ends. h ps://doi.o g/10.48550/ARXIV.2407.07403
a Xi :2407.07403
[188] F ank Weizhen Liu and Chenhui Hu. 2024. Explo ing Vulne abili ies and P o ec ions in La ge Language Models: A
Su ey. h ps://doi.o g/10.48550/ARXIV.2406.00240 a Xi :2406.00240
[189]
Hanchao Liu, Wenyuan Xue, Yi ei Chen, Dapeng Chen, Xiu ian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei
Peng. 2024. A Su ey on Hallucina ion in La ge Vision-Language Models. h ps://doi.o g/10.48550/ARXIV.2402.00253
a Xi :2402.00253
[190]
Lei Liu, Xiaoyan Yang, Junchi Lei, Xiaoyang Liu, Yue Shen, Zhiqiang Zhang, Peng Wei, Jinjie Gu, Zhixuan Chu, Zhan
Qin, and Kui Ren. 2024. A Su ey on Medical La ge Language Models: Technology, Applica ion, T us wo hiness,
and Fu u e Di ec ions. h ps://doi.o g/10.48550/ARXIV.2406.03712 a Xi :2406.03712
[191]
Peng Liu, Lemei Zhang, and Jon A le Gulla. 2023. P e-T ain, P omp , and Recommenda ion: A Comp ehensi e
Su ey o Language Modeling Pa adigm Adap a ions in Recommende Sys ems. T ansac ions o he Associa ion o
Compu a ional Linguis ics 11 (2023), 1553–1571. h ps://doi.o g/10.1162/TACL_A_00619
[192] Wen ao Liu, Hanglei Hu, Jie Zhou, Yuyang Ding, Junsong Li, Jiayi Zeng, Mengliang He, Qin Chen, Bo Jiang, Aimin
Zhou, and Liang He. 2023. Ma hema ical Language Models: A Su ey. h ps://doi.o g/10.48550/ARXIV.2312.07622
a Xi :2312.07622
[193]
Xiaoxia Liu, Jingyi Wang, Jun Sun, Xiaohan Yuan, Guoliang Dong, Peng Di, Wenhai Wang, and Dongxia Wang.
2023. P omp ing F amewo ks o La ge Language Models: A Su ey. h ps://doi.o g/10.48550/ARXIV.2311.12785
a Xi :2311.12785
[194]
Xiaoyu Liu, Paiheng Xu, Junda Wu, Jiaxin Yuan, Yi an Yang, Yuhang Zhou, Fuxiao Liu, Tian ui Guan, Haoliang
Wang, Tong Yu, Julian J. McAuley, Wei Ai, and Fu ong Huang. 2024. La ge Language Models and Causal In e ence in
Collabo a ion: A Comp ehensi e Su ey. h ps://doi.o g/10.48550/ARXIV.2403.09606 a Xi :2403.09606
[195]
Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. 2024. Da ase s o La ge Language Models: A
Comp ehensi e Su ey. h ps://doi.o g/10.48550/ARXIV.2402.18041 a Xi :2402.18041
[196]
Yiheng Liu, Hao He, Tianle Han, Xu Zhang, Mengyuan Liu, Jiaming Tian, Yu ong Zhang, Jiaqi Wang, Xiaohui Gao,
Tianyang Zhong, Yi Pan, Shaochen Xu, Zihao Wu, Zhengliang Liu, Xin Zhang, Shu Zhang, Xin ao Hu, Tuo Zhang,
Ning Qiang, Tianming Liu, and Bao Ge. 2024. Unde s anding Llms: A Comp ehensi e O e iew om T aining o
In e ence. h ps://doi.o g/10.48550/ARXIV.2401.02038 a Xi :2401.02038
[197]
Yang Liu, Yuanshun Yao, Jean-F ancois Ton, Xiaoying Zhang, Ruocheng Guo, Hao Cheng, Yego Klochko , Muham-
mad Faaiz Tau iq, and Hang Li. 2023. T us wo hy Llms: A Su ey and Guideline o E alua ing La ge Language
Models’ Alignmen . h ps://doi.o g/10.48550/ARXIV.2308.05374 a Xi :2308.05374
[198]
Ze ang Liu. 2024. A Re iew o Ad ancemen s and Applica ions o P e-T ained Language Models in Cybe secu i y. In
12 h In e na ional Symposium on Digi al Fo ensics and Secu i y, ISDFS 2024, San An onio, TX, USA, Ap il 29-30, 2024.
IEEE, San An onio, TX, USA, 1–10. h ps://doi.o g/10.1109/ISDFS60797.2024.10527236
[199]
Lin Long, Rui Wang, Ruixuan Xiao, Junbo Zhao, Xiao Ding, Gang Chen, and Haobo Wang. 2024. On Llms-D i en
Syn he ic Da a Gene a ion, Cu a ion, and E alua ion: A Su ey. In Findings o he Associa ion o Compu a ional
Linguis ics, ACL 2024, Bangkok, Thailand and Vi ual Mee ing, Augus 11-16, 2024, Lun-Wei Ku, And e Ma ins,
and Vi ek S ikuma (Eds.). Associa ion o Compu a ional Linguis ics, Bangkok, Thailand and i ual mee ing,
11065–11082. h ps://doi.o g/10.18653/ 1/2024. indings-acl.658
[200]
Siqu Long, Feiqi Cao, Soyeon Ca en Han, and Haiqin Yang. 2022. Vision-and-Language P e ained Models: A Su ey.
In P oceedings o he Thi y-Fi s In e na ional Join Con e ence on A i icial In elligence, IJCAI 2022, Vienna, Aus ia,
23-29 July 2022, Luc De Raed (Ed.). ijcai.o g, Vienna, Aus ia, 5530–5537. h ps://doi.o g/10.24963/IJCAI.2022/773
[201]
Jinliang Lu, Ziliang Pang, Min Xiao, Yaochen Zhu, Rui Xia, and Jiajun Zhang. 2024. Me ge, Ensemble, and Coope a e! A
Su ey on Collabo a i e S a egies in he E a o La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2407.06089
a Xi :2407.06089
[202]
Weizheng Lu, Jiaming Zhang, Jing Zhang, and Yueguo Chen. 2024. La ge Language Model o Table P ocessing: A
Su ey. h ps://doi.o g/10.48550/ARXIV.2402.05121 a Xi :2402.05121
[203]
Haoyan Luo and Lucia Specia. 2024. F om Unde s anding o U iliza ion: A Su ey on Explainabili y o La ge
Language Models. h ps://doi.o g/10.48550/ARXIV.2401.12874 a Xi :2401.12874
[204]
Man Luo, Sh inidhi Kumbha , Ming Shen, Mihi Pa ma , Nee aj Va shney, P a yay Bane jee, Somak Adi ya, and Chi a
Ba al. 2023. Towa ds LogiGLUE: A B ie Su ey and A Benchma k o Analyzing Logical Reasoning Capabili ies o
Language Models. h ps://doi.o g/10.48550/ARXIV.2310.00836 a Xi :2310.00836
La ge Language Models: A Su ey o Su eys 39
[205]
Man Luo, Xin Xu, Yue Liu, Panupong Pasupa , and Meh an Kazemi. 2024. In-Con ex Lea ning wi h Re ie ed
Demons a ions o Language Models: A Su ey. h ps://doi.o g/10.48550/ARXIV.2401.11624 a Xi :2401.11624
[206]
Xudong Luo, Zhiqi Deng, Binxia Yang, and Michael Y. Luo. 2024. P e-T ained Language Models in Medicine: A
Su ey. A i icial In elligence in Medicine 154 (2024), 102904. h ps://doi.o g/10.1016/J.ARTMED.2024.102904
[207]
Qun Ma, Xiao Xue, Deyu Zhou, Xiangning Yu, Donghua Liu, Xuwen Zhang, Zihan Zhao, Yi an Shen, Peilin Ji,
Juanjuan Li, Gang Wang, and Wanpeng Ma. 2024. Compu a ional Expe imen s Mee La ge Language Model Based
Agen s: A Su ey and Pe spec i e. h ps://doi.o g/10.48550/ARXIV.2402.00262 a Xi :2402.00262
[208]
Xianzheng Ma, Yash Bhalga , B andon Sma , Shuai Chen, Xinghui Li, Jian Ding, Jindong Gu, Da e Zhenyu Chen,
Songyou Peng, Jia-Wang Bian, Philip H. S. To , Ma c Polle eys, Ma hias Nießne , Ian D. Reid, Angel X. Chang, I o
Laina, and Vic o Ad ian P isaca iu. 2024. When Llms S ep in o he 3D Wo ld: A Su ey and Me a-Analysis o 3D
Tasks ia Mul i-Modal La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2405.10255 a Xi :2405.10255
[209] Yueen Ma, Zixing Song, Yuzheng Zhuang, Jianye Hao, and I win King. 2024. A Su ey on Vision-Language-Ac ion
Models o Embodied AI. h ps://doi.o g/10.48550/ARXIV.2405.14093 a Xi :2405.14093
[210]
Mohammad A. Makhzan and Hadi Ma dani Kamali. 2024. E olu iona y La ge Language Models o Ha dwa e
Secu i y: A Compa a i e Su ey. h ps://doi.o g/10.48550/ARXIV.2404.16651 a Xi :2404.16651
[211]
Amogh Manneko e. 2024. Towa ds Composi ionally Gene alizable Seman ic Pa sing in La ge Language Models: A
Su ey. h ps://doi.o g/10.48550/ARXIV.2404.13074 a Xi :2404.13074
[212]
Qiheng Mao, Zemin Liu, Chenghao Liu, Zhuo Li, and Jianling Sun. 2024. Ad ancing G aph Rep esen a ion Lea ning
wi h La ge Language Models: A Comp ehensi e Su ey o Techniques. h ps://doi.o g/10.48550/ARXIV.2402.05952
a Xi :2402.05952
[213]
Yu en Mao, Yuhang Ge, Yijiang Fan, Wenyi Xu, Yu Mi, Zhonghao Hu, and Yunjun Gao. 2024. A Su ey on LoRA o
La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2407.11046 a Xi :2407.11046
[214]
Johana Cab e a Medina and Rod igo Rojas And ade. 2024. Ad ancemen s in A i icial In elligence o Heal h: A
Rapid Re iew o AI-based Men al Heal h Technologies Used in he Age o La ge Language Models. In Bioin o ma ics
and Biomedical Enginee ing - 11 h In e na ional Con e ence, IWBBIO 2024, Melone as, G an Cana ia, Spain, July 15-17,
2024, P oceedings, Pa I (Lec u e No es in Compu e Science, Vol. 14848), Ignacio Rojas, F ancisco O uño, Fe nando
Rojas, Luis Ja ie He e a, and Olga Valenzuela (Eds.). Sp inge , Cham, 318–343. h ps://doi.o g/10.1007/978-3-031-
64629-4_26
[215]
G égoi e Mialon, Robe o Dessì, Ma ia Lomeli, Ch is o o os Nalmpan is, Ramakan h Pasunu u, Robe a Raileanu,
Bap is e Roziè e, Timo Schick, Jane Dwi edi-Yu, Asli Celikyilmaz, Edoua d G a e, Yann LeCun, and Thomas Scialom.
2023. Augmen ed Language Models: A Su ey. T ansac ions on Machine Lea ning Resea ch 2023 (2023).
[216]
Xupeng Miao, Gab iele Olia o, Zhihao Zhang, Xinhao Cheng, Hongyi Jin, Tianqi Chen, and Zhihao Jia. 2023.
Towa ds E icien Gene a i e La ge Language Model Se ing: A Su ey om Algo i hms o Sys ems. h ps:
//doi.o g/10.48550/ARXIV.2312.15234 a Xi :2312.15234
[217]
Bonan Min, Hayley Ross, Elio Sulem, Ami Pou an Ben Veyseh, Thien Huu Nguyen, Osca Sainz, Eneko Agi e,
Ilana Hein z, and Dan Ro h. 2024. Recen Ad ances in Na u al Language P ocessing ia La ge P e-T ained Language
Models: A Su ey. Acm Compu ing Su eys 56, 2 (2024), 30:1–30:40. h ps://doi.o g/10.1145/3605943
[218]
She in Minaee, Tomás Mikolo , Na jes Nikzad, Meysam Chenaghlu, Richa d Soche , Xa ie Ama iain, and Jian eng
Gao. 2024. La ge Language Models: A Su ey. h ps://doi.o g/10.48550/ARXIV.2402.06196 a Xi :2402.06196
[219]
A suyuki Miyai, Jingkang Yang, Jingyang Zhang, Yi ei Ming, Yueqian Lin, Qing Yu, Go I ie, Sha iq Jo y, Yixuan Li,
Hai Li, Ziwei Liu, Toshihiko Yamasaki, and Kiyoha u Aizawa. 2024. Gene alized Ou -o -Dis ibu ion De ec ion and
beyond in Vision Language Model E a: A Su ey. h ps://doi.o g/10.48550/ARXIV.2407.21794 a Xi :2407.21794
[220]
And ea Moglia, Kons an inos Geo giou, Pie o Ce e i, Luca T. Maina di, Richa d M. Sa a a, and Al ed Cuschie i.
2024. La ge Language Models in Heal hca e: F om a Sys ema ic Re iew on Medical Examina ions o a Compa a i e
Analysis on Fundamen als o Robo ic Su ge y Online Tes . A i icial In elligence Re iew 57, 9 (2024), 231. h ps:
//doi.o g/10.1007/S10462-024-10849-5
[221]
Philipp Mondo and Ba ba a Plank. 2024. Beyond Accu acy: E alua ing he Reasoning Beha io o La ge Language
Models - A Su ey. h ps://doi.o g/10.48550/ARXIV.2404.01869 a Xi :2404.01869
[222]
Raji Mo a, Sidhika Balachanda , Kenny Peng, Gab iel Agos ini, Nikhil Ga g, and Emma Pie son. 2024. Topics,
Au ho s, and Ins i u ions in La ge Language Model Resea ch: T ends om 17K a Xi Pape s. In P oceedings o he
2024 Con e ence o he No h Ame ican Chap e o he Associa ion o Compu a ional Linguis ics: Human Language
Technologies (Volume 1: Long Pape s), NAACL 2024, Mexico Ci y, Mexico, June 16-21, 2024, Ke in Duh, Helena Gómez-
Ado no, and S e en Be ha d (Eds.). Associa ion o Compu a ional Linguis ics, Mexico Ci y, Mexico, 1223–1243.
h ps://doi.o g/10.18653/V1/2024.NAACL-LONG.67
[223]
Humza Na eed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwa , Muhammad Usman, Nick Ba nes, and
Ajmal Mian. 2023. A Comp ehensi e O e iew o La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2307.06435
a Xi :2307.06435
40 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
[224]
Zabi Al Nazi and Wei Peng. 2024. La ge Language Models in Heal hca e and Medical Domain: A Re iew. h ps:
//doi.o g/10.48550/ARXIV.2401.06775 a Xi :2401.06775
[225]
Se h Neel and Pe e W. Chang. 2023. P i acy Issues in La ge Language Models: A Su ey. h ps://doi.o g/10.48550/
ARXIV.2312.06717 a Xi :2312.06717
[226]
Subhash Ne ella, Sabyasachi Bandyopadhyay, Jiaqing Zhang, Miguel Con e as, Sco Siegel, Aysegul Bumin, B andon
Sil a, Jessica Sena, Benjamin Shickel, Az a Biho ac, Kia Khezeli, and Pa isa Rashidi. 2024. T ans o me s and
La ge Language Models in Heal hca e: A Re iew. A i icial In elligence in Medicine 154 (2024), 102900. h ps:
//doi.o g/10.1016/J.ARTMED.2024.102900
[227]
Thong Nguyen, Yi Bin, Junbin Xiao, Leigang Qu, Yicong Li, Jay Zhangjie Wu, Cong-Duy Nguyen, See-Kiong Ng, and
Anh Tuan Luu. 2024. Video-Language Unde s anding: A Su ey om Model A chi ec u e, Model T aining, and Da a
Pe spec i es. In Findings o he Associa ion o Compu a ional Linguis ics, ACL 2024, Bangkok, Thailand and Vi ual
Mee ing, Augus 11-16, 2024, Lun-Wei Ku, And e Ma ins, and Vi ek S ikuma (Eds.). Associa ion o Compu a ional
Linguis ics, Bangkok, Thailand and i ual mee ing, 3636–3657. h ps://doi.o g/10.18653/ 1/2024. indings-acl.217
[228]
Yuqi Nie, Yaxuan Kong, Xiaowen Dong, John M. Mul ey, H. Vincen Poo , Qingsong Wen, and S e an Zoh en.
2024. A Su ey o La ge Language Models o Financial Applica ions: P og ess, P ospec s and Challenges. h ps:
//doi.o g/10.48550/ARXIV.2406.11903 a Xi :2406.11903
[229]
Sau abh Pahune and Manoj Chand asekha an. 2023. Se e al Ca ego ies o La ge Language Models (Llms): A Sho
Su ey. h ps://doi.o g/10.48550/ARXIV.2307.10188 a Xi :2307.10188
[230]
Medha Pala alli, Amanda Be sch, and Ma hew R. Go mley. 2024. A Taxonomy o Da a Con amina ion in La ge
Language Models. h ps://doi.o g/10.48550/ARXIV.2407.08716 a Xi :2407.08716
[231]
Liangming Pan, Michael Saxon, Wenda Xu, Deepak Na hani, Xinyi Wang, and William Yang Wang. 2024. Au oma -
ically Co ec ing La ge Language Models: Su eying he Landscape o Di e se Au oma ed Co ec ion S a egies.
T ansac ions o he Associa ion o Compu a ional Linguis ics 12 (2024), 484–506. h ps://doi.o g/10.1162/ acl_a_00660
[232]
Seungcheol Pa k, Jaehyeon Choi, Sojin Lee, and U Kang. 2024. A Comp ehensi e Su ey o Comp ession Algo i hms
o Language Models. h ps://doi.o g/10.48550/ARXIV.2401.15347 a Xi :2401.15347
[233]
Ye-Jean Pa k, Abhina Pillai, Jiawen Deng, Eddie Guo, Mehul Gup a, Mike Page , and Ch is ophe Naugle . 2024.
Assessing he Resea ch Landscape and Clinical U ili y o La ge Language Models: A Scoping Re iew. BMC Medical
In o ma ics Decis. Mak. 24, 1 (2024), 72. h ps://doi.o g/10.1186/S12911-024-02459-6
[234]
Sau a Pawa , S. M. Towhidul Islam Tonmoy, S. M. Mehedi Zaman, Vinija Jain, Aman Chadha, and Ami a a Das.
2024. The Wha , Why, and How o Con ex Leng h Ex ension Techniques in La ge Language Models - A De ailed
Su ey. h ps://doi.o g/10.48550/ARXIV.2401.07872 a Xi :2401.07872
[235]
Ji-Lun Peng, Sijia Cheng, Egil Diau, Yung-Yu Shih, Po-Heng Chen, Yen-Ting Lin, and Yun-Nung Chen. 2024. A Su ey
o Use ul LLM E alua ion. h ps://doi.o g/10.48550/ARXIV.2406.00936 a Xi :2406.00936
[236]
Michal Pe elkiewicz and Ra al Poswia a. 2024. A Re iew o he Challenges wi h Massi e Web-Mined Co po a Used
in La ge Language Models P e-T aining. h ps://doi.o g/10.48550/ARXIV.2407.07630 a Xi :2407.07630
[237]
F ancesco Pe i i and S e ano Mon anelli. 2024. Lexical Seman ic Change h ough La ge Language Models: A Su ey.
Acm Compu ing Su eys 56, 11 (2024), 282:1–282:38. h ps://doi.o g/10.1145/3672393
[238]
And eas Pes e , Ahmed Tammaa, Ch is ian Gü l, Alexande S einmau e , and Sami Abou El-Seoud. 2024. Con e sa-
ional Agen s, Vi ual Wo lds, and beyond: A Re iew o La ge Language Models Enabling Imme si e Lea ning. In
IEEE Global Enginee ing Educa ion Con e ence, EDUCON 2024, Kos Island, G eece, May 8-11, 2024. IEEE, Kos Island,
G eece, 1–6. h ps://doi.o g/10.1109/EDUCON60312.2024.10578895
[239]
F ed Philippy, Siwen Guo, and Shoh eh Haddadan. 2023. Towa ds a Common Unde s anding o Con ibu ing Fac o s
o C oss-Lingual T ans e in Mul ilingual Language Models: A Re iew. In P oceedings o he 61s Annual Mee ing
o he Associa ion o Compu a ional Linguis ics (Volume 1: Long Pape s), ACL 2023, To on o, Canada, July 9-14, 2023,
Anna Roge s, Jo dan L. Boyd-G abe , and Naoaki Okazaki (Eds.). Associa ion o Compu a ional Linguis ics, To on o,
Canada, 5877–5891. h ps://doi.o g/10.18653/V1/2023.ACL-LONG.323
[240]
Fa man Ali Pi zado, Awais Ahmed, Román A. Mendoza-U diales, and Hugo Te ashima-Ma ín. 2024. Na iga ing
he Pi alls: Analyzing he Beha io o Llms as a Coding Assis an o Compu e Science S uden s - A Sys ema ic
Re iew o he Li e a u e. IEEE access : p ac ical inno a ions, open solu ions 12 (2024), 112605–112625. h ps:
//doi.o g/10.1109/ACCESS.2024.3443621
[241]
Aske Plaa , Annie Wong, Suzan Ve be ne, Joos B oekens, Niki an S ein, and Thomas Bäck. 2024. Reasoning wi h
La ge Language Models, a Su ey. h ps://doi.o g/10.48550/ARXIV.2407.11511 a Xi :2407.11511
[242]
Shushan a Pudasaini, Luis Mi alles-Pechuán, Da id Lillis, and Ma isa Llo ens-Sal ado . 2024. Su ey on Plagia ism
De ec ion in La ge Language Models: The Impac o Cha GPT and Gemini on Academic In eg i y. h ps://doi.o g/10.
48550/ARXIV.2407.13105 a Xi :2407.13105
[243]
Shuo ei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun
Chen. 2023. Reasoning wi h Language Model P omp ing: A Su ey. In P oceedings o he 61s Annual Mee ing o
La ge Language Models: A Su ey o Su eys 41
he Associa ion o Compu a ional Linguis ics (Volume 1: Long Pape s), ACL 2023, To on o, Canada, July 9-14, 2023,
Anna Roge s, Jo dan L. Boyd-G abe , and Naoaki Okazaki (Eds.). Associa ion o Compu a ional Linguis ics, To on o,
Canada, 5368–5393. h ps://doi.o g/10.18653/V1/2023.ACL-LONG.294
[244]
Libo Qin, Qiguang Chen, Xiachong Feng, Yang Wu, Yongheng Zhang, Yinghui Li, Min Li, Wanxiang Che, and Philip S.
Yu. 2024. La ge Language Models Mee NLP: A Su ey. h ps://doi.o g/10.48550/ARXIV.2405.12819 a Xi :2405.12819
[245]
Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S. Yu. 2024.
Mul ilingual La ge Language Model: A Su ey o Resou ces, Taxonomy and F on ie s. h ps://doi.o g/10.48550/
ARXIV.2404.04925 a Xi :2404.04925
[246]
Weicong Qin and Zhongxiang Sun. 2024. Explo ing he Nexus o La ge Language Models and Legal Sys ems: A Sho
Su ey. h ps://doi.o g/10.48550/ARXIV.2404.00990 a Xi :2404.00990
[247]
Zhen Qin, Daoyuan Chen, Wenhao Zhang, Liuyi Yao, Yilun Huang, Bolin Ding, Yaliang Li, and Shuiguang Deng. 2024.
The Syne gy be ween Da a and Mul i-Modal La ge Language Models: A Su ey om Co-De elopmen Pe spec i e.
h ps://doi.o g/10.48550/ARXIV.2407.08583 a Xi :2407.08583
[248]
XiPeng Qiu, TianXiang Sun, YiGe Xu, YunFan Shao, Ning Dai, and XuanJing Huang. 2020. P e-T ained Models
o Na u al Language P ocessing: A Su ey. Science China Technological Sciences 63, 10 (Oc . 2020), 1872–1897.
h ps://doi.o g/10.1007/s11431-020-1647-3
[249]
Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Ji-Rong Wen. 2024.
Tool Lea ning wi h La ge Language Models: A Su ey. h ps://doi.o g/10.48550/ARXIV.2405.17935 a Xi :2405.17935
[250]
Guanqiao Qu, Qiyuan Chen, Wei Wei, Zheng Lin, Xianhao Chen, and Kaibin Huang. 2024. Mobile Edge In elligence
o La ge Language Models: A Con empo a y Su ey. h ps://doi.o g/10.48550/ARXIV.2407.18921 a Xi :2407.18921
[251]
Youyang Qu. 2024. Fede a ed Lea ning D i en La ge Language Models o Swa m In elligence: A Su ey. h ps:
//doi.o g/10.48550/ARXIV.2406.09831 a Xi :2406.09831
[252]
Md. Abdu Rahman. 2023. A Su ey on Secu i y and P i acy o Mul imodal Llms - Connec ed Heal hca e Pe spec i e.
In IEEE Globecom Wo kshops 2023, Kuala Lumpu , Malaysia, Decembe 4-8, 2023. IEEE, Kuala Lumpu , Malaysia,
1807–1812. h ps://doi.o g/10.1109/GCWKSHPS58843.2023.10465035
[253]
Daking Rai, Yilun Zhou, Shi Feng, Abulhai Sapa o , and Ziyu Yao. 2024. A P ac ical Re iew o Mechanis ic In e -
p e abili y o T ans o me -Based Language Models. h ps://doi.o g/10.48550/ARXIV.2407.02646 a Xi :2407.02646
[254]
Mohaimenul Azam Khan Raiaan, Md. Saddam Hossain Muk a, Kaniz Fa ema, Nu Mohammad Fahad, Sadman Sakib,
Mos Ma u a ul Janna Mim, Jubae Ahmad, Mohammed Eunus Ali, and Sami Azam. 2024. A Re iew on La ge
Language Models: A chi ec u es, Applica ions, Taxonomies, Open Issues and Challenges. IEEE access : p ac ical
inno a ions, open solu ions 12 (2024), 26839–26874. h ps://doi.o g/10.1109/ACCESS.2024.3365742
[255]
Mayk Caldas Ramos, Ch is ophe J. Collison, and And ew D. Whi e. 2024. A Re iew o La ge Language Models and
Au onomous Agen s in Chemis y. h ps://doi.o g/10.48550/ARXIV.2407.01603 a Xi :2407.01603
[256]
Ma hieu Ra au , Bosheng Ding, Fangkai Jiao, Hailin Chen, Xingxuan Li, Ruochen Zhao, Chengwei Qin, Caiming
Xiong, and Sha iq Jo y. 2024. How Much A e Llms Con amina ed? A Comp ehensi e Su ey and he Llmsani ize
Lib a y. h ps://doi.o g/10.48550/ARXIV.2404.00699 a Xi :2404.00699
[257]
Qingyang Ren, Zilin Jiang, Jinghan Cao, Sijia Li, Chiqu Li, Yiyang Liu, Shuning Huo, and Tiange He. 2024. A
Su ey on Fai ness o La ge Language Models in E-Comme ce: P og ess, Applica ion, and Challenge. h ps:
//doi.o g/10.48550/ARXIV.2405.13025 a Xi :2405.13025
[258]
Xubin Ren, Jiabin Tang, Dawei Yin, Ni esh V. Chawla, and Chao Huang. 2024. A Su ey o La ge Language Models
o G aphs. In P oceedings o he 30 h ACM SIGKDD Con e ence on Knowledge Disco e y and Da a Mining, KDD
2024, Ba celona, Spain, Augus 25-29, 2024, Rica do Baeza-Ya es and F ancesco Bonchi (Eds.). ACM, Ba celona Spain,
6616–6626. h ps://doi.o g/10.1145/3637528.3671460
[259]
Da id Res epo, Chenwei Wu, Cons anza Vásquez-Venegas, João Ma os, Jack Galli an , and Luis Filipe. 2024. Analyzing
Di e si y in Heal hca e LLM Resea ch: A Scien ome ic Pe spec i e. h ps://doi.o g/10.48550/ARXIV.2406.13152
a Xi :2406.13152
[260]
Zhya Rzga K. Ros am, Sándo Szénási, and Gábo Ke ész. 2024. Achie ing Peak Pe o mance o La ge Language
Models: A Sys ema ic Re iew. IEEE access : p ac ical inno a ions, open solu ions 12 (2024), 96017–96050. h ps:
//doi.o g/10.1109/ACCESS.2024.3424945
[261]
Paul Rö ge , Fabio Pe nisi, Be ie Vidgen, and Di k Ho y. 2024. Sa e yP omp s: A Sys ema ic Re iew o Open
Da ase s o E alua ing and Imp o ing La ge Language Model Sa e y. h ps://doi.o g/10.48550/ARXIV.2404.05399
a Xi :2404.05399
[262]
Ta a Sa a i and Danai Kou a. 2021. Rela ional Wo ld Knowledge Rep esen a ion in Con ex ual Language Models:
A Re iew. In P oceedings o he 2021 Con e ence on Empi ical Me hods in Na u al Language P ocessing, EMNLP 2021,
Vi ual E en / Pun a Cana, Dominican Republic, 7-11 No embe , 2021, Ma ie-F ancine Moens, Xuanjing Huang, Lucia
Specia, and Sco Wen- au Yih (Eds.). Associa ion o Compu a ional Linguis ics, Online and Pun a Cana, Dominican
Republic, 1053–1067. h ps://doi.o g/10.18653/V1/2021.EMNLP-MAIN.81
48 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
[383]
Paul Yousse , Osman Alpe en Ko as, Meijie Li, Jö g Schlö e e , and Ch is in Sei e . 2023. Gi e Me he Fac s! A Su ey
on Fac ual Knowledge P obing in P e-T ained Language Models. In Findings o he Associa ion o Compu a ional
Linguis ics: EMNLP 2023, Singapo e, Decembe 6-10, 2023, Houda Bouamo , Juan Pino, and Kalika Bali (Eds.). Associa ion
o Compu a ional Linguis ics, Singapo e, 15588–15605. h ps://doi.o g/10.18653/V1/2023.FINDINGS-EMNLP.1043
[384]
Huizi Yu, Lizhou Fan, Lingyao Li, Jiayan Zhou, Zihui Ma, Lu Xian, Wenyue Hua, Sijia He, Mingyu Jin, Yong eng Zhang,
Ash in Gandhi, and Xin Ma. 2024. La ge Language Models in Biomedical and Heal h In o ma ics: A Bibliome ic
Re iew. h ps://doi.o g/10.48550/ARXIV.2403.16303 a Xi :2403.16303
[385]
Mingze Yuan, Peng Bao, Jiajia Yuan, Yunhao Shen, Zi an Chen, Yi Xie, Jie Zhao, Yang Chen, Li Zhang, Lin Shen,
and Bin Dong. 2023. La ge Language Models Illumina e a P og essi e Pa hway o A i icial Heal hca e Assis an : A
Re iew. h ps://doi.o g/10.48550/ARXIV.2311.01918 a Xi :2311.01918
[386]
Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi
Gu, Yong Jae Lee, Yan Yan, Beidi Chen, Guangyu Sun, and Ku Keu ze . 2024. LLM In e ence Un eiled: Su ey and
Roo line Model Insigh s. h ps://doi.o g/10.48550/ARXIV.2402.16363 a Xi :2402.16363
[387]
Munazza Zaib, Quan Z. Sheng, and Wei Emma Zhang. 2020. A Sho Su ey o P e-T ained Language Models
o Con e sa ional AI-A New Age in NLP. In P oceedings o he Aus alasian Compu e Science Week, ACSW 2020,
Melbou ne, VIC, Aus alia, Feb ua y 3-7, 2020, P em P akash Jaya aman, Dimi ios Geo gakopoulos, Timos K. Sellis,
and Abdu Fo kan (Eds.). ACM, Melbou ne VIC Aus alia, 11:1–11:4. h ps://doi.o g/10.1145/3373017.3373028
[388]
Daoguang Zan, Bei Chen, Fengji Zhang, Dianjie Lu, Bingchao Wu, Bei Guan, Yongji Wang, and Jian-Guang Lou. 2023.
La ge Language Models Mee Nl2code: A Su ey. In P oceedings o he 61s Annual Mee ing o he Associa ion o
Compu a ional Linguis ics (Volume 1: Long Pape s), ACL 2023, To on o, Canada, July 9-14, 2023, Anna Roge s, Jo dan L.
Boyd-G abe , and Naoaki Okazaki (Eds.). Associa ion o Compu a ional Linguis ics, To on o, Canada, 7443–7464.
h ps://doi.o g/10.18653/V1/2023.ACL-LONG.411
[389]
Fanlong Zeng, Wensheng Gan, Yongheng Wang, Ning Liu, and Philip S. Yu. 2023. La ge Language Models o Robo ics:
A Su ey. h ps://doi.o g/10.48550/ARXIV.2311.07226 a Xi :2311.07226
[390]
Pai Zeng, Zhenyu Ning, Jie u Zhao, Weihao Cui, Mengwei Xu, Liwei Guo, Xusheng Chen, and Yizhou Shan. 2024.
The CAP P inciple o LLM Se ing: A Su ey o Long-Con ex La ge Language Model Se ing. h ps://doi.o g/10.
48550/ARXIV.2405.11299 a Xi :2405.11299
[391]
Chen Zhang, Zhuo ui Liu, and Dawei Song. 2024. Beyond he Specula i e Game: A Su ey o Specula i e Execu ion
in La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2404.14897 a Xi :2404.14897
[392]
Dingkai Zhang, Huan an Zheng, Wenjing Yue, and Xiaoling Wang. 2024. Ad ancing ITS Applica ions wi h Llms:
A Su ey on T a ic Managemen , T anspo a ion Sa e y, and Au onomous D i ing. In Rough Se s - In e na ional
Join Con e ence, IJCRS 2024, Hali ax, NS, Canada, May 17-20, 2024, P oceedings, Pa II (Lec u e No es in Compu e
Science, Vol. 14840), Mengjun Hu, Ch is Co nelis, Yan Zhang, Pawan Ling as, Dominik Slezak, and JingTao Yao (Eds.).
Sp inge , Cham, 295–309. h ps://doi.o g/10.1007/978-3-031-65668-2_20
[393]
Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2024. A Su ey o Con ollable Tex
Gene a ion Using T ans o me -Based P e-T ained Language Models. Acm Compu ing Su eys 56, 3 (2024), 64:1–64:37.
h ps://doi.o g/10.1145/3617680
[394]
Jie Zhang, Haoyu Bu, Hui Wen, Yu Chen, Lun Li, and Hongsong Zhu. 2024. When Llms Mee Cybe secu i y: A
Sys ema ic Li e a u e Re iew. h ps://doi.o g/10.48550/ARXIV.2405.03644 a Xi :2405.03644
[395]
Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. 2024. Vision-Language Models o Vision Tasks: A Su ey.
IEEE T ansac ions on Pa e n Analysis and Machine In elligence 46, 8 (2024), 5625–5644. h ps://doi.o g/10.1109/TPAMI.
2024.3369699
[396]
Liang Zhang and Zhelun Chen. 2023. Oppo uni ies and Challenges o Applying La ge Language Models in Building
Ene gy E iciency and Deca boniza ion S udies: An Explo a o y O e iew. h ps://doi.o g/10.48550/ARXIV.2312.11701
a Xi :2312.11701
[397]
Lingzhe Zhang, Tong Jia, Mengxi Jia, Yi an Wu, Aiwei Liu, Yong Yang, Zhonghai Wu, Xuming Hu, Philip S. Yu,
and Ying Li. 2024. A Su ey o Aiops o Failu e Managemen in he E a o La ge Language Models. h ps:
//doi.o g/10.48550/ARXIV.2406.11213 a Xi :2406.11213
[398]
Qiang Zhang, Keyang Ding, Tianwen Ly , Xinda Wang, Qingyu Yin, Yiwen Zhang, Jing Yu, Yuhao Wang, Xiao ong
Li, Zhuoyi Xiang, Xiang Zhuang, Zeyuan Wang, Ming Qin, Mengyao Zhang, Jinlu Zhang, Jiyu Cui, Renjun Xu,
Hongyang Chen, Xiaohui Fan, Huabin Xing, and Huajun Chen. 2024. Scien i ic La ge Language Models: A Su ey on
Biological & Chemical Domains. h ps://doi.o g/10.48550/ARXIV.2401.14656 a Xi :2401.14656
[399]
Quanjun Zhang, Chun ong Fang, Yang Xie, Yuxiang Ma, Weisong Sun, Yun Yang, and Zhenyu Chen. 2024. A
Sys ema ic Li e a u e Re iew on La ge Language Models o Au oma ed P og am Repai . h ps://doi.o g/10.48550/
ARXIV.2405.01466 a Xi :2405.01466
[400]
Quanjun Zhang, Chun ong Fang, Yang Xie, Yaxin Zhang, Yun Yang, Weisong Sun, Shengcheng Yu, and Zhenyu Chen.
2023. A Su ey on La ge Language Models o So wa e Enginee ing. h ps://doi.o g/10.48550/ARXIV.2312.15223
La ge Language Models: A Su ey o Su eys 49
a Xi :2312.15223
[401]
Shengyu Zhang, Lin eng Dong, Xiaoya Li, Sen Zhang, Xiao ei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei
Zhang, Fei Wu, and Guoyin Wang. 2023. Ins uc ion Tuning o La ge Language Models: A Su ey. h ps:
//doi.o g/10.48550/ARXIV.2308.10792 a Xi :2308.10792
[402]
Xiyuan Zhang, Ranak Roy Chowdhu y, Rajesh K. Gup a, and Jingbo Shang. 2024. La ge Language Models o Time
Se ies: A Su ey. h ps://doi.o g/10.48550/ARXIV.2402.01801 a Xi :2402.01801
[403]
Xuanliang Zhang, Dingzi ui Wang, Longxu Dou, Qing u Zhu, and Wanxiang Che. 2024. A Su ey o Table Reasoning
wi h La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2402.08259 a Xi :2402.08259
[404]
Yu Zhang, Xiusi Chen, Bowen Jin, Sheng Wang, Shuiwang Ji, Wei Wang, and Jiawei Han. 2024. A Comp ehensi e
Su ey o Scien i ic La ge Language Models and Thei Applica ions in Scien i ic Disco e y. h ps://doi.o g/10.48550/
ARXIV.2406.10833 a Xi :2406.10833
[405]
Yue Zhang, Ya u Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xin ing Huang, Enbo Zhao, Yu Zhang, Yulong
Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, F eda Shi, and Shuming Shi. 2023. Si en’s Song in he AI Ocean: A
Su ey on Hallucina ion in La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2309.01219 a Xi :2309.01219
[406]
Yue Zhang, Ziqiao Ma, Jialu Li, Yanyuan Qiao, Zun Wang, Joyce Chai, Qi Wu, Mohi Bansal, and Pa isa Ko djamshidi.
2024. Vision-and-Language Na iga ion Today and Tomo ow: A Su ey in he E a o Founda ion Models. h ps:
//doi.o g/10.48550/ARXIV.2407.07035 a Xi :2407.07035
[407]
Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Ad ian de Wyn e , Yan Xia, Wenshan Wu, Ting Song, Man
Lan, and Fu u Wei. 2024. LLM as a Mas e mind: A Su ey o S a egic Reasoning wi h La ge Language Models.
h ps://doi.o g/10.48550/ARXIV.2404.01230 a Xi :2404.01230
[408]
Yang Zhang, Yu ei Wang, Kai Wang, Quan Z. Sheng, Lina Yao, Adnan Mahmood, Wei Emma Zhang, and Rongying
Zhao. 2023. When La ge Language Models Mee Ci a ion: A Su ey. h ps://doi.o g/10.48550/ARXIV.2309.09727
a Xi :2309.09727
[409]
Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen.
2024. A Su ey on he Memo y Mechanism o La ge Language Model Based Agen s. h ps://doi.o g/10.48550/ARXIV.
2404.13501 a Xi :2404.13501
[410]
Ziyin Zhang, Chaoyu Chen, Bingchang Liu, Cong Liao, Zi Gong, Hang Yu, Jianguo Li, and Rui Wang. 2023. A Su ey
on Language Models o Code. h ps://doi.o g/10.48550/ARXIV.2311.07989 a Xi :2311.07989
[411]
Zihan Zhang, Meng Fang, Ling Chen, Mohammad-Reza Namazi-Rad, and Jun Wang. 2023. How Do La ge Language
Models Cap u e he E e -Changing Wo ld Knowledge? A Re iew o Recen Ad ances. In P oceedings o he 2023
Con e ence on Empi ical Me hods in Na u al Language P ocessing, EMNLP 2023, Singapo e, Decembe 6-10, 2023,
Houda Bouamo , Juan Pino, and Kalika Bali (Eds.). Associa ion o Compu a ional Linguis ics, Singapo e, 8289–8311.
h ps://doi.o g/10.18653/V1/2023.EMNLP-MAIN.516
[412]
Zijian Zhang, Yujie Sun, Zepu Wang, Yuqi Nie, Xiaobo Ma, Peng Sun, and Ruolin Li. 2024. La ge Language Models
o Mobili y in T anspo a ion Sys ems: A Su ey on Fo ecas ing Tasks. h ps://doi.o g/10.48550/ARXIV.2405.02357
a Xi :2405.02357
[413]
Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and
Mengnan Du. 2024. Explainabili y o La ge Language Models: A Su ey. ACM T ansac ions on In elligen Sys ems
and Technology 15, 2 (2024), 20:1–20:38. h ps://doi.o g/10.1145/3639372
[414]
Pengyu Zhao, Zijian Jin, and Ning Cheng. 2023. An In-Dep h Su ey o La ge Language Model-Based A i icial
In elligence Agen s. h ps://doi.o g/10.48550/ARXIV.2309.14365 a Xi :2309.14365
[415]
Shuai Zhao, Meihuizi Jia, Zhongliang Guo, Leilei Gan, Jie Fu, Yichao Feng, Fengjun Pan, and Luu Anh Tuan.
2024. A Su ey o Backdoo A acks and De enses on La ge Language Models: Implica ions o Secu i y Measu es.
h ps://doi.o g/10.48550/ARXIV.2406.06852 a Xi :2406.06852
[416]
Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji-Rong Wen. 2024. Dense Tex Re ie al Based on P e ained Language
Models: A Su ey. ACM T ans. In . Sys . 42, 4 (2024), 89:1–89:60. h ps://doi.o g/10.1145/3637870
[417]
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang,
Junjie Zhang, Zican Dong, Yi an Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yi an
Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A Su ey o La ge Language Models.
h ps://doi.o g/10.48550/ARXIV.2303.18223 a Xi :2303.18223
[418]
Chaoqi Zhen, Yanlei Shang, Xiangyu Liu, Yi ei Li, Yong Chen, and Dell Zhang. 2022. A Su ey on Knowledge-Enhanced
P e-T ained Language Models. h ps://doi.o g/10.48550/ARXIV.2212.13428 a Xi :2212.13428
[419]
Junhao Zheng, Shengjie Qiu, Chengming Shi, and Qianli Ma. 2024. Towa ds Li elong Lea ning o La ge Language
Models: A Su ey. h ps://doi.o g/10.48550/ARXIV.2406.06391 a Xi :2406.06391
[420]
Yanxin Zheng, Wensheng Gan, Ze eng Chen, Zhenlian Qi, Qian Liang, and Philip S. Yu. 2024. La ge Language Models
o Medicine: A Su ey. h ps://doi.o g/10.48550/ARXIV.2405.13055 a Xi :2405.13055
50 Max Ho , Fe nando Vallecillos-Ruiz, and Leon Moonen
[421]
Zibin Zheng, Kaiwen Ning, Yanlin Wang, Jingwen Zhang, Dewu Zheng, Mingxi Ye, and Jiachi Chen. 2023. A Su ey
o La ge Language Models o Code: E olu ion, Benchma king, and Fu u e T ends. h ps://doi.o g/10.48550/ARXIV.
2311.10372 a Xi :2311.10372
[422]
Hongjian Zhou, Boyang Gu, Xinyu Zou, Yi u Li, Sam S. Chen, Peilin Zhou, Junling Liu, Yining Hua, Cheng eng Mao,
Xian Wu, Zheng Li, and Fenglin Liu. 2023. A Su ey o La ge Language Models in Medicine: P og ess, Applica ion,
and Challenge. h ps://doi.o g/10.48550/ARXIV.2311.05112 a Xi :2311.05112
[423]
Hao Zhou, Chengming Hu, Ye Yuan, Yu ei Cui, Yili Jin, Can Chen, Haolun Wu, Dun Yuan, Li Jiang, Di Wu,
Xue Liu, Cha lie Jianzhong Zhang, Xianbin Wang, and Jiangchuan Liu. 2024. La ge Language Model (LLM)
o Telecommunica ions: A Comp ehensi e Su ey on P inciples, Key Techniques, and Oppo uni ies. h ps:
//doi.o g/10.48550/ARXIV.2405.10825 a Xi :2405.10825
[424]
Pengyuan Zhou, Lin Wang, Zhi Liu, Yanbin Hao, Pan Hui, Sasu Ta koma, and Jussi Kangasha ju. 2024. A Su ey on
Gene a i e AI and LLM o Video Gene a ion, Unde s anding, and S eaming. h ps://doi.o g/10.48550/ARXIV.2404.
16038 a Xi :2404.16038
[425]
Xin Zhou, Sicong Cao, Xiaobing Sun, and Da id Lo. 2024. La ge Language Model o Vulne abili y De ec ion and
Repai : Li e a u e Re iew and he Road Ahead. h ps://doi.o g/10.48550/ARXIV.2404.02525 a Xi :2404.02525
[426]
Yue Zhou, Li ong Feng, Yiping Ke, Xue Jiang, Junchi Yan, Xue Yang, and Wayne Zhang. 2024. Towa ds Vision-Language
Geo-Founda ion Model: A Su ey. h ps://doi.o g/10.48550/ARXIV.2406.09385 a Xi :2406.09385
[427]
Yuxiang Zhou, Jiazheng Li, Yanzheng Xiang, Hanqi Yan, Lin Gui, and Yulan He. 2023. The Mys e y and Fascina ion
o Llms: A Comp ehensi e Su ey on he In e p e a ion and Analysis o Eme gen Abili ies. h ps://doi.o g/10.
48550/ARXIV.2311.00237 a Xi :2311.00237
[428]
Zixuan Zhou, Xue ei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan,
Xiuhong Li, Shengen Yan, Guohao Dai, Xiao-Ping Zhang, Yuhan Dong, and Yu Wang. 2024. A Su ey on E icien
In e ence o La ge Language Models. h ps://doi.o g/10.48550/ARXIV.2404.14294 a Xi :2404.14294
[429]
Hongyan Zhu, Shuai Qin, Min Su, Chengzhi Lin, Anjie Li, and Jun eng Gao. 2024. Ha nessing La ge Vision and
Language Models in Ag icul u e: A Re iew. h ps://doi.o g/10.48550/ARXIV.2407.19679 a Xi :2407.19679
[430]
Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2023. A Su ey on Model Comp ession o La ge Language
Models. h ps://doi.o g/10.48550/ARXIV.2308.07633 a Xi :2308.07633
[431]
Yu ao Zhu, Huaying Yuan, Shu ing Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Zhicheng Dou, and Ji-Rong
Wen. 2023. La ge Language Models o In o ma ion Re ie al: A Su ey. h ps://doi.o g/10.48550/ARXIV.2308.07107
a Xi :2308.07107
[432]
Ziyu Zhuang, Qiguang Chen, Longxuan Ma, Mingda Li, Yi Han, Yushan Qian, Haopeng Bai, Zixian Feng, Weinan
Zhang, and Ting Liu. 2023. Th ough he Lens o Co e Compe ency: Su ey on E alua ion o La ge Language Models.
h ps://doi.o g/10.48550/ARXIV.2308.07902 a Xi :2308.07902
[433]
Shi Zong and Jimmy Lin. 2024. Ca ego ical Syllogisms Re isi ed: A Re iew o he Logical Reasoning Abili ies o Llms
o Analyzing Ca ego ical Syllogism. h ps://doi.o g/10.48550/ARXIV.2406.18762 a Xi :2406.18762
