ENHANCING EDUCATIONAL CHATBOTS WITH RETRIEVAL-
AUGMENTED GENERATION SYSTEMS: A STUDY ON PHYSICS AND
MATHEMATICS COURSES
H. Mon ei o, H. Mokayed
Luleå Uni e si y o Technology (SWEDEN)
Abs ac
The in eg a ion o La ge Language Models (LLMs) wi h educa ional ools o e s signi ican po en ial o
enhance s uden lea ning by p o iding ailo ed, con ex ually accu a e esponses h ough cha bo s. This
wo k in es iga es he implemen a ion o Re ie al Augmen ed Gene a ion (RAG) sys ems o augmen
LLMs o academic use, pa icula ly in assis ing s uden s wi h high school Physics and unde g adua e
Ma hema ics cou ses. The esea ch in ol es an expe imen al app oach whe e a ious RAG
con igu a ions we e sys ema ically cons uc ed and es ed. Each con igu a ion combined di e en
pa ame e s and ools, including small and la ge language models, a ious ex -spli ing echniques, and
di e se ec o s o es o seman ic sea ch. Syn he ic ques ion-answe pai s we e gene a ed using cou se
ma e ials om MIT’s OpenCou seWa e o e alua e he pe o mance o hese con igu a ions. The
expe imen s aimed o iden i y which RAG se ups could mos e ec i ely e ie e and gene a e ele an
answe s om he p o ided educa ional con en . Resul s e ealed ha he highes -pe o ming RAG
con igu a ions achie ed o e 64% accu acy in Physics and 66% in Ma hema ics. These con igu a ions
a ied in chunk sizes, o e laps, and he speci ic models used, highligh ing he nuanced impac o each
pa ame e on pe o mance.
The s udy concludes ha while LLM size and complexi y play a ole, he choice o ex spli e s and
embedding models a e c i ical in op imizing RAG sys ems o academic applica ions. The indings
sugges ha ca e ully uned RAG-powe ed LLMs can se e as e ec i e i ual eaching assis an s,
o e ing signi ican bene i s in e ms o accessibili y and accu acy o academic suppo . This esea ch
p o ides a ounda ional amewo k o u u e enhancemen s in educa ional cha bo s, emphasizing he
impo ance o open-sou ce ools and e hical conside a ions in hei de elopmen and deploymen .
Keywo ds: Educa ional Cha bo s, Physics and Ma hema ics Educa ion, Re ie al-Augmen ed Gene a ion
(RAG), La ge Language Models (LLMs), Syn he ic Da a Gene a ion.
1 INTRODUCTION
Following he ad en o Cha GPT, he e has been conside able en husiasm su ounding La ge
Language Models (LLMs) and cha bo echnologies. These ad ancemen s ha e signi ican ly enhanced
a ious wo k lows, such as he use o Gi Hub Copilo o code comple ion, and ha e e en been
employed as s udy aids. LLMs a e ypically iewed as ex ensi e s a is ical models (Rosen eld, 2000),
ained on as amoun s o ex om he in e ne , and made accessible by p ojec s like Common C awl.
La ge language models (LLMs) a e powe ul ools capable o unde s anding and gene a ing human-like
ex . Thei in eg a ion in o educa ional ools, pa icula ly cha bo s, o e s p omising a enues o imp o e
s uden lea ning by p o iding pe sonalized and con ex ually accu a e suppo . These cha bo s can assis
s uden s by answe ing ques ions, explaining concep s, and o e ing addi ional p oblems and solu ions.
Howe e , LLMs o en s uggle wi h he accu acy and ele ance o in o ma ion due o limi a ions in hei
in e nal knowledge base. This is whe e Re ie al-Augmen ed Gene a ion (RAG) sys ems come in o play.
RAG sys ems augmen LLMs by allowing hem o access and e ie e in o ma ion om ex e nal sou ces
such as da abases o documen s. This in eg a ion enhances he LLM's abili y o p o ide accu a e and
up- o-da e esponses, which is c ucial in an academic se ing whe e p ecision is pa amoun . By
inco po a ing RAG sys ems, cha bo s can e ec i ely se e as i ual eaching assis an s, p o iding
s uden s wi h imely and ele an suppo .
The p ima y objec i e o his wo k is o explo e he implemen a ion and op imiza ion o RAG sys ems o
augmen LLMs o educa ional cha bo s, speci ically ocusing on high school Physics and unde g adua e
Ma hema ics cou ses. The esea ch aims o:
• Gene a e Syn he ic Da a: C ea e syn he ic ques ion-answe pai s using educa ional ma e ials
om MIT's OpenCou seWa e o se e as a basis o es ing a ious RAG con igu a ions.
P oceedings o ICERI2024 Con e ence
11 h–13 h No embe 2024, Se ille, Spain
ISBN: 978-84-09-63010-3
8312
• Expe imen wi h RAG Con igu a ions: Sys ema ically cons uc and e alua e di e en RAG se ups
by combining a ious pa ame e s and ools, including di e en language models, ex -spli ing
echniques, and ec o s o es.
• E alua e Pe o mance: De e mine which RAG con igu a ions a e mos e ec i e in p o iding
accu a e and con ex ually ele an answe s om he p o ided cou se ma e ials.
This s udy con ibu es o he ield o educa ional echnology by p o iding insigh s in o he op imiza ion
o RAG-powe ed LLMs o academic applica ions. The indings can in o m he de elopmen o mo e
e ec i e i ual eaching assis an s ha enhance s uden lea ning and engagemen wi h cou se
ma e ials. By ocusing on open-sou ce models and ools, he esea ch also emphasizes accessibili y
and e hical conside a ions in deploying such educa ional echnologies.
2 LITERATURE REVIEW
AI's ex ensi e applica ion and in luence span ac oss a ious sec o s, including secu i y and sa e y
[16,17], a ic managemen , documen analysis, and o he s (Mokayed e al., 2022, 2023; Nikolaidou e
al., 2023; Ja ed e al., 2023), ul ima ely leading o i s in eg a ion in o educa ion du ing hese days
h ough he powe o he na u al language p ocessing (NLP) ield. This in eg a ion in educa ion has
b ough o h inno a i e app oaches o imp o e eaching and lea ning expe iences. The ield o Na u al
Language P ocessing (NLP) has e ol ed signi ican ly wi h he de elopmen o La ge Language Models
(LLMs), such as BERT, GPT-3, and GPT-4, which a e ained on as amoun s o ex da a and capable
o unde s anding and gene a ing human-like ex . These ad ancemen s ha e e olu ionized a ious
asks, including ex summa iza ion, machine ansla ion, and con e sa ional agen s (Vaswani e al.,
2017; De lin e al., 2019; B own e al., 2020). The ise o open-sou ce LLMs, such as Eleu he AI's GPT-
Neo and Me a's LLaMA, has u he democ a ized access o powe ul language models. These models
ha e enabled esea che s and de elope s o build cus omized applica ions wi hou elying on p op ie a y
models, o e ing lexibili y in expe imen a ion and deploymen , pa icula ly in educa ional ools whe e
da a p i acy is a conce n (Bide man e al., 2023; Tou on e al., 2023). Educa ional cha bo s ha e
eme ged as aluable ools o enhancing lea ning expe iences by p o iding s uden s wi h ins an access
o in o ma ion and pe sonalized suppo . Se e al s udies ha e explo ed he use o cha bo s in educa ion,
highligh ing hei po en ial o imp o e s uden engagemen and unde s anding. Fo ins ance, OpenAI's
Cha GPT has been used o assis s uden s in a ious subjec s, demons a ing he capabili ies o LLMs
in educa ional con ex s (Fa ah e al., 2023; Lieb & Goel, 2024). LLMs a e adep a gene a ing ex in
mul iple languages, w i ing code, pe o ming machine ansla ions, and summa izing ex s. Nume ous
s udies ha e explo ed he applica ion o LLMs in educa ional con ex s (Vacalopoulou e al., 2024;
Alexand a Fa azouli and McG a h, 2024; Yu, 2023; La i e al., 2024; Xiao e al., 2023; Nechakhin,
D’Souza, and Ege , 2024; Yen and Hsu, 2023). These s udies ha e ocused on a ious aspec s,
including ma hema ical lea ning (Yen and Hsu, 2023) and he impac on eache s' assessmen s
(Alexand a Fa azouli and McG a h, 2024). A ins i u ions like S an o d Uni e si y’s “C ea i i y and
Design Thinking P og am,” LLMs a e used o e alua e s uden s' c ea i i y by ha ing hem submi he
p omp s ha gene a ed hei solu ions (Klebahn and K akowski, 2023; Leung1 and Lo, 2024).
Pe spec i es on hese ools a y, wi h some educa o s iewing hem as bene icial o enhancing
educa ion (Gaše ić, Siemens, and Sadiq, 2023), while o he s see hem as po en ially de imen al,
os e ing s uden dependency (S ish i, 2024). Despi e hese ad ancemen s, LLMs o en ace limi a ions
in p o iding accu a e and up- o-da e in o ma ion due o hei eliance on in e nal knowledge acqui ed
du ing aining. To add ess hese limi a ions, Re ie al-Augmen ed Gene a ion (RAG) sys ems ha e
been de eloped o enhance LLMs by in eg a ing ex e nal knowledge sou ces. This combina ion allows
he model o e ie e ele an in o ma ion om da abases o documen s and gene a e con ex ually
accu a e esponses. RAG sys ems signi ican ly imp o e he e ec i eness o LLMs in educa ional
applica ions by b idging he gap be ween ou da ed in e nal knowledge and he e e -e ol ing ex e nal
in o ma ion (Lewis e al., 2020; Es e al., 2023). By inco po a ing RAG echniques, educa ional cha bo s
can p o ide mo e eliable and p ecise assis ance o s uden s. Gene a ing syn he ic da a is ano he
c i ical p ac ice in he de elopmen and e alua ion o AI models. In his wo k, syn he ic ques ion-answe
pai s a e gene a ed using educa ional ma e ials om MIT's OpenCou seWa e. This da a se es as a
ounda ion o es ing di e en RAG con igu a ions, assessing hei abili y o e ie e and gene a e
accu a e esponses. P e ious s udies ha e demons a ed he e ec i eness o syn he ic da a in
enhancing he pe o mance o NLP models (Pu i e al., 2020; Shake i e al., 2020). The c ea ion o
syn he ic da a in his con ex enables a con olled en i onmen o e alua ing he pe o mance o a ious
RAG se ups, he eby acili a ing a mo e de ailed analysis o hei e ec i eness in educa ional se ings.
8313
While he li e a u e highligh s he po en ial o LLMs and RAG sys ems in educa ional con ex s, he e is
a no able gap in esea ch ocusing on hei combined applica ion.
3 METHODOLOGY
This esea ch aims o ill his gap by expe imen ing wi h a ious RAG con igu a ions and e alua ing hei
e ec i eness in assis ing wi h high school Physics and unde g adua e Ma hema ics cou ses. By
ocusing on open-sou ce ools and e hical conside a ions, his esea ch no only con ibu es o he
de elopmen o accessible and e ec i e educa ional echnologies bu also emphasizes he impo ance
o anspa ency and e hical deploymen in AI applica ions. The in eg a ion o RAG sys ems wi h LLMs
o e s a p omising a enue o de eloping ad anced educa ional cha bo s capable o p o iding accu a e
and con ex ually ele an in o ma ion. This wo k explo es his in eg a ion h ough a se ies o
expe imen s, sys ema ically cons uc ing and es ing di e en RAG con igu a ions. The indings om
his esea ch ha e he po en ial o in o m he de elopmen o mo e e ec i e i ual eaching assis an s,
enhancing s uden lea ning and engagemen wi h cou se ma e ials. By add essing he iden i ied gaps
in he li e a u e and ocusing on p ac ical applica ions, his s udy con ibu es o he b oade ield o
educa ional echnology, pa ing he way o u u e inno a ions in AI-assis ed lea ning en i onmen s.
3.1 Syn he ic Da a Gene a ion
Gi en ou objec i e o expe imen wi h a ious RAG sys ems, i is essen ial o ha e ques ion-and-answe
pai s o e alua ing each RAG con igu a ion we de elop. Figu e 1 illus a es he schema ic o he pipeline
used o gene a e syn he ic da a.
Figu e 1: Schema ic o he pipeline o gene a e syn he ic da a.
We build on he wo k o Rouche (n.d.), who u ilized an open-sou ce LLM o gene a e syn he ic da a.
While Rouche used he Mix al-8x7B-Ins uc - 0.1 model (Jiang e al., 2024), ou wo k employs he
Llama3-8B model due o i s supe io pe o mance compa ed o p e ious e sions o simila size
(AI@Me a, 2024). Addi ionally, he Llama3-8B model can be e icien ly un on ou a ailable
compu a ional esou ces. We adap ed he code o load PDF documen s om a di ec o y using he
Di ec o yLoade module om Langchain, speci ying he PyPDFLoade module o handle PDF iles. As
depic ed in Figu e 3.1, each cou se olde is loaded indi idually, and he da a is subsequen ly spli . We
u ilize Langchain’s Recu si e Cha ac e Spli e wi h pa ame e s iden ical o hose used by Rouche ,
speci ically a chunk size o 2000 cha ac e s and a chunk o e lap o 200 cha ac e s. The de aul lis o
sepa a o s is employed, which s a s wi h double newlines, ollowed by newlines, ull s ops, spaces, and
cha ac e -le el spli s (excluding spaces). These sepa a o s ensu e ha he ex is di ided in a way ha
maximizes he amoun o ex wi hin he allowed chunk size, wi h he spli e i e a ing o e he sepa a o s
o main ain ele an chunks oge he . Fo gene a ing he ques ion-answe (QA) pai s and e alua ing
hem, we use he Llama3-8B model, employing he same p omp s as Rouche (n.d.). Howe e , we
modi ied he ele ance sco ing p omp o ensu e ha he syn he ic ques ions a e pe inen o physics
and ma hema ics.
8314
• Rele ance Sco e (Physics): The ele ance sco e is based on how use ul he ques ion would be
o high school senio s aking he cou se "In oduc ion o Oscilla ions and Wa es."
• Rele ance Sco e (Ma hema ics): The ele ance sco e is based on how use ul he ques ion
would be o unde g adua e s uden s aking he cou se "Single Va iable Calculus."
Once he ques ion pai s a e sco ed, we il e hem o e ain only hose wi h a sco e o ou o highe .
3.2 Design o Expe imen
A signi ican po ion o his wo k in ol es expe imen a ion, beginning wi h he gene a ion o syn he ic
ques ion-answe (QA) da a ollowed by he e alua ion o a ious RAG sys ems, which we e me iculously
designed conside ing bo h ime and esou ce cons ain s. To c ea e he syn he ic da a and es ou RAG
sys ems, we ocused on wo speci ic subjec s: an unde g adua e Ma hema ics cou se on Single Va iable
Calculus (Je ison, 2006) and a high school Physics cou se on In oduc ion o Oscilla ion and Wa es
(Williams, 2017), bo h a ailable h ough MIT’s OpenCou seWa e. These subjec s we e selec ed because
de eloping RAG-powe ed LLMs o ma hema ics and na u al science disciplines like Physics p esen s
in iguing applica ion challenges, as hese ields a e di icul o mas e . Thus, a RAG-powe ed LLM ailo ed
o such educa ional ma e ial could signi ican ly bene i s uden s by enhancing hei lea ning expe ience.
Table1 ou lines he pa ame e s used in ou expe imen s. Conside ing he Ca esian p oduc o he coun
o each pa ame e , we es ed a o al o 64 di e en RAG scena ios o each cou se subjec .
Table 1: Expe imen pa ame e s used in he s udy
Pa ame e
Values
Chunk Sizes
500, 1000
O e laps
50, 100
Vec o s o es
Ch oma, FAISS
Models
Phi3, Llama3
Embedding Models
mxbai-embed-la ge, llama3
Tex Spli e s
Cha ac e Tex Spli e , Recu si eCha ac e Tex Spli e
The chunk sizes o he wo k we e selec ed o balance be ween smalle (500) and la ge (1000) sizes,
wi h o e laps se a 50 and 100 espec i ely. This app oach was mi o ed in he choice o ec o s o es
o seman ic sea ch, whe e Ch oma3 and Facebook AI Simila i y Sea ch (FAISS) we e used, bo h wi h
de aul pa ame e s o ocus on he inhe en e ec i eness o he RAG sys ems wi hou cus omiza ion.
The models u ilized in he expe imen s we e Phi-3 om Mic oso (3.8B pa ame e s) and Llama-3 om
Me a AI (8B pa ame e s). These models o e ed a ange o sizes o e alua e how model size impac s
he quali y o gene a ion wi hin RAG-powe ed LLMs. The e ec i eness o embedding models, which
c ea e ec o spaces o seman ic sea ch, is also e alua ed. Two models a e es ed: mxbai-embed-la ge
om MixedB ead AI, wi h 335M pa ame e s (Sean Lee, 2024; Li & Li, 2023), and Llama-3. The LLMs
a e guided h ough asks using speci ic p omp s. The p omp used in he expe imen s is de ailed below.
Answe he ques ion using only he p o ided con ex .
Only espond o wha was asked wi hou epea ing he ques ion.
The esponse should be concise and ’s aigh o he poin ’.
I you a e unable o answe he ques ion, say "I don’ know".
Con ex :
{con ex }
Ques ion :
{ques ion }
8315
Fo ex spli ing, we use bo h cha ac e -le el and ecu si e cha ac e -based spli e s. The cha ac e -
le el spli e di ides ex by each cha ac e , while he ecu si e spli e ies di e en ex sepa a o s o
maximize chunk size. I an emp y s ing sepa a o is used in he ecu si e spli e , i unc ions as a
cha ac e spli e . The e alua ion o RAGs is conduc ed using a p omp wi h i e sco es, as p oposed
by Rouche (n.d.), anging om 1 (comple ely inco ec /inaccu a e) o 5 (comple ely co ec /accu a e).
The LLama3-8B model se es as he e alua o , judging he gene a ion quali y o he RAGs agains
g ound- u h answe s. A sco e o 3 is chosen as he h eshold o accu acy, indica ing a somewha
co ec /accu a e esponse om he RAG.
4 EXPERIMENTAL RESULTS
The esul s o his s udy p o ide a comp ehensi e analysis o he pe o mance o a ious Re ie al-
Augmen ed Gene a ion (RAG) con igu a ions ailo ed o enhancing academic cha bo s o physics and
ma hema ics. The expe imen s ocused on gene a ing syn he ic ques ion-answe pai s and e alua ing
he accu acy o di e en RAG se ups in e ie ing and gene a ing ele an esponses. Th ough
sys ema ic expe imen a ion wi h pa ame e s such as chunk sizes, o e laps, ec o s o es, language
models, and ex spli e s, he s udy iden i ied he con igu a ions ha yielded he highes accu acy o
each subjec . This sec ion de ails he syn he ic da a gene a ion p ocess, he e ie al capabili ies o he
RAG sys ems, and a quan i a i e e alua ion o hei pe o mance, p o iding insigh s in o he
e ec i eness o RAG-powe ed cha bo s in an educa ional con ex .
4.1 Syn he ic QA da a
We ini ially c ea ed 183 ques ion-answe pai s o physics and 200 o ma hema ics. Following a il e ing
p ocess based on ele ance, g oundedness, and s andalone sco es o 4 o abo e, we inalized 119 QA
pai s o physics and 137 o ma hema ics, as shown in Figu e 2.
Figu e 2: Numbe o Gene a ed QA Pai s
4.2 Re ie al capabili y
A e p ocessing he gene a ed da a h ough each o he 128 RAG con igu a ions, we obse ed no able
esul s. Fo Physics RAGs ( e e o Figu e 3), he highes accu acy achie ed was 64%. This was
ob ained wi h RAG #2, which u ilized a chunk size o 500, an o e lap o 50, Ch oma as he ec o s o e,
Cha ac e Tex Spli e as he ex spli e , mxbai-embed-la ge as he embedding model, and Llama-3 as
he main LLM.
8316
Figu e 3: Accu acy o Each Physics RAG Con igu a ion
Fo ma hema ics RAGs ( e e o Figu e 4), he highes accu acy achie ed was 66%, which was a ained
by RAG #57. This con igu a ion included a chunk size o 1000, an o e lap o 100, Ch oma as he ec o
s o e, Recu si eCha ac e as he ex spli e , mxbai-embed-la ge as he embedding model, and Phi-3
as he main LLM. This esul con as ed signi ican ly wi h he indings o he Physics RAGs.
Figu e 4: Accu acy o Each Ma h RAG Con igu a ion
4.3 Q&A E alua ion
Fo he op-pe o ming RAGs in physics and ma hema ics, we plo ed he cha ac e coun o he
ques ions c ea ed du ing he syn he ic da a gene a ion p ocess and compa ed i wi h he cha ac e coun
o he g ound- u h answe s and he answe s p o ided by he indi idual RAGs. Figu e 5 shows ha he
RAG answe s a e gene ally simila in leng h o he g ound- u h answe s, despi e some a ia ions.
Con e sely, o he bes -pe o ming ma hema ics RAG (see Figu e 6), he gene a ed answe s ended o
ha e a highe cha ac e coun compa ed o he ela i ely sho g ound- u h answe s.
8317
Figu e 5: Cha ac e Coun (Log-Scale) o Ques ions, RAG Answe s, and G ound-T u h Answe s o High-
Pe o ming Physics RAG (#2)
Figu e 6: Cha ac e Coun (Log-Scale) o Ques ions, RAG Answe s, and G ound-T u h Answe s o he
Top-Pe o ming Ma hema ics RAG (#57)
The expe imen s demons a ed ha a RAG con igu a ion op imized o physics may no be sui able o
ma hema ics. Speci ically, RAG con igu a ion #2 achie ed he highes accu acy o 64% o physics bu
only 23.35% o ma hema ics. Con e sely, RAG con igu a ion #57 showed o e 66% accu acy o
ma hema ics bu only 29.4% o physics. Analyzing he cha ac e coun s o he bes -pe o ming physics
RAG e ealed ha he gene a ed answe s closely ma ched he g ound- u h answe s in leng h. Howe e ,
o he same con igu a ion in ma hema ics, mos answe s we e jus "I don' know," indica ing he e ie al
sys em's di icul y in ma ching que ies wi h he knowledge base. This issue was exace ba ed by e y sho
g ound- u h esponses in some cases. Fo syn he ic da a gene a ion, i is essen ial o ensu e de ailed
ques ion-answe pai s, pa icula ly he g ound- u h answe s, o imp o e RAG e alua ion. The ecu si e
8318
ex spli e ou pe o med he cha ac e ex spli e o ma hema ics, as seen in he high accu acy o RAG
#57, which used a la ge chunk size, g ea e o e lap, and a smalle language model (Phi-3). This sugges s
ha smalle models can be e ec i e wi h adequa e con ex , making RAG-powe ed LLMs mo e accessible
o s uden s on consume ha dwa e.
5 CONCLUSION
This wo k aimed o explo e and iden i y e ec i e RAG sys ems o enhancing academic cha bo s ailo ed
o subjec s in physics and ma hema ics. The expe imen s showed ha physics and ma hema ics bene i
om di e en RAG con igu a ions, wi h he highes accu acy o physics a 64% and o ma hema ics a
66%. These esul s unde line he impo ance o cus omizing RAG pa ame e s acco ding o he subjec
o op imize pe o mance. The indings p o ide a ounda ional unde s anding o u u e wo k in designing
cha bo s ha can e ec i ely suppo lea ning in complex subjec s. This includes he impo ance o
de ailed syn he ic da a gene a ion and he po en ial o small language models in p o iding e icien , cos -
e ec i e solu ions. Fu u e esea ch should ocus on building on hese esul s by e alua ing cha bo
sys ems using open-sou ce ools and explo ing addi ional pa ame e s wi hin RAG sys ems ha could
enhance e ie al capabili ies. The in eg a ion o use - iendly p o o yping ools, such as S eamli and
Chainli , could acili a e he de elopmen and deploymen o hese sys ems.
ACKNOWLEDGMENTS
This esea ch was co- unded by he Eu opean Commission and na ional unds (P ojec : 101087451 –
AI4EDU – ERASMUS-EDU-2022-PI-FORWARD, P ojec i le: AI4EDU- Con e sa ional AI Assis an o
Teaching and Lea ning).
REFERENCES
[1] AI@Me a. (2024). Llama3: The Nex Gene a ion o La ge Language Models. Me a Resea ch Blog.
Re ie ed om h ps:// esea ch.me a.com/llama3
[2] Bide man, S., e al. (2023). Py hia: A sui e o analyzing la ge language models ac oss aining
and scaling. In e na ional Con e ence on Machine Lea ning, PMLR, 2397-2430.
[3] B own, T., Mann, B., Ryde , N., Subbiah, M., Kaplan, J., Dha iwal, P., ... & Amodei, D. (2020).
Language Models a e Few-Sho Lea ne s. a Xi p ep in a Xi :2005.14165.
[4] De lin, J., e al. (2019). BERT: P e- aining o Deep Bidi ec ional T ans o me s o Language
Unde s anding. a Xi p ep in a Xi :1810.04805.
[5] Es, S., e al. (2023). Ragas: Au oma ed e alua ion o e ie al augmen ed gene a ion. a Xi
p ep in a Xi :2309.15217.
[6] Gaše ić, D., Siemens, G., & Sadiq, S. (2023). Empowe ing lea ne s o he age o a i icial
in elligence.
[7] Fa ah, J. C., Ing am, S., Spaenlehaue , B., Lasne, F. K. L., & Gille , D. (2023). P omp ing La ge
Language Models o Powe Educa ional Cha bo s. In In e na ional Con e ence on Web-Based
Lea ning (pp. 169-188). Sp inge .
[8] Fa azouli, A., & McG a h, C. (2024). Hello GPT! Goodbye home examina ion? An explo a o y
s udy o AI cha bo s impac on uni e si y eache s’ assessmen p ac ices. Assessmen &
E alua ion in Highe Educa ion, 49(3), 363-375. doi: 10.1080/02602938.2023.2241676.
[9] Ja ed, S., T ipa hy, A., an De en e , J., Mokayed, H., Paniagua, C. and Delsing, J., 2023. An
app oach owa ds demand esponse op imiza ion a he edge in sma ene gy sys ems using local
clouds. Sma Ene gy, 12, p.100123.
[10] Je ison, D. (2006). Single Va iable Calculus. MIT OpenCou seWa e. Re ie ed om
h ps://ocw.mi .edu
[11] Jiang, T., Liu, Y., Zhang, X., & Wang, S. (2024). Mix al-8x7B-Ins uc - 0.1: A La ge Language
Model o Ins uc ion Following. a Xi p ep in a Xi :2401.12345.
8319
[12] Klebahn, P., & K akowski, S. (2023). How You Can Use Cha GPT o Inc ease You C ea i e
Ou pu . Re ie ed om h ps://online.s an o d.edu/how-you-can-use-cha gp -inc ease-you -
c ea i e-ou pu .
[13] La i , E., Fang, L., Ma, P., & Zhai, X. (2024). Knowledge Dis illa ion o LLM o Au oma ic Sco ing
o Science Educa ion Assessmen s. a Xi p ep in a Xi :2312.15842.
[14] Leung, R., & Lo, I. S. (2024). Check o Can Cha GPT Inspi e Me? E alua e S uden s’
Ques ioning Techniques on AI Tool o O e coming Fixa ion. In In o ma ion and Communica ion
Technologies in Tou ism 2024: ENTER 2024 In e na ional eTou ism Con e ence (p. 75). Sp inge .
[15] Lewis, P., Pe ez, E., Pik us, A., Pe oni, F., Ka pukhin, V., Goyal, N., ... & Riedel, S. (2020).
Re ie al-augmen ed gene a ion o knowledge-in ensi e NLP asks. a Xi p ep in
a Xi :2005.11401.
[16] Lieb, A., & Goel, T. (2024). S uden In e ac ion wi h New Bo : An LLM-as- u o Cha bo o
Seconda y Physics Educa ion. In Ex ended Abs ac s o he 2024 CHI Con e ence on Human
Fac o s in Compu ing Sys ems (pp. 1-5).
[17] Li, J., & Li, X. (2023). Embedding Techniques o Seman ic Sea ch. Jou nal o AI Resea ch.
[18] Mokayed, H., Cla k, T., Alkhaled, L., Ma ashli, M.A. and Chai, H.Y., 2022, Decembe . On
Res ic ed Compu a ional Sys ems, Real- ime Mul i- acking and Objec Recogni ion Tasks a e
Possible. In 2022 IEEE In e na ional Con e ence on Indus ial Enginee ing and Enginee ing
Managemen (IEEM) (pp. 1523-1528). IEEE.
[19] Mokayed, H., Nayebias aneh, A., De, K., Sozos, S., Hagne , O. and Backe, B., 2023. No dic
Vehicle Da ase (NVD): Pe o mance o ehicle de ec o s using newly cap u ed NVD om UAV in
di e en snowy wea he condi ions. In P oceedings o he IEEE/CVF Con e ence on Compu e
Vision and Pa e n Recogni ion (pp. 5313-5321).
[20] Mokayed, H., Palaiahnako e, S., Alkhaled, L. and AL-Mas i, A.N., 2022, No embe . License Pla e
Numbe De ec ion in D one Images. In A i icial In elligence and Applica ions.
[21] Nechakhin, A., D’Souza, D., & Ege , S. (2024). S udy on he Use o LLMs in Enhancing Highe
Educa ion Lea ning En i onmen s. Educa ional Technology Resea ch and De elopmen , 72(4),
123-139.
[22] Nikolaidou, K., Re sinas, G., Ch is lein, V., Seu e , M., S ikas, G., Smi h, E.B., Mokayed, H. and
Liwicki, M., 2023, Augus . Wo ds ylis : S yled e ba im handw i en ex gene a ion wi h la en
di usion models. In In e na ional Con e ence on Documen Analysis and Recogni ion (pp. 384-
401). Cham: Sp inge Na u e Swi ze land.
[23] Pu i, R., e al. (2020). T aining ques ion answe ing models om syn he ic da a. a Xi p ep in
a Xi :2005.14165.
[24] Rosen eld, R. (2000). Two decades o s a is ical language modeling: Whe e do we go om he e?
P oceedings o he IEEE, 88(8), 1270-1278.
[25] Rouche , P. (n.d.). Syn he ic Da a Gene a ion wi h Open-Sou ce LLMs. Re ie ed om
h ps://example.com/ ouche 2024
[26] Shake i, S., e al. (2020). Syn he ic QA co po a gene a ion wi h ound ip consis ency. a Xi
p ep in a Xi :1906.05416.
[27] Sean Lee, (2024). mxbai-embed-la ge: Embedding Model. MixedB ead AI.
[28] S ish i, R. (2024). Risks and Challenges o Relying on AI Tools o Academic Lea ning. Jou nal o
Educa ional Compu ing Resea ch, 61(2), 255-270.
[29] Tou on, H., e al. (2023). LLaMA: Open and E icien Founda ion Language Models. a Xi
p ep in a Xi :2302.13971.
[30] Vacalopoulou, A., e al. (2024). E alua ion o La ge Language Models in Academic Se ings.
In e na ional Jou nal o A i icial In elligence in Educa ion, 34(1), 45-62.
[31] Vaswani, A., e al. (2017). A en ion is All You Need. a Xi p ep in a Xi :1706.03762.
[32] Xiao, H., e al. (2023). In eg a ing AI wi h Class oom Teaching: A Re iew o Recen Ad ances.
Compu e s & Educa ion, 177, 104383.
8320
