Educational Artificial Intelligent Chatbot: Teacher Assistant & Study Buddy

DEGREE PROJECT
Educa ional A i icial In elligen Cha bo :
Teache Assis an & S udy Buddy
Dimi ios Za is
S e gios Sozos
Mas e P og amme in Da a Science
2023
Luleå Uni e si y o Technology
Depa men o Compu e Science, Elec ical and Space Enginee ing
[This page in en ionally le blank]
1
Luleå Uni e si y o Technology
Educa ional A i icial In elligen Cha bo :
Teache Assis an & S udy Buddy
Membe s:
S e gios Sozos
Dimi ios Za is
Da e: 2023-07-26
2
Abs ac
In he apidly e ol ing landscape o a i icial in elligence, he po en ial o la ge language models (LLMs)
emains a ocal poin o explo a ion, especially in he domain o educa ion. This esea ch del es in o he
capabili ies o AI-enhanced cha bo s, wi h a spo ligh on he "Teache Assis an " & "S udy Buddy"
app oaches. The s udy highligh s he ole o AI in o e ing adap i e lea ning expe iences and pe sonalized
ecommenda ions. As educa ional ins i u ions and pla o ms inc easingly u n o AI-d i en solu ions,
unde s anding he in icacies o how LLMs can be ha nessed o c ea e meaning ul and accu a e educa ional
con en becomes pa amoun .
The esea ch adop s a sys ema ic and mul i- ace ed me hodology. A i s co e, he s udy in es iga es he
in e play be ween p omp cons uc ion, enginee ing echniques, and he esul ing ou pu s o he LLM. Two
p ima y me hodologies a e employed: he applica ion o p omp s uc u ing echniques and he in oduc ion
o ad anced p omp enginee ing me hods. The o me in ol es a p og essi e applica ion o echniques like
pe sona and empla e, aiming o disce n hei indi idual and collec i e impac s on he LLM's ou pu s. The
la e del es in o mo e ad anced echniques, such as he ew-sho p omp and chain-o - hough p omp , o
gauge hei in luence on he quali y and cha ac e is ics o he LLM's esponses. Complemen ing hese is
he "S udy Buddy" app oach, whe e cu icula om domains like biology, ma hema ics, and physics a e
u ilized as ounda ional ma e ials o he expe imen s.
The indings om his esea ch a e poised o ha e signi ican implica ions o he u u e o AI in educa ion.
By o e ing a comp ehensi e unde s anding o he a iables ha in luence an LLM's pe o mance, he s udy
pa es he way o he de elopmen o mo e e ined and e ec i e AI-d i en educa ional ools. As educa o s
and ins i u ions g apple wi h he challenges o mode n educa ion, ools ha can gene a e accu a e, ele an ,
and di e se educa ional con en can be in aluable. This hesis no only con ibu es o he academic
unde s anding o LLMs and p o ides p ac ical insigh s ha can shape he u u e o AI-enhanced educa ion,
bu as educa ion con inues o e ol e, he indings unde sco e he need o ongoing explo a ion and
e inemen o ully le e age AI's bene i s in he educa ional sec o .
3
Table o Con en s
1. In oduc ion ........................................................................................................................................... 6
1.1 Cu en p oblems in Educa ion ..................................................................................................... 7
1.2 Objec i e ....................................................................................................................................... 7
1.3 Thesis ou line ................................................................................................................................ 8
2. Li e a u e Re iew .................................................................................................................................. 9
2.1 E olu ion o Compu e Technology ............................................................................................. 9
2.2 Mul i-Domain Impac o Compu e Technology: A Re iew Ac oss Di e se Sec o s .................. 9
2.3 A i icial In elligence in Educa ion ............................................................................................. 10
2.4 Resea ch Gap .............................................................................................................................. 15
2.5 GPT s Wa son s DialogFlow? ................................................................................................. 16
a. Ad anced Na u al Language Capabili ies ................................................................................... 16
b. E icien Tex Re ie al wi h llama-index .................................................................................. 16
c. Cus omizabili y and Fine-Tuning ............................................................................................... 16
d. De elopmen Flexibili y ............................................................................................................. 16
e. Cos -E ec i eness ...................................................................................................................... 16
2.6 GPT - La ge Language Model .................................................................................................... 17
2.6.1. Limi a ions o GPT-4: ......................................................................................................... 17
2.6.2. E hical P oblems o GPT-4 ................................................................................................. 18
2.7 P omp Enginee ing o Teache Assis an ................................................................................. 19
2.7.1. De ini ion ............................................................................................................................ 19
2.7.2. P omp S uc u ing .............................................................................................................. 19
2.7.3. P omp Enginee ing Techniques ......................................................................................... 22
2.7.3.1 Ze o-Sho P omp ing....................................................................................................... 22
2.7.3.2 Few-Sho P omp ing ....................................................................................................... 22
2.7.3.3 Chain-o -Though P omp ing .......................................................................................... 24
2.7.3.4 O he echniques ............................................................................................................. 24
3. Resea ch App oach ............................................................................................................................. 25
3.1 O e iew ..................................................................................................................................... 25
3.2 Sys em A chi ec u e .................................................................................................................... 25
3.3 Design and Implemen a ion ........................................................................................................ 25
4. S udy Buddy ........................................................................................................................................ 27
4.1 Me hodology ............................................................................................................................... 27
4.2 Resul s ......................................................................................................................................... 29

4
a. Biology ........................................................................................................................................ 29
b. Ma hema ics ................................................................................................................................ 30
c. Physics ........................................................................................................................................ 32
5. Teache Assis an ................................................................................................................................ 35
5.1 Me hodology ............................................................................................................................... 35
5.2 Resul s ......................................................................................................................................... 36
5.2.1 P omp S uc u ing .............................................................................................................. 36
a. Pe sona ........................................................................................................................................ 36
b. Templa e ..................................................................................................................................... 37
5.2.2 P omp Enginee ing ............................................................................................................ 38
a. Few sho ...................................................................................................................................... 38
b. Chain-o - hough ......................................................................................................................... 38
5.2.3 E alua ion: .......................................................................................................................... 39
6. Discussion ........................................................................................................................................... 40
6.1 E hics & Legal Issues on AI in educa ion ................................................................................... 40
6.2 Jailb eaks and o ensi e con en by LLMs ................................................................................. 41
6.3 The E hical Implica ions o AI O e use in Educa ional Se ings ............................................... 41
a. Limi a ion on Usage .................................................................................................................... 41
b. Moni o ing and Feedback ........................................................................................................... 41
c. Educa ional Guidelines ............................................................................................................... 42
d. E hical Design ............................................................................................................................. 42
e. Ongoing Assessmen ................................................................................................................... 42
7. Conclusion .......................................................................................................................................... 43
Re e ences ................................................................................................................................................... 44
Acknowledgemen s ..................................................................................................................................... 47
Appendix A ................................................................................................................................................. 48
Biology .................................................................................................................................................... 48
Ma hema ics ............................................................................................................................................ 49
Physics .................................................................................................................................................... 50
Appendix B ................................................................................................................................................. 54
a. Baseline answe .............................................................................................................................. 54
a. P omp S uc u ing .......................................................................................................................... 55
i. Pe sona ........................................................................................................................................ 55
ii. Templa e ..................................................................................................................................... 55
5
P omp Enginee ing ................................................................................................................................ 57
i. Few sho ...................................................................................................................................... 57
ii. Chain-o - hough ......................................................................................................................... 57
Lis o Figu es
FIGURE 1: ILLUSTRATION OF THE APPLICATION USER INTERFACE ................................................................................................. 26
FIGURE 2. STUDY BUDDY METHODOLOGY STEPS ..................................................................................................................... 27
FIGURE 3. OVERALL PERFORMANCE SCORE - STUDY BUDDY ...................................................................................................... 34
FIGURE 4. TEACHER ASSISTANT METHODOLOGY STEPS ............................................................................................................ 35
Lis o Tables
TABLE 1. SUMMARY OF PAPERS ON AI IN EDUCATION ............................................................................................................. 15
TABLE 2. BIOLOGY - EVALUATION OF CRITERIA ....................................................................................................................... 30
TABLE 3. MATHEMATICS - EVALUATION OF CRITERIA ............................................................................................................... 32
TABLE 4. PHYSICS - EVALUATION OF CRITERIA ........................................................................................................................ 34
TABLE 5. BIOLOGY - ANSWER COMPARISON ........................................................................................................................... 48
TABLE 6. MATHEMATICS - ANSWER COMPARISON .................................................................................................................. 49
TABLE 7. PHYSICS - ANSWER COMPARISON ............................................................................................................................ 53
TABLE 8. BASELINE QUESTION RESULTS ................................................................................................................................ 54
TABLE 9. PROMPT STRUCTURING - PERSONA RESULTS ............................................................................................................. 55
TABLE 10. PROMPT STRUCTURING - TEMPLATE RESULTS .......................................................................................................... 56
TABLE 11. PROMPT ENGINEERING - FEW SHOT RESULTS .......................................................................................................... 57
TABLE 12. PROMPT ENGINEERING - CHAIN-OF-THOUGHT RESULTS ............................................................................................ 58
6
1. In oduc ion
A i icial In elligence (AI) is he abili y o machines o pe o m complex asks ypically equi ing human
in elligence. These asks can include p oblem-sol ing, planning, unde s anding na u al language,
ecognizing pa e ns, lea ning, and he lis keeps g owing.
The apid e olu ion o A i icial In elligence esea ch con inually p esen s new solu ions o complex
p oblems and signi ican imp o emen s o he quali y o human li e. One no ewo hy example is he s udy
conduc ed by Mokayed e al., (2023), whe e he pe o mance o ehicle de ec o s unde a ied snowy
wea he condi ions was analyzed using da a cap u ed by Unmanned Ae ial Vehicles (UAVs). The esea ch
p o ides a comp ehensi e accoun o da a p epa a ion p ocedu es and in oduces a mul i- ea u e
decon olu ional Fas e R-CNN model o accu a e ehicle de ec ion in ae ial image y, u ilizing he No dic
Vehicle Da ase (NVD) o expe imen al alida ion. Howe e , ad ancemen s in he ield o medical
imaging a e p esen ed in he wo k by Voon e al., (2022), whe e hey assessed he e ec i eness o se en
dis inc Con olu ional Neu al Ne wo ks (CNNs) in g ading In asi e Duc al Ca cinoma (IDC) in b eas
his opa hological images using ans e lea ning. This esea ch no only highligh s he p ac ical applica ion
o CNNs o IDC g ade classi ica ion bu also p esen s a compa a i e pe o mance analysis o he se en
CNN models agains he image dis ibu ion da a om B eaKHis and he BCG da ase . Las ly, add essing
he challenge o sca ce da a in documen image classi ica ion, Kanchi e al., (2022) p opose an e icien
mul imodal me hod ha combines ex ual s eams wi h dynamic wo d embedding and hie a chical a en ion
ne wo ks. Thei inno a i e app oach, encompassing echniques such as adap i e h esholding and
mul imodal side- uning, yields highe o e all accu acy compa ed o ea lie models. This con luence o
esea ch illus a es he b oad impac and po en ial o AI ac oss a ied domains and condi ions, Also, AI can
be used in di e en applica ions like Secu i y and su eillance (Mokayed e al., 2022), in elligen
anspo a ion sys em (Khan e al., 2022), and nume ous o he ields. This s udy speci ically concen a es
on he use o AI on he educa ion. Educa ion is he mos c ucial elemen in os e ing a p ospe ous socie y.
A h i ing communi y depends on a solid educa ional ounda ion o p omo e he g ow h and well-being o
i s membe s. The de elopmen o educa ion has been cha ac e ized by ongoing ad ancemen s and
b eak h oughs, leading o a weal h o knowledge and possibili ies o all. Mo ing o wa d, i is essen ial o
p io i ize he c ea ion o well- ounded educa ional p og ams, build inclusi e lea ning spaces, and encou age
c i ical hinking. By laying a s ong educa ional g oundwo k, we enable indi iduals o make meaning ul
con ibu ions o socie y, nu u e a sense o global esponsibili y, and pa e he way o endu ing p og ess
o e e yone.
The ime is ipe o AI o make i s way in o schools and begin deli e ing signi ican bene i s. One such
example is IBM Wa son, a supe compu e -powe ed AI, se ing as an adminis a i e assis an o s uden s a
Deakin Uni e si y. This inno a i e use o AI exempli ies he po en ial o in eg a ing ad anced echnology
in academia, o e ing a dynamic, in e ac i e lea ning expe ience while s eamlining adminis a i e asks
(Boa eng e al., 2022). Wa son can p o ide se ice o he s uden s a any ime. This esul s in a di e en
s uc u e wi hin he adminis a i e powe o he uni e si y wi h ewe pe sonnel. Now he s uden s ge an
immedia e answe depending on hei p o ile.
In ecen yea s, he educa ional sys ems ha e in eg a ed compu e s and o he o ms o echnology in hei
eaching me hodologies. This ansi ion has led o he widesp ead use o de ices such as able s and lap ops
in class ooms, enhancing he lea ning expe ience o s uden s. Teache s a e now inco po a ing a ious
digi al ools and pla o ms o acili a e collabo a i e lea ning. The implemen a ion o echnologies like
in e ac i e whi eboa ds, lea ning managemen sys ems, and online esou ces has no only made lessons
mo e engaging bu also expanded he bounda ies o he adi ional class ooms, connec ing s uden s and
7
educa o s ac oss he globe. As a esul , s uden s a e acqui ing aluable digi al skills ha p epa e hem o
success in an inc easingly echnology-d i en wo ld.
1.1 Cu en p oblems in Educa ion
Despi e he in eg a ion o compu e ized echnology in o he educa ional sys ems, he e emain se e al
challenges o be add essed in o de o uly e olu ionize educa ion. Some o hese issues include:
1. Uni o mi y in educa ion: Cu en educa ional sys ems o en ollow a one-size- i s-all app oach, ailing
o ca e o he indi idual s eng hs and weaknesses o each s uden . This can hinde s uden s om
ealizing hei ull po en ial and ecei ing he a ge ed suppo hey need o excel.
2. Slow eedback: Teache s may some imes p o ide eedback slowly, esul ing in delayed no i ica ions
o s uden s who need ex a ime o s udy. This can lead o missed oppo uni ies o imp o emen and
a lack o imely suppo o hose s uggling wi h speci ic concep s.
3. O e bu dened eache s: Teache s a e o en s e ched hin, managing la ge class sizes and an ex ensi e
wo kload, which can a ec hei abili y o p o ide indi idualized a en ion and suppo o s uden s.
1.2 Objec i e
In his s udy we aim o explo e he easibili y o implemen ing ad anced AI echnologies o add ess he
challenges in educa ion om bo h he pe spec i es o eache s and s uden s. We p opose he concep o a
eache assis an and a s udy buddy, powe ed by s a e-o - he-a AI echniques. By le e aging hese
in elligen sys ems, we aspi e o o e e ec i e solu ions ha suppo educa o s in hei ins uc ional asks
and empowe s uden s in hei lea ning jou ney. This s udy is ca ied ou om wo unique pe spec i es.
One is om a eache 's iewpoin , whe e he AI unc ions as a suppo i e eaching assis an . The o he is
om a s uden 's iewpoin , whe e he AI se es as a help ul s udy companion.
1. Teache Assis an : In ou in es iga ion, we will ocus on he po en ial o he AI o gene a e exam
subjec s ailo ed o he cu iculum being u ilized by he eache . By u ilizing hese ad anced
echnologies, we aim o p o ide educa o s wi h a aluable ool ha can assis in he c ea ion o ele an
and a ge ed exam ma e ials.
2. S uden Buddy: As pa o ou esea ch, ou objec i e is o assess i s abili y o p o ide accu a e
answe s o ques ions di ec ly ela ed o speci ic cu iculum. By employing hese sophis ica ed AI
echnologies, we endea o o c ea e a s udy companion ha can o e s uden s aluable assis ance in
hei lea ning jou ney, ensu ing hey ha e access o p omp and eliable in o ma ion.
Upon conduc ing esea ch, i is e iden ha GPT cu en ly s ands a he o e on o AI and la ge language
models. Howe e , i p o okes se e al in iguing ques ions. Could i s le el o in elligence se e e ec i ely
as a eaching assis an , c ea ing exam ques ions akin o hose cons uc ed by human educa o s? Is i
su icien ly sophis ica ed o p o ide accu a e and meaning ul esponses o s uden s' que ies? Wha is he
ideal s uc u ing o ques ions o maximize he accu acy o esponses and he e ec i eness o exam
ques ionnai es as gene a ed by he AI model?
14
A i icial
In elligence (AI)
S uden
Assis an s in he
Class oom:
Designing
Cha bo s o
Suppo S uden
Success
1. Teach basic
con en &
answe basic
ques ions
2. Encou age
explo a ion o
addi ional
esou ces
3. Assis wi h
ca ee ‑ and
li e‑ ela ed
issues
4. 24/7
esponsi eness
5. Assis
s uden s h ough
con e sa ions
which leads o
engagemen
6. Shy s uden s
can ask any
ques ion eely
1. Losing he low
o he dialogue
2. Lacking he
capaci y o
emo ional
comp ehension in
s uden s
3. No able o
unde s and open-
ended answe s
4. Inabili y o
eac in eal ime
o s uden
commen s and
ques ions
1. Mul iple-
choice quizzes
2. Answe ing
equen ly asked
ques ions
3. Answe ing
basic cou se and
con en ques ions
4. Abili y o
encou age u he
explo a ion o
addi ional
esou ces
5. Language
lea ning
1. By quiz esul s
2. Objec i e da a,
es sco es, and
s uden opinion o
hei lea ning
expe ience
Educa ion 4.0:
A i icial
In elligence
Assis ed Task-
and Time
Planning Sys em
1. Au oma ically
gene a ed asks
om he AI
Engines
2. Secu e da a
exchange ia he
Open API
Se ice
3. Abili y o
moni o and
iew lea ning
p og ess and
success
4. Gami ica ion
o wo king
h ough cou ses
5. Conside a ion
o di e en
lea ning ypes,
s eng hs, and
weaknesses
6. No i ica ion
messages o ale
s uden s o
lea ning a ea s
1. Lack o
pe sonaliza ion
and cus omiza ion
o indi idual
s uden s
2. Po en ial o
da a secu i y and
p i acy issues
3. Di icul y in
unde s anding
complex asks and
ins uc ions
4. Di icul y in
p o iding accu a e
and imely
eedback,
no i ica ions and
eminde s, and
assessmen asks
1. Tailo ed
lea ning con en
2. Reminde s and
no i ica ions
abou asks and
lea ning p og ess
3. P o ide
assessmen asks
4. Coo dina e
au oma ically and
manually
gene a ed asks
5. Moni o and
iew lea ning
p og ess and
success
1. Usabili y
2. Use expe ience
3. Da a p o ec ion
4. Da a secu i y
5. Communica ion
be ween laye s
Kwame o
Science: An AI
Teaching
Assis an o
1. P o iding
ins an answe s
o science
ques ions
1. Challenging
cases when he e
a e ypos in he
1. P o iding
ins an answe s o
Science ques ions
o s uden s
1. Top 1 accu acy
quan i y
pe o mance
assuming only one

15
Science
Educa ion in
Wes A ica
2. Abili y o
ine- une he
SBERT model
o imp o e
accu acy
3. Making
Kwame o
Science
a ailable in local
languages ac oss
A ica
4. Making i
a ailable ia
o line channels
such as SMS,
USSD, and oll-
ee calling
spelling o
scien i ic wo ds
2. Ques ions
ela ed o opics
ou side he scope
o he knowledge
sou ce
3. Unhelp ul
answe s due o
issues wi h he
da ase
answe was e u ned
and o ed on
2. Top 3 accu acy
e e s o he
pe o mance whe e
o each ques ion
ha ecei ed a o e,
a leas one answe
was a ed as help ul
ou o he 3 answe s
ha we e e u ned
Table 1. Summa y o Pape s on AI in Educa ion
2.4 Resea ch Gap
Despi e he g owing body o esea ch on he use o con e sa ional AI cha bo s in educa ion, many s udies
o da e ha e ocused p ima ily on he cha bo s' abili ies o answe ques ions based on speci ic cu iculums
wi hou he p o ision o associa ed ex books (Chen e al., 2022; Pe ei a, 2016; Benede o & C emonesi,
2019). These models a e ypically p e- ained on speci ic educa ional ma e ials and a e hus limi ed in hei
lexibili y and adap abili y o ca e o di e en lea ning ma e ials and di e se educa ional needs.
One no ewo hy excep ion is he s udy by Boa eng e al. (2022), which p oposed a sys em ha allowed
use s o p o ide hei own ex books o he AI o gene a e esponses. None heless, he sys em equi ed
u he aining on he p o ided ma e ials o deli e accu a e esponses, a ime-consuming p ocess ha
could po en ially hinde i s wide-scale applica ion.
Despi e i s pionee ing e o s, Boa eng e al.'s (2022) s udy illumina es a signi ican esea ch gap: he lack
o cha bo applica ions capable o gene a ing accu a e esponses om use -p o ided ex books wi hou he
need o addi ional aining. I also unde sco es he need o he de elopmen o mo e lexible, adap able,
and use - iendly educa ional cha bo s.
The p esen s udy aims o add ess his esea ch gap by using he GPT, a p e- ained La ge Language Model
(LLM), in conjunc ion wi h he llama-index lib a y, which has he capaci y o ead and index documen s
wi hou he need o addi ional aining. This app oach no only elimina es he ime and esou ces equi ed
o aining bu also enhances he lexibili y and adap abili y o cha bo s o ca e o a wide ange o
educa ional ma e ials and use equi emen s.
16
2.5 GPT s Wa son s DialogFlow?
In he de elopmen o an educa ional AI cha bo aimed a se ing as a eache assis an and s udy buddy,
he choice o unde lying echnology plays a pi o al ole. Th ee p ima y pla o ms we e conside ed:
Google's Dialog low, IBM's Wa son, and OpenAI's GPT coupled wi h he llama-index lib a y. A e
ho ough e alua ion, he combina ion o GPT and llama-index was selec ed o he ollowing easons:
a. Ad anced Na u al Language Capabili ies
GPT model excel in unde s anding and gene a ing human-like, con ex - ele an ex . This ea u e is c ucial
o a cha bo ha needs o in e p e complex educa ional que ies and espond in an in ui i e manne . While
Wa son and Dialog low o e obus Na u al Language Unde s anding (NLU), GPT's capabili ies a e mo e
ad anced, especially when ine- uned o a speci ic domain like educa ion.
b. E icien Tex Re ie al wi h llama-index
One o he unique equi emen s o his p ojec was he abili y o si h ough ex books o p o ide accu a e
answe s. Llama-index complemen s GPT by e icien ly indexing la ge olumes o ex , making he e ie al
o ele an sec ions as and accu a e. This ills a gap ha would o he wise equi e a sepa a e ool like
Elas icsea ch, he eby s eamlining he echnology s ack.
c. Cus omizabili y and Fine-Tuning
GPT models can be ine- uned o specialize in speci ic asks o domains. This le el o cus omizabili y
allows o be e alignmen wi h he educa ional ocus o he cha bo . While Wa son does o e specialized
models, he ine- uning capabili ies o GPT a e mo e ex ensi e, o e ing ine con ol o e he model's
beha io .
d. De elopmen Flexibili y
The in eg a ion o GPT wi h llama-index can be achie ed wi h Py hon, which o e s a ich se o lib a ies
and ools o machine lea ning, and NLP.
e. Cos -E ec i eness
While i is ue ha GPT API calls can be expensi e, he combined u ili y o GPT and llama-index educes
he need o mul iple se ices ha could incu addi ional cos s, as would be he case wi h Wa son o
Dialog low. Addi ionally, he ine- uning capabili ies o GPT can esul in mo e accu a e and e icien
que ying, po en ially educing he numbe o equi ed API calls.
17
In summa y, he syne gy be ween GPT's ad anced na u al language capabili ies and llama-index's e icien
ex e ie al p esen s a compelling case o hei selec ion. This combina ion o e s a obus , cus omizable,
and e icien solu ion o de eloping an educa ional AI cha bo ha can e ec i ely se e as bo h a eache
assis an and a s udy buddy.
2.6 GPT - La ge Language Model
In his sec ion we will gi e an o e iew o GPT and La ge Language Models.
The Gene a i e P e-T ained T ans o me (GPT) is an inno a i e Na u al Language P ocessing (NLP)
model, spea headed by he es eemed esea ch ins i u e OpenAI. This sophis ica ed deep lea ning model
le e ages he powe o he ans o me a chi ec u e, enabling i o p oduce ex eminiscen o human-like
seman ics and syn ax. T ained on an ex ensi e co pus o ex , GPT epi omizes wha a e o en e e ed o as
La ge Language Models (LLMs), exhibi ing a ema kable abili y o gene a e con ex ually pe inen ex
ela i e o he inpu p o ided. This model's e sa ili y allows i o be p o icien ly employed o a di e se
a ay o asks ha include, bu a e no limi ed o, ex summa iza ion, ques ion answe ing, and ex
gene a ion.
E en hough GPT-4 which is he la es e sion o GPT model a he ime o s udy, is eally good in
answe ing ques ions and making discussions on di e en subjec s, i is no pe ec and in many cases i s
answe s a e no accu a e o e en wo se inapp op ia e. Acco ding o OpenAI, (2023), he e a e no only
limi a ions bu some e y c i ical e hical issues ha we should conside when using GPT-4 model.
2.6.1. Limi a ions o GPT-4:
a. Hallucina ions: GPT-4 is p one o hallucina ions, a e m e e ing o si ua ions whe e he model
p o ides alse o made-up in o ma ion wi hou con ex . This migh include he p esen a ion o
ic i ious da a as ac ual, o he con iden asse ion o alse eali ies. These hallucina ions unde mine
he eliabili y o he model, po en ially misleading use s o comp omising he in eg i y o ou pu s.
b. P oducing biased and un eliable con en : GPT-4 has been obse ed gene a ing con en ha is biased
o un eliable. This includes po en ially ha m ul con en such as ha e speech, inci emen s o iolence,
and alse na a i es, which can exploi indi iduals and cause ha m. I can also gene a e con en
p omo ing sel -ha m, g aphic ma e ial, and ins uc ions o illegal ac i i ies, u he emphasizing he
need o esponsible use and moni o ing.
c. Finding websi es selling illegal goods o se ices: A limi a ion o GPT-4 includes i s capaci y o
acili a e illegal ac i i ies. This can in ol e he iden i ica ion and p omo ion o illici web pla o ms
like da k ma ke s, websi es selling coun e ei o s olen goods, o se ices like hacking o money
launde ing. Such capabili y poses a se ious e hical dilemma in he use o his echnology.
d. Planning a acks: An ala ming limi a ion lies in GPT-4's abili y o plan ha m ul ac ions, such as
conduc ing a phishing a ack o e en se ing up an open-sou ce language model on a new se e o
malicious ac i i ies. I can iden i y key ulne abili ies, hide aces, and le e age human esou ce
pla o ms o i s asks, hus posing a h ea o digi al secu i y.
18
e. Gene a ing belie able and pe suasi e con en : GPT-4 has he po en ial o gene a e con en ha is
con incing and pe suasi e. This capabili y can be misused o c ea e poli ically cha ged con en ,
ealis ic bu misleading na a i es, o o al e he discou se a ound ce ain opics, he eby sp eading
misin o ma ion and manipula ing public opinion.
. Lack o knowledge o pos - aining e en s: GPT-4's aining da a is limi ed o he poin o p e-
aining da a cu -o . The e o e, i lacks knowledge o subsequen e en s, changes in laws, egula ions,
social no ms, ad ances in echnology, o new disco e ies in a ious ields, esul ing in ou da ed o
po en ially i ele an esponses.
g. Simple easoning e o s: GPT-4 can o en make simple easoning e o s, such as s e eo yping
demog aphic g oups o making assump ions based on limi ed in o ma ion. This could lead o
inaccu a e, biased, o disc imina o y ou pu s.
h. Gullibili y owa ds alse s a emen s: GPT-4 can display a deg ee o gullibili y, o en accep ing
clea ly alse s a emen s wi hou any e idence. This gullibili y ex ends o conspi acy heo ies, alse
poli ical p omises, and un ounded claims om a ious sou ces, which can p opaga e misin o ma ion
and alsehoods.
i. Failing a ha d p oblems: GPT-4 may unde pe o m when aced wi h complex p oblems. Fo
ins ance, accu a ely p edic ing e y low pass a es, e e sing he end o dec easing pe o mance as
a unc ion o scale, and unde pe o ming on seemingly easy asks a e among he limi a ions ha may
lead o ine ec i e o misleading ou comes.
j. Negligence in double-checking wo k: GPT-4 does no inhe en ly double-check i s ou pu , leading o
a po en ial sp ead o misin o ma ion. Dependence on he model's ou pu wi hou e i ica ion and
igno ing he model's e usals and hedging cues can lead o inaccu acies and e o s.
2.6.2. E hical P oblems o GPT-4
a. P i acy conce ns: AI sys ems like GPT-4 could po en ially ein o ce ideologies, and e en p opaga e
alse in o ma ion, which aises conce ns abou p i acy and he au onomy o hough . Fu he mo e,
hese sys ems can also cause dispa i ies in se ice quali y due o pe o mance di e ences ac oss
demog aphics and asks.
b. Da a secu i y isks: GPT-4 has he po en ial o exace ba e da a secu i y isks. Fo example, i could
assis in iden i ying indi iduals when combined wi h ou side da a, acili a e social enginee ing o
cybe a acks, p o ide guidance o ha m ul ac i i ies, o e en enhance exis ing secu i y h ea s.
c. Po en ial ha ms o ulne able popula ions: GPT-4 can unin en ionally cause ha m o ulne able
g oups. This could be h ough exposu e o haza dous ma e ials o en i onmen s, disc imina ion,
unsa e wo king condi ions, subs anda d heal hca e, iolence o abuse, cybe bullying, scams, aud, o
he sp ead o alse o misleading in o ma ion.
d. Da a b eaches: GPT-4 may indi ec ly con ibu e o da a b eaches, such as unau ho ized access o
sensi i e da a, malicious a acks on da a sys ems, o da a leakage due o weak secu i y measu es,
causing signi ican ha m and aising e hical issues.
e. Misuse o pe sonal in o ma ion: Misuse o pe sonal in o ma ion is a se ious e hical conce n. GPT-
4 can be used o ein o ce ideologies, in o m esou ce alloca ion, o p o ide legal o heal h ad ice,
po en ially leading o misuse o exploi a ion o pe sonal da a.
. Algo i hmic bias: GPT-4 can exhibi biases which can esul in unequal se ices ac oss di e en
demog aphics o domains. These biases can ein o ce ideologies, and no ms, and can pa icula ly
19
a ec g oups ha a e unde ep esen ed in he aining da a, u he aising conce ns abou ai ness and
equi y.
2.7 P omp Enginee ing o Teache Assis an
This chap e aims o p o ide a comp ehensi e o e iew o p omp enginee ing as a echnique ha can be
used o c ea e e ec i e assessmen s o s uden s.
Fi s ly, an explana ion o p omp enginee ing will be p esen ed, ou lining he main concep s and p inciples
unde lying his app oach. By de ining he key aspec s o p omp enginee ing, eade s will be able o gain a
clea unde s anding o how i can be applied in he con ex o au oma ed assessmen design and c ea ion.
Then, he ocus o his chap e will be on demons a ing and summa izing echniques ha ha e been
documen ed in esea ch pape s. These echniques ha e been p o en o ha e di e en e ec i eness on he
ou pu , ega dless o he con ex o he p omp .
Finally, by analyzing hese app oaches, aken by o he esea che s in his a ea, we will be able o iden i y
bes p ac ices, and we will demons a e how p omp enginee ing can be applied o c ea e e ec i e
assessmen s o s uden s.
This chap e will se e as a ounda ion o he subsequen chap e o his hesis “Teache Assis an ” Th ough
his wo k, we hope o con ibu e o he ongoing de elopmen o using cha bo s o c ea ing e ec i e
assessmen s a egies ha can suppo s uden lea ning and e alua ion.
2.7.1. De ini ion
P omp enginee ing is abou how he s uc u e and he con en o a p omp can signi ican ly a ec he esul
o he model. Since he p omp de ines he ask, selec ing he igh p omp has a signi ican impac on bo h
he accu acy and he ini ial ask ha he model does (Liu e al., 2021). To p o ide wi h a mo e echnical
de ini ion in he con ex o cha bo s, we can use Whi e (e al) de ini ion, which de ines p omp as a se o
di ec ions gi en o an LLM, which enables he cus omiza ion, enhancemen , o e inemen o he LLM's
capabili ies (Whi e e al., 2023).
The use o p omp s can signi ican ly in luence he ou comes o in e ac ions wi h a La ge Language Model
(LLM), as hey p o ide speci ic ules and guidelines o con e sa ions. A p omp help o es ablish he
con ex o he con e sa ion, highligh ing he mos ele an in o ma ion and indica ing he desi ed o m and
con en o he ou pu . Fo ins ance, a p omp can speci y ha an LLM should gene a e code ha adhe es o
a pa icula coding s yle (Whi e e al., 2023).
2.7.2. P omp S uc u ing
Resea ch has been conduc ed o p o ide a eusable solu ion o p omp s. The ca alog o p omp pa e ns
seeks o o e eusable answe s o issues use s encoun e when wo king wi h LLMs o comple e a a ie y
o ac i i ies (Whi e e al., 2023). O he esea ches ha e al eady p o en he e iciency o p omp ing in
di e en a eas, such as o designing p omp s o be used in asks o classi ica ion (Wang e al., 2023) o

20
o designing boolean p omp s ha can be used in li e a u e que ies (Xu e al., 2022). This solu ion ha is
p oposed in Whi e e al pape is mo e gene al, so ha i can be used o ecu ing p oblems wi hin a pa icula
con ex , which in ou case is he assessmen design and c ea ion.
This chap e is aimed o p o ide an o e iew o he c ucial aspec s ha equi e conside a ion when
cons uc ing an e icien p omp . We will ocus on speci ic a eas ha we belie e can ha e a signi ican
in luence on he e icacy o he p omp o he pu pose o assessmen c ea ion. Then, in he o hcoming
chap e en i led "Teache Assis an ," we will employ he amewo k es ablished in his chap e o e alua e
i s e ec i eness. Addi ionally, we will compa e he ou comes gene a ed by his s uc u e o hose o a
simpli ied p omp ha de ia es om he p oposed amewo k.
The sys em ha has been p oposed by Whi e e al. o compiling and using a lis o p omp pa e ns o
Cha GPT and o he la ge language models (LLMs) consis s o he ollowing aspec s (Whi e e al., 2023):
 Inpu Seman ics
 Ou pu Cus omiza ion
 E o Iden i ica ion
 P omp Imp o emen
 In e ac ion
 Con ex Con ol
The key ac o in ou case is he ou pu cus omiza ion, which is abou limi ing o modi ying he ypes,
o ma s, s uc u es, o o he elemen s o he ou pu p oduced by he LLM (Whi e e al., 2023). We need o
in es iga e how a p omp can a ec he ou pu , so ha he assessmen s ha a e p oduced by a cha bo mee
he eache ’s expec a ions. The pa e ns ha a e p oposed o he ou pu cus omiza ion a e he ollowing:
1. Ou pu Au oma e , wi h which he use can w i e sc ip s o au oma e any asks ha he LLM ou pu
p oposes him o do. This is no ele an o ou esea ch abou assessmen c ea ion (Whi e e al., 2023).
2. Pe sona, wi h which he use can gi e o LLM a ole o embody when i p o ides an answe . Pe sona
pa e n can be explained by wo key elemen s, he In en & Con ex o i , and i s Mo i a ion (Whi e e
al., 2023).
In e ms o he in en ion and con ex , i en ails gi ing he LLM a ce ain poin o iew o pe spec i e
o ake when p oducing p oduc s. This iewpoin may ake he shape o a "pe sona" ha di ec s he
LLM in de e mining which speci ics o emphasize. Fo ins ance, use s migh ask he LLM o analyze
he code om he s andpoin o a secu i y specialis . Wi h he use o his me hod, he LLM can p o ide
ou pu s ha a e in line wi h he ask's s a ed pu pose and desi ed esul s (Whi e e al., 2023).
Rega ding he pa e n's mo i a ion, i acknowledges ha use s migh no be awa e o he p ecise ou pu s
o in o ma ion needed o comple e a ask. They migh , howe e , be amilia wi h he posi ion o kind
o pe son who equen ly o e s suppo in hese a eas. Use s can exp ess hei desi es wi hou ha ing
o be as speci ic as possible abou he needed ou comes by gi ing he LLM a pe sona. Wi h he use o
his me hod, he LLM can p o ide esul s ha mee he needs o he in ended audience (Whi e e al.,
2023).
O e all, he Pe sona pa e n is a po en ool ha enables use s o cus omize he ou pu s o hei pa icula
needs, which can inc ease he e ec i eness o LLMs. Use s can ensu e ha he ou pu s a e in line wi h
21
he ask's in ended pu pose and planned ou comes by gi ing he LLM wi h a pe sona wi hou needing
o be awa e o he p ecise in o ma ion o ou pu s equi ed (Whi e e al., 2023).
Finally, i is p oposed o s uc u e his pa e n by including he ollowing ph ase in he p omp :
“Ac as pe sona X” o “P o ide ou pu s ha pe sona X would c ea e” (Whi e e al., 2023).
3. Visualiza ion Gene a o , wi h which he use can make he LLM o p oduce ex ual ou pu s ha may
be ed o o he ools, such o he AI-based pic u e gene a o s enabling he use o c ea e isualiza ions.
This is no ele an o ou esea ch abou assessmen c ea ion (Whi e e al., 2023).
4. Recipe, wi h which he use is able o ecei e a se o s eps o ac ions ha will help hem achie e a
speci ied end goal, e en i hey a e wo king wi h unknown o limi ed knowledge. This is no ele an
o ou esea ch abou assessmen c ea ion (Whi e e al., 2023).
5. Templa e, wi h which he use can designa e an ou pu empla e, which he LLM will hen ill up wi h
i s ou pu . This pa e n can, also, be explained by he wo key elemen s, which a e he In en & Con ex
o i , and i s Mo i a ion (Whi e e al., 2023).
Rega ding he In en and Con ex , o make su e ha an LLM's ou pu ollows a ce ain empla e
s uc u e elemen o he Templa e pa e n ha i was c ea ed. When cus ome s ask he LLM o gene a e
ou pu s in a o ma ha is no commonly u ilized o he in o ma ion being gene a ed, his is especially
help ul. Fo ins ance, a use could need o c ea e a URL ha places c ea ed da a a pa icula poin s
along he URL pa h. Use s can di ec he LLM o p oduce ou pu ha ollows he de ined empla e by
u ilizing he Templa e pa e n, gua an eeing ha he inal ou pu is p ope ly o ganized as equi ed
(Whi e e al., 2023).
Mo i a ion is he second componen o explaining he Templa e pa e n. This componen
acknowledges ha occasionally ou pu needs o be c ea ed in a p ecise o ma ha is pa icula o a
gi en applica ion o use-case. Use s mus speci y he o ma and he loca ions o a ious sec ions o
he ou pu because he LLM is unawa e o he necessa y empla e s uc u e. This could be done by
c ea ing a sample da a s uc u e o by illing ou a se ies o o m le e s. Use s can make su e ha he
ou pu p oduced by he LLM accu a ely adhe es o he de ined empla e s uc u e by employing he
Templa e pa e n, ensu ing ha he ou pu mee s he necessa y o ma and quali y c i e ia (Whi e e al.,
2023).
In gene al, he Templa e pa e n is a use ul ool o di ec ing he ou pu p oduced by an LLM. Use s
can gua an ee ha he ou pu mee s he necessa y o ma and quali y c i e ia by gi ing he LLM a
speci ied empla e s uc u e, e en i he LLM isn' gene ally used o c ea ing ou pu in ha manne . The
Templa e pa e n is an e ec i e me hod o making su e ha LLM ou pu is p ecisely o ma ed as
needed, in a way ha sui s he needs o use s and s akeholde s (Whi e e al., 2023).
Finally, i is p oposed o s uc u e his pa e n by including he ollowing ph ase in he p omp :
“I am going o p o ide a empla e o you ou pu ” o “X is my placeholde o con en ” o “T y o i
he ou pu in o one o mo e o he placeholde s ha I lis ” o “Please p ese e he o ma ing and o e all
empla e ha I p o ide” o “This is he empla e: PATTERN wi h PLACEHOLDERS” (Whi e e al.,
2023).
22
2.7.3. P omp Enginee ing Techniques
2.7.3.1 Ze o-Sho P omp ing
Mode n La ge Language Models (LLMs), such as GPT-3, ha e he capabili y o pe o m ce ain asks "ze o-
sho " due o hei abili y o ollow ins uc ions and ex ensi e aining on as amoun s o da a. Such models
ha e ze o-sho capabili ies. Fo example, in a classi ica ion-asking p omp such as he ollowing:
“Classi y his sen ence as posi i e o nega i e: I am e y happy wi h his p oduc ”
We go he ollowing answe using Cha Gp :
“The sen ence is posi i e.”
This means ha such models can comp ehend and classi y a sen imen om a ex wi hou any examples
being gi en wi h hei classi ica ions in he p omp . And his is wha is e e ed o as ze o sho p omp ing.
This can be ex ended o any pu pose; in ou case i can be on he educa ion con ex .
Ze o-sho p omp ing is a p omising app oach o gene a ing ex wi h li le o no supe ision. Howe e , i
is no always e ec i e o ad anced o non-gene ic ques ions. In such cases, p o iding demons a ions o
examples in he p omp can imp o e he pe o mance o he model, leading o wha is known as ew-sho
p omp ing. In he nex sec ion, we will demons a e how his app oach can be u ilized o imp o e he quali y
o ex gene a ed by he model.
2.7.3.2 Few-Sho P omp ing
Lea ning asks by p o iding a ew examples, o ew-sho lea ning, is a signi ican academic challenge.
Mode n NLP does ew-sho lea ning by e o mula ing asks as "p omp s" in na u al language and illing in
hose p omp s wi h ained language models (Logan IV e al., 2022).
Recen esea ch on ad anced p e- ained language models, including GPT-3, Ins uc GPT, and Cha GPT
2, indica es ha hese models can e ec i ely pe o m a a ie y o asks wi hou he need o adjus hei
pa ame e s. Ins ead, hey can u ilize jus a small se o examples as ins uc ions. This demons a es he
imp essi e capabili ies o LLMs on a la ge scale (Wei e al., 2023).
Few-sho p omp ing is also e e ed as “in-con ex ” lea ning. As men ioned abo e, he in-con ex lea ning
allows he model o pe o m many asks wi hou adjus ing he pa ame e s o he model. The e ec i eness
o his echnique has been also measu ed and been compa ed o he ze o-sho p omp ing. The esul s showed
ha in a b oad a ay o asks, he ew-sho p omp ing is consis en ly p o ed o be mo e e ec i e han he
ze o-sho one (Zhao e al., 2021).
Howe e , i needs o be no ed a his poin ha he academic esea ch abou his echnique is limi ed. I is
no ye ully explained nei he how i exac ly wo ks, no which pa o he ew-sho p omp is mos e ec i e
o he bes ou pu (Min e al., 2022). Thus, i is known ha gi ing some inpu -label pai s be o e he ac ual
p omp is con ibu ing o a be e ou pu (B own e al., 2020), bu i is no knowing which pa o hese
inpu -label pai s ac ually a ec he end pe o mance o he ou pu .
An example o how ew-sho p omp ing can be implemen ed and how i can be be e han ze o-sho is
p esen ed below:
Le ’s assume abou a medical diagnosis ask. This is a ask whe e ew-sho lea ning is likely o
ou pe o m ze o-sho lea ning due o he speci ici y and complexi y o he ask.
23
He e's a se o symp oms we will be using in ou p omp :
"The pa ien is expe iencing sho ness o b ea h, swelling in he legs, and a igue."
 Ze o-Sho P omp ing:
We would eed he language model (LM) his symp om and ask i o diagnose wi hou any speci ic
aining o p io examples o his ask.
Inpu :
"The pa ien is expe iencing sho ness o b ea h, swelling in he legs, and a igue. Wha is you
diagnosis?"
Ou pu :
"Based on hese symp oms, he pa ien could be su e ing om a numbe o condi ions. I could be
a espi a o y issue, a ca dio ascula p oblem, o e en a side e ec o ce ain medica ions. I 's
essen ial o seek p o essional medical ad ice o a p ope diagnosis."
The model p o ides a ague answe ha doesn' speci y a single diagnosis because i doesn' ha e
he con ex o make ha kind o judgemen .
 Few-Sho P omp ing:
In ew-sho p omp ing, we p o ide he LM wi h a ew examples o he ask be o e p esen ing i
wi h he es case.
Inpu :
"Example 1: The pa ien is expe iencing ches pain, sho ness o b ea h, and nausea. The diagnosis
o hese symp oms is hea a ack.
Example 2: The pa ien has equen u ina ion, excessi e hi s , and unexplained weigh loss. The
diagnosis o hese symp oms is diabe es.
Example 3: The pa ien expe iences join pain, s i ness, and dec eased ange o mo ion. The
diagnosis o hese symp oms is a h i is.
The pa ien is expe iencing sho ness o b ea h, swelling in he legs, and a igue. Wha is you
diagnosis?"
Ou pu :
"Based on hese symp oms, he pa ien could po en ially ha e conges i e hea ailu e. Howe e , i
is s ill c ucial o consul wi h a heal hca e p o essional o an accu a e diagnosis."
Compa ing he 2 echniques:
In he ze o-sho p omp ing, he model ga e a b oad and ague answe because i didn' ha e a speci ic
con ex o example o base i s answe on. This demons a es he limi a ions o he ze o-sho app oach o
ce ain speci ic asks, especially in a eas ha equi e expe knowledge like medical diagnosis.
In con as , ew-sho p omp ing wi h p io examples allowed he model o pe o m be e , iden i ying he
symp oms as po en ially indica i e o conges i e hea ailu e, which aligns mo e accu a ely wi h he
p esen ed symp oms.
30
No e abou ques ion 6: S udy Buddy (AI) did no unde s and he ques ion, because p obably i couldn’ ind
any co ela ion o he wo d “ ole” in he ex book a ound he exis ence o DNA, hence he answe was
w ong saying ha he ole o he DNA is no men ioned in he con ex .
#
Ques ion
C i e ion 1
(Comp ehensi e
Con en )
C i e ion 2
(In o ma ion
Consis ency)
C i e ion 3
(Rele an
Con en )
Sco e
1
Who coined he
e m cell, in
e e ence o he
iny s uc u es seen
in li ing
o ganisms?

(Teache
men ioned: "The
i s ime he wo d
cell was used o
e e o hese iny
uni s o li e was in
1665.")






0.66
2
Who iden i ied
animalcules? Wha
a e animalcules?



1
3
Wha a e he h ee
main pa s o he
cell heo y?



1
4
Lis he ou pa s
common o all
cells.



1
5
Wha a e he cell
s uc u es whe e
p o eins a e made?



1
6
Wha is he ole o
DNA?

(no answe )


0
Table 2. Biology - E alua ion o C i e ia
To al sco e o Biology is: 4.6 / 6 = 0.76
b. Ma hema ics
The ques ion s a ed in he ex book is as ollows: “Find he pa e n ules o he ollowing nume ical
pa e ns.”, ollowed by a lis o numbe s’ sequences, see column “Ques ion asked” in Table 2.
S udy Buddy did no in e p e he ques ion p ope ly and i couldn’ gi e any answe , hen we ied di e en
e sions o he ques ion in o de o make i mo e sui able o S udy Buddy o unde s and.

31
The ques ion we ga e o S udy Buddy is as ollows: “Can you ollow he book's heo y in o de o ind he
pa e n ha applies o he ollowing numbe s' sequence: 1, 6, 21, 66. And do i . Bu gi e me he inal answe
wi hou explaining he s eps. Be o e you answe make su e ha you answe is co ec by applying he
calcula ions o he numbe sequence I ga e you.”
By obse ing he co ec and w ong answe s gi en by S udy Buddy, we see ha i gi es he w ong answe
when he pa e n ule consis s o mo e han one calcula ion. This is jus an obse a ion o he es s we ha e
un and i may no be he ule o o he examples.
#
Ques ion
( Sequence o
Numbe s)
C i e ion 1: Solu ion
Accu acy
C i e ion 2: Rele an
Con en
Sco e
1
1, 6, 21, 66


Only con ains in o ma ion
ele an o he cou sebook.
0.5
2
95, 80, 65, 50


Only con ains in o ma ion
ele an o he cou sebook.
1
3
3, 10, 17, 24


Only con ains in o ma ion
ele an o he cou sebook.
1
4
256, 64, 16, 4


Only con ains in o ma ion
ele an o he cou sebook.
1
5
3, 11, 43, 171


Only con ains in o ma ion
ele an o he cou sebook.
0.5
6
81, 27, 9, 3


Only con ains in o ma ion
ele an o he cou sebook.
1
7
4, 13, 40, 121


Only con ains in o ma ion
ele an o he cou sebook.
0.5
8
1, 6, 31, 156

0.5
32

Only con ains in o ma ion
ele an o he cou sebook.
Table 3. Ma hema ics - E alua ion o C i e ia
To al sco e o Ma hema ics is: 6 / 8 = 0.75
c. Physics
F om Table 6 in Appendix A, we can see ha he answe s om S udy Buddy a e e y simila o he ones
gi en by he eache , bu he e a e no ewo hy di e ences, bo h in some o he esul s, bu also in he way
he solu ions a e explained and p esen ed:
Ques ion 1. De e mining he ime a which you ha e he same speed as you b o he .
S udy Buddy and he eache bo h ag ee on he ime a which you each he same speed as you b o he .
Bo h gi e he answe as 1 second.
Ques ion 2. Calcula ing he maximum heigh a ball eaches when launched upwa ds.
In his scena io, bo h S udy Buddy and he eache use he same app oach - using he p inciple o
conse a ion o ene gy. Howe e , he AI p o ides a mo e s ep-by-s ep, ma hema ical solu ion, while he
eache 's answe is concise and assumes ha he s uden is amilia wi h he ele an concep s.
Ques ion 3. Finding he eloci y o he ball jus be o e i hi s he op o a building.
The AI calcula es he eloci y as app oxima ely 15.99 m/s, whe eas he eache calcula es i as
app oxima ely 16 m/s. This disc epancy is mino and could be due o ounding o he speci ics o how he
calcula ions we e ca ied ou .
Ques ion 4. Finding ou how long a ball is in he ai om launch ill i hi s he g ound.
He e we see a signi ican disc epancy. S udy Buddy calcula es ha he ball is in he ai o app oxima ely
14.27 seconds, while he eache calcula es a o al ime o abou 8.77 seconds. The answe om S udy
Buddy is w ong, because i assumes ha he ime going up is equal o he ime alling.
Ques ion 5. Iden i ying he equi ed accele a ion o inc ease speed o e a speci ic dis ance.
S udy Buddy calcula es he accele a ion 1.78 m/s2, while he eache answe s wi h 2 m/s2. This is a
signi ican disc epancy and could be due o ei he ounding o an e o in he calcula ions.
Ques ion 6. De e mining he accele a ion o a ock d opped om a cli .
S udy Buddy answe s co ec ly ha he accele a ion due o g a i y is cons an o 9.8 m/s2.
Ques ion 7. Finding he heigh o a cli om which a ock is d opped.
S udy Buddy calcula es he heigh o he cli o app oxima ely 60.725 me e s, while he eache gi es he
heigh o 60 me e s. This di e ence is ela i ely mino and likely due o ounding.
O e all, i appea s ha he S udy Buddy is mo e inclined owa ds de ailed ma hema ical explana ions, while
he eache ’s answe uses a mix o ma h and concep ual unde s anding.
33
#
Ques ion
C i e ion 1:
Solu ion
Accu acy
C i e ion 2:
Me hodological
Alignmen
C i e ion 3:
Rele an
Con en
Sco e
1
Bike p oblem:
A wha ime
do you ha e he
same speed as
you b o he ?

S udy Buddy
gi es he
co ec answe
o be 1 second.

The eache solu ion is
gi en by obse ing a
diag am, whe e S udy
Buddy is ollowing he
heo y’s equa ions o ind
he answe .

Only con ains
in o ma ion
ele an o he
cou sebook.
0.66
2
Building
p oblem: How
high does he
ball go?

S udy Buddy
gi es he
co ec answe
o be
app oxima ely
250m.

The me hod used by S udy
Buddy, does no align wi h
he eache 's me hod.

Only con ains
in o ma ion
ele an o he
cou sebook.
0.66
3
Building
p oblem: Wha
is he eloci y
o he ball igh
be o e i hi s
he op o he
building?

S udy Buddy
gi es he
co ec answe
o be
app oxima ely
16 m/s.

The S udy Buddy's solu ion
aligns wi h he me hod used
in he eache ’s solu ion.

Only con ains
in o ma ion
ele an o he
cou sebook.
1
4
Building
p oblem: Fo
how many
seconds is he
ball in he ai
om he
momen i was
launched ill i
hi s back o he
g ound?

The answe
om S udy
Buddy is
14.27s which is
w ong. The
co ec answe
is 8.77s.

The me hod used by he
S udy Buddy does no align
wi h he eache 's me hod.
S udy Buddy w ongly
assumes ha he alling
ime is equal o he ime
going up.

Only con ains
in o ma ion
ele an o he
cou sebook.
0.33
5
Speed p oblem:
Wha
accele a ion
should you use
o inc ease
you speed
om 10 m/s o
18 m/s o e a
dis ance o 55
m?

The answe
om S udy
Buddy is 1.78
m/s² which is
w ong. The
co ec answe
is 2 m/s².

Bo h solu ions use he
co ec kinema ic equa ion.

Only con ains
in o ma ion
ele an o he
cou sebook.
0.66
34
6
Cli p oblem:
Wha is he
magni ude o
he accele a ion
o he ock a
he momen i
is d opped?

S udy Buddy
gi es he
co ec answe
9.8 m/s2.

Bo h solu ions use he
concep o g a i a ional
accele a ion.

Only con ains
in o ma ion
ele an o he
cou sebook.
1
7
Cli p oblem:
Wha is he
heigh o he
cli ?

S udy Buddy
gi es he
co ec answe
o be
app oxima ely
60m.

Bo h solu ions use he
co ec kinema ic equa ion.

Only con ains
in o ma ion
ele an o he
cou sebook.
1
Table 4. Physics - E alua ion o C i e ia
The o al sco e o Physics is: 5.31 / 7 = 0.76
Figu e 3. O e all Pe o mance Sco e - S udy Buddy
F om he abo e g aph we see ha he pe o mance o he S udy Buddy is good enough and wi h some
weaking (p omp enginee ing) we can po en ially make i e en be e .
35
5. Teache Assis an
5.1 Me hodology
This chap e b eaks down he s eps we ook o ca y ou ou esea ch. We used a i e-s ep p ocess ha
in ol es wo king wi h a La ge Language Model (LLM) o make a mul iple-choice es based on Biology
book “CK-12 Biology o High School” om chap e “4.1 Cen al Dogma”. We' e pa icula ly in e es ed
in seeing how di e en ways o c ea ing and weaking p omp s can change he way he LLM esponds.
We ollowed 5 s eps o un he Teache Assis an examples.
Figu e 4. Teache Assis an Me hodology S eps
Following a e de ails o each s ep:
1. Selec ion o he Chap e : The i s s ep in ou me hodology is o choose a speci ic chap e om he
book ha se es as he ounda ion o ou es . Fo he pu pose o his s udy, he chap e “2.1 Pa s o
he Cell” which ocuses on a pa icula heo y om he book “CK-12 Biology o High School” has
been selec ed.
2. C ea ion o he Baseline P omp (Ze o-P omp ): Following he selec ion o he chap e , we p oceed
o de elop a simple p omp ha guides he LLM o o mula e a ‘mul iple-choice’, ‘ ill in he gaps’,
‘ ue- alse’, ‘ma ch he ollowing’, and ‘open-ended’ ques ions es based on he selec ed heo y. This
p omp is in ended o ac as a e e ence poin , enabling compa isons wi h he ou comes o subsequen
s eps. The simplici y o he ini ial p omp is delibe a ely main ained o disce n he e ec s o la e
modi ica ions clea ly.

36
3. Applica ion o P omp S uc u ing Techniques: The hi d s age o ou me hodology in ol es
applying p omp s uc u ing echniques p og essi ely. Techniques such as he pe sona and he empla e
a e in eg a ed in o he ini ial p omp one by one, as discussed in he li e a u e e iew. We aim o obse e
how hese modi ica ions al e he LLM's ou pu s, hus o e ing insigh in o he indi idual and collec i e
impac s o he s uc u ing echniques.
4. Employmen o P omp Enginee ing Techniques: The inal phase in ol es in oducing p omp
enginee ing echniques, moni o ing he changes in he LLM's esponses ollowing each change o
echnique. Techniques like ew-sho p omp and chain-o - hough p omp a e used o e alua e hei
in luence on he LLM's ou pu quali y and cha ac e is ics. By sys ema ically and independen ly
applying each echnique, we can disce n he p ecise impac o hese changes on he LLM's gene a ed
con en .
5. Es ablishing E alua ion C i e ia: The e alua ion will be in compa ison o he ‘baseline’ answe . We
ha e de ined he ollowing e alua ion c i e ia:
E alua ion C i e ia:
C i e ion 1: F om how many concep s a e he ques ions aken?
We check he di e si y o he answe om he book.
C i e ion 2: Rele an Con en
The con en o he answe should always be ele an o he con en o he book.
In summa y, he me hodology ou lined abo e o e s a sys ema ic app oach o assessing he po en ial o a
la ge language model in he gene a ion o knowledge-based es s. By explo ing he in e ac ions be ween
p omp cons uc ion, enginee ing echniques, and he LLM's ou pu s, we es ablish a comp ehensi e
unde s anding o he a iables in luencing he LLM's pe o mance. In he nex chap e we will p esen he
esul s de i ed om his me hodological app oach.
5.2 Resul s
The i s p omp es ed was a simple ques ion like i was e e ed o a human eache . The p omp is he
ollowing:
“Gene a e 3 mul iple-choice ques ions wi h 4 possible answe s each ega ding molecula biology. The
ques ions should es unde s anding o he undamen al p inciples o he opic.”
See Table 8. Baseline Ques ion Resul s o he esul s.
5.2.1 P omp S uc u ing
a. Pe sona
Inco po a ing he pe sona o a eache in an LLM's p omp can enhance he c ea ion o exam ques ions due
o wo main ac o s: con ex -in en ion and mo i a ion. Con ex ually, he eache pe sona enables he LLM
o ocus on educa ional p inciples and goals ypically associa ed wi h exam design, such as app op ia e
di icul y and con en ele ance. In e ms o mo i a ion, use s who may no know he speci ics needed o
e ec i e exam ques ions can bene i om he LLM embodying a eache 's pe sona, as i inhe en ly cap u es
37
aspec s like ai ness, comp ehensi eness, and di e si y o ques ion o ma s. Hence, by assigning he
pe sona o a eache o he LLM, use s can ob ain ailo ed ou pu s ha align wi h educa ional s anda ds and
mee hei needs, despi e po en ial lack o speci ic inpu de ails.
See Table 9. P omp S uc u ing - Pe sona Resul s o he esul s o his echnique.
E alua ion o Teache Assis an answe s gi en using he “pe sona” echnique
Bo h e sions a e echnically co ec , and hey bo h con ain clea and concise exam-s yle ques ions.
Howe e , he e a e some di e ences in s yle and con en ha migh be a ibu ed o he "ac as a eache "
ins uc ion.
Simpli ica ion and cla i ica ion: The "ac as a eache " e sion seems o use sligh ly simple language and
clea e ph asing. Fo example, in he i s ques ion, he "ac as a eache " e sion simply gi es "DNA →
RNA → P o ein" as an op ion, whe eas he o iginal e sion p o ides a longe desc ip ion ("DNA is
ansc ibed in o RNA, which is hen ansla ed in o p o eins"). This migh e lec a eache 's in en ion o
p o ide clea , unambiguous choices.
Di ec ness: The "ac as a eache " e sion ends o use mo e di ec language. Fo example, he ques ion
"Wha is he p ocess o copying gene ic in o ma ion om DNA o RNA called?" di ec ly asks abou a
speci ic p ocess, whe eas he equi alen ques ion in he o iginal e sion asks abou he ole o RNA in he
cen al dogma, which is a bi mo e abs ac .
Tes ing Hie a chical Unde s anding: In he hi d ques ion, he "ac as a eache " e sion adds he e m
HIV in he choices, while he o iginal e sion uses only he e m "Re o i us". This change may aim o
e alua e s uden s' comp ehension o he hie a chical ela ionship be ween speci ic examples (HIV) and
b oade ca ego ies ( e o i uses). I 's a eaching ac ic o es whe he s uden s can dis inguish be ween he
wo, which is c ucial o a deep unde s anding o he subjec .
In summa y, he "ac as a eache " ins uc ion seems o ha e led o ques ions ha a e sligh ly simple , mo e
di ec , and possibly mo e engaging due o he use o speci ic examples. Howe e , he o e all con en and
di icul y le el appea o be simila be ween he wo e sions. I 's also wo h no ing ha hese di e ences
migh no be obse ed consis en ly ac oss all p omp s.
b. Templa e
The empla e is o ganized in o se e al sec ions. The i s placeholde is o he ques ion numbe , which
allows o easy acking and o ganiza ion o di e en ques ions. This is ollowed by placeholde s o he
ques ion con en and ou po en ial answe op ions (labeled A h ough D), gi ing s uc u e o he ques ion
and i s possible solu ions.
Wha se s his empla e apa is he inclusion o placeholde s o bo h he co ec answe and an explana ion
o ha answe . This design ensu es ha he exam doesn' jus e alua e s uden knowledge bu also p o ides
eedback and aids in lea ning, p omo ing an unde s anding o why a pa icula answe is co ec .
O cou se, as men ioned in he li e a u e, he main ad an age o p o iding he empla e is ha he expec ed
ou pu has always he same o ma , based on he use ’s needs.
See Table 10. P omp S uc u ing - Templa e Resul s o he esul s o his echnique.
38
E alua ion o Teache Assis an answe s gi en using he “ empla e” echnique
The addi ion o he empla e o he o iginal p omp c ea ed a signi ican change in he o ma o he exam
ques ions p oduced by he la ge language model (LLM). He e's a de ailed analysis:
The key di e ences wi h adding he empla e o he la ge language model (LLM) include:
Fo ma ing: The empla e has s uc u ed he ou pu in a mo e o ganized way. Each componen ( he
ques ion, answe op ions, co ec answe , and explana ion) now has a clea and de ined place, making i
easie o pa se and unde s and. This o ma also emphasizes he impo ance o p o iding a de ailed
explana ion, which could enhance he lea ning p ocess.
Comple eness: In con as o he o iginal e sion, he empla e has p omp ed he LLM o p o ide
explana ions o he co ec answe s. This helps o elabo a e on he easoning behind he co ec answe s
and could po en ially aid s uden s in hei unde s anding o he ma e ial.
As such, adding he empla e is abou imposing s uc u e and ensu ing comple eness, which guides he
model o ou pu con en in a speci ic, desi ed o ma .
5.2.2 P omp Enginee ing
a. Few sho
Fo his echnique we consul ed a documen gi ing some ules o c ea ing a good quali y mul iple-choice
ques ions exam pape , (san ound y.com).
See Table 11. P omp Enginee ing - Few sho Resul s o he esul s o his echnique.
E alua ion o Teache Assis an answe s gi en using he “ ew sho ” echnique
We do no obse e wo h men ioning di e ences when we used he examples o he ew sho p omp ing.
Thus his change in he p omp was no success ul.
b. Chain-o - hough
Fo “chain-o - hough ” we used he guide by BRIGHAM YOUNG UNIVERSITY (2001).
This guide has he p inciples o how o w i e an e ec i e mul iple choice ques ion. As s a es in he li e a u e
we can use his logic, his chain-o - hough o help he LLM o ollow he same logic when p o iding an
answe .
In o de o gi e he AI o unde s and his ules, we passed hem as a p omp as is, om he pape . And a
he end we added he ollowing p omp :
“Acco ding o he abo e ex , which explains he ules o c ea ing a mul iple-choice ques ionnai e, gene a e
h ee mul iple choice ques ions wi h ou possible op ions each on he opic o Molecula Biology, designed
o es s uden s' unde s anding o he undamen al p inciples o his subjec .”
See Table 12. P omp Enginee ing - Chain-o - hough Resul s o he esul s o his echnique.
39
E alua ion o Teache Assis an answe s gi en using he “Chain-o - hough ” echnique
E en hough i ollowed he ules we can see ha he chap e i chose is w ong.
5.2.3 E alua ion:
P omp
s uc u e/Technique
used
C i e ion 1: F om how many
concep s a e he ques ions aken?
We check he di e si y o he answe
om he book.
C i e ion 2: Rele an Con en
The con en o he answe should
always be ele an o he con en
o he book.
Pe sona echnique


Templa e


Few-sho p omp


Chain-o - hough


46
28 OpenAI, 2023. 'GPT-4 echnical epo '. a Xi .o g. A ailable a : h ps://a xi .o g/abs/2303.08774
(Accessed: 03 July 2023).
29 Pe ei a, J., 2016. 'Le e aging Cha bo s o imp o e sel -guided lea ning h ough con e sa ional
quizzes'. P oceedings o he Fou h In e na ional Con e ence on Technological Ecosys ems o
Enhancing Mul icul u ali y [P ep in ]. doi:10.1145/3012430.3012625.
30 san ound y.com, 'Molecula biology MCQ (mul iple choice ques ions)'. San ound y. A ailable a :
h ps://www.san ound y.com/1000-molecula -biology-ques ions-answe s/ (Accessed: 02 July
2023).Yao, S. e al. (2023) Reac : Syne gizing easoning and ac ing in language models,
a Xi .o g. A ailable a : h ps://a xi .o g/abs/2210.03629 (Accessed: 03 June 2023).
31 Voon, W., e al., 2022. Pe o mance analysis o se en Con olu ional Neu al Ne wo ks (cnns)
wi h ans e lea ning o in asi e duc al ca cinoma (IDC) g ading in b eas his opa hological
images. Na u e News. A ailable a : h ps://www.na u e.com/a icles/s41598-022-21848-3
[Accessed 02 July 2023].
32 Wang, X., e al., 2023. Sel -consis ency imp o es chain o hough easoning in language models.
a Xi .o g. A ailable a : h ps://a xi .o g/abs/2203.11171 [Accessed 03 June 2023].
33 Wei, J., e al., 2023. Chain-o - hough p omp ing elici s easoning in la ge language models.
a Xi .o g. A ailable a : h ps://a xi .o g/abs/2201.11903 [Accessed 03 June 2023].
34 Wei, X., e al., 2023. Ze o-sho in o ma ion ex ac ion ia cha ing wi h cha gp . a Xi .o g.
A ailable a : h ps://a xi .o g/abs/2302.10205 [Accessed 13 May 2023].
35 Whi e, J., e al., 2023. A p omp pa e n ca alog o enhance p omp enginee ing wi h cha gp .
a Xi .o g. A ailable a : h ps://a xi .o g/abs/2302.11382 [Accessed 13 May 2023].
36 Za olokina, L., Dola a, M. and Schwabe, G., 2016. The in ech phenomenon: An eceden s o
inancial inno a ion pe cei ed by he popula p ess - inancial inno a ion. Sp inge Open.
A ailable a : h ps://j in-swu e.sp inge open.com/a icles/10.1186/s40854-016-0036-7 [Accessed
30 May 2023].
37 Zhang, Z., e al., 2023. Mul imodal chain-o - hough easoning in language models. a Xi .o g.
A ailable a : h ps://a xi .o g/abs/2302.00923 [Accessed 03 June 2023].
38 Zhao, T.Z., e al., 2021. Calib a e be o e use: Imp o ing ew-sho pe o mance o language
models. a Xi .o g. A ailable a : h ps://a xi .o g/abs/2102.09690 [Accessed 02 June 2023].

47
Acknowledgemen s
S e gios and Dimi is wish o exp ess hei p o ound g a i ude o D . Hamam Mokayed o his in aluable
guidance and unwa e ing suppo h oughou his esea ch endea o . His dep h o knowledge and expe ise
played a pi o al ole in shaping he di ec ion and ou comes o ou s udy.
We also acknowledge and app ecia e he inancial backing om he AI4EDU Eu opean p ojec , which
pa ially unded ou esea ch. Thei con ibu ion was ins umen al in acili a ing a ious ace s o ou
in es iga i e pu sui s.
On a pe sonal no e, Dimi is ex ends hea el app ecia ion o his son, Ch is os, o his endu ing pa ience
and unde s anding du ing he du a ion o he mas e 's p og am. The sac i ices made in e ms o sha ed
momen s ha e been deeply acknowledged, and Dimi is emains indeb ed o Ch is os's con inual suppo .
Addi ionally, Dimi is would like o exp ess his big g a i ude o his amily, which e en been a away, hey
ha e all be suppo i e gi ing him s eng h and cou age o inishing his mas e ’s wi h success.
Fu he mo e, Dimi is wishes o commend Geo gios o his exempla y collabo a ion du ing g oup
assignmen s. His insigh s, dedica ion, and commi men o excellence signi ican ly en iched he academic
expe ience.
S e gios, om his side wan s o exp ess deep g a i ude o his amily o hei cons an suppo . To his dad,
Elias, and mom, I igeneia, o hei endless lo e; o his b o he , Lab os, and sis e , Vily, o always being
he e; and a special men ion o his g andpa en s o always li ing his spi i s and s anding by his side.
Also, du ing his mas e 's, S e gios had he oppo uni y o wo k wi h Soma, And eas, and An onio. Thei
knowledge and eamwo k ha e been in aluable, and he has lea ned so much om he ime oge he .
S e gios is also hank ul o his colleagues and his manage , Sone , o hei unde s anding and lexibili y as
he balanced wo k and s udies.
Mo eo e , S e gios wan s o gi e a big hank you o all his iends who ha e been pa o his jou ney wi h
him.
In closing, S e gios and Dimi is wan o highligh he impo ance o eamwo k. W i ing his hesis oge he
had i s challenges, bu he inal esul shows wha can be achie ed when wo people collabo a e e ec i ely.
The jou ney was illed wi h momen s o sha ed disco e y, mu ual lea ning, and unwa e ing suppo .
Th ough open communica ion and espec o each o he 's pe spec i es, we we e able o combine ou
s eng hs, na iga e challenges, and p oduce a piece o wo k ha we' e bo h p oud o .
48
Appendix A
Following a e lis ed he ques ions, and he answe s gi en by S udy Buddy in compa ison o he Teache ’s,
o he h ee subjec s o in e es , Biology, Ma hema ics, and Physics.
Biology
Ques ion
Answe by S udy Buddy
Answe gi en by he eache
1. Who coined he e m cell, in
e e ence o he iny s uc u es
seen in li ing o ganisms?
Robe Hooke coined he e m
cell in e e ence o he iny
s uc u es seen in li ing
o ganisms.
The i s ime he wo d cell was
used o e e o hese iny uni s
o li e was in 1665 by a B i ish
scien is named Robe Hooke.
(O can jus answe Robe
Hooke.)
2. Who iden i ied animalcules?
Wha a e animalcules?
An on an Leeuwenhoek
iden i ied animalcules, which
a e now known as bac e ia.
Animalcules we e disco e ed by
An on an Leeuwenhoek.
Today, we call Leeuwenhoek’s
animalcules bac e ia.
3. Wha a e he h ee main
pa s o he cell heo y?
The h ee main pa s o he cell
heo y a e: all o ganisms a e
made up o one o mo e cells, all
he li e unc ions o an o ganism
occu wi hin cells, and all cells
come om p eexis ing cells.
The h ee main pa s o he cell
heo y: a. all o ganisms a e
made up o one o mo e cells b.
all he li e unc ions o an
o ganism occu wi hin cells c.
all cells come om p eexis ing
cells
4. Lis he ou pa s common
o all cells.
The ou pa s common o all
cells a e plasma memb anes,
cy oplasms, ibosomes, and
DNA.
The ou componen s common
o all cells a e he plasma
memb ane, cy oplasm,
ibosomes and DNA.
5. Wha a e he cell s uc u es
whe e p o eins a e made?
The cell s uc u es whe e
p o eins a e made a e
ibosomes.
Ribosomes a e s uc u es in he
cy oplasm whe e p o eins a e
made.
6. Wha is he ole o DNA?
The ole o DNA is no speci ied
in he gi en con ex in o ma ion.
DNA con ains he gene ic
ins uc ions ha cells need o
make p o eins.
Table 5. Biology - Answe compa ison
49
Ma hema ics
#
Ques ion
(Sequence o
Numbe s)
Answe by S udy Buddy
Answe gi en by
he eache
Resul
Compa ison
1.
1, 6, 21, 66
Yes, I can ollow he book's heo y o ind he
pa e n ha applies o he sequence 1, 6, 21,
66. The pa e n ule is "mul iply by 3 and
sub ac 3."
x 3 + 3

2.
95, 80, 65, 50
The pa e n ule is o "sub ac 15."
– 15

3.
3, 10, 17, 24
Yes, he book's heo y can be ollowed o ind
he pa e n o he sequence 3, 10, 17, 24 and
he answe is o "add 7" o each e m.
+ 7

4.
256, 64, 16, 4
Yes, I can ollow he book's heo y o ind he
pa e n ha applies o he sequence 256, 64,
16, 4. The pa e n ule o his sequence is
"di ide by 4."
÷ 4

5.
3, 11, 43, 171
Yes, I can ollow he book's heo y o ind he
pa e n ha applies o he sequence 3, 11, 43,
171. The pa e n ule is o mul iply by 4 and
add 1.
x 4 – 1

6.
81, 27, 9, 3
Yes, I can ollow he book's heo y o ind he
pa e n ha applies o he sequence 81, 27, 9,
3. The pa e n ule is "di ide by 3."
÷ 3

7.
4, 13, 40, 121
Yes, I can ollow he book's heo y o ind he
pa e n ha applies o he sequence 4, 13, 40,
121, and he inal answe is "mul iply by 3 and
sub ac 5."
X 3 + 1

8.
1, 6, 31, 156
Yes, I can ollow he book's heo y o ind he
pa e n ha applies o he sequence 1, 6, 31,
156. The pa e n ule is "mul iply by 5 and
sub ac 4."
X 5 + 1

Table 6. Ma hema ics - Answe compa ison
50
Physics
#
Ques ion
Answe by S udy Buddy
Answe gi en by he eache
1.
I will gi e you he ollowing
p oblem: You a e si ing on you
bike a es . You b o he comes
unning a you om behind a a
speed o 2 m/s. A he exac
momen he passes you, you s a
up on you bike wi h an
accele a ion o 2 m/s2. A wha
ime do you ha e he same
speed as you b o he ? Gi e an
answe acco ding o he book's
heo y and examples.
The ime a which you ha e
he same speed as you
b o he can be ound by
se ing you eloci y
equa ion equal o his
eloci y equa ion and
sol ing o ime. You
b o he 's eloci y is cons an
a 2 m/s, so his eloci y
equa ion is _b = 2 m/s.
You eloci y equa ion can
be ound using he kinema ic
equa ion _ = _i + a ,
whe e _ is you inal
eloci y, _i is you ini ial
eloci y (which is 0 m/s), a
is you accele a ion o 2
m/s2, and is ime.
Subs i u ing in hese alues,
we ge _ = 2 . Se ing his
equal o you b o he 's
eloci y equa ion, we ge 2
= 2, which means = 1
second. The e o e, you will
ha e he same speed as you
b o he a e 1 second.
Le ’s conside he p oblem as
a whole (see diag am). You
b o he mo es a a cons an
eloci y; he e o e his
posi ion inc eases linea ly
om 0 wi h a slope o 2.
You posi ion is a pa abola
ha in e sec s you b o he ’s
when you a e age eloci y is
equal o his, ha is, a 2
seconds. You b o he ’s
eloci y is a cons an 2m,
while you s is an upwa ds
sloping line wi h a slope o 2.
They in e sec when you
ins an aneous eloci ies a e
equal, ha is, a 1 second. See
he diag ams below o
p oblems 6 and 7 o an
example o how o d aw such
a g aph.
2.
I will gi e you he ollowing
p oblem: You a e s anding a he
oo o he Bank o Ame ica
building in San F ancisco, which
is 52 loo s (237 m) high. You
launch a ball s aigh up in he
ai om he edge o he oo o
he building. The ini ial e ical
speed is 70 m/s. (Fo his
p oblem, you may igno e you
own heigh , which is e y small
compa ed o he heigh o he
building.). How high up does he
ball go? Gi e an answe by
ollowing he book's heo y and
examples.
The ball's maximum heigh
can be ound using he
equa ion:
Δy = _iy* + 1/2*a_y* 2
whe e Δy is he change in
heigh , _iy is he ini ial
e ical eloci y, a_y is he
accele a ion due o g a i y (-
9.8 m/s2), and is he ime i
akes o he ball o each i s
maximum heigh (which can
be ound using _ y = _iy +
a_y* = 0).
Plugging in he gi en alues:
Δy = (70 m/s)*(7.14 s) +
1/2*(-9.8 m/s2)*(7.14 s)2
Δy = 249.8 m
Again, we use mgh = .5m 2,
we ind ha h = 250m.
51
The e o e, he ball eaches a
heigh o app oxima ely 250
me e s.
3.
I will gi e you he ollowing
p oblem: You a e s anding a he
oo o he Bank o Ame ica
building in San F ancisco, which
is 52 loo s (237 m) high. You
launch a ball s aigh up in he
ai om he edge o he oo o
he building. The ini ial e ical
speed is 70 m/s. (Fo his
p oblem, you may igno e you
own heigh , which is e y small
compa ed o he heigh o he
building.). Wha is he eloci y
o he ball igh be o e i hi s he
op o he building? Gi e an
answe by ollowing he book's
heo y and examples.
The eloci y o he ball igh
be o e i hi s he op o he
building can be ound using
he kinema ic equa ion: 2 =
i2 + 2ad, whe e is he
inal eloci y (which is 0 m/s
when he ball hi s he op o
he building), i is he ini ial
eloci y (which is 70 m/s), a
is accele a ion due o g a i y
(which is -9.8 m/s2 because
he ball is mo ing upwa d),
and d is he displacemen
(which is he heigh o he
building, 237 m).
Rea anging he equa ion o
sol e o , we ge :
= sq ( i2 + 2ad)
Plugging in he alues, we
ge :
= sq ((70 m/s)2 + 2(-9.8
m/s2)(237 m))
= sq (4900 m2/s2 - 4644.4
m2/s2)
= sq (255.6 m2/s2)
= 15.99 m/s
So, he eloci y o he ball
igh be o e i hi s he op o
he building is
app oxima ely 15.99 m/s.
We know ha a he op o i s
jou ney, he ball has a speed
o 0. The e o e, since i
a els 13 me e s a cons an
accele a ion be o e i hi s he
building, we can use he hi d
’big h ee’ equa ion o ind
2 = 02 +2ax = 2×9.8×13
=⇒ =≈ 16m/s
4.
I will gi e you he ollowing
p oblem: You a e s anding a he
oo o he Bank o Ame ica
building in San F ancisco, which
is 52 loo s (237 m) high. You
launch a ball s aigh up in he
ai om he edge o he oo o
he building. The ini ial e ical
speed is 70 m/s. (Fo his
p oblem, you may igno e you
own heigh , which is e y small
compa ed o he heigh o he
building.). Fo how many
Using he o mula, h = i +
1/2g 2, whe e h is he heigh ,
i is he ini ial eloci y, g is
he accele a ion due o
g a i y (-9.81 m/s2), and is
he ime:
A he highes poin , he
ball's speed is 0 m/s, so we
can use i = 70 m/s as he
ini ial eloci y.
We can do his in wo pa s
using he second ’big h ee’
equa ion (wa ch ou o signs,
hough!) : ( ) = 0 + a (a he
op, he eloci y is 0; his is
he inal eloci y in he i s
pa , and he ini ial eloci y in
he second pa , so he
equa ion is simpli ied bo h
imes): in he ligh up, he
ball spends
= 70/9.8 ≈ 7.14s.

52
seconds is he ball in he ai om
he momen i was launched ill
i hi s back o he g ound? Gi e
an answe by ollowing he
book's heo y and examples.
When he ball hi s he
g ound, he heigh is 0 m.
So, h = 0 m.
0 = (70 m/s) + 1/2(-9.81
m/s2) 2
Simpli ying, we ge :
0 = 70 - 4.905 2
Sol ing o , we ge :
= 14.27 s
The e o e, he ball is in he
ai o app oxima ely 14.27
seconds.
Du ing he ligh down, i
spends
16/9.8 ≈ 1.63s
So he o al is abou 8.77
seconds.
5.
I will gi e you he ollowing
p oblem: Wha accele a ion
should you use o inc ease you
speed om 10 m/s o 18 m/s
o e a dis ance o 55 m? Gi e an
answe by ollowing he book's
heo y and examples.
To sol e his p oblem, we
can use he o mula:
accele a ion = ( inal eloci y
squa ed - ini ial eloci y
squa ed) / (2 x dis ance)
Plugging in he gi en alues,
we ge :
accele a ion = (182 - 102) / (2
x 55)
accele a ion = 196 / 110
accele a ion = 1.78 m/s2
The e o e, he accele a ion
equi ed o inc ease he
speed om 10 m/s o 18 m/s
o e a dis ance o 55 m is
1.78 m/s2.
Use big h ee equa ion h ee,
182 = 102 + 2 × a × 55 =⇒ a ≈
2 m/s2
6.
I will gi e you he ollowing
p oblem: You d op a ock om
he op o a cli . The ock akes
3.5 seconds o each he bo om.
Wha is he magni ude (i.e.,
nume ical alue) o he
accele a ion o he ock a he
momen i is d opped? Gi e an
answe by ollowing he book's
heo y and examples.
The magni ude o
accele a ion due o g a i y is
9.8 m/s2. When he ock is
d opped, i expe iences a
cons an accele a ion o 9.8
m/s2 owa ds he g ound due
o g a i y. The e o e, he
magni ude o he
accele a ion o he ock a
he momen i is d opped is
9.8 m/s2.
G a i a ional accele a ion is
cons an : a = 9.8m/s2
53
Table 7. Physics - Answe compa ison
7.
I will gi e you he ollowing
p oblem: You d op a ock om
he op o a cli . The ock akes
3.5 seconds o each he bo om.
Wha is he heigh o he cli ?
Gi e an answe by ollowing he
book's heo y and examples.
The heigh o he cli can be
calcula ed using he o mula
h = (1/2)g 2, whe e g is he
accele a ion due o g a i y
(9.8 m/s2) and is he ime
aken o he ock o all (3.5
seconds). The e o e, h =
(1/2)(9.8 m/s2)(3.5 s)2 =
60.725 m. Thus, he heigh
o he cli is app oxima ely
60.725 me e s.
Use big h ee equa ion one
(down is posi i e, label cli
as ze o poin ):
x = 0+0+.5×9.8×(3.5)2 =⇒ x
= 60m
54
Appendix B
a. Baseline answe
Ques ion
Answe
Gene a e 3 mul iple-choice ques ions wi h 4
possible answe s each ega ding molecula
biology. The ques ions should es
unde s anding o he undamen al p inciples o
he opic.
1. Which o he ollowing bes desc ibes he
cen al dogma o molecula biology?
a) DNA is ansc ibed in o RNA, which is hen
ansla ed in o p o eins.
b) RNA is ansc ibed in o DNA, which is hen
ansla ed in o p o eins.
c) P o eins a e ansc ibed in o DNA, which is
hen ansla ed in o RNA.
d) DNA is ansla ed in o p o eins, which a e hen
ansc ibed in o RNA.
2. Wha is he ole o RNA in he cen al dogma
o molecula biology?
a) RNA ca ies he ins uc ions om p o eins o
DNA.
b) RNA ca ies he ins uc ions om DNA o
p o eins.
c) RNA ca ies he ins uc ions om p o eins o
o he p o eins.
d) RNA ca ies he ins uc ions om DNA o
o he DNA molecules.
3. Which ype o i uses do no adhe e o he
cen al dogma o molecula biology?
a) Re o i uses
b) Adeno i uses
c) He pes i uses
d) In luenza i uses
Table 8. Baseline Ques ion Resul s
55
a. P omp S uc u ing
i. Pe sona
Ques ion
Answe
Ac as a eache and gene a e 3 mul iple-choice
ques ions wi h 4 possible answe s each ega ding
molecula biology. The ques ions should es
unde s anding o he undamen al p inciples o he
opic.
1. Wha is he cen al dogma o molecula
biology?
A. DNA → RNA → P o ein
B. P o ein → RNA → DNA
C. DNA → P o ein → RNA
D. RNA → P o ein → DNA
2. Wha is he p ocess o copying gene ic
in o ma ion om DNA o RNA called?
A. T ansc ip ion
B. T ansla ion
C. Replica ion
D. Re e se T ansc ip ion
3. Wha ype o i us does no obey he cen al
dogma o molecula biology?
A. HIV
B. In luenza
C. He pes
D. Re o i us
Table 9. P omp S uc u ing - Pe sona Resul s
ii. Templa e
Ques ion
Answe
Gene a e h ee mul iple-choice ques ions wi h
ou possible op ions each on he opic o
Molecula Biology, designed o es s uden s'
unde s anding o he undamen al p inciples o
his subjec . Please adhe e o he ollowing
empla e o each ques ion:
Ques ion # [placeholde o ques ion numbe ]:
Ques ion #1:
Wha is he cen al dogma o molecula biology?
A. DNA → RNA → P o ein
B. RNA → P o ein → DNA
C. P o ein → DNA → RNA
D. DNA → P o ein → RNA