AnEx ensibleF amewo k o llm-enhanced
Sc a chS a icCodeAnalysis
Demo h ps://sc a ch. im.uni-passau.de/sc a ch/
�h ps://gi hub.com/se2p/Li e Box
�h ps://a xi .o g/abs/2509.12021
Benedik Fein, Flo ian Obe mülle , Go don F ase
ASE 2025: Tool Demons a ion T ack
Command Line In e ace
All ea u es a e a ailable ia he cli, o example o allow o ba ch p ocessing o s uden submissions:
ja a -Dli e box.llm.openai.api-key=KEY -ja Li e Box.ja llm ix
^--pa h=submissions/ ^-- a ge =Boa
Via he cli use s can also access addi ional ea u es no ye in eg a ed in o he ui:
• eques he llm o ind bo h new issues (smells) and commen on good code (pe umes), and
• ask he llm o ill in missing pa s o he code.
P oblem and Idea
The p oblem:
• P og amming lea ne s equi e eedback on hei code
• False posi i es a e unaccep able in an educa ional con ex
•llms unde pe o m wi h isual Sc a ch code
•llms canno p oduce alid Sc a ch code
The solu ion:
• Use llms o explain issues ound by s a ic analysis (Li e Box)
•Li e Box+con e s Sc a ch code o ex ual sc a chblocks o p omp ing
•Li e Box+con e s llm esponses back o Sc a ch code
• Enables con e sa ions abou code issues and p og ams
Li e Box: Gene ic s a ic code analysis eedback
Li e Box
+
: Ex ended wi h p og am-speci ic eedback gene a ed by he llm
Use In e ace
The Sc a ch gui is ex ended wi h a code analysis in e ace:
• Lis s code issues ound by Li e Box
• Use s can ask o explana ions
• Use s can ask o ix sugges ions
• Use s can ask hei own ques ions abou he code
Sc a ch P og am Rep esen a ion
Li e Box
+
p omp s he llm o explain he obse able beha iou , he un-
de lying p og amming concep he s uden migh ha e misunde s ood,
and s eps he s uden can ollow o ix hei speci ic issue.
When eques ing ex ended eedback om he llm o a Li e Box s a ic
code analysis wa ning, we add bo h he o iginal code and he gene ic
issue desc ip ion o he p omp . Li e Box con e s he isual code om
he Sc a ch-in e nal json ep esen a ion in o sc a chblocks syn ax as used
on he public Sc a ch communi y o ums:
when clicked
mo e -1 s eps
i ouching colo hen
^// Sc ip : "in e nal sc ip -ID"
when g een lag clicked
i < ouching colo (#5ce581)> hen
mo e (-1) s eps
end
In i s esponse, he llm should again use he sc a chblocks syn ax. The
sc a chblocks elemen s a e ende ed as inline images in he llm explana-
ion. Howe e , o he ‘GPT: Fix!’ ea u e he ixed sc ip needs be in e-
g a ed di ec ly in o he cu en p og am and loaded in o he edi o so
ha he s uden can con inue o make changes. To enable his, ou newly
de eloped sc a chblocks-pa se con e s code blocks in he esponse in o
he as ep esen a ion o Li e Box, which in u n is mapped o he na i e
ep esen a ion o Sc a ch, which hen can be loaded as edi able code.
The Sc a ch-in e nal json is no di ec ly sui able o llms since i con ains
abs ac opcodes and con ains ‘nex ’-poin e s o he ollowing blocks,
which may howe e be s o ed in a -away loca ions wi hin he json and
need o be upda ed consis en ly when making changes.
Sys em Design
L
i e
B
ox
I
ssues
R
eques
H
in
R
eques
F
ix
A
sk
Q
ues ion
S
a ic
C
ode
A
nalysis
LLM
P
omp
D
esigns
S
c a ch
B
locks
P
a se
REST
API
O
llama
langchain
Li e Box
+
is implemen ed as wo main componen s: The use in e ace in eg a ion and a backend unning
he s a ic code analysis and he communica ion wi h he llms.
By using gene ic in e aces i is designed o be con igu able and ex ensible wi h
• cus om p omp designs (e.g., o suppo o he na u al languages),
• al e na i e llm p o ide s, and
•
new llm-based ea u es making use o he s a ic code analysis and sc a chblocks pa sing in as uc u e.
Besides he in eg a ion in he Sc a ch gui and he cli,Li e Box
+
can also be used as a lib a y p o iding a
Ja a api o suppo he in eg a ion wi h o he ools.
1
