scieee Science in your language
[en] (orig)

Artificial Intelligence and Machine Evolution

Abstract

This working paper provides a brief overview on Artificial Intelligence (AI) and Machine Evolution. Machine Evolution can be understood as a situation where machines autonomously create new machines which create further machines and so on. While such an evolution is not yet possible, substantial progress was made in multiple areas which bring machines and AI much closer to full autonomy, self-learning, and adaptation for their self-improvement. These developments accelerate the technological progress, but make huma oversight and interventions increasingly difficult, i.e., instead of being in the loop, humans are moved out of the loop. Advances include the AI-assisted design of computer chips, the AI-assisted design of entire computers, self-learning and improving AI agents in experimental settings as well as automated fabrications. Already in 2024, the machine learning-based AI program AlphaChip generated superhuman chip layouts and created in 2025 the new Ironwood TPU as one of the fastest marketed chips. China is advancing as well with the chip-designing AI QiMeng that already designed chips comparable with commercial chips. The chip design research is advancing rapidly. The first three-dimensional monolithic 3D chip to accelerate AI was presented by researchers of the Stanford University in December 2025. The Shanghai-based company CHIPX released a photonic quantum chip that for some mathematical operations was 1000 times faster than conventional chips. Quilter's physics-driven reinforcement learning AI system designed in the Project Speedrun a working 843-component Linux computer in only 38 hours and autonomously completed 98% of placement, routing, and physics. New ways of AI training and learning were proposed in 2025; the networking and information exchange between AI agents in the first step and the open-ended evolution of AI agents in the second step by the so-called Darwin Gödel Machine (DGM). Researchers from Google DeepMind proposed that AI systems learn from each other and share their experience permanently with unlimited development above humans, but networking and knowledge exchange between AI system may amplify the risk for system errors and malfunction of the AI agent ecosystem by exchange of inaccurate data. In 2025, the Darwin Gödel Machine (DGM) was developed, a self-improving Coding AI Agent system that iteratively modifies its own code and empirically validates each change using coding benchmarks which was effective in experiments. The spread of robots may open the way into fully automated fabrications. The human control of AI can be done by observation of output and action, but can be complemented by monitoring of the chain of thoughts, i.e., the internal AI protocols that show their reasoning (thinking) steps. The ability to observe the chain of thought of an AI is called monitorability. There are concerns that novel AI architectures may reduce the monitorability despite findings of hidden misalignment (scheming) where frontier AI models hide thoughts e.g., to get rewards or to avoid reinforcement learning. Various researchers express serious concerns about a potentially uncontrolled evolution e.g., by self-learning. The key concern is that humans are currently not really prepared for this situation and may not be able to provide sufficient oversight, i.e., the situation may get out of human control and for this reason, a strict governance with regulations and supervision should be established.

Read accessible full text

Artificial Intelligence and Machine Evolution

Author: Saalbach, Klaus
Year: 2026
DOI: 10.48693/817
Source: https://osnadocs.ub.uni-osnabrueck.de/bitstream/ds-2026010113958/1/Artificial_Intelligence_Machine_Evolution_2026_Saalbach.pdf
A i icial_In elligence_and_Machine_E olu ion 1
Apl. P o . D . D . K. Saalbach
Geos a egy and Geopoli ics
Depa men 1
49069 Osnab ueck
A i icial In elligence and Machine E olu ion
02 Janua y 2026
Summa y
This wo king pape p o ides a b ie o e iew on A i icial In elligence (AI) and Machine
E olu ion. Machine E olu ion can be unde s ood as a si ua ion whe e machines au onomously
c ea e new machines which c ea e u he machines and so on. While such an e olu ion is no
ye possible, subs an ial p og ess was made in mul iple a eas which b ing machines and AI
much close o ull au onomy, sel -lea ning, and adap a ion o hei sel -imp o emen .
These de elopmen s accele a e he echnological p og ess, bu make huma o e sigh and
in e en ions inc easingly di icul , i.e., ins ead o being in he loop, humans a e mo ed ou o
he loop. Ad ances include he AI-assis ed design o compu e chips, he AI-assis ed design
o en i e compu e s, sel -lea ning and imp o ing AI agen s in expe imen al se ings as well as
au oma ed ab ica ions. Al eady in 2024, he machine lea ning-based AI p og am AlphaChip
gene a ed supe human chip layou s and c ea ed in 2025 he new I onwood TPU as one o he
as es ma ke ed chips. China is ad ancing as well wi h he chip-designing AI QiMeng ha
al eady designed chips compa able wi h comme cial chips. The chip design esea ch is
ad ancing apidly. The i s h ee-dimensional monoli hic 3D chip o accele a e AI was
p esen ed by esea che s o he S an o d Uni e si y in Decembe 2025. The Shanghai-based
company CHIPX eleased a pho onic quan um chip ha o some ma hema ical ope a ions
was 1000 imes as e han con en ional chips. Quil e 's physics-d i en ein o cemen lea ning
AI sys em designed in he P ojec Speed un a wo king 843-componen Linux compu e in only
38 hou s and au onomously comple ed 98% o placemen , ou ing, and physics.
New ways o AI aining and lea ning we e p oposed in 2025; he ne wo king and in o ma ion
exchange be ween AI agen s in he i s s ep and he open-ended e olu ion o AI agen s in he
second s ep by he so-called Da win Gödel Machine (DGM). Resea che s om Google
DeepMind p oposed ha AI sys ems lea n om each o he and sha e hei expe ience
pe manen ly wi h unlimi ed de elopmen abo e humans, bu ne wo king and knowledge
exchange be ween AI sys em may ampli y he isk o sys em e o s and mal unc ion o he AI
agen ecosys em by exchange o inaccu a e da a. In 2025, he Da win Gödel Machine (DGM)
was de eloped, a sel -imp o ing Coding AI Agen sys em ha i e a i ely modi ies i s own code
and empi ically alida es each change using coding benchma ks which was e ec i e in
expe imen s. The sp ead o obo s may open he way in o ully au oma ed ab ica ions.
The human con ol o AI can be done by obse a ion o ou pu and ac ion, bu can be
complemen ed by moni o ing o he chain o hough s, i.e., he in e nal AI p o ocols ha show
hei easoning ( hinking) s eps. The abili y o obse e he chain o hough o an AI is called
moni o abili y. The e a e conce ns ha no el AI a chi ec u es may educe he moni o abili y
despi e indings o hidden misalignmen (scheming) whe e on ie AI models hide hough s
e.g., o ge ewa ds o o a oid ein o cemen lea ning. Va ious esea che s exp ess se ious
conce ns abou a po en ially uncon olled e olu ion e.g., by sel -lea ning. The key conce n is
ha humans a e cu en ly no eally p epa ed o his si ua ion and may no be able o p o ide
su icien o e sigh , i.e., he si ua ion may ge ou o human con ol and o his eason, a s ic
go e nance wi h egula ions and supe ision should be es ablished.
A i icial_In elligence_and_Machine_E olu ion 2
Con en
1 Machine E olu ion ................................................................................................................................ 3
1.1 In oduc ion ................................................................................................................................... 3
1.2 AI-assis ed Chip Design .................................................................................................................. 3
1.3 AI-assis ed Compu e Design ........................................................................................................ 4
1.4 Sel -Lea ning AI .............................................................................................................................. 4
1.5 Fully Au oma ed Fab ica ion ......................................................................................................... 6
1.6 Moni o ing and O e sigh o AI ..................................................................................................... 6
2 Summa y ............................................................................................................................................... 7
3 Re e ences ............................................................................................................................................ 8
A i icial_In elligence_and_Machine_E olu ion 3
1 Machine E olu ion
1.1 In oduc ion
This wo king pape p o ides a b ie o e iew on A i icial In elligence and Machine E olu ion.
Machine E olu ion can be unde s ood as a si ua ion whe e machines au onomously c ea e new
machines which c ea e u he machines and so on. While such an e olu ion is no ye possible,
subs an ial p og ess was made in mul iple a eas which b ing machines and AI much close o
ull au onomy, sel -lea ning, and adap a ion o hei sel -imp o emen .
These de elopmen s accele a e he echnological p og ess, bu make huma o e sigh and
in e en ions inc easingly di icul , i.e., ins ead o being in he loop, humans a e mo ed ou o
he loop. Ad ances include he AI-assis ed design o compu e chips, he AI-assis ed design o
en i e compu e s, sel -lea ning and imp o ing AI agen s in expe imen al se ings as well as
au oma ed ab ica ions. The key challenge o he human de elope s is o ensu e o e sigh and
moni o ing o ac i i ies o a oid sys em e o s, hallucina ions, and con amina ion o AI
ecosys ems wi h inaccu a e da a.
AI models combine lea ning algo i hms wi h up o billions o pa ame e s and la ge aining da a
se s, which esul s in opaque black-box sys ems, his is known as explainabili y, in e p e abili y
o p edic abili y issue
1
. How does an AI ‚know‘ an objec ? Some imes, an AI is e y good in
iden i ying objec s o animals, bu nobody can explain which pa ame e s we e used, e.g.,
s uc u e, colo , size, pa e ns and so on. This makes AI sys ems unp edic able o some ex en
as nobody can say unde which condi ions an e o may occu . This p oblem could be ampli ied
in au onomous sys ems.
1.2 AI-assis ed Chip Design
Al eady oday, he design o compu e chips is dependen on compu e suppo . Due o he
ex eme complexi y, he design canno be done anymo e by humans alone, i equi es design
so wa e (called elec onic design au oma ion, o EDA, so wa e).
Al eady in 2024, he machine lea ning-based p og am AlphaChip gene a ed supe human chip
layou s, e.g., esul ing in a 6.2% size educ ion o T illium Tenso P ocessing Uni (TPU)
2
compa ed o he p e ious chip gene a ion. This also educes he design ime om weeks o e en
mon hs o hou s. AlphaChip is imp o ing chip loo -planning by s epwise imp o emen o
ci cui placemen s
3
. In la e 2025, he new I onwood TPU became ope a ional which means ha
AI-assis ed chip design now p oduced one o he as es ma ke ed chips
4
. This chip will be
o e ed as a 256-chip clus e and a 9,216-chip clus e
5
.Google is designing chips “ o AI wi h
AI”
6
, while he o he companies ei he p oduce chips o AIs which makes AI-assis ed design o
a s a egic ma ke ad an age.
In pa allel, China is ad ancing in he AI ace as well, e.g., by DeepSeek, Manus, Kimi K2, Qwen,
and he chip-designing AI QiMeng
7
. QiMeng consis s o h ee laye s, a domain-specialized
La ge P ocesso Chip Model (LPCM) o no el a chi ec u es as bo om-laye , a Ha dwa e
Design Agen and a So wa e Design Agen o au oma ed ha dwa e and so wa e design in he
middle-laye , and a ious p ocesso chip design applica ions in he op-laye
8
. While he
1
A ie a e al. 2020, p.83, Chaud y/Klein 2023
2
Goldie e al. 2024
3
Goldie/Mi hoseini 2024
4
Ma ini 2025
5
Spence 2025
6
In Decembe 2025, Google Gemini e sion 3.0 was conside ed as one o he leading on ie AI models, such
as DeepSeek e sion 3.2 and Cha GPT e sion 5.2.
7
Zhang, R. e al. 2025
8
Zhang R. e al. 2025
A i icial_In elligence_and_Machine_E olu ion 4
de elope s o QiMeng see oom o u he imp o emen , QiMeng was al eady able o design
he QiMeng-Compu e P ocessing Uni (CPU)- 1 compa able o In el 486 om he1990ies,
while he second e sion QiMeng-CPU- 2 was compa able o A m Co ex A53 om he
2010ies, i.e., chip designs compa able wi h comme cial chips
9
.
The p oblem ha he AI is an opaque black-box sys em is also p esen in chip design and makes
a oo cause analysis in case o p oblems di icul
10
.
The chip design esea ch is ad ancing apidly. The i s h ee-dimensional 3D chip o accele a e
AI was p esen ed by esea che s o he S an o d Uni e si y in Decembe 2025. I is a eal 3D
chip, i.e., no simply consis ing o laye s o 2D chips, bu a monoli hic design ha can ully
u ilize he hi d dimension esul ing in a wel e- old imp o emen on eal AI wo kloads and an
imp o ed ene gy e iciency
11
.
The Shanghai-based company CHIPX eleased a pho onic quan um chip ha can accele a e AI,
he coming 6G ne wo k, and quan um compu ing de elopmen . Fo some ma hema ical
ope a ions, he chip was 1000 imes as e han con en ional chips
12
. The chip is al eady
p oduced wi h abou 12,000 wa e s a yea , each wa e wi h a ound 350 chips
13
. In he long un,
pho onic chips, i.e., using ligh ins ead o elec ons o in o ma ion ans e , could eplace
con en ional chips as hey a e as e and mo e ene gy-e icien . China al eady has he la ges
global ma ke sha e in he pho onic indus y which o iginally was led by he Uni ed S a es and
Eu ope
14
.
1.3 AI-assis ed Compu e Design
The company Quil e p esen ed he i s compu e ha was a chi ec ed, placed, ou ed, and
alida ed by a physics-d i en AI which was ained o easoning on physics, i.e., is di e en
om con en ional la ge language models
15
.
Quil e 's physics-d i en ein o cemen lea ning AI sys em designed in he P ojec Speed un a
wo king 843-componen Linux compu e a ound a NXP i.MX 8M Mini chip in 38 hou s ins ead
o he 11 weeks ha human enginee s would ha e needed o his ask and he compu e could
be success ully boo ed on he i s y
16
. The AI o Quil e au onomously comple ed 98% o
placemen , ou ing, and physics, i.e., he enginee ac ed mainly as supe iso . The ou come o
he p ojec was ha AI is now capable o designing compu e s ha ope a e in he eal wo d
en i onmen s
17
.
I he AI akes o e , chips and new compu e s will be designed by compu e s, i.e., he humans
will no know anymo e how he AI and he compu e a e wo king which makes su eillance
and con ol qui e challenging. The c ea ion o compu e s by o he compu e s which hen design
again new compu e s is he s a o he machine e olu ion.
1.4 Sel -Lea ning AI
AI lea ning and obo lea ning inc ease in simila ways: Robo lea ning has meanwhile eached
s ages o 1022 o 1024 FLOPS ( loa ing comma ope a ions) while on ie AI models cu en ly
exceeded 1024 FLOPS o hei machine lea ning
18
. The aining o AI is dependen om high-
9
Zhang R. e al. 2025
10
Chen 2025
11
Sche le 2025
12
Hi e lab 2025, Swayne 2025
13
Swayne 2025
14
EAC 2024
15
Ta di 2025: Pa ame e s used by he AI included signal in eg i y condi ions, ace impedance, ji e and skew
he mal p opaga ion, cu en -ca ying capaci y, elec omagne ic conside a ions and physical manu ac u abili y.
16
Ta di 2025
17
Ta di 2025
18
Beye e /Ki chne e al. 2ß25
A i icial_In elligence_and_Machine_E olu ion 5
quali y da a c ea ed by humans, because a aining o AI by AI-gene a ed da a only leads o a
apid loss o quali y, he model collapse.
New ways o AI aining and lea ning we e p oposed in 2025: he ne wo king and in o ma ion
exchange be ween AI agen s in he i s s ep and he open-ended e olu ion o AI agen s in he
second s ep by he so-called Da win Gödel Machine (DGM)
19
.
AI agen s a e so wa e p og ams whe e LLMs wi h easoning and disco e y abili y p o ide
inc easing au onomy o achie emen o gi en objec i es
20
. The de elopmen is apdily
ad ancing om simple wo k lows wi h LLMs o au onomously ac ing AI Agen s and mul i-
agen ic sys ems (MAS) which can un inc easingly complex p ocedu es on hei own
21
.
Resea che s om Google DeepMind no ed ha in scien i ic a eas like ma hema ics, coding, and
science, he knowledge ex ac ed om human da a is apidly app oaching a limi and ha mos
high-quali y da a sou ces we e al eady u ilized. The p oposed solu ion is ha AI sys ems lea n
om each o he and sha e hei expe ience pe manen ly.
22
Wi h he g owing p esence o AI
agen s, hey can lea n much mo e om each o he han om humans alone. DeepMind expec s
an A i icial Gene al In elligence AGI a ound 2030 and ha he e will be unlimi ed
de elopmen abo e humans.
23
The nex logical s ep was he e o e o allow he communica ion be ween AI agen s and he
in e ope abili y i AI agen s and mul i-agen sys ems by new p o ocols ha acili a e
communica ion (such as Glibbe link o communica ion ia phone), in e ac ions (as he new
Agen 2Agen A2A open p o ocol o Mic oso
24
) and o lea ning by knowledge exchange
be ween AI sys ems
25
.
Howe e , he ne wo king and knowledge exchange be ween AI sys em ampli ies he isk o
sys em e o s and mal unc ion o he AI agen ecosys em pollu ion by inaccu a e da a. Google
DeepMind has de eloped he FACTS G ounding Benchma k which co e s he model’s abili y
o access i s in e nal knowledge accu a ely, o use sea ch as a ool o e ie e in o ma ion and
syn hesize i co ec ly abili y o answe p omp s ela ed o inpu images in a ac ually co ec
manne
26
. In a s udy om Decembe 2025, Gemini 3 P o led in o e all pe o mance, wi h a
FACTS Sco e o 68.8% which showed a limi ed accu acy o da a handling by he es ed models.
Wha will happen, i mul iple agen s s a o exchange inaccu a e da a? This con ibu es o he
Mul i-Agen -Pa adox which was shown in a s udy: adding mo e agen s o a sys em makes i
slowe , mo e expensi e, and less accu a e
27
.
In mul i-agen ic AI sys ems, a malicious a acke who ge s access o one AI agen may be able
o damage u he pa s i he in uded AI agen s p o ide alse inpu o he o he agen s, esul ing
in a model pollu ion
28
.
The ques ion o how AI sys ems can endlessly imp o e hemsel es while ge ing be e a
sol ing ele an p oblems esul ed in he Da win Gödel Machine (DGM), a sel -imp o ing
19
Zhang e al. 2025
20
DSIT 2025
21
La sen 2024
22
Sil e /Su on 2025
23
Shah e al. 2025. Ano he aspec is ha he AI sys ems can exis in mul iple copies and gain knowledge in
pa allel which is no possible o human indi iduals, Hin on 2025
24
A enas/B ekelmans 2025
25
Rosenbush 2025
26
Deep Mind 2025
27
Zia 2025
28
He e al. 2024

A i icial_In elligence_and_Machine_E olu ion 6
sys em ha i e a i ely modi ies i s own code and empi ically alida es each change using
coding benchma ks
29
.
The concep o a Gödel machine was p oposed in 2007 whe e a machine should be able o p o e
a sys em imp o emen . In con as , he Da win Gödel Machine (DGM) only needs o show
empi ical e idence ha an imp o emen ook place. The DGM is a sel - e e en ial sel -
imp o emen o Coding AI Agen s
30
. The DGM s a s wi h one coding agen ha s a s sel -
modi ica ion wi h checks agains coding benchma ks. Each agen e sion is a chi ed by he
DGM and can hen be used o sel -modi ica ion h ough open-ended explo a ion. The p ocess
is going on i e a i ely. In he expe imen ha o sa e y easons was done unde s ic human
supe ision in a sandbox, he DGM au oma ically imp o ed i s coding capabili ies (e.g., be e
code edi ing ools, long-con ex window managemen , pee - e iew mechanisms), inc easing
pe o mance. The DGM is he p ac ical s ep o a sel -imp o ing AI o endless inno a ion
31
.
1.5 Fully Au oma ed Fab ica ion
Sam Al man, CEO o he Cha GPT p o ide OpenAI, expec s a wide-sp ead use o obo s in
p oduc ion o 2027 and en isions a comple ely new de elopmen pa h: “we ha e o make he
i s million humanoid obo s he old- ashioned way, bu hen hey can ope a e he en i e supply
chain—digging and e ining mine als, d i ing ucks, unning ac o ies, e c. - o build mo e
obo s, which can build mo e chip ab ica ion acili ies, da a cen e s, e c.”
32
. As a majo s ep
owa ds his ision, Foxconn and N idia in end o deploy humanoid obo s a a Foxconn plan
in Hous on o p oduce N idia GB300 AI se e s om 2026 on
33
.
1.6 Moni o ing and O e sigh o AI
The human con ol o AI is dependen om he abili y o obse e he AI. This can be done by
obse a ion o ou pu and ac ions, bu can be complemen ed by moni o ing o he chain o
hough s, i.e., he in e nal AI p o ocols ha show hei easoning ( hinking) s eps. The abili y
o humans o obse e he chain o hough o an AI is called moni o abili y
34
.
The leading Tech companies Google, Me a, OpenAI and An h opic eleased a join pape whe e
hey exp essed majo sa e y conce ns abou no el AI a chi ec u es which educe he need o
‘loud hinking’ o AI models, i.e., he need o c ea e chain o hough s wi h w i en p o ocols
o impo an easoning s eps. Wi hou hese p o ocols, i will be e y di icul o AI de elope s
o ind ou wha he AI is hinking and how decisions we e made which could esul in loss o
con ol o humans
35
.
A s udy o OpenAI p oposes o in es iga e change o hough s by asking ollow-up ques ions
o s a emen s which may esul in mo e insigh
36
. This is necessa y as on ie AI models show
hidden misalignmen (scheming) whe e he AI hides i s hough s e.g., o ge ewa ds o o a oid
ein o cemen lea ning.
37
Scheming is accompanied by o he isks o ad anced AI models like
decep ion, si ua ional awa eness (when he AI knows ha i is in a es si ua ion), powe
seeking, au onomous eplica ion and agency (goal-di ec ed beha io whe e he AI ies o push
h ough i s own goals)
38
.
29
Zhang, J. e al. 2025. The name Da win e e s o e olu ion, Gödel o ma hema ics
30
Zhang, J. e al. 2025
31
Zhang, J. e al. 2025
32
Al man 2025
33
Lee 2025
34
Ko bak e al. 2025, Guan 2025
35
Ko bak e al. 2025
36
Guan 2025, OpenAI 2025a
37
OpenAI 2025b
38
G ey/Sege ie 2025
A i icial_In elligence_and_Machine_E olu ion 7
Va ious esea che s exp ess se ious conce ns abou a po en ial e olu ion e.g., by sel -lea ning.
The key conce n is ha humans a e cu en ly no eally p epa ed o his si ua ion and may no
be able o p o ide su icien o e sigh
39
, i.e., he si ua ion may ge ou o human con ol
40
. Fo
his eason, a s ic go e nance wi h egula ions and supe ision should be es ablished
41
. I no
done, esea che s o he Machine In elligence Resea ch Ins i u e in Be keley concluded ha he
mos likely ou come will be human ex inc ion
42
. An h opic’s chie scien is Ja ed Kaplan iews
le ing AI models ain hemsel es as he mos c i ical isk o an uncon ollable in elligence
explosion which could al eady happen in 2027
43
.
2 Summa y
This wo king pape p o ided a b ie o e iew on A i icial In elligence and Machine E olu ion.
Machine E olu ion can be unde s ood as a si ua ion whe e machines au onomously c ea e new
machines which c ea e u he machines and so on. While such an e olu ion is no ye possible,
subs an ial p og ess was made in mul iple a eas which b ing machines and AI much close o
ull au onomy, sel -lea ning, and adap a ion o imp o emen .
These ad ances accele a e he echnological p og ess, bu make huma o e sigh and
in e en ions inc easingly di icul , i.e., ins ead o being in he loop, humans a e mo ed ou o
he loop. Ad ances include he AI-assis ed design o compu e chips, he AI-assis ed design o
en i e compu e s, sel -lea ning and imp o ing AI agen s in expe imen al se ings as well as
au oma ed ab ica ions. Al eady in 2024, he machine lea ning-based AI p og am AlphaChip
gene a ed supe human chip layou s and c ea ed in 2025 he new I onwood TPU as one o he
as es ma ke ed chips. China is ad ancing as well wi h he chip-designing AI QiMeng ha
al eady designed chips compa able wi h comme cial chips. The i s h ee-dimensional
monoli hic 3D chip o accele a e AI was p esen ed by Resea che s o he S an o d Uni e si y
in Decembe 2025. The Shanghai-based company CHIPX eleased a pho onic quan um chip
ha o some ma hema ical ope a ions was 1000 imes as e han con en ional chips. Quil e 's
physics-d i en ein o cemen lea ning AI sys em designed in he P ojec Speed un a wo king
843-componen Linux compu e in only 38 hou s and au onomously comple ed 98% o
placemen , ou ing, and physics.
New ways o AI aining and lea ning we e p oposed in 2025; he ne wo king and in o ma ion
exchange be ween AI agen s in he i s s ep and he open-ended e olu ion o AI agen s in he
second s ep by he so-called Da win Gödel Machine (DGM). Resea che s om Google
DeepMind p oposed ha AI sys ems lea n om each o he and sha e hei expe ience
pe manen ly wi h unlimi ed de elopmen abo e humans, bu ne wo king and knowledge
exchange be ween AI sys em may ampli y he isk o sys em e o s and mal unc ion o he AI
agen ecosys em pollu ion by inaccu a e da a. In 2025, he Da win Gödel Machine (DGM) was
de eloped, a sel -imp o ing Coding AI Agen sys em ha i e a i ely modi ies i s own code and
empi ically alida es each change using coding benchma ks which was e ec i e in
expe imen s. The sp ead o obo s may open he way in o ully au oma ed ab ica ions.
The human con ol o AI can be done by obse a ion o ou pu and ac ion, bu can be
complemen ed by moni o ing o he chain o hough s, i.e., he in e nal AI p o ocols ha show
hei easoning ( hinking) s eps. The abili y o obse e he chain o hough o an AI is called
moni o abili y. The e a e conce ns ha no el AI a chi ec u es may educe he moni o abili y
despi e indings o hidden misalignmen (scheming) whe e on ie AI models hide hough s
39
Ba ne /Sche 2025, Losey 2025, MIRI 2025
40
Adewumi e al. 2025, Tonsen 2025
41
Adewumi e al. 2025, Ba ne /Sche 2025, Losey 2025, MIRI 2025,Tonsen 2025
42
Ba ne /Sche 2025, MIRI 2025
43
Landymo e 2025
A i icial_In elligence_and_Machine_E olu ion 8
e.g., o ge ewa ds o o a oid ein o cemen lea ning. Va ious esea che s exp ess se ious
conce ns abou a po en ially uncon ollable e olu ion e.g., by sel -lea ning. The key conce n is
ha humans a e cu en ly no eally p epa ed o his si ua ion and may no be able o p o ide
su icien o e sigh , i.e., he si ua ion may ge ou o human con ol and o his eason, a s ic
go e nance wi h egula ions and supe ision should be es ablished.
3 Re e ences
Adewumi, T. e al. (2025): AI Mus no be Fully Au onomous. Machine Lea ning G oup,
EISLAB, Sweden a Xi :2507.23330 1 [cs.AI] 31 Jul 2025
Al man, S. (2025): Gen le Singula i y. Sam Al man’s Blog 10 June 2025
A enas, Y., B ekelmans, B. (2025): Empowe ing mul i-agen apps wi h he open Agen 2Agen
(A2A) p o ocol. Mic oso News 07 May 2025
A ie a, A.B. e al. (2020): Explainable A i icial In elligence (XAI): Concep s, axonomies,
oppo uni ies, and challenges owa d esponsible AI. In o ma ion Fusion 58(2020), p. 82–111
Beye e , J., Ki chne , E. e al. (2025): KI in de Robo ik. Flexible und anpassba e Sys eme
du ch in e ak i es Le nen. DOI: h ps://doi.o g/10.48669/pls_2025
Ba ne , P., Sche , A. (2025): AI Go e nance o A oid Ex inc ion: The S a egic Landscape
and Ac ionable Resea ch Ques ions Machine In elligence Resea ch Ins i u e Be keley,
Technical Go e nance Team May 2025
Chaudh y, H., Klein, L. (2023): A i icial In elligence and Nuclea Weapons: P oblem
Analysis and US Policy Recommenda ions. policy@ u u eo li e.o g 14 h No embe 2023
Fu u e o Li e Ins i u e (FLI)
Chen, N. (2025): In eg a ing AI in o Semiconduc o Design and Fab ica ion: Me hodologies,
Challenges and Fu u e P ospec s. ITM Web o Con e ences 78, 03007 (2025)
h ps://doi.o g/10.1051/i mcon /20257803007 CSEIT 2025
Deep Mind (2025): FACTS Benchma k Sui e: Sys ema ically e alua ing he ac uali y o
la ge language models. The FACTS eam o Google Deep Mind. Decembe 9, 2025
DSIT (2025): AI Insigh s - Agen ic AI. Pape o he Depa men o Science, Inno a ion and
Technology and he Go e nmen Digi al Se ice o he Uni ed Kingdom
EAC (2024): EAC In e na ional Consul ing Execu i e Repo . Poli ical S ee ing P ocesses in
China in Co e Segmen s o he Pho onics Indus y. Commissioned by he Eu opean
Technology Pla o m Pho onics21
Goldie, A. e al. (2024): Addendum: A g aph placemen me hodology o as chip design
doi.o g/10.1038/s41586-024-08032-5 Published online 26 Sep embe 2024
Goldie, A., Mi hoseini, A. (2024): How AlphaChip ans o med compu e chip design
Science Sep embe 26, 2024
G ey, M., Sege ie, C.R. (2025): The AI Risk Spec um F om Dange ous Capabili ies o
Exis en ial Th ea s. F ench Cen e o AI Sa e y (CeSIA) 20 Augus 2025
Guan, M.Y. e al. (2025): Moni o ing Moni o abili y. OpenAI Decembe 18, 2025
He, Y. e al. (2024): Secu i y o AI Agen s a Xi :2406.08689 3 [cs.CR] 17 Dec 2024
Hin on, G. (2025): KI wi d bald deu lich in elligen e als wi sein. Handelsbla 23-25 Mai
2025, S.26-27
A i icial_In elligence_and_Machine_E olu ion 9
Hi e lab (2025): China’s CHIPX Pho onic Chip P oduc ion Line Se o Accele a e AI, 6G,
and Quan um Compu ing De elopmen : SCMP h ps://hi e lab.com/china-chipx-pho onic-
chip-ai-6g-quan um-compu ing/ June 17, 2025
Khan, S.M., Mann, A., Pe e son, D. (2021): The Semiconduc o Supply Chain: Assessing
Na ional Compe i i eness Cen e o Secu i y and Eme ging Technology (CSET) Issue B ie
Janua y 2021
Ko bak, T. e al. (2025): Chain o Though Moni o abili y: A New and F agile Oppo uni y o
AI Sa e y. a Xi :2507.11473 1 [cs.AI] 15 Jul 2025
Landymo e, F. (2025): An h opic's Chie Scien is Says We' e Rapidly App oaching he
Momen Tha Could Doom Us All. Fu u ism Published Dec 13, 2025
La sen, B. e al. (2024): Na iga ing he AI F on ie : A P ime on he E olu ion and Impac o
AI - Wo ld Economic Fo um Ai Go e nance Alliance Sa e Sys ems and Technologies Wo king
G oup - Wo ld Economic Fo um Whi epape
Lee, W.-Y. (2025): Reu e s Exclusi e: N idia, Foxconn in alks o deploy humanoid obo s a
Hous on AI se e making plan . June 20, 2025 h ps://www. eu e s.com/wo ld/china/n idia-
oxconn- alks-deploy-humanoid- obo s-hous on-ai-se e -making-plan -2025-06-20/
Losey, R. (2025): A aid o AI? Lea n he Se en Ca dinal Dange s and How o S ay Sa e – AI
E hics. Ap il 25, 2025
Ma ini, A. (2025): 3 hings o know abou I onwood, ou la es TPU. Google Cloud, No 25,
2025
MIRI (2025): The P oblem - Machine In elligence Resea ch Ins i u e Be keley Oc obe
2025We
Open AI (2025a): E alua ing chain-o - hough moni o abili y OpenAI Resea ch Publica ion
Decembe 18, 2025
OpenAI (2025b): De ec ing and educing scheming in AI models. Publica ion Resea ch
Sep embe 17, 2025
Sche le , I. (2025): Resea che s un eil g oundb eaking 3D chip o accele a e AI. S an o d
Repo Decembe 10, 2025 h ps://news.s an o d.edu/s o ies/2025/12/monoli hic-3d-chip-
ound y-b eak h ough-ai
Shah, R. e al. (2025): An App oach o Technical AGI Sa e y and Secu i y Google DeepMind
Resea ch Pape
Swayne, M. (2025): China’s New Pho onic Quan um Chip P omises 1,000-Fold Gains o
Complex Compu ing Tasks. The Quan um Inside h ps:// hequan uminside .com/au ho /ma -
swayne/) No embe 15, 2025
Ta di , A. (2025): Quil e Shows AI Can Now Design Re Ha dwa e Wi h he Wo ld’s Fi s
Mac Enginee ed Compu e . Published Decembe 10, 2025 By An oine Ta di , CEO &
Founde o Uni e.AI
Zhang, J. e al. (2025): Da win Gödel Machine (DGM) – Open-ended E olu ion o Sel -
imp o ing Agen s. a Xi :2505.22954 2 [cs.AI] 26 Sep 2025
Zhang, R. e al. (2025): QiMeng: Fully Au oma ed Ha dwa e and So wa e Design o
P ocesso Chip. a Xi :2506.05007 1 [cs.AR] 5 Jun 2025
Zia, T. (2025): The Mul i-Agen Pa adox: Why Mo e Agen s Can Lead o Wo se Resul s.
Uni e.AI Decembe 22, 2025