Discrete Information Geometry of Computational Universes:\\ Relative Entropy, Fisher Structure, and Task-Aware Distances

Disc e e In o ma ion Geome y o Compu a ional
Uni e ses:
Rela i e En opy, Fishe S uc u e, and Task-Awa e
Dis ances
Haobo Ma1Wenlin Zhang2
1Independen Resea che
2Na ional Uni e si y o Singapo e
No embe 24, 2025
Abs ac
Wi hin he axioma ic amewo k o “compu a ional uni e se” Ucomp = (X, T,C,I),
complexi y geome y cha ac e izes “how much ime/cos is needed o each a con-
igu a ion.” Howe e , complexi y geome y alone is insu icien o desc ibe “wha
quali y o in o ma ion is gained o hese cos s.” To add ess his, we cons uc a
“disc e e in o ma ion geome y” heo y compa ible wi h compu a ional uni e ses
wi hin a ully disc e e se ing.
We i s in oduce obse a ion ope a o amilies O={Oj}j∈J, whe e each Oj
maps con igu a ion x∈X o a p obabili y dis ibu ion p(j)
xo e some ini e ou come
se . Unde ixed asks o obse a ion schemes, hese dis ibu ions p o ide “ isible
in o ma ion s a es” o each con igu a ion x. We de ine ask-awa e ela i e en opy
s uc u es DQ(x∥y) and de i e in o ma ion dis ances such as Jensen–Shannon dis-
ance dJS,Q(x, y). These dis ances locally induce disc e e Fishe s uc u es: nea a
e e ence con igu a ion x0, he Hessian o second-o de ela i e en opy DQ(x∥x0)
yields a disc e e in o ma ion me ic enso a ound x0.
We p o e ha unde na u al egula i y assump ions, disc e e in o ma ion s uc-
u es con e ge in app op ia e limi s o a Riemannian in o ma ion mani old (SQ, gQ)
wi h Fishe - ype me ic gQ. Co espondingly, “in o ma ion geome y on con igu a-
ion space” is ealized h ough mapping ΦQ:X→ SQsending each con igu a ion
x o i s isible in o ma ion s a e. We u he discuss olume g ow h o in o ma-
ion balls Bin o
R(x0) and “in o ma ion dimension,” p o iding gene al inequali ies
be ween in o ma ion dimension and complexi y dimension, cha ac e izing “limi s
o in o ma ion esolu ion achie able unde gi en complexi y budge s.”
Finally, we cons uc a ask-awa e in o ma ion–complexi y join ac ion AQwhose
local Eule –Lag ange equa ions p o ide local desc ip ions o op imal compu a-
ional ajec o ies “maximizing in o ma ion quali y” unde ini e ime budge s,
es ablishing disc e e in o ma ion geome y ounda ions o subsequen comple e
“ ime–in o ma ion–complexi y a ia ional p inciples.”
Keywo ds: Compu a ional Uni e se, In o ma ion Geome y, Rela i e En opy, Fishe
In o ma ion, Task-Awa e Dis ance, In o ma ion Dimension
MSC 2020: 94A17, 62B10, 68Q15, 53B12
1
1 In oduc ion
In he axioma ic sys em o compu a ional uni e ses, he uni e se is abs ac ed as disc e e
con igu a ion space X, one-s ep upda e ela ion T, single-s ep cos C, and in o ma ion
quali y unc ion I, whe eby any ac ual compu a ional p ocess co esponds o a ini e pa h
on he con igu a ion g aph, wi h complexi y dis ance d(x, y) cha ac e izing minimal cos
equi ed om x o y. P e ious wo k has cons uc ed “disc e e complexi y geome y”
based on his ounda ion, desc ibing p oblem di icul y and complexi y ho izons h ough
complexi y ball olumes and disc e e Ricci cu a u e.
Howe e , complexi y geome y conce ns “how a a eled” a he han “wha was
obse ed.” Unde s anding geome ic s uc u e o “in o ma ion quali y” in compu a ional
uni e ses equi es in oducing ano he dimension: obse a ion and asks. Speci ically,
“use ul in o ma ion” o he same con igu a ion xdepends no only on xi sel bu also on
how we ead i ou and wha asks we ca e abou . Di e en asks co espond o di e en
“in o ma ion geome ies,” and compu a ional p ocess ajec o ies on hese in o ma ion
geome ies uly e lec “how much in o ma ion we ex ac ed in gi en ime.”
This pape ’s goal is o es ablish ask- ela ed “disc e e in o ma ion geome y” o com-
pu a ional uni e ses in comple ely disc e e se ings:

A disc e e le el, assign each con igu a ion xa p obabili y s a e pxde e mined
by obse a ion scheme, cons uc ing in o ma ion dis ances using ela i e en opy,
Jensen–Shannon dis ance, e c.;

Locally, ob ain Fishe - ype me ics h ough second-o de expansion o ela i e en-
opy, es ablishing disc e e in o ma ion mani old s uc u es;

Globally, cha ac e ize “complexi y o dis inguishable s a es in he uni e se unde
ce ain asks” h ough in o ma ion ball olumes and in o ma ion dimension.
Mo e impo an ly, in o ma ion geome y mus coo dina e wi h complexi y geome y:
complexi y geome y ells us which con igu a ions we can mo e be ween unde esou ce
cons ain s; in o ma ion geome y ells us how much in o ma ion gain hese mo emen s
b ing in “ ask- ele an s a e spaces.” Coupling o he wo ul ima ely leads o uni ied
“ ime–in o ma ion–complexi y ac ion.”
Main s uc u e: Sec ion 2 in oduces obse a ion ope a o s and ask-awa e disc e e
ela i e en opy s uc u es. Sec ion 3 cons uc s disc e e in o ma ion dis ances and in o -
ma ion balls, de ining in o ma ion dimension. Sec ion 4 discusses local Fishe s uc u e
and in o ma ion mani old limi s. Sec ion 5 p o ides in o ma ion–complexi y inequali ies
and a ask-awa e join ac ion p o o ype. Appendices p o ide de ailed p oo s o main
p oposi ions and heo ems.
2 Obse a ion Ope a o s and Task-Awa e Rela i e
En opy
This sec ion in oduces obse a ion ope a o s and ask-awa e p obabili y s uc u es a
con igu a ion le el o compu a ional uni e ses.
2
2.1 Obse a ion Ope a o Families and Visible S a es
In compu a ional uni e se Ucomp = (X, T,C,I), con igu a ion x∈Xis he uni e se’s
in e nal s a e. Obse e s can only access i h ough ini e expe imen al o eadou p o-
cesses wi hin ime windows. We in oduce obse a ion ope a o amilies o cha ac e ize
his.
De ini ion 2.1 (Obse a ion Ope a o Family).Le (Yj)j∈Jbe a amily o ini e ou come
se s. An obse a ion ope a o amily is a se o mappings
O={Oj:X→∆(Yj)}j∈J,
whe e ∆(Yj) is he p obabili y simplex on Yj, and o each x∈X,j∈J,Oj(x) = p(j)
xis
he ou come dis ibu ion on esul se Yj om one expe imen .
In ui i ely, Ojdesc ibes an obse a ion p ocess implemen able on con igu a ion x,
wi h ou pu dis ibu ion p(j)
xbeing s a is ical in o ma ion he obse e “sees” on ha
con igu a ion.
To a oid edundancy, we o en deno e asks o obse a ion schemes as ini e subse s
Q⊂J, de ining “join isible s a es” unde ha ask.
De ini ion 2.2 (Join Visible S a e Unde Task Q).Fo gi en ini e ask se Q⊂J,
de ine isible ou come se
YQ=Y
j∈Q
Yj,
and de ine con igu a ion x’s join isible s a e as a join dis ibu ion p(Q)
xon YQ. The
simples cons uc ion assumes independen obse a ions:
p(Q)
x(y) = Y
j∈Q
p(j)
x(yj), y = (yj)j∈Q∈YQ.
Mo e gene ally, known coupling s uc u es be ween di e en obse a ions can be al-
lowed, whe e p(Q)
xis gi en by a ask-speci ic obse a ion model. This pape mainly con-
side s independen cases.
2.2 Task-Awa e Rela i e En opy
A e ixing ask Q, each con igu a ion xis mapped o p obabili y dis ibu ion p(Q)
x∈
∆(YQ). This allows us o in oduce ela i e en opy o ask Q.
De ini ion 2.3 (Rela i e En opy Unde Task Q).Fo con igu a ions x, y ∈X, i o all
z∈YQ,p(Q)
y(z)>0 implies p(Q)
x(z)>0, de ine
DQ(x∥y) = X
z∈YQ
p(Q)
x(z) log p(Q)
x(z)
p(Q)
y(z),
o he wise de ine DQ(x∥y) = +∞.
DQ(x∥y) is he “dis inguishabili y deg ee” o con igu a ions xand yunde ask Q:
la ge alues mean xand ya e mo e “in o ma ionally dis an ” unde ha ask.
Clea ly, DQ(x∥y)≥0, and DQ(x∥y) = 0 i and only i p(Q)
x=p(Q)
y.
No e DQis gene ally no symme ic and does no sa is y iangle inequali y, hus is
no a me ic. To ob ain in o ma ion dis ance, we use symme ized o ms de i ed om
DQ.
3
3 Disc e e In o ma ion Dis ance, In o ma ion Balls,
and In o ma ion Dimension
This sec ion de ines ask-awa e in o ma ion dis ances and in o ma ion balls based on DQ,
in oducing he concep o in o ma ion dimension.
3.1 Jensen–Shannon In o ma ion Dis ance
Fo p obabili y s uc u es on ini e se s, Jensen–Shannon di e gence p o ides na u al
symme iza ion, wi h i s squa e oo being a me ic. We adop simila cons uc ions in
ask in o ma ion con ex s.
De ini ion 3.1 (Jensen–Shannon Di e gence and In o ma ion Dis ance Unde Task Q).
Fo x, y ∈X, de ine mix u e dis ibu ion
m(Q)
x,y =1
2p(Q)
x+p(Q)
y,
Jensen–Shannon di e gence
JSQ(x, y) = 1
2Dp(Q)
x∥m(Q)
x,y +1
2Dp(Q)
y∥m(Q)
x,y ,
and in o ma ion dis ance
dJS,Q(x, y) = q2 JSQ(x, y).
whe e D(·∥·) is s anda d Kullback–Leible ela i e en opy.
By known esul s, dJS,Q is a me ic on X ela i e o ask Q: sa is ies non-nega i i y,
symme y, iangle inequali y, and dJS,Q(x, y) = 0 i and only i p(Q)
x=p(Q)
y.
We call dJS,Q he ask-awa e in o ma ion dis ance.
3.2 In o ma ion Balls and In o ma ion Volume
De ini ion 3.2 (In o ma ion Ball and In o ma ion Volume).Fo e e ence con igu a ion
x0∈X, ask Q, and adius R > 0, de ine in o ma ion ball
Bin o,Q
R(x0) = {x∈X:dJS,Q(x, x0)≤R},
in o ma ion olume
Vin o,Q
x0(R) = Bin o,Q
R(x0).
This olume cha ac e izes he numbe o con igu a ions “in o ma ion dis ance a mos
R om x0” om in o ma ion geome y pe spec i e o ask Q.
3.3 In o ma ion Dimension
Simila o complexi y dimension, we de ine in o ma ion dimension using g ow h a e o
in o ma ion ball olumes.
4
De ini ion 3.3 (In o ma ion Dimension).Fo gi en ask Qand e e ence x0, de ine uppe
in o ma ion dimension
dimin o,Q(x0) = lim sup
R→∞
log Vin o,Q
x0(R)
log R,
lowe in o ma ion dimension
dimin o,Q(x0) = lim in
R→∞
log Vin o,Q
x0(R)
log R.
I he wo a e equal, hei common alue is called in o ma ion dimension, deno ed
dimin o,Q(x0).
In ui i ely, dimin o,Q(x0) desc ibes g ow h o de o dis inguishable con igu a ion num-
be s wi hin in o ma ion dis ance adius Runde ask Q. I in o ma ion dimension is ini e,
ask Qac ually only in ol es some low-dimensional in o ma ion s uc u e; i in ini e, he
ask has high complexi y and high dis inguishabili y a in o ma ion le el.
3.4 P elimina y Rela ionship Be ween In o ma ion and Com-
plexi y Dimensions
Le complexi y dis ance be dcomp(x, y), complexi y ball olume Vcomp
x0(T), complexi y
dimension dimcomp(x0). Gene ally, in o ma ion dimension and complexi y dimension ha e
no simple equali y, bu we can p o ide ough inequali y cha ac e izing “uppe bound o
in o ma ion dis inc ion abili y unde complexi y cons ain s.”
P oposi ion 3.4 (In o ma ion Volume Cons ained by Complexi y Volume).Assume
he e exis s cons an LQ>0such ha o all adjacen con igu a ions x, y (i.e., (x, y)∈T),
dJS,Q(x, y)≤LQC(x, y),
hen he e exis s cons an C > 0such ha o all R > 0,
Vin o,Q
x0(R)≤Vcomp
x0R
C.
Thus
dimin o,Q(x0)≤dimcomp(x0).
P oo in Appendix A.1. This inequali y shows ha unde local Lipschi z condi ions,
in o ma ion geome y “dimension” does no exceed complexi y geome y “dimension,”
con o ming o in ui ion: compu able dis inguishing abili y is limi ed by achie able com-
plexi y esou ces.
4 Local Fishe S uc u e and In o ma ion Mani old
Limi s
This sec ion in oduces local Fishe s uc u e unde ask Q’s in o ma ion dis ance, con-
s uc ing Riemannian in o ma ion me ics nea e e ence con igu a ions h ough second-
o de expansion o ela i e en opy. We hen discuss how in o ma ion geome y con e ges
o Fishe mani olds in limi s when con inuous pa ame e iza ions exis be ween con igu-
a ions.
5

4.1 Second-O de Expansion o Rela i e En opy and Disc e e
Fishe Ma ix
Le x0∈Xbe e e ence con igu a ion wi h isible s a e p0=p(Q)
x0unde ask Q. Conside
se e al adjacen con igu a ions x(1), . . . , x(k)wi h isible s a es pi=p(Q)
x(i). Assume he e
exis s local pa ame e iza ion
θ∈Θ⊂Rk7−→ p(θ)∈∆(YQ),
such ha p(0) = p0, and each pican be w i en as p(εei), whe e eiis s anda d basis and
ε > 0 is small pa ame e . We can use θas “in o ma ion coo dina es” nea x0.
De ini ion 4.1 (Local Task Fishe Ma ix).Unde abo e se ing, de ine local Fishe
in o ma ion ma ix o ask Qas
g(Q)
ij (0) = X
z∈YQ
p0(z)∂θilog p(θ)(z)θ=0 ∂θjlog p(θ)(z)θ=0.
This is he Fishe in o ma ion ma ix a θ= 0, comple ely de e mined by local a i-
a ion o p(θ).
Theo em 4.2 (Fishe Fo m o Rela i e En opy Second-O de Expansion).Unde abo e
se ing and s anda d egula i y condi ions, o su icien ly small θ∈Θ,
DQθ∥0=Dp(θ)∥p(0)=1
2X
i,j
g(Q)
ij (0) θiθj+o(|θ|2).
P oo in Appendix B.1. This heo em shows ask-awa e ela i e en opy locally has
s anda d Fishe second-o de s uc u e: i s Hessian yields a local Riemannian in o ma ion
me ic.
4.2 In o ma ion Mani olds and Con igu a ion- o-In o ma ion Map-
ping
The abo e discussion is based only on ini ely many adjacen con igu a ions and local
pa ame e iza ion. Howe e , in many cases, he se o isible s a es {p(Q)
x:x∈X}o
con igu a ion space Xunde ask Qcan be app oxima ed by some con inuous pa ame e
mani old SQ.
Assump ion 4.3 (Mani old S uc u e o Task Visible S a es).The e exis s ini e-dimensional
mani old SQwi h embedding map
ΨQ:SQ,→∆(YQ),
and mapping
ΦQ:X→ SQ,
such ha :
1. Fo each x∈X,p(Q)
xapp oxima es ΨQ(ΦQ(x));
2. S anda d Fishe in o ma ion me ic on SQ ia ΨQis consis en wi h second de i a-
i e o ela i e en opy.
6
Unde his assump ion, we can iew SQas “in o ma ion mani old o ask Q,” while
ΦQp o ides mapping om con igu a ion space X o in o ma ion mani old.
De ini ion 4.4 (Task In o ma ion Mani old and In o ma ion Me ic).Unde Assump ion
4.3, he in o ma ion mani old o ask Qis (SQ, gQ), whe e gQis Fishe in o ma ion me ic.
Fo con igu a ion x∈X, i s in o ma ion geome y posi ion is ΦQ(x)∈ SQ.
4.3 Consis ency o In o ma ion Dis ance and Fishe Dis ance
Unde sui able egula i y condi ions, Jensen–Shannon in o ma ion dis ance dJS,Q nea x0
is consis en wi h Fishe dis ance.
Theo em 4.5 (Local In o ma ion Dis ance Consis ency).Le x, x0∈Xsuch ha ΦQ(x0) =
θ0,ΦQ(x) = θ, wi h θclose o θ0. Then
dJS,Q(x, x0) = q(θ−θ0)⊤gQ(θ0)(θ−θ0) + o(|θ−θ0|).
P oo in Appendix B.2. This heo em shows ha in local coo dina es, Jensen–Shannon
in o ma ion dis ance is i s -o de equi alen o geodesic dis ance induced by Fishe me -
ic, hus Riemannian in o ma ion geome y o SQis locally compa ible wi h disc e e
in o ma ion geome y on X.
5 In o ma ion–Complexi y Inequali y and Task-Awa e
Ac ion
This sec ion p o ides in o ma ion–complexi y inequali ies a he in e sec ion o disc e e
in o ma ion geome y and complexi y geome y, cons uc ing a ask-awa e join ac ion
p o o ype.
5.1 In o ma ion–Complexi y Inequali y
P oposi ion 3.4 al eady p o ided gene al cons ain be ween in o ma ion ball olume
and complexi y ball olume. We can s eng hen his o a local “in o ma ion g adi-
en –complexi y g adien ” ela ion in he in o ma ion mani old amewo k.
Le γ= (x0, x1, . . . , xn) be a complexi y sho es pa h wi h complexi y leng h
C(γ) =
n−1
X
k=0
C(xk, xk+1),
wi h co esponding in o ma ion pa h ΦQ(γ) = (θ0, θ1, . . . , θn) and in o ma ion dis ance
LQ(γ) =
n−1
X
k=0
dSQθk, θk+1,
whe e dSQis geodesic dis ance induced by Fishe me ic.
Unde local Lipschi z assump ions, we ha e he ollowing p oposi ion.
7
P oposi ion 5.1 (Local In o ma ion–Complexi y Lipschi z Inequali y).I he e exis s
cons an Lloc
Q>0such ha o all adjacen con igu a ions x, y (i.e., (x, y)∈Twi h x, y
in some local egion),
dSQΦQ(x),ΦQ(y)≤Lloc
QC(x, y),
hen o any local pa h γ,
LQ(γ)≤Lloc
QC(γ).
In pa icula , minimal in o ma ion dis ance and minimal complexi y dis ance sa is y
dSQΦQ(x0),ΦQ(x)≤Lloc
Qdcomp(x0, x).
P oo in Appendix C.1. This inequali y shows ha in local egions, “in o ma ion
displacemen ” is con olled by “complexi y displacemen ,” wi h complexi y p o iding
esou ce uppe bound o in o ma ion geome y.
5.2 Task-Awa e Join Ac ion
To uni y complexi y geome y and in o ma ion geome y, we cons uc a ask-awa e dis-
c e e ac ion o e alua ing “cos -e ec i eness” o compu a ional pa hs unde gi en ask
Q.
De ini ion 5.2 (Join Ac ion P o o ype o Task Q).Le γ= (x0, x1, . . . , xn) be a pa h
wi h complexi y leng h C(γ) and e minal in o ma ion quali y IQ(xn) (quali y unc ion
de ined by ask). De ine join ac ion o ask Q:
AQ(γ) = αC(γ)−βIQ(xn),
whe e α, β > 0 balance complexi y and in o ma ion.
In con inuous limi s, in oducing ime pa ame e on in o ma ion mani old (SQ, gQ)
and complexi y mani old (M, G), le ing con igu a ion pa h x( ) and in o ma ion pa h
θ( ) = ΦQ(x( )), wi h complexi y eloci y qGab(θ)˙
θa˙
θband in o ma ion quali y IQ(θ(T)),
he con inuous o m o join ac ion is
AQ[θ(·)] = ZT
0
αqGab(θ( )) ˙
θa( )˙
θb( ) d −β IQ(θ(T)).
This ac ion balances “complexi y leng h o pa h” and “in o ma ion gain a e minal,”
wi h minimizing ajec o ies co esponding o op imal in o ma ion acquisi ion s a egies
unde esou ce cons ain s. Speci ic o ms o Eule –Lag ange equa ions o disc e e and
con inuous e sions a e le o u u e wo k; his pape only p o ides s uc u al p o o ype.
6 Conclusion
Unde disc e e axioma ic amewo k o compu a ional uni e ses, his pape in oduces ob-
se a ion ope a o amilies and ask-awa e ela i e en opy s uc u es, cons uc s disc e e
in o ma ion dis ances, in o ma ion balls, and in o ma ion dimension, and locally ob ains
Fishe in o ma ion ma ices h ough second-o de expansion o ela i e en opy, es ablish-
ing he concep o ask in o ma ion mani old (SQ, gQ). Th ough mapping ΦQ:X→ SQ,
8
con igu a ion space o compu a ional uni e se is embedded unde ask Qin o a ini e-
dimensional in o ma ion mani old, wi h in o ma ion dis ance locally compa ible wi h dis-
c e e Jensen–Shannon dis ance, and in o ma ion dimension sa is ying na u al inequali y
wi h complexi y dimension.
Based on hese s uc u es, we p opose in o ma ion–complexi y Lipschi z inequali y
and p o o ype ask-awa e join ac ion, p o iding disc e e in o ma ion geome y oun-
da ions o cons uc ing comple e “ ime–in o ma ion–complexi y a ia ional p inciples”
unde uni ied ime scales. Nex s eps will combine his pape ’s in o ma ion geome y wi h
p e ious complexi y geome y, sys ema ically cons uc ing compu a ional wo ldlines on
join mani old (M, G;SQ, gQ) and in e acing wi h bounda y ime geome y and uni ied
sca e ing ime scales o physical uni e ses.
A P oo o In o ma ion–Complexi y Dimension In-
equali y
A.1 P oo o P oposi ion 3.4
P oposi ion Res a emen
Assume he e exis s cons an LQ>0 such ha o all adjacen con igu a ions x, y,
dJS,Q(x, y)≤LQC(x, y).
Then he e exis s cons an C > 0 such ha o all R > 0,
Vin o,Q
x0(R)≤Vcomp
x0R
C,
hus
dimin o,Q(x0)≤dimcomp(x0).
P oo
Fo any x∈Bin o,Q
R(x0), by de ini ion dJS,Q(x, x0)≤R. Take any complexi y sho es
pa h γ= (x0, x1, . . . , xn) om x0 o xwi h cos C(γ) = dcomp(x0, x).
By iangle inequali y and local Lipschi z condi ion,
dJS,Q(x, x0)≤
n−1
X
k=0
dJS,Q(xk, xk+1)≤LQ
n−1
X
k=0
C(xk, xk+1) = LQC(γ).
I dJS,Q(x, x0)≤R, hen dcomp(x0, x)≤R
LQ.
Thus Bin o,Q
R(x0)⊆Bcomp
R/LQ(x0).
Taking C=LQgi es equi ed inclusion, hus Vin o,Q
x0(R)≤Vcomp
x0R
C.
Taking uppe limi as R→ ∞ yields dimension inequali y.
9