Poster for "On-demand Memory Compression of Stream Aggregates through Reinforcement Learning"

Enginee ing'and'
Physical'Sciences'
Resea ch'Council'
G an 'numbe '
EP/X029174/1
Ho izon'Eu ope'2021-2027'
F amewo k'P og amme'
G an 'Ag eemen 'numbe '101072456
Disclaime :+Funded+by+ he+Eu opean+Union.+
Views+and+opinions+exp essed+a e+howe e + hose+o +
he+au ho (s)+only+and+do+no +necessa ily+ e lec + hose+
o + he+EU.+The+EU+canno +be+held+ esponsible+ o + hem.
Funded&by
he&Eu opean&Union
Chalme s Uni e si y o Technology
and Uni e si y o Go henbu g,
Sweden
ICPE 2025
Resea ch T ack
S eam P ocessing and Agg ega es Rein o cemen Lea ning
E alua8on
Usecases and Se up
SPE
Con olle
En i onmen
!s a e 𝑠
"ac'on 𝑎
RL Agen
# ewa d 𝑟
good ac'on
posi' e ewa d
bad ac'on
nega' e ewa d
- 𝑖𝑛𝑝𝑢𝑡'𝑟𝑎𝑡𝑒 - 𝑡ℎ𝑟𝑜𝑢𝑔ℎ𝑝𝑢𝑡
- 𝑜𝑢𝑡𝑝𝑢𝑡'𝑟𝑎𝑡𝑒 - 𝑙𝑎𝑡𝑒𝑛𝑐𝑦
- 𝑛/𝑐 𝑟𝑎𝑡𝑖𝑜 - 𝐶𝑃𝑈'𝑐𝑜𝑛𝑠𝑢𝑚𝑝𝑡𝑖𝑜𝑛
-𝑙𝑎𝑡𝑒𝑛𝑐𝑦
-
𝑛/𝑐'𝑟𝑎𝑡𝑖𝑜
- #𝑠𝑡𝑒𝑝𝑠'𝑝𝑒𝑟'𝑒𝑝𝑖𝑠𝑜𝑑𝑒
send
da a
A
8:00
20
A
8:03
15
F
F
F
F
F
F
Inpu s eam S eam P ocessing Engine (SPE)
Can un dis ibu ed/in pa allel
à sp ead in he Cloud-IoT con'nuum
Di ec ed Acyclic G aph
Ou pu s
𝐴𝑊𝐴,𝑊𝑆, 𝑆, 𝑓
!, 𝑓
"##, 𝑓
$%&, 𝑓
'(
Func'on 𝑓
!𝑡
𝑊𝐴 (window ad ance)
𝑊𝑆 (window size)
Func'on 𝑓
"## Γ,𝑡
Func'on 𝑓
$%& Γ
Func'on 𝑓
'( Γ,𝑡 (op .)
e en &me
size
ad ance
emo e 𝒕
ou pu
Agg ega e
S eam 𝑆
ØEn i onmen
oin e ace o connec he SPE and RL Agen
ØAgen
oimplemen aining algo i hm by Neu al Ne wo k (DQN)
oge an ac'on o in e ac wi h he en i onmen
ØRewa d
o eedback o he Agen o ein o ce good ac'ons
RL agen
ewa d
⓵
⓶
⓷
en i onmen
ØLinea Road benchma k
oVehicles a elling in highways epo hei posi'on/speed
oEach ehicle epo s i s posi'on e e y 30 seconds
oAgg ega e: coun he numbe o non-consecu' e s ops
oWS = 10 mins, WA = 5 secs
Ø Syn he;c (s ess- es )
oDa a is gene a ed ollowing a saw oo h wa e whose peaks’
alues and dis ances a e chosen andomly
oThe key aY ibu e is gene a ed om a Gaussian dis ibu'on
wi h changing 𝜇/𝜎)
oAgg ega e: pe o m ma h ope a'ons on a andom alue
ca ied by each uple
oWS = 15 mins, WA = 1 sec
ØSe up
oJa a (OpenJDK 17.0.7), Py hon 3.7.6
oSPE: Lieb e
oComp ession lib a y: snappy
oAgen : openAI Gym
o120 episodes wi h maximum 1000 s eps o each
On-demand Memo y Comp ession o S eam
Agg ega es h ough Rein o cemen Lea ning
Compa ison discussions o he Agen wi h diffe en comp ession le els
Scalabili y discussions o he Agg ega e
ØWi hou an Agen ( op):
oa e age CPU cons.: 0.33 (Linea Road), 0.59 (Syn he ic)
oa e age la ency: 0.98s (Linea Road), 0.53s (Syn he ic)
ØWi h an Agen (bo om):
odi . in CPU cons. and la ency a e almos 0
I does no become a scaling bo5leneck o
he Agg ega e by in oducing he Agen .
Ø Linea Road (WS = 10 mins, WA = 5 secs)
oall baselines a e sa e excep o 𝐷0.0
osimila policy beha io s excep o WEL-OB
Ø Syn he ic (WS = 15 mins, WA = 1 sec)
o𝑛/𝑐 a io dec eases linea ly wi h lowe 𝐷
o ine- une abili y
(ini'al) s a e
ac'on
once pe day
once pe sec
send
da a
- 𝐶𝑜𝑚𝑝𝑟𝑒𝑠𝑠 𝑚𝑜𝑟𝑒
•𝑒. 𝑔. 𝐷 ↑ 10%
-𝑆𝑡𝑎𝑦
•𝑢𝑛𝑐ℎ𝑎𝑛𝑔𝑒𝑑
- 𝐶𝑜𝑚𝑝𝑟𝑒𝑠𝑠 𝑙𝑒𝑠𝑠
•𝑒. 𝑔. 𝐷 ↓ 10%
oRL-based adap i e memo y comp ession scheme o s eam Agg ega es
oAllowing eal- ime balancing o pe o mance and memo y usage unde la ency cons ain s
oCap u e applica ion- and da a-speci ic beha io s o Agg ega es
oHighligh he ade-o be ween RL aining imeliness and policy e ec i eness
10:00:00
ac'on 'me
•I a window hasn’ been upda ed o a while…
•Comp ess i fi s ly, and la e decomp ess i
example
Abou his pape
Jingyu Liu, Vincenzo Gulisano
in equen ly
equen ly
Comp ess!
'me o he
nex ou pu
10:05:00
RL Agen SPE
ac&on (comp ess)
s a e, ewa d
Cyclical dependency
Ø Agen compu es i s nex ac'on
o ecei e he s a e and ewa d fi s !
baseline (X alue) baseline (X alue)
Each 𝐷𝑋#baseline always se s he 𝐷 alue o 𝑋 ∗ 𝑊𝑆#Each 𝐷𝑋 baseline always se s he 𝐷 alue o 𝑋 ∗ 𝑊𝑆
Ou pu
S eam
Ø Agg ega e wai s o he Agen ’s ac'on
o sha e a new s a e and ewa d fi s !
WEL-OB
(WEL
-OBli ious
)
EL-OB
(EL
-OBli ious
)
L-OB
(L-OBli ious)
WEL
-
AW
(WEL-AWa e)
wallclock
ime (W)
✘✓ ✓ ✓
e en ime (E)
✘ ✘ ✓ ✓
nex ou pu (L)
(e.g. la ency)
✘ ✘ ✘ ✓
-𝐷𝑄𝑁
policy
obse a&on
Linea Road Syn he ic Linea Road Syn he<c
(comp ession h eshold) 𝑫= 𝑋 ∗ 𝑊𝑆, 𝑋 ∈ 0.0,1.0 :
o 𝒍𝒂𝒕𝒆𝒔𝒕𝑻𝑺 −𝒕𝒔 ≥𝑫 à comp ess ( he condi ion o igge ing comp ession)
•𝑙𝑎𝑡𝑒𝑠𝑡𝑇𝑆: he imes amp (e en ime) o he la es uple p ocessed by he Agg ega e 𝐴
•𝑡𝑠: he imes amp (e en ime) o he la es uple ha con ibu ed o he window ins ance
•e.g., 0.0 ∗ 𝑊𝑆: all window ins ances main ained by he Agg ega e 𝐴 a e comp essed
1.0 ∗ 𝑊𝑆: no window ins ance main ained by he Agg ega e 𝐴is comp essed
uple
window ins ance
Why a e ou policies?
ØTh ee ways he sys em makes p og ess
(1) wallclock 'me mo es o wa d
(2) e en 'me ad ances
(3) ge a new s a e (e.g. la ency) measu emen
•(2) implies (1) because e en 4me ad ances only as 𝐴
p ocesses inpu , which depends on wallclock 4me.
•(3) implies (2) because la ency upda es occu only
when e en 4me ad ances enough o p oduce ou pu .
Based on hese dependencies, ou policies can be es ablished:
I he s a e comes om be o e 10:05:00, he
effec s on la ency a e no measu ed ye …
TIME ALIGNMENT (in ou policies)
up
down
100%
WS
𝑫
0%
WS
100%
WS
𝑫
0%
WS