Benchma king Pe o mance o Va ious MQTT
B oke Implemen a ions in a Compu e Con inuum
Jasenka Dizda e ić∗, Ma c Michalke∗, Admela Jukan∗, Xa i Masip-B uin†and F ancesco D’And ia‡
∗Technische Uni e si ä B aunschweig, Ge many
†Uni e si a Poli ècnica de Ca alunya, Ba celona, Spain
‡E iden, Ba celona, Spain
{j.dizda e ic, m.michalke, a.jukan}@ u-bs.de, x, y
Abs ac —Wi h he inc easing adop ion o IoT de ices and
applica ions, signi ican esea ch and de elopmen e o s ha e
been cen e ed a ound enginee ing no el ecosys ems e e ed o
as he IoT-edge-cloud compu e con inuum. In his a icle, we
implemen , analyze and p esen a case s udy o pe o mance
benchma king o i e well known and selec open sou ce MQTT
b oke implemen a ions in an open-sou ce compu e con inuum
es bed. The p oposed MQTT b oke implemen a ions a e
e alua ed in e ms o esponse ime, di e en payload sizes
and h oughpu . Measu emen s and esul s show ha he
ha dwa e pla o m used, he message size, as well as he
ne wo k pa ame e s (la ency, packe loss and ji e ) ha e a
signi ican impac on he esul ing pe o mance o a ious
b oke implemen a ions and he e o e ha e o be ca e ully
conside ed in he selec ion p ocess o he building blocks o he
con inuum. All implemen a ions and measu emen s a e made
o be ully ep oducible and ee and open sou ce.
I. In oduc ion
Wi h he inc easing expansion o In e ne o Things
(IoT) applica ions, edge and cloud compu ing e ol ed
as pi o al echnologies, wi h a la ge spec um o se ice
o e ings. Combined wi h IoT echnologies and de ices,
edge and cloud compu ing c ea e a no el ecosys em, o en
e e ed o as IoT-edge-cloud compu e con inuum [1], [2],
- a combina ion o echnologies ha join ly p ocess and
s o e da a in a a ie y o he so-called e icals, including
heal h ca e [3], manu ac u ing [4], and anspo a ion [5].
Enginee ing a unc ional compu e con inuum oday is
an open challenge, bo h concep ually and a he sys em
le el. Concep ually, applica ion equi emen s d i e he
op imal placemen o so wa e componen s on a he e oge-
neous IoT-edge-cloud sys em ha needs o ac as single
dis ibu ed en i y (a ”me a ope a ing sys em”). A he
sys em le el, he majo challenge is in he he e ogenei y
o de ices, so wa e ools and p o ocols.
To us, he choice o an applica ion laye communica ion
p o ocol is o pa icula in e es . Since se ices p e iously
deployed a he cloud p emises can now also be loca ed
a he edge o suppo la ency-sensi i e IoT applica-
ions, he op imal placemen o communica ion p o ocol
b oke /se e ins ances ega ding hei pe o mance is
an open challenge [6]. O e he pas yea s, a numbe
o he p o ocol solu ions, including Message Queuing
Teleme y T anspo (MQTT), Cons ained Applica ion
P o ocol (CoAP), Hype ex T ans e P o ocol (HTTP),
Ex ensible Messaging and P esence P o ocol (XMPP),
Ad anced Message Queuing P o ocol (AMQP), and Da a
Dis ibu ion Se ice (DDS), ha e ound hei use in IoT-
edge-cloud con inuum sys ems, each wi h a ious deg ee
o adop ion in a ious applica ion scena ios. In a compu e
con inuum, we no only need o conside hese p o ocols
o he exchange o messages be ween he di e en sys-
em’s componen s, bu also he co esponding message
b oke s and/o se e s. O special in e es a e open-sou ce
MQTT b oke implemen a ions and hei in eg a ion in o
he compu e con inuum, as MQTT emains he cu en ly
mos widely adop ed publish/subsc ibe p o ocol despi e
i s known sho comings, such as limi ed scalabili y o cen-
alized b oke a chi ec u es o i s TCP-based anspo .
In his pape , we s udy and implemen a pe o mance
benchma king o MQTT in a compu e con inuum es bed.
We deploy and es i e well-known open sou ce MQTT
b oke implemen a ions, i.e. Mosqui o, EMQX, Rab-
bi MQ, Ve neMQ, and Hi eMQ and hei in eg a ion in o
edge-cloud ne wo k eco-sys ems. The de eloped es bed
allows o in e ope abili y wi h a ious MQTT p o ocol
amewo ks, whose u iliza ion can be o in e es o bo h,
esea che s and de elope s. I consis s o wo ha dwa e
se ups; a clus e o wo i ual machines o measu e he
pe o mance when ypical AMD64 nodes a e used, as
well as a clus e o wo Raspbe y Pis o analyze he
pe o mance i hese nodes a e eplaced by less powe -
consuming ARM64 de ices. To compa e he pe o mance
o di e en b oke implemen a ions, we con igu e he
unde lying ne wo k sys em ha in e connec s he con in-
uum in he es bed by adjus ing he pa ame e s ne wo k
delay, a iance and packe loss. The b oke pe o mance
is hen benchma ked in e ms o es bed pe o mance
o se , message exchange la ency o di e en payload
sizes and h oughpu unde a high load, bo h on he
publishe and subsc ibe sides. The conduc ed expe i-
men s showcase he impac on he esul ing pe o mance
when using di e en ha dwa e pla o ms, message sizes,
ne wo k pa ame e s and numbe o clien s. Among hem,
he la ges pe o mance di e en ia o is showcased o be
ha dwa e wi h ARM64 based p ocesso s esul ing in much
highe esponse ime o all es ed b oke s. The mos
unexpec ed esul s ha e been no ed o he h oughpu
measu emen s, whe e he b oke implemen a ions wi h a
highe publishing h oughpu (e.g. Rabbi MQ) p o ide
a signi ican ly lowe h oughpu on he subsc ibe side
compa ed o he o he solu ions.
The es o he pape is o ganized as ollows. Sec ion
II p esen s he backg ound in e ms o ela ed wo k and
communica ion p o ocol con ex in he con inuum. Sec ion
III desc ibes he es bed and i s con igu a ions. Expe i-
men al benchma king es s and esul s a e p esen ed in
sec ion IV. Sec ion V concludes he pape and p o ides
an ou look.
II. Backg ound
A. Communica ion p o ocols in a con inuum
In a compu e con inuum, compu ing and da a p o-
cessing can happen a a ious deg ees o in ensi y and
pe o mance in all pa s o he sys em. Fo he sake
o his pape , le us assume ha he highes le el o
p ocessing capabili ies is o e ed by he cloud, he lowes
le el can be ound in IoT, wi h edge scaling in be ween
om easonably powe ul og compu ing nodes close o
he cloud, o low-ene gy mis compu ing nodes close o he
IoT. The e is a wide ange o de ices in he edge compu ing
ca ego y, o en le e aging almos any cen al p ocessing
uni (CPU) a chi ec u e, depending on hei en i onmen
and powe supply. Fo ins ance, de ices wi h low powe
consump ion and conside able compu ing capabili ies such
as Raspbe y Pi single boa d compu e s o o he ARM
based de ices can i in o bo h, he IoT and mis (and
he e o e edge) compu ing ca ego ies, depending on he
deploymen . Fo simplici y, we op o he de ini ion o
he de ices in he edge as mo e esou ce cons ained
in compa ison o he cloud, hence assuming medium
p ocessing, s o age and communica ion capabili ies. Due
o he s o age and p ocessing equi emen s and o e head
imposed by con aine iza ion and con aine o ches a ion,
we assume, wi hou loss o gene aliza ion, ha con aine -
ized se ices can only be used in he edge and he cloud
pa s o he sys em, as he IoT de ices lack he espec i e
capabili ies in hese a eas. Fo he assump ions ega ding
he ne wo king pe o mance, lowe ne wo k delay alues
a e assumed in he edge pa o he con inuum, as i is
expec ed ha he edge compu ing esou ces a e close o
he end de ices.
The communica ion in he con inuum is achie ed by
using di e en communica ion p o ocols. Rega ding he
choice o hese p o ocols o he di e en in e aces ac oss
he con inuum, a a ie y o solu ions is conside ed due
o he a ie y o Quali y o Se ice (QoS) equi emen s
o he applica ions and he capabili ies o he sys em.
As he cloud assumes compu a ionally in ensi e asks
and ewe esou ce cons ain s, making i less sensi-
i e o he communica ion o e head, he de elope s can
choose p o ocols simply based on hei popula i y and
compa ibili y wi h he expec ed in as uc u e. In ac ,
he mos p ominen p o ocols he e include (REST ul)
HTTP(S) and he ple ho a o p o ocols based on i ,
as well as publish-subsc ibe p o ocols like AMQP. To
enable message exchange h ough hese p o ocols, so wa e
componen s in he cloud ha e o hos co esponding
message b oke s and/o se e s. On he o he hand, o
es ablish communica ion wi h he IoT de ices and collec
hei da a, he de ices in he edge ha e o hos message
b oke s and/o se e s wi h less compu a ionally in ensi e
p o ocols, such as MQTT and CoAP, hus minimizing
he communica ion o e head. I is impo an o no e ha
o he communica ion aspec , we mus no only conside
he p o ocols o he exchange o messages be ween he
di e en sys em’s componen s in he con inuum, bu also
he co esponding message b oke s and/o se e s.
In his wo k we ocus on MQTT p o ocol solu ions,
as i emains a highly ele an p o ocol in IoT ela ed
con inuum sys ems [7]. When choosing MQTT b oke ap-
plica ions o analyze, he e a e no es ic ions in oduced
upon hem by he cloud esou ces o he con inuum. We
do howe e conside he abili y o deploy he b oke s on
esou ce cons ained a chi ec u es and unde lowe la ency
es ic ions as a c ucial ea u e o hei deploymen
in he edge pa o he con inuum, as his allows o
placemen o he communica ion solu ion on ARM64
de ices, close o he end de ices, ins ead o es ic ing
he placemen op ions o he cloud and i s mo e esou ce
capable AMD64 de ices. F om all a ailable MQTT b oke
implemen a ions, we e alua e he i e mos widely adop ed
open-sou ce op ions. Table I shows he speci ic de ails o
hese chosen MQTT b oke s.
TABLE I
Suppo ed MQTT b oke implemen a ions
Name Sou ce Language ARM64 3.1.1 5.0
Mosqui o mosqui o.o g C ✓ ✓ ✓
EMQX emqx.io E lang ✓ ✓ ✓
Rabbi MQ abbi mq.com E lang ✓ ✓ –
Ve neMQ e nemq.com E lang – ✓ ✓
Hi eMQ hi emq.com Ja a – ✓ ✓
We ocus on b oke s wi h official con aine images ha
suppo he ARM64 a chi ec u e, which cu en ly includes
h ee implemen a ions. Namely, hese a e he mos widely
used Mosqui o, he highly scalable EMQX as well as
Rabbi MQ which was ini ially designed o he AMQP
p o ocol bu also suppo s MQTT. In addi ion, we analyze
wo o he b oke applica ions ha ha e been used in
bo h dis ibu ed edge and cloud compu ing; Hi eMQ and
Ve neMQ [8].
B. Rela ed wo k
While he compu e con inuum makes i possible o
di e en applica ions and sys ems o bene i om he
ad an ages o e ed by bo h edge and cloud compu ing
pa adigms, i esul s in a complex he e ogeneous execu ion
en i onmen , which i sel p esen s a numbe o challenges
and open issues. The e ha e been a ious signi ican e o s
in de ining gene alized con inuum e e ence a chi ec u es
[9]–[11]. O he wo ks ha e ocused on he o ches a ion o
he applica ion wo k lows ac oss he con inuum esou ces
[12]. Inc easingly impo an di ec ions o he con inuum
a e ela ed o se ice scheduling [13], [14], as well as he
de elopmen o amewo ks and es beds ha acili a e
applica ion and se ice deploymen s in edge-cloud en-
i onmen s, ocusing on au oma ed deploymen o bo h
scien i ic and p oduc ion wo k lows [15], [16]. Fu he
wo k [17]–[19] add esses he placemen o applica ions
ac oss he con inuum wi h machine lea ning and a i icial
in elligence, indica ing he complexi y o he applica ion
placemen p oblem. These au oma ed placemen a emp s
howe e ha e no ye esul ed in p oduc ion- eady solu-
ions, which lea es p ede ined placemen o se ices as
he cu en me hod o choice. Such decisions he e o e
s ill need e e ence alues ega ding he expec ed ela i e
pe o mance when se ices a e placed a di e en posi ions
among edge and cloud. While hese e e ence alues ha e
s a ed o be explo ed o p ocessing h ough se e less
unc ions [1], [20], [21] o machine lea ning [22], [23], his
gap s ill exis s o messaging, which we con ibu e o
closing he e.
In gene al, he mos p omising angle o gene a e pe -
o mance indica ing numbe s o an applica ion is he
benchma king o he so wa e unde ci cums ances ha e-
semble he p oduc ion en i onmen o be expec ed. To his
end, he au ho s in [11], de eloped a amewo k o c ea e
an edge-cloud ne wo k and benchma k ML applica ions
ac oss i . They expe imen wi h di e en o loading models
and es he esul ing pe o mance o he ML applica ion
unde di e en condi ions. This is done by emula ing he
cha ac e is ics o di e en pa s o he con inuum. Simila
benchma king wo k has been conduc ed by he au ho s
o [24] o he OpenWhisk se e less pla o m. Ou wo k
aims o p o ide a simila amewo k ha can be eely
epu posed o o he ne wo king en i onmen s.
Fo he communica ion aspec o he con inuum, pa-
pe s [25]–[27] compa e di e en communica ion p o ocol
solu ions and lay ou hei espec i e ad an ages and
disad an ages. Rega ding he MQTT p o ocol chosen in
his pape , ela ed wo k compa es he pe o mance o
di e en message b oke s simila o [28]. The la e s udy
ocused on wo es ing en i onmen s; a local se up and
a p op ie a y cloud solu ion, wi h he Mosqui o b oke
solu ion ou pe o ming o he s o mos o he obse ed
me ics. O he wo k ocuses on sma ci ies ega ding
he pe o mance [29] o MQTT b oke s when connec ed
h ough di e en bandwid h-limi ed links o an a chi ec-
u e o in eg a e hem [30] albei wi hou conside a ion
o link delay despi e he ocus on IoT de ices.
While he e is no lack o s udies o di e en aspec s o
MQTT communica ion in he cloud en i onmen s, pa ic-
ula ly in e ms o a ious whi epape s p o ided by b oke
endo s, s udies ocused on he edge o he IoT-edge-cloud
con inuum a e ew and a be ween. Wo k [31] conduc s
benchma king o he pe o mance o mul iple MQTT b o-
ke s in clus e ed con igu a ion while conside ing di e en
locali y se ups, meaning he publishing and subsc ibing
clien s being connec ed o b oke s on ei he a di e en
o he same si e as he publishe s. I also in oduces
ei he 5 o 50 ms o o al delay be ween he si es. I es s
and compa es he mos known MQTT b oke s EMQX,
Ve neMQ, Rabbi MQ, Hi eMQ, Mosqui o. These aspec s
a e simila o he s udy in his pape . We do no howe e
exclusi ely ocus on con en ional x86/AMD64 ha dwa e
bu also conside he ARM64 a chi ec u e in he con ex
o non-clus e ed b oke deploymen s. In addi ion, we
ocus on he message size and concu en eques s o
close he knowledge gaps o hese dimensions. Finally,
ou measu emen s a e also ocused on he subsc ibe
h oughpu , which is in s a k con as o o he ela ed
wo k which mos ly ocuses on publishe h oughpu . This
is c i ical o b oke implemen a ions unde a ious affic
and load condi ions.
We c ea e a es bed ha allows o au oma ed schedul-
ing o di e en benchma ks and manipula ion o link
me ics like delay, delay a iance o packe loss o asses
he pe o mance o a ious MQTT message b oke s unde
di e en ne wo king ci cums ances. We choose he ne -
wo king me ics based on he pe o mance o he cloud
and edge analyzed in [32] and he alues used in [21] and
e alua e he pe o mance o b oke implemen a ions unde
a ious affic and load condi ions including di e en
emula ed loca ions ac oss he compu e con inuum. Ou
no el con ibu ion can he e o e be summa ized as ollows:
•C ea ion o pe o mance e e ence alues o messag-
ing/communica ion solu ions ac oss he con inuum.
•C ea ion o a ep oducible amewo k o e alua e
MQTT b oke s a di e en loca ions ac oss he
compu e con inuum by emula ing cloud and edge
ne wo king cha ac e is ics.
•Focus on bo h; con en ional x86/AMD64 ha dwa e
and he ARM64 a chi ec u e in he con ex o non-
clus e ed b oke deploymen s.
•Conside a ion o he impac ha he numbe o
concu en eques s has on he message h oughpu ,
no jus on he publishe bu also on he subsc ibe
side.
•Publishing his open-sou ce benchma king amewo k
such ha i can be eely epu posed o e alua ion
o di e en communica ion p o ocol b oke /se e
applica ions o o he ne wo king en i onmen s o
b oke solu ions.
III. Tes bed
Fig. 1a shows he ne wo k es bed enginee ed o
expe imen al benchma king o di e en MQTT b oke
implemen a ions as p esen ed in Table I.
The ha dwa e con igu a ion o ou es bed consis s o
wo Raspbe y Pis (ARM Co ex A72, 8GB) and wo
Vi ual Machines (4 CPU, 8GB), as shown in Fig.1a,
which a e deployed on a hype iso (In el i9-10900X,
64GB), as well as a Desk op PC (In el i5-6500, 16GB)
as a es e node. All de ices a e in e connec ed h ough
a Gigabi E he ne swi ch. The ope a ing sys ems include
Ubun u e sion 22.04.2 o he es e , Ubun u 20.04.6 o
all Raspbe y Pis (ARM64) and VMs (AMD64), and A ch
Linux o he hype iso .
When a benchma king es is execu ed h ough he
benchma king ool on he es e node, one o wo in as-
uc u e se ups is used as a benchma king a ge . The
i s se up consis s o wo i ual AMD64 machines; one
Headnode and a wo ke node. The second se up consis s
o wo ARM64 Raspbe y Pis wi h he same kind o ole
assignmen . Ac oss ei he se up, he nodes a e connec ed
h ough an o e lay ne wo k (Nebula1) on op o which
he con aine o ches a ion (k3s) o ms a Kube ne es
clus e . This clus e is hen le e aged o deploy each
o he espec i e con aine ized b oke implemen a ions
h ough Kube ne es deploymen and se ice de ini ions,
one b oke a a ime, which ul ima ely a e a ge ed o he
pe o mance es . Once a es is comple ed, he nex b oke
implemen a ion is deployed and used as a a ge , wi h all
o hese es s being epea ed on bo h pla o m se ups;
AMD64 and ARM64. Fo he benchma king p ocedu e,
ou se up uses he wo benchma king ools jme e and
mq -s esse , wi h he ool and pa ame e s being de ined
pe benchma king desc ip ion ha is applied o he
es bed.
Nex , we p o ide a de ailed o e iew o he es bed
deploymen wo k low as shown in Fig.1b o he bench-
ma king o he di e en open sou ce MQTT b oke appli-
ca ions. This wo k low is execu ed h ough he au oma-
ion ool Ansible2based on so-called Ansible playbooks
(YAML iles), ha de ine all necessa y equi emen s and
s eps o be execu ed agains he ac ual es bed in as uc-
u e. The wo k low can be di ided in o se en s eps which
a e explained as ollows:
1) Ins alla ion o all equi emen s on o he machines,
includes he benchma king ools o he es e node
and basic u ili ies like cu l o he Headnode and
wo ke nodes
2) Ins alla ion o he Nebula o e lay ne wo k o ensu e
connec ion be ween he nodes ha shall o m a
Kube ne es clus e , e en i hey should no eside
in he same LAN
3) Ins alla ion o he Kube ne es clus e wi h he
Headnode hos ing all con ol se ices and all po en-
ial applica ions es ic ed o be exclusi ely deployed
on o he wo ke nodes
1h ps://gi hub.com/slackhq/nebula
2h p://ansible.com/
Tes e node
Benchma king
Tools
Vi ual machine
AMD64, 8GB
Headnode Wo ke node
Vi ual machine
AMD64, 8GB
Headnode
Raspbe y Pi
ARM64, 8GB
Raspbe y PI
ARM64, 8GB
Wo ke node
Con aine o ches a ion
O e lay ne wo k
B oke /se e applica ions
Con aine o ches a ion
O e lay ne wo k
B oke /se e applica ions
(a) Tes bed in as uc u e
5. Deploy con aine ized
b oke applica ions
3. Ini ia e and manage k3s
clus e om headnode
4. Modi y he in e aces o
he o e lay ne wo k
Tes e node
1. Ini ialize deploymen
on he VM/RP
Nebula
Wo ke node
Headnode
Ansible
SSH SSHSSH
In as uc u e
in en o y
(YAML ile)
Nebula
(YAML ile)
2. Deploy o e lay
ne wo k
k3s
o ches a o
(YAML ile)
Nebula
k3s mas e k3s wo ke
ne em
(YAML ile)
la ency, a iance, p. loss la ency, a iance, p. loss
B oke
con aine s
(YAML ile)
Mosqui o, Hi eMq, EMQX,
Rabbi MQ, Ve neMQ
6.1. Run pe o mance o se
es agains b oke s
Bechma king
ools
Jme e
6.2. Run payload size es
agains b oke s
6.3. Run scalabili y es
agains b oke s
mq -s esse
Resul s
(.cs and
. x ) 7. Gene a e esul s and ini ialize ese
(b) Deploymen wo k low
Fig. 1. Tes bed o expe imen al benchma king o MQTT b oke
implemen a ions
4) Manipula ion o he in e ace pa ame e s o in o-
duc ion o addi ional delay, packe loss o delay
a iance
5) Deploymen o he con aine ized applica ion (one
o he i e b oke s) as Kube ne es deploymen and
se ice on o he wo ke node
6) Execu ion o he h ee pe o mance es s agains
he Headnode (who o wa d he eques s o he
applica ion)
7) Gene a ion o he esul iles and cleanup o he
in as uc u e
He e, s eps 6.1 o 6.3 ep esen he indi idual bench-
ma k ope a ions o be execu ed agains he clus e . They
a e desc ibed in mo e de ail in he espec i e subsec ions
unde sec ion IV.
A. In as uc u e and wo k low ini ializa ion (S ep 1)
In o de o ini ialize he deploymen o equi ed con-
igu a ions and ins alla ions on he a ge ed de ices ha
o m he es bed in as uc u e, he p e equisi es de ined
as pa o he es bed a e ins alled o he de ices based
on hei ole in he o e all sys em. These include he
necessa y benchma king ools on o he es e node, basic
u ili ies used o download sc ip s o he wo ke nodes
and Headnode, as well as copying iles ha a e needed a
la e s ages o he p ocess o he espec i e nodes, c ea ion
o esul s olde s wi h cu en imes amps, p epa a ion o
Nebula ce i ica es and cleaning up p e ious deploymen s
om all machines, in case hey a e s ill unning.
B. Con aine iza ion and ne wo king (S eps 2, 3 and 4)
To connec he Headnode and all wo ke nodes con-
side ed in each se up h ough a Nebula o e lay ne wo k
wi h es ablished i ual in e aces, a Nebula ligh house is
ins alled on o he Headnode i s . Once his succeeded,
he wo ke nodes join he o e lay ne wo k by le e aging
he ce i ica es p e iously gene a ed o hem. Once he
o e lay ne wo k is es ablished, he Headnode c ea es a
Kube ne es clus e by ins alling k3s wi h ins alla ion
a gumen s like he e sion o be used being de ined in he
dedica ed YAML con igu a ion ile. The wo ke nodes hen
join his clus e . A Kube ne es node selec o es ic ion
is c ea ed o he namespaces such ha applica ions
a e la e only deployed on o he wo ke nodes and no
he headnode. This ensu es ha he esou ces o he
Kube ne es con ol plane a e consumed on he Headnode,
which ees he wo ke nodes o un he b oke implemen-
a ions. The e o e, he impac o Kube ne es con ol plane
ope a ions on o he es esul s can be minimized. These
s eps can be seen in Fig.1b as s eps 2 and 3, espec i ely.
S ep 4 consis s o applying he link me ics speci ied in he
playbook o he co esponding ne wo k in e aces o he
machines o emula e di e en ne wo king en i onmen s,
connec ions o dis ances ac oss he edge-cloud con inuum.
Fo he es bed o be used in he con inuum compu ing
con ex , he e is an addi ional dimension o conside
besides po en ially di e en CPU a chi ec u es and capa-
bili ies o ha dwa e pla o ms; he ne wo king pe o mance
impac ed by he di e en affic pa ame e s be ween
all nodes. To explo e po en ial pe o mance di e ences
be ween he b oke s when his ac o is conside ed,
he es bed enables unning es s o e di e en ne wo k
con igu a ion scena ios, designed o app oxima e eal-
wo ld scena ios. The emula ion o di e en ne wo king
con igu a ions is modi ied h ough he ne em con igu a ion
YAML ile (s ep 4 in he deploymen wo k low), whe e
di e en la ency, ji e and packe loss alues a e de ined,
which a e hen applied o he i ual o e lay ne wo k
in e aces. Highe amoun s o delay can o example
show how a cloud node has a highe dis ance o o he
nodes o an end de ice, while highe delay a iance and
packe loss could also eplica e a wi eless connec ion.
The exac alues o he benchma king o MQTT b oke
implemen a ions a e lis ed in Table II, wi h each lis ed
alue ep esen ing he pa ame e s ha can be measu ed
ac oss he connec ion o any wo nodes o he ne wo k, as
well as he connec ion o he es e o any node. Based
on he la ency alues measu ed in [32] and he scena ios
de ined in [21], we assume a o al delay o 2.5 ms, a delay
a iance o 0.5 ms and a packe loss o 0.04% o ou edge
scena io and a delay o 6.25 ms wi h a delay a iance o
1.25 ms and a packe loss o 0.1% o cloud da a cen e s
as shown in Table II.
TABLE II
Emula ed pa ame e s o ne wo k con igu a ions
Scena io La ency (ms) Va iance (ms) Packe Loss (%)
local 0 0 0
edge 2.5 0.5 0.04
cloud 6.25 1.25 0.1
Acco ding o hese ne wo k pa ame e s, we de ine
h ee es bed ne wo k scena ios in which he b oke
implemen a ions will be benchma ked: local, edge, and
cloud case. While he local scena io se es as a baseline
wi hou any addi ionally in oduced la ency, a iance o
packe loss, he edge scena io ep esen s wha we conside
an achie able bes -case scena io o an edge compu ing
deploymen wi h e y low la ency. The cloud scena io on
he o he hand ep esen s he lowes la ency ha would
be easonable o assume o a cloud deploymen .
C. Deploying b oke applica ions (S ep 5)
We deploy he i e open-sou ce b oke implemen a-
ions shown p e iously in Table I (Mosqui o, EMQX,
Rabbi MQ, Ve neMQ and Hi eMQ) and Fig. 1b in s ep
5. To deploy he selec ed b oke s in he es bed, he
co esponding con aine s a e deployed on he wo ke node
o each es bed se up. Mosqui o, EMQX and Rabbi MQ
a e analyzed on he Raspbe y Pi se up since hey a e he
only solu ions wi h suppo o he ARM64 a chi ec u e,
while all i e b oke s a e analyzed on he AMD64 se up.
D. Benchma king ools (S eps 6 and 7)
The benchma king p ocess i sel is execu ed in s ep 6,
which loops o e he di e en es de ini ions, execu ing
hem one by one. Fo his ask, he espec i e es ing ool
is used along wi h he pa ame e s de ined in each indi-
idual benchma king ask. He e, we use wo es ing ools
wi h MQTT suppo ; an MQTT plugin o he JMe e
load es ing ool3 o he pe o mance o se and payload
size es s, as well as he mq -s esse scalabili y es ing
ool4 o he scalabili y es . The pa ame e s equi ed
o he u iliza ion o hese benchma king ools (MQTT
po s o be used, payload message size, numbe o clien s,
e c.) a e also p o ided h ough YAML con igu a ion iles,
and a e un agains he espec i e b oke s (see s eps 6.1,
6.2 and 6.3 o Fig.1b). Once his is accomplished, he
esul s a e e ie ed and he en i onmen is cleaned up
by unins alling he o e lay ne wo k and he con aine
o ches a o which also dele es all b oke s deployed on
op o i .
An example o Ansible playbooks o scalabili y es ing
o EMQX b oke applica ion is shown in a ollowing YAML
ile:
3h ps://gi hub.com/emqx/mq -jme e
4h ps://gi hub.com/ino ex/mq -s esse
1- name: Playbook example o emqx and s e s s e
2hos s : es e s
3 a s :
4applica ion : emqx
5benchma k_ ool : mq - s e s s e
6mq _s esse :
7u l : ”{{ add ess_ es bed }}”
8po : 31884
9num_clien s_se : [800 , 700 , 600 , 500 , 400]
10 num_messages : 1000
11 global_ imeou : 60s
12 ne wo k_se s :
13 name: lo c a l
14 delay : 0
15 a iance :0
16 loss : 0
17 egis y :
18 use : use _name
19 ip : ip_add ess
20 po : po _numbe
21 asks :
22 - name: Launch emqx/mq - s e s s e wo k low
23 ansible . bui l in . include_ asks : wo k lows/
emqx/main . yml
24
This playbook ile is con igu ed o se up an EMQX
b oke ins ance (emqx) o be i s ins alled on he es bed
in as uc u e (add ess_ es bed) and hen es ed wi h he
mq -s esse ool by speci ying he po on which he
b oke applica ion can be accessed in he es bed. The pa-
ame e s o he es e i sel a e also con igu ed, including
numbe o emula ed concu en clien s (num_clien s_se ),
numbe o messages (num_messages) and he imeou
o he en i e ope a ion (global_ imeou ). He e we also
speci y which ne wo k scena io is o be deployed ac oss
he es bed ia ne wo k_se s.
De ailed ins uc ions on he es bed used and deploy-
men o MQTT b oke s o e he AMD64 o ARM64 se up
a e eady o be published on Gi hub, wi h all esul s,
so wa e ools and code ully ep oducible and ee and
open sou ce.
IV. Expe imen al esul s
In his sec ion, we desc ibe he benchma king es s
conduc ed as well as he measu emen s ob ained. We
de ine h ee ypes o es s, i.e., (i) es bed pe o mance
o se , (ii) payload size esponse ime and (iii) h oughpu .
The de ailed speci ica ions o each es s a e shown in
Table III.
We u ilize MQTT p o ocol e sion 3.1.1. Fo he i s
wo es s we measu e he la ency as he ime elapsed
om sending a message om he MQTT publishe o
he MQTT subsc ibe . Fo his ype o measu emen s
we use an MQTT plugin o he es e ool JMe e
wi h cus omized es plans. Fo he h oughpu es s, we
s ess he MQTT b oke wi h an inc easing numbe o
concu en clien s publishing a numbe o messages, and
hen measu e he h oughpu in o m o he messages pe
second success ully published o he b oke and he mes-
sages pe second success ully ecei ed by he subsc ibe s.
In his case, we use he mq -s esse scalabili y es ing
ool, con igu ing i s pa ame e s o he numbe o MQTT
clien s, as well as he messages pe clien . The mq -
s esse ool is selec ed in his case as i allows o easie
es con igu a ions han is cu en ly possible wi h he open
sou ce e sion o he JMe e MQTT plugin.
We conside wo ha dwa e se ups o he i s wo es s,
i.e., AMD64 and ARM64, o h ee di e en ne wo k
scena ios, i.e., local, edge and cloud. In each o hese
es s, we use one subsc ibe h ead and adjus he numbe
o publishe h eads, wi h each publishe h ead sending
a single message o he a ge b oke . The published
messages a e se o include imes amps, which a e hen
sub ac ed om he co esponding imes amps on he
subsc ibe side. Each es is epea ed a leas 10 imes
o minimize he po en ial ma gin o e o and ensu e
ep oducibili y o he esul s. Fo he hi d es , we use
he AMD64 se up only, and a local scena io.
A. Tes bed La ency O se
This ini ial measu emen se es o e alua e he pe o -
mance o se in oduced when publishing a simple hello-
wo ld s ing o he MQTT b oke deployed on he es bed.
JMe e is used o measu e he esponse imes in e ms o
la ency, as p e iously de ined, and collec hem in a sepa-
a e .cs ile pe execu ed es . He e, we conduc wo se s o
measu emen s o he esponse ime on he subsc ibe side.
In he i s se , we analyze b oke implemen a ions wi h
official ARM64 con aine suppo (Mosqui o, EMQX and
Rabbi MQ). In he second se , we compa e all i e o
he ini ially selec ed implemen a ions (Mosqui o, EMQX,
Rabbi MQ, Hi eMQ and Ve neMQ). On he publishe
side, we con igu e a u iliza ion alue o 100 h eads, each
sending he same message wi h an in e al o 250ms.
The QoS alue o bo h, he MQTT subsc ibe and he
publishe , in he p o ocol is se o he eliable le el 1,
which implies ha a success ul message ans e o he
b oke is gua an eed. I should be no ed ha nei he he
au hen ica ion no he secu i y mechanism was used.
Fo he i s se o measu emen s, he esponse ime
alues o he b oke s unning on he ARM64 se up a e
shown in Fig.2a wi h boxplo g aphics showing he median
alue o each o he b oke s as well as a ia ions in he
da a ob ained. He e, we can see ha he median alue
o he pe o mance o se es s is a ound 5 ms in he
local scena io, 10 - 12 ms in he edge scena io, and 18
- 19 ms in he cloud scena io, o all implemen a ions.
The igu e also shows ha he boxplo s a e almos o
he same size o each scena io wi h low a ia ion o he
alues. While Mosqui o seems o p e o m sligh ly be e ,
he compa able delays in all cases a e e y low and can
be conside ed negligible.
Fo he second se o measu emen s, he esponse ime
alues o all i e b oke s unning on he AMD64 se up a e
shown in Fig.2b. The median alue o pe o mance o se
is a ound 4.8 - 8 ms (local), 8 - 13 ms (edge), and 11 -
18 ms (cloud scena io), o all implemen a ions. This case
TABLE III
Benchma king es s speci ica ions
Tes ype Tes ing Se up Ne wo k No. o No. o QoS Le el Payload No. o messages
ool scena ios publishe s subsc ibe s size pe publishe
Pe o mance JMe e AMD64, local, edge, cloud 100 1 1 10B 1
o se ARM64 local, edge, cloud 100 1 1 10B 1
Payload size JMe e AMD64, local, edge, cloud 100 1 1 1KB, 10KB, 1MB 1
esponse ime ARM64 local, edge, cloud 100 1 1 1KB, 10KB, 1MB 1
Th oughpu mq -s esse AMD64 local 100 - 800 100 - 800 0 10B 100, 1000
mosqui o_l
emqx_l
abbi mq_l
mosqui o_e
emqx_e
abbi mq_e
mosqui o_c
emqx_c
abbi mq_c
0
5
10
15
20
Pe o mance o se [ms]
mosqui o
emqx
abbi mq
(a) ARM64 se up
mosqui o_l
emqx_l
abbi mq_l
mosqui o_e
emqx_e
abbi mq_e
mosqui o_c
emqx_c
abbi mq_c
0
5
10
15
20
Pe o mance o se [ms]
mosqui o
emqx
abbi mq
(b) AMD64 se up
Fig. 2. Tes bed pe o mance o se
howe e shows highe a ia ion o he alues, pa icula ly
in he cloud scena io, and especially o he Hi eMQ
b oke , which shows he highes de ia ion o all b oke s,
e en unde compa ably good condi ions. This beha io
becomes mo e p onounced he wo se he ne wo king
condi ions (delay, packe loss and ji e ) become, no ably
exceeding he o he b oke s’ de ia ions while main aining
a simila median esponse ime. The opposi e e ec can
be obse ed wi h he Rabbi MQ b oke , which shows
sligh ly ele a ed median esponse imes while main aining
signi ican ly mo e p edic able esponse imes han he
o he b oke s. When compa ing he median alues o he
h ee b oke implemen a ions ha ha e been es ed on
bo h se ups in Fig.2a and Fig.2b, we can also see a sligh ly
lowe esponse ime by oughly one ms which can be
conside ed negligible. This di e ence can also be aced
back o he sligh pe o mance ad an age ha Vi ual
Machines ha e o e Raspbe y Pis since VM’s in e -node
affic does no ha e o pass h ough a physical ne wo k
medium bu emains in he same physical de ice ins ead.
O e all when analyzing he pe o mance o se o he
es bed i sel , we ound compa able esponse imes o all
b oke s, when deployed in he same ne wo k scena ios and
on he same ha dwa e pla o m se ups.
B. La ency s. payload size
Fo he payload es s, we also con igu e a u iliza ion
o 100 h eads on he publishe side, sending messages
o di e en payload sizes wi h an in e al o 250ms. We
modi y he message size by de ining s ings o di e en
ixed leng hs, conside ing 1KB, 10KB and 1MB sizes.
Nex , we measu e he la ency on he subsc ibe side o
each payload size and o each b oke implemen a ion
in all h ee ne wo k scena ios, o he AMD64 and o
he ARM64 se up. The QoS o bo h, he subsc ibe and
publishe , emains on le el 1.
Fo he i s se o b oke implemen a ions es ed on
bo h se ups, i.e., ARM64 and AMD64, he median la ency
alues and in e -qua ile ange (IQR), which shows how
sp ead he boxplo is, a e shown in Table IV, wi h hei
alues in milliseconds. When sending a 1KB message,
he esul ing median alues in bo h se ups a e e y
close in all h ee measu emen scena ios (local, edge,
cloud), wi h negligible di e ences be ween he b oke
implemen a ions. E en he di e ences in e ms o la ency
alues a e low be ween he local and cloud scena io (e.g.,
Mosqui o, local median 5.39 ms and cloud case median
18.6 ms). I we compa e hese median alues wi h hose
ob ained as pe o mance o se and shown in Fig.2a (e.g.,
app oxima ely 4.8 ms and 17 ms median alues o local
and cloud case scena io o Mosqui o) we can see ha he
1KB payload size in oduces a e y low addi ional la ency.
TABLE IV
Payload size: la ency median and IQR - Mosqui o, EMQX and Rabbi MQ
Mosqui o EMQX Rabbi MQ
local edge cloud local edge cloud local edge cloud
1KB ARM64 5.39 - 1.00 10.67 - 1.00 18.60 - 2.00 6.04 - 0.25 11.54 - 1.00 18.93 - 2.00 7.00 - 1.00 12.00 - 1.00 19.87 - 2.25
AMD64 3.94 - 0.00 8.59 - 2.00 15.23 - 6.00 4.24 - 0.00 9.14 - 3.00 15.25 - 7.00 4.83 - 1.00 9.65 - 3.00 15.59 - 6.00
10KB ARM64 10.68 - 1.00 15.91 - 1.00 24.90 - 2.00 11.44 - 2.00 16.56 - 2.00 24.71 - 2.00 13.16 - 2.00 17.73 - 1.00 26.15 - 2.00
AMD64 4.33 - 0.00 8.83 - 2.00 16.85 - 7.00 4.66 - 1.00 9.43 - 2.00 16.67 - 7.00 5.76 - 1.00 9.85 - 1.00 17.25 - 6.00
1MB ARM64 272.48 - 32.00 332.42 - 48.00 643.17 - 181.25 274.11 - 35.00 333.04 -49.00 641.02 - 193.75 297.74 - 53.00 379.31 - 55.00 679.16 - 167.00
AMD64 66.27 - 13.00 153.96 - 75.00 437.10 - 388.25 65.2 - 14.00 149.60 - 69.25 418.81 - 356.50 71.97 - 13.00 161.38 - 64.25 429.83 - 379.50
m_l
pi_l
m_e
pi_e
m_c
pi_c
0
200
400
600
800
1000
La ency [ms]
RPi se up
VM se up
Fig. 3. Payload size 1MB: ARM64 se up s AMD64 se up
The IQR alues also a e in a low ange, wi h sligh ly highe
a iance o he cloud case ne wo k scena io o Rabbi MQ
and EMQX implemen a ions. Simila ends con inue o
10KB message size, wi h he expec ed inc ease o la ency
in all cases compa ed o a message size o 1KB. This
inc ease emains in he ange o app oxima ely 5 ms in
all scena ios and all implemen a ions.
Mo ing o 1MB message size, we can obse e signi i-
can pe o mance di e ences. Fi s , when compa ing he
same b oke implemen a ions on he AMD64 and ARM64
se up, we can obse e ha in each b oke implemen a ion
and in each ne wo k scena io, VMs pe o m be e wi h
no iceably lowe median esponse imes, as well as lowe
a iance. The di e ence in la ency be ween hese wo
ha dwa e se ups is app oxima ely 200ms o all imple-
men a ions, which can be a limi ing ac o in low-la ency
demanding IoT applica ions, whe e MQTT is o en he
p o ocol o choice. On he o he hand, when compa ing
he median esponse imes o he same scena io on
he ARM64 se up, e.g. 643.17 ms (Mosqui o), 641.02
ms (EMQX) and 679.16 ms (Rabbi MQ) o he cloud
scena io, we can see ha he b oke implemen a ion i sel
does no play a key ole in he o e all esul s. The same
can be said o AMD64 se up. In con as o wha was
obse ed o he small payload sizes, in he case o 1MB,
he IQR ange is highe , indica ing a highe a iance o
he ob ained alues.
TABLE V
Payload size: la ency median and IQR - Hi eMQ and Ve neMQ
Hi eMQ
local edge cloud
1KB AMD64 9.70 - 5.00 13.46 - 5.00 18.40 - 7.00
10KB AMD64 9.95 - 4.00 13.94 - 5.00 19.75 - 8.00
1MB AMD64 125.00 - 24.00 194.72 - 73.00 374.80 - 320.00
Ve neMQ
local edge cloud
1KB AMD64 4.37 - 1.00 8.30 - 3.00 13.64 - 6.00
10KB AMD64 5.18 - 1.00 9.23 - 2.00 15.23 - 7.00
1MB AMD64 76.25 - 12.00 144.85 - 71.00 345.64 - 292.50
The e ec ha he speci ic es bed se up has is also
isually shown on he example o he Rabbi MQ b oke
in Fig. 3 o 1MB message payload size. As i can be seen,
he di e ence in he esponse ime on he AMD64 and
ARM64 se up is no iceable in each o he h ee scena ios.
The median la ency alues and IQR o he emaining
b oke implemen a ions, which a e es ed only on he
AMD64 se up, a e shown in Table V. Fo a message size
o 1KB we can see he same end as in he p e iously
discussed implemen a ions, wi h he median la ency alues
being sligh ly highe in he local ne wo k scena io o
Hi eMQ. I we compa e hese median alues wi h he
ones ob ained as pe o mance o se and shown in Fig.2b
(e.g. app oxima ely 8 ms and 18 ms median alues o
local and cloud scena io o Hi eMQ) we can again see
ha 1KB payload size in oduces negligible la ency. The
same end con inues o 10KB message size, wi h he
inc ease o la ency compa ed o 1KB message being less
han 1 ms. Mo eo e , his esul can be obse ed in all
i e b oke implemen a ions es ed on he AMD64 se up.
As shown p e iously, he mo e signi ican de e io a ions
in pe o mances o bo h median la ency alues and IQR
ange can be no iced wi h 1MB payload size. Howe e , i
we now compa e all median esponse imes o he edge
scena io on he AMD64 se up, e.g., 153.9 ms (Mosqui o),
149.6 ms (EMQX), 161.38 ms (Rabbi MQ), 194.72ms
(Hi eMQ) and 144.85 ms (Ve neMQ), we can see ha
he di e ences a e no e y p onounced, wi h Hi eMQ
pe o ming sligh ly wo se han o he implemen a ions.
Howe e , i we conside he cloud scena io (AMD64
se up), we can see a signi ican di e ence be ween he
wo addi ional b oke s Ve neMQ and Hi eMQ and he
emaining b oke s. Namely, hei mean alues o 345.64
ms (Ve neMQ) and 374.80 ms (Hi eMQ) a e conside ably
lowe han 429.83 ms (Rabbi MQ), 418.81 ms (EMQX),
and 437.10 ms (Mosqui o) sco ed by he emaining
b oke s. This is especially in e es ing gi en ha Hi eMQ
p e o med wo se han he o he s when using smalle
payloads.
C. Th oughpu and Scalabili y
In he inal es s, we inc ease he numbe o concu en
publishing and subsc ibing clien s, scaling om 100 o
800 clien s. Each publishe clien is se o send a numbe
o messages (100 and 1000 pe clien ) o 10B payload
size. These pa ame e s a e speci ied in he playbook
con igu a ion ile o each o he b oke s (example shown
in Sec ion III-D), and each es has been epea ed a leas
10 imes. I is impo an o no e, ha he de aul pause
be ween messages was kep a 0s, in o de o inc ease he
load o he b oke s. The QoS o bo h ypes o MQTT
clien s is se o le el 0 (no gua an ee o he deli e y), as we
assume a local ne wo k scena io wi h a s able connec ion
and hus wi hou added la encies, ji e and packe loss.
The measu emen s we e pe o med on he AMD64 se up,
in o de o be able o compa e all i e b oke s.
Fo he i s se o measu emen s wi h each clien
publishing 100 messages, he published and ecei ed
h oughpu in e ms o numbe o messages published
and ecei ed pe second, a e shown in Fig.4a and 4b,
espec i ely. The esul s in Fig.4a, show a he cons an
publishing h oughpu alues when scaling om 100 o
800 clien s o Mosqui o (a ound 80000 msg/s), EMQX
(a ound 75000 msg/s) and Ve neMQ (a ound 60000 ms-
g/s). On he o he hand, bo h Rabbi MQ and Ve neMQ
show a sha p inc ease in publishing h oughpu , wi h he
inc eased numbe o clien s. O e all he highes publishing
h oughpu alues a e measu ed o Rabbi MQ, while he
wo s pe o mances a e measu ed o Hi eMQ.
In o de o ully unde s and b oke beha io unde
hese high loads, we also s udy he ecei ed h oughpu ,
which is a salien ea u e o ou app oach. I is measu ed
on he side o subsc ibe clien s, shown he e in Fig.4b.
The ecei ed h oughpu o all b oke s ollows almos
a linea ly dec easing end, wi h he deli e ed message
a es signi ican ly lowe han he one published o he
b oke s o each numbe o clien s. In addi ion o he
educed ecei ed h oughpu pe o mance, he b oke s
wi h he highes publishing h oughpu do no gua an ee
highes ecei ed h oughpu . This is mos ob ious in he
case o he Rabbi MQ b oke , which while allowing o
highes published h oughpu , pe o ms almos he wo s ,
wi h he ecei ed h oughpu a ound 400 msg/s, wi h
lowe alues measu ed only o Hi eMQ. The highes
ecei ed h oughpu is measu ed o Ve neMQ (2000-3000
msg/s) and EMQX (1800-2500 msg/s). When scaling he
numbe o messages om 100 o 1000 o he clien s, he
published h oughpu ends we e compa able. Howe e ,
he e he ecei ed h oughpu o all b oke s ollows a
mo e no iceable dec easing end, as seen in Fig.5, which
is shown in loga i hmic scale due o ex eme di e ence
in he ecei ed h oughpu be ween he EMQX and
Ve neMQ implemen a ions (in ange 1×104), Rabbi MQ
and Mosqui o implemen a ions (in ange 1×103), and
he Hi eMQ implemen a ion (less han 1×102). As we
can see he e is a di e ence o mo e han 10 000 msg/s
in case o 100 concu en clien s. Fo small numbe s o
concu ency (100-300), he di e ence om EMQX and
Ve neMQ o he o he b oke s is e y high, wi h he gap
closing he mo e he clien numbe inc eases. The mos
su p ising esul is ha he b oke implemen a ion wi h
he highes publishing h oughpu - Rabbi MQ, p o ides
a much lowe ecei ed h oughpu han he b oke s wi h
lowe published h oughpu - EMQX, Mosqui o and
Ve neMQ. These esul s showcase he impo ance on
measu ing h oughpu on bo h MQTT clien sides, as high
h oughpu pe o mances in e ms o messages a i ing
a he b oke migh no ge e lec ed in he numbe o
messages ha he b oke is capable o o wa ding o he
end use s.
V. Conclusion and ou look
We expe imen ally benchma ked he pe o mance o i e
open-sou ce MQTT B oke implemen a ions a di e en
loca ions ac oss he compu e con inuum. To his end, we
enginee ed a ep oducible ne wo k es bed by emula ing
cloud and edge ne wo king cha ac e is ics, suppo ing de-
ices o bo h, AMD64 and ARM64 ha dwa e a chi ec u es.
When analyzing he pe o mance o se o he es bed
i sel , we ound compa able esponse imes o all b oke s
when deployed in he same ne wo k scena ios and on
he same ha dwa e pla o m. In cases o small message
payload sizes, wi h deg ada ion o ne wo k condi ions, a
small la ency inc ease could be no iced o each o he
b oke implemen a ions. Unde he same condi ions, he
la ency di e ences be ween hem could be conside ed neg-
ligible. Once he message payload size inc eased o 1MB,
he pe o mance o all implemen a ions in all ne wo k
scena ios de e io a es wi h espec o bo h; he median
la ency and i s de ia ion. Howe e , he la ges pe o -
mance di e en ia o has been in he es bed’s ha dwa e
se up i sel , wi h he AMD64 se up esul ing in much
lowe esponse imes han he ARM64 se up. Finally, he
mos unexpec ed esul s we e ob ained o he h oughpu
measu emen s unde a high message and clien load. The
esul s showcased he impo ance o measu ing h ough-
pu on bo h MQTT clien sides, as he high published
h oughpu pe o mance we e no e lec ed in he same
numbe o messages he b oke was capable o o wa ding
