A ailable online www.ejae .com
Eu opean Jou nal o Ad ances in Enginee ing and Technology, 2021, 8(11):152-155
Resea ch A icle
ISSN: 2394 - 658X
152
Real-Time Big Da a P ocessing wi h Edge Compu ing
Rajesh Kuma Kanji
Independen Resea che ,
Plano, TX, USA-75024
_____________________________________________________________________________________________
ABSTRACT
The su ge o In e ne o Things (IoT) de ices and he apid inc ease in da a gene a ion ha e equi ed
imp o emen s in big da a p ocessing echniques. Con en ional cloud-based sys ems, al hough esilien ,
equen ly ace issues conce ning la ency, bandwid h limi a ions, and eal- ime p ocessing. Edge compu ing
ep esen s a e olu iona y model ha posi ions compu a ion and da a s o age in p oximi y o he da a sou ce. By
le e aging edge compu ing's p oximi y o da a sou ces, his s udy aims o educe la ency, enhance da a
p ocessing speeds, and imp o e o e all sys em e iciency. The indings highligh he po en ial o edge-based
a chi ec u es in add essing he challenges o adi ional cloud-based models, pa icula ly in ime-sensi i e
applica ions. This pape examines s a egies o enhancing la ency and pe o mance in edge-based big da a
a chi ec u es. We illus a e how edge compu ing can ans o m eal- ime da a p ocessing h ough heo e ical
insigh s and s a is ical e e ences.
Keywo ds: Big da a analysis, Edge compu ing, In e ne o Things (IoT)
____________________________________________________________________________________
INTRODUCTION
The exponen ial inc ease in da a gene a ed by In e ne o Things (IoT) de ices and o he sou ces has necessi a ed
no el app oaches o da a p ocessing. T adi ional cloud compu ing models, while e ec i e, equen ly expe ience
la ency p oblems due o he dis ance be ween da a sou ces and cen alized se e s. Edge compu ing o e s a
p omising solu ion by b inging compu a ion close o he da a sou ce, he eby educing la ency and imp o ing
esponse imes [1]. Edge compu ing, which p ocesses da a a o nea he sou ce, add esses hese challenges by
educing he need o da a ansmission o cen alized se e s. By le e aging edge de ices and nodes, o ganiza ions
can achie e as e esponse imes and enhanced pe o mance. This pape del es in o he p inciples o edge
compu ing in big da a p ocessing, wi h a ocus on op imizing la ency and pe o mance [2].
Figu e 1: Edge Compu ing In as uc u e
Edge compu ing decen alizes da a p ocessing by placing compu a ional esou ces a he "edge" o he ne wo k,
close o whe e da a is gene a ed. This app oach no only minimizes la ency bu also educes he amoun o da a ha
needs o be ansmi ed o cen alized da a cen e s, he eby lowe ing bandwid h usage and associa ed cos s.
Kanji RK Eu o. J. Ad . Engg. Tech., 2021, 8(11):152-155
153
Mo eo e , edge compu ing enhances da a secu i y and p i acy by allowing sensi i e in o ma ion o be p ocessed
locally a he han being sen o ex e nal se e s [3]. These ad an ages make edge compu ing an a ac i e op ion
o a ious applica ions, including au onomous ehicles, sma ci ies, heal hca e moni o ing sys ems, and indus ial
au oma ion [4].
The in eg a ion o a i icial in elligence (AI) wi h edge compu ing u he ampli ies i s po en ial. AI algo i hms can
be deployed a he edge o pe o m eal- ime analy ics, p edic i e main enance, anomaly de ec ion, and o he
ad anced asks. This combina ion o edge compu ing and AI enables mo e in elligen and esponsi e sys ems,
capable o making decisions and aking ac ions wi hou elying on dis an cloud se e s [5]. As a esul , edge-based
a chi ec u es a e inc easingly being adop ed o mee he demands o mode n, da a-in ensi e applica ions.
Despi e i s po en ial, he adop ion o edge compu ing in big da a p ocessing is no wi hou challenges. Issues such as
esou ce managemen , in e ope abili y, and scalabili y need o be add essed o ully ealize he bene i s o edge
compu ing. Addi ionally, he in eg a ion o AI a he edge equi es obus ha dwa e and so wa e solu ions o ensu e
e icien and eliable pe o mance. This esea ch pape aims o explo e hese aspec s and p o ide insigh s in o
op imizing la ency and pe o mance in edge-based big da a a chi ec u es.
LITERATURE REVIEW
A s udy by Fu and Lee (2019) examined he ole o edge compu ing in enhancing eal- ime da a p ocessing o IoT
applica ions. They highligh ed how edge compu ing add esses la ency and bandwid h challenges by p ocessing da a
close o he sou ce, he eby imp o ing e iciency and enabling eal- ime decision-making. This app oach is
pa icula ly bene icial in applica ions equi ing imely esponses, such as heal hca e moni o ing and indus ial
au oma ion [6]. A comp ehensi e su ey by Zhou e al. (2019) explo ed he in eg a ion o edge compu ing wi h
a i icial in elligence, highligh ing how Edge AI can expedi e da a p ocessing and acili a e eal- ime in e ence [7].
Mo eo e , Anaya, Acos a-Be mejo, and Salinas-Rosales (2018) discussed he in eg a ion o edge, IoT, and cloud
compu ing in a dis ibu ed en i onmen , highligh ing he ad an ages o p ocessing da a locally o alle ia e da a
ans e challenges and enhance secu i y [8]. Simila ly, Langona e al. (2016) examined he ole o og compu ing in
da a analy ics and cloud dis ibu ed p ocessing a he ne wo k edges, u he suppo ing he bene i s o edge
compu ing in eal- ime da a p ocessing [9].
These s udies collec i ely unde sco e he po en ial o edge compu ing in op imizing la ency and pe o mance in big
da a a chi ec u es. By p ocessing da a close o he sou ce, edge compu ing minimizes delays and enhances he
e iciency o eal- ime applica ions, making i a iable solu ion o indus ies equi ing immedia e da a analysis,
such as au onomous ehicles and heal hca e moni o ing sys ems.
METHODOLOGY
Algo i hm: Big Da a P ocessing a Edge Nodes in a Cloud Compu ing Sys em
Inpu :
• Da a S eam (D): Con inuous inpu da a om IoT de ices o senso s.
• P ocessing Tasks (T): A se o da a p ocessing asks (e.g., il e ing, agg ega ion, and analy ics).
• Edge Nodes (EN): A se o edge compu ing nodes a ailable o p ocessing.
• Resou ce Limi s (R): Compu a ional and memo y limi s o each edge node.
Ou pu :
P ocessed Da a (P): Agg ega ed and analyzed da a eady o cloud s o age o eal- ime decision-making.
Inpu : Da a S eam D, P ocessing Tasks T, Edge Nodes EN, Resou ce Limi s R
Ou pu : P ocessed Da a P
Begin
// Phase 1: Ini ializa ion
Ini ialize Node Regis y NR wi h me ada a o edge nodes EN
De ine p ocessing asks T wi h p io i y le els and esou ce equi emen s
// Phase 2: Da a Inges ion and Pa i ioning
While Da a S eam D is ac i e do
Pa i ion da a D in o chunks d_i based on size o ype
// Phase 3: Resou ce Alloca ion
Fo each da a chunk d_i do
Selec edge node EN_j om NR wi h su icien esou ces R
Kanji RK Eu o. J. Ad . Engg. Tech., 2021, 8(11):152-155
154
Assign d_i o EN_j
EndFo
EndWhile
// Phase 4: Edge Node P ocessing
Fo each edge node EN_j do
Fo each assigned da a chunk d_i do
P ocess d_i using p ede ined asks T
S o e in e media e esul s locally o ansmi o neighbo ing nodes i needed
EndFo
EndFo
// Phase 5: Load Balancing
Pe iodically assess load on all edge nodes in NR
Redis ibu e asks om o e loaded nodes o unde u ilized nodes
// Phase 6: Da a Compila ion and T ansmission
Collec p ocessed da a chunks P_i om edge nodes
Agg ega e P_i in o inal esul s P
T ansmi agg ega ed da a P o cloud s o age
// Phase 7: Pe o mance Moni o ing and Adap a ion
Con inuously moni o pe o mance me ics (la ency, h oughpu , scalabili y)
Adap ask scheduling and da a pa i ioning dynamically based on moni o ed da a
End
The me hodology o implemen ing he p oposed algo i hm o big da a p ocessing a edge nodes is s uc u ed in o
he ollowing phases:
1. Sys em Ini ializa ion:
O Edge Node Regis a ion: Ini ialize he sys em by egis e ing a ailable edge nodes wi h hei espec i e
compu a ional and memo y capabili ies in a cen al Node Regis y.
O Task De ini ion: De ine da a p ocessing asks, including il e ing, agg ega ion, and analy ics, along wi h hei
esou ce equi emen s and p io i y le els.
2. Da a Inges ion and Pa i ioning:
O Real- ime da a s eams om IoT de ices a e moni o ed and inges ed.
O Da a is pa i ioned in o manageable chunks o sub-s eams based on size, da a ype, o p ocessing p io i y.
3. Resou ce Alloca ion:
O Assign da a chunks o edge nodes dynamically based on hei cu en load and esou ce a ailabili y.
O Implemen esou ce-awa e scheduling o ensu e asks a e dis ibu ed e enly and c i ical asks a e p io i ized.
4. Edge Node P ocessing:
O Each edge node p ocesses he assigned da a chunks using p ede ined asks such as p ep ocessing, machine
lea ning in e ence, o anomaly de ec ion.
O In e media e esul s a e s o ed locally o ansmi ed o neighbo ing nodes o u he p ocessing, depending on
ask equi emen s.
5. Load Balancing:
O Pe iodic load assessmen is conduc ed ac oss all edge nodes o iden i y bo lenecks o unde u ilized esou ces.
O Tasks a e edis ibu ed dynamically o ensu e balanced u iliza ion and minimize p ocessing delays.
6. Da a Compila ion and T ansmission:
O P ocessed da a om edge nodes is compiled and agg ega ed in o inal esul s.
O Agg ega ed da a is ansmi ed o he cloud o long- e m s o age o u he analysis, ensu ing minimal la ency.
This me hodology ensu es ha he algo i hm achie es op imal esou ce u iliza ion, low la ency, and high
h oughpu while main aining scalabili y in edge-based big da a a chi ec u es.
This me hodology ensu es ha he algo i hm achie es op imal esou ce u iliza ion, low la ency, and high
h oughpu while main aining scalabili y in edge-based big da a a chi ec u es.
RESULTS & DISCUSSION
The implemen a ion o he p oposed algo i hm demons a ed signi ican imp o emen s in eal- ime big da a
p ocessing using edge nodes. The esul s we e analyzed based on key pe o mance me ics, and he indings a e
summa ized as ollows:
Kanji RK Eu o. J. Ad . Engg. Tech., 2021, 8(11):152-155
155
1. La ency Reduc ion:
O The algo i hm achie ed a educ ion in la ency compa ed o adi ional cloud-cen ic a chi ec u es. The a e age
end- o-end p ocessing ime o da a chunks dec eased signi ican ly.
2. Th oughpu Enhancemen :
O Wi h dynamic esou ce alloca ion and e icien load balancing, he h oughpu inc eased by a e y good amoun .
3. Scalabili y:
O The edge-based a chi ec u e main ained consis en pe o mance unde high da a loads, scaling be e han he
baseline olume wi hou signi ican deg ada ion in la ency o h oughpu .
CONCLUSION
The in eg a ion o edge compu ing wi h eal- ime big da a p ocessing o e s signi ican ad ancemen s in add essing
he la ency, h oughpu , and scalabili y challenges o adi ional cloud-based sys ems. By le e aging dynamic
esou ce alloca ion, e icien load balancing, and localized p ocessing a he edge, pe o mance is no ably enhanced
while educing ene gy consump ion. The inco po a ion o machine lea ning models a edge nodes u he augmen s
decision-making accu acy, enabling eal- ime insigh s ac oss a ious applica ions such as sma ci ies and
heal hca e. These imp o emen s highligh edge compu ing's po en ial as a ans o ma i e pa adigm in big da a
p ocessing, p o iding a scalable and e icien solu ion o mode n da a-in ensi e applica ions.
Fu u e esea ch should ocus on e ining secu i y measu es, op imizing he syne gy be ween edge and cloud
compu ing, and ex ending he algo i hm o handle mo e complex and dynamic da ase s. This will ensu e he
adap abili y and obus ness o edge compu ing in e ol ing echnological landscapes. The con inued e olu ion and
in eg a ion o ad anced AI echniques will u he enhance he capabili ies o edge-based sys ems, d i ing
inno a ion and imp o ing se ice quali y in c i ical sec o s.
REFERENCES
[1]. Sa yana ayanan, M., e al. (2017). The ole o edge compu ing in cloud compu ing. IEEE Compu e
Socie y.
[2]. Mille , P. (2018, May). Wha is edge compu ing? The Ve ge.
[3]. Li, S., T y onas, T., & Li, H. (2016). The In e ne o Things: A secu i y poin o iew. In e ne Resea ch,
26(2), 337–359.
[4]. Gubbi, J., Buyya, R., Ma usic, S., & Palaniswami, M. (2013). In e ne o Things (IoT): A ision,
a chi ec u al elemen s, and u u e di ec ions. Fu u e Gene a ion Compu e Sys ems, 29(7), 1645–1660.
[5]. Bonne , P., Geh ke, J., & Seshad i, P. (2001). Towa ds senso da abase sys ems. Mobile Da a
Managemen , 3(1), 3–14.
[6]. Fu, Z., & Lee, Y. C. (2019). Edge compu ing: A u u e end o IoT and big da a p ocessing. In M. U. S.
Khan, S. U. Khan, & A. Y. Zomaya (Eds.), Big Da a-enabled In e ne o Things (pp. 373–386). Ins i u ion
o Enginee ing and Technology.
[7]. Zhou, Z., Li, D., Zeng, L., & Chen, X. (2019). Edge AI: On-demand accele a ing deep neu al ne wo k
in e ence ia edge compu ing. In P oceedings o he 2019 IEEE In e na ional Con e ence on Edge
Compu ing (EDGE) (pp. 1–6). IEEE.
[8]. Anaya, E., Acos a-Be mejo, R., & Salinas-Rosales, M. (2018). Dis ibu ing compu ing in he In e ne o
Things: Cloud, og, and edge compu ing o e iew. In NEO 2016: Resul s o he Nume ical and
E olu iona y Op imiza ion Wo kshop NEO 2016 and he NEO Ci ies 2016 Wo kshop (pp. 87–115).
Sp inge In e na ional Publishing.
[9]. Langona, K., Sil a, E. A., de B i o Ca alho, T. C. M., & Se idcon, A. (2016). Fog compu ing: Da a
analy ics and cloud dis ibu ed p ocessing on he ne wo k edges. In 2016 35 h In e na ional Con e ence o
he Chilean Compu e Science Socie y (SCCC) (pp. 1–9). IEEE.