Expe imen al Demons a ion o Real-Time Image-P ocessing using a
VLSI Analog P og ammable A ay P ocesso
Gus a o Lj iáa, Ra ael DomInguez-Cas o, Se ando Espejo, Elisenda Rocaa,), Pe e Fo1desy
and Angel Rod Iguez-Vazquez
de Mic oelec ónica de Se illa, Cen o Nacional de Mic oelec ónica, A da. Reina Me cedes
s/n, Campus Uni e sidad de Se illa, E-41012 Se illa (Spain).
bA ea de Elec ónica, Escuela Supe io de Ingenie os, Uni e sidad de Se illa, Camino de los
Descub imien os s/n, Isla de la Ca uja, E-41092 Se illa (Spain).
Hunga ian Academy o Sciences, SzTAKI, Analog and Neu al Compu ing Labo a o y, Kende u ca 13-
17, H- 1 1 1 1 , Budapes (Hunga y)
ABSTRACT1
This pape desc ibes a ull-cus om mixed-signal chip which embeds dis ibu ed op ical signal acquisi ion, digi ally-
p og ammable analog pa allel p ocessing, and dis ibu ed image memo y —cache—on a common silicon subs a e. This chip,
designed in a O.5p m CMOS s anda d echnology con ains a ound 1, 000, 000 ansis o s, 80% o which ope a e in analog
mode; i is hence one he mos complex mixed-signal chip epo ed o now. Chip unc ional ea u es a e in acco dance o he
CNN Uni e sal Machine pa adigm: cellula , spa ial-in a ian a ay a chi ec u e; p og ammable local in e ac ions among
cells; andomly-selec able memo y o ins uc ions (elemen a y ins uc ions a e de ined by speci ic alues o he cell local
in e ac ions); andom s o age/ e ie al o in e media e images; capabili y o comple e algo i hmic image p ocessing asks
con olled by he use -selec ed s o ed ins uc ions and in e ac ing wi h he cache memo y, e c. Thus, as illus a ed in his
pape , he chip is capable o comple e complex spa io- empo al image p ocessing asks wi hin sho compu a ion ime
(200ns o linea con olu ions) and using a low powe budge (<1.2W o he comple e chip). The in e nal ci cui y o he
chip has been designed o ope a e in obus manne wi h >7-bi equi alen accu acy in he in e nal analog ope a ions, which
has been con i med by expe imen al measu emen s. Hence, o all p ac ical pu poses, p ocessing asks comple ed by he chip
ha e he same accu acy han hose comple ed by digi al p ocesso s p eceded by 7-bi digi al- o-analog con e e s o image
digi aliza ion. Such 7-bi accu acy is enough o mos image p ocessing applica ions.
The pape b ie ly desc ibes he chip a chi ec u e and ocus mos ly on p esen ing expe imen al e idences o he chip
unc ionali y. Mul iscale low-pass and high-pass il e ing o g ay-scale images, analog edges ex ac ion, image segmen a ion,
h esholded g adien de ec ion, ma hema ical mo phology ope a ions, sho es pa h de ec ion in a laby in h, skele onizing,
image econs uc ion, se e al non-linea ype image p ocessing aks like absolu e alue calcula ion o g ay-scale g adien
de ec ion and eal- ime mo ion de ec ion in QCIF ideo sequences a e some o he e y in e es ing applica ions ha ha e
been demons a ed as a ailable when using he p o o ype.
1. INTRODUCTION
In mos mode n elec onic sys ems he ole o analog is basically limi ed o ha o an in e ace be ween eal-wo ld analog
signals and he numbe s handled by he co e digi al p ocesso s. In he case o sys ems in ended o p ocessing uni-
dimensional signals, in e ace unc ions comp ise basically impedance ma ching, ampli ica ion, il e ing and analog- o-
digi al con e sion. P ocessing is ealized in digi al domain by subsequen s ages o he p ocessing chain. Howe e , in he
case o sys ems in ended o p ocessing la ge, mul i-dimensional, in e ela ed signals — such as image lows — i may be
in e es ing o inco po a e pa allel p ocessing a he analog in e ace in o de o educe he amoun o da a been ans e ed o
he digi al sec ion, and hence o imp o e e iciency o he whole p ocessing chain.
i. This wo k has been pa ially unded by ONR-NICOP N68171-98-C-9004, DICTAM IST-1999-19007 and TIC 990826.
In Applica ions o A i icial Neu al Ne wo ks in Image P ocessing V, Nasse M. Nas abadi, Aggelos K.
Ka saggelos, Edi o s, P oceedings o SPIE Vol. 3962 (2000) • 0277-786X1001$15.00 235
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use
Ac ually, such analog pa allel p ep ocessing is obse ed al eady in he e ina — he senso y on -end o na u al "image
p ocessing" sys ems. I embeds pho o ecep o cells o acqui e images, and p ocessing cells o pe o m non-linea spa ial-
empo al p ocessing ope a ions on he incoming low o images h ough a sequence o laye s. Mo i a ed by he e iciency o
na u al ision sys ems, uni e si ies and companies ha e ocused hei e o s on he de elopmen o new gene a ions o
de ices capable o o e coming he d awbacks o adi ional ones h ough he inco po a ion o dis ibu ed pa allel p ocessing,
and, by making his p ocessing ac concu en ly wi h he acquisi ion o he signal. One possible s a egy o achie e ha is
h ough lip-chip bonding o sepa a e sensing and p ocessing de ices; ano he possibili y is o inco po a e he senso y and he
p ocessing ci cui y on he same semiconduc o subs a e. "Silicon e inas", "sma -pixel chips" and " ocal-plane a ay-
p ocesso s" a e membe s o his la e class o ision chips 2Thei de elopmen is expec ed o ha e a signi ican impac in
qui e di e se scena ios. Howe e , indus ial applica ions demand chips capable o lexible ope a ion, wi h p og ammable
ea u es and s anda d in e acing o con en ional equipmen . A powe ul me hodological amewo k o a sys ema ic
de elopmen o hese ypes o chips is using he pa adigm o analog cellula compu ing whose mos ad anced
implemen a ion is he so-called CNN uni e sal machine (CNN-UM) .
Main p ocessing componen s o his a chi ec u e a e: a) pa allel a chi ec u e consis ing o an a ay o locally-connec ed
analog p ocesso s; b) a means o s o ing, locally, pixel-by-pixel, he in e media e compu a ion esul s, and 3) s o ed on-chip
p og ammabili y. When implemen ed as a mixed-signal VLSI chip wi h embedded dis ibu ed op ical senso s, image
p ocessing a es o illions o ope a ions pe second can be achie ed wi h e y small size and low powe consump ion.
Ac ually, his has been al eady demons a ed by he au ho s h ough a O.8 m mixed-signal chip con aining 20 x 22 cells and
capable o only bina y image p ocessing . This pape desc ibes, om a high le el poin o iew, ano he chip belonging o
he same amily which ou pe o ms p e ious ealiza ions ega ding bo h size, and unc ionali y; among o he hings his new
chip is capable o g ay-scale p ocessing. The pape is o ganized as ollows: Sec ion 2 desc ibes he chip om a sys em poin
o iew. Sec ion 3 p o ides he in o ma ion abou he speci ic elec onic implemen a ion ocussing in o he mapping o he
cell s a e equa ion and in o he desc ip ion o he enhanced unc ionali ies ha ha e been added o each elemen a y
p ocessing uni o inc ease he inal pe o mances o he p o o ype. Sec ion 4 shows wo applica ion examples while Sec ion
5 deals wi h he implemen a ion o non-linea image p ocessing asks.
2. CHIP DESCRIPTION
The chip consis s basically o a squa e a ay o 64 x 64 iden ical cells (plus a su ounding ing o bo de cells which a e
ob iously di e en ), each o which in e ac s wi h i s nea es neighbo s in acco dance o he so called ull signal ange
model (FSR4).
dx'J( ) = —g[x,j )]+ :A(i,j;k,l).xkl( )+ :B(i,j;k,l).ukl+z (1)
C(k,I) E S (1,J) C(k, 1) E S (E,J)
whe e x, ep esen s he s a e o he gene ic i — h cell; Xkl ep esen s he s a e o he cells loca ed in he in e ac ion
neighbo hood o he i — h cell; A(i,j;k, 1) , B(i,j;k,1) and z a e p og ammable pa ame e s; and,
In addi ion o he ci cui y needed o implemen (I), each cell con ains he ollowing: a) an op ical senso ; b) ou
andomly-add essable analog memo y poin s plus o he ou digi al memo y poin s (i means ha he chip can s o e ou
g ay-scale images plus ou bina y images); c) a p og ammable local logic uni o ealiza ion o logical ope a ions among
bina y images; d) inpu /ou pu and con ol ci cui y.
mJ ,x1( )<—l
mR ,x1( )> I
mL
mR
(2)
236
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use
Fo gi en inpu image ( ep esen ed by he ma ix o u1 alues), ei he cap u ed by he op ical senso s o e ie ed om
some o he cache memo ies, and gi en alue ze o-s a e image ( ep esen ed by he ma ix o x(O) alues, he chip can
ealize a la ge a ie y o basic p ocessing asks (ins uc ions) h ough adjus men o he p og ammable pa ame e s in (1);
namely he bias e m z and he in e ac ion pa ame e s which can be g ouped in o he so-called eedback empla e and con ol
empla e,
A = [A(i,j;k,l)] =B = [B(i,j;k,1)] (3)
whe e means op, c cen e , b bo om, 1 le and igh . Thus, o ins ance, A1 deno es he scaling coe icien o he
con ibu ion o he cen e -le cell s a e a iable in o equa ion (1).
Basic chip ins uc ions a e hence de ined by 19 (ac ually 20 ) alues. In he chip hese alues a e p og ammable wi h a
esolu ion o se en bi s plus sign. F om an ex e nal poin o iew, images may be analog (g ay-scale) o bina y (black &
whi e). In e nally, pixel alues a e ea ed as analog in gene al, wi h black & whi e images ha ing ex eme analog le els
co esponding o he limi s o he linea egion. Su ounding bo de cells a e used o es ablish he necessa y spa ial bounda y
condi ions o p ocesses. O he miscellaneous unc ions like analog and digi al bu e ing, con ol, and I/O asks, a e also
included wi hin he bo de cells.
Figu e 1 (a) shows he chip a chi ec u e. In addi ion o he co e a ay p ocesso , he chip includes some global con ol and
p og amming ci cui y loca ed in he pe iphe y o he cell a ay. This includes memo y o 32 a bi a y se s o coe icien s,
which a e being p og ammed can be a bi a ily selec ed om he ou side. Some o he analog alues ela ed o he
p ocessing ci cui y, like he limi s o he linea egion and o he s, can also be p og ammed. Digi al o analog (DA)
con e e s gene a e he analog-p og am signal le els ansmi ed o he cell a ay om he selec ed se o coe icien s. Also,
64 a bi a y se s o 35 digi al signals ha a e used as digi al ins uc ions o con igu e p ope ly he cell in o de o pe o m
di e en ask anging om unning a p ocess o con igu e he cell I/O ci cui y. These memo ies can be andomly add essed
om he hos ing pla o m once hey ha e been p og ammed. Figu e 1 (b) shows he chip mic opho og aph.
The ex e nal con ol is comple ely digi al. The in e ace has been designed o be easily embedded in con en ional digi al
sys ems cen e ed a ound a CPU o a DSP uni . Two bidi ec ional da a-buses, one analog and one digi al, a e employed o
image ans e ences.
Figu e 1. (a) Chip A chi ec u e. Figu e 1. (b) Chip Mic opho og aphy
237
Digi al I/O Da a bus - - - -
9.145mm
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use
The p o o ype has been designed and manu ac u ed in a 0.5 im,
single-poly, 3-me al laye s, CMOS echnology. Table 1 shows he
mos impo an physical and elec ical da a o he p o o ype.
3. ARRAY CIRCUITRY
In an elec onic implemen a ion, equa ion (1) mus be eplaced
by a simila one in which a iables and pa ame e s ha e physical ____________________ ___________________
meaning. The p ocessing ci cui y employed in he p o o ype
ep esen s he s a e a iable x and he cell' inpu u as ol ages
ac oss capaci o s. Con ibu ions om cell o cell a e in he o m o
cu en , and he e o e, empla e elemen s ha e he dimension o
conduc ances. In his manne , incoming con ibu ions, in he o m o
cu en s, a e easily added a a common node, and easily in eg a ed in ___________________
a capaci o , while cell' s a e and inpu a iables, in he o m o
ol age, a e easily dis ibu ed o he synapses. The cell s a e
equa ion can he e o e be w i en, in in eg al o m, as ollows :
( ) = (O)+[-I( ( )) +d1G ( ) +
d=1 G + JD]d (4) ______________ _____________
The nonlinea dissipa i e e m Ig(V) . ske ched in Figu e 2, pe o ms
an e ec i e ol age limi a ion o he s a e a iable x wi hin he
linea egion [Vsa Vsa ] by "elimina ing" wha e e cu en is _____________________
necessa y. The same signal- ange is employed o he inpu signal
U, which is sampled and s o ed in a capaci o C iden ical o ha employed o he in eg a ion.
I , a we assign he index 0 o an imagina y neighbo gene a ing he bias, o o se , e m 'D and se i s inpu and s a e
a iables o he sa u a ion le el Vsa
238
ii. The index d e e s o he eigh nea es neighbo s o he cell and he cell i sel
Table 1. P o o ype Da a.
# o Cells 4096 (64 x 64 A ay)
# o T ansis o s —l.OOO.OOO
# T ansis o s on he cell I 72
Cell Size l2O m x lO2.2m
Cell Densi y -82 cells/mm2
Signal Swing [0.6, l.41V (P og am-
mable.)
Weigh Swing [2.15, 2.95]V (P og am-
mable.)
Time Cons an -l.2 s
Time Cons an o Linea
Con olu ions 2OOns
Spa ial Uni o mi y on he
Weigh Signals. 7.6-bi s
110 Digi al Ra e 10MHz
110 Analog Ra e 1MHz
Powe Supply 3.3V
Powe pe cell 250 W
Powe Consump ion 1 .2W (wo s case)
# o Templa es Memo-
ized 32
# o lns uc ions 64
Die Size 9l45.10 m x 9534 j m
C
mL ,Vx<Vsa mJ
mR —*
C
mR ,Vx>Vsa
(5)
—V
Figu e 2. Cell' Non-Linea i y.
C
Vx
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use
00
x = u =Vsa (6)
hen, (4) can be ew i en as:
( ) = (O)+;L[- J( ( ))+
dO d=O1 (7)
whe e he o se e m is now exp essed as:
'D (G +G)Vsa (8)
which is now gene a ed in he same o m (i.e., wi h he same ci cui blocks) and wi h he same e e ence (sa u a ion)
le el
Vsa han any o he con ibu ion o he in eg andin (4). The use o he sa u a ion le el as a e e ence o he gene a ion o he
bias e m p o ides some addi ional homogenei y, since con ibu ions om sa u a ed cells a e p opo ional o hesa u a ion
le el. No e also ha wo addi i e coe icien s a e employed o de ine he o se e m hus p o iding a double ange
o he
o se e m.
3.1 Enhanced Func ionali ies
Some addi ional unc ionali ies ha e been inco po a ed which ha e speci ic applica ion in ele an p ocessing unc ions.
3.1.1 Image Memo ies
E e y cell has he capabili y o s o ing ou analog (g ay-scaled) and ou bina y (black & whi e) pixel alues. A a sys em
le el, his means ha he chip allows he s o age o a o al o eigh images. The use o any o hese images as inpu da a o
some p ocess, o o some speci ic memo y as des ina ion o he esul o he p ocessing unc ion, can be ealized in a e y
sho ime (a ound 0. 1 s). In u n, his esul s in a signi ican imp o emen in he compu a ion ime o mo e complex
algo i hms equi ing i e a i e empla e applica ions, wi h he possibili y o s o ing in e media e esul s o bi u ca ed- low
algo i hms. Bina y memo ies employ con en ional digi al la ches, while analog memo ies elay on "bo om-pla e sampling"
swi ched-capaci o s ages ollowing he guidelines gi en in .
3.1.2 Local Logic Uni
The local logic uni (LLU) is a p og ammable boolean ga e whose u h able is de ined as pa o he digi al ins uc ions
s o ed in he p og amming ci cui y. I allows a comple ely pa allel ealiza ion o a bi a y bi - o-bi logic ope a ions be ween
images s o ed a wo speci ic bina y memo ies. The esul ing image can be down-loaded o s o ed in anyo he ou bina y
memo ies. Con en ional digi al ci cui y is employed o his pu pose.
3.1.3 F eezing Mask
Ha ing a " eezing" mask means ha he con en o one speci ic bina y image memo y can (op ionally) be used as a lag
which disables he e olu ion o he ma ked pixels du ing p ocessing ansien s, keeping hei s a e a iables ime-in a ian .
The ealiza ion o his unc ion equi es jus a ew analog swi ches.
11.4 Global Ga es
In many cases i is in e es ing o ind ou i some speci ic image ( o ins ance he ou come o some p ocess) is comple ely
whi e o comple ely black, wi hou was ing he ime equi ed o download he whole image. The p o o ype inco po a es wo
global ga es, one NAND and one NOR, o pe o m hese logic ope a ions o e he pixel alues o one speci ic bina y
memo y. Wi h his unc ionali y, he ime equi ed o check i some image is comple ely black o
whi e is abou 3ps.
239
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use

3.1.5 Op ical Inpu
In many eal-li e high-speed applica ions, he in o ma ion o be p ocessed by he ne wo k is an image ha is a ailable in
op ical o m while he ou pu con ains only a ew de ails ex ac ed om he inpu . In hese si ua ions, he eadou p ocess is
ex emely simpli ied and hence speeded up. Howe e , he hpu image is always a comple e ame and he e o e, he ime
needed o ans e he image o he a ay can cons i u e a bo leneck. In hose cases, he capabili y o combining he senso y
and he p ocessing planes, p o ides a d ama ic inc ease o he sys em pe o mances, since i p oduces sys ems ha do no
only exploi he ad an ages o he ully pa allel p ocessing bu also hose o he ully pa allel image acquisi ion ha a e
p o ided by a ma ix o pho osenso s me ged wi h ha o p ocesso s. Acco dingly o his, he chip inco po a es a
pho osensing de ice wi hin each cell ha allows he acquisi ion o images ha a e di ec ly p ojec ed o e he silicon su ace.
The sensing scheme is based on he in eg a ion, in he capaci o o any o he analog image memo y, o he cu en ha is
gene a ed by di usion-subs a e pho odiode.
4. APPLICATION EXAMPLES
4.1 Mo emen De ec ion
One o he key-ideas when coding a ideo sequence is he in e ame mo ion compensa ion 6• This s a egy codi ies he
i s image on he sequence as an independen i em while he emaining ames a e codi ied as a p edic ed ame and he
p edic ion e o . I he mo ion o he objec s in he scene is small enough and mos ly ansla ional, his scheme comp ises he
ideo sequence in a e y e icien manne .
Ou om all asks needed o MPEG-1,2 ideo comp ession, block mo ion es ima ion is c ucial in e ms o ime
consump ion. Block mo ion es ima ion is based on sys ema ic block-ma ching sea ch. This sea ch is based on inding he
candida e block, conside ing a Mean Absolu e Ji e ence c i e ion (MAD) which is he mos simila o he p edic ed one. In
his subsec ion, we will show, only wi h example pu pose, how he i s low-le el, bu e y in ensi e asks, a e pe o med by
he new p o o ype. Namely, he ex ac ion o blocks ha a e changed.
A i s s ep in he block mo ion es ima ion, is he de ec ion o he changes among wo consecu i e ames. Du ing his ask,
image subs ac ion, low-pass il e ing, and double- h esholding a e pe o med on chip (see Figu e 3). The ime needed o
accomplish his i s s ep is l1Oxs , ha is, he p ocessing ime is 25ns pe pixel, while he o al numbe o ope a ions pe
pixel is I subs ac ion (eigh bi s o esolu ion) + 1 hea di usion (N x (10 mul iplica ions+ 10 addi ions) whe e N is he
numbe o i e a ions needed o achie e he desi ed equilib ium poin ) + 1 logic ope a ion.
The second s ep is o ma k he changed blocks. The goal o his s ep is o ma k he blocks whe e a leas one pixel has
changed. This is done by using he ixed s a e map and black wa e p opaga ion. The wa e ills up he block-sized a eas
whe e a black pixel exis s, while he ixed s a e map s ops he wa e e olu ion a he bo de o each block. The equi ed ime
is 26 s , hus p o iding a p ocessing ime pe pixel o abou 6ns .The esul o his s ep can be seen also in Figu e 3.
.N
________
(b) Di e ence om 4 h ame (c) Hea Di usion (Low (d) Ma ked blocks
Pass Fil e ing) + Double
h esholding+ OR
Figu e 3. De ec ion o he changes in a ideo sequence.
240
,
(a) 5 h F ame
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use
4.2 Impulse Noise Remo al
The p oblem o emo ing noisy pixels om an inpu image can be o mula ed in a gene al way as ollows. Gi en a se o
pixel in ensi ies I(i,j) whe e 1 I NROWS 1Nc015 I ld a maximum a e o change among pixels R ,all hose pixels
pi whe e he local a e o change is la ge han R should be subs i u ed by he local a e aging M o he in ensi y o i s
neighbo s, whe e:
m1 k1
:
M=m-4k--1
8(9)
The low cha o he algo i hm ha we a e using in o de
o implemen his unc ion is shown in Figu e 4 and i wo ks
as ollows:
•Fi s o all, he use has o de ine wha is noise, ha is,
he use has o de e mine which is he maximum alue o
he di e ence among wo pixels ha makes a pixel o be
conside ed as co ec ins ead o as noisy'.
•Second, he local maxima and minima pixels a e
de ec ed. Du ing his calcula ion, only hose pixels whe e
he di e ence be ween i s in ensi y and ha o any o i s
neighbo s is la ge (o lowe , i depends on whe he we
look o maxima o minima loca ions) han he p e iously
de ined h eshold will u n o black.
•The esul o he maxima and minima de ec ions a e
agg ega ed by using a logic NOR ia he Local Logic
Uni . A e his las ope a ion, he esul ing image con ains
whi e do s only whe e a local maxima o minima was
de ec ed.
•This las image is sen o he ixed-s a e image memo y,
and a hea di usion (a e aging) p ocess is pe o med o e
he inpu image. Due o ixed s a e unc ion, ha is
included wi hin each cell, only hose pixels ha a e no
ma ked (black) will be p ocessed. Since hose pixels a e
conside ed as noise con ibu ions o he image, he a e
eplaced by he local a e aging o hei neighbo s.
Figu e 4 shows he low cha o his algo i hm while ..
.....Figu e 4. Algo i hm o Impulse Noise Remo al.
Figu e 5 shows an example o applica ion o a noisy image.
I can be obse ed how noisy pixels a he inpu a e eplaced by he a e aging o he in ensi ies o hei nea es neighbo s hus
inc easing signi ican ly he quali y o he image. The inpu image is an s anda d QCIF (176 x 144 )
size image ( he image is
cu in o nine o e lapped pieces ha a e p ocessed one a e he o he ) and he o al ime consump ion (including he ime o
9loading and 9 downloading p ocess) is sligh ly below 5.2ms hus yielding a pixel a e o 200ns .Among his ime, he
p ocessing akes 23ns . Du ing his 23ns each pixel pe o ms 18 compa isons, I logic ope a ion and a hea di usion
(N x (10 mul iplica ions+ 10 addi ions)) whe e N is he numbe o i e a ions needed o achie e he desi ed equilib ium
poin .
iii.We conside ha noisy pixels a e always sepa a ed by co ec pixels ( hus dis inguishing noise om sha p edges)
241
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use
Figu e 5. l'he Impulse Noise Remo al Applica ion
5. IMPLEMENTATION OF NON-LINEAR INTERACTIONS
5.1 Theo e ical Backg ound
Non-linea cell in e ac ions can be ealized by decomposing hem in o a sequence o se e al linea ones . We will deal
wi h he decomposi ion o empla es whe e only he B e m is a non-linea unc ion. Fu he mo e, we will assume ha he
non-linea i ies appea ing on he eed o wa d e m a e piecewise-linea unc ions and ha he inpu image is ime in a ian .
Le us suppose ha he non-linea piecewise unc ion can be exp essed as:
(10)
whe e a and 3 a e eal numbe s, and ha he linea egions a e de ined by a se o m b eaking poin s E '2''' . In ha
case, ha is also he mos common in p ac ice. he non-linea empla e can be decomposed in o a sequence o linea empla e
execu ions. The algo i hm8 uns as ollows:
•The p ocess s a s by selec ing he i s linea egion o he non-linea unc ion. Le us call R1 his egion ha is de ined
by he b eaking poin s E
•The nex s ep is o selec which a e he cells belonging o ha egion. This calcula ion is ealized by wo empla es exe-
cu ions and a logic ope a ion (all o hem a e done on-chip). Wi h he i s empla e, he so called h eshold empla e, we
d i e o black all hose cells ha ing >E,while wi h he second one, he so called in e se h eshold, we d i e o black all
hose cells ha ing <.Finally a logic AND ope a ion o bo h esul s will selec hose pixels whe e < <'2 " l
•Equa ions (Ii). (12). show he h eshold and he in e se h eshold empla e'.
000 30()
A= 010 B= OaO (II)
000 000
i . Keep in mind ha = au11 + and he subindex ki deno es he cell' neighbo s.
. These a e he FSR e sion o he empla es .
242
(a) Noisy Inpu Image (b) Ou pu
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use
000 —IO
A= B= O—aO =2 (12)
000 000
•The non-selec ed cells a e " ozen", by using he eezing mask p o ided by he chip, while in heselec ed ones he co -
esponding con ibu ion o he s a e equa ion is e alua ed and s o ed as a "bias map" ha will be upda ed (o no ) in he
nex i e a ion by adding he new esul o he one ha was p e iously s o ed. The upda inglaw o he s a e a iables o he
cells ha a e selec ed mus be gi en by he equa ion o a s aigh line (due o he ac ha kP() is linea be ween each wo
b eaking poin s) c ossing he poin s E and 2 All he poin s belonging o his line sa is y:
E—1 =qI()kp(1) (13)
21 '(2)'(1)
and om heoiy, i can be demons a ed ha his ela ionship is ob ained i he ollowing empla e is execu ed"1:
000 00
A= _i B= k-aO (14)
000 0 00
z=
whe e,
k = (2)'1) (15)
•The p ocess con inues o he nex linea egion.
•Finally, a empla e execu ion is needed. In his empla e he eedback e m is he same as in he o iginal one, he eed o -
wa d e m is se o ze o (modi ied B empla e), since i has been al eady calcula ed, and he o se e m is he addi ion o
he o iginal one z , and he "bias map " ha is s o ed in some memo y on he cell.
5.1.1 Absolu e Value Calcula ion
In his subsec ion we conside only pixel-wise ans o ma ions, o wi h o he wo ds, B empla es wi h he size o lx
1
As a consequence o missing neighbo connec ions he decomposi ion me hod can be simpli ied, a oiding he
accumula ion o he pa ial esul s. Mo eo e , he selec ion o cells belonging a gi en in e al is done by he wo h eshold
empla es, which also con ain only cen al elemen s (whe e a = 1, 0 and = 0):
000 000
A= 020 B= OaO Z1 (16)
000 000
i. The posi ion o he coe icien mus be o a ed in o de o pe o m his ope a ion o each o he neighbo s o he cell
appea ing as a non-linea connec ion on he o iginal B empla e. The e o e, each linea egion could equi e up o 16 em-
pla es and 8 logic ope a ions o be selec ed, 8 empla es o upda e he s a e a iable, and 8 empla es o pe o m he addi ion
o he esul s, ha is 32 empla es and 8 logic ope a ions.
243
Downloaded F om: h ps://www.spiedigi allib a y.o g/con e ence-p oceedings-o -spie on 27 Ap 2020
Te ms o Use: h ps://www.spiedigi allib a y.o g/ e ms-o -use