A low cost 3D vision system for positioning welding mobile robots using a FPGA prototyping system

A Low Cos 3D Vision Sys em o Posi ioning Welding Mobile Robo s using a FPGA p o oyping sys em I. Ibáñez, M.A. Agui e, A. To alba and L.G. F anquelo Escuela Supe io de Ingenie os. Uni e sidad de Se illa A da. Camino de los Descub imien os s/n, 41092 Se illa (SPAIN) [email p o ec ed], o[email p o ec ed], leopold[email p o ec ed] Abs ac – The aim o his wo k is o explo e some solu ions o a i icial ision sys ems applied o welding au onomous obo s. We ake ad an age om he UNSHADES-1 sys em, de eloped in Filia ion (blind o e alua ion). This sys em can be used o building a powe ul 3D ision sys em. The sys em concen a es and p ocess he h ee images in pa allel, p oducing an accu a e alue o he posi ion, ha can be imp o ed i he ela i e posi ion o he came as a e well de ined. I. INTRODUCTION Au onomous Robo s a e comple e sys ems ha ope a e inside complex en i onmen s wi hou being guided and con olled by human ope a o s. They can es ablish ela ionships, ake decisions in un- ime and adap hei ac ion plans o di e en ex e nal ci cums ances. The ecei ed in o ma ion make hem au o-p og ammables [1-2], al e ing hei p og am in o de o sol e he ac ual si ua ion. Those sys em ha e some AI in hei beha iou . Mos o cases, he equi ed in o ma ions a e ela ed o posi ion, speed, accele a ion, s eng h, o que, sizes and shapes om objec s and empe a u e. Quan i ica ion o hose measu es a e made by means o mechanical, op ical, e mical, elec ical, ul asounds, ... senso s. Howe e , ad anced esea ch specially akes in o accoun a i icial ision ansduce s. This esea ch can ake ad an age om he newes mic oelec onic sys ems, whe e design schedule ha e been sho ened using he ad anced FPGA’s. These de ices can p ocess images in “ eal ime”, p ocessing ame by ame on he ly. Be ween he a i icial ision based solu ions, he mos a ac i e ones o Robo ics applica ions a e objec ecogni ion and classi ica ion, ensambling, welding, s ee ing and synch onisa ion wi h o he obo s. One o he mos complex applica ion is welding sys ems based on au onomous obo s. An eme ging echnology ha concen a es s a egies based on me alu gy, obo ics, elec ical and elec onic enginee ing and mic oelec onics. Welding obo s a e widesp ead demanded sys ems, specially whe e he indus y has a high le el o au oma ion. This pape ela es a new cus om sys em o de e mining he exac posi ion o he welding poin . The me hod is based on a sys em wi h h ee low cos came as ha a e placed and ixed in he h ee space axes, and compu es he posi ion h ough he simul aneous p ocessing o he images aken by each came a. Each image is bina ised and he welding poin is ob ained using a p og ammable bina y h eshold. So, o each came a, he whole image is educed o a single poin . As he h ee images a e p ocessed simul aneously, e e y coo dina e is compu ed as pa o wo o hem and hen he inal coo dina e is gi en as an imp o ed p ecision numbe . Using a gene al pu pose sys em based on FPGA’s called UNSHADES [4] we ha e de eloped and es ed he comple e sys em. Typically his sys em would be made wi h a DSP sys em ha would spend many ime p ocessing e e y image in a se ial way and wouldn’ ake ad an age om he pa allelism in e ms o accu acy. This pape shows how o ake ad an age om he UNSHADES-1 sys em. We ha e de eloped a ull con olled sys em ha compu es he posi ion o he welding poin . In sec ion II we’ll desc ibe he UNSHADES amewo k, in sec ion III he Vision sys em is p esen ed, sec ion IV shows he sys em design and inally he conclusions will be p esen ed. II. UNSHADES FRAMEWORK A Ha dwa e Debugge is a comple e p og ammable sys em ha can be cus omized o di e en applica ions. Fig 1. UNSHADES-1 sys em   Hos Con olle XC4010XL In e ace & Con olle EPP Da alink P og ammable Oscilla o 0 o 100MHz VIRTEX De ice XCV50 o XCV800 Low Speed Con ol Signals Gene al pu pose Signals o Sys em In e ace Main ad an age is ha he comple e sys em can be moni o ized du ing execu ion ime. Main p ocesso s a e subs i u ed by an ad anced FPGA and un embedded in o he sys em i sel . I needed a o ally cus omized sys em can be buil using he ac ual ne lis . UNSHADES-1 gene al pu pose amewo k ha is based on one Xilinx ad anced FPGA o emula ing he cus om p ocesso o , in his case, image p ocessing. Main ad an age o UNSHADES sys em is ha he use ha e ull con ol o he execu ion o he FPGA due o he special cha ac e is ics o his de ice. UNSHADES means Uni e sidad de Se illa Ha dwa e Debugge Sys em, and allows a apid p o o yping and he debugging o he p ocesso ci cui du ing execu ion. Figu e 1 shows a gene al scheme o he sys em: Main componen s o UNSHADES sys em a e lis ed below: - An ad anced FPGA om Xilinx Vi ex amily [3]. This FPGA hos s he cus om design. - PC so wa e, he human in e ace, ha p o ides a comp ehensi e way o manipula ing he inne in o ma ion o he FPGA. - A p og ammable econoscilla o om Dallas Semiconduc o DS-1075, ha gene a es he sys em clock. This de ice can be uned om he PC. - The e’s ano he FPGA ac ing as he in e ace be ween he PC and he ad anced FPGA. All he communica ions, p o ocol adap ing and signal gene a ion a e suppo ed by de ice, which is smalle han ad anced FPGA an has ixed i s p og am. Communica ion be ween PC and UNSHADES ha dwa e is pe o med using he EPP p o ocol ha can each 2 Mby es/s, ha is speed enough o ou pu pose. UNSHADES so wa e is a comple e sys em ha allows a bi s eam managemen including some in e es ing ools: downloade , in e nal egis e s inspec o , in e nal egis e modi ie , oscilla o manage and design selec o , all ully in eg a ed wi hin he Xilinx s anda d design low. Fo an op imal speed in he bi s eam ans e , all he w i e and ead ope a ions a e pe o med using pa ial econ igu a ion echniques. III. POSITIONING SYSTEM DESCRIPTION The aim o his wo k is o build a simple, low cos and e ec i e 3D digi al ision sys em o acking he posi ion and p o ide a eedback in posi ioning au onomous obo s. Sys ems like he one p esen ed can enla ge he e ec i e li e o he au onomous obo s. Main d awbacks o cons uc ing hese sys ems a e [5]: 1s . Vision sys ems a e as expensi e as he obo i sel . 2nd. In his kind o obo ics sys ems high h oughpu echniques ha could gene a e he in o ma ion in eal- ime a e needed. 3 d. The e’ e many p oblems ela ed wi h isual in o ma ion, like shadows, illumina ions, uni, bi and h ee-dimensional, hidden objec s,… e c. The main objec i e is o o e come mos o hese di icul ies: 1s We p o ide wi h low cos CCD came as and ideo decode s, he cos o he sys em s ongly depends on he equi ed p ecision. Fo ou p o o ype we used small digi al came as. 2nd The p ocessing echniques do no need o p ocess he whole image o each came a. A simpli ied me hod allows a esou ce sa ing ha makes a de ice like Vi ex V300 big enough o he h ee came as. They a e p ocessed in pa allel and he in o ma ion gene a ed simul aneously, so he esul is p oduced using he same image ame aken in he same ins an . 3 d All he imaging p oblems a e sol ed due o he use o h ee came as placed a di e en posi ions. T inocula s e eo ision inc eases he geome ic cons ain s and educes he in luence o heu is ic cons ain s o s e eo ma ching [6-7]. Mos o cases we’ll ha e edundan in o ma ion, bu hidden objec s will be o one came a, while he o he s will con inue p oducing he in o ma ion. When he in o ma ion is edundan , some p ecision imp o emen echniques can be applied and be e weld acking will be ob ained. 4 h Ano he imp o emen comes om he image dis o ion due o he lenses ha a e commonly cu e. Fo i s compensa ion, a dis o ion ac o is in oduced o modi ying he ypical linea model o he came a. The pa ame e s o he compensa ion a e measu ed using pho og ame ic me hods o calib a ion [8-9 ]. Ano he ad an ages a e in oduced when compa ed wi h o he welding sys ems: The e’s no need o p oduce s imuli o in e ac ion measu emen . Ou sys em can be conside ed a passi e sys em, due o he e’s no need emi e s, because he welding poin ligh ing is he signal aken o measu emen s. The e’ e ano he echniques o p o iding he in o ma ion ha ha e been ejec ed o ou pu pose: ul asound a e slow and a objec s a e bad measu ed wi h many well known p oblems. X ays need o use be ween he objec and he senso an ampli ie o ans o ming X ays in o isible spec um. Finally, in a ed ha e many p oblems ela ed wi h abso p ion, many dis o ions coming om he su ounding emi e s. In ou sys em he h ee images a e mixed, compensa ed, s o ed and p ocessed in he same de ice. IV. PROTOTYPE DESCRIPTION Ou p o o ype has been designed and implemen ed in he ela ed FPGA. Ou ex e nal ci cui only needs he ideo acqui ing sys ems: TVCCD-30M Came as and he digi ize s. Inside he FPGA, he compensa ions sys em and coo dina e compu ing blocks a e p og ammed. Figu e 2 p esen s a block diag am o he PCB boa d. I ’s compound by he h ee digi ize s, he Phillips SAA7113H, and ano he communica ion ea u es, like a RS232 line, a VGA ou pu and a pa allel po . came a 1 came a 2 came a 3 Video Decode 3 F P G A In e a ce Pa allel Po Se ial Po CDA connec o VGA 3D ision boa d Video Moni o Compu e UNSHADES Sys em Boa d Video Decode 2 Video Decode 1 Fig. 2. Main PCB blocks. Inside he FPGA, a ull digi al ci cui p ocess e e y image. Figu e 3 shows he block diag am o he p ocesso . The i s s age is o compensa e he lense dis o ion coming om he came a lenses. Secondly a bina isa ion o he image sea ches he welding poin , ead like a small a ea ob ained ha is a small b igh ci cle. A e , his pa is s o ed in a memo y o pos p ocessing. P e iously a ci cui con igu es he digi ize s o an adequa e image o ma . The mos impo an block is he image p ocesso . The i s s ep o he image p ocessing consis o inding he app opia e h eshold ha would lead o a bina ised ound shape co esponding o he welding poin p esen in he image. E e y pixel is compa ed wi h he p e iously compu ed h eshold, ha is ob ained om he image condi ion i sel . We ha e selec ed he h eshold alue be ween he mean o he luminance o he image and i s maximum alue. I he luminance is below his alue he pixel has 0 alue (black) and 1 alue (whi e) in ano he case. The line and pixel numbe o whi e alues a e s o ed in he memo y. Due o he ixed posi ion o e e y came a e e y decode can p oduce wo alues (x,y), (y,z) o (z,y). The ideo decode s p o ide in o ma ion wi h 625 lines and 1440 pixels pe line. This in o ma ion is use ul o gene a ing e e y coo dina e using simple coun e s. Video acqui e sys em Dis o ion compens i Dis o ion compens Dis o ion compens Bina isa ion Bina isa ion Bina isa ion Memo y Memo yMemo y P ocesso Con igu e and synch onise decode s Main con ol In e ace VGA In e ace (X,Y,Z) (X,Y,Z) Robo Con ol Sys em di i li ió Main Blocks in FPGA PCB PCB Fig. 3. Inside FPGA p ocessing blocks. The exac cen e o he welding poin is compu ed by means o a g a i y cen e o each coo dina e, using hese o mulas: We’ll ob ain wo numbe s o each coo dina e because hey’ e epea ed in wo memo ies. Some p ecision imp o emen s can be p oduced i he came as a e placed wi h a hal pixel o de ia ion o e e y coo dina e. ∑ ∑ = l li l li x x X 1 2 1 1∑ ∑ = m mi m mi y y Y 1 2 1 1∑ ∑ = n ni n ni z z Z 2 2 2 2 ∑ ∑ = q qi q qi y y Y 2 2 2 2∑ ∑ = i i z z Z 3 2 3 3 ∑ ∑ = p pi p pi x x X 3 2 3 3 2 31 XX X+ =2 21 YY Y+ =2 32 ZZ Z+ = The esul s a e ans e ed o he PC ha p o ides he in o ma ion o he con olle s. The implemen a ion o his sys em e esh he coo dina es e e y wo ames, due o he pos -p ocessing da apa h. The comple e sys em has been implemen ed in o a Vi ex 300, wi h a 60% o he occupa ion. Sys em clock uns a 50MHz. V. CONCLUSIONS The pape p esen s a simple sys em o de e mine he absolu e posi ion o a welding poin o an au onomous obo s. Main ad an ages a e he cos , he p ecision and he compu ing speed. In he inal e sion o his pape he compu ing ime will be gi en. I ’s been es ima ed like a coo dina e compu ing pe wo images, which can be conside ed in e ms o obo speed eal- ime. This sys em akes ad an age o he UNSHADES-1 sys em ha p o ides a comple e, lexible and e y com o able pla o m o he ision sys em. V. REFERENCES [1] D. Rdine Hundsdö e “Cos u as pe ec as. Aumen a el empleo de la soldadu a po láse en el mecanizado de chapa”. Re is a de Soldadu a y Tecnología de unión. Año XII- Num 72, pp. 9-11 No iemb e/Dic 2001. [2] P. Coi e , M. Chi ouze, “An In oduc ion o Robo Technology”, Mc G aw Hill, 1983. [3] Xilinx Da a Book. 2002 [4] M.A. Agui e, J. Tombs, A. To alba, L.G. F anquelo “HADES-1: A apid p o o yping en i onmen based on aad ances FPGA´s”. P oceedings o he Design o Ci cui s and In eg a ed Sys ems, DCSI’01 Opo o 2001. [5] J. M. Angulo Usa egui “Robó ica P ác ica. Tecnología y aplicaciones”. Capí ulo 7: “Visión de Máquinas”, Pa anin o, 5.ª ed., 2000. [6] Ayache, “T inocula S e eo Vision o Robo ics”,IEEE T ans. Pa e n Anal. & Machine In elligence, ol. 13, no. 1, Jan. 1991. [7] Yachida, “3D Da a Acquisi ion by Mul iple View”, Robo ics Resea ch: Thi d In . Symp., Fauge as, O.D. and Gi al , G., Eds. Camb idge, MA, MIT P ess, pp. 11- 18, 1986. [8] B own D., 1971: Close-Range Came a Calib a ion. Pho og amme ic Enginee ing, Vol. 37, No.8 [9] El-Hakim, S.F., 1986: Real-Time Image Me ology wi h CCD Came as. Pho og amme ic Enginee ing and Remo e Sensing, Vol. 52, No 11, pp. 1757-1766.