Multiobjective RBFNNs Designer for Function Approximation: An Application for Mineral Reduction

Mul iobjec i e RBFNNs Designe o Func ion App oxima ion: An Applica ion o Mine al Reduc ion Albe o Guill´en, Ignacio Rojas, Jes´us Gonz´alez, H´ec o Poma es, L.J. He e a and F ancisco Fe n´andez Uni e si y o G anada Abs ac . Radial Basis Func ion Neu al Ne wo ks (RBFNNs) a e well known because, among o he applica ions, hey p esen a good pe o - mance when app oxima ing unc ions. The unc ion app oxima ion p ob- lem a ises in he cons uc ion o a con ol sys em o op imize he p ocess o he mine al educ ion. In o de o egula e he empe a u e o he o ens and o he pa ame e s, i is necessa y a module o p edic he inal concen a ion o mine al ha will be ob ained om he sou ce ma e ials. This module can be o med by an RBFNN ha p edic s he ou pu and by he algo i hm ha designs he RBFNN dynamically as mo e da a is ob ained. The design o RBFNNs is a e y complex ask whe e many pa ame e s ha e o be de e mined, he e o e, a gene ic algo i hm ha de e mines all o hem has been de eloped. This algo i hm p o ides sa - is ac o y esul s since he ne wo ks i gene a es a e able o p edic qui e p ecisely he inal concen a ion o mine al. 1 In oduc ion Many dynamic op imiza ion p oblems can be ound du ing he p ocess o he nickel p oduc ion wi h he CARON echnology. These p oblems equi e each- ing a balance be ween he immedia e gaining and he op imum beha io o he sys ems h ough he ime. As an example, i can be conside ed he p ocess o he mine al educ ion. In his p ocess, an expendi u e o echnologic pe oleum is used o es ablish he he mic p o ile o he o ens ha de e mine he di - e en chemis eac ions o he co ec p ocess. This is a complex ask ha nowadays equi es a human ope a o o ake decisions based on his expe ience and in ui ion. The e o e, i would be e y help ul i a suppo decision sys em can be designed and implemen ed. Figu e 1 shows he inpu , ou pu and con- ol a iables ha will be used o cha ac e ize he model. All hese a iables a e egis e ed by a SCADA sys em ha gene a es he da a used o he expe imen s. The p oblem consis s in he op imiza ion o he necessi ies o he echnologi- cal pe oleum h ough he analysis o he da a dynamically ob ained. Since he e a e se e al necessi ies, we a e ackling a mul iobjec i e op imiza ion p oblem. These kind o p oblems do no ha e an unique solu ion since he se o solu ions canno be comple ed so ed because, o some cases, i is impossible o decide which solu ion is be e . The se o solu ions ha canno be imp o ed is know as he op imal Pa e o. The mine al educ ion p ocess can be cha ac e ized by he ollowing unc- ions: –Ex ac ions = 1(Inpu mine al, O en empe a u e, Reducing agen s) –O en empe a u es = 2( Inpu mine al, Chambe empe a u es) –Reducing agen s = 3( Inpu mine al, Addi i e pe oleum, Pe oleum in chambe s) The p oblem hen consis s in he lea ning o he h ee di e en unc ions ha ela e he inpu ec o s wi h he co esponding ou pu . This is possible since he da a will be measu ed di ec ly om he sou ce, and once hese unc ions a e lea ned, i will be possible o gene a e new alues no de ined in he aining se s ha will help o ake decisions in o de o op imize he p ocess. Figu e 2 shows an hyb id p ocess o he dynamic op imiza ion. In he p ocess, he e is a module ha has o app oxima e he beha io o he sys em in o de o p edic i and o gi e his in o ma ion o he Neu o-p og amming op imiza ion module ha will p o ide he in o ma ion o ake decisions. Mos o he dynamic neu o-p og amming me hods s a wi h an ini ial se- quence o con ol ac ions ha a e used o compu e an ini ial alue o he i ness unc ion ha will be used. This ini ial si ua ion can be imp o ed by modi ying he he ini ial con ol ac ions. The algo i hm p oposed in his pape will be used in he unc ion app oxima ion module shown in Figu e 2. This module has o include a p edic o elemen , he RBFNN, and an algo i hm ha designs he ne - wo k since he numbe o inpu s ec o s g ows dynamically each 8 hou s. This ime ame is big enough o be able o use a gene ic algo i hm ha will design a RBFNN ha will app oxima e he inpu da a. The algo i hm p esen ed in his pape is able o design his kind o ne wo ks p o iding excellen esul s as i will be shown in he expe imen sec ion. 2 RBFNN desc ip ion The p oblem o be ackled consis s in designing an RBFNN ha app oxima es a se o gi en alues. The use o his kind o neu al ne wo ks is a common solu ion since hey a e able o app oxima e any unc ion [10, 11]. Fo mally, a unc ion app oxima ion p oblem can be o mula ed as, gi en a se o obse a ions {(xk;yk); k= 1, ..., n}wi h yk=F(xk)∈IR and xk∈IRd, i is desi ed o ob ain a unc ion Fso n P k=1 ||yk− F(xk)||2is minimum. The pu pose o he design is o be able o ob ain ou pu s om inpu ec o s ha we e no speci ied in he o iginal aining da a se . An RBFNN Fwi h ixed s uc u e o app oxima e an unknown unc ion F wi h nen ies and one ou pu s a ing om a se o alues {(xk;yk); k= 1, ..., n} wi h yk=F(xk)∈IR and xk∈IRd, has a se o pa ame e s ha ha e o be op imized: Fig. 1. Mine al educ ion p ocess F(xk;C, R, Ω) = m X j=1 φ(xk;cj, j)·Ωj(1) whe e C={c1, ..., cm}is he se o RBF cen e s, R={ 1, ..., m}is he se o alues o each RBF adius, Ω={Ω1, ..., Ωm}is he se o weigh s and φ(xk;cj, j) ep esen s an RBF. The ac i a ion unc ion mos commonly used o classi ica ion and eg ession p oblems is he Gaussian unc ion because i is con inuous, di e en iable, i p o ides a so e ou pu and imp o es he in e po- la ion capabili ies [2,13]. The p ocedu e o design an RBFNN s a s by se ing he numbe o RBFs in he hidden laye , hen he RBF cen e s cjmus be placed and a adius jhas o be se o each o hem. Finally, he weigh s Ωjcan be calcula ed op imally by sol ing a linea equa ion sys em [4]. 3 Mul iobjec i e Algo i hm o Func ion App oxima ion: MOFA This sec ion desc ibes he algo i hm ha could be used in he p edic ion and unc ion app oxima ion module wi hin he sys em desc ibed in he p e ious Fig. 2. Hyb id p ocess o he dynamic op imiza ion sec ion. This algo i hm is based in he popula mul iobjec i e non-domina ed so ing gene ic algo i hm in i s second e sion (NSGA-II) [3]. The wo objec i es in he algo i hm a e o ob ain he ne wo k wi h he smalles e o and wi h he smalles numbe o RBFs. The bigge he ne wo ks become, he mo e expensi e is i s manipula ion wi hin he gene ic algo i hm, making i un e y slowly and he ne wo k mus be e ained each 8 hou s. This sec ion will in oduce he new elemen s ha ha e been inco po a ed o i he o iginal algo i hm o he design o RBFNNs. 3.1 Rep esen ing RBFNN in he Indi iduals As i was shown in he In oduc ion, o design an RBFNN i is needed o speci y: 1. he numbe o RBFs 2. he posi ion o he cen e s o he RBFs 3. he leng h o he adii 4. he weigh s o he ou pu laye The indi iduals in he popula ion o he algo i hm will con ain he i s h ee elemen s in a ec o o eal numbe s. Ins ead o including he weigh s, he ap- p oxima ion e o is s o ed in o de o sa e compu a ional e o by he ime he indi iduals will be compa ed. In he ollowing subsec ions he concep o local e o o an RBF will be e e ed. The local e o is de ined as he sum o he e o s be ween he eal ou pu and he ou pu gene a ed by he RBFNN bu , ins ead o conside ing all he inpu ec o s, only he ones ha ac i a e each RBF will be selec ed. To know i an inpu ec o ac i a es a neu on, i s ac i a ion unc ion is calcula ed o each inpu ec o and i i is highe han a de e mined h eshold, he inpu ec o ac i a es he neu on. 3.2 Ini ial Popula ion The ini ial popula ion is gene a ed using clus e ing algo i hms in o de o supply good indi iduals ha will make easie and as e o ind good solu ions. These clus e ing algo i hms a e: –Fuzzy C-means (FCM): This clus e ing algo i hm [1] pe o ms a uzzy pa - i ion o he inpu da a whe e he same inpu ec o can belong o se e al clus e s a he same ime wi h a membe ship deg ee. –Imp o ed Clus e ing o Func ion App oxima ion (ICFA): his algo i hm [5] uses supe ised clus e ing in o de o iden i y he a eas whe e he unc ion is mo e a iable. To do his, i de ines he concep o es ima ed ou pu o a cen e o assign a alue o he cen e in he ou pu axis. –Possibilis ic Cen e s Ini ialize (PCI) and Fuzzy-Possibilis ic Clus e ing o Func ion app oxima ion (FPCFA): hese algo i hms [6] modi y he way he inpu ec o s a e sha ed be ween he cen e s o he clus e s. In he ICFA algo i hm, a uzzy pa i ion was de ined. In hese wo algo i hms he uzzy pa i ion is eplaced by he ones used in [14] and in [9] espec i ely. The e a e also included indi iduals gene a ed andomly in o de o no o loose di e si y in he popula ion. A e his ini ializa ion o he popula ion, e y ew i e a ions o a local sea ch algo i hm (Le enbe g−Ma qua d [8]) a e un and he esul s a e conca ena ed o he popula ion. This ha e been p o ed o imp o e he quali y o he esul s because he popula ion becomes mo e di e se since he clus e ing algo i hms a e qui e obus and could gene a e indi iduals ha a e oo simila . The size o he RBFNNs belonging o he i s gene a ion should be small o wo easons: 1. make he ini ializa ion as as as possible 2. allow he gene ic algo i hm o de e mine he sizes o he RBFNNs om an inc emen al poin o iew, sa ing he compu a ional e o ha would suppose o deal wi h big ne wo ks om he i s gene a ions. The c oss ope a o s will ha e he chance o inc emen he numbe o RBFs and wi h he mu a ion ope a o s he e will be he possibili y o emo ing useless RBFs. 3.3 C osso e ope a o s The o iginal c osso e ope a o o e a bina y o eal coded ch omosome can- no be pe o med wi h he indi iduals o his algo i hm because each g oups o genes ha e di e en meanings. Two c osso e ope a o s we e designed o hese indi iduals, and expe imen ally i was concluded ha he applica ion o bo h ope a o s wi h he same p obabili y p o ided be e esul s han applying only one o hem. C osso e ope a o 1: Neu ons exchange This c osso e ope a o , con- cep ually, would be he mos simila one o he o iginal c osso e . Since he indi iduals ep esen an RBFNN wi h se e al neu ons, he c oss o wo indi id- uals will be he esul o exchanging one neu on. This is exchange is ep esen ed in Figu e 3. The ad an ages o his c osso e ope a o is ha i exploi s he ge- ne ic ma e ial o each indi idual wi hou modi ying he s uc u e o he ne wo k, he o he ad an age is i s simplici y and e iciency. C osso e ope a o 2: Addi ion o he neu on wi h he smalles e o This ope a o consis s in he addi ion o he neu on wi h he smalles local e o belonging o he o he indi idual and i is ep esen ed in Figu e 3. I he neu on wi h he smalles local e o is e y simila o ano he in he o he ne wo k, he neu on wi h he second smalles e o is chosen and so on. This ope a o will gi e he oppo uni y o inc ease he numbe o RBFs in one indi idual, allowing he algo i hm o explo e mo e opologies. A e inemen s ep is pe o med igh a e he c ossing, his e inemen consis s in he p une o he RBFs which does no in luence he ou pu o he RBFNN, o do his, all he weigh s ha connec he p ocessing uni s o he ou pu laye a e calcula ed and he neu ons ha do no ha e a signi ican weigh will be emo ed. C osso e 1 C osso e 2 Fig. 3. C osso e ope a o s 1 and 2 3.4 Mu a ion Ope a o s The mu a ion ope a o s p oposed o his algo i hm can be sepa a ed in wo ca ego ies: –mu a ions wi hou any knowledge –mu a ions using expe knowledge The mu a ion wi hou any knowledge e e s o hose changes ha a e pe - o med in a andom way, hose changes can a ec bo h he s uc u e and he pa ame e s o he RBFNNs. The objec i e o hese ope a o s is o add an- domness in he sea ch p ocess o a oid he con e gence o local minima. The mu a ion ope a o s wi h expe knowledge a e mu a ions ha a ec also he s uc u e and he pa ame e s o he RBFNNs bu using some in o ma ion in such a way ha he changes won’ be comple ely andom. As i occu s wi h he c osso e ope a o s, i we di ide hese subse o mu a ion ope a o s and pe o m di e en uns, he esul s ob ained a e wo se han i we un he algo i hm using bo h kind o mu a ion ope a o s. Mu a ions wi hou any knowledge. The e a e ou ope a o s ha a e com- ple ely andom: –The i s one is he dele ion o an RBF in one andom posi ion o e he inpu ec o s space se ing his adio also wi h a andom alue. All he andom alues a e in he in e al [0,1] since he inpu ec o s and hei ou pu a e no malized. –The second ope a o is he opposi e o he p e ious one, dele ing an exis - ing RBF. This mu a ion mus be cons ained and no be applied when he indi idual has less han wo neu ons. –The hi d one adds o all he coo dina es o a cen e a andom dis ance which alue is chosen in he in e al [-0.5,0.5]. –The o h one has exac ly he same beha io han he hi d one bu changing he alue o he adius o he selec ed RFB. The wo is ope a o s modi y he s uc u e o he ne wo k meanwhile he hi d and he o h modi y he pa ame e s o he ne wo k. The hi d and he ou h ope a o s e e o he eal coded gene ic algo i hms as p esen ed in [7]. Mu a ions wi h expe knowledge. These mu a ion ope a o s use he in- o ma ion p o ided by he ou pu o he unc ion o be app oxima ed. As he p e ious ope a o s, hese will ake ca e o he s uc u e o he ne wo k, adding and emo ing RBFs in a RBFNN, and will also modi y he alue o he pa a- me e s o he RBFs. The ope a o s a e: –The i s ope a o inse s one RBF in he posi ion o he inpu ec o wi h he highes e o . To selec his posi ion he ou pu o he RBFNN is calcula ed and hen i is compa ed wi h he ou pu o he a ge unc ion, he cen e will be placed in he posi ion o he poin whe e he di e ence be ween he eal ou pu and he gene a ed ou pu is g ea e . –The second ope a o emo es one RBF om he RBFNN, he RBF o be emo ed is he one wi h less local e o . This could seem no oo logical a i s , bu i allows o keep mo e di e si y in he popula ion since one o he c oss ope a o s adds he neu on o he indi idual wi h less local e o , so combining his o elemen s, he genes will emain in he popula ion bu a oiding edundan elemen s and allowing o sea ch o new a eas in he inpu ec o space. –The hi d ope a o consis s in he applica ion o a local sea ch algo i hm (Le enbe g-Ma dqua d ) o une he posi ions o he cen e s and hei adii bu being awa e ha wi h hese mo emen s he e o will be dec eased o su e. This ope a o mus be used ca e ully and only ew i e a ions should be done, o he wise he popula ion will con e ge oo as o a local minima. 4 Expe imen s The da a used in he expe imen s we e ob ained by measu ing he ollowing pa ame e s om he eal sys em: –Inpu s: Ni, Fe, Co, Si, Mg, Ton, Temp O ens (9), Ou pu : Addi i e pe oleum index The da a we e ob ained measu ing each 8 hou s he di e en elemen s, so he p edic ion o he nex alue mus be done in 8 hou s ime, he p oposed gene ic algo i hm is able o p o ide app op ia e esul s in ha ime ame. Fo he expe imen s, i was used he da a ob ained in 70 days so he size o he da a se has 210 ins ances o 26 a iables each. F om his 210 ins ances, 180 we e used o aining and he es o es . The algo i hm p oposed in his pape will be compa ed wi h o he e olu- iona y s a egy p esen ed in [12] whe e he au ho s p opose a new e olu iona y p ocedu e o design op imal RBFNNs. I de ines a sel -o ganizing p ocess in o a popula ion o RBFs based on he es ima ion o he i ness o each neu on in he popula ion, which depends on h ee ac o s: he weigh and wid h o each RBF, he closeness o o he RBFs, and he e o inside each RBF wid h. The algo- i hm also uses ope a o s ha , acco ding o a se o uzzy ules, ans o m he RBFs. All hese elemen s allowed he au ho s o de ine coope a ion, specia ion, and niching ea u es in he e olu ion o he popula ion. Table 1 shows espec i ely he app oxima ion e o s using he aining se and he es se . The wo e olu iona y algo i hms a e compa ed also wi h he CFA algo i hm ha was designed speci ically o pe o m he ini ializa ion s ep in he design o RBFNN o unc ion app oxima ion. Once he algo i hms we e execu ed a local sea ch algo i hm was applied o hei esul s. These ables show how he p oposed algo i hm o e comes he o he app oaches when app oxima - ing he aining da a and he es da a se . The o he wo algo i hms a e able o app oxima e he aining se wi h a e y small e o when many RBFs a e used, howe e , when hey y o app oxima e he es da a, he e o inc eases signi - ican ly. The ne wo ks gene a ed by he p oposed algo i hm ha e he abili y o app oxima ing he aining e o qui e p ecisely bu wi hou loosing gene ali y so he es e o is s ill small, no like when he o he algo i hms a e used. Table 1. Mean o he app oxima ion e o (NRMSE) o he aining and es da a T aining Tes RBFs CFA Ri e a MOFA 3 0.623 0.428 0.282 4 0.566 0.327 0.140 5 0.590 0.313 0.170 7 0.182 0.169 0.0140 8 0.061 0.054 3.333e-5 9 0.006 0.007 6.264e-6 10 0.003 0.002 4.513e-6 11 0.026 0.006 4.385e-6 12 0.017 0.002 2.946e-6 13 0.004 0.001 2.939e-6 14 8.264e-4 4.854e-5 2.896e-6 16 6.264e-4 3.644e-5 2.527e-6 RBFs CFA Ri e a MOFA 3 4.963 4.963 1.995 4 1.204 2.235 0.954 5 1.644 1.382 1.405 7 1.309 1.410 0.870 8 1.495 1.403 0.047 9 1.479 1.339 0.047 10 1.125 1.129 0.047 11 1.128 1.105 0.043 12 1.024 0.995 0.045 13 0.730 0.755 0.047 14 0.830 0.520 0.030 16 0.519 0.312 0.029 5 Conclusions This pape has p esen ed a sys em ha can be used o con ol and op imize he mine al ex ac ion om sou ce ma e ials. One o he modules ha builds he sys em is in cha ge o p edic ing he inal amoun o ex ac ed mine al om empi ical da a ob ained p e iously. The module consis s in an RBFNN, ha is able o p edic qui e p ecisely he eal ou pu o ma e ial, and in a gene ic algo i hm ha ains he ne wo k wi hin he ime ame equi ed by he sys em. A mul iobjec i e gene ic algo i hm ha designs he RBFNNs o he p edic ion module was p esen ed, ob aining a e y good pe o mance when i was compa ed agains o he echniques o he design o RBFNNs. Acknowledgemen s This wo k has been pa ially suppo ed by he Spanish CICYT P ojec TIN2004-01419 and he Eu opean Commission’s Resea ch In- as uc u es ac i i y o he S uc u ing Eu opean Resea ch A ea p og amme, con ac numbe RII3-CT-2003-506079 (HPC-Eu opa) Re e ences 1. J. C. Bezdek. Pa e n Recogni ion wi h Fuzzy Objec i e Func ion Algo i hms. Plenum, New Yo k, 1981.