Toward an optimal and structured feature subset selection for multi-target regression using genetic algorithm

Recei ed 8 Oc obe 2023, accep ed 22 Oc obe 2023, da e o publica ion 26 Oc obe 2023, da e o cu en e sion 6 No embe 2023. Digi al Objec Iden i ie 10.1109/ACCESS.2023.3327870 Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o Mul i-Ta ge Reg ession Using Gene ic Algo i hm FARRUKH HASAN SYED1, MUHAMMAD ATIF TAHIR 1, JAROSLAV FRNDA 2,3, (Senio Membe , IEEE), MUHAMMAD RAFI 1, MUHAMMAD SHAHID ANWAR 4, AND JAN NEDOMA 3, (Senio Membe , IEEE) 1School o Compu ing, Depa men o Compu e Science, Na ional Uni e si y o Compu e and Eme ging Sciences, Islamabad 44000, Pakis an 2Depa men o Quan i a i e Me hods and Economic In o ma ics, Facul y o Ope a ion and Economics o T anspo and Communica ions, Uni e si y o Žilina, 01026 Žlina, Slo akia 3Depa men o Telecommunica ions, Facul y o Elec ical Enginee ing and Compu e Science, VSB—Technical Uni e si y o Os a a, 70800 Os a a, Czech Republic 4Depa men o AI and So wa e, Gachon Uni e si y, Seongnam-si 13120, Sou h Ko ea Co esponding au ho s: Muhammad Ra i ([email p o ec ed]) and Muhammad Shahid Anwa (shahidanwa [email protected] ) This wo k was suppo ed in pa by he Minis y o Educa ion, You h and Spo s o he Czech Republic conduc ed by he VSB—Technical Uni e si y o Os a a, Czechia, unde G an 2023/39 and G an 2023/42; and in pa by he Eu opean Union wi hin he REFRESH P ojec —Resea ch Excellence Fo Region Sus ainabili y and High-Tech Indus ies, o he Eu opean Jus T ansi ion Fund’’ unde G an ‘‘CZ.10.03.01/00/22003/0000048.’’ ABSTRACT Mul i Ta ge Reg ession (MTR) is a machine lea ning me hod ha simul aneously p edic s mul iple eal- alued ou pu s using a se o inpu a iables. A lo o eme ging applica ions ha can be mapped o his class o p oblem. In MTR me hod one o he c i ical aspec is o handle s uc u al in o ma ion like ins ance and a ge co ela ion. MTR algo i hms a emp o exploi hese in e dependences when building a model. This esul s in inc eased model complexi ies, which in u n, educe he in e p e abili y o he model h ough manual analysis o he esul . Howe e , da a d i en eal-wo ld applica ions o en equi e models ha can be used o analyze and imp o e eal-wo ld wo k lows. Le e aging dimensionali y educ ion echniques can educe model complexi y while e aining he pe o mance and boos in e p e abili y. This esea ch p oposes mul iple ea u e subse al e na i es o MTR using gene ic algo i hm, and p o ides a compa ison o he di e en ea u e subse selec ion al e na i es in conjunc ion wi h MTR algo i hms. We p oposed a gene ic algo i hm based ea u e subse selec ion wi h all a ge s and wi h indi idual a ge keeping he s uc u al in o ma ion in ac in he selec ion p ocess. Expe imen s a e pe o med on eal wo ld benchma ked MTR da a se s and he esul s indica e ha a signi ican imp o emen in pe o mance can be ob ained wi h compa a i ely simple MTR models by u ilizing op imal and s uc u ed ea u e selec ion. INDEX TERMS Mul i- a ge eg ession, ea u e selec ion, gene ic algo i hm, single a ge , mul iple objec i es. I. INTRODUCTION Mul i- a ge Reg ession (MTR) has been ecei ing inc easing a en ion in he esea ch communi y [1],[2],[3]. Wi h he exponen ial inc ease in au oma ion and da a collec ion, da a-d i en decision-making is becoming a no m in almos The associa e edi o coo dina ing he e iew o his manusc ip and app o ing i o publica ion was Wen ao Fan . all a eas o li e. Inc easing eal-wo ld si ua ions a e being iden i ied whe e he objec i e is o make in e ence abou se e al a ge a iables a he same ime. Reg ession asks in gene al and mul i- a ge eg ession asks in pa icula ha e a as numbe o applica ions anging om economics, business and inance o heal h, enginee ing and space esea ch, e c. [4],[5],[6]. Since MTR p oblems deal wi h mul iple a ge a iables, la ge sys ems wi h complex 121966 2023 The Au ho s. This wo k is licensed unde a C ea i e Commons A ibu ion-NonComme cial-NoDe i a i es 4.0 License. Fo mo e in o ma ion, see h ps://c ea i ecommons.o g/licenses/by-nc-nd/4.0/ VOLUME 11, 2023 F. H. Syed e al.: Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o MTR FIGURE 1. Classi ica ion o sola la e in ensi ies ([7]). in e ela ed componen s can be modeled using mul i- a ge eg ession algo i hms. Fo ins ance, Figu e 1(a) shows sola la es o di e en in ensi ies while Figu e 1(b) shows hei i adiance in ensi- ies. The in ensi ies o he emissions a e used o ca ego ize he sola la es. A sola e en can esul in emissions o mul iple in ensi ies, and is he e o e a classical mul i- a ge p oblem. The al eady challenging p oblem o mul i- a ge p edic ion and analysis is u he agg a a ed by (i) complex in e depen- dencies be ween he a ge a iables, and (ii) issues a ising om high dimensionali y o he ea u e space, popula ly e e ed o as he ‘‘Cu se o Dimensionali y’’. To he bes o ou knowledge, mos o he esea ch un il now has ocused on he a ge in e dependency modeling p oblem, bu e y li le wo k has been done on add essing he dimensionali y issues o imp o e model pe o mance. This esea ch aims o in es iga e ea u e selec ion as a iable pe o mance imp o emen app oach o mul i- a ge eg ession p oblems. This esea ch is ela ed o i s semi-supe ised coun e pa p oposed in [8]. The p oblem o cu se o dimensionali y is inhe en in all machine lea ning asks. I collec i ely e e s o he p oblems which a ise in modeling o high dimension da a [9],[10]. As a esul , models become complex [11]. This complexi y leads o low p edic i e accu acy and in e p e abili y, and an inc eased un ime. This p oblem is compounded when he numbe o a ailable examples o lea ning is ew, which is usually he case in p ac ical cases [12]. To sol e hese p oblems, a numbe o echniques exis , which gene ally do so by educing he dimensions o he da a. Fea u e selec ion echniques [13] help ule ou ea u es which a e i ele an o edundan . The o iginal ea u es a e no ans o med o modi ied in any way; hus, he unde s andabili y and he con ex is p ese ed. The issue wi h ea u e selec ion is o decide which ea u es o keep and which ones o d op om he se o a ailable ea u es. Gi en he la ge s a e space usually p esen ed by he ea u es, an exhaus i e sea ch is almos always ou o ques ion. One me hod ha has been used ex ensi ely o he pu pose o ea u e selec ion is he use o E olu iona y Algo i hms which a e me aheu is ic algo i hms inspi ed by he e olu iona y p ocesses in na u e [14],[15]. One commonly used e olu iona y algo i hm is he Gene ic Algo i hm (GA) [16]. GAs wi h hei ope a o s o c osso e and mu a ion, allow a as e explo a ion o di e en ea u e combina ions in an e ec i e manne . They a e mos commonly used algo i hms o op imiza ion p oblems, gi en hei lexible na u e, ease o adap a ion and a iabili y o pa ame e uning possibili ies [17],[18]. The p ocess o selec ing ea u es o mul i- a iable lea ning aces added complica ions since he e a e mul iple dependen a iables. Fo example, does he e exis a common ea u e subse which can be used o all a ge a iables? O , is using a di e en subse o ea u es o each a ge he way o go? I a common ea u e subse is used, dec easing e o o one a ge may inc ease he e o o some o he a ge and ice e sa. The e a e a huge numbe o eal-wo ld analy ics and op imiza ion p oblems ha ace his issue [19],[20]. In such a case, e en i a e age e o is used as an e alua ion me ic, he e o in some a ge s migh educe while o some a ge s, he e o may inc ease o an unaccep able le el. Keeping he abo e ques ions in mind, his pape in es i- ga es he e ec o ea u e selec ion on mul i- a ge eg ession asks using gene ic algo i hms. The objec i e o his esea ch is as ollows: OI: To show ha by educing ea u es using ea u e selec ion, a pa o be e esul s can be ob ained compa ed o exis ing MTR me hods. OII: To alle ia e he issues collec i ely e e ed o as he cu se o dimensionali y issues o high dimension MTR p oblems. OIII: To imp o e in e p e abili y and pe o mance o exis - ing MTR models. Fo any mul i- a ge ask, he e a e wo basic app oaches a ailable. 1) To app oach he p oblem as independen modeling p oblems o each o he a ge s. 2) To use mul i- a ge eg ession algo i hms ha can also model he in e ac ions be ween he a ge s in o de o explo e in e a ge dependencies. Based on he abo e men ioned app oaches, we p opose wo a ia ions as pa o his esea ch. Fo he i s app oach, we will be using he baseline app oach called Single Ta ge (ST) [21], while o he second app oach, we will be using wo me hods mul i- a ge eg ession algo i hms, S acked Single Ta ge (SST) and Ensemble o Reg esso Chains (ERC) [22]. Al hough a numbe o o he mul i- a ge eg ession algo i hms exis , we ha e chosen hese h ee algo i hms due o he ac ha he same base eg esso can be used o all h ee algo i hms. Thus only he e ec o ea u e selec ion can be isola ed and in es iga ed. VOLUME 11, 2023 121967 F. H. Syed e al.: Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o MTR Based on he abo e men ioned pa ame e s, he ollowing s eps shown in igu e 2 o iden i y and p opose ele an solu ion: FIGURE 2. S udy p ocess lowcha . The es o he pape is o ganized as ollows: Sec ion II p esen s a discussion o he exis ing easea ch in he a ea o mul i- a ge eg ession and ea u e selec ion. I also gi e a b ie o e iew o using GA as a ool o ea u e selec ion. Sec ion III desc ibes he de ails o he p oposed me hod, along wi h a desc ip ion o he algo i hms and echniques used in he expe imen s. I also lis s he da ase s used in he expe imen s. Sec ion IV p esen s comp ehensi e esul s o he expe imen s and a discussion on he ex en o which he objec i es o he esea ch ha e been achie ed. Sec ion V ends he pape wi h a discussion on he ad an ages and disad an ages o he p oposed app oach and some u u e di ec ions o he esea ch. II. RELATED WORK Mul i- a ge eg ession [23],[24] has gained ocus ecen ly wi h many p ac ical applica ions and p oblems being unco e ed which equi e p edic ion o alues o mul iple a ge a iables ha ing con inuous ou pu s. Some in e es ing examples include p edic ing cus ome p epaid op-ups [25], p edic ion o d ug e icacy [26], p edic ion o lou quali y [27], human b ain ac i i y p edic ion [28] whe e he ac i i y o adjacen a eas o he b ain need o be mapped simul aneously o a be e unde s anding o b ain ac i i y, ecosys em s udy such as in [29] whe e eadings o a ious a ibu es o ege a ion a a ce ain loca ion need o be s udied oge he , in epidemiological s udies, by s udying mul iple a iables o a disease o diseases, and hei in e dependence [30], and many mo e. Up o a ce ain ex en , he echniques de eloped o Mul ilabel Classi ica ion (MLC) can be adap ed o MTR p oblems as well [21], while some me hods, o example, Mul i-Objec i e Decision T ee [31], Mul i-Objec i e Ran- dom Fo es (MORF) [32], Ensembles o Mul i-Objec i e Decision T ees [33] ha e been de eloped speci ically o Mul i- a ge Reg ession (MTR). Al hough a lo o wo k has been done on ea u e selec ion o adi ional machine lea ning algo i hms, no much wo k has been done o imp o e MTR algo i hm pe o mance using ea u e selec ion and mos wo k has ocused on model de elopmen . Table 1p esen s a summa y demons a ing his concep . TABLE 1. MTR model de elopmen . As can be om able 1, mos g ound-b eaking MTR pape s discuss algo i hm de elopmen and adap ing MTC algo i hms o MTR. Fo all machine lea ning asks, he quali y o ea u es plays he mos impo an ole. E en he pe o mance o he mos powe ul algo i hms su e s i a sui able se o ea u es is no used. In his ega d, ea u e selec ion me hods play a e y impo an ole. Fea u e selec ion can help in no only imp o ing he esul s by educing he p oblems associa ed wi h cu se o dimension- ali y and o e i ing, bu also in dec easing he un ime o he model [34]. Fea u e selec ion app oaches a e gene ally di ided in o h ee ca ego ies: Fil e app oach, W appe app oach and Embedded app oach. E olu iona y algo i hms, and especially GA and i s a ia ions ha e ound hei way in o a numbe o ea u e selec ion applica ions and ha e been used since long. They a e used as w appe me hods, which means hey use he unde lying machine lea ning algo i hm o e alua e subse pe o mance. GA wi h i s c osso e and mu a ion ea u es o e s a good balance o exploi a ion and explo a ion in o de o sea ch h ough a la ge space o ea u e combina ions ha make up he sea ch space. III. METHODOLOGY A. DATASETS Fo expe imen s and e alua ion, expe imen s we e pe o med using benchma k mul i- a ge eg ession da ase s.1 1h p://mulan.sou ce o ge.ne /da ase s-m .h ml 121968 VOLUME 11, 2023 F. H. Syed e al.: Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o MTR TABLE 2. Mul i- a ge Reg ession Da ase s. ddeno es he numbe o dimensions o ea u es, while mdeno es he numbe o a ge s o each da a se . The da ase s a e a ailable o download publicly. A sum- ma y o he da ase s is gi en in Table 2. B. EVALUATION MEASURE Fo mul i- a ge eg ession, Rela i e Roo Mean Squa ed E o (RRMSE) [22] is used as he e alua ion measu e. RRMSE is de ined as: RRMSE = u u P(x,y)ϵD es (ˆyj−yj)2 P(x,y)ϵD es (¯ Yj−yj)2(1) In he abo e gi en equa ion (1),yjis he g ound u h alue, ˆyjis he p edic ed alue and ¯ Yjis he mean alue o he a ge a iable Yjob ained om he aining se . C. BENCHMARK MTR METHODS A b ie desc ip ion o he algo i hms is gi en below: •Single Ta ge : Single Ta ge me hod is he baseline me hod o mul i- a ge eg ession and igno es a ge in e dependencies. I is inspi ed by i s mul ilabel coun e pa called Bina y Rele ance [42]. The idea is o decompose he ‘m’ a ge p oblem in o ‘m’ independen single a ge p oblems and hen applying any sui able adi ional eg ession algo i hm. •S acked Single Ta ge : In SST, he aining phase has wo s ages. In he i s s age, ‘m’ independen models a e ained, one o each a ge (whe e ‘m’ is he numbe o a ge a iables). In he second s age, he ‘m’ models a e used o p edic he alues o he ‘m’ a ge s. Then, he inpu ea u es a e augmen ed using hese p edic ions. Now, a second se o ‘m’ models, i.e. one o each a ge , is lea ned using he augmen ed ea u e space. A es ime, he models ained in he i s phase a e i s used o ou pu ‘m’ p edic ions, and based on hese p edic ions, he ea u e space is augmen ed and inpu o he second phase om which he inal ou pu is ob ained. •Ensemble o Reg esso Chains: ERC is based on he Reg esso Chains [23] concep and consis s o an ensemble o Reg esso Chains (RC). A chain o a ge a iables in cons uc ed by o de ing he a iables andomly. A sepa a e eg ession model is de eloped o each a ge by using a chain o a ge s o augmen he ea u e space. Since using only one chain can cause bias due o chaining o de , in ERC an ensemble o such chains is used. Final p edic ion is ob ained by a e aging he p edic ions om each chain. Since a p edic ion ime, he ac ual alues o he a ge s a e no known, he p edic ed alues a e ob ained simila o SST. D. WRAPPER BASED FEATURE SELECTION APPROACH Fo any ea u e selec ion echnique, he gene al p ocess is ela i ely he same as shown in Figu e 3: FIGURE 3. The p ocess o ea u e selec ion. Since he objec i e o his esea ch was o in es iga e ea u e selec ion in conjunc ion wi h MTR algo i hms, he e o e, a w appe app oach was used. The eason o including MTR algo i hms was ha each algo i hm di e s in how he a ge s a e included in he inal model, so using he algo i hms hemsel es as he e alua ion unc ion will s ill allow any use ul a ge in e ac ion o be cap u ed. The e o e, he powe o MTR algo i hms will be e ained wi h he added bonus o a educed ea u e coun . E. FEATURE SELECTION ALGORITHM Gene ic algo i hm (GA) uses he ope a o s o selec ion, c osso e and mu a ion. Each o he ope a o s pe o m hei espec i e ope a ions on a se o candida e solu ions ( ea u e subse s) called ‘popula ion’. The selec ion ope a o selec s he candida e solu ions o c osso e based on hei pe o mance. The c osso e ope a o combines wo (o some imes mo e) candida e solu ions o p oduce new candida e solu ions. The in ui ion is ha he child solu ions om good pa en solu ions will combine he s eng hs o he pa en solu ions. Since con inuous c osso e s can esul in con e gence o he popula ion owa ds one op imum esul , possibly a local op imum, as simila solu ions a e epea edly combined, mu a ion ope a o is used o add di e si y o he popula ion by andomly changing some indi iduals p esen in he popula ion. Mu a ion allows he solu ions o escape om local op ima by c ea ing solu ions ha a e di e en om he cu en popula ion. GA allow as e explo a ion o he s a e space along wi h ele an exploi a ion o p omising solu ions. Ano he ad an age o GA is ha hese beha iou s can be modi ied by uning o he GA pa ame e s which a e gene ally simple, and hus, do no equi e ex ensi e addi ional knowledge. GA is also domain independen and can be applied o p oblems o da a se i espec i e o hei domains. Howe e , nume ous modi ica ions a e also possible in case VOLUME 11, 2023 121969 F. H. Syed e al.: Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o MTR domain knowledge o heu is ics need o be inco po a ed o ce ain si ua ions. F. METHODOLOGY The p oposed me hod uses he Single Ta ge (ST) me hod o ind sui able ea u e subse s o each a ge . Fo GA-FS- Ta ge , a ea u e selec ion o each a ge is ound indepen- den ly by conside ing he m- a ge mul i- a ge eg ession ask as msepa a e single a ge eg ession asks. Fo GA-FS- MTR, a common ea u e subse is ound by inding a ea u e subse which minimizes he a e age e o o all he a ge s. A s ep by s ep explana ion o he whole p ocess is gi en below: •Encoding and Popula ion Ini ializa ion: Fea u e sub- se s a e encoded in he o m o s ings o 0’s and 1’s. The 1’s deno e he p esence o a ea u e in a subse , while 0’s deno e i s absence. A numbe o candida e solu ions o ea u e subse s in he o m o s ings o 0’s and 1’s a e gene a ed andomly. •Fi ness Calcula ion: Each candida e solu ion is passed o he eg esso and he e o is ob ained. In he case o GA-FS-Ta ge , he candida e solu ion co esponds o one a ge a a ime, while in he case o GA-FS-MTR, he candida e solu ion co esponds o all he a ge s oge he . The e o e, o he o me case, Rela i e Roo Mean Squa ed E o (RRMSE) o one a ge ha is unde conside a ion is conside ed as he i ness o he candida e solu ion, while o he la e case, he A e age Rela i e Roo Mean Squa ed E o s (ARRMSE) o all a ge s is conside ed as he i ness o he candida e solu ion. •Selec ion: The bes candida e solu ions i.e. he ea u e subse s wi h he lowes e o a e selec ed o u he p ocessing. •C osso e : C osso e ope a o is used o gene a e new candida e solu ions by combining any wo candida e solu ions as depic ed in Figu e 4. FIGURE 4. Single poin c osso e . The bi s om Pa en 1 and Pa en 2 and combined using he c osso e ope a o o ob ain Child 1 and Child 2. The c osso e p obabili y is dependen on he i ness o solu ions. Candida e solu ions wi h highe i ness a e mo e likely o be selec ed as pa en s o c osso e . •Mu a ion: Mu a ion ope a o is used o main ain a le el o di e si y in he solu ions. In his case, his change consis ed o changing a ‘0’ in he candida e solu ion s ing o a ‘1’ o ice e sa as depic ed in Figu e 5. FIGURE 5. One bi mu a ion. A a iable a e o mu a ion was used o hese expe - imen s. Fo he i s ew gene a ions, he mu a ion a e was kep e y high, as mo e explo a ion was equi ed. The mu a ion a e was dec eased a e a ew gene a ions, and hen dec eased u he a e a ew mo e. To compensa e o he high a e o mu a ion and a oid mu a ion o good solu ions, he mu a ion p ocess was se up such ha he bes solu ions in he popula ion we e ne e mu a ed. •Gene a ion: This cycle con inues un il he s opping c i e ia is ob ained. Fo hese expe imen s, he e we e wo s opping c i e ia: ei he i a be e solu ion was no ound o wen y consecu i e gene a ions, o a p ede ined numbe o gene a ions was eached. Complexi y Analysis: •I M is he dimension o he da a se , hen o gene a e N indi iduals, he ime complexi y would be close o O(M*N). •I F is he ime aken o e alua e a i ness unc ion, hen he ime complexi y o popula ion would be O(F*N) •Time complexi y o c osso e O(L), mu a ion O(MU) •I G is he numbe o gene a ions, hen he o al ime complexi y o he algo i hm may be deno ed as: Complexi yGA =G∗O(M∗N∗F∗L∗MU) (2) Equa ion 2can be used o ob ain a ough es ima e o he unning ime o he p oposed app oach as well. Gene ic algo i hms o en p o ide he mos ad an age o op imiza ion p oblems whe e he popula ion size is ela i ely small compa ed o he size o he p oblem. The s a e space o ea u e selec ion can be deno ed as: S=2m, whe e mis he numbe o ea u es. In he case o ea u e selec ion, he numbe o ea u es can exceed hund eds o housands which esul s in a huge s a e space and he numbe o popula ion (N) is ex emely small as compa ed o (S). Since one o he objec i es o using GA o ea u e selec ion was o explo e la ge s a e space o ea u es quickly, he e o e, apa om mu a ion, we ha e also in oduced one mo e mechanism o inc ease he p opo ion o s a e space explo a ion. I he numbe o candida e solu ion goes below a h eshold due o he selec ion p ocess, new andom candida e solu ions a e gene a ed and added o he popula ion. This allows o he numbe o solu ions o emain a a ce ain numbe , while also in oducing di e se candida e solu ions in o he popula ion. Two a ia ions o p oposed app oach ha e been e alua ed o hese expe imen s. Figu es 6(a) and 6(b) show he 121970 VOLUME 11, 2023 F. H. Syed e al.: Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o MTR p oposed me hods g aphically and hei desc ip ion is gi en below: FIGURE 6. G aphical ep esen a ions o he p oposed me hods. 1) GENETIC ALGORITHM FEATURE SELECTION MULTI-TARGET (GA-FS-MTR) The p oposed me hod GA-FS-MTR wo ks by inding a common ea u e subse which educes he a e age e o o all a ge s. Fo hese expe imen s, he ST app oach was used o ind he common ea u e subse which was hen e alua ed using di e en MTR algo i hms. A popula ion o candida e solu ions was ini ialized andomly. One by one, each ea u e subse was passed o he ST algo i hm. Since ST e alua es he ea u es pe a ge , he ob ained esul s we e a e aged o ind he ARRMSE. The ea u e subse s wi h he lowes ARRMSE we e selec ed o u he p ocessing. A e a p econ igu ed numbe o gene a ions was eached, he subse wi h he lowes ARRMSE was conside ed as he bes one. Finally, he ob ained ea u e subse s we e used in conjunc ion wi h he MTR algo i hms; Single Ta ge (GA-FS-MTR(ST)), S acked Single Ta ge (GA-FS-MTR(SST)) and Ensemble o Reg esso Chains (GA-FS-MTR(ERC)), o ob ain he inal esul s. The ad an age o using his app oach is ha any a ge in e dependencies a e also inco po a ed in he inal p edic ion model. 2) GENETIC ALGORITHM FEATURE SELECTION USING SEPARATE FEATURES (GA-FS-TARGET) The p oposed me hod GA-FS-Ta ge wo ks by inding a ge speci ic ea u es o each a ge sepa a ely. One ad an age o using his echnique is ha no MTR algo i hms a e needed. The e o e, he complexi y o he model is dec eased. Ano he si ua ion whe e his me hod may also be use ul is when o any MTR p oblem, he e a e some a ge s ha a e mo e impo an han o he s. When using MTR algo i hms, each a ge is conside ed equally. This p oposed me hod can p o ide an imp o emen o e baseline me hods while allowing lexibili y o handling each a ge di e en ly. Simila o he i s a ia ion, a popula ion o candida e solu ions we e ini ialized andomly. The candida e solu ions we e e alua ed using ST, bu he RRMSE o a single a ge was conside ed each ime. E en ually, he ea u e subse which minimized he RRMSE o a ge ‘mi’ was conside ed he bes ea u e subse o a ge ‘mi’. The p ocess was hen epea ed o each a ge . In his way, o a da ase wi h ‘m’ a ge s, ‘m’ ea u e subse s we e ound. I would be pe inen o men ion he e ha in each case, he ST me hod was used o ind he ea u e subse . The eason is ha since he MTR algo i hms also inco po a e a ge in o ma ion when gene a ing he models, only he ea u e subse s e u ned by using MTR algo i hms would no ha e gi en an accu a e pic u e o he ea u e impo ance. G. EXPERIMENTAL SETTINGS All expe imen s we e ca ied ou using Mulan and Weka [43]. The da a was di ided in o wo equal hal . 15- old c oss alida ion was used o ea u e selec ion expe imen s using hal o he da a. The expe imen s we e pe o med using ‘ ep ee’ as base eg esso . Once he bes ea u e subse s we e iden i ied, he educed ea u es we e used o aining and es ing on he second hal o he da a o ob ain inal esul s which a e epo ed in he pape . This was also ca ied ou using 10- old c oss alida ion. Fo he gene ic algo i hm, a s anda d single poin c osso e was used. The mu a ion a e was se andomly. Ini ially, a highe mu a ion a e was se o encou age explo a ion. I was dec eased o e he cou se o he subsequen ly gene a ions. IV. RESULTS AND DISCUSSIONS This sec ion p esen s he esul s o he expe imen s and sepa a e discussions o each compa ed me hod. The esul s a e p esen ed o show a compa ison o he p oposed me hods wi h hei espec i e baseline me hods. In each case, he p oposed a ia ion can be seen o ha e imp o ed he pe o mance o he he baseline me hod. In he Pe Ta ge me hod, a Single Ta ge (ST) app oach is employed. The e o e, his p oposed me hod is compa ed wi h all he baseline me hods using Table 12. A. COMPARISON WITH (GA-FS-MTR) GA-FS-MTR-SO: The p oposed GA-FS-MTR-SO me hod p o ides an imp o emen in pe o mance compa ed o he benchma k me hods in all ins ances excep a ew. Tables 3, 4and 5show he esul s. The ST me hod achie es an a e age pe o mance imp o e- men o a ound 7.42% compa ed o i s baseline by using he p oposed app oach in he majo i y o he da a se s, as well as on a e age. The s anda d de ia ion is also less, showing ha esul s do no a y e y la gely. The SST me hod also achie es a pe o mance imp o e- men o a ound 14.5% compa ed o i s baseline me hod by using he p oposed app oach in he majo i y o he da a se s. The a e age pe o mance and he s anda d de ia ion also e lec he pe o mance imp o emen . VOLUME 11, 2023 121971 F. H. Syed e al.: Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o MTR TABLE 3. Table showing he compa ison be ween he ARRMSE ob ained om baseline me hod ST [21] wi h he ARRMSE ob ained using he p oposed Single objec i e me hod GA-FS-MTR(ST)-SO. TABLE 4. Table compa ing he ARRMSE ob ained om baseline me hod SST [22], wi h he ARRMSE ob ained using he p oposed Single objec i e me hod GA-FS-MTR(SST)-SO. TABLE 5. Table compa ing he ARRMSE ob ained om baseline me hod ERC [22], wi h he ARRMSE ob ained using he p oposed Single objec i e me hod GA-FS-MTR(ERC)-SO. The ERC me hod also achie es a pe o mance imp o e- men o a ound 11% compa ed o i s baseline me hod by using he p oposed app oach in he majo i y o he da a se s. In each case, in addi ion o he dec ease e o , he s anda d de ia ion is also educed, hus leading o a mo e con iden p edic ion using he p oposed app oach. To es i he di e ences in pe o mances we e signi ican , pai ed samples - es was ca ied ou be ween each MTR algo i hm and i s p oposed coun e pa . Table 6p esen s he esul ing p- alues o Single Objec i e me hods. The signi icance es shows ha he p oposed me hods ha e signi ican ly be e pe o mance a alpha alue o 0.05 in TABLE 6. Signi icance es esul s showing p- alues be ween Benchma k me hods ST, SST and ERC e sus p oposed me hods GA-FS-MTR(ST)-SO, GA-FS-MTR(SST)-SO and GA-FS-MTR(ERC)-SO. some cases, while a he alpha alue o 0.1, he p oposed me hods ha e signi ican ly be e pe o mance in almos all cases. Fu he , F iedman es was ca ied ou o be e unde s and he pe o mance di e ences be ween he me hods. Figu e 7 shows c i ical dis ance diag am o he compa ed echniques a alpha=0.1. FIGURE 7. C i ical dis ance diag am compa ing Single Objec i e based GA-FS-MTR me hods wi h he baseline me hods o alpha=0.1. Lowe anks a e be e . I can be seen om Figu e 7 ha he p oposed me hods gene ally ou pe o m he baseline me hods and he e o e ha e lowe anks in he diag am. (Objec i es I and I) GA-FS-MTR-MO: I he ea u e selec ion p oblem is o mula ed as a mul i-objec i e (MO) p oblem, a mul i- objec i e i ness unc ion is used in he GA. The GA(MO) e u ns a Pa e o on , om which he ea u e subse ha educes he e o he mos is selec ed. Tables 7,8and9p esen he esul s o hese expe imen s. TABLE 7. Table compa ing he ARRMSE ob ained om baseline me hod ST [21] e sus he ARRMSE ob ained using he p oposed Mul i-objec i e me hod GA-FS-MTR(ST)-MO. The p oposed GA-FS-MTR-Mul i Objec i e me hods also p o ides an o e all pe o mance imp o emen as shown in Tables 7,8and 9. In he a e age case an imp o emen o a ound 4% o e ST, 15.5% o e SST and 10.5% o e ERC is ob ained. In his case as well, he s anda d de ia ion ac oss all da a se s is also lowe o he p oposed me hod. To es i he di e ences in pe o mances we e signi ican , pai ed samples - es was ca ied ou be ween each MTR 121972 VOLUME 11, 2023 F. H. Syed e al.: Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o MTR TABLE 8. Table compa ing he ARRMSE ob ained om baseline me hod SST [22] e sus he ARRMSE ob ained using he p oposed Mul i-objec i e me hod GA-FS-MTR(SST)-MO. TABLE 9. Table compa ing he ARRMSE ob ained om baseline me hod ERC [22] e sus he ARRMSE ob ained using he p oposed Mul i-objec i e me hod GA-FS-MTR(ERC)-MO. algo i hm and i s p oposed coun e pa . Table 10 p esen s he esul ing p- alues o Mul i-Objec i e me hods. TABLE 10. Signi icance es esul s showing p- alues be ween Benchma k me hods ST, SST and ERC e sus p oposed me hods GA-FS-MTR(ST)-MO, GA-FS-MTR(SST)-MO and GA-FS-MTR(ERC)-MO. The p oposed me hods show signi ican di e ences a alpha alue o 0.1 indica ing ha he p oposed app oach imp o es he pe o mance conside ably. F iedman es was also ca ied ou be ween he p oposed and baseline me hods. Figu e 8shows c i ical dis ance diag am o he compa ed echniques a alpha=0.1 o Mul i-Objec i e based GA-FS-MTR me hods. (Objec i es I and I). FIGURE 8. C i ical dis ance diag am compa ing Mul i-Objec i e based GA-FS-MTR me hods wi h he baseline me hods o alpha=0.1. Lowe anks a e be e . I is seen om he igu es ha he p oposed me hods gene ally ou pe o m he baseline me hods and he e o e ha e lowe anks in he diag am. The GA-FS-MTR(ERC) me hod pe o ms bes among he p o- posed me hods o bo h he cases. In o de o analyze how much ea u e educ ion he p o- posed app oaches p o ide, he numbe o ea u es ob ained a e ea u e selec ion a e analyzed. Table 11 shows he pe cen age educ ion in he numbe o ea u es when he p oposed GA-FS-MTR (GA-FS-MTR-Single Objec i e o GA-FS-MTR-Mul i-Objec i e) me hods a e used. TABLE 11. Pe cen age educ ion in numbe o ea u es using GA-FS-MTR-Single Objec i e and GA-FS-MTR-Mul i-Objec i e. The esul s show ha he p oposed me hods p o ide a ema kable educ ion in he numbe o ea u es. Wi h a minimum o 38 pe cen educ ion, ea u e selec ion p o ides a subs an ial imp o emen in in e p e abili y by iden i ying he ea u es which a ec he a ge a iables he mos . Ano he poin o in e es in he esul s is ha e en o he lowe dimensional da ase s, he p oposed me hods s ill allow o a leas an a pa pe o mance wi h he educed ea u e se . This scale o educ ion allows o a g ea e possibili y o isualizing he esul s using g aphical echniques and hus gaining mo e insigh in o he da a. In o de o inspec he e ec o ea u e selec ion on each a ge , Figu es 9(a) and 9(b), show a compa ison o he Rela i e Roo Mean Squa ed E o o each a ge when ea u es a e educed using GA-FS-MTR-Single Objec i e. Figu es 10(a) and 10(b) show a compa ison o he Rela i e Roo Mean Squa ed E o o each a ge when ea u es a e educed using GA-FS-MTR-Mul i-Objec i e. As can be seen om he igu es, he pe o mance imp o emen is obse ed in almos all o he a ge s using he p oposed app oaches. Del ing u he in o he ea u e selec ion p ocess o ind how o en any ea u e is used when making p edic ions. In o de o do ha , Local in e p e able model-agnos ic explana ions LIME [44] was used o e alua e he ea u e con ibu ions o wo da a se s ‘ju a’ and ‘edm’ and he p edic ions we e made using a Decision T ee eg esso and a Ridge eg esso implemen ed using py hon’s sklea n lib a y. Figu es 11(a) and 11(b) show he pe cen age o he numbe o a imes a ea u e was used by he model o make a p edic ion. As can be seen om he igu es, al hough some ea u es a e used mo e equen ly in making he p edic ions, i is no possible o decide he ex en o which he models will be a ec ed i one o he ea u es is d opped. The e o e, any da a d i en ac i i y will equi e an assessmen o he ea u es indi idually. Fea u e selec ion iden i ies he mos impo an VOLUME 11, 2023 121973 F. H. Syed e al.: Towa d an Op imal and S uc u ed Fea u e Subse Selec ion o MTR FIGURE 9. Compa ison o Rela i e Roo Mean Squa ed E o o each a ge in he da a se o da a se s ‘and o’ and ‘ju a’ when ea u e educ ion is ob ained using GA-FS-MTR-Single Objec i e. FIGURE 10. Compa ison o Rela i e Roo Mean Squa ed E o o each a ge in he da a se o da a se s ‘and o’ and ‘ju a’ when ea u e educ ion is ob ained using GA-FS-MTR-Mul iObjec i e. ea u es and he e o e allows inc eased in e p e abili y and unde s anding in o he wo king o he model. B. COMPARISON WITH (GA-FS-TARGET) The esul s o he p oposed me hod GA-FS-Ta ge a e p esen ed in he Table 12. Al hough his p oposed echnique igno es he possible a ge in e dependencies, i s ill p o ides an a e age pe o mance imp o emen o oughly 8.6% in compa ison o all he MTR me hods. FIGURE 11. Pe cen age o imes a ea u e con ibu ed o a p edic ion o wo machine lea ning models. TABLE 12. Table compa ing he ARRMSE ob ained om baseline me hods ST [21], SST and ERC [22], e sus he ARRMSE ob ained using he p oposed Pe a ge me hod. Table 13 shows he p- alues o - es be ween he benchma ked me hods and he p oposed me hod GA-FS- Ta ge . The esul s how a signi icance in all cases a alpha alue 0.1 and a signi icance in all bu one case o alpha 0.05. TABLE 13. Signi icance es esul s showing p- alues be ween Benchma k me hods ST, SST and ERC e sus p oposed me hod GA-FS-Ta ge . Figu e 12 shows he c i ical dis ance diag am o GA-FS-Ta ge me hod a alpha=0.1. The p oposed me hod 121974 VOLUME 11, 2023