MLDA: A tool for analyzing multi-label datasets

Original software publication MLDA: A tool for analyzing multi-label datasets Jose M. Moyano a , Eva L. Gibaja a , Sebastián Ventura a , b , ∗ a Department of Computer Science and Numerical Analysis, University of Cordoba, Spain b Department of Information Systems, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah, Saudi Arabia a r t i c l e i n f o Article history: Received 3 October 2016 Revised 12 January 2017 Accepted 13 January 2017 Available online 14 January 2017 Keywords: Multi-label learning Software Data exploration Preprocessing Data analysis Dataset description a b s t r a c t The objective of this paper is to present MLDA, a tool for the exploration and analysis of multi-label datasets with both simple and multiple views. MLDA comprises a GUI and a Java API, providing the user with a wide set of charts, metrics, methods for transforming and preprocessing data, as well as comparison of several datasets. The paper introduces the main features of the framework, and introduces its use toward some illustrative examples. 1. Introduction Multi-label learning (MLL) is a supervised learning paradigm in which a pattern may have associated some classes or labels simultaneously. This learning paradigm has become a challenging research area with an increasing number of papers and application domains, including multimedia classification, medical diagnosis or social network mining, among others [1] . In MLL, datasets usually include characteristics such as imbalance, high dimensionality or relationship among labels [2] . These characteristics could hamper the performance of the algorithms, so characterizing and analyzing the data is essential [3] . To the best of our knowledge, only mldr [4] has been proposed for data characterization in MLL. It is an R package focused on analyzing the imbalance and relationship among labels, which offers both an API and a GUI. Mulan [5] and Meka [6] are two APIs to develop MLL algorithms which also include some basic metrics, data transformations and preprocessing tasks. Finally, Chekina et al. [3] provided a wide study on the datasets’ characteristics to select the best MLL algorithm, giving rise to a large number of metrics on this sense. For that, a wide number of metrics have been proposed. With the aim of extending the offer of data characterization tools we have develop the MLDA tool, a GUI and an API for char- ∗Corresponding author. E-mail address: sv[email protected] (S. Ventura). acterization, exploration and analysis of multi-label and multi-view multi-label (MVML) datasets. MLDA includes all the functionality of previous works, also expanding the metrics to analyze the imbalance and dependences among labels as well as different preprocessing methods. Moreover, MLDA includes new tools to convert dataset’s format, to analyze MVML data and to compare several datasets. The rest of the manuscript is organized as follows: Section 2 presents the software framework, Section 3 shows some illustrative examples and finally Section 4 presents some conclusions. 2. Software framework The MLDA tool code and its documentation (javadoc and user guide) are available at the GitHub repository https://github.com/ i02momuj/MLDA . The software has been released under the GPLv3 license. 2.1. Software architecture Both MLDA GUI and API have been implemented in Java and have been built over Mulan and Weka [7] . The libraries JGraphX [8] and JFreeChart [9] have been used to represent the charts. The API class diagram is shown in Fig. 1 . The base package includes the base classes from which all the metrics inherit; metricsTaxonomy package enables to represent the taxonomy of metrics; utils package includes extra methods to calculate metrics; and dimensionalhttp://dx.doi.org/10.1016/j.knosys.2017.01.018 0. 2 Fig. 1. MLDA class diagram. Fig. 2. Main characteristics and preprocessing. Fig. 3. Measuring imbalance and relationship among labels. ity, labelsRelation, labelsDistribution, imbalance and attributes packages include the implementation of the corresponding metrics. 2.2. Software functionalities MLDA provides a wide set of 57 metrics, which can be saved in different file formats ( csv, txt, tex and arff) for further analysis. It includes specific metrics and charts to measure the imbalance of the labels, such as histograms and boxplot diagrams with the labels distribution, and Imbalance Ratio for labels (IR both inter-class and intra-class) and combinations of labels (also called labelsets). It also includes Chi and Phi coefficients, as well as co-occurrence and heatmap graphs to analyze the relationships among labels. Furthermore, MLDA enables to compare several datasets. Additionally, MLDA includes different instance and feature selection methods. It can also perform data partition in both holdout and k -folds, including random and stratified partitioning specific for MLL. MLDA is able to load, transform and save datasets in both Mulan and Meka file formats. It includes some transformation methods which generate one or more single-label datasets. For MVML datasets, MLDA also shows a set of MVML specific metrics and allows to save a set of user-selected views. Finally, the API includes the full set of metrics included in the GUI and a framework to implement new metrics. 3 Table 1 Software metadata. Nr. (executable) Software metadata description Please fill in this column S1 Current software version 1.2.2 S2 Permanent link to executables of this version https://github.com/i02momuj/MLDA/releases/tag/1.2.2 S3 Legal Software License GPLv3 S4 Computing platform/Operating System Linux, Microsoft Windows, OS X S5 Installation requirements & dependencies Java version 1.8 or higher S6 If available, link to user manual - if formally published include a reference to the publication in the reference list https://github.com/i02momuj/MLDA/blob/master/doc/MLDA _ Doc.pdf S7 Support email for questions [email protected] Table 2 Code metadata. Nr. Code metadata description Please fill in this column C1 Current code version v1.117 C2 Permanent link to code/repository used of this code version https://github.com/i02momuj/MLDA/releases/tag/1.2.2 C3 Legal Code License GPLv3 C4 Code versioning system used git C5 Software code languages, tools, and services used Java C6 Compilation requirements, operating environments & dependencies Java version 1.8 or higher C7 If available Link to developer documentation/manual https://github.com/i02momuj/MLDA/releases/download/1.2.2/javadocs.zip C8 Support email for questions [email protected] 3. Illustrative examples In this section, an example of use of MLDA with emotions dataset is shown (available at https://www.uco.es/grupos/kdis/ kdiswiki/index.php/Resources ). Its main characteristics can be obtained loading the dataset with MLDA ( Fig. 2 (a)). It provides researcher with a first characterization of the dataset. Usually train and test or k -folds partitions are needed to run the algorithms, which can be generated by the Preprocess tab ( 2 (b)). As many algorithms are influenced by the imbalance of labels, it might be analyzed in order to use a rebalancing technique or not. The Labels tab shows the IR inter-class chart ( Fig. 3 (a)). As some labels has an IR > 1.5, a rebalancing technique could be necessary. In order to use an algorithm either considering label relationships or not, the Dependences tab provides a co-occurrence graph, which shows the co-occurrence between each pair of labels ( Fig. 3 (b)). As all labels are related, using a technique that takes into account the relationship among labels might be more recommendable. Finally, the frequency of the labelsets ( Fig. 3 (c)) also could increase the complexity of certain algorithms [1] . As many labelsets are very infrequent, an algorithm that prune infrequent labelsets could be recommendable. 4. Conclusions In this paper MLDA have been proposed, including a GUI tool and a Java API, which is a useful tool in the MLL research field since it enables to characterize, analyze and preprocess MLL and MVML datasets in an easy way. The analysis of the datasets are highly important since it helps researchers to select the most appropriate algorithm depending on the characteristics of the dataset. Required Metadata Current executable software version Table 1 Current code version Table 2 Acknowledgements This research was supported by the Spanish Ministry of Education under FPU Grant FPU15/02948, by the Spanish Ministry of Economy and Competitiveness, Project TIN-2014-55252-P, and by FEDER funds. References [1] E. Gibaja, S. Ventura, Multi-label learning: a review of the state of the art and ongoing research, WIREs Data Mining Knowl Discov 2014. doi: 10.1002/widm. 1139 . [2] E. Gibaja , S. Ventura ,A tutorial on multilabel learning, ACM Comput. Surv. 47 (3) (2015) . [3] L. Chekina , L. Rokach , B. Shapira , Meta-learning for selecting a multi-label classification algorithm, 2011, pp. 220–227 . [4] F. Charte , D. Charte , Working with multilabel datasets in r: the mldr package, R. J. 7 (2) (2015) 149–162 . [5] G. Tsoumakas , E. Spyromitros-Xioufis , J. Vilcek , I. Vlahavas , Mulan: a java library for multi-label learning, J. Mach. Learn. Res. 12 (2011) 2411–2414 . [6] Meka: A multi-label extension to weka, ( http://meka.sourceforge.net/ ) Last access: 16-11-2016. [7] M. Hall , E. Frank , G. Holmes , B. Pfahringer , P. Reutemann , I.H. Witten , The weka data mining software: an update, SIGKDD Explor. Newsl. 11 (1) (2009) 10–18 . [8] Jgraphx, ( https://www.jgraph.com ), Last access: 16-11-2016. [9] Jfreechart, ( http://www.jfree.org/jfreechart/ ), Last access: 16-11-2016.