DMN for Data Quality Measurement and Assessment

DMN for Data Quality Measurement and Assessment ´ Alvaro Valencia-Parra1(B), Luisa Parody2,´ Angel Jes´us Varela-Vaca1, Ismael Caballero3,andMar´ıa Teresa G´omez-L´opez1 1Universidad de Sevilla, Seville, Spain {avalencia,ajvarela,maytegomez}@us.es 2Universidad Loyola Andaluc´ıa, C´ordoba, Spain [email protected] 3Universidad Castilla-La Mancha, Ciudad Real, Spain [email protected] http://www.idea.us.es/ Abstract. Data Quality assessment is aimed at evaluating the suitability of a dataset for an intended task. The extensive literature on data quality describes the various methodologies for assessing data quality by means of data profiling techniques of the whole datasets. Our investigations are aimed to provide solutions to the need of automatically assessing the level of quality of the records of a dataset, where data profiling tools do not provide an adequate level of information. As most of the times, it is easier to describe when a record has quality enough than calculating a qualitative indicator, we propose a semi-automatically business rule-guided data quality assessment methodology for every record. This involves first listing the business rules that describe the data (data requirements), then those describing how to produce measures (business rules for data quality measurements), and finally, those defining how to assess the level of data quality of a data set (business rules for data quality assessment). The main contribution of this paper is the adoption of the OMG standard DMN (Decision Model and Notation) to support the data quality requirement description and their automatic assessment by using the existing DMN engines. Keywords: Data quality ·Decision Model and Notation ·Data quality measurement ·Data quality assessment 1 Introduction Globalization and emerging technologies are bringing new challenges to companies in the contexts in which enterprises should use massive amounts of data, most of them provided by third parties. In order to guarantee the success of the various tasks, and to satisfy the necessities of the business processes that use the data, it is paramount to face up with the quality of these data [12]. Just to mention an example of a critic task, let us think about the integration of data coming from different and heterogeneous sources in complex scenarios [3], and the many problems related to data quality that can arise when it comes to build a new dataset. To achieve the largest benefits of data, companies will need to find out ways to automatically manage the levels of data quality; this is even more important, in scenarios where it is required high efficiency in terms of computational cost of the operations per second. The assessment of data quality largely depends on the context of the use of data. Despite of the extensive literature around data quality (e.g., definition of methodologies for requirement definitions, selection of criteria to judge data qualitydata quality dimensions [1]), assessment and measurement procedures have been typically developed ad-hoc [13]. The data quality context-awareness needs the description of the business rules representing the data quality requirements. In order to describe the data quality requirements, we propose the description of three sets of business rules: those describing the data requirements (BR), those describing how to measure the data quality dimensions (BR.DQM), and those describing how to assess the data quality (BR.DQA) in terms of the measures of the data quality. The assessment follows a procedure based on the sequencing of the verification in cascade of the three previously-mentioned types of business rules in two phases: first, we conduct the verification of the BRs to estimate a measurement for every data quality dimension according to BR.DQM; and, then, we use the verification of the BR.DQM to produce an estimation of the assessment of the level of data quality, according to the stated BR.DQA. Based on the result of this assessment, the organization should make a decision on the use of the record based on its risk appetite. In this paper, we propose the use of DMN (Decision Model and Notation) [15], a declarative language proposed by OMG to facilitate the description of the business rules, as well as their evaluation [6]. DMN provides human readers with a more understandable and visual representation of business rules [8], being data quality requirements a new scenario where DMN can be used. The remainder of this paper is organized as follows: Sect. 2details the foundations involved in this paper; Sect. 3introduces a case study to make the proposal accessible; Sect. 4presents the proposal of application of DMN to data quality measurements and assessment respectively; Sect. 5presents the related work and Sect. 6concludes and remarks the lessons learned. 2 Foundations 2.1 Data Quality Management: Rules for Measurement and Assessment The cornerstone of our proposal of data quality management is grounded in the difference between two important concepts typically used as synonymous: “measurement” and “assessment”. This differentiation is based on the two definitions of quality: the “meeting requirements” by Crosby (or “how well data is built”) and the “fitness for use” by Juran (or “how usable the data is”)[19]. The basis Table 1. Data quality dimensions from Wang [19] Data quality category Data quality dimension Intrinsic Accuracy, Objectivity, Believability, Reputation Accessibility Access, Security Contextual Relevancy, Value-Added, Timeliness, Completeness, Amount of data Representational Interpretability, Ease of understanding, Concise representation, Consistent representation of our proposal is to describe by means of business rules when “data is well built” according to several data quality dimensions and when “data is usable” according to the assessment of the quality including all data quality dimensions. Data quality dimensions (criteria used to evaluate the quality of data) are at the core of data quality management [20] because they represent users’ data quality requirements. Several authors have proposed their own set of data quality dimensions, both generic ones (like the ones proposed by [19] -see Table 1-orthe introduced in ISO 25012 [10]) or for specific context. When it comes to estimate the amount of data quality that a dataset has, data profiling tools are typically employed to produce some measures that data quality processes use as indicators [9]. But, without knowing how the semantics of data has been implemented in the data model, the results cannot be interpreted, and cannot be used to diagnose the root causes of a low level of data quality. However, it is more than enough to have a qualitative indications of whether data is usable or not. This strategy is specially recommendable when it comes to determine if a record should be used as part of the execution of an instance of a business process. More specifically, when the decision should be made according to the quality of the used data, desirably in an automatic way. At this point, let us recall that every record is a set of attributes ai; every attribute aior every set of attributes ai,a j,a k,... must meet some data requirements specified by means of several Business Rules (BR). Some typical statements of business rules look like: –BR.01. The attribute a1must meet the regular expression RE –BR.02. The attribute a2(datatype numeric) should be lower than the attribute a3(datatype numeric) The measurement implies the verification of the business rules associated to every chosen data quality dimension (e.g. completeness, consistency, ...). To produce a value for measurement, we need a BR.DQM that describes the possible values that every data quality dimension could obtain. Depending on the granularity, a BR.DQM can be defined in terms of one or more attributes and one or more BRs. We propose to use Likert scales to define the possible values for the results of measurement. As an example of possible values of completeness is the set {“Complete Enough”, “Not Enough Complete”, “Dramatically Noncomplete”}. Typically, BR.DQM sentences for completeness can look like: –BR.DQM.01. A record can be considered as “Complete Enough” if it meets BR.01 and BR.02 –BR.DQM.02. A record can be considered as “Not Enough Complete” if it only meets BR.01 –BR.DQM.03. A record can be considered as “Dramatically Non-complete” if it does not meet neither of BR.01 nor BR.02 Finally, and after defining how to measure the data quality dimensions, it is time for aggregating the measures for the various data quality dimensions to produce an indication of the amount of data quality that a record has. The result of this aggregation will represent the level of usability of a record for a task in terms of the risk appetite of the organization for the underlying task. Once again, a Likert scale (e.g., “Usable”, “Potentially usable but risky”, “Nonusable”) is proposed and the statement of the corresponding business rules for the assessment (BR.DQA) should be done in terms of the chosen data quality dimensions and the corresponding BR.DQM. Typically, BR.DQA can look like: –BR.DQA.01. A record can be considered as “Usable” if it meets BR.DQM.01 –BR.DQA.02. A record can be considered as “Potentially usable but Risky” if it meets BR.DQM.02 –BR.DQA.03. A record can be considered as “Non-usable” if it meets BR.DQA.03 Please see Sect. 3for a larger motivating example of the description of an assessment. 2.2 Decision Model and Notation Decision Model and Notation (DMN) is a modelling language and notation defined to describe business rules [15]. DMN provides a simple way to define the decision logic model understandable by all users, in our case, from the business experts in charge of describing the processes to the data quality experts responsible for defining the quality requirements. The DMN standard includes two components: – Decision requirements diagram that defines the decisions to be made, their interrelationships, and their requirements for decision logic. – Decision logic that defines the required decisions in sufficient details to allow validation and/or automation. An example of a Decision Requirement Diagram is presented and detailed in Fig. 1.Thedecision task describes a specific task that includes a decision logic, that is, depending on some input values set output values as described in a decision table. The input data, as the name implies, is the necessary data that the decision logic needs to determine the output value. For instance, in the example, -- Decision task RelaƟons between Decisions tasks Input data Comment Fig. 1. Decision requirement diagram example the Verified Data according to Data Requirements feeds Measurement of Completeness task. On the other side, the arrows that connect two decisions tasks indicates the relationships between these two tasks. In the example presented in Fig. 1, the output of the decision task Measurement of Completeness feeds the Assessment of Data Quality task, which means that the decision output of the first task is considered as input for the second decision. Each decision task includes a decision table. The decision table used in this article has an horizontal orientation: the input and outputs are defined in columns and the rules as rows. An example is presented in Fig. 2, which also indicates the different components of the table. Output BR.06 BR.07 BR.08 Completeness boolean boolean boolean {complete, non-complete} 1true true true complete BR.DQMN.01 2- - - non-complete - AnnotaƟons InputF Measurement of Completeness Decision_Completeness DMN Table DescripƟon InformaƟon item name Hit policy indicator Rules in rows Input expression Rules numbers Irrelevant Input entry Allowed values Output component name Output entry OpƟonal annotaƟons AnnotaƟon entry OpƟonal default Output entry Fig. 2. Decision table example for the Measurement of Completeness task (see Fig. 1) The information item name is the name of the variable (i.e. information item) for which the decision table provides the decision logic. The hit policy indicator indicates how to handle the multiple matches. In our case, the Fmeans that although multiple rules can match, only the first hit by rule order is returned. There are other possible hit policy indicators such as Unique (U), Any (A), and Priority (P) (see [15] for further details). There is also a set of input clauses composed of an input expression and optional allowed values for the input entries, for instance BR.06 is an input expression whose possible value is a boolean (true and false values). An input entry is contained in a rule: the value true corresponds to the input entry for the BR.06 in the rule number 1. The input cell entry ‘-’ means irrelevant, i.e. it can have any of the allowed values. Moreover, a set of output clauses are also included in a decision table. An output clause consists of an output component name and its allowed values. The allowed value that is underlined corresponds to the default value. Finally, a set of annotations clauses can also be included in the decision table. In our case, rule number 1 is annotated with the entry DQ.DQMN.01. The decision table displays the rules in an abbreviated notation organizing the entries in table cells. For example, the rule number 1 can be read as: If (BR.06 and BR.07 and BR.08) then Completeness =‘complete’. 3 Motivating Example In order to illustrate how a semi-automatically business rule guided-data quality assessment can be done, we adapted a well-known example of a movie-database introduced in [2] and shown in Table 2. The adaptation just consist of changing some values to have a much better casuistry in the data quality assessment. Table 2. Example of dataset with data quality problems. Id Title Director Year #Remakes LastRemakeYear 1 Casablanca Weir 1942 3 1940 2Dead Poets Society Weir 1989 0 NULL 3Rman Holiday Wylder 1953 0 NULL 4 Sabrin NULL 1964 0 1985 To illustrate the data quality assessment, we introduce several business rules (BR,BR.DQM,andBR.DQA): –Data Requirements: Associated to the given dataset, some business rules (BR) describing some syntactic and/or semantic data requirements are listed: •BR.01. The attribute Title contains a string no longer than 256 characters •BR.02. The attribute Title must exists in the IMDB database •BR.03. The attribute Director contains an string no longer than 30 characters •BR.04. The attribute Director must appear in the IMDB database associated to the movie having the title specified in the attribute Title •BR.05. The attribute Year must be a positive number between 1895 and 2030 •BR.06. The attribute LastRemakeYear must be always greater than Year •BR.07, BR.08 and BR.09. The attributes Title,Director,andYear can not be null •BR.10. If the attribute #Remakes is zero, then the attribute LastRemakeYear must be null –Data Quality Measurement Requirements: The data quality dimensions along with possible values for expressing the results of measurements are the following: •Completeness: the possible values are {“Complete”, “Non-Complete”} *BR.DQM.01. A record is Complete when meet the business rules BR.06,BR.07 and BR.08 *BR.DQM.02. A record is Non-complete when does not meet BR.DQM.01 •Accuracy, having the values {“Very Accurate”, “Accurate”, “Inaccurate”}: *BR.DQM.03. A record is Very accurate when meets BR.02 and BR.04 *BR.DQM.04. A record is Accurate when meets BR.02 *BR.DQM.05. A record is Inaccurate when does not meet neither BR.DQM.03 nor BR.DQM.04 •Consistency, having values from the set {“High Consistency”, “Consistency”, “Low consistency”}: *BR.DQM.06. A record is High Consistency when meets BR.04 and BR.09 *BR.DQM.07. A record is Consistency when meets BR.04 *BR.DQM.08. A record is Low consistency when does not meet neither BR.DQM.06 nor BR.DQM.07 –Data Quality Assessment Requirements: After measurement, it is the time to aggregate the results previously obtained, in order to generate a judgment about the usability of a record. For this example, we consider the following values: {“suitable quality”, “enough-adequate quality”, “non-usable”} •A record is said to be of suitable quality when meet BR.DQM.01, BR.DQM.02,BR.DQM.04,a ndBR.DQM.05. •BR.DQA.02. A record is said to have adequate quality for use when meet BR.DQM.01, BR.DQM.04,andBR.DQM.05. •BR.DQA.03. A record is said to be non-usable when does not meet any of BR.DQM.01, BR.DQM.02, BR.DQM.03, BR.DQM.04 or BR.DQM.05. 4 Decision Model in DMN for Data Quality The decision model presented in Fig. 3establishes two hierarchical levels to reduce the DMN complexity [7]. The bottom level corresponds to the data quality dimensions to be measured. Following the motivating example, these dimensions are Completeness,Accuracy,andConsistency. The upper level is related to the assessment of the level of data quality. 4.1 Data Quality Measurement Data quality measurement is at the bottom level in the Decision Model. As aforementioned, it is related to each dimension to be measured in accordance with the Data Quality Measure Requirements presented in Sect. 3. Fig. 3. Decision model diagram for data quality assessment The first dimension modeled is Completeness (see Fig. 2). The input for building the DMN table for Completeness are: BR.06,BR.07,andBR.08.Eachof these columns might take a Boolean value (i.e., true or false) after the verification of the corresponding rule. The different entries (i.e., rows) return an output assessment value for this dimension (cf., Output column). In this example, the completeness dimension has two entries (two rows). The first entry sets true to each one of the input value, associating the value complete to the output column. It indicates that, in order to consider that the data is complete, then it must fulfill the requirements BR.06,BR.07,andBR.08. The second entry basically indicates that if the data does not meet none of the aforementioned requirements, then it is considered non-complete. The same logic applies for building the DMN tables of the other two dimensions. Figure 4depicts the DMN table for Accuracy. It is composed of two input columns, that correspond with BR.02 and BR.04. Three entries establish the data quality measurement requirements for Accuracy. The first entry indicates that both requirements must be fulfilled in order to consider the data as very accurate. The second entry specifies that only the requirement BR.02 must be met in order to consider the data as accurate. The last entry indicates that the data is inaccurate if it does not fulfill none of both requirements. Fig. 4. DMN for measurement accuracy. The last dimension is Consistency, shown in Fig. 5. In this case, BR.04 and BR.09 are the requirements used to evaluate this dimension. If both requirements are fulfilled, then the data is considered highly consistent. If only the requirement BR.04 is met, then the data is considered consistent. In any other case, the consistency of the data is low. Fig. 5. DMN for measurement consistency. 4.2 Data Quality Assessment Data quality assessment is at the top level in our decision model (cf, Fig. 6). In this level, there is only one DMN table, which takes as input the values returned by the previous DMN tables corresponding to data quality measurement of all dimensions. Fig. 6. DMN for assessment of data quality The DMN table consists of three input columns, one per each data quality dimension measured. The value which each column might take depends on the value returned by the corresponding dimension. In this case: Completeness could take these values complete or non-complete;Accuracy can be very accurate, accurate or inaccurate;andConsistency could take high consistency,consistency or low consistency. The output represents the overall level of data quality which can be one of these values: adequate,suitable,orenough. The table is composed of three rules. The first rule indicates that the overall level of data quality is adequate if Completeness is complete,Accuracy is not inaccurate,andConsistency is not low consistency. It must be highlighted that two of the entries of this rule support more than one valid value. For example, the entry for Accuracy indicates that must not be inaccurate, which means the value might be either very accurate or accurate. The same applies for Consistency. Thus, the second rule is similar to the first one, except for the Accuracy entry.