Quality evaluation for Model-Driven Web Engineering methodologies

Quality evaluation for Model-Driven Web Engineering methodologies F.J. Domínguez-Mayo a, , M.J. Escalona a , M. Mejías a , M. Ross b,c , G. Staples c a Department of Computer Languages and System, University of Seville, Seville, Spain b Faculty of Maritme and Technology, Southampton Solent University, Southampton, United Kingdom c BCS Software Quality Specialist Group, United Kingdom Keywords: Software engineering Standards Metrics/measurement Methodologies Quality analysis and evaluation Software quality/SQA abstract Context: There are lots of approaches or methodologies in the Model-Driven Web Engineering (MDWE) context to develop Web Applications without reaching a consensus on the use of standards and scarcity of both, practical experience and tool support. Objective: Model-Driven Web Engineering (MDWE) methodologies are constantly evolving. Moreover, Quality is a very important factor to identify within a methodology as it defines processes, techniques and artifacts to develop Web Applications. For this reason, when analyzing a methodology, it is not only necessary to evaluate quality, but also to find out how to improve it. The main goal of this paper is to develop a set of Quality Characteristics and Sub-Characteristics for MDWE approaches based on ISO/ IEC standards. Method: From the software products context, some widely standards proposed, such as ISO/IEC 9126 or ISO/IEC 25000, suggest a Quality Model for software products, although up to now, there are no standard methods to assess quality on MDWE methodologies. Such methodologies can be organized into Properties, thus, a methodology has artifacts, processes and techniques. Then, each item is evaluated through a set of appropriate Quality Characteristics, depending on its nature. This paper proposes to evaluate a methodology as a product itself. Results: This paper recommends a set of Quality Characteristics and Sub-Characteristics based on these standards in order to evaluate MDWE methodologies quality. Additionally, it defines an agile way to relate these Quality Sub-Characteristics to Properties with the sole purpose of not only analyzing, but also assessing and improving MDWE methodologies. Conclusions: The application of these Quality Characteristics and Sub-Characteristics could promote efficiency in methodologies since this kind of assessment enhances both the understanding of strengths and weaknesses of approaches. 1. Introduction Quality is a relevant aspect to consider in the Software Engineering context, although there are several different definitions, for example, conformance to user expectations, which is often described as the ‘‘fitness for purpose’’ of a piece of software. Another quality definition attending to software quality measures deals with the high quality of software design (quality of design) and the high level the software conforms with that design (quality of conformance). The definition of quality in ISO 9000:2005, as described in [21], includes the notion of ‘‘degree’’ meaning that quality is not an absolute, but a changeable aspect. The concept of ‘‘degree’’ is illustrated in Fig. 1, showing that needs, requirements and expectations are constantly changing. Therefore, quality is the difference between the model state implied or required, and the model state reached. Thus, satisfactory quality takes place when the state reached is within the range of acceptability defined by the required model; superior quality occurs when the state reached is above the required model and inferior quality appears when the state reached is below the required state. In the final evaluation, it is the user who sets the quality standards by deciding which products should be purchased and whom they should be purchased. We need to express our relative satisfaction with products and, as a consequence, use subjective terms. When a product satisfies the user’s needs, it can be said that the product is either a high quality or a satisfactory quality product and in the same way, when the user is dissatisfied with the product, it can be said that it is either, a poor or a low quality product. When the product exceeds the user’s needs, it is considered to be either a high quality or a superior quality product, whereas if it is below the user’s expectations, it is considered to be either a low quality or unsatisfactory quality product. The Model-Driven Engineering (MDE) is a software development paradigm dealing with the creation of models or abstractions closer to a particular domain than to concepts or specific syntax. A Web Engineering domain specific to MDE is called MDWE (ModelDriven Web Engineering). A Model-Driven Architecture (MDA) [37] is an architecture platform approach to develop software systems under the MDE paradigm which provides a set of guidelines to structure specifications that are expressed as models. In recent years, the growing interest in the Internet has generated a high number of MDWE approaches [45] that offer a frame of reference for Web environment. Nowadays, there are many approaches and diverse MDWE methodologies such as OOHDM (Oriented Hypermedia Design Method) [40], UWE (UML-based Web Engineering) [44], WebML (The Web Modeling Language) [48], OOH4RIA [34], RUX-Method [39] or NDT (Navigational Development Techniques) [36], which do not reach a consensus on the use of standards, on the one hand, and show scarcity of both practical experience and tool support, on the other. Thus, each methodology has different tools, such as metamodels (some of them are based on other methodologies) or transformations, which can implement different levels of abstractions, for instance, the Computer Independent Model (CIM), the Platform Independent Model (PIM) and/or the Platform Specific Model (PSM), among others. In this situation, it is necessary to characterize these methodologies in order to analyze and evaluate them. In addition, it is important to know both, the real needs that designers have to cover regarding users of these approaches and also the Quality Characteristics these needs should guarantee. Thus, designing a clear strategy is essential since it will allow designers to efficiently outline these methodologies. QuEF [13] is a framework which manages quality in MDWE approaches. It is based on quality management, but it focuses on the Quality Model lifecycle. This lifecycle is composed of a set of phases (Strategy phase, Design phase, Transition phase, Operation phase and Quality Continuous Improvement phase) that helps quality management work effectively. In QuEF, the Quality Model is the key element in quality management since it describes all the necessary elements that make the automatic generation of artifacts reduce the estimated time and cost. The Strategy and Design phase must start with defining all users’ needs in general. Once all these needs are customized then, in turn, they are defined with a Quality Model. During the Operation phase, the most appropriate one is selected and both users and designers can analyze, control and evaluate the quality of their approaches, as shown in the sample application included in this paper. Users of methodologies need to find out the most appropriate one for themselves and their work environment. Due to their experience, they would prefer some aspects rather than others In any case, users have the last word to decide and designers must offer only what users need. All metrics studied in this paper focus on determining the aspects that must be included in a methodology. Thus, in the Operation phase, users select their preferences by customizing a set of weight values associated to the Quality Characteristics analyzed in this paper as well as the Properties containing users’ needs and environmental description of approaches in terms of the importance given. This paper focuses on developing a set of Quality Characteristics and Sub-Characteristics for MDWE approaches based on ISO/IEC standards. It aims to propose the bases of a set of these Quality Characteristics as part of the Strategy and Design phase of QuEF. We also suggest how to analyze, control and evaluate the quality of MDWE approaches as part of the Operation phase. Besides, this paper deals with defining Quality Characteristics. The idea is to provide an environment that allows users and designers to figure out which of the Quality Characteristics have to be guaranteed when particular users apply these methodologies. The paper is organized into the following sections: after this introduction, Section 2presents a global analysis of the situation and all necessary elements to elaborate this work according to some quality standards and contexts. Section 3proposes quality concepts such as Quality Characteristic and Quality Sub-Characteristic as part of the Strategy and Design phase in QuEF. Section 4explains how all these concepts are related to a Matrix of Influences (MoIs) as well as how it provides a set of formulas to analyze and evaluate MDWE methodologies as part of the Operation phase in QuEF. In Section 5, a set of Quality Characteristics and Quality Sub-Characteristics are identified. Section 6provides a sample of the proposed analysis and NDT methodology assessment. Finally, Section 7 and 8 offer a set of conclusions and contributions and suggest possible future work. 2. Work context and related work Few years ago, several research groups began to analyze the characteristics of new emerging software systems known as hypermedia systems, which have eventually evolved into Web systems. It was the birth of a new line of Software Engineering currently known as Web Engineering [16]. It is a specific domain within MDE (Model-Driven Engineering) paradigm where an application can be used [15]. The application of MDE Engineering to Web sites is called Model-Driven Web Engineering (MDWE) and, as it can be observed in different studies [7,45,15], it is offering very good results. Nowadays, there are several proposals on MDWE in the literature that are very useful for designing such applications. Some of them almost entirely cover every level of abstraction and they even have tools that support transformations automation in development and evaluation processes. There is a variety of proposals in Web Engineering, as shown in [45]. This range of possibilities and the trend towards using MDE in proposals, open such a wide range of offers that, in many cases, it is difficult to select the most appropriate one. MDE was launched by the Object Management Group (OMG). The OMG has also developed the proposed MDA (Model-Driven Architecture) that provides standard platform architecture for proposals based on the ModelDriven paradigm. MDA was created with the idea of separating a system logical specification from the operational details that define how the system uses the technology platform capabilities to be implemented. In this regard, the goals of MDA are portability, interoperability and reusability through architectural separation. The need, requirement or expectation The perfomance level reached Superior quality Satisfactory quality Inferior quality Quality Model Time Fig. 1. Quality in ISO/IEC. Besides, the concept of platform independence frequently appears in MDA, which models the independence of the characteristics of any technological platform in CIM and PIM level of Abstraction. Among the benefits provided by MDE, it can automate the software development, documentation, code generation, testing and project management. This fact favors a growing productivity in software development and maintenance, as well as time and cost reduction. Finally, it also increases the quality of the products obtained. By applying this paradigm, the lifecycle of a software system is completely covered, ranging from requirements capture to maintenance, or code generation. There are several methodological approaches and many comparative studies in the MDWE area. In this sense, Schwinger et al. [45] must be taken into account. The authors specifically consider modeling concepts in their ubiquitous nature. They also include a study regarding the available support for Model-Driven Development in a comprehensive way, using a well-defined and finegrained catalogue with more than 30 evaluation criteria. As far as quality is concerned, Heitlager et al. [19] discuss that the amount of effort needed to maintain a software system is related to the technical quality of the source code of that system. Besides, the ISO/IEC 9126 does not provide a consensual set of measures for estimating maintainability on the basis of the source code of a system. Heitlager et al. [19] identify a number of requirements to be fulfilled by a maintainability model in order to be usable in practice. They also discuss their experiences by using such a system for IT management consultancy activities. Azuma [3], on the contrary, introduces an overview of current ISO/IEC 9126, SQuaRE and ISO/IEC 14598 series, followed by some problems to be solved. Besides, Azuma explains new parts, such as Quality in Use, Quality Requirements and Elementally Metrics. Herrera et al. [20], on their part, aim at defining a Quality Model to assess Web Portals quality. The model is based on ISO/IEC 25010 standard and some related work found in the literature. Finally, Calero et al. [7] present the WQM (Web Quality Model), which distinguishes three dimensions related to web features, lifecycle processes and Quality Characteristics, and classify the most relevant web metrics through the framework. The idea of developing a MDE framework for evaluating quality has been applied in several studies by Mohagheghi and Dehlen [35], where they state that the quality of models is affected by the quality of modeling languages, tools, modeling processes, the knowledge and experience of modelers and the quality assurance techniques applied. Moreover, there are different proposals concerning conceptual frameworks for the metrics and indicators domains, such as different metamodels or tools like GenMETRIC [17], a proposal for the integrated management of software measurement. GenMETRIC aims at providing companies with a generic and flexible environment for software measurement, by facilitating and setting the bases for a common and effective measurement process, not only restricted to one kind of software entity, a single quality or an evaluation mode. Another framework is C-INCAMI [6], a goal-oriented and organization Measurement and Evaluation Framework. C-INCAMI framework creates an ontological domain model defining all the concepts and relations needed to design and implement measurement and evaluation processes. It is an approach in which the requirements specification, measurement and evaluation design, and analysis of results are elaborated to satisfy specific information in a given context. This ontology also considers sources in terms of WebQEM terminology such as ISO/IEC standards, articles and books. Olsina and Rossi [39] explain the use of WebQEM, a quantitative evaluation strategy to assess web sites and the quality of applications. Defining and measuring quality indicators can help stakeholders understand and improve web products. An e-commerce case study illustrates the use of methodology in systematically assessing attributes that influence the quality of the product. Olsina et al. [38] describe more than a hundred characteristics and attributes for the academic sites domain in order to analyze the quality requirement tree and the way to specify them. These elements are used in a quantitative methodology for assessment, comparison and ranking processes. The proposed web site Quality Evaluation Methodology (QEM) can be useful to assess quality in different phases of a web product lifecycle. Punter et al. [43] suggest the W-process. It supports the selection and assessment of appropriate techniques in an evaluation, for example, by choosing the right type of metrics to measure software product quality. The W-process is an extension of ISO/IEC 14598, taking the activities of the standard into account, but expressing the relations among them more explicitly than the standard does. The process carries out a categorization of the ISO/IEC 14598 activities in three levels, namely: Goal, Question and Metric. In this line, the open literature typically describes GQM (Goal Question Metric) [5] in terms of a six-step process where the first three steps deal with using business goals to drive the identification of the right metrics and the last three steps deal with gathering measurement data and making effective use of measurement results to drive decision-making and improvements. Plösch et al. [41] present a method to systematically collect quality requirements for software products based on quality models and potential obstacles that may obstruct quality goals. In previous work [11,10], a framework to analyze, evaluate and improve quality of MDWE approaches is posed but this version of QuEF was actually extended in [13] with several phases: Strategy phase, Design phase, Transition phase, Operation phase and Quality Continual Improvement phase. Such phases have been defined together with processes and artifacts to cope with a complete Quality Model lifecycle. In QuEF extension the main difference among frameworks is that quality management is focused on the Quality Model lifecycle. The purpose of QuEF extension is to converge on a continuous automatic quality improvement by means of generating Checklists and documentation as well as automatic evaluations and plans in order to control and better quality and, in turn, automatically reduce effort and time. On the one hand, this paper points out some relevant details when defining a set of Quality Characteristics and Quality SubCharacteristics for the quality assessment of MDWE approaches as a part of the Strategy and Design phase of QuEF and, on the other, it also proposes a way to analyze them as a part of the Operation phase. As far as Quality standards are concerned, ISO published [1] an expanded version, containing ISO quality models and a consensus decision on inventories of proposed measures for these models. The current version of ISO/IEC 9126 series of standards consists of four documents: ISO/IEC 9126-1: Quality Models [25]. ISO/IEC TR 9126-2: External Metrics [26]. ISO/IEC TR 9126-3: Internal Metrics [27]. ISO/IEC TR 9126-4: Quality in Use Metrics [28]. As a result of Information Technology advances and environmental changes, a new version of ISO/IEC 9126 has recently appeared, known as ISO/IEC 25000:2005 [29]. It guides on the use of the new series of International Standards named Software product Quality Requirements and Evaluation (SQuaRE). The purpose of this guidance is to provide an overview on SQuaRE contents, common reference models and definitions, and relations among documents. This makes users properly understand these series of International Standards according to their purposes. The document explains the transition process from the old ISO/IEC 9126 and ISO/ IEC 14598 [31] series to SQuaRE, and it also gives information about using ISO/IEC 9126 and ISO/IEC 14598 series in their previous form. The aim is to guide the software product development with quality requirements specification and evaluation. The document set criteria for quality requirements specification in software products, their metrics and evaluation. SQuaRE consists of the following divisions [30]: ISO/IEC 2500n: Quality Management Division. ISO/IEC 2501n: Quality Model Division. ISO/IEC 2502n: Quality Measurement Division. ISO/IEC 2503n: Quality Requirements Division. ISO/IEC 2504n: Quality Evaluation Division. One of the objectives of ISO/IEC 25000 series (and what makes them different from the current ISO/IEC 9126 series) is the harmonization of its contents with the software measurement terminology of ISO/IEC 15939:2007, which determines a measurement process applicable to System and Software Engineering and management disciplines alike. The process is described through a model that defines the necessary measurement process activities to adequately specify what measurement information is required, how the measures and analysis results are applied and how to consider their validity. The measurement process is flexible, tailorable and adaptable to different users’ needs. ISO/IEC 15939:2007 identifies a process that enhances defining a suitable set of measures addressing specific Properties. Other rigorous and disciplined methodology is Six sigma [21] which uses data and statistical analysis to measure and improve a company’s operational performance by identifying and eliminating process ‘‘defects’’. In this line, Software Quality Assurance (SQA) is responsible for monitoring Software Engineering processes and methods used to ensure quality. The methods by which this is accomplished are many and varied, and may ensure conformance to one or more standards, such as ISO 9000 or a model such as CMMI. SQA encompasses the entire software development process, which includes processes such as requirements definition, software design, coding, source code control, code reviews, management change, management configuration, testing, management release and product integration. SQA is organized into goals, commitments, abilities, activities, measurements and verifications. In this context, ISO/IEC 9126 is a standard Quality Characteristic terminology for software product evaluation. ISO/IEC 15504 [32], also known as SPICE (Software Process Improvement and Capability determination), consists of a set of technical standard documents for the computer software development process and the related business management functions. It is the reference model for the maturity models (dealing with capability levels whichinturnhavetodowiththeprocessattributesandgenericpractices) against which the assessors can place the evidence collected during their assessment, so that, they can give an overall idea of the organization’s abilities for delivering products (software, systems, and IT services). In this line, ISO/IEC 12207:2008 [33] establishes a common framework for software lifecycle processes, with well-defined terminology, that can be referenced by the software industry. ISO/IEC12207:2008alsoprovidesaprocessthatcanbeusedfordefining, controlling and improving software lifecycle processes. Processes, activities and tasks of ISO/IEC 12207:2008 - either alone or in conjunction with ISO/IEC 15288 – may also be applied during the acquisition of a system containing software. ISO/IEC 14598 series of standards provide methods for measuring, assessing and evaluating software products quality. They describe neither methods for evaluating software production processes nor methods for cost prediction (software product quality measurements may, of course, be used for both purposes) [21]. ISO/IEC 19796-1:2005 is an interesting standard to consider when developing a framework for the quality evaluation of MDWE methodologies. This standard is focused on e-learning and aims to harmonize the different approaches used around the world for assessing e-learning quality initiatives. The e-learning market acceptance depends on the quality of the related products, services and tools. A harmonized conception of e-learning quality is a prerequisite for a properly functioning market on e-learning products and services and their constant quality improvement.[23] 3. A quality management framework based on the Quality Model Lifecycle Given the high number of methodologies available and recently proposed, it has become necessary to define objective management tools to enable organizations to improve their methodological environment and help web methodologies designers create new effective and efficient tools, processes and techniques. Since methodologies are constantly evolving, it is not only necessary to manage and improve quality but also to optimize the process in order to reduce costs and time. QuEF [11,10] is a quality management framework for Model-Driven Web methodologies. In the present work, a Model-Driven Web methodology is an approach for Web Applications development based on the MDE paradigm that provides a set of guidelines, techniques, processes and/or tools to structure specifications which are expressed as models. QuEF was initially developed for the analysis and evaluation of these kinds of methodologies and it has recently been extended to cover different phases of the quality management. In Domínguez-Mayo et al. [13], all the phases have been described together with the objectives and artifacts they used. These phases, as shown in Fig. 2,have been identified in a lifecycle focusing on the Quality Model. Quality Model Strategy phase: This is the initial phase and the main objective concludes that quality management becomes strategically active. In this phase all the strategic assests have to be identified. Quality Model Design phase: In this phase the Quality Model has to be designed in terms of all strategic views resulting from the previous phase. Significantly, a design within QuEF is understood to encompass all relevant elements that are to be designed within the Quality Model. This Quality Model will be used in the Operation phase for the quality management. Quality Model Transition phase: The Quality Model transition phase includes changing the Quality Model, without influencing the operation phase. This phase is very important and it needs to keep pace with changes in technology, processes or techniques. These changes do not have to influence quality management but do have to be considered in the Operation phase. Quality Model Operation phase: In this phase the Quality Model is used to manage the quality of approaches. All artifacts like Checklists, evaluations or plans, among others, are automatically generated in terms of the Quality Model and designed in the Design phase. The analysis and evaluation of quality in the Operation phase is carried out to control and evaluate the current state of quality. Finally, all results are used for the continuous improvement of quality in the Continual Improvement phase. Quality Continual Improvement phase: The Quality Model can change due to changes of trends or technologies. In addition, this phase aims to align and realign the Quality Model with the Properties to be covered and Quality Characteristics to be assured along with the improvement of every phase in the lifecycle. Basically, this way to manage quality consists of defining a Quality Model containing all past, present and future require- ments, expectations, goals or trends to reach. The idea is to centralize all efforts and costs concerning quality management in a unique model. Therefore, the effort is reduced not only by centralizing the model but also by generating all necessary artifacts like Checklists, Improvement plans or Charts, among others, by means of a Model-Driven Quality. QuEF can be applied to any domain that may define goals and requirements to reach. The benefits or ROI are costs and time reduction when improving the domain under study. Besides, knowledge can be shared by stakeholders of the same domain. Another advantage lies in our strategic map for managing quality efficiently. Despite the initial effort to implement QuEF, if the defined strategy is clear and well defined, effort is a good investment since it increases quality control, faster quality improvement and competitiveness. The lifecycle deals with the set of phases performed to carry out quality management. In this lifecycle, past, present and future views of the Quality Model elements in the domain under study must be considered in order to get quality management in an efficient way. This phase represents a strategic assest that focuses on defining a quality management strategy. Moreover, the Quality Model Strategy phase of QuEF is the key issue in quality management. The Quality Model is finally designed in the QuEF Quality Model Design phase, where all strategic actives from the previous phase are taken into consideration. In fact, this Quality Model will be the one used for Quality analysis, control and evaluation in the next Operation phase. The Quality Model Operation phase is used to carry out the Quality management. Thus, the Analysis and evaluation management processes are performed within this phase which means that we can know the current state of the domain under study. If the domain or context changes due to the appearance of new trends, then the Quality Model needs to be changed in the Quality Model Transition phase without affecting the Operation phase. Finally, the Quality Continual Improvement phase executes all processes to improve quality in all the lifecycle processes as well as in the Quality Model itself. The use of QuEF focuses on medium-long term quality management. Additionally, a short-term use of QuEF for Quality management is also possible, but considering some aspects. Particularly, in a short-term quality management, Checklists are used to assure quality (meeting the Quality Model, for example, in terms of requirements or objectives, among others) rather than to analyze the current state domain under study. On the contrary, if a medium-long term Quality management is considered, then, the Checklists determine the current state rather than ensuring compliance with the Quality Model. In this sense, despite performing a short or a medium-long term quality management, introducing QuEF always requires a great initial effort rewarded with the aforementioned benefits. In the Strategy phase of QuEF, some of the main necessary processes to be carried out are, among others, the Property Portfolio Management, the Quality Characteristic Portfolio Management and the Quality Model Portfolio Management. Different portfolios are obtained as a result of these processes: The Portfolio of Properties. The Portfolio of Quality Characteristics. The Portfolio of Quality Model. These portfolios are used as strategic assets to obtain the final Properties, the Quality Characteristics and the Quality Model in Fig. 2. Quality model lifecycle in QuEF framework. the Design phase. When the Portfolios are active and have past, present and future information, the Design phase uses such information to outline the models. Both Properties and Quality Characteristics have to be described and structured in a two-levelled hierarchy which represents a higher and lower level of granularity of an approach. Properties are used to describe and categorize the proposals in two levels (Feature and Sub-Feature) and they indicate the degree to which a methodology is implemented. For instance, the MDE Feature may have some Sub-Features such as, Language Definition, Transformations and Trace Generation. On the contrary, Quality Characteristics and Quality Sub-Characteristics represent the topic of this work. They need to be clarified, but they are quality aspects influenced by an environment description or Properties. In other words, a Quality Characteristic is a higher-level quality aspect. In a hierarchy of Quality Characteristics, higher-level attributes are called Quality Characteristics and lower-level attributes are called Quality Sub-Characteristics. For example, Quality Characteristics could be Usability, Functionality or Portability. Usability, for instance, is defined by different Quality Sub-Characteristics such as Learnability, Understandability or Operability. Fig. 3 shows the relation among the aspects highlighted in this paper. A Quality Characteristic has a set of Quality Sub-Characteristics and a Feature has a set of Sub-Features. 3.1. A Strategy phase based on a Systematic Literature Review In QuEF, the Technological Watch Management is a process of the Quality Model Strategy phase focused on the observation of trends and technologies on the rise. It also explores the state of the art of the domain. Due to the definition of Properties (where CheckLists are defined) and Quality Characteristics involves a large initial effort thistaskismadebycarryingoutaSystematicLiteratureReview(SLR). There are many reasons for undertaking a SLR such as the identification of concepts and gaps for MDWE methodologies quality management. The results of this process are used to capture information for Properties and Quality Characteristics Portfolios. The review method is based on the research protocol and it is in this section where sources, studies selection and selection execution are defined. Sources Selection: The objective of this section is to select the sources where searches for primary studies will be executed. First of all, we carried out preliminary searches aimed at identifying existing systematic reviews and assessing the volume of potentially relevant studies. In this sense, we found the following relevant initiatives: Schwinger et al., 2008, Meliá et al., 2008, Preciado et al., 2008, Escalona and Koch 2004 or Escalona and Aragón, 2008, among others. The proposed study by Schwinger et al. was a good basis for this framework, although it is not sufficient because the environment does not only aim at assessing approaches, but managing quality as a whole. In addition, in a good strategy, the Properties that are shown in the study are not sufficient since they must be considered in a quality continuous improvement regarding past, present and future trends. This favors a good quality design and leads to a quality management based on quality continuous improvement. After that, a Web-search was performed to find other relevant and new concepts related to our Properties and Quality Characteristics. Several sets of keywords were used by combining the concepts of our study. Some examples are: ‘‘Model-Driven’’, ‘‘Web’’, ‘‘development’’, ‘‘methodology’’, ‘‘methodologies’’, ‘‘approach’’, ‘‘approaches’’, ‘‘CIM’’, ‘‘PIM’’, ‘‘SPM’’, ‘‘Computer Independent Model’’, ‘‘Platform Independent Model’’, ‘‘Specific Platform Model’’, Code’’, ‘‘Level of abstraction’’, ‘‘Tool Support’’, ‘‘evaluation’’, ‘‘assessment’’, ‘‘Quality’’, ‘‘ISO 9126’’, ‘‘ISO 25000’’, ‘‘CMMI’’ or ‘‘ITIL’’, among others. The last step consisted in looking for references of papers included in the previous reviews in order to identify more concepts. Studies selection: Then the selection criteria to evaluate studies sources were: the availability to consult articles on the Internet or on the digital library of the University of Seville, which has e-books and also access to other resources like Google Scholar, Scopus, Mendeley, Science Direct, ISI Web of Knowledge, ACM Digital Library, CiteSeerX or the IEEE digital library; availability for search mechanism through keywords; and literature from companies, books, journals and conferences published by experts in the field. As there are plenty of work published, our search started excluding those issued prior to 2006 because we were interested in current work. Then we analyzed which of them covered our domain, and all studies dealing with the Model-Driven Web development environments domain were included. The process to consider a paper was as follows: First, the title was considered, then the Abstract, Keywords and Content and finally the Conclusions of the analysis. Selection execution: Then, the procedures for studies selection were applied to obtain articles in order to verify if the studies fit the inclusion and exclusion criteria. Finally, attending to the Quality Characteristic Quality Sub-Characteristic Feature Sub-Feature Property 1..* 1..* 1..* 1..* 1..* Properties Quality Characteristics Quality Model Fig. 3. Quality model metamodel. results of the SLR, a set of Properties describing the MDWE methodologies were identified and classified into Features and Sub-Features in relation to work and the current literature. In addition, after applying the SLR, a MDWE methodology was evaluated as a product itself thus, a set of Quality Characteristics based on the ISO Standard was defined. A methodology resembles a software product in that it defines a set of methods, tools or techniques that conforms the Internal Quality. Moreover, these methodologies are established in a work context or environment and users behave in a different manner regarding these Properties depending on the work context (External Quality). Finally, we evaluated the effects of applying the methodology in a work context (Quality in Use). Then, these Properties and Quality Characteristics have to be related to this environment to indicate the influence of each Sub-Feature on each Quality Sub-Characteristic. Thus, every SubFeature is related to every Quality Sub-Characteristics so as to point out its influence. That is to say that, if designers have to improve methodologies, they only need to develop all phases of QuEF starting with the Strategy and Design phase in order to define a first Quality Model. The designer can either share the Quality Model with other designers or use a Quality Model reached on consensus, instead. QuEF could also be applied in other areas and domains such as the implementation of a recognized standard in a company. The implementation can be executed by defining a Quality Model with the necessary tasks to be performed and implemented within the phases of QuEF to cover all the Quality Model lifecycle. In contrast to the immature domain of MDWE methodologies, a standard model is a mature domain since it is agreed, stable and does not suffer many changes over time. Nevertheless, QuEF-TS can generate all artifacts to perform the standard model in the company. 4. Analysis and evaluation of Quality Characteristics Quality Characteristics and Properties are defined in the Strategy and Design phases of QuEF and their relevance has to be determined in terms of user’s needs, requirements and expectations. Although many methodology designers may have different viewpoints, due to their experience, in relation to the importance of Properties and Quality Characteristics, methodology designers have to identify these needs. We can evaluate those Quality Characteristics and Properties as elements of MDWE methodology from either a general or a specific granularity respectively. Each Quality Characteristic has a set of Quality Sub-Characteristics, however, in Properties, a Feature has a set of Sub-Features containing Properties. This granularity depends on the level in which the evaluation value of MDWE methodology is obtained. For instance, if we assess Quality Characteristics in general, we will obtain values for Usability, Functionality, Portability, Reliability and Maintainability. For specific granularity, for instance, particularly for the Usability Quality Characteristics, we can get evaluation values for Learnability, Understandability, Simplicity, Interpretability, Operability and Attractiveness. For that reason, there are two levels for Quality Characteristics and Properties which have, in turn, been divided into two levels: firstly, Quality Characteristics and Sub-Characteristics for Quality Characteristics and, secondly Features and Sub-Features for Properties. In the same way, methodology designers could have different viewpoints about the influence of Properties on Quality Characteristics. When the domain is immature or not completely agreed, the Strategy and Design phases are essential as designers have to identify Properties demanded by users in a methodology and the Quality Characteristics these Properties have to follow to be offered to users. Once the Properties and Quality Characteristics have been described in a first version of the Quality Model, users and designers have to define the importance of each element and the relationship among them i.e. how Quality Characteristics are influenced by Properties. It is very difficult to define the importance of Properties and Quality Characteristics in an immature domain. A consensus decision should be reached and delimited in the Strategy and Design phases however, if the domain is too immature, it is very difficult to reach this consensus. The reason lies in the fact that this kind of domain is left to the Operation phase to be solved. Then, users and designers can define the importance of each element in the Operation phase, the Quality Analysis management and the Quality Evaluation management. Therefore, once users and designers have reached a consensus decision on the Strategy and Design phases, it is then fixed in the Quality Model design. In these lines, the Quality Model Management is a process carried out in the Design phase by defining the current Quality Model as well as the association links between Properties and Quality Characteristics. Another important artifact to define and that provides the association links between the current Properties and Quality Characteristics is the MoIs (Matrix of Influences). Table 1 shows the association links between Quality Sub-Characteristics and SubFeatures determined by a matrix named MoI. In this matrix, elements of the Quality Model are organized in rows and columns; Properties (Hierarchical in Features and SubFeatures) are listed in rows and Quality Characteristics (Hierarchical in Quality Characteristics and Sub-Characteristics) are represented in columns. In the MoI, the Sub-Features value is calculated by the Checklists containing Properties (Hierarchical in Features and Sub-Features) used to obtain the methodology value. A qualitative value can determine the degree of influence on each relation modeled by a cell. In this case, the degree of influence is an association link between Sub-Features and Sub-Characteristics, regardless of the degree of importance. Thus, the MoI represents the degree of influence of each Sub-Feature f i;j i on each Quality Sub-Characteristic q k;l k . This is a qualitative value that has to be transformed in a quantitative value v ½f i;ji ;q k;lk  . For example, two value scales of the influence degree can only be defined as: Influence (p) or Not Influence (). Another possibility could be setting four different grades: High Influence (p"), Moderate Influence (p), Low Influence (p;) or No Influence (). Some formulas have been defined in order to calculate the values determining the methodology state reached for Quality Characteristics and Sub-Characteristics. These Quality Characteristics and Sub-Characteristics values are calculated in terms of Features and Sub-Features values that are currently being studied in other Table 1 Matrix of Influences (MOIs) between properties (Hierarchical by Sub-Features and Features) and quality characteristics (Hierarchical by Quality Characteristics and SubCharacteristics). Q 1 Q 2 ... Q k q 11 ... q 1;l 1 q 2,1 ... q 2;l 2 ... q k,1 ... q k;l k F 1 f 1,1 ... f 1;j 1 F 2 f 2,1 ... v ½f i;ji ;q k;lk  f 2;j 2 ... ... F i f i,1 ... f i;j i work. The Quality Sub-Characteristic values s q k;lk are calculated according to formula (1), which considers each Sub-Feature value r f i;ji of a Feature F i . This formula calculates the Quality Sub-Characteristic value in terms of those Sub-Features that influence the Quality Sub-Characteristic. This is defined by an association link that is a qualitative value in the MoI, which is, then, transformed into a quantitative value v ½f i;ji ;q k;lk  . Besides, the importance of each Sub-Feature in the set of Sub-Features is considered by the SubFeature weight value w f i;ji . s q k;lk ¼P n j¼1 v ½f i;ji;qk;lk r f i;ji w f i;ji nð1Þ s q k;lk Z. It is a Quality Sub-Characteristic value. It is between ð06s q k;lk 61Þbecause this value actually represents whether the state reached by this value satisfies or not the required model (value = 1). r f i;ji Z. It is a Sub-Feature value between ð06r f i;ji 61Þsince this value is actually a degree representing whether the state reached by this value satisfies or not the required model (value = 1).  v ½f i;ji ;q k;lk  Z. It is the degree of influence value or the relation value between the Sub-Feature f i;j i and the Quality Sub-Characteristic q k;l k . In the MoI, it is a qualitative value that has to be changed into a quantitative value between ð06 v ½f i;ji ;q k;lk  61Þ. It depends on the value scale defined indicating the influence. w f i;ji Z. It is the weight value of the Sub-Feature f i;j i . It is between ð06w f i;ji 61Þand represents the importance of the Sub-Feature in the set of Sub-Features. The sum of this value for the set of Sub-Features of the same Feature has to be 1. nZ. It is the number of Sub-Feature values of a Feature F i whose degree of influence value v ½f i;ji ;q k;lk  is not 0. Nevertheless, it can be generalized and the Quality Sub-Characteristic value s q k;lk can be calculated in terms of a Feature F i or, on the contrary, considered as a combination of a set of Features. For instance, these two Features are considered F 1 and F 2 ,as indicated in formula (2). s q k;lk ¼P n 1 j¼1 v ½f 1;j1;qk;l1 r f 1;j1 w f 1;j1 þP n 2 j¼1 v ½f 2;j2;qk;lk r f 2;j2 w f 2;j2 n 1 þn 2 ð2Þ In formula (2), n 1 and n 2 are the numbers of Sub-Feature values of each Feature F 1 and F 2 whose weight value v ½f 1;j1 ;q k;lk  and v ½f 2;j2 ;q k;lk  respectively, are not 0. In other words, it can be generalized to this other formula (3) where a Quality Sub-Characteristic value s q k;lk is calculated in terms of different Features F 1 ,F 2 , ..., F t selected. s q k;lk ¼P n 1 j¼1 v ½f 1;j1;qk;l1 r f 1;j1 w f 1;j1 þ...þP n t j¼1 v ½f t;jt;qk;lk r f t;jt w f t;jt n 1 þ...þn t ð3Þ In a similar manner to formula 2, n 1 , ..., n t represents the number of Sub-Feature values of each Feature F 1 , ..., F t whose degree of influence values v ½f 1;j1 ;q k;lk  ;...;v ½f t;jt ;q k;lk  respectively, are not 0. Finally, every Quality Characteristic value s Q k is calculated by considering each Quality Sub-Characteristic q k;l k and the necessary degree to influence it with a weight value. In the following formula (4),w q k;lk represents the weight value that indicates the influence of each Quality Sub-Characteristic value q k;l k on the Quality Characteristic value s q k;lk associated. s Q k ¼P m l¼1 w q k;lk s q k;lk mð4Þ s Q k . It is a Quality Characteristic value. It is between ð06 s Q k 61Þbecause this value actually represents a degree showing whether the state reached by this value satisfies or not the required model (value = 1). w q k;lk . It is a Quality Sub-Characteristic weight value. It is between ð06w q k;lk 61Þand represents the importance of Quality Sub-Characteristic in the set of Quality Sub-Characteristics. The sum of this value for the set of Quality Sub-Characteristics of the same Quality Characteristic has to be 1. s q k;lk . It is a Quality Sub-Characteristic value. It is between ð06s q k;lk 61Þbecause this value is actually a degree representing whether the state reached by this value satisfies or not the required model (value = 1). m. It is the number of Quality Sub-Characteristic values of a Feature F i whose weight value w q k;lk is not 0. A formulation is given in order to define chart representations of a set of values. The main goal of this kind of formulation is to clarify the results showed in the charts since there are several alternatives to carry out an analysis from different points of view. For instance, we can get either a general view of Features with every Feature value or a set of Sub-Features value of a specific Feature. Besides, we can have a general view about Quality Characteristics with every Quality Characteristic value or a set of Quality Sub-Characteristics value of a specific Quality Characteristic, depending on different Features or just one. Thus, this last representation case has to indicate the Sub-Features or Sub-Feature considered calculating this Quality Sub-Characteristics. The notation is explained below: R F . It represents the set of Feature values fr F 1 ;r F 2 ;...;r F n gin a general view. All Features are considered with this terminology. R F i . It represents a set of Sub-Feature values fr f i;1i ;r f i;2i ;...;r f i;ji gof a Feature F i . In the same way, in order to represent Quality Sub-Characteristic values of a Quality Characteristic, the term Q k is equivalent to the term q k;l k . S QjF Z. It represents a set of Quality Characteristic values fs Q 1 ;s Q 2 ;...;s Q m gfor each Quality Characteristic in a general view, according to the Sub-Feature values fr f i;1i ;r f i;2i ;...;r f i;ji gof each Feature F i . All Features are considered. S Q k jF . It represents a set of Quality Sub-Characteristic values fs q k;1k ;s q k;2k ;...;s q k;lk gof a Quality Characteristic Q K in terms of Sub-Feature values fr f i;1i ;r f i;2i ;...;r f i;ji gof each Feature F i . S Q k jF 1 ;F 2 ;...;F n . It represents a set of Quality Sub-Characteristic values fs q k;1k ;s q k;2k ;...;s q k;lk gof a Quality Characteristic Q K according to a set of Sub-Feature values fr f i;1i ;r f i;2i ;...;r f i;ji gof the Features F i considered. S Q k jFF t . It represents a set of Quality Sub-Characteristic values fs q k;1k ;s q k;2k ;...;s q k;lk gof a Quality Characteristic Q K in terms of the Sub-Feature values fr f i;1i ;r f i;2i ;...;r f i;ji gof each Feature F i , without considering the Sub-Feature values of the Feature F t . This MoI could be used to optimize the costs of implementing several new Features and Sub-Features in a methodology. Designers could predict the costs of implementing a new Feature in their methodologies. For instance, they can decide to implement the minimal number of Features and Sub-Features which have influence on a major number of Quality Characteristics and Quality Sub-Characteristics. 5. Identifying Quality Characteristics and Quality SubCharacteristics for MDWE In QuEF, the Technological Watch Management is a process of the Strategy phase used for the observation of trends and technologies on the rise. It also assesses the state of the art of the domain. The results of this process are used to capture information for the Portfolios. In these lines, the Portfolio of Quality Characteristics is an artifact of the Quality Model Strategy phase that defines all important aspects that designers have to guarantee to users of these methodologies. Users, on their part, have to identify the importance of these elements in the future, bearing in mind the removal or insertion of more Quality Characteristics, depending on the current trends captured by the Technological Watch Management. As far as Quality Characteristics are concerned, the methodology can be evaluated as a product itself. For instance, we can evaluate either Usability or Functionality of a methodology. Depending on its nature, each item is evaluated through a set of appropriate Quality Characteristics. As there is no standard glossary for MDWE terminology, it is necessary to standardize the terminology to improve the access channel for communication in MDWE. This paper presents a set of Quality Characteristic and Quality Sub-Characteristic based on ISO/IEC 25000. Consequently, methodologies are evaluated as products. To this end, a set of Quality Characteristics, based on current literature such as ISO/IEC 9126, ISO/IEC 25000 (SQuaRE), IEEE [22] and other standards adapted to MDWE methodologies are being identified and hierarchically classified. ISO/IEC 2501n standard (Quality Model Division) that conforms this division presents a detailed quality model including characteristics for internal and external quality in use. Furthermore, the internal and external software quality characteristics are organized into sub-characteristics. Practical guidance on the use of the quality model is also provided. Quality Characteristics are classified according to ISO/IEC ISO/IEC 2501n into three different groups: External Quality, which measures the software (ISO/IEC 2501n). Internal Quality, which measures the system behavior (ISO/IEC 2501n). Quality in Use, which measures the effect of using the software in a specific context (ISO/IEC 2501n). Thus, the Quality Characteristics for MDWE methodologies could be classified into two categories: External/Internal Quality, which measures the approach features and their behavior. They are Usability, Functionality, Reliability, Maintainability and Portability. Quality in Use, which measures the effect of using the approach features in a specific context. They are Effectiveness, Productivity and Satisfaction. As Fig. 4 shows, both categories have influence on each other. Moreover, the Quality in Use category of a Web Application depends on the Web Application External and Internal Quality category. We have selected the above Quality Characteristics because we consider that a methodology could be evaluated as a product itself. A methodology resembles a software product in that it defines a set of methods, tools or techniques that conforms Internal Quality. On the contrary, these methodologies are implanted in a work context or environment and users behave in a different manner in relation to all these Properties depending on the work context (External Quality). Finally, we can evaluate the effects of applying the methodology in a working context (Quality in Use). Therefore, in spite of using ISO/IEC 9126 or ISO/IEC 25000, Quality Characteristics work as a basis adapted to the evaluation of methodologies. It must be noted that we do not talk about software product, but MDWE methodologies, instead. In this line, the use of a methodology ensures output consistency, although it does not guarantee Web application quality. This is due to the fact that using a methodology does not ensure Web Application quality, but a uniformity of results. This fact may even institutionalize the creation of inappropriate Web Applications. Thus, in future work, application features will be related to methodology features in order to define how each methodology feature influences each Web Application feature. For this reason, we are working on the analysis of efficiency and productivity with the aim of relating these methodologies methods to Web Applications. However, the first step must deal with characterizing these methodologies to know what Properties must be offered to users and what Quality Characteristics must be guaranteed according to users’ needs. In this sense, it is essential to describe the Properties of Web Application and the relation between Properties of methodologies and Properties of Web Applications in the results. They must also be related to Properties of methodologies, as it is necessary to know what Properties of methodologies have influence on Properties of Web Applications. In this paper, each Quality Characteristic in ISO/IEC 25000 is described in relation to a software product, but, in this case, all Quality Characteristics and Quality Sub-Characteristic are described in relation to approach characteristics as follows: Usability: In ISO/IEC 25000, it is a Quality Characteristic defined as: ‘‘The capability of the software to be understood, learned, used and be attractive to the user when used under specific conditions’’. This definition could be adapted to more closely fit our specific domain as: ‘‘The capability of an approach feature to be understood, learned, used and attractive to user when applied in specific conditions’’ or in a general way as: ‘‘A set of attributes that bears influence on the effort needed for use, and on the individual assessment of such an use, by a stated or implied set of users’’. A set of Quality Sub-Characteristics is identified for every Quality Characteristic. Quality Sub-Characteristics associated to each Quality Characteristic of Usability are similarly described by adapting other definitions from ISO/IEC, IEEE, other standards and work already published. Quality Sub-Characteristics of Usability are described as: Learnability. The capability of an approach feature to enable the user to learn how to use it. [Adapted from ISO/IEC 25000]. Understandability/ Comprehensibility. The capability of being understood and the extent to which an approach feature is clear, without ambiguity, and easily comprehensible. [Adapted from ISO/IEC 25000]. Simplicity. The degree to which an approach feature has a straightforward design easily understood. [Adapted from IEEE]. MDWE Methodology External and Internal Quality MDWE Methodology QualityinUse Web Application External and Internal Quality Web Application Quality in Use Depends on Depends on Influences on Influences on Depends on Influences on Fig. 4. Influences and dependencies between the methodology and the product. This development occurs because NDT methodology implements those Sub-Features of Tool Support that affect these Quality Sub-Characteristics. Nevertheless, values 2 are closer to values 1 and the reached state is not the required model. Thus, the designers could highly improve NDT methodology. If the Tool Support Characteristic were not considered (Fig. 7), it would be observed that this Feature would get lower values for Maintainability. The cause could be that not all features defined by NDT methodology are implemented by the Tool support. In other words, the Tool Support Feature includes aspects that NDT methodology does not implement. For this reason, the results are lower without considering this Feature. In this sense, the framework could be used for predicting and simulating different situations and designers could decide what features should be implemented, attending to the analyzed results. This is an important aspect, as they can optimize the implementation cost of those features that highly affect the Quality Characteristic that must be firstly implemented. Generally, NDT methodology has sensitively best scores in Usability, being an approach easy to understand, learn, use and attractive to users. Additionally, it also has best scores in Maintainability, being an easy approach to be modified when correcting defects, meeting new requirements, making future maintenance easier or coping with a changeable environment. Besides, it shows good performance in Functionality, Reliability andPortability,althoughitislesssensitive toQualityCharacteristics such as Usability and Maintainability. Finally, it could be improved in all Quality Characteristics due to the required Quality Model. 7. Future work Although Microsoft Excel was used to implement the sample application, we are currently working on QuEF-TS, a general tool support for implementing the QuEF framework. This tool support is composed of different tools that are focused on each phase of QuEF (Strategy phase, Design phase, Transition phase, Operation phase and Quality Continuous Improvement phase): QuEF-S: It is the tool that supports the Quality Model Strategy phase. (It is being developed). QuEF-D: It is the tool that supports the Quality Model Design phase. (It is not developed yet). QuEF-T: It is the tool that supports the Quality Model Transition phase. (It is not developed yet). QuEF-O: It is the tool that supports the Quality Model Operation phase. (It is being developed). QuEF-QCI: It is the tool that supports the Quality Continual Improvement in quality management. (It is not developed yet). QuEF-TS can generate all artifacts (for instance the Checklists and MoIs) in terms of the Quality Model specified in the Strategy phase and Design phase with QuEF-S and QuEF-D tools. Thus, an important issue is to define Quality Model to be used in the Operation phase with QuEF-O, which contains specific Checklists and all the necessary elements based on the Quality Model to be analyzed, controled and evaluated for this type of methodologies. If some changes have to be applied in the Quality Model, the QuEF-T is the tool to be used for applying the Transition phase without affecting the quality management in the Operation phase. Finally, QuEF-QCI is a tool that supports the Quality Continuous Improvement phase of QuEF. Using this tool support could help improve efficiency not only in methodologies, but also in other contexts, since this approach enhances the understanding of strengths and weaknesses in different contexts. In this sense, we are working to apply QuEF-TS in other contexts. The specific contexts have only to be defined in the Quality Model. QuEF-TS generates Checklists and MoIs to start with the Operation phase (analysis, control, and evaluation). As a future work, we are going to use QuEF-TS to work with other contexts like CMMI [8] and standards like ISO/IEC 15504 and ISO/IEC 12207. This use of the tool would help companies implement and manage their capability and maturity. Another specific context to apply QuEF is the Green IT aspects for sustainability management and costs reduction in companies as well as the implementation of different agile software development methodologies like XP (Extremme Programming), Scrum or RUP (Rational Unified Process). Nowadays, we are also working in the Strategy and Design phase with specific strategic models to establish some criteria for a methodology designers group and reach a consensus on the definition of several aspects, such as the Properties hierarchy or Quality Characteristics, despite the importance of these elements and the influence of Properties on Quality Characteristics. Thus, these methods could help define the weights for Properties, Quality Characteristics and Quality Sub-Characteristics. Furthermore, this process would help reach a consensus on the definition of Sub-Features influence on Quality Sub-Characteristics. In this line, a first prototype of QuEF-S (tool support for the Strategy phase) for a first statistical study is being currently developed to gather the general opinion of MDWE experts, designers and users. We are using a fuzzy group analytical hierarchy process (AHP) [44] approach to manage this task. In these studies, a set of web pages for a first statistical study is being currently developed to know the opinion of MDWE experts, designers and users. In this statistical study, designers and users have to set the necessary weight to consider a Quality Characteristic, Quality Sub-Characteristic, Feature and relevant Sub-Feature. The weight would indicate the degree of importance. Another objective is that QuEF-TS is connected to well known social networks like Facebook [17] or Twitter [46] to make the Strategy and Design phase easier and reach the real quality management objectives. The use of social networks does not only involve the number of people that already use them, but it entails the fact that these methodologies are carried out by people and these type of systems make easy the communication among them. Besides, lots of methods in the Strategy phase of QuEF include statistical methods focused on stakeholders in order to detect the real needs and demand as well as the quality aspects that have to be guaranteed. Social networks, in particular Facebook and Twitter, offer a complete API that permit developers to improve the connectivity of stakeholders in order to better the communication and setting statistical methods for the Strategy phase and the final design of the Quality Model in the Design phase of QuEF. We think that having access to social networks results crucial to execute an effective quality management in the future. 8. Conclusions A set of Quality Characteristics and Quality Sub-Characteristics have been proposed according to ISO/IEC 9126, SQuaRE (ISO/IEC 25000), IEEE and other standards which have been adapted to MDWE standard in order to manage quality in MDWE methodoloTable 10 Quality Characteristic values in terms of all Features (values 1) and all Features except Tool Support (values 2). Quality Characteristic name Values 1 Values 2 Usability 3/4 2/3; Functionality 2/3 3/5; Portability 2/3 5/8; Reliability 2/3 3/5; Maintainability 5/7 7/8" gies. Although there are some work to converge towards a common metamodel to achieve the interoperability of these methodologies, this kind of methodologies has different metamodels and models, transformations and tool support that work in a different way. Even though users are the design target for these approaches, this situation does not let them know how to use these approaches since they do not know how these approaches can help them and which of the approaches is more suitable for them. In addition, the diversity in some features of the design of these approaches reflects that designers of these approaches do not have a clear idea of users’ needs and the Quality Characteristics they have to guarantee these needs. Therefore, designers have a twofold task; to identify the real Propertiesthat cover users’ needs and theQuality Characteristics that users demand. The convergence towards a standardization of these methods together with a consensus decision on quality management and a common Quality Model would result in that both, users and designers, apply and design these methodologies efficiently. This would be possible because they would have a common strategy, the same objectives and purposes as well as the same design of the model to be achieved. At the same time and due to their shared knowledge, designers would have a clearer idea of the changes that should be performed in the Quality Model, in case of changes in technologies or trends while designing these methodologies. Users will have the chance of comparing methodologies, and designers, on their part, will have the chance of controlling and improving the design of the methodologies. In this line, it is necessary to standardize the terminology in order to improve the access channel for communication in MDWE. Furthermore, a brief example shows how these Quality Characteristics (Hierarchical by Quality Characteristics and Sub-Characteristics) are related to Properties (Hierarchical by Features and SubFeatures). A set of formulas has been suggested in order to analyze quality in MDWE methodologies. Additionally, we have proposed some formulas to evaluate quality in terms of different Properties and define a Matrix of Influences (MoIs). As this matrix determines the influence of each Sub-Feature on each Quality Sub-Characteristics, designers are able to guess the costs of implementing a new Feature on their methodology with it. In future work, the Quality in Use is a quality aspect that has to be adapted to evaluate MDWE methodologies in the use context. Regarding the contributions obtained from this research work, it can be stated that an environment is required either to improve the current MDWE approaches or develop new MDWE methodologies. Despite the fact that the domain is immature, we consider that using this framework to manage quality would enhance the design and methods of these MDWE approaches as a consequence of a clear strategy focused on quality continuous improvement of the Quality Model. Another important aspect is that the use of a methodology ensures output consistency. However, it does not guarantee Web Application quality, but uniformity of results. Thus, in future work, application features will be related to methodology features in order to define how each methodology feature influences each Web Application Feature. For this reason, we are working on analyzing efficiency and productivity with the aim of relating these methodologies approaches to Web applications. Nevertheless, the first step must consist in characterizing these methodologies to identify Properties that must be offered to users and Quality Characteristics that must be guaranteed according to users’ needs. To conclude, we consider essential to satisfy user’s demands and expectations. This aspect can be observed in the Quality Model and designers perfectly know the aspects that should be considered. Therefore, if users fulfil their requirements, the results obtained (Web applications) will probably improve, but always depending on the use they make of Properties. This could be interesting because if these relations are defined, it will be possible to guess the Properties of a methodology that need to obtain a set of requirements in a Web Application. Even designers will optimize the implementations of Features and Sub-Features in their methodologies according to the results they can get. NDT methodology has been analyzed and evaluated for the Functionality Quality Characteristic in a brief example about the use of Quality Characteristics and Sub-Characteristics. We have checked that this entire environment could be also useful for either simulating different situations or making decisions on Properties designers must implement in methodologies in terms of the analyzed results. Acknowledgements This research has been supported by the Project QSimTest (TIN2007-67843-C06_03) and by the Tempros Project of the Ministerio de Educación y Ciencia, (TIN2010-20057-C03-02), Spain. References [1] A. Abran, R.E. Al-Qutaish, J. Desharnais, Harmonization issues in the updating of the ISO standards on software product quality, Metrics News Journal 10(2) (2005) 35–44, (Otto-von-Guericke University of Magdeburg, Germany, December, (ISSN: 1431–8008)). [2] M. Azuma, SQuaRE: the next generation of the ISO/IEC 9126 and 14598 international standards series on software product quality, ESCOM (European Software Control and Metrics conference), 2001, pp. 337–346. [3] Alcer. Federación Nacional de Asociaciones para la lucha contra las enfermedades renales. <www.alcer.org> (retrieved 04.11). [4] V. Basili, H.D. Rombach, The TAME project: towards improvement-oriented software environments, IEEE Transactions on Software Engineering 14 (6) (1989) 758–773. [5] P. Becker, L. Olsina, Towards support processes for web projects, in: Proceedings of the 10th International Conference on Current Trends in Web Engineering, Springer-Verlag, 2010, pp. 102–113. [6] C. Calero, J. Ruiz, M. Piattini, Classifying Web Metrics Using the Web Quality Model. 29 (3) (2005) 227–248. [7] C.P. Team, CMMI for Development, Version 1.2, Preface, Software Engineering Institute, Carnegie Mellon University, August, 2006. [8] Consejería de Cultura. <http://www.juntadeandalucia.es/organismos/ cultura.html> (retrieved 04.11). [9] F.J. Domínguez-Mayo, M.J. Escalona, M. Mejías, I. Ramos, L.F. Sanz, A Quality Evaluation Framework for MDWE Methodologies, in: Proceedings of the 18th International Conference on Softwrae Quality Management (SQM), The British Computer Society, London, UK, 2010, pp. 171–184, (ISBN: 978-0-9557300-85). [10] F.J. Domínguez-Mayo, M.J. Escalona, M. Mejías. Quality Issues on Model-Driven Web Engineering Methodologies, Information Systems Development Springer Verlag, 2011, pp. 295–306, (ISBN: 10.1007/978-1-4419-7355-9_25). [11] F.J. Dominguez-Mayo, M. Espinilla, M.J. Escalona, M. Mejias, A Weighted Quality Evaluation Framework by applying the Analytic Hierarchy Process, Nineteenth International Conference on Software Quality Management, Global Quality Issues, 2011, pp. 55–66 (ISBN 978 1 907382 44 4). [12] F.J. Domínguez-Mayo, M.J. Escalona, M. Mejías, M. Ross, G. Staples, A Quality Management Based on the Quality Model Life Cycle, Computer Standards and Interfaces, January 2012, doi: 10.1016/j.csi.2012.01.004 (ISSN 0920–5489). [13] Emasesa. <www.aguasdesevilla.com> (retrieved 04.11). [14] M.J. Escalona, G. Aragón, NDT. A model-driven approach for web requirements, IEEE Transactions on Software Engineering 34 (3) (2008) 377–390. [15] M.J. Escalona, N. Koch, Requirements engineering for web applications – a comparative study, Journal of Web Engineering 2 (3) (2004) 193–212. [16] Facebook for Developers. <http://developers.facebook.com> (retrieved 04.11). [17] I. Heitlager, T. Kuipers, J. Visser, A Practical Model for Measuring Maintainability, in: 6th International Conference on the Quality of Information and Communications Technology (QUATIC 2007), 2007. [18] M. Herrera, M.Á. Moraga, I. Caballero, C. Calero, Quality in Use Model for Web Portals (QiUWeP), in: Proceedings of the 10th International Conference on Current Trends in Web Engineering, Springer-Verlag, 2010, pp. 91–101. [19] D. Hoyle, ISO 9000 Quality Systems Handbook – Updated for the ISO 9001:2008 Standard, Using the standards as a framework for business improvement, Elsevier, 2009 (ISBN: 978-1-85617-684-2). [20] IEEE Std 610.12-1990. IEEE Standard Glossary of Software Engineering Terminology. [21] ISO/IEC IS 9126, Software Product Evaluation – Quality Characteristics and Guidelines for Their Use, Geneva, Switzerland: International Organization for Standardization, 1991. [22] ISO/IEC 9126–1: Software Engineering – Product Quality – Part 1: Quality Model, Geneva, Switzerland: International Organization for Standardization, 2001. [23] ISO/IEC TR 9126–2: Software Engineering – Product Quality – Part 2: External Metrics, Geneva, Switzerland: International Organization for Standardization, 2003. [24] ISO/IEC TR 9126–3: Software Engineering – Product Quality – Part 3: Internal Metrics, Geneva, Switzerland: International Organization for Standardization, 2003. [25] ISO/IEC TR 9126–4: Software Engineering – Product Quality – Part 4: Quality in Use Metrics, Geneva, Switzerland: International Organization for Standardization, 2004. [26] ISO/IEC 25000:2005. Software Engineering – Software product Quality Requirements and Evaluation (SQuaRE) – Guide to SQuaRE. [27] SC7, ISO/IEC DTR 25021: Software Engineering – Software Product Quality Requirements and Evaluation (SQuaRE) – Quality Measure Elements ISO/IEC JTC1/SC7 WG6, January 11, 2006, 6N–565, 2006. [28] ISO/IEC 14598–1:1999. Information technology – Software product evaluation – Part 1: General overview. [29] ISO/IEC 15504–1:2004 Information technology – Process assessment – Part 1: Concepts and vocabulary. [30] ISO/IEC 12207:2008 Systems and software engineering – Software life cycle processes. [31] S. Meliá, J. Gómez, S. Pérez, O. Díaz, A Model-Driven Development for GWTBased Rich Internet Applications with OOH4RIA, in: ICWE ‘08 Proceedings of the 8th International Conference on Web, Engineering, 2008. [32] P. Mohagheghi, V. Dehlen, Developing a quality framework for model-driven engineering, Models in Software Engineering: Workshops and Symposia at MoDELS 2007 (2008) 275–286. [33] NDT (Navigational Development Techniques). <http://www.iwt2.org/en/ ndt.php> (retrieved 03.11). [34] OMG: MDA. <http://www.omg.org/mda/> (retrieved 01.10). [35] L. Olsina, G. Lafuente, G. Rossi, Specifying Quality Characteristics and Attributes for Websites, Web Engineering (2001) 266–278. [36] L. Olsina, G. Rossi, Application Quality with WebQEM, 2002, pp. 20–29. [37] OOHDM (Oriented Hypermedia Design Method). <http://www-di.inf.pucrio.br/schwabe/HT96WWW/section1.html> (retrieved 03.11). [38] R. Plösch, A. Mayr, C. Körner, Collecting Quality Requirements Using Quality Models and Goals, in: International Conference on the Quality of Information and Communications Technology, IEEE, 2010, pp. 198–203. [39] J.C. Preciado, M. Linaje, R. Morales-Chaparro, F. Sanchez-Figueroa, G. Zhang, C. Kroib, N. Koch, Designing Rich Internet Applications Combining UWE and RUX-Method, in: ICWE ‘08 Proceedings of the 2008 8th International Conference on Web Engineering, 2008 (ISBN: 978-0-7695-3261-5). [40] T. Punter, R.O.B. Kusters, J.O.S. Trienekens, T. Bemelmans, A. Brombacher, The W-Process for Software Product Evaluation: A Method for Goal-Oriented Implementation of the ISO 14598 Standard, Review Literature And Arts Of The Americas, 2004, pp. 137–158. [41] T.L. Saaty, Introduction to a modeling of social decision process, Mathematics and Computers in Simulation 25 (1983) 105–107. [42] W. Schwinger, W. Retschitzegger, A. Schauerhuber, G. Kappel, M. Wimmer, B. Pröll, C. Cachero Castro, S. Casteleyn, O. De Troyer, P. Fraternali, I. Garrigos, F. Garzotto, A. Ginige, G-J. Houben, N. Koch, N. Moreno, O. Pastor, P. Paolini, V. Pelechano Ferragud, G. Rossi, D. Schwabe, M. Tisi, A. Vallecillo, van der Sluijs, G. Zhang, A survey on web modeling approaches for ubiquitous web applications, International Journal of Web Information Systems 4 (3) (2008) 234–305. [43] Twitter for developers. <http://dev.twitter.com> (retrieved 04.11). [44] UWE (UML-Based Web Engineering). <http://uwe.pst.ifi.lmu.de> (retrieved 03.11). [45] WebML (The Web Modeling Language). <http://www.Webml.org> (retrieved 03.11).