Requirements change management based on web usage mining

FACULTY OF ENGINEERING • UNIVERSITY OF PORTO Requirements Change Management based on Web Usage Mining Jorge Manuel Esparteiro Garcia January 2016 Scientific Supervision by PhD, Ana Cristina Ramada Paiva, Assistant Professor Departmento de Engenharia Informática In partial fulfillment of requirements for the degree of Doctor of Philosophy in Informatics Engineering by the ProDEI Doctoral Programme Contact Information: Jorge Manuel Esparteiro Garcia Faculdade de Engenharia da Universidade do Porto Departamento de Engenharia Informática Rua Dr. Roberto Frias, s/n 4200-465 Porto Portugal Tel.: +351 22 508 1400 Fax.: +351 22 508 1440 Email: jorge.gar[email protected] URL: http://portal.ipvc.pt/images/ipvc/esce/docentes/jgarcia/ Jorge Manuel Esparteiro Garcia “Requirements Change Management based on Web Usage Mining” Copyright © 2015 Jorge Esparteiro Garcia. All rights reserved. ...to my sweet daughter Carolina ...to my lovely wife Ana This page was intentionally left blank. "The consequences of things are not always proportionate to the apparent magnitude of those events that have produced them." Charles Caleb Colton This page was intentionally left blank. Abstract In recent years, the use of the World Wide Web (WWW) has had a huge growth and there is a greater variety of web applications with an increasing importance in society and in supporting the development to all kinds of business. Often, most of websites are providing support services that must be maintained and improved over time. This maintenance and upgrade can be difficult because frequently the requirements are no longer actual and/or often not even exist documented. Furthermore, it can also be difficult to assess what are the most critical features in order to define the changes to implement first (in the case of several requests). Websites are increasingly monitoring usage data, and this type of information is increasingly abundant as the meta-data that is possible to extract from the web navigation. However, commonly, the data about the usage of web sites is used only for reporting and for analysing the usage traffic not adding any value to the strengthening and improvement of web sites. Extracting and analysing the information about the usage of the websites can help identify improvements on the website and help to maintain requirements updated which can be a contribution to the overall quality of the service provided. Along the research work described in this dissertation, we develop an approach and supporting tool (called REQAnalytics) to collect the information about the usage of the website, to analyse it and to build reports with recommendations that can, hopefully, contribute to increase the quality of the requirements specification and the website itself. The contributions of this research work are: a set of rules to analyse usage data; an high-level mapping tool that through a web-based application, maps ii abstract the functional requirements of the software requirements specification with the web pages and elements of the website; a novel approach to Requirements Management through a Recommender System, that collects information on the usage of a website, relates that information back to the requirements and generates reports with recommendations to the requirements specification (e.g. change the requirements priority, create new requirements, remove existing requirements). In order to evaluate the overall approach, we performed evaluation of case studies over existing websites. The results showed that REQAnalytics can produce reports in a language closer to the business, recommend new workflows, identify functionalities to create and remove, and, ultimately, give support to the maintenance of requirements of the website being analysed. Resumo Nos últimos anos, a World Wide Web (WWW) tem tido um enorme crescimento e existe uma maior variedade de aplicações web com uma importância crescente na sociedade e no apoio ao desenvolvimento de todos os tipos de negócio. Muitas vezes, a maioria dos sítios web oferecem serviços de apoio que devem ser mantidos e melhorados ao longo do tempo. Esta manutenção e atualização pode ser difícil porque muitas vezes os requisitos não estão atualizados e / ou muitas vezes nem sequer podem já existir documentados. Além disso, também pode ser difícil avaliar quais são as características críticas para determinar as alterações a serem implementadas primeiro (no caso de vários solicitações). Os sítios web estão monitorizando cada vez mais os seus dados de utilização, e este tipo de informação é cada vez mais abundante, como por exemplo os meta-dados que são possíveis extrair a partir da navegação web. No entanto, normalmente, os dados de utilização dos sítios web são utilizados apenas para fins de comunicação e marketing e para a análise do tráfego de utilização, não acrescentando qualquer valor para a melhoria do sítio web. A extração e a análise da informação da utilização dos sítios web conseguem ajudar a identificar melhorias e ajudam a manter o sítio web e os seus requisitos, o que pode ser um contribuição para a qualidade global do serviço prestado. Ao longo deste trabalho de investigação, desenvolvemos uma abordagem e uma ferramenta de apoio (chamada REQAnalytics) para recolher a informação de utilização do sítio web, analisá-la e gerar relatórios com recomendações que irão, esperamos, contribuir para aumentar a qualidade da especificação de requisitos e do próprio sítio web. As ferramentas desenvolvidas durante este trabalho de investigação foram xCONTENTS List of Figures 2.1Requirements engineering processes .................. 23 2.2Major RM activities [Wie03]....................... 25 2.3Change management workflow process ................ 26 2.4Bidirectional - Forward and Backward Traceability [Wes06].... 29 2.5Requirement Engineering tools’ scores and prices [CNA+11]... 35 2.6Screenshot of IBM Rational Doors ................... 36 2.7Screenshot of CaliberRM ......................... 36 3.1Web Mining Categories ......................... 41 3.2Best Key Practices: Web Analytics Process Guide [BJ08, Chr] .... 45 3.3Metrics Categories ............................ 46 4.1 Combined mapping of the RE activity, recommendation technique, and type of items recommended [MRE12]............... 56 6.1 An overview of the Architectural Design of REQAnalytics recommender system .............................. 75 6.2Open Web Analytics Dashboard .................... 78 6.3Functional requirements mapped with the web pages and elements 80 6.4 XSD Schema Diagram of the XML file necessary to import the functional requirements ......................... 81 6.5Mapping tool inside a bookmarklet .................. 82 6.6 Mapping tool inside a bookmarklet to create the traceability links between requirements and the implementation ............ 83 xii LIST OF FIGURES 6.7 Screenshot of the component Requirements Import XML of REQAnalytics. .................................. 85 6.8 List of the Functional Requirements previously imported to REQAnalytics .................................. 86 6.9 Screenshot of REQAnalytics with a Tooltip available in the Requirements List view, with a mapping and their HTML element id and HTML element type. ........................... 87 6.10 REQAnalytics component Details of Requirement. .......... 89 6.11 Screenshot of REQAnalytics main Dashboard ............ 90 6.12 Requirements Dependencies Directed Graph ............. 91 6.13 Screenshot of Most used Requirements Navigation Paths - Table view ..................................... 93 6.14 Screenshot of the Traceability Matrix .................. 94 6.15 Screenshot of Requirements Analytics available in REQAnalytics - Statistics And Main Metrics correlated with the functional requirements ................................ 95 6.16 Screenshot of Recommendation to Creation of New Functional Requirements ............................... 97 6.17 Screenshot of Recommendation to Change the Priority of Functional Requirements ........................... 98 6.18 Recommendation of split a Functional Requirement in two or more New Requirements ............................ 99 6.19 Recommendation of Creation of a New Dependency for a Functional Requirement ............................101 7.1The Case Study Process from Robert Yin’s guidelines [Yin09]...106 7.2 Screenshot of REQAnalytics with the Requirements Priority Recommendations in Case Study 1.....................109 7.3 Recommendation to Delete an existing Functional Requirement based on its Web Usage .........................110 7.4 Case Study 1: Recommendation to Creation of New Functional Requirements ...............................111 7.5Most used navigation path. .......................111 LIST OF FIGURES xiii 7.7 Pages Per Visit with and without recommendations applied using the same period of time, 14 days ....................112 7.6Navigation path redesigned. .......................112 7.8 Screenshot of REQAnalytics with the Requirements Priority Recommendations in Case Study 2.....................115 7.9 Most Used Navigation Paths related with Requirements in Case Study 2...................................116 7.10 Traceability Matrix between functional requirements and Web pages and elements in Case Study 2..................117 7.11 Entry Features ...............................118 7.12 Exit Features ................................118 7.13 Split a Functional Requirement in two or more New Requirements 119 7.14 Creation of a New Dependency for a Functional Requirement ...119 7.15 Requirements Dependencies Directed Graph of Case Study 2...120 xiv LIST OF FIGURES List of Tables 2.1An example of a Requirements Traceability Matrix [KS09]..... 30 3.1Web Server Log File Types and Content ................ 42 6.1Web Analytics Tools Comparison .................... 77 6.2Prioritization of the Requirements ................... 98 7.1Functional requirements analysed in the Case Study 1.......108 7.2Functional requirements analysed in the Case Study 2.......114 xvi LIST OF TABLES Acknowledgments This thesis is the outcome of my doctoral research carried out at Faculty of Engineering of University of Porto and I would like to thank the collaboration and the support of all those who contributed to its accomplishment. First and foremost, I wish to express my sincere gratitude to my supervisor, Prof. Ana Paiva, for her continuous scientific support, feedback and suggestions throughout the course of my research and whose expertise, understanding, and patience, added considerably to my graduate experience. Thank you for your valuable insights and comments and for providing everything I needed to my research work. I would like to endorse my thanks to Prof. Eugénio Oliveira and Prof. Augusto Sousa of the Doctoral Program in Informatics Engineering (ProDEI) for all the support provided. I would also like to thank all my teachers and colleagues of ProDEI who accompanied me during these years at the PhD course and at different levels contributed to I could do this work. I also thank all other researchers from the Software Engineering Research Group of the Faculty of Engineering of University of Porto for providing a very good environment for research. A very special acknowledgement to my parents Manuel and Josefina, my sister Cláudia, my brother Luís and the rest of my family, for your constant and unconditional support that helped and inspired me to complete this thesis. My final thanks goes to my sweet daughter Carolina and my lovely wife Ana, to whom I dedicate this work. xviii acknowledgments —An idea not coupled with action will never get any bigger than the brain cell it occupied. Arnold H Glasow 1 Introduction 1.1Problem and Motivation ....................... 3 1.2Research Questions and Goals .................... 5 1.3Research Strategy ............................ 7 1.4Contributions .............................. 9 1.5Results .................................. 9 1.6Structure of Thesis ........................... 10 1.7Publications ............................... 12 In Software Engineering, software quality has become a topic of major concern. Particularly, websites are being used not only for displaying static information but increasingly as core business tools, particularly through web services, intranets and web applications that run as support applications to business development [SS10,TKG07]. The websites and web applications are a type of software that must be available 8introduction settle the directions of the research. The topics that have been identified with this research were: Requirements Engineering; Requirements Management; Requirements Management tools; requirements evolution and traceability; Web usage data; Web usage mining approaches; Web Analytics tools; Recommender Systems; Recommender Systems for Software Engineering; and Recommender Systems approaches. The methods used in this phase were historical research methods like literature search and review. The achieved results from this phase are described in Part I - Background and State of the Art ( Chapter 2(p. 17) - Requiremens Engineering, Chapter 3(p. 39) - Web Usage Mining and Chapter 4(p. 51) - Recommender Systems). •Hypothesis Definition. With the research carried out and with the information gathered and studied, an hypothesis was formulated Section 5.2(p. 65). The solution is based on a new approach for Requirements Management and maintenance through recommendations to the requirements specification based on the web usage data of the website. •Approach Development. Based on the hypothesis formulated, the proposed approach was developed. The approach presented in Chapter 6 (p. 71) was implemented through a Recommender System, REQAnalytics that collects information on the usage of a web application, relates that information back to the requirements, and generates reports with recommendations and change suggestions that can increase the quality of that service. The research methods used in this phase were observational ones. •Approach Evaluation. After the proposed solution was developed for the identified problem, observational methods, like case studies, and controlled methods, like replicated experiments, were conducted to validate the solution Chapter 7(p. 105). The results obtained during this research work were presented in peer-reviewed international conferences and published in peer-reviewed international journals. contributions 9 1.4Contributions The main contributions of this research work are: • An high-level mapping tool that through a web-based application, maps the functional requirements of the software requirements specification with the implemented functionalities (web pages and elements) of the website. The main goals of this tool are: – to reduce the manual work required to maintain the traceability relationship information; – maintain traceability relationships between the requirements and web pages and elements of the website; –to identify the implemented functional requirements presented in the software requirements specification of a website and therefore trace functional requirement coverage. • A novel approach to Requirements Management through a Recommender System, REQAnalytics that collects information on the usage of a web application, relates that information back to the requirements, and generates reports with recommendations and change suggestions that can increase the quality of that service; • Provide more readable reports of web usage data in a language more closer to the business; • Improve the development of the system, supporting the evolution of the system proposing changes and improvements to the requirements specification. 1.5Results This research work has the following results: • An integrated Recommendation System, REQAnalytics, which provides features for managing functional requirements of services or web applications 10 introduction and for analyzing information of the usage of such services relating it back to functional requirements. The recommender system is able to provide the follow functionalities: – Lists of all functional requirements mapped with the web pages and elements of the website; – Identifies the most used functional requirements, the top entry requirements and the top exit requirements; –Identifies the most used website navigation paths; – Identifies all traversed sequences of functional requirements along the web application; – Automatic generation of the traceability matrix between functional requirements and implementation artifacts; –Recommendations to the web application Functional Requirements: *Change the priority of a requirement; *Create new requirement; *Delete existing requirement; *Split existing Requirement; *Create new requirement dependency; *Remove requirement dependency. • Assist in the task of determining the website’s coverage of the requirements through mapping tool developed. • Provide detailed reports that can effectively give hints/suggestions of how to improve the quality and level of service of websites and web applications. Traditionally, Web Analytics tools offer reports based on the web usage. However, there is a gap in the real utility and comprehensibility of the information gathered. 1.6Structure of Thesis This dissertation thesis is logically organized in three parts. The First Part, comprising Chapters 2to 4, gives a brief background to support the problem structure of thesis 11 research and presents the proposed solution presented in the Second Part, that is composed by chapters 5to 6. Finally, the Third Part presents the Validation and summarizes with the Conclusions and Opportunities for Future Research Work. A brief description of each Part and its Chapters is next provided. Part I: Background and State of the Art The First Part reviews the most important concepts and issues relevant to the thesis, namely: •Chapter 1: Introduction. Provides a brief introduction to the main research area of this thesis, the problem and the motivation, the main goals, the research strategy, the contributions and results achieved, as well as the structure of the thesis and a list of the publications achieved during the development of this research. •Chapter 2: Requirements Management. Focus on the concepts and terminology related to the field of Requirements Management, the research area of the problem identified. It also gives a state of the art of existing techniques and tools used in the Requirements Management process and in the Requirements Evolution. •Chapter 3: Web Usage Mining. Gives a detailed depiction of the state of the art of Web Mining, particularly Web Usage Mining, that allows for the collection of Web access information for Web pages. It also presents the application of these techniques in the context of Requirements Management addressing the identified research opportunity. •Chapter 4: Recommender Systems for Software Engineering. Discusses the evolution of Recommender Systems. Further presents a state of the art review on the topic of Recommender Systems for Software Engineering, describing existing approaches and identifying the flaws and the open issues. 12 introduction Part II: Research Approach and Solution The Second Part states the problem research approach and presents the proposed solution: •Chapter 5: Research Problem. Considers the state of the art, formulates the thesis statement, overviews the solution proposed, the methodology that was followed and draws the validation strategy. •Chapter 6: REQAnalytics. Presents the proposed solution describing a novel approach for Requirements Management through a Recommender System that generates recommendations to the software requirements specification based on the web usage data of the website. Part III: Validation and Conclusions The Third and final Part presents two experimental evaluations for the validation of the thesis, and presents the conclusions of this research work and set future directions: •Chapter 7: Validation. Gives two thorough experimental evaluations of different type of websites, assessing the efficiency of the recommendations to the requirements specification presented by the proposed recommender system and discussing the results achieved with REQAnalytics. •Chapter 8: Conclusions and Future Work. Discusses the research work done, summarizing the contributions made and points opportunities for future work directions. 1.7Publications Parts of this thesis dissertation have already been published in international conferences and journals during the author’s Ph.D. research work. A list of those is given next chronologically ordered: •A Requirements-to-Implementation Mapping Tool for Requirements Traceability , a state of the art on Requirements traceability, with an approach that supports the mapping of functional requirements with the publications 13 web pages and elements of a web site to help maintaining requirements traceability information updated and, ultimately, increasing the efficiency of requirements change management process itself that may contribute to the overall quality of the service provided. This work was presented in the 8th International Conference on Computer Science and Information Technology (ICCSIT 2015) , Amsterdam, Netherlands, and a revised version was selected to publication in the Journal of Software [GP16a]. •REQAnalytics: A Recommender System for Requirements Maintenance , a preliminary approach of REQAnalytics, a recommender system that collects information on the usage of a web application, relates that information back to the requirements, and generates reports with recommendations and change suggestions to the requirements specification baseline like features that can be removed, change the priority of requirements and presents the relationship between requirements and the proposed changes helping to maintain the software requirements specification updated and useful that can increase the quality of that service. This paper has been published in the International Journal of Software Engineering and Its Applications [GP16b]. •An Automated Approach for Requirement Specification Maintenance , presents an experimental evaluation of a case study based on an online newspaper website using an approach through a recommender system that collects the information about the usage of a website using a Web Analytics tools and generate recommendations reports that may help the requirements maintenance and increase the quality of the software requirements specification of the website. This paper was accepted and will be presented in the 4th World Conference on Information Systems and Technologies , Recife, Brazil and will be published in the Journal of Advances in Intelligent Systems and Computing Series. 14 introduction Part I Background and State of the Art —What gets measured gets managed. Peter Drucker 2 Requirements Engineering 2.1Introduction ............................... 18 2.2Requirements Engineering ...................... 22 2.3Requirements Management Tools .................. 30 2.4Discussion ................................ 37 2.5Summary ................................ 37 The main goal of this work is to improve the maintenance of services and their requirements based on the usage of a website. The purpose of this chapter is therefore to introduce the reader to fundamental concepts related to Requirements Engineering (RE) and Requirements Management (RM). 24 requirements engineering 2.2.1Requirements Elicitation The Requirements elicitation, sometimes also called Requirements discovery is the process of discovering, reviewing, documenting, and understanding the user’s needs and constraints for the system. Involves working with the customers to understand the application domain and may involve other people like end-users, managers, domain experts. These are called the stakeholders. 2.2.2Requirements Specification Requirements specification is the process of documenting the user’s needs and constraints clearly and precisely. On this process it is necessary to fully describe what the application will do and how it is expected to perform. 2.2.3Requirements Validation Requirements validation is an iterative process of ensuring that the system requirements are complete, correct, consistent, and clear. Is also necessary to verify if the requirements statement themselves are complete, correct, feasible, necessary, prioritized, unambiguous and verifiable. There are some validation techniques like requirement reviews, prototyping and test-case generation. 2.2.4Requirements Management Requirements management can be described as the process of managing changing requirements during the requirements engineering process and system development [Som04]. During the development of a system and also after it being delivered to the customer, new requirements emerge and it is necessary to keep track of this new requirements. The process of software requirements management practices is believed to be one of the first process improvements steps that a software development organisation should take [EMM01,DCVP03]. In the last years, Requirements Management is increasingly recognised as crucial [ NE00 ], due to the needing of writing requirements readable and traceable, in order to manage their evolution over time. The requirements evolution requirements engineering 25 consists in the changes made to the requirements after the initial deploy of the requirements specification document [ AP01 ]. Requirement changes may consist in additions, omissions or change and can occur in any process, elicitation, analysis, specification or validation. A software requirements specification evolves when a system change is performed, its behavior changes and it is necessary to change some of the requirements initially specified. Nowadays, the task of requirements management is considered extremely important to high quality software and to project success [ ED , LvDNdZ04 , SAW13 , LQF10 ], however the requirements management is often ignored. Figure 2.2shows the major activities that are related to Requirements Management that have been identified by Wiegers [Wie03]. Figure 2.2:Major RM activities [Wie03] Sommerville and Kotonya specified these activities in three main areas related to requirements management [ SK98 ]: (i) Change management, (ii) Requirements attributes, (iii) Requirements traceability. Change Management Change management describes the procedures and tasks needed to be followed when analysing a requirements change. Sommerville [ Som04 ] proposed a workflow process, shown in Figure 2.3, to change management. Starts with problem analysis in which requirements are discussed and changes are proposed. Then 26 requirements engineering change requests are analysed in order to assess their impact. Finally, the change is implemented, i.e., the documents are changed to reflect the change. Figure 2.3:Change management workflow process Requirement errors are the largest type of errors typically found in a project, and it is estimated that 56 percent of the discovered errors are related to the requirements specification [ HF01 , DS08 ]. Reducing requirement errors can be the single most effective action developers can take to improve project outcomes and identifying omitted requirements and finding errors during the requirements stage, as opposed to later stages of the life cycle, provides great leverage and cost savings [Wil11]. Traditionally, most requirements engineering tasks, including the requirements elicitation, are carried out in the beginning of the system development lifecycle [ Roy90 ]. However, when the software application is a service provided continually over time, as are web applications and services, requirements may change or new requirements may come up for several reasons, for instance, new laws, new needs, etc. Requirements Attributes Requirements attributes are information necessary to better describe and interpret the requirements. This include the requirement classification (e.g. functional or non-functional, priority) and the verification method or acceptance test plan. It can also include other information like the source of the requirement, change history and the unambiguously requirement ID. Requirements Traceability Requirements traceability consist in the recovering the source of requirements and is used to perform impact analysis of a requirement change. A requirement requirements engineering 27 should be traceable backwards to its origin (e.g. use cases, test cases, source code) and forwards into requirements and design entities that satisfy it. 2.2.5Requirements Evolution The requirements evolution consists in the changes made to the requirements after the initial deploy of the requirements specification document [AP01]. Requirement changes may consist in additions, omissions or change [ SOSA99 ] and can occur in any process, elicitation, analysis, specification or validation. A software requirements specification evolves when a system adaption is performed, its behaviour changes and it is necessary to change some of the requirements initially specified. 2.2.6Traceability Keep traceability information between web pages and elements of the website and requirements may be useful to evaluate the impact of change requests and help maintenance activities. Software traceability has been increasingly recognized as an important quality of a well-engineered software system [ RJ01 ]. It is defined by the Center of Excellence for Software and Systems Traceability (CoEST) as the ability to interrelate any uniquely identifiable software engineering artifact to any other, maintain required links over time, and use the resulting network to answer questions of both the software product and its development process [coe]. Software traceability is recognized as a critical success factor in software development [ DP98 ] and has been recognized as an important quality of a well-engineered software system [ GF94 ]. Requirements traceability has been demonstrated to provide many benefits to organizations that make proper use of traceability techniques [KS09]. The USA Food and Drug Administration (FDA) states that traceability analysis must be used to verify that the software design implements the specified software requirements, that all aspects of the design are traceable to software requirements, and that all code is linked to established specifications and test procedures [ Foo05 ]. In another report, "Critical Code: Software Producibility for Defense", the Committee for Advancing Software Intensive Systems Producibility 28 requirements engineering of the U.S. Department of Defense identified requirements traceability as one of the seven technology areas on which research should be targeted in order to assure the safe and correct operation of current and future software intensive systems [Com11]. Requirements traceability has been defined by Gotel and Finkelstein [ GF94 ] as the ability to describe and follow the life of a requirement, in both a forward and backward direction (i.e. from its origins, through its development and specification, to its subsequent deployment and use, and through periods of ongoing refinement and iteration in any of these phases). These traceability chains are shown in Figure 2.4. Despite the benefits resulting from the use of requirements traceability techniques, the poor tool support is perhaps the biggest challenge to the implementation of traceability [ Wil11 ] [ KS09 ]. Furthermore, the use of requirements traceability tools is just used for about 50% of the software engineering industry [ LM06 ]. One of the main reasons for this low rate is because in the existing requirement management tools, exists poor support for traceability and are inadequate for the needs of the software engineering industry [Geo]. Kannenberg and Saiedian [ KS09 ] states that creating cost-effective requirements traceability tools that improve upon the design and feature set of currently available tools would serve to greatly improve the practice of traceability in the software engineering industry. In addition, Di and Zhang [ DZ09 ] stated that the recovering traceability links between requirement and other artifacts is not yet well investigated and there is a stringent need for an automatic tracing method to trace high-level requirements to other software artifacts which are expressed in natural language and may evolve autonomously. Moreover, these techniques are used mainly between requirements and software test cases and the majority of these tools are unable to automatically generate and maintain traceability relationships [HDO03,TWF+15]. Forward traceability enables to trace the requirements sources to their resulting product requirement(s) to ensure the completeness of the product requirement specification. Tracing each unique product requirement forward into the design that implements that requirement, the code that implements that design and the tests that validate that requirement and so on. The objective is to ensure that requirements engineering 29 Sources of the Requirement Requirements Work Products that implements the Requirements Forward Traceability Backward Traceability Sources of the Requirement Requirements Work Products that implements the Requirements Pre-Requirement Traceability Post-Requirement Traceability Figure 2.4:Bidirectional - Forward and Backward Traceability [Wes06] each requirement is implemented in the product and that each requirement is thoroughly tested. [Wes06]. Backward Tracing is related to trace each unique work product (e.g., design element, object/class, code unit, test) back to its associated requirement. Backward traceability can verify that the requirements have been kept current with the design, code, and tests [Wes06]. Requirements Traceability Matrix The most used method for tracing requirements to their outcomes is a Requirements Traceability Matrix (RTM). A RTM is defined as a table that illustrates logical links between individual functional requirements and other system artifacts [ Wie03 ]. However, manual traceability methods are not suitable for the needs of the software engineering industry since traceability links that need to be captured grow exponentially with the size and complexity of the software system [ CHCC03 ]. Traceability analysis is related to the process of tracking forward or backward through a network of interrelationships between components of a system and their documentation. An example Requirements Traceability Matrix is shown in Table 2.1. Requirements Traceability is also an important methodology during the maintenance phase of an application or web application. The initially defined require- 30 requirements engineering Table 2.1:An example of a Requirements Traceability Matrix [KS09] System Requirement Software Requirement Design Element Code Module Test Case A00500150-8000505 005-0015080-00112 Airspeed Calculation Calculate_airspeed() Tc_103.doc A00500150-8000506 005-0015080-00234 Airspeed Display Display_airspeed() Tc_125.doc ments in the software requirements specification often change during the lifecycle of the project and it is very important to assess the impact of these changes. Traceability allows to determine what requirement, test cases or other artifacts need to be changed and can also determinate the costs and risks associated with that change. Pohl [ Poh96 ] states that a traceability approach should provide answers to the following questions: what traceability information should be captured, how traceability information should be captured and how traceability information should be stored. Sherba [ SA03 ] added another question that a traceability approach should also answer: how traceability relations are going to be viewed and queried. 2.3Requirements Management Tools There are several research and commercial tools that provide capabilities for documenting requirements, managing their changes and support traceability between artifacts [ NE00 , CNA+11 , HKL10 , KKKC08 ]. Requirements management tools are increasingly used to ease the Requirements Engineering processes and allow for more systematic and formalized maintenance of requirements, change management and traceability. Figure 2.5shows a table from a deep survey [ CNA+11 ] with the main software solutions for Requirements Engineering, detailing its features, price, and a score to classify the best feature for each solution . requirements management tools 31 Due to the large number of RE tools, we discussed here only three, which are recognized as noteworthy based on usage and market presence 2.3.1IBM Rational Doors DOORS is a requirements management application for optimizing requirements communication, collaboration and verification throughout your organization and supply chain. Figure 2.6shows a screenshot of the tool. It provides several features of Requirements Management, Traceability and Test tracking. Doors also provides visualisation of documents as hierarchies and its extension language enables a wide range of supporting tools to be built, and many are provided as menu commands and examples. 2.3.2Rational RequisitePro RequisitePro is specially designed for project teams who want to manage their application software requirements, write good use cases, improve traceability, strengthen collaboration, reduce project risk, and increase quality. Rational RequisitePro is an easy to use requirements management tool that lets teams share their requirements using familiar document-based methods while leveraging database-enabled capabilities such as requirements traceability and impact analysis. The result is better communication and management of requirements with the increased likelihood of completing projects on time. Successful projects start with requirements management - the more effective the maintenance of requirements, the greater the resulting quality and customer satisfaction. 2.3.3CaliberRM by Borland CaliberRM is a complete requirements solution which ensures compliance and alignment of development to the business needs. It facilitates stakeholder collaboration, rich visualization, robust management, and traceability of requirements to Agile delivery plans. Some of the main features of this tool are: 32 requirements engineering • Requirements management. Manage the requirements with version control, traceability, impact analysis, and requirement reuse. Also, provides the traceability to model internal relationships between requirements, as well as external relationships with regulatory controls. This traceability supports impact analysis, so you can understand when requirements change and how those changes impact the project. • Requirements visualization. Allows to visualize requirements as interactive, testable user flows to increase clarity and precision and automatically generate test cases from defined scenarios and collect feedback in context to ensure that requirements are defined completely and unambiguously. • Traceability and real-time impact analysis. Provides automatic generation and reporting of customized trace tables. The built-in control detects orphans and highlights suspect traces. • Collaboration and review. Facilitate collaboration across business stakeholders with Caliber Review. See related attributes, traces and discussions, to ensure stakeholder participation and feedback. Allows to create filters to show only items which require review/approval, and approve baselines with electronic signatures. 2.3.4Requirements Management Approaches In other traceability approaches, there is some work related to different tools and methodologies for requirements traceability. Saiedian [ SKM11 ] stated that existing traceability tools focus primarily on requirements traceability or traceability among the various artifacts of a software product, however, there is still an open issue in end-to-end traceability. For instance, Sherba and Anderson [ SA03 ] proposed a traceability, which has an evolution service that analyzes the changes to a set of relationships over time. It analyzes existing links in different versions without evolving the links themselves. Huffman Hayes et al. [ HDS06 ] developed a traceability tool, called RETRO (Requirements Tracing On-target [ Wes06 ]), to trace requirements. RETRO implements both VSM and LSI for determining requirements similarity. requirements management tools 33 Egyed and Grunbacher [ EG02 ] presented a tool-supported technique easing trace acquisition by generating trace information automatically. Neumuller and Grunbacher [ NG06 ] developed a traceability environment that was introduced in a very small software company where they have found out that comparably simple automation techniques were surprisingly effective. Cuddeback et al. [ CDH10 ] presented a framework for the study of analyst interaction with artifacts generated automatically during the tracing. Neumuller and Grunbacher [ NG06 ] developed a traceability environment and introduced in a very small software company where they have found that comparably simple automation techniques are surprisingly effective. Cuddeback et al. [ CHGH+14 ] presented a framework for the study of analyst interaction with artifacts generated automatically during the tracing. Saiedian [ SKM11 ] stated that existing traceability tools focus primarily on requirements traceability or traceability among the various artifacts of a software product, however, there is still an open issue in end-to-end traceability. For instance, Sherba and Anderson [ SA03 ] proposed a traceability, which has an evolution service that analyzes the changes to a set of relationships over time. It analyzes existing links in different versions without evolving the links themselves. Huffman Hayes et al. [ HDS06 ] developed a traceability tool, called RETRO (Requirements Tracing On-target [20]), to trace requirements. RETRO implements both VSM and LSI for determining requirements similarity. Egyed and Grunbacher [ EG02 ] presented a tool-supported technique easing trace acquisition by generating trace information automatically. Gotel and Finkelstein [ GF94 ] detected that traceability methods were preferred in the industry due to shortcomings in available traceability tools. However, this problem still exists today because manual traceability methods are still preferred by a significant percentage of software organizations [ KS09 ]. Some research has also been done to investigate the methods to recovery traceability links between design and implementation. For instance, Egyed and Grunbacher [ EG02 ] proposed a method to recover traceability links between requirements and Java programs by monitoring the Java programs to record the usage of the program classes when scenarios are executed. Despite this, the majority of the related techniques are used mainly between 40 web usage mining website data usage [KSK12]. This kind of data, however, is heterogeneous, semi structured or unstructured in contrast to the standard data that the common data mining methods have to deal with [ZS08]. Researchers are increasingly studying new methods to manage and analyse web data in order to structure, standardize and organize it. Therefore, new mining algorithms were created to better answer the demands of web data. Then, Web Mining (or Web Data Mining) can be described as the whole of data mining and related techniques used to automatically discover an extract information from web documents and services [ CL96 , KB00 ]. Web data can be in several formats (e.g. HTML, XML, Javascript, plain text) and can be stored in different servers just for a single website. 3.1Web Mining Web Mining can be categorized to three areas based on objective of information or knowledge intended to extract [KB00] as shown if Figure 3.1: • Web Content Mining: describes the discovery of useful information or knowledge from web documents. The web content data is considerably heterogeneous and unstructured and consists of several type of data like text or multimedia like image, audio or video. In web content mining there is also two groups of researching: web page content mining, that is related to retrieval of information available on the web; search result mining, that refers to different approaches of indexing data for an easier knowledge discovery. • Web Structure Mining: refers to analysing and discovering the model of the link structure of the Web. Throughout these methods it is possible to categorize web pages and extract information like the similarity and relationship between them, using the hyperlink structure. These techniques and methods can be also used to redesign the website that leads [SS10] to an improvement of the quality of a website. web usage mining 41 • Web Usage Mining: Collects and analyse the data generated through websites visits and transactions. Figure 3.1:Web Mining Categories 3.2Web Usage Mining Web usage mining refers to the discovery of user access patterns from Web usage logs. The kind of data collected can be data stored in server access logs, referrer logs, agent logs, user profiles, bookmark data, user sessions, cookies stored by browsers, and any other data generated by the interaction between the user and the web page. Web Usage Mining have an important part in understanding the usability of the website design, the improvement of user’s relations and improving the necessity of system presentation [MA15]. One important kind of data is the data generated by users’ click stream. This data after analysed can be very useful to predict user behaviour and to redirect the user to the information which he demands more easily. The main sources to get the row log data from a website are: i) Client Log File; ii) Proxy Log File; and (iii) Web Server Log File. 3.2.1Web Server Log File The most significant and frequently used source for web usage mining is web server log data. This web log data is generated automatically by a web server 42 web usage mining Table 3.1:Web Server Log File Types and Content Log File Type Content Access Log All resource access request sent by user Agent Log User’s browser, version, OS Error Log Details of errors occurred while processing user access request Referrer Log Contains information about referrer page. when its services user requests, which contains all information about visitor’s activity [ KB00 ]. The common server log file types are access log, agent log, error log and referrer log [SK]. Table-1summarizes each. 3.2.2Web Usage Mining Tasks The process of web usage mining can be divided in three independent tasks [ CMS99 , Shr12 ]: Preprocessing : do the conversion of the raw data into the data abstraction (users, sessions, episodes, clicktreams, and pageviews) necessary for further applying the data mining algorithm; Pattern Discovery which converges the algorithms and techniques from data mining, machine learning, statistics and pattern recognition; Pattern Analysis : validation and interpretation of the mined patterns. 3.2.3Preprocessing The preprocessing of Web usage mining is usually complex. Purpose of preprocessing is to offer structural, reliable and integrated data source to pattern discovery. It consists of four steps: (i) Data cleaning, (ii) Transaction Identification; (iii) Data Integration; (iv) Tranformation [CMS99]. Data Cleaning Data cleaning is the first step performed in the preprocessing of Web usage mining. The entries which are irrelevant in data analyzing and mining are removed. In data cleaning process, firstly, entries that have status of "error" or "failure" should be removed. Secondly, some access records generated by automatic search engine agent should be identified and removed from the access log. web usage mining 43 Transaction Identification After the data cleaning, the log entries must be partitioned into logical clusters using one or a series of transaction identification modules. These transaction identification modules can be called as either a merge or a divide module. The aim of transaction identification is creating meaningful clusters of references for each user. Both types of modules take a transaction list and possibly some parameters as input, and as output a transaction list that has been operated on by the function in the module in the same format as the input [CMS99]. Data Integration The integration of content, structure, and userdata in other phases of the Web usage mining may also be essential in providing the ability to further analyze and reason about the discovered patterns. Transformation Once the domain-dependent data transformation phase is completed, the resulting transaction data must be formatted to conform to the data model of the appropriate data mining task. For example, the format of the data for the association rule discovery task may be different than the format necessary for mining sequential patterns [CMS99]. 3.2.4Pattern Discovery After data preparation phase, the pattern discovery method should be applied. This phase consists of different techniques derived from various fields such as statistics, machine learning, data mining, pattern recognition applied to the Web domain and to the available data. The task for discovering the patterns offer some techniques as statistical analysis, association rules, sequential pattern analysis, clustering and so on. Here we will briefly describe some techniques to discover patterns from processed data [Shr12]. • Statistical Analysis such as frequency analysis, mean, median. 44 web usage mining • Clustering of users help to discover groups of users with similar navigation patterns (provide personalized Web content). • Classification is the technique to map a data item into one of several predefined classes. • Association Rules discover correlations among pages accessed together by a client. • Sequential Patterns extract frequently occurring intersession patterns such as the presence of a set of items followed by another item in time sequence. • Dependency Modelling determines if there are any significant dependencies among the variables on the website. 3.2.5Pattern Analysis Pattern Analysis is the final stage of Web Usage Mining which involves the validation and interpretation of the mined pattern. Validation To eliminate the irrelevant rules or patterns and to extract the interesting rules or patterns from the output of the pattern discovery process. Interpretation The output of mining algorithms is mainly a mathematic form and not suitable for direct human interpretations. 3.3Web Usage Mining Approaches In [ VMKR13 ], Varnagar et al. identify some work done on data collection and pre-processing stage of web usage mining. They state that Web log data preprocessing is very important and a crucial task in the entire mining process and can be strengthened by choosing and neatly applying various heuristic techniques. Further, the author adds that exploiting the usefulness of both client and server web analytics 45 side log data, is useful to produce result that are more efficient and a better match to empirical observations. 3.4Web Analytics One of the most common form of analysis of web usage data is the statistical analysis. The data is aggregated by predetermined metrics such as sessions, visits, domains, page views. Studying the behaviour of the website users, can lead to an optimization of usage and of the user experience through the website. To measure the different events of the website, web analytics tools use different Metrics. Metrics are different types of available user information, are used as a way of analysing Web traffic and can help improving a Website to meet its goals. These metrics can be divided in four categories: website usage, referrers, website content analysis, and quality assurance [ SKS14 ]. The Figure 3.3[ BJ08 ] shows some examples of metrics related to each of these categories. Figure 3.2:Best Key Practices: Web Analytics Process Guide [BJ08,Chr] Web analytics deals with several methods for measurement, data collection and data analysis. These are the main feature of commercial solutions available 46 web usage mining for web usage analysis. The web analytics process involves several steps to be implemented. First, must be defined the goals, i.e. the objectives for each metric defined. Then it is necessary to gather and collect de usage data from several sources that are considered relevant. With this collected data, an analysis for each metric is done and finally changes are implemented. Figure 3.3:Metrics Categories 3.4.1Tools There are several tools of web analytics to collect and analyse web usage data from a website. Google Analytics [ goo15 ], Piwik [ piw15 ], CrazyEgg [ cra15 ] are some of the most used web analytics tools in the market. They have different capabilities and functions and also differ in the kind of data they collect. Although there are several tools, they focus mainly on statistical analysis of website data usage [ KSK12 ] not providing detailed reports as intended in this proposal. By collecting various Web analytics metrics, such as number of visits and visitors and visit duration, one can develop key performance indicators (KPIs - Key Performance Indicators) – a versatile analytic model that measures diverse metrics against each other to define visitor trends. KPIs use these dynamic numbers to get an in-depth picture at visitor behavior on a site. This information allows businesses to align their Websites’ goals with their business goals for the purpose of identifying areas of improvement, promoting popular parts of the site, testing new site functionality, and ultimately increasing revenue [BJ08]. The kind of data gathered is also related to the metrics and KPIs that each tool analyses and gives report. There are several metrics and KPIs available in each tool, however, there are some that are present in web analytics tools: discussion 47 • Conversion Rate: KPI that measures the percentage of total visitors to a Website that perform a specific action • Exit page: The pages visitors viewed last on the site. • Landing Page: The pages through which visitors entered the site. • New Visitor: A user who is accessing a Website for the first time • New Visitor Percentage: KPI that measures the ratio of new visitors to unique visitors. • Repeat Visitor: A user who has been to a Website before and is now returning • Returning Visitor: KPI that measures the ratio of unique visitors to total visits • Total Bounce Rate: KPI that measures the percentage of visitors who scan the site and then leave. • Unique Visit: One visit to a Website (regardless of if the user has previously visited the site); an alternative to unique visitors. • Unique Visitor: A specific user who accesses a Website. 3.5Discussion Information like number of visits, duration of visit, page views, click path and exit rate, can be used for different purposes as assessment of quality of webproducts (i.e., websites, web videos) [ SKS14 ] or to statistical analysis of website data usage [KSK12]. This set of data can be collected through web analytics tools and can be helpful to maintain the requirements, particularly with regard to traceability, extensions and prioritization that, ultimately, can help in the improvement and maintenance of the web systems. Currently, web analytics tools are able to collect diverse data about the usage of a website but they only generate reports with navigation statistics, duration 48 web usage mining of navigation on the website and other metrics that are mostly often used simplistically to see which content has more adherence by users [ Ver11 ]. Either, web analytics has focused on analysis and reporting of business metrics of interest mainly to marketers [PRRS13]. Despite of continuous research in the field of Web Analytics, there are still some challenges which researchers need to work upon. The analysis and research done till this moment show that some of the open challenges of this field are: • There are several web analytics tools with large volumes of data, but they do not give any recommendations to the improvement of the website based on the data collected. • Stakeholders are often skeptical regarding a new form of automated tool support [ MT13 ]. Therefore, high quality recommendations should be presented to answer the stakeholder’s doubts. • No focus on the improvement of the software requirements specification (SRS). • The current web analytics tools have a number of weaknesses. For instance, they do not analyze the service based on different kinds (roles) of users; do not analyze typical navigational paths (which may be useful to define or improve workflows); do not produce reports based on the requirements and, therefore, in a language closer to the business. Web Mining has an immense potential in itself for recommendations to users based on their preferences [ Ver11 ]. Web usage mining is also a field with immense potential itself for recommendations for systems developers to help in the website improvement [KSK12]. The potential of the analysis of usage data is yet to be explored [ ABB12 ]. An analysis directed to the improvement is not currently done and this data is disregarded for the development and improvement of the quality of a web application. Web Analytics tools available [ SKS14 ] do not provide functionalities that enable a more intelligent analysis to suggest improvements to the website, such as suggest new workflows, identify and remove unused features or present more readable and legible reports. summary 49 3.6Summary Web usage mining has emerged as the essential tool for realizing more personalized, user-friendly and business-optimal Web services. Moreover, Web Usage Mining may be used to support user personalization, web site design and other business making decision. The key is to use the user-clickstream data for many mining purposes. Traditionally, Web usage mining is used by e-commerce sites to organize their sites and to increase profits. It is now also used by search engines to improve search quality and to evaluate search results, and by many other applications. 56 recommender systems Figure 4.1: Combined mapping of the RE activity, recommendation technique, and type of items recommended [MRE12] In Figure 4.1it is shown a combined mapping of each activity of Requirements Engineering with the recommendation technique and the type of items recommend by each recommendations system of software engineering analysed in a systematic mapping study of RSSEs [ MRE12 ]. Most of the RSSEs analysed address the elicitation and analysis activities of the Requirements Engineering process with very few studies related to the validation technique. 4.6Recommender Systems Approaches There are Recommendation Systems that have been widely used in e-commerce websites to provide user personalization [ PB07 ] like product, content or service recommendations. Recommendation Systems provide information items estimated to be also valuable for a software engineering task in a given context [ RWZ10 ]. Recommender Systems for Software Engineering (RSSE) is a novel approach to support developers in decision making. RSSE can help developers to find alternative decisions in a wide range of software engineering tasks from reusing code to writing effective bug reports. The overall goal is to provide the right information, at the right time, to the right person. This would allow requirements engineers to spend their limited time on more important aspects of the project [RWZ10]. Ghezzi [ GPST14 ] presented an approach that automates the acquisition of recommender systems approaches 57 user-interaction requirements through web logs and analyses them by means of probabilistic model checking to identify navigation anomalies and emerging users behaviours. SRRS [ RMZR08 ] proposes a Security Requirements Recommendation System that uses prior knowledge about security requirements approaches, combined with user preferences, to recommend the most appropriate approach for specific project characteristics, and in fact present a full-order ranking of all approaches. Castro-Herrera [ CH10 ] presented a hybrid Recommendation System to identify potential users who could reply to unanswered posts in online forums. A content-based recommender technique analyses the text of unanswered posts, and compares them to previous posts. Cleland-Huang [ CHM08 ] proposed a feasible approach that utilizes data mining and recommender systems to scale-up the fundamental processes of requirements elicitation and prioritization. Castro-Herrera [ CHCH09 ] presented a novel technique, that utilizes data mining techniques to analyse stakeholders’ contributions to a project in order to identify potential experts. A method for automatic classification of informal requirements is presented by Ko, Youngjoong et al. [ KPSC07 ], which is particularly useful in large-scale projects. StakeNet [ LQF10 ] is an approach for stakeholder identification which is based on the concepts of social network analysis. Fitzgerald et al. [ FLF12 ] developed an approach to feature requirements management based on predicting software failures (e.g., abandoned implementation of a feature) by analysing the communication threads in feature management systems. In a survey related to software requirements specification in model-driven development, Valderas [ VP11 ] demonstrated that few of the existing approaches are specifically defined for the specification of web application requirements. Furthermore, once requirements are specified, there is little support for allowing the systematic or automatic derivation of the conceptual model that properly satisfies the software requirements specification. Currently there are several research works in the field of recommender systems for software engineering (RSSE) [ MRE12 ]. They are mainly focused on the coding phase where the developed tools give recommendations to assist developers on several programming tasks like, suggesting code reuse opportunities [ YF05 ] or support software requirements elicitation [18]. For instance, Maalej [ MT09 ] 58 recommender systems proposed a continuous and context-aware approach for communicating user input to engineering team. 4.7Discussion Research in the field of Requirements Management based on data collected of the usage of website is few [ GP14 ], or not exactly related with the research proposed on this research work. For instance, Gao [ GLX+11 ] proposed a solution where the evolution of software requirements models is based on the feedback collected. In addition, nowadays, the Requirements Management of a website is only done to solve detected problems, satisfy or correct the failures of the software requirements specification [ KMA+13 ] not focused in its improvement, and therefore in the improvement of the quality of the website. Several studies [ MT13 , RWZ10 ] have been developed related to Recommender Systems of Software Engineering , however none of them produce recommendations to the Software Requirements Specification based on the web usage data of the website. In other related work, Danylenko and Lowe [ DL12 ] suggest recommendation systems based on context-aware composition to enable a system designer to postpone and automate decisions regarding efficiency non-functional requirements, such as performance. Though, this approach focus on handling efficiency non-functional requirements during the software development process and is carried out before deploy of the system. A recommendation-based approach to requirements reuse is presented by Dumitru [ DGH+11 ]. The proposed recommendation approach is content-based filtering, where a vector of keywords (derived from the description of the new software project) is matched with the keywords extracted from requirements artifacts from the repository of already completed software projects. A recent recommender system, INTELLIREQ [ FRST15 ] is based on different recommendation approaches that support stakeholders in requirements-related activities such as definition, quality assurance, reuse, and release planning. However, on this approach, the recommender system is used just through the elicitation phase. summary 59 4.8Summary The available RSSE [ MRE12 , LADR12 ] differ from our proposal since the data collected to generate recommendations is not based on the data on the use of the website. In addition, our approach differs from all the related works described, because we have developed a recommender system (REQAnalytics) that supports the evolution of the website and allows the requirements maintenance throughout its lifecycle. Despite the fact of several studies [ RWZ10 ] [ MT13 ] [ RMWZ14 ] have been carried related to RSSEs, none of them produce recommendations to the Software Requirements Specification based on techniques of web mining or based on web usage data. Using the data gathered from a web analytics tool, REQAnalytics generates recommendations to the software requirements specification and to the website itself. These recommendations also allow to analyze typical paths taken by users (which may be useful to define or improve workflows) and suggest changes to the functional requirements and to the website in a language closer to the business. 60 recommender systems Part II Research Approach and Solution —We are drowning in information but starved for knowledge. Rutherford D. Rogersh 5 Research Problem 5.1Research Challenges .......................... 64 5.2Thesis Statement ............................ 65 5.3Research Questions .......................... 65 5.4Research Goals ............................. 66 5.5Validation Methodology ....................... 68 5.6Summary ................................ 68 Uncontrolled, incomplete or incorrect requirements lead to many software project failures [ Sta95 , KS09 , LFVV11 ]. Programmers and software organizations are improving the methods they use to gather, analyze, document, and maintain their requirements. Nowadays, project teams document their requirements in a structured software requirements specification written in natural language. However, a Software Requirements Specification has some limitations [Wie99]: • It is difficult to keep current 64 research problem • It is hard to communicate changes to the affected team members • It is difficult to store supplementary information about each requirement • It is hard to define links between functional requirements and corresponding use cases, designs, code, tests, and project tasks. Requirements management tools can diminish these limitations since these tools provide functions to store requirements and related information, import and export requirements and define links between requirements. However, requirements management tools, particularly when used with the requirements of web applications, which are permanently evolving during its lifecycle, disregard one important kind of information that is the web usage data. Web usage data is commonly used to generate reports with the navigation statistics, duration of navigation on the website and other metrics of interest mainly to increase the number of visits of the website. In this research work, it is intended to use web usage data to support requirements maintenance in order to improve the quality of the website. In this chapter, there are identified and described several research challenges that led to the proposed novel approach of supporting the requirements management of a website through a recommender system using the web usage data. Then the thesis statement of this research work is defined and explained. The chapter concludes presenting the methodology that was used to validate the thesis. 5.1Research Challenges There are still some challenges which researchers need to work upon. The analysis and research done till this moment show that some of the open challenges of this field are: • It is extracted little knowledge of the data collected. • There are many web analytics tools with large volumes of data, but they do not give any recommendations to the improvement of the website based on the data collected. thesis statement 65 • Stakeholders are often skeptical regarding a new form of automated tool support [ MT13 ]. Therefore, should be delivered high quality recommendations to answer the stakeholders’s doubts. • No focus on the improvement of the software requirements specification. • Available tools support some kind of analysis, however, are not able to perform other kind of analysis. For instance, do not analyse typical paths taken (which may be useful to define or improve workflows); do not produce reports based on the requirements and, therefore, a language closer to the business. In a service context in which maintenance gains importance, the analysis of the use of the service may also be useful for the maintenance of the requirements as it may, for example, suggest prioritization changes which may have an impact on new updates/changes of software system. 5.2Thesis Statement Considering the literature review presented in Chapter 2,3and 4, the study and analysis performed, and the research challenges identified in Section 5.1(p. 64), the thesis statement is defined as: The analysis of the use of a web evolving system can identify possible recommendations for changes in requirements that will improve the system quality and can help maintain the requirements (e.g., traceability, prioritization) with further improvements in the own system 5.3Research Questions Given this thesis statement, the research work is focused on the following research questions: •Q1How web usage mining can support requirements change management? •Q2 How to create traceability links between the functional requirements and the website application? 72 reqanalytics maintenance which can be a contribution to the overall quality of the service provided. Existing approaches and tools do not take advantage of this data, which could improve the process of Requirements Management of a website during its lifecycle and therefore help to improve the quality of the website itself. This chapter focuses on the primary goal of this research work, which is to improve the requirements maintenance based on the usage of a website using a web analytics tool and generate recommendations reports that may help the requirements maintenance and increase the quality of the software requirements specification of the website. This is achieved by providing software engineers and software analysts with REQAnalytics, a novel approach through a web based Recommender System that supports the task of Requirements Management. In addition, it is also presented a novel approach to create traceability links with functional requirements of a software requirements specification and the implementation artifacts of a website through a high-level web based mapping tool. The mapping tool focuses on the secondary goal of this research work which is to map the functional requirements of the software requirements specification of a given website with the features available in the scope of the web pages that allows to identify traceability links between the functional requirements and the developed features of the website. The chapter also presents how the recommendations to the requirements provided by REQAnalytics are generated. These recommendations are focused on the functional requirements of the software requirements of the website. Hence, this focus on other secondary goal of this research work which is to generate reports of web usage data written in a language easier to read and understand to the business than the reports provided by existing web analytics tools. This chapter starts by presenting an overall insight on the approach, contextualizing web analytics data and how it can be used with a recommender system for generating recommendations to requirements. Then, it is presented the REQAnalytics platform that supports this approach and methodology, with an overview of its main features and describing each functionality. overview 73 6.1Overview REQAnalytics is a recommender system that using the web usage data of a website, and the information of the mapping of the functional requirements with the web pages and elements, suggest recommendations to the software requirements specification. Despite having some features commonly available in a Requirements Management tool, it cannot be categorized as so, due to the fact that its goal is to complement and support the task of requirements maintenance through several recommendations to the requirements. The main features of REQAnalytics include: •Integration with a Web Analytics Tool. Integrate the data provided by a Web Analytics tool in REQAnalytics and correlate this data with the functional requirements. The REQAnalytics reads the web usage data from the web analytics tool and analyze them in order to suggest recommendations to the SRS. •Requirements Import. Reads the functional requirements from the software requirements specification in a XML file with the structure as described in Section 6.8.1(p. 85) and imports them to the REQAnalytics system. •Mapping Tool. This tool is integrated within REQAnalytics and relates functional requirements with the web pages and elements of the website (see Section 6.4(p. 79)). •Functional Requirements List Report. This report lists the requirements with the mapping information, i.e., for each requirement it shows the web pages and elements that are related to such requirement. •Details of the Requirement Report. Shows all the details related to the requirement in REQAnalytics (e.g. Requirement dependencies; Mappings with web pages and elements; DOM element number of clicks; Most visited page comparison). •Dashboard Visualization Report. A report that shows the following information: Requirements that were already mapped; Number of mapping 74 reqanalytics established; Charts indicating the percentage of the requirements of the priority ranking; Number of recommendations generated by REQAnalytics. •Requirements Dependencies Directed Chart . A directed chart that shows the dependencies (detected by REQAnalytics) of each requirement with previous and subsequent requirement. The nodes of this chart represent the functional requirements and the edges represent a dependency between two requirements. This chart allows to visualize execution order of the requirements along the web application (see Section 6.8.5(p. 91)). •Most used Requirements Navigation Paths Report. A report that shows the most frequent paths taken by users along the web application. However, instead of showing the paths as sequences of web pages visited, as it is common in other tools, it shows the functional requirements executed along those paths (see Section 6.8.6(p. 93)). •Traceability Matrix Report. Correlate the links between the functional requirements and the web pages and elements of the website (see Section 6.8.7(p. 93)). •Requirements Analytics (Statistics and Main Metrics) Report. This report lists the most accessed Requirements, and other metrics like Entry Features, Exit Features and Requirements Bounce Rate. •Report of Recommendations to the Software Requirements Specification based on web usage data. This report shows the main recommendations to the Software Requirements Specification. These recommendations are mainly focused in the requirements and include: –Create New Requirement –Requirements Prioritization Change. –Delete existing Requirement. –Split a Requirement in two or more Requirements. –New Requirement Dependency. –Delete Requirement Dependency. architectural design 75 Software Requirements Specification Website pages Requirements Recommendations (New Requirements, Changes,...) Recommendation Engine (Data Analysis) Functional Requirements XML Document Web Usage Database Website Usage Logs Web Analytics Tool Mapping Tool Functional Requirements mapped to Website pages/HTML elements REQAnalytics REQAnalytics Database Figure 6.1: An overview of the Architectural Design of REQAnalytics recommender system 6.2Architectural Design This system is designed to be used totally through a web browser. Figure 6.1 shows an architecture model of the developed recommender system. Developed in PHP and using a MySQL database as support, REQAnalytics analyses the web usage and navigation of a specific website to generate recommendations to improve the quality of the software requirements specification. To generate these recommendations, we developed a web based mapping tool included in REQAnalytics that allows to map the functional requirements of the website with the web pages and elements of the features and functionalities of the website. To collect the web data usage from the website, REQAnalytics uses a web analytics tool, OWA (Open Web Analytics) that allows to gather this kind of data and save it to a database. The reasons why Open Web Analytics was selected as the web analytics tool to support the task of collecting the data of web usage of the websites are detailed inSection 6.3(p. 76). The REQAnalytics system is divided in four different main phases: • Requirements mapping - map the functional requirements with the web pages and elements of the website. (see Section 6.4(p. 79)) 76 reqanalytics • Collect Web usage data - use of the web analytics tool OWA for collecting web usage data (pages viewed, clicked web elements, click paths, session duration, entry pages, exit pages). (see Section 6.5(p. 83)) • Analysis of the data collected - the data provided by OWA is analysed and intersected with the mapping information defined on the first phase. (see Section 6.6(p. 83)) • Generation of recommendation report - it is generated a high level recommendations report with possible improvements of the requirements specification and of the website itself (see Section 6.7(p. 84)) 6.3Web Analytics Tool Selection The selection of the web analytics tool to use with REQAnalytics followed a process of features comparison. Based on the problem and the solution provided, the following criteria were identified and considered mandatory for the selection of web analytics tool: •Criteria 1:Free or Open-Source software. •Criteria 2: API and Database access to provide a way to request and work with web usage data outside of the tool. •Criteria 3: Different Web Metrics and KPIs (Key Performance Indicators) to allow different types of correlations with the requirements and analysis. •Criteria 4: Allow to work with several projects at same time for analyzing different web applications. •Criteria 5:Track clicks in all DOM elements on each web page. •Criteria 6: Identify the individual web stream paths for each user session - Navigation Paths. Eight web analytics tools available on the market that could meet the identified criteria were examined more carefully. Table 6.1summarizes the results of the comparison of features present in each web analytics tool. web analytics tool selection 77 Table 6.1:Web Analytics Tools Comparison Web Analytics Tool Crit. 1Crit. 2Crit. 3Crit. 4Crit. 5Crit. 5 Google Analytics Yes Yes1Yes Yes No No Open Web Analytics Yes Yes Yes Yes Yes Yes Piwik Yes Yes Yes Yes No2Yes Woopra Yes3Yes4Yes Yes No Yes WebTrends No No Yes Yes No Yes Clicky Yes5Yes Yes6Yes No No CrazyEgg Yes No No No No Foresee No No Yes Yes No No After analyzing all tools, the web analytics tool selected for gather the web usage data was Open Web Analytics (OWA), since it is the only that answers all previously required criteria and provides the features set essential for the development of this project. Figure 6.2shows the Dashboard of the OWA tool. One of the features considered essential was to Track clicks in all DOM elements on each web page , since it allows to map requirements with web pages and with other HTML elements, because OWA also collects the type of element clicked and its HTML identifier. As shown in Table 6.1, Piwik answers almost all features except to track clicks in the DOM elements of the web page. To perform this task, Piwik requires to change the source code of all web pages of the website to insert tracker links in all HTML elements. Tracking the clicks in all DOM elements of the web page allows to know exactly what was the clicked element and its HTML identifier. This allows to map a requirement with just a single element of a web page (e.g. input textbox, image) and not just with the URL of the web page. With this information is possible to map more than one requirement to elements inside one web page, since it is common to have more than one functionality in a single web page. 1No access to the database. 2 Piwik allows to track DOM element clicks but requires to change the source code of website to insert tracker links in all HTML elements. 3Limited to 30.000 user actions/month. 4No access to the database. 5Limited to 3.000 daily page views. 6Limited to 1website in free version. 78 reqanalytics Figure 6.2:Open Web Analytics Dashboard 6.3.1Open Web Analytics REQAnalytics uses a MySQL database to store all the data gathered from the web usage of the websites. Since MySQL is an Open Source Relational Database Management System (RDBMS) it is possible to execute SQL queries directly in the tool. Additionally, Open Web Analytics has a REST based API that can be used to export data in JSON, XML, serialized PHP, and even basic HTML. To collect the data of web usage, OWA uses a tracking code that needs to be inserted in each web page which is intended to analyze. This kind of procedure is similar to any web analytics tool like Google Analytics [ goo15 ], Woopra [ woo15 ] or Piwik [piw15]. The tracking code is a snippet of JavaScript that collects and sends data to OWA from each web page. An example of a tracking code is shown in 6.1. Listing 6.1:Example of Javascript tracking code used by Open Web Analytics <script type="text/javascript"> //<![CDATA[var owa_baseUrl=’http://www.reqanalytics.eu/’; var owa_cmds=owa_cmds || []; owa_cmds.push([’setSiteId’, ’a8e423432e9892a42124a31’]); owa_cmds.push([’trackPageView’]); mapping tool 79 owa_cmds.push([’trackClicks’]); owa_cmds.push([’trackDomStream’]); (function() { var _owa=document.createElement(’script’); _owa.type=’text/javascript’; _owa.async=true; owa_baseUrl=(’https:’==document.location.protocol ? window.owa_baseSecUrl || owa_baseUrl.replace(/http:/,’https:’) z:owa_baseUrl ); _owa.src=owa_baseUrl + ’owa.tracker-combined-min.js’; var _owa_s=document.getElementsByTagName(’script’)[0]; _owa_s.parentNode.insertBefore(_owa, _owa_s); }());//]]> </script> 6.4Mapping Tool This section describes the proposed approach and tool to create traceability links between software requirements of a web application and the web pages and elements of the website, similar to the one presented in [PFTV05]). The main idea is to trace functional requirements with the implementation artifacts like web pages and elements of the website. Identifiers for each traceability element such as requirement identifiers, web page URL and HTML element identifiers are stored within the database and correlated to the functional requirements. This mapping is a very important task in the project, since it will allow to establish correlations between the functional requirements and the implementation in order to allow REQAnalytics analyze and then generate recommendations to the software requirements specification baseline. Figure 6.3shows a diagram of the proposed solution. The tool runs through a bookmarklet and runs on any web browser. This allows the traceability tool to 80 reqanalytics Software Requirements Specification Functional Requirements exported to a XML document FR01: Requirement... Requirement selection and mapping with the HTML element or web page Traceability links database Traceabilty Relations Traceabilty Visualization HTML Reports Matrix View Figure 6.3:Functional requirements mapped with the web pages and elements work with any available website on the web. To develop a tool for the proposed traceability approach we needed to divide the project in three different tasks: •What Information? Identify the information to be captured and traced •How capture the information? Identify how this information is captured and managed in the database •Development of the tool . Create the tool itself with the different traceability views and functionalities to analyze the traced requirements with the implementation artifacts 6.4.1What Information? The first task toward the development of the tool was to identify the information that will be traced. Since we are analyzing the functional requirements of a web application, the traceability information necessary is the set of functional requirements, and the web pages and elements. mapping tool 81 Figure 6.4: XSD Schema Diagram of the XML file necessary to import the functional requirements 6.4.2How capture the information? The next task was to identify how the information of the functional requirements and web pages would be captured. Firstly, to fulfill this task, the functional requirements of the website under analysis are mapped with the web pages and elements present in the website. The mapping may be established between a single requirement and a web page, but it can also be the relation between a sequence of different web pages or elements and a requirement. With this traceability tool it is possible to map a requirement to multiple web pages and elements and an web page to multiple requirements. 6.4.3Development of the tool To perform this mapping, an XML document is previously generated with the functional requirements defined in the requirements specification of the website. This XML Schema was created during this research project and intends to ease the task of describing the functional requirements. This XML document must comply with the definitions of the XML Schema created for this purpose. The XML Schema Diagram is shown in Figure 6.4. We have chosen XML on this approach due to the fact that XML is the most widely used data-interchange language among different tools and applications. Nevertheless, this could be done with another data-interchange format like JSON. Then, this XML document with the functional requirements is imported to the system to be mapped. This mapping is established manually using a high-level 88 reqanalytics 6.Number of Visits. Displays a chart with the number of visits on the date range of analysis. 7.Most visited Page comparison. Displays a chart comparing the present requirement visits with the most visited page on the date range of analysis. 8.DOM Element Number of Clicks. Displays a bar chart with the number of clicks in the DOM element that was mapped with requirement, on the date range of analysis. 9.Top 5DOM elements clicked on this page. Displays a pie-chart with the top 5DOM elements most clicked on this page. This chart is to show REQAnalytics’ users a comparison between the DOM element that was mapped with the present requirement and the most clicked DOM elements in that web page. components of reqanalytics 89 Figure 6.10:REQAnalytics component Details of Requirement. 6.8.4Dashboard Visualization The component Dashboard Visualization shows the main dashboard of REQAnalytics. This user interface allows users to check several data related to their projects. The information is organized and presented in a way that it is easy to read the most relevant information about the project The features included in this dashboard are shown in 6.11 with the respective number in the follow list 90 reqanalytics identifying the features: 1. Number of mapping established and number of recommendations generated by REQAnalytics. 2. Bookmarklet link to include in the bookmarks of the web browser. This bookmarklet will allow to map the requirements with the web pages or elements in any website. It is also provided some instructions to help users in the task of mapping the requirements. 3. Charts indicating the percentage of the requirements of the priority ranking and percentage of mapping established and requirements mapped; 4. List of the functional requirements of the project that have been previously imported to REQAnalytics with the indication of its priority and the number of mappings that have been established with each requirement 5. Feature that allows to switch between projects that the user has privileges to access in REQAnalytics. Figure 6.11:Screenshot of REQAnalytics main Dashboard components of reqanalytics 91 6.8.5Requirements Dependencies Graph The component Requirements Dependencies Network Chart allows to view a directed chart with all the navigations paths taken by the users of the website under analysis, from the point of view of functionalities performed (the requirements describing such functionalities). Figure 7.15 (p. 120) shows a screenshot of the directed graph automatically generated by REQAnalytics. Figure 6.12:Requirements Dependencies Directed Graph REQAnalytics automatically generates this graph based on the information of web usage supplied by Open Web Analytics correlated with the mapping information of the requirements. A Lemma 6.8.1is shown to clear describe how this chart is generated. Lemma 6.8.1. The mapping can be established from a requirement to a list of web elements, a set of web elements or one web element. For that we have 92 reqanalytics tree functions: F1 : req →[webElement] \\relates requirements with sequences of web elements F2 : req → {webElement} \\relates requirements with sets of web elements F3 : req →webElement \\relates requirements with web elements We have defined a auxiliary function ( Fn ) to transform previous functions into one returning always a set of web elements: Fn :req → {webElement} if req in dom F1 return elems F1(req) else if req in dom F2 return F2(req) else if req in dom F3 return {F3(req)} So, now, in order to transform a sequence of logs into a sequence of requirements, we have defined a function TransformLogsIntoSeqOfReqs defined as: TransformLogsIntoSeqOfReqs(log : seq of webElement) =⇒seq of req if log = [] return [] else let l1_webElements _l3=log in return TransformLogsIntoSeqOfReqs(l1)_ [r|rin set elems RESULT & Fn[r] = elems webElement]_ TransformLogsIntoSeqOfReqs(l3) In the directed graph the user can for each requirement visualize its previous requirement and its post requirement that allows to identify possible new requirement dependencies. The width of each arrow is correlated with the number of clicks between the requirements. components of reqanalytics 93 6.8.6Most used Requirements Navigation Paths The component Most used Requirements Navigation Paths allows to view the most common navigations paths taken by the users of the website under analysis, associated with the requirements. Figure 6.13 (p. 93) shows a screenshot of the table with the list of most used requirements navigation paths. These requirements navigations paths were created using the same definitions described in Lemma 6.8.1. Figure 6.13:Screenshot of Most used Requirements Navigation Paths - Table view 6.8.7Traceability Matrix The mapping tool presented at Section 6.4(p. 79) allows to identify traceability links between the functional requirements and the developed features of the website. The presented methodology for requirements traceability can be applied in an evolutionary context that supports and manages functional requirements during software lifetime. Most of existing techniques of traceability are used commonly between require- 94 reqanalytics ments and software test cases[ HDO03 ] and these tools are unable to automatically generate and maintain traceability relationships. With the information of the traceability links and the functional requirements stored in the database, REQAnalytics will provide several reports of traceability visualization like a traceability matrix as shown in Figure 6.14 and lists with the tracing links between the functional requirements and the implementation artifacts. Figure 6.14:Screenshot of the Traceability Matrix 6.8.8Requirements Analytics - Statistics and Main Metrics The component Requirements Analytics - Statistics and Main Metrics displays several tables with different statistics and metrics of the functional requirements under analysis on REQAnalytics (See Figure 6.15 (p. 95)). The metrics that are possible to view are: (i) Requirements Visits; (ii) Entry Features; (iii) Exit Features; (iv) Bounce Rate. These metrics are similar to the metrics used by the traditional web analytics tools like Google Analytics [ goo15 ] or Piwik [ piw15 ]. However, on these metrics, the web pages and elements are correlated with the requirements which allow to have a novel kind report of web analytics that is the "Requirement Analytics". With this novel approach in the domain of web analytics, it is possible to analyze the following also novel metrics: components of reqanalytics 95 •Requirements Visits. Displays the number of visits of the web pages and elements that were mapped with each requirement. •Entry Features. Displays the number of visits as first page of the website correlated with the web pages and elements that were mapped with each requirement, i.e., the first functionality performed when starting interaction with the web application. •Exit Features. Displays the number of visits as last page of user session correlated with the web pages and elements that were mapped with each requirement. •Requirements Bounce Rate. Displays the percentage of visitors who enter the site and then leave ("bounce") rather than continuing on to view other pages within the same site correlated with the web pages and elements that were mapped with each requirement. Figure 6.15: Screenshot of Requirements Analytics available in REQAnalytics - Statistics And Main Metrics correlated with the functional requirements 96 reqanalytics 6.8.9Recommendations to the Software Requirements Specification (SRS) The REQAnalytics using the web usage data from Open Web Analytics and the mapping information with the correlation between the functional requirements of the website and the web pages and elements, automatically generate several recommendations that supports the task of requirements change management. Despite, the accuracy of these recommendations, the users of the REQAnalytics need to analyze the suggested changes before implementing them into the software requirements specification of the website, because despite being sustained recommendations on real web usage data the process of recommendation is an automatic process without human intervention. The recommender system provides the following recommendations specifically directed to the functional requirements of the website and to its maintenance and improvement: Create New Requirement The first type of recommendations generated by the recommender system is the creation of new requirements. This recommendation is generated based on the web pages that have been visited for at least 10 times during the period of analysis and do not have any requirement associated. During previous empirical experiments, we have noticed that when this occurs it is because there exists a functionality that has been probably implemented after the deploy of the website and the requirements specification was not updated accordingly. So, the recommender system, analyzes what are the visited pages that meet these conditions and suggests the creation of new requirements as it is shown in Figure 6.16. In addition to this analysis, the system also analyzes if there are several pages with a high similarity pattern URL ( > 90%). This occurs when there are pages that have the same functionality and the same name but with different values components of reqanalytics 97 Figure 6.16: Screenshot of Recommendation to Creation of New Functional Requirements in the URL variables. REQAnalytics aggregates these pages and suggests the creation of a single requirement for these pages. Requirements Prioritization Change Figure 6.17 shows the recommendations to the priority of each requirement. The possible recommendations defined for prioritization of the requirements are: Maintain, Increase or Decrease the priority of the requirement. —The greatest value of a picture is when it forces us to notice what we never expected to see. John Tukey 7 Validation 7.1Case Study 1: Website of Regional Newspaper ..........107 7.2Case Study 2: SIGARRA: Faculty of Engineering of University of Porto Website ............................113 7.3Discussion ................................120 7.4Summary ................................123 To validate the approach proposed in this research work, including the methodology along with processes defined for the design and the development of the approach, two case studies were conducted. The first case study provides an experimental evaluation of a website. It is a website of a regional newspaper, cidadetomar.pt, and is described in Section 7.1(p. 107). The second case study was carried out through an experimental evaluation of a module of SIGARRA, the web information system of Faculty of Engineering of University of Porto. The first case study was chosen to allow to carry an experimental evaluation 106 validation on a website that is public and the type of users is very heterogeneous. This case study will allow to understand if the software requirements specification complies with the usage of a traditional news website. In the second case study, since the module under analysis is just available in an intranet, the experiment will allow to analyse in a more controlled web environment with trained users to the core functions of the web application, how the recommendations to the requirements specification can be correct and assertive. Both cases studies were developed following Robert Yin’s guidelines [ Yin09 ] which have nearly universal acceptance on qualitative case study methods. Yin’s six-stage case study process is shown on Figure 7.1(p. 106). Qualitative case study has been gaining acceptance as a valid and valuable research method in a large number of diverse scientific domains [ EG07 , BCL11 , Yin09]. Figure 7.1:The Case Study Process from Robert Yin’s guidelines [Yin09] This method comprises all aspects of a case study research method. Therefore, when conducting a case study, there are six major activities to be followed: (1) Plan – consists in identifying the research questions and idealize the case study; case study 1:website of regional newspaper 107 (2) Design – define data to be collected as well as procedures to maintain the case study quality (external validity and internal validity) and identify the criteria to interpret the findings; (3) Prepare – identifying teams to be part of the case study and define procedures for data collection; (4) Collect – case study execution with data gathering; (5) Analyze – consists in examining, categorizing, tabulating data to draw empirically based conclusions; and (6) Share – report the case study demonstrating its findings and results. The objective of the first case study is to assess the REQAnalytics approach, through a news website, in order to evaluate the ability to suggest recommendations to the software requirements specification using the web usage data collected through a web analytics tool. It is also intended to evaluate what type of recommendations can REQAnalytics provide, and what are the most appropriate metrics to use in order to support the task of requirements management and maintenance. The second case study provides a deeper analysis by evaluating the functional requirements navigation paths and the requirements dependencies so that REQAnalytics can suggest other type of recommendations to the software requirements specification. 7.1Case Study 1: Website of Regional Newspaper This section presents the results of the application of REQAnalytics in a news website: Cidade Tomar, a regional newspaper of Portugal. In this case study it is intended to assess the real ability of the system to suggest recommendations to the software requirements specification and to the navigation paths. In particular, this case study is designed to answer the research questions of this research work that were defined in Section 5.3(p. 65). 7.1.1Goals The main objectives defined for this case of study are: 1. Analyse the data collected from the usage of the website and produce 108 validation Table 7.1:Functional requirements analysed in the Case Study 1 Id Description Priority RF01 The website must have a menu with all the categories of the news High RF02 The first web page must have a section with the latest news Medium RF03 There must exist a news search box in all web pages Low RF05 The website shall have a section called "Desporto" High RF06 The website shall have a section called "Cultura" Medium RF07 The website shall have a section called "Freguesias" Medium RF08 The website shall have a section with the Latest print edition of the newspaper High RF09 The website shall have a Contacts page Low RF10 The website must have a Home button redirecting to the first page Low RF11 The website should have a Disclaimer Page Low RF12 The website shall have a detail of each news story High RF13 The website shall have a section called "Opiniao" High recommendations to the software requirements specification and to the website itself. 2. Identify common navigation paths of sessions of the website and propose changes to the navigation paths. 3. Suggest recommendations to assign the priority to requirements under analysis. 4. Suggest recommendations to create new requirements. 5. Suggest recommendations to delete existing requirements. 7.1.2Setup As described in Section 6.2(p. 75), the first phase of the REQAnalytics is to identify the functional requirements of the website requirements specification document that will be used to the analysis of the recommender system engine. Table 7.1(p. 108) describes the functional requirements that were identified and used for analysis in this case study: case study 1:website of regional newspaper 109 7.1.3Procedure The functional requirements described in Table Table 7.1(p. 108) were firstly exported to an XML document in the format accepted by REQAnalytics. Then the mapping between each requirement and the web page and element that implements it. This mapping was made using the web based mapping tool include in the REQAnalytics system. The web usage data for this case study was collected using OWA. The time period used in this study was 2weeks (14 days) before the recommender system analysis was carried out. 7.1.4Results With the data collected by the web analytics tool, REQAnalytics linked the web usage data with the mapping information of the requirements in order to analyze the gathered information. After this data has been analysed, REQAnalytics generated a set of recommendations to the software requirements specification: Requirements Priority Change Figure 7.2: Screenshot of REQAnalytics with the Requirements Priority Recommendations in Case Study 1 110 validation The first recommendation generated by REQAnalytics was to the priority of the functional requirements shown in Figure 7.2. Here are some of the observations/recommendations generated by REQAnalytics based on the analysed data that may help the software engineer to improve the software requirements specification of the website and the website itself. • Recommendations to create new requirements (Figure 7.4). • Recommendations to delete existing requirements (Figure 7.3). Figure 7.3: Recommendation to Delete an existing Functional Requirement based on its Web Usage • List of the requirements analysed with the respective mapping with the web page and elements of the website. This mapping allows to identify if all requirements are implemented in the website • Recommendations to the priority of each requirement analysed (Increase, Maintain, Lower) (Figure 7.2) • Most used pages/funcionalities of the website (Figure 7.2) • More clicked page - tema.php?id=1 , (4198 clicks in the period) matches the requirement with the id RF10: The portal should have a Home button that redirects to the first page. Recommendation: Increase priority of RF10 requirement to High (previous value: Low) (Figure 7.2) • Most used navigation path of the website (Figure 7.5) With this achieved results/recommendations, changes were proposed to the requirements specification. The changes proposed to the functional requirements of the website are shown in Figure 7.2. It was detected that requirement with the id RF10 is associated with the page tema.php?id=1 , that was the most clicked url in the period of time analysed. Therefore, REQAnalytics recommends case study 1:website of regional newspaper 111 Figure 7.4: Case Study 1: Recommendation to Creation of New Functional Requirements to increase the priority of the requirement RF10 to high. These recommendations allow to support the requirements maintenance and bring the software requirements specification closer to the user needs and preferences. Navigation Path Change REQAnalytics also detected the most used navigation path shown in Figure 7.5 and proposed to redesign the navigation path detected. Page 1 Page 2 Page 3 Page 4 Figure 7.5:Most used navigation path. Regarding the most used navigation path detected, it was noticed that after users clicked Page 2, Page 3or Page 4, that were pages of news, user always need to go back to the homepage to read another news. 112 validation 0246 8 10 12 14 1.4 1.6 1.8 2 2.2 2.4 Days Average Pages Per Visit w/o recommendations w/ recommendations Figure 7.7: Pages Per Visit with and without recommendations applied using the same period of time, 14 days Page 1 Page 3 Page 4 Page 2 Figure 7.6:Navigation path redesigned. Therefore, this could be a factor for the user to leave the website before it is desired, due to the excessive number of clicks the user has to perform to read the news of the website So it was proposed to create a "Related News" section next to the news content with the navigation path presented in Figure 7.6. After the changes made to the website, we analysed what were the improvements to the web usage of the website using the same time period of 2weeks. The results show that before the changes the average number of pages per visit was 1.7and after the change of the navigation path, the average number of pages per visit was 2.11 pages using the same time period of 2weeks. Figure 7.7shows case study 2:sigarra:faculty of engineering of university of porto website 113 the results obtained for the period before the analysis carried out by REQAnalytics and the results obtained after the changes have been implemented. This corresponds to an increase of almost 25% in number of pages per visit, which for a newspaper website is a substantial value as it may correspond to an increase of their revenue from advertising. 7.2Case Study 2: SIGARRA: Faculty of Engineering of University of Porto Website This section presents the results of the application of REQAnalytics in the web information system SIGARRA: Faculty of Engineering of University of Porto Website. Due to the complexity of SIGARRA, it is analyzed only one Module called "Expenditure Authorization Requests Management". In this case study is intended to assess new type of recommendations to the software requirements specification and to analyze and generate recommendations to requirements dependencies. 7.2.1Goals The main objectives defined for this case of study are: • Identify the most used navigation paths of the website. • Automatic generate a Traceability Matrix between functional requirements and implementation (Web pages and elements). • Study what type of recommendations to the software requirements specification can be made based on the website data usage. • Identify the most used functional requirements, the top entry features and the top exit features. • Generate Recommendations to the Functional Requirements: –Change the priority of Requirements –Split existing Requirement 120 validation of the requirements. This graph allows to view a directed chart with all the navigations paths taken by the users of SIGARRA from the perspective of the requirements performed, i.e., functionalities described by requirements. Figure 7.15 (p. 120) shows a screenshot of the directed graph automatically generated by REQAnalytics in Case Study 2. Figure 7.15:Requirements Dependencies Directed Graph of Case Study 2 7.3Discussion At the end of the case studies it was possible to analyze the results and findings in order to answer the research questions that were identified in Section 5.3(p. 65) and to analyze the case study goals. discussion 121 7.3.1Case Study 1 In the first Case Study, it was possible to verify that the proposed approach is capable of analyzing the web usage of a website in order to support the requirements maintenance. Within a period of 2weeks, the REQAnalytics system used the web usage collected by the web analytics tools to help in the task of requirements management. This results allowed to answer to the research question Q1 - (How web usage mining can support requirements change management?). Analysing the web usage data as described in the proposed approach in Chapter 6 (p. 71) it was possible to understand how the behaviour of website users is important and how can it be correlated with the requirements specification. The mapping tool used to correlate the functional requirements of the software requirements specification with the web pages and elements allowed to answer the research question Q2 - (How to create traceability links between the functional requirements and the website application?). It was possible to understand the efficiency of mapping each requirement with the web pages, since the mapping tool is fully integrated in the web browser and allows to easily point and select the web pages and elements and then associate it with the functional requirements. Regarding research question Q2 , the REQAnalytics recommender system also provide several reports of traceability visualization like a traceability matrix and lists the tracing links between the functional requirements and the implementation artifacts (web pages and elements). The findings of this case study also allowed to answer the research question Q3 -How to analyze web usage data in order to suggest new navigation paths related with the functional requirements?. The results show that the analysis of web usage data with the requirements allowed to redesign a navigation path of the website. The outcome showed that before these changes have been implemented, the average number of pages per visit was 1.7and after the change of the navigation path, the average number of pages per visit was 2.11 pages using the same time period of 2weeks. At the end of this case study it was also possible to answer the research question Q4 - (How to generate recommendations to the software requirements specification?). The results obtained show that REQAnalytics was able to suggest recommendations to change the priority of each functional requirement. 122 validation This recommendation was based on the number of accesses to the web pages and elements related with the requirements. Hence, the recommender system was able to suggest recommendations to create new requirements and suggest recommendations to delete existing requirements. 7.3.2Case Study 2 At the end of Case Study 2, it was possible to confirm the research questions that were already answered in the first Case Study, namely Q1,Q2,Q3and Q4. In a different website like is SIGARRA, with different purposes, since the module of this website analyzed is private and is used as a web application to support the core business of the Faculty of Engineering of University of Porto, it was possible to achieve successful results in the goals that were initially set. The findings of this case study also allowed to answer the research question Q4 -How to generate recommendations to the software requirements specification?. As in the first case study, the REQAnaytics system was able to generate several recommendations based on the web usage data gathered by OWA, like suggest recommendations to the priority change of each functional requirement and to create new requirements and suggest recommendations to delete existing requirements. Furthermore, it was also possible to answer the research question Q5 -What type of recommendations can be suggested to functional requirements?. Using the same web usage data, the results obtained show that the system generated other type of recommendations to the software requirements specification. Analyzing the web usage data and the mapping information, three other type of recommendations were identified: (i) Split existing Requirement; (ii) New requirement dependency; (iii) Remove requirement dependency. The recommendation to split existing requirements is when REQAnalytics detects a mapped requirement with a considerable number of visits without user switching to other requirement of the website. A new requirement dependency recommendation is generated when a requirement has always a previous requirement before it is accessed. In this case it means that the requirements depend on the previous requirement. Finally, the recommendation to delete a requirement summary 123 dependency is generated when REQAnalytics cannot confirm the existence of such dependency. At the end of this case study it was also possible to answer the final research question Q6 -How to provide more readable reports in a language closer to the business?. The results obtained with this case study show that it is possible to generate several reports that allow a better understanding of the web usage data when compared to results obtained with traditional web analytics tools. The reports obtained in this case study were: (i) Functional Requirements List Report; (ii) Details of the Requirement Report; (iii) Requirements Dependencies Directed Chart; (iv) Most used Requirements Navigation Paths Report; (iv) Requirements Analytics (Statistics And Main Metrics) Report. 7.4Summary This chapter presented two case studies that were designed and executed to validate the REQAnalytics approach and its supporting processes. After finishing this experimental evaluation of these two Case Studies, the recommendations given by REQAnalytics system was able to generate a set of recommendations to the functional requirements of the software requirements specification. Changing the priority of the requirements; Creation of new requirements and Delete existing requirements. Thus, considering this case study, the observations/recommendations given by REQAnalytics were: increase the priority of functional requirement RF10 from low to high since it was the most used functionality of the website; redefine the most used navigation path; list most used pages/funcionalities of the website and a list of the requirements analysed with the respective mapping with the web page and elements. The recommendations generated by REQAnalytics are presented in a language closer to the business and with a kind of analysis that the existing analytics tools do not offer, which commonly only generate reports with the navigation statistics. These results suggest that recommender systems for software engineering can be used in a meaningful way to help the requirements maintenance during the 124 validation lifecycle of a website which requirements are in constant change and evolution and therefore help to improve the quality of the website. The next chapter discusses the research done in this work, summarizing the contributions made and points future work directions. —We worship perfection because we can’t have it; if we had it, we would reject it. Perfection is inhuman, because humanity is imperfect. Fernando Pessoa 8 Conclusions 8.1Summary ................................126 8.2Main Contributions ..........................127 8.3Opportunities for Future Research Work ..............128 8.4Conclusion ...............................129 This chapter concludes this dissertation. It begins with a summary of the progresses and the main results obtained. The main contributions to academic knowledge arising from the research reported in this dissertation are after reviewed and discussed. Taking into consideration the different research contributions, this chapter concludes pointing new opportunities for further research work. 126 conclusions 8.1Summary This research work presents an approach that collects information about the usage of a website through a web analytics tool and generate recommendations reports that may help the requirements maintenance and increase the quality of the software requirements specification and the overall website. Through a recommender system, REQAnalytics maps requirements with the functionalities of a website that implements them, and relates this information with the web usage data, in order to generate recommendations to the website under review. This mapping also allows to identify traceability links between the functional requirements and the developed features of the website. The presented methodology for requirements traceability and software requirements management can be applied in an evolutionary context that support and manage functional requirements during software lifetime. Moreover, these techniques are used commonly between requirements and software test cases and the majority of these tools are unable to automatically generate and maintain traceability relationships. In this research work we also demonstrated that creating traceability links between functional requirements and implementation artifacts like web pages and elements may provide a valuable help to the requirements maintenance of a website, since existing traceability methods like traceability matrices are also prone to errors and are vulnerable to changes in the system. It also presents experimental evaluations on two case studies where some relevant results of recommendations to requirements specification of the website were achieved. These recommendations allow to change the priority of requirements and create or remove new requirements and detect shorter paths for a given website’s functionality. The results obtained in the experiments suggest that recommender systems for software engineering can be used in a meaningful way to help the requirements maintenance during the lifecycle of a website during which requirements are constantly changing and evolving. Overall this can therefore help to improve the quality of the website. We also demonstrated that recommendations generated by REQAnalytics, like the detection of most used navigation paths, significantly improved the number of pages per visit in 25%. main contributions 127 The kind of analysis performed by REQAnalytics is not performed by existing web analytics tools that only generate reports with navigation statistics. In addition, the recommendations generated by REQAnalytics are presented in a language closer to the business. The results show that the analysis of the use of a website provide recommendations that allow the website to meet the expectations of its customers and users. Furthermore, this approach when compared to existing web analytics and other related tools, presents more readable and understandable reports for users and stakeholders. This approach also helps in the task of requirements management, which contributes to the quality of the web application itself. The results also indicated that the recommendations provided by REQAnalytics performed better than the random case; however additional work is needed to compare these results to those obtainable using other type of recommendations. 8.2Main Contributions Summarizing, the main contributions of this dissertation are: •An integrated Recommender System, REQAnalytics. Provides functionalities to manage the functional requirements of a website or web application and to analyze the web usage information relating that information with the functional requirements. The recommender system is able to provide the follow contributions: – List of all functional requirements mapped with the web page and element; – Identify the most used functional requirements, the top entry requirements and the top exit requirements; – Detection of most used navigation paths and identification of shorter paths for a given functionality provided by the website; – Identify all the functional requirements paths that were made by the users of the website; 128 conclusions –Identify the most used navigation paths of the website; – Automatic generated Traceability Matrix between functional requirements and implementation artifacts; – Recommendations to the Functional Requirements: *Priority of change of a requirement; *Create new requirement; *Delete existing requirement; *Split existing Requirement; *Create new requirement dependency; *Remove requirement dependency. •Mapping Tool. Tool to assist the task of tracing requirement coverage of a website through a high-level mapping tool that correlates the functional requirements of the website with the web pages and elements; •More Detailed Reports based on Web Usage. REQAnalytics provide detailed reports that can effectively give hints/suggestions of how to improve the quality and level of service of websites. Traditionally, web analytics tools offer reports based on the web usage. However, there is a gap in the real utility and comprehensibility of the information gathered; •Experimental Evaluation. Two case studies to assess the recommendations to the requirements specification provided by REQAnalytics, the Recommender System developed during this research work. 8.3Opportunities for Future Research Work The analysis over the previous research contributions allowed the author to identify several opportunities for future research arising from this work. These opportunities are: •Use other Web Mining methods and techniques. Since there are emerging new methods and techniques applied to web usage mining, an opportunity for future work would be apply other Web Mining methods to achieve new type of recommendations to the requirements specification. conclusion 129 •Develop other evaluation case studies. We intend to develop other case studies on more complex websites to study other kind of recommendations. •Automatically separate different roles. Using the web usage data to identify user roles could help to improve and personalize the recommendations to the software requirements specification. •Allow REQAnalytics to work with different Web Analytics tools. Since each of these Web Analytics tools can provide different kinds of data and metrics to analyze and since each website uses its tool, supporting different tools would be an important improvement to REQAnalytics. •Study how to automatically apply the recommendations to requirements specification. An interesting improvement to REQAnalytics would be to automatically apply the suggested recommendations to the software specification requirements in order to automatize the process of requirements management. •Integrate REQAnalytics with Requirements Management Tools. Add the capabilities of the REQAnalytics along with the features of a traditional requirements management tool can be an interesting approach to research. •Extend the tool to analyze dependencies between requirements which belong to different sessions. Analyse dependencies of different sessions would be an interesting research to allow to identify new requirements dependencies and to remove invalid dependencies. •Apply this approach to other type of applications. REQAnalytics can only be used in websites or web applications. Applying this same approach in mobile applications would be a promising research work. 8.4Conclusion Requirements Management is a challenging task for developers and software engineers while developing complex software systems. This dissertation has delivered a research study which allowed to understand the process of requirements