A framework for modeling and improving agile requirements engineering.

A Framework for Modeling and Improving Agile Requirements Engineering - PhD Thesis - Eva-Maria Schön September | 2017 Advisors: Dr. Mª José Escalona Cuaresma Dr. Jörg Thomaschewski Computer Languages and Systems Department University of Seville 2 A Framework for Modeling and Improving Agile Requirements Engineering 3 A Framework for Modeling and Improving Agile Requirements Engineering Contents Chapter 1 Introduction.......................................................................................................................... 13 Chapter 2 Related Work ....................................................................................................................... 17 2.1. Foundations Agile Methodologies, Human-Centered Design and Requirements Engineering 17 2.1.1. Agile Methodologies ............................................................................................................ 17 2.1.2. Human-Centered Design .................................................................................................... 18 2.1.3. Requirements Engineering ................................................................................................. 19 2.1.4. Agile Requirements Engineering ........................................................................................ 20 2.2. Relevant Work from other Research Groups ........................................................................... 22 2.2.1. Salah et al. ........................................................................................................................... 22 2.2.2. Silva et al.............................................................................................................................. 22 2.2.3. Méndez Fernández et al. .................................................................................................... 23 2.3. State of the Art Agile Requirements Engineering .................................................................... 24 2.3.1. Summary of Related Literature Reviews ........................................................................... 24 2.3.2. Gap Analysis in Related Literature Reviews...................................................................... 25 2.3.3. Review Method .................................................................................................................... 26 2.3.4. Results and Discussion – State of the Art ........................................................................ 31 2.3.5. Revision of State of the Art Analysis .................................................................................. 43 2.4. Implications and Gap Analysis .................................................................................................. 48 2.5. Chapter Summary....................................................................................................................... 49 Chapter 3 Challenges and Objectives ................................................................................................. 50 3.1. Challenges .................................................................................................................................. 50 3.2. Objectives .................................................................................................................................... 51 3.2.1. Perform a Systematic Literature Review ........................................................................... 53 3.2.2. Propose a Metamodel for Agile RE .................................................................................... 54 3.2.3. Provide Domain Specific Models of the Metamodel ........................................................ 54 3.2.4. Identify the Most Important Problems in Agile RE ............................................................ 54 3.2.5. Create Agile RE Patterns ..................................................................................................... 55 3.2.6. Share the Knowledge Concerning Agile RE ....................................................................... 55 3.3. Chapter Summary....................................................................................................................... 56 Chapter 4 Metamodel for Agile Requirements Engineering .............................................................. 57 4.1. Summary of Related Work ......................................................................................................... 57 4.2. Gap Analysis of Related Work ................................................................................................... 58 4.3. Research Approach .................................................................................................................... 59 4.3.1. Objectives and Research Questions .................................................................................. 59 4 A Framework for Modeling and Improving Agile Requirements Engineering 4.3.2. Developing the Agile RE Metamodel ................................................................................. 59 4.3.3. Developing a Profile for the Agile RE Metamodel ............................................................. 60 4.4. Results and Discussion – Agile RE Metamodel ....................................................................... 61 4.4.1. Description of the Agile RE Metamodel ............................................................................. 63 4.4.2. Additional Views on the Agile RE Metamodel ................................................................... 76 4.4.3. Profile for Agile RE Metamodel .......................................................................................... 78 4.4.4. Impact and Limitations ....................................................................................................... 79 4.5. Chapter Summary....................................................................................................................... 80 Chapter 5 Identification of Agile RE Problems .................................................................................... 81 5.1. Summary of Related Work ......................................................................................................... 81 5.2. Gap Analysis of Related Work ................................................................................................... 82 5.3. Research Method ....................................................................................................................... 83 5.3.1. Objectives and Research Questions .................................................................................. 84 5.3.2. General Study Design ......................................................................................................... 84 5.3.3. Panel of Experts................................................................................................................... 85 5.3.4. Round 1................................................................................................................................ 87 5.3.5. Round 2................................................................................................................................ 88 5.3.6. Round 3................................................................................................................................ 91 5.4. Results and Discussion – Agile RE Problems .......................................................................... 93 5.4.1. (RQ-5.1) What Are The Key Problems in Agile Requirements Engineering? .................. 93 5.4.2. (RQ-5.2) How Can We Deal With The Identified Key Problems? ................................... 102 5.4.3. Meaning of Findings .......................................................................................................... 104 5.4.4. Limitations ......................................................................................................................... 104 5.5. Chapter Summary..................................................................................................................... 105 Chapter 6 Creation of Agile RE Patterns ........................................................................................... 106 6.1. Summary of Related Work ....................................................................................................... 106 6.2. Gap Analysis of Related Work ................................................................................................. 107 6.3. Research Method ..................................................................................................................... 107 6.3.1. Objectives and Research Questions ................................................................................ 108 6.3.2. Pattern Mining Process ..................................................................................................... 108 6.4. Results and Discussion – Agile RE Patterns .......................................................................... 110 6.4.1. Overview Agile RE Patterns............................................................................................... 111 6.4.2. Examples of Agile RE Patterns ......................................................................................... 114 6.4.3. Knowledge Sharing by Means of a Web Application ...................................................... 119 6.5. Chapter Summary..................................................................................................................... 121 5 A Framework for Modeling and Improving Agile Requirements Engineering Chapter 7 Application of the Framework in Industry........................................................................ 122 7.1. General Application of the Framework ................................................................................... 122 7.2. Research Method ..................................................................................................................... 124 7.2.1. Objectives and Research Questions ................................................................................ 124 7.2.2. Tailoring the Metamodel into Domain Specific Models ................................................. 124 7.3. Results and Discussion – Application of the Framework ..................................................... 125 7.3.1. Case A: Tailoring Within a Kanban-based Environment in Germany ............................ 125 7.3.2. Case B: Tailoring Within a Scrum-based Environment in Spain .................................... 128 7.3.3. Impact and Limitations ..................................................................................................... 131 7.4. Chapter Summary..................................................................................................................... 132 Chapter 8 Results, Future Work and Conclusions .......................................................................... 133 8.1. Results....................................................................................................................................... 133 8.2. Future Work .............................................................................................................................. 136 8.3. Relation to IWT2 research group ............................................................................................ 137 8.4. Conclusions ............................................................................................................................... 138 References 140 Appendix I Glossary.............................................................................................................................. 148 Appendix II Protocol of SLR .................................................................................................................. 150 Appendix III Change Log Agile RE Metamodel ..................................................................................... 156 Appendix IV Agile RE Problems ............................................................................................................. 159 Appendix V Publications and Experiences .......................................................................................... 210 6 A Framework for Modeling and Improving Agile Requirements Engineering Figures Figure 1: Graphical abstract highlighting the contribution of this PhD thesis by means of summarizing the connections of the outcomes. .................................................................................... 15 Figure 2: Agile Values stated by the Manifesto for Agile Software Development (cf. (Beck et al., 2001)) ........................................................................................................................................................ 18 Figure 3: Human-Centred Design ISO 9241-210:2010 (International Organization for Standardization, 2010) ............................................................................................................................ 18 Figure 4: Sequential phases approach to software development ............................................... 19 Figure 5: In ASD people focus on outcomes and how they can fulfil human needs through the outputs they produce. ............................................................................................................................... 20 Figure 6: Venn diagram showing intersection of research areas influencing agile RE .............. 21 Figure 7: Related work has not investigated Agile RE from the perspective of HCD up to now25 Figure 8: Phases of a SLR (cf. (Kitchenham and Charters, 2007)) ............................................. 26 Figure 9: Search process comprising six phases and snowballing. After analyzing the data, we included 27 papers into our study........................................................................................................... 28 Figure 10: Quality assessment ........................................................................................................ 32 Figure 11: Revision of search process comprising six phases and snowballing. After analyzing the data, we included 5 papers into the revision of our study. ............................................................. 44 Figure 12: Distribution of search results among digital libraries ................................................. 45 Figure 13: Conceptual schema representing the objectives to be achieved with the PhD thesis .................................................................................................................................................................... 53 Figure 14: Mapping between vision of this PhD thesis, objectives and research questions .... 56 Figure 15: Phases to create the agile RE metamodel................................................................... 59 Figure 16: Creating a profile with Enterprise Architect ................................................................. 61 Figure 17: Agile RE metamodel (v.1.8) ........................................................................................... 62 Figure 18: Detailed view on metaclass <<User>> ........................................................................ 76 Figure 19: Detailed view on metaclass <<Stakeholder>> ........................................................... 77 Figure 20: Detailed view on metaclass <<AgileTeam>> .............................................................. 77 Figure 21: Profile for agile RE metamodel ..................................................................................... 78 Figure 22: Phases of a Delphi study ............................................................................................... 83 Figure 23: General process of study ............................................................................................... 84 Figure 24: Type of organization for which experts work ............................................................... 86 Figure 25: Process models used by experts .................................................................................. 86 Figure 26: Excerpt of questionnaire of round 1 ............................................................................. 87 Figure 27: Exemplary item of round 2 ............................................................................................ 89 Figure 28: Degree of refinement of requirements ........................................................................ 91 Figure 29: Categorized key problems in agile RE .......................................................................... 95 Figure 30: Importance key problem C1 (N=13) ............................................................................. 96 Figure 31: Importance key problem C2 (N=14) ............................................................................. 97 Figure 32: Importance key problem C3 (N=15) ............................................................................. 98 Figure 33: Importance key problem C4 (N=15) ............................................................................. 99 Figure 34: Importance key problem C5 (N=12) ........................................................................... 100 Figure 35: Importance key problem C6 (N=14) ........................................................................... 101 Figure 36: Steps for creation of agile RE patterns ...................................................................... 108 Figure 37: Pattern mining process and knowledge sharing ....................................................... 108 7 A Framework for Modeling and Improving Agile Requirements Engineering Figure 38: Relation between agile RE problem and agile technique ......................................... 109 Figure 39: Overview of types of agile techniques ........................................................................ 110 Figure 40: Agile RE patterns are composed of dynamic and static parts ................................. 112 Figure 41: Metaclass AgileREPattern ........................................................................................... 114 Figure 42: Example of a Kanban board........................................................................................ 115 Figure 43: Example of a story map ............................................................................................... 116 Figure 44: Review Meeting in form of an exhibition (Glase, 2016) ........................................... 118 Figure 45: Landing page agileRE.org ............................................................................................ 119 Figure 46: Example of agile RE pattern (Story Map) presented on agileRE.org ....................... 120 Figure 47: General application of the framework by means of tailoring domain specific models for agile RE .............................................................................................................................................. 123 Figure 48: Modeling agile RE by instantiating the agile RE metamodel.................................... 124 Figure 49: Domain specific model for agile RE in a Kanban-based environment .................... 126 Figure 50: Example of multiple Kanban boards for pattern Continuous management of requirements by means of tools ............................................................................................................ 128 Figure 51: Domain specific model for agile RE in a Scrum-based environment ...................... 129 Figure 52: Search process............................................................................................................. 151 Figure 53: Item 1.1 – direct cooperation with end users ........................................................... 178 Figure 54: Item 1.2 – iterative involvement of stakeholders ..................................................... 178 Figure 55: Item 1.3 – face to face communication ..................................................................... 179 Figure 56: Item 1.4 – evaluation of requirements with end users ............................................ 179 Figure 57: Items 1.5 – review of implemented requirements.................................................... 180 Figure 58: Item 1.6 – continuous communication ...................................................................... 180 Figure 59: Item 2.1 – understanding of agile values .................................................................. 181 Figure 60: Item 2.2 – avoiding large requirements analysis in the beginning ......................... 181 Figure 61: Item 2.3 – decisions of the development team ........................................................ 182 Figure 62: Item 2.4 – handling requirements within an organization ....................................... 182 Figure 63: Item 2.5 – understanding agile values in terms of fears of losing control ............. 183 Figure 64: Item 2.6 – perception of requirement analysis ......................................................... 183 Figure 65: Item 3.1 – impact analysis of components ............................................................... 184 Figure 66: Item 3.2 – impact of styleguides and prototypes on user experience .................... 185 Figure 67: Item 3.3 – restricting solution finding with finalized prototypes ............................. 185 Figure 68: Item 3.4 – continuous requirements management .................................................. 186 Figure 69: Item 3.5 – classical methods for elicitation and evaluation .................................... 186 Figure 70: Item 3.6 – sharing insights with the whole development team ............................... 186 Figure 71: Item 3.7 – traceability of requirements ..................................................................... 187 Figure 72: Item 3.8 – point of time for carrying out RE in agile projects .................................. 187 Figure 73: Item 3.9 – handling non functional requirements .................................................... 188 Figure 74: Item 3.10 – changes in requirements documentation ............................................. 188 Figure 75: Item 4.1 – detailing requirements for short-term iterations .................................... 189 Figure 76: Item 4.2 – short-term changes in priorities of requirements ................................... 189 Figure 77: Item 4.3 – outlook on upcoming iterations ............................................................... 190 Figure 78: Item 4.4 – avoiding effort estimation in hours .......................................................... 190 Figure 79: Item 4.5 – slicing requirements to fit into iteration .................................................. 191 Figure 80: Item 4.6 – losing sight of the big picture due to complex requirements ................ 191 Figure 81: Item 5.1 – coordination effort due to dependencies ................................................ 192 Figure 82: Item 5.2 – analyzing requirements to avoid interdependencies ............................. 192 Figure 83: Item 5.3 – describing requirements in terms of objectives ..................................... 193 8 A Framework for Modeling and Improving Agile Requirements Engineering Figure 84: Item 5.4 – deviating test cases from requirements ................................................. 193 Figure 85: Item 5.5 – clarity of requirements .............................................................................. 193 Figure 86: Item 5.6 – reasoning about benefits of a requirement ............................................ 194 9 A Framework for Modeling and Improving Agile Requirements Engineering Tables Table 1: Activities in agile RE .......................................................................................................... 21 Table 2: Keywords used for search ................................................................................................ 27 Table 3: Search space ..................................................................................................................... 28 Table 4: Quality checklist ................................................................................................................. 30 Table 5: Distribution according to research methods ................................................................... 32 Table 6: Sub-criteria RQ1................................................................................................................. 33 Table 7: Sub-criteria RQ2................................................................................................................. 36 Table 8: Sub-criteria RQ3................................................................................................................. 38 Table 9: Artifacts in Agile RE............................................................................................................ 38 Table 10: Overview of parameters for revision of search ............................................................. 43 Table 11: Findings of revision related to RQ1.1 ............................................................................ 45 Table 12: Findings of revision related to RQ1.2 ............................................................................ 46 Table 13: Overview of objectives of this PhD thesis...................................................................... 52 Table 14: Overview of conceptual improvements of the agile RE metamodel ........................... 60 Table 15: Color coding scheme for the agile RE metamodel ....................................................... 62 Table 16: Problems in agile RE reported by related work ............................................................. 82 Table 17: Overview of rounds .......................................................................................................... 85 Table 18: Know-how of panel (N=26) in terms of ASD ................................................................. 86 Table 19: Exemplary item in round 1 ............................................................................................. 88 Table 20: Problems in agile RE (key problems are highlighted in green, whereas problems where only 1/3 of the experts rate as problem are highlighted in yellow)........................................... 92 Table 21: Key problems in agile RE ................................................................................................ 94 Table 22: Calculations of importance concerning key problem C1 ............................................. 96 Table 23: Calculations of importance concerning key problem C2 ............................................. 97 Table 24: Calculations of importance concerning key problem C3 ............................................. 98 Table 25: Calculations of importance concerning key problem C4 ............................................. 99 Table 26: Calculations of importance concerning key problem C5 ........................................... 100 Table 27: Calculations of importance concerning key problem C6 ........................................... 101 Table 28: Agile RE patterns matched to agile RE problems ....................................................... 111 Table 29: Classification of Agile RE patterns in dynamic and static .......................................... 113 Table 30: Search space ................................................................................................................. 151 Table 31: Full list of search keywords .......................................................................................... 152 Table 32: Inclusion and exclusion criteria.................................................................................... 153 Table 33: Quality assessment criteria .......................................................................................... 154 Table 34: Data Collection Form .................................................................................................... 154 16 A Framework for Modeling and Improving Agile Requirements Engineering Chapter 4 presents the agile RE metamodel, which is the heart of our framework. We will explain the metamodel and provide an abstract view on the complex field of agile RE. Moreover, we will discuss the benefits and limitations of the metamodel. In addition, we create a visual language for the agile RE metamodel by means of a profile. This profile can be used to build instances of the metamodel, which allows solving real problems in industry. In chapter 5, we will present the catalogue of agile RE problems. Therefore, we identify the most important problems in agile RE industry has to cope with today. We will discuss the results of our iterative expert judgement process with 26 experts working in the field of ASD. Subsequent, chapter 6 shows the catalogue of agile RE patterns. To this end, the chapter introduces the concept of agile RE patterns, with which the identified agile RE problems can be solved by means of examples. In addition, the web application (agileRE.org) will be explained. Chapter 7 provides an overview of how the framework for modeling agile RE can be applied in industry. Therefore, we present domain specific models of the metamodel in order to show how the metamodel can be instantiated using the profile. One process model is based on Kanban and is used in the area of e-commerce. The other process model is based on Scrum and applied in the area of e-government Finally, chapter 8 presents the final conclusion of this research, points out future lines of research and outlines the relation of this work to the IWT2 research group. This work also contains five Appendixes: Appendix I lists a glossary of the different terms used throughout the work, in order to ease its understanding. Appendix II provides additional material to the state of the art analysis by means of the underlying review protocol. Then, Appendix III gives an overview of the improvements of the metamodel by means of a change log. Appendix IV includes additional material of the iterative expert judgement process and shows original statements given by the experts as well as the results reports. In conclusion, Appendix V enumerates all publications and research projects. 17 A Framework for Modeling and Improving Agile Requirements Engineering Chapter 2 Related Work he previous chapter presented the context of the problem domain we want to address with this PhD thesis, which refers to the three main fields of Agile, HCD and RE. In this chapter, we will explore the problem domain in detail and perform a deep analysis of the state of the art related to agile RE with focus on user and stakeholder involvement. In section 2.1, foundations concerning the three main fields (Agile, HCD, and RE) will be discussed. Based on these foundations, we will give a definition of agile RE, which is used in this PhD thesis. Section 2.2 presents the related work from other research groups. Then, section 2.3 aims to capture the current state of the art of research related to agile RE with focus on stakeholder and user involvement. Therefore, we carried out a secondary study of existing research by means of conducting a Systematic Literature Review (SLR). In particular, we investigate what approaches exist to involve stakeholder in the process, which methodologies are accepted to present the user perspective and how requirements management is been carried out. Section 2.4 discusses on the implications of the findings and outlines the gaps in existing research. 2.1. Foundations Agile Methodologies, Human-Centered Design and Requirements Engineering In this section, we will present the foundations related to agile methodologies, HCD, and RE. In addition, we will provide a definition for agile RE, which builds the basis for this work and which we elaborate in the following chapters. 2.1.1. Agile Methodologies In the mid-80s, Takeuchi and Nonaka (Takeuchi et al., 1986) already stated that a sequential phases approach to product development is not well suited due to the lack of flexibility. In the 90s, there have been different movements concerning lightweight process models like Scrum (Schwaber, 1997), (Schwaber, 2004), Extreme Programming (XP) (Beck, 2000), or Feature-Driven Development (Palmer and Felsing, 2001). In 2001, the leaders of these different movements came together and tried to find a common ground for their work. As result, the Agile Manifesto (Beck et al., 2001) emerged. The agile manifesto comprises agile values (see Figure 2), which are assisted by 12 principles (see (Beck et al., 2001)). Although the agile manifesto was created in 2001, agile values still guiding agile teams and most of the principles even play an important role in today´s agile community (Williams, 2012), (Schön et al., 2015). T I 18 A Framework for Modeling and Improving Agile Requirements Engineering Figure 2: Agile Values stated by the Manifesto for Agile Software Development (cf. (Beck et al., 2001)) Nowadays, ASD is becoming more popular in all fields of industry due to enabling immediately changes in the direction of product development. Current surveys reveal that Scrum and hybrid models based on Scrum/XP are agile methodologies followed most closely (VersionOne, 2016). Moreover, Kanban (Anderson, 2004), (Anderson, 2010) is playing an important role among the agile community (Komus et al., 2017). 2.1.2. Human-Centered Design During the last decades a lot of different terms evolved describing quality models from a user´s perspective as well as process models to create interactive products addressing these quality models. HCD is defined by ISO 9241-210:2010 (International Organization for Standardization, 2010) and describes an approach to interactive product development where the ergonomics of humansystem interaction plays an important role. ISO 9241-210:2010 offers generic recommendations for every step of the HCD process (see Figure 3). The document explicitly does not recommend specific methods. In order to implement HCD into an organization ISO/DIS 9241-220.2 (International Organization for Standardization, 2017) has been created. ISO/DIS 9241-220.2 provides processes for enabling, executing and assessing human-centered design within organizations. Figure 3: Human-Centred Design ISO 9241-210:2010 (International Organization for Standardization, 2010) 19 A Framework for Modeling and Improving Agile Requirements Engineering The concept of HCD is elaborated on User-Centered Design (UCD) (International Organization for Standardization, 1999) and covers a broader view on human needs. Compared to UCD, HCD emphasizes the impacts on further stakeholders besides the user. The User Experience (UX) is a “person's perceptions and responses resulting from the use and/or anticipated use of a product, system or service” (International Organization for Standardization, 2010). Whereas the Usability is defined as extent “to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”(International Organization for Standardization, 1998). 2.1.3. Requirements Engineering Requirements are the base of all software products and consequently Requirements Engineering (RE) plays and important role in system development. The term Requirements Engineering was already used in the 70-s (Richter et al., 1976). Then, RE gained in popularity with the first conferences in the 90-s (RE´93 and ICRE´94 2 ). Traditional process models, such as the waterfall model (Royce, 1970) consist of sequential phases and start with an upfront design. Figure 4 shows the underlying sequential phases. In the beginning of these projects, all requirements are consolidated into a specification document. Based on this, schedule and budget are estimated. Although Royce (Royce, 1970) stated that there are iterations in the process of development of software, he conveyed that the requirements analysis is completed at some point. In industry, this leads to the assumption that requirements are seen as true and changes are made using a heavyweight change request process while using traditional process models. Figure 4: Sequential phases approach to software development Sommerville and Sawyer (Sommerville and Sawyer, 1997) describe RE as combination of all activities concerning discovering, documenting, and maintaining a set of requirements. They recommend using systematic and repeatable techniques to ensure that product requirements are complete, consistent and relevant. The requirements are consolidated in a specification document, where details are added while product development proceeds. 2 http://requirements-engineering.org 20 A Framework for Modeling and Improving Agile Requirements Engineering 2.1.4. Agile Requirements Engineering The established definition of RE, outlined in section 2.1.3 fails in terms of ASD. Agile methodologies are used to cope with increasing complexity in product development. The former definition of RE and the understanding of it the does not work in an agile environment. Since the agile movement, software development has moved away from plan-driven to valuedriven process models (see Figure 5). People in plan-driven environments often negotiate about pricing models, project plans and how many product features they can develop with the available resources. They are emphasizing the generated outputs (e.g. number of created features or number of releases during a time period). In contrast, people in value-driven environments discuss visions, experiences and human values as well as how they can address them through the product. They concentrate on outcomes, which means that they are focused on the difference that outputs entail. Therefore, product development with agile methodologies is mainly driven by human values. Figure 5: In ASD people focus on outcomes and how they can fulfil human needs through the outputs they produce. In the context of agile methodologies, RE should be carried out iteratively during the whole development process instead of a closed phase in the beginning. Compared to established RE approaches ((Sommerville and Sawyer, 1997), (Pohl, 2010)), a list of prioritized requirements (Product Backlog (Schwaber, 2004)) is used instead of a requirements specification document. In agile RE, requirements are often treated as hypotheses that are continuously validated through frequent stakeholder and user feedback (Olsson and Bosch, 2015). In addition, requirements are regularly described from a user perspective in the form of epics and user stories (Cohn, 2004). The main agile RE activities are not clearly separated activities, since they can take place in parallel. Moreover, the wording of the activities is adapted to an agile environment: discovery, refinement, prioritization, review, and documentation, see (Schön et al., 2017d). The activities are repeated each iteration and only required information is elaborated before the next iteration starts. For this purpose, RE in agile environments is carried out just-in-time with a Little Design Up Front (Adikari et al., 2009). This leads to a kind of ad hoc nature of agile RE. 21 A Framework for Modeling and Improving Agile Requirements Engineering Agile techniques like Continuous Delivery (CD) (Humble and Farley, 2010) have an impact on the manner and the frequency of usability testing nowadays (Larusdottir et al., 2010). New information is given along the product development by the user and the system itself. This knowledge is processed during further steps and it conditions the decision-making process. Therefore, requirements are treated like assumptions, which are validated continuously. Hybrid development models (Schön et al., 2016b), (Pfeiffer et al., 2016) with the integration of HCD, are applied with the aim to deliver competitive products with a suitable UX. Therefore, stakeholder and user involvement during RE are essential in order to establish a collaborative environment with constant feedback loops. Agile RE is a cross-functional research area comprising areas like HCD, ASD, RE (see Figure 6). Contributing to the body of knowledge of agile RE implies considering research from all of the aforementioned areas. Figure 6: Venn diagram showing intersection of research areas influencing agile RE Agile RE activities (see Table 1) are slightly different compared to the activities known from established RE (elicitation, analysis and negotiation, and validation (Sommerville and Sawyer, 1997)). Agile RE is carried out in an iterative manner and the activities are not separated in phases. In comparison to established RE, they can take place in parallel. Table 1: Activities in agile RE Agile RE activity Description Discovery Eliciting new ideas/requirements Refinement Clarifying and analyzing new ideas/requirements Prioritization Measuring the value that the development will add to the system Review Checking if the requirement is implemented in the manner to deliver value Documentation Capturing discussion and decisions around the requirement 22 A Framework for Modeling and Improving Agile Requirements Engineering 2.2. Relevant Work from other Research Groups Besides the state of the art analysis of agile RE, this section presents the relevant work from other research groups in the field, which is related to the work of this PhD thesis. In this section, we want to highlight the most related authors by giving a short overview of their work. The relation between their work and this PhD thesis is additionally outlined in appropriate chapters of this work in the sections Gap analysis of Related Work (see 2.3.2, 4.2, 6.2). 2.2.1. Salah et al. Salah et al. study the integration of UCD and Agile. Hence, Salah (Salah, 2011) contributes a framework for the integration of UCD and ASD. The framework consists of two components. On the one hand, a capability maturity model that measures the maturity of an organization in terms of the integration of UCD and ASD. On the other hand, a framework implementer, who is an individual person responsible for implementing the framework. Salah et al. (Salah et al., 2016) elaborated the former part of the framework and presented a descriptive maturity model for integrating UCD and Agile. The maturity model is evaluated by means of a SLR (Salah et al., 2014) and evaluation forms (expert review). The SLR conducted by Salah et al. (Salah et al., 2014) aims to identify challenges with the integration of UCD and ASD as well as propose practices, which are applied to address these challenges (see section 2.3.1). In addition, Salah et al. (Salah et al., 2015) created patterns, which support the integration of UCD and ASD (see section 6.1). 2.2.2. Silva et al. The research group around Silva et al. is investigating a similar research line like Salah et al. They conducted several SLRs in the field of the integration of HCD and Agile. For instance, Silva et al. (Silva da Silva et al., 2011) contribute a SLR with the aim to analyze the key aspects concerning the integration of UCD and Agile. In this SLR, they identified several artifacts, practices and needs, which are applied to integrate UCD and Agile (see section 2.3.1). Based on this study, Silva et al. (Silva da Silva et al., 2012) propose a framework for integrating UX and agile. The framework was assessed in a field study, where they found some issues regarding the collaboration of UX designer and developers due to organizational conditions. Furthermore, Silva contribute Agile Usability Patterns in collaboration with Bertholdo et al. (Bertholdo et al., 2014). These Agile Usability Patterns are focused on the early stages of HCD (see section 6.1.). Silva (Silva Da Silva et al., 2015) carried out multiple case studies to investigate how usability evaluation is carried out in agile environments. They identified new usability evaluation methods, which emerged with agile (e.g. peer review by pairing). Another systematic mapping study was conducted in collaboration with Garcia, Silva and Selbach Silveira (Garcia et al., 2017). The aim of tis study was to analyze which artifacts are used to facilitate the communication in agile environments, where UCD plays an important role. 23 A Framework for Modeling and Improving Agile Requirements Engineering 2.2.3. Méndez Fernández et al. Méndez Fernández et al. study the field of RE process improvements. This field is very close to the one, which is investigated by this PhD work. However, their work has a slightly different viewpoint since our work is focused on RE in agile environments with strong focus on stakeholder and user involvement. Their viewpoint is more artifact-centered (plan-driven), whereas our viewpoint is more human-centered (value-driven). Méndez Fernández et al. (Méndez Fernández et al., 2010) contribute a metamodel for artifactbased RE, which is inferred by two existing RE models in industry. The metamodel is focused on the topic of handling artifacts in RE. Furthermore, they (Méndez Fernández et al., 2011) conducted a case study with an artefactbased RE approach, which defines two artefact types (business specification and requirements specification) as a domain specific interpretation of RE. Méndez Fernández and Wieringa (Méndez Fernández and Wieringa, 2013) elaborated on the ideas and presented an artefact-based, problem-driven RE improvement approach. The approach starts with an empirical analysis of the improvement problem in terms of understanding the practice of modelling and documentation in a company. Based on this analysis, improvements are defined by means of an artefact model. Subsequent, a RE reference model is defined. In addition, Méndez Fernández et al. (Méndez Fernández et al., 2016) work on the Naming the Pain in Requirements Engineering (NaPiRE) initiative, which study the status quo and problems in practical RE by means of an international series of surveys. The NaPiRE team (Méndez Fernández et al., 2017) consists of more than 50 researchers, who analyze problems in terms of RE in more than 200 companies worldwide by means of survey studies. In this context, they also publish a paper concerning RE problems in agile projects (Wagner et al., 2017). 24 A Framework for Modeling and Improving Agile Requirements Engineering 2.3. State of the Art Agile Requirements Engineering In this section, we will present the current state of the art of research related to agile RE with focus on stakeholder and user involvement. Therefore, we will present the results of our Systematic Literature Review (SLR) (Schön et al., 2017a) and discuss the findings of a revision of the study. 2.3.1. Summary of Related Literature Reviews In the literature, many reviews are conducted in order to do research on ASD. The next paragraphs briefly summarize the most related ones. Silva et al. (Silva da Silva et al., 2011) carried out a SLR on the integration of ASD and HCD and analyzed how usability issues are addressed in agile projects. They included a comprehensive classification process based on a system covering research-related and content-related information. The authors identified the following key aspects, which play an important role for the integration: little upfront design, prototyping, user stories, user testing, inspection evaluation and one sprint ahead. Besides, they presented a process model for the integration of ASD and HCD that took into account their findings. Salah et al. (Salah et al., 2014) addressed a similar area. Their review aimed to identify challenging factors for the integration of ASD and HCD. They presented the challenges in a very understandable manner with good examples. Additionally, they explored practices and success factors to face these challenges. The reported challenges are: lack of allocated time for upfront activities, difficulty of modularization, optimizing the work between developers and HCD practitioners, performing usability testing and lack of documentation. Brhel et al.‘s (Brhel et al., 2015) literature review, examined hybrid development models, such as ASD and HCD. Their main objective was to capture the state of the art of the integration of ASD and HCD. Compared to (Silva da Silva et al., 2011) and (Salah et al., 2014), they addressed a more holistic view of ASD. Thus, they focused on four dimensions (process, people/social, technology and practices) with a coding system and five derived principles: 1) separate product discovery and product creation, 2) iterative and incremental design and development, 3) parallel interwoven creation tracks, 4) continuous stakeholder involvement, and 5) artifact-mediated communication. Furthermore, they contributed to a classification system for existing work in the field of usercentered ASD. Heikkilä et al. (Heikkilä et al., 2015) conducted a mapping study of RE in ASD in order to identify gaps in the scientific knowledge. Their results indicate that the definition of agile RE is vague. Furthermore, they identified the following benefits from agile RE: lower process overheads, improved requirements understanding, a reduced tendency to overallocate development resources, responsiveness to change, rapid delivery of value, and improved customer relationships. The following problems were identified: use of customer representatives, insufficiency of the user story format, prioritization of requirements, growing technical debt, tacit requirements knowledge, and imprecise effort estimation. The main purpose of Inayat et al.’s (Inayat et al., 2015) literature review is to make clear Agile RE challenges and practices, including a good discussion on related work. Moreover, they aimed to understand how traditional RE problems are resolved using Agile RE. In summary, they provided 17 commonly used practices and also practical challenges that agile teams had to face. The practices are: Face-to-face communication, customer involvement and interaction, user stories, iterative 25 A Framework for Modeling and Improving Agile Requirements Engineering requirements, requirement prioritization, change management, cross-functional teams, prototyping, testing before coding, requirements modeling, requirements management, review meetings and acceptance tests, code refactoring, shared conceptualization, pairing for requirements analysis, retrospectives and continuous planning. Soares et al. (Soares et al., 2015) combined a literature review with an exploratory study. They analyzed difficulties while working with requirements in an agile environment, particularly, causes that can lead to documentation debt (e.g. missing, inadequate or incomplete requirements). They contribute with their work to an important research topic, for documentation in ASD is often treated in an inadequate manner. The authors defined 10 difficulties that occur when identifying and managing agile requirements: requirement prioritization, non-functional requirements identification, lack of information, volatility of requirements, requirements definition, dependence between requirements, to predict impacts of changes, user dependence, communication and collaboration with users, and requirements validation. Furthermore, they uncovered difficulties when using user stories instead of use cases. 2.3.2. Gap Analysis in Related Literature Reviews To sum up, it can be said that related literature reviews cover many aspects of Agile RE. Nevertheless, analyzing the existing work, we observed some shortcomings. Silva et al. (Silva da Silva et al., 2011) and Salah et al. (Salah et al., 2014) worked on the integration of HCD and ASD. In this context, they studied the collaboration between HCD specialists and developers but they did not pay the same attention to stakeholder and user involvement. The other published reviews (Brhel et al., 2015), (Heikkilä et al., 2015), (Inayat et al., 2015) and (Soares et al., 2015) identified stakeholder and user involvement as one of the key aspects in ASD, although they only presented partially how this problem might be solved. As they treated this problem as one out of many, we consider that they only scratch the surface. Figure 7: Related work has not investigated Agile RE from the perspective of HCD up to now Since human beings and their values play one of the most important roles in value-driven organizations (see Figure 5), it is necessary to further investigate this aspect in the agile RE field. To this end, we conducted this SLR. However, to the best of our knowledge, no systematic review has 32 A Framework for Modeling and Improving Agile Requirements Engineering Table 5: Distribution according to research methods Research Method Paper total Percentage Case Study 15 56% Case Study in Laboratory 2 7% Multi-Case Study 2 7% Description of Approach 5 19% Experiment 1 4% Research Perspective 1 4% Semi-Structured Interviews 1 4% We used the quality checklist presented in Table 4 in order to evaluate each study. The overall results from the quality assessment are shown in Figure 10. Figure 10: Quality assessment The first criterion (QA1) examines whether the proposal is validated. For 15 papers this is true, as they used case studies in order to validate their proposal. Nine papers either validated their proposals in a laboratory or assessed only parts of the proposal. We also included three papers whose proposals were not validated at all. With QA2 we confirmed whether the study presented a detailed description of the approach. In 17 papers the approach is described with enough details, so that other researchers could use it. In comparison, when the approach of seven papers should be used again, included references have to be read. Two papers missed details. QA3 queried whether the study provided a personal opinion or viewpoint. 20 out of the 27 papers were based on a clearly defined research design. For 6 papers, the related work was explained and the paper was set into context. Nonetheless, there was also one study that did not clearly describe the research method. QA4 wondered how many times the study had been cited in other papers. For this purpose, we used the number of citation from Google scholar (Assessment date 2015-11-20). 15 studies had more than five citations in other papers. Six papers had been cited in among 1-5 articles and six papers had no citation until the assessment date. With the last criterion (QA5) we tested if the 33 A Framework for Modeling and Improving Agile Requirements Engineering aims of the study were included in the paper. In 26 of the works, the aims were well described and clear. Only one paper lacked presenting the aims, since they were described very unclearly. To sum up, six papers fulfiled every quality criterion ((Cajander et al., 2013), (Losada et al., 2013), (Abdullah et al., 2011), (Kautz, 2010), (Ramesh et al., 2010), (Obendorf and Finck, 2008)). We have to be aware that the results might be different at publication date of the results, due to the different number of citations at assessment date that QA4 required. 2.3.4.2. (RQ 1.1) What Approaches Exist, Which Involve Stakeholders in the Process of RE and Are Compatible With ASD? Table 6 presents the results from the evaluation of the sub-criteria related to RQ1. Therefore, we list the studies that clearly have a positive answer, “Yes”. Additionally, the distribution according the items is also shown. Table 6: Sub-criteria RQ1 ID Sub-criteria Covered by study Overall C1.1 Stakeholders are involved directly. (Bellucci et al., 2015), (Harbers et al., 2015), (Olsson and Bosch, 2015), (Dragicevic et al., 2014), (Liskin et al., 2014), (Rivero et al., 2014), (Wanderley et al., 2014), (Cajander et al., 2013), (Losada et al., 2013), (Kamthan, 2013), (Maguire, 2013), (Abdullah et al., 2011), (Näkki et al., 2011), (Mahmud and Veneziano, 2011), (Bourimi et al., 2010), (Kautz, 2010), (Lee et al., 2010), (Lucia and Qusef, 2010), (Ramesh et al., 2010), (Obendorf and Finck, 2008), (Memmel et al., 2007) N = 27 Yes = 77% Part = 19% No = 4% C1.2 The user is involved directly. (Bellucci et al., 2015), (Blomkvist et al., 2015), (Olsson and Bosch, 2015), (Rivero et al., 2014), (Wanderley et al., 2014), (Cajander et al., 2013), (Kamthan, 2013), (Losada et al., 2013), (Maguire, 2013), (Näkki et al., 2011), (Mahmud and Veneziano, 2011), (Kautz, 2010), (Ramesh et al., 2010), (Obendorf and Finck, 2008), (Memmel et al., 2007) N = 27 Yes = 56% Part = 33% No = 11% C1.3 They use a process in order to involve stakeholders. (Bellucci et al., 2015), (Harbers et al., 2015), (Olsson and Bosch, 2015), (Buchan, 2014), (Dragicevic et al., 2014), (Liskin et al., 2014), (Nawrocki et al., 2014), (Rivero et al., 2014), (Wanderley et al., 2014), (Cajander et al., 2013), (Kamthan, 2013), (Losada et al., 2013), (Maguire, 2013), (Issa and AlAli, 2011), (Näkki et al., 2011), (Mahmud and Veneziano, 2011), (Bourimi et al., 2010), (Kautz, 2010), (Ramesh et al., 2010), (Obendorf and Finck, 2008), (Memmel et al., 2007) N = 27 Yes = 78% Part = 19% No = 4% C1.4 There are iterations during the development process. (Bellucci et al., 2015), (Blomkvist et al., 2015), (Olsson and Bosch, 2015), (Buchan, 2014), (Dragicevic et al., 2014), (Liskin et al., 2014), (Nawrocki et al., 2014), (Rivero et al., 2014), (Wanderley et al., 2014), (Cajander et al., 2013), (Losada et al., 2013), (Kamthan, 2013), (Maguire, 2013), (Näkki et al., 2011), (Bourimi et al., 2010), (Kautz, 2010), (Ramesh et al., 2010), (Obendorf and Finck, 2008), (Memmel et al., 2007) N = 27 Yes = 70% Part = 30% No = 0% C1.5 They use methods in order to gather data. (Bellucci et al., 2015), (Blomkvist et al., 2015), (Harbers et al., 2015), (Olsson and Bosch, 2015), (Buchan, 2014), (Dragicevic et al., 2014), (Nawrocki et al., 2014), (Adikari et al., 2013), (Cajander et al., 2013), (Maguire, 2013), (Näkki et al., 2011), N = 27 Yes = 70% Part = 30% No = 0% 34 A Framework for Modeling and Improving Agile Requirements Engineering (Issa and AlAli, 2011), (Mahmud and Veneziano, 2011), (Kautz, 2010), (Lee et al., 2010), (Lucia and Qusef, 2010), (Ramesh et al., 2010), (Obendorf and Finck, 2008), (Memmel et al., 2007) Subsequent to the quantitative analysis presented in Table 6, the following paragraphs will put forward the highlights we found in the included studies concerning sub-criteria by means of a qualitative analysis. Stakeholder and User Involvement Bellucci et al. (Bellucci et al., 2015) combined XP with co-design sessions in order to develop a product with strong user involvement. They explored in a field study, how users interact and work with a prototype. Based on their findings, the prototype was developed iteratively. The authors considered this approach as a tactic to quickly deploy an evolving prototype. Harbers et al. (Harbers et al., 2015) studied the application of a Value Story workshop for the elicitation process of user stories. The aim of this workshop is to embed stakeholder values into the RE process. Therefore, direct and indirect stakeholders have to be identified. In a second step, the values of each stakeholder group are revealed. Then, a situation for each value is provided and stakeholder needs in this situation are analyzed. Olsson et al. (Olsson and Bosch, 2015) derived a conceptual model from multiple case studies that emphasized the need for combining qualitative feedback in early stages of development with quantitative customer observation in later stages of development. In the Qualitative/Quantitative Customer-driven Development (QCD) model requirements are treated as hypotheses that are validated through continuous customer feedback. Several studies provided additional roles, which should be applied to an agile environment in order to address communication gap or take over responsibility for particular tasks. Dragicevic et al. (Dragicevic et al., 2014) claimed that business users could collect necessary data and documents (e.g. templates, scans of documents or screens) and provide information to developers who model those data in UML. Moreover, Lee et al. (Lee et al., 2010) stressed the role of an Agile-HCD specialist (AUS), which has to be responsible for bridging the communication gap between developer and UX designer. They also present a usability-pattern-based requirement-analysis method that help AUSs to do their tasks of requirements specification. Kautz (Kautz, 2010) wondered how customers and users participate in ASD with Participatory Design. He reports that customer and users were involved indirectly and directly through different kind of activities. Additionally, the role of an onsite customer was applied and they had weekly feedback loops. Kautz states that one of the important benefits of the frequent feedback loops is that misunderstandings are detected in time and changes can be applied early before they grow into larger problems. For this reason, the user generates a feeling of trust that has impact on the development. Collaboration and shared understanding are essential to ASD (Beck et al., 2001). To this end, and attending to the results of a Multi-Case Study, Ramesh et al. (Ramesh et al., 2010) described informal and frequent communication as the core of agile RE. They stated that customers were directly involved in each iteration. Requirements were elicited, refined and validated through face- 35 A Framework for Modeling and Improving Agile Requirements Engineering to-face communication with the customer. They claim requirements analysis as a social–political process that depends on human interaction and is influenced by several contextual factors. Data Gathering in Agile RE The results of sub-criteria C1.5 (Table 6) revealed that 70% of the studies reviewed used methods in order to gather data. In addition to traditional methods known from agile methodologies (e.g. planning or reviews) reported by (Buchan, 2014) and (Ramesh et al., 2010), the reviewed studies included further methods for data gathering within Agile RE. For instance, we found representative examples in the following studies: Bellucci et al. (Bellucci et al., 2015) applied weekly co-design sessions as a meeting point between users and designers. During these sessions, researchers collected impression and feedback concerning users’ experience with the system and usage scenarios. They also gathered information about nonusage and misusing of implemented features. Similarly, Kautz (Kautz, 2010) studied Participatory Design activities within ASD. He reported that data gathering with customer and user involvement took place on an ongoing basis. Communication was structured through planning games, presentation of working software and acceptance tests. Lucia et al. (Lucia and Qusef, 2010) present an overview regarding agile RE. Interviews, Brainstorming, Ethnography and Use Case analysis are the most important elicitation techniques from their point of view. Näkki et al. (Näkki et al., 2011) describe users as a source of ideas and looked at them as decision makers throughout the design process. For this purpose, they enabled Lead users to participate via online co-creation tools with social media mechanisms. Considering that analyzing the context of use is an important activity to achieve a humancentered process, several studies recommend using a structured process for this activity. They describe how to carry out a Contextual Inquiry (Beyer and Holtzblatt, 1998) as an appropriate manner (Obendorf and Finck, 2008), (Maguire, 2013). With regard to Human-Centered Design, Maguire (Maguire, 2013) propose different methods following activities from ISO 9241-210:2010. In addition to a Contextual Inquiry, he suggests to accomplish a stakeholder analysis to explore the context of use. Furthermore, he proposes surveys, interviews, discussions, focus groups, competitor analysis and user journeys for specifying user requirements. Maguire recommends gathering data with conceptual design meetings and co-design workshops during the creation of design solutions. For the evaluation, he highlights the use of user walkthroughs and usability tests. 2.3.4.3. (RQ 1.2) Which Agile Methodologies, Which Are Capable of Presenting the User Perspective to Stakeholders, can be Found? Table 7 shows the results from the evaluation of the sub-criteria related to RQ2. Therefore, we list the studies that clearly answered the questions positively. Moreover, we display the distribution in terms of the items. 36 A Framework for Modeling and Improving Agile Requirements Engineering Table 7: Sub-criteria RQ2 ID Sub-criteria Covered by study Overall C2.1 The proposal used a methodology. (Bellucci et al., 2015), (Harbers et al., 2015), (Olsson and Bosch, 2015), (Buchan, 2014), (Dragicevic et al., 2014), (Nawrocki et al., 2014), (Rivero et al., 2014), (Wanderley et al., 2014), (Adikari et al., 2013), (Kamthan, 2013), (Losada et al., 2013), (Maguire, 2013), (Farid, 2012), (Issa and AlAli, 2011), (Näkki et al., 2011), (Bourimi et al., 2010), (Kautz, 2010), (Lee et al., 2010), (Obendorf and Finck, 2008), (Memmel et al., 2007) N = 25 Yes = 80 % Part = 16% No = 4% C2.2 The knowledge about the user requirements is shared among the stakeholders. (Bellucci et al., 2015), (Olsson and Bosch, 2015), (Buchan, 2014), (Liskin et al., 2014), (Rivero et al., 2014), (Cajander et al., 2013), (Maguire, 2013), (Kamthan, 2013), (Abdullah et al., 2011), (Bourimi et al., 2010), (Kautz, 2010), (Lee et al., 2010), (Lucia and Qusef, 2010), (Memmel et al., 2007) N = 27 Yes = 52% Part = 48% No = 0% Subsequent to the quantitative analysis presented in Table 7, the following sub-sections will present the highlights we found in the included studies concerning sub-criteria by means of a qualitative analysis. User Perspective in ASD Cajander et al. (Cajander et al., 2013) interviewed 21 IT professionals in order to analyze the user perspective in ASD. On one hand, they found that the responsibility for the user perspective is often unclear and on the other hand, they discover that in some cases the user perspective is neither discussed nor described. Furthermore, Cajander et al. state that ad hoc nature of user involvement and design feedback exist in agile projects. Moreover, they conclude that new usability methods arose because of the agile requirements (e.g. speed, efficiency or focus on deliverables instead of documentation). Methodologies in Agile RE Several studies deal with methodologies that are used to extend agile methodologies like scrum and XP, with the aim to better understand the user perspective. Human-Centered Design (HCD): Maguire (Maguire, 2013) extended the HCD framework (International Organization for Standardization, 2010) for agile development. There are four main activities in HCD, which are performed iteratively: a) understand and specify the context of use, b) specify user requirements, c) produce design solutions and d) evaluate designs against requirements. Maguire suggests different methods in order to perform each activity. Additionally, he recommends producing clearly defined artifacts based on the gathered information. Design Thinking: A study by Adikari et al. (Adikari et al., 2013) propose a framework based on three methodologies: Design Thinking, UX design and ASD. A real world system context was being explored with other relevant systems using Design Thinking. As a result, a reframed context was build. They report that the knowledge of the reframed contexts could be used to create products, systems or services using UX design and ASD. 37 A Framework for Modeling and Improving Agile Requirements Engineering Contextual Inquiry: Several studies (e.g. (Obendorf and Finck, 2008) and (Maguire, 2013)) state that performing a Contextual Inquiry (Beyer and Holtzblatt, 1998) is useful to explore the user perspective and to gather data concerning both, users and context of use. Participatory Design: Kautz (Kautz, 2010) studied how customers and users participated in ASD with Participatory Design. Kautz focused on the role of the customers and users and how they were involved through different activities in design and development. Similarly, Bellucci et al. (Bellucci et al., 2015) investigated an approach to design with and by the user. For this reason, they carried out a field study to evaluate how users interacted with a prototype. Based on their findings, the prototype was developed iteratively through constant feedback loops. In addition, Näkki et al. (Näkki et al., 2011) worked on an application of a lead-user approach in the context of ASD. The chosen lead users participated actively in the innovation process through idea generation as well as in all phases of the development process via online co-creation tools. With regard to Participatory Design, Olsson et al. (Olsson and Bosch, 2015) developed the QCD model based on a multi-case study. They confirm that it is important to combine qualitative and quantitative feedback techniques in order to achieve continuous customer validation. The authors treat requirements as hypotheses that are validated with customers before they are taken into account for development. Shared Understanding Abdullah et al. (Abdullah et al., 2011) cope with communication patterns in an agile team and in particular, how communication and collaboration supported RE activities (gathering, clarifying and evolving) in an agile environments. They built the concept of shared conceptualization, which mean that the development team shares a common understanding of the requirements, which is deeper than shared understanding. They describe that there is a link between communication and memory. Little information about a requirement was documented on a story card, but members of the agile team built a related concept in their minds that was based on discussion concerning the requirement. A study by Buchan (Buchan, 2014) also provides insights into the concept of shared understanding. Buchan developed a theory of shared understanding of requirements (SUR). He states that SUR is a specialized form of a Team Mental Model with focus on RE. Furthermore, he defines two activities in SUR development: 1) uncovering a gap collaboratively, and 2) addressing this gap to achieve a new state of SUR. The goal is to enable team members achieve a consistent understanding of the requirements. 2.3.4.4. (RQ 1.3) What are the Common Ways for Requirements Management in ASD? Table 8 presents the results from the evaluation of sub-criteria related to RQ3 as well as it lists the studies that clearly answered the questions positively. In addition, it also shows the distribution according the items. 38 A Framework for Modeling and Improving Agile Requirements Engineering Table 8: Sub-criteria RQ3 ID Sub-criteria Covered by study Overall C3.1 They are using artifacts. (Bellucci et al., 2015), (Blomkvist et al., 2015), (Harbers et al., 2015), (Olsson and Bosch, 2015), (Buchan, 2014), (Dragicevic et al., 2014), (Liskin et al., 2014), (Nawrocki et al., 2014) (Wanderley et al., 2014), (Rivero et al., 2014), (Cajander et al., 2013), (Losada et al., 2013), (Maguire, 2013), (Abdullah et al., 2011), (Issa and AlAli, 2011), (Mahmud and Veneziano, 2011), (Näkki et al., 2011), (Bourimi et al., 2010), (Lee et al., 2010), (Lucia and Qusef, 2010), (Ramesh et al., 2010), (Kautz, 2010), Farid, (Obendorf and Finck, 2008), (Memmel et al., 2007) N = 27 Yes = 93% Part = 7% No = 0% C3.2 The documentation is understandable without further knowledge. (Bellucci et al., 2015), (Blomkvist et al., 2015), (Harbers et al., 2015), (Olsson and Bosch, 2015), (Buchan, 2014), (Liskin et al., 2014), (Rivero et al., 2014), (Cajander et al., 2013), (Maguire, 2013), (Abdullah et al., 2011), (Issa and AlAli, 2011), (Näkki et al., 2011), (Lucia and Qusef, 2010), (Ramesh et al., 2010), (Kautz, 2010), (Obendorf and Finck, 2008), (Memmel et al., 2007) N = 27 Yes = 63% Part = 33% No = 4% C3.3 They distinguish between functional and non-functional requirements. (Harbers et al., 2015), (Dragicevic et al., 2014), (Nawrocki et al., 2014), (Rivero et al., 2014), (Cajander et al., 2013), (Losada et al., 2013), (Maguire, 2013), (Farid, 2012), (Issa and AlAli, 2011), (Bourimi et al., 2010), (Lucia and Qusef, 2010), (Ramesh et al., 2010), (Memmel et al., 2007) N = 26 Yes = 50% Part = 42% No = 8% Subsequent to the quantitative analysis presented in Table 8, the following sub-sections will present the highlights we found in the included studies concerning sub-criteria by means of a qualitative analysis. Artifacts in Agile RE Artifacts are used for communication, elaboration, validation and documentation of requirements in agile environment. In sum, we identified 57 different artifacts mentioned in the included studies. 17 out of 57 are mentioned more than twice (see Table 9). Table 9: Artifacts in Agile RE Artifact Description Reference Perc. User Story User story is a description of a feature written from the perspective of the person who needs this. It consists of a written text, conversation about it and acceptance criteria. (Blomkvist et al., 2015), (Harbers et al., 2015), (Buchan, 2014), (Liskin et al., 2014), (Nawrocki et al., 2014), (Rivero et al., 2014), (Wanderley et al., 2014), (Kamthan, 2013), (Maguire, 2013), (Farid, 2012), (Näkki et al., 2011), (Kautz, 2010), (Lee et al., 2010), (Ramesh et al., 2010), (Obendorf and Finck, 2008) N=27 56% 39 A Framework for Modeling and Improving Agile Requirements Engineering Prototype Prototype is a software application model that supports the evaluation of design alternatives and communication. (Bellucci et al., 2015), (Blomkvist et al., 2015), (Nawrocki et al., 2014), (Rivero et al., 2014), (Maguire, 2013), (Kamthan, 2013), (Losada et al., 2013), (Näkki et al., 2011), (Bourimi et al., 2010), (Lucia and Qusef, 2010), (Ramesh et al., 2010), (Obendorf and Finck, 2008), (Memmel et al., 2007) N=27 41% Use Case Use case describes an action or event steps, which are needed to achieve a goal. (Harbers et al., 2015), (Nawrocki et al., 2014), (Losada et al., 2013), (Farid, 2012), (Issa and AlAli, 2011), (Lucia and Qusef, 2010), (Memmel et al., 2007) N=27 26% Scenario Scenario is a textual representation of a problem and describes the interaction between user and system in a specific context. (Blomkvist et al., 2015), (Harbers et al., 2015), (Nawrocki et al., 2014), (Maguire, 2013), (Obendorf and Finck, 2008), (Memmel et al., 2007) N=27 22% Story Card Story Card is a physical representation for the written text and shows details from a user story. (Buchan, 2014), (Farid, 2012), (Abdullah et al., 2011), (Kautz, 2010), (Ramesh et al., 2010) N=27 22% Persona Persona describes a fictitious person that represents a larger part of the target group. (Blomkvist et al., 2015), (Liskin et al., 2014), (Kamthan, 2013), (Maguire, 2013) N=27 15% Vision Vision is an abstract description of the overarching goal that guides product development and aligns development, business people and other stakeholders. (Buchan, 2014), (Liskin et al., 2014), (Memmel et al., 2007) N=27 11% UML diagram Unified Modeling Language (UML) provides a standard to visualize the design of a system. (Dragicevic et al., 2014), (Bourimi et al., 2010), (Lee et al., 2010) N=27 11% Storyboard Storyboard presents the workflow of the user in a sequence of pictures. (Näkki et al., 2011), (Lucia and Qusef, 2010), (Obendorf and Finck, 2008) N=27 11% Task Each of the parts into which one user story is split. It describes more technical requirements. (Liskin et al., 2014), (Obendorf and Finck, 2008) N=27 7% Kanban board Kanban board visualizes the progress of a requirement through the workflow of the development team. (Buchan, 2014), (Liskin et al., 2014) N=27 7% UI pattern UI pattern describes an abstract solution for recurring design problems and give inspiration to designer. (Lee et al., 2010), (Memmel et al., 2007) N=27 7% Essential use case Essential use case describes user tasks and it represents a simplified and generalized form of use cases. (Lee et al., 2010), (Memmel et al., 2007) N=27 7% 40 A Framework for Modeling and Improving Agile Requirements Engineering Picture Picture is a visual representation (e.g. photograph or painting) (Bellucci et al., 2015), (Näkki et al., 2011) N=27 7% Video Video consists of a sequence of images processed electronically that are seen in a recording and are displayed on a screen. (Bellucci et al., 2015), (Näkki et al., 2011) N=27 7% Mind Map Mind map is a diagram used to visualize and organize information. (Wanderley et al., 2014), (Mahmud and Veneziano, 2011) N=27 7% UI specification Written specification that describes the UI of a system. The text is enriched by mock-ups or icons, among other elements. (Blomkvist et al., 2015), (Maguire, 2013) N=27 7% At this point, we would like to highlight some key artifacts (usage <20%). User stories are the most frequent used artifact in ASD. The included studies describe how they can be created and represented. Näkki et al. (Näkki et al., 2011) use the concept of needsbased user stories. Therefore, they collected users’ everyday needs and challenges regarding a specific domain. Users were involved during requirements elaboration, by commenting and rating features in order to allow the prioritization of features. Harbers et al. (Harbers et al., 2015) suggest using a Value Story workshop to embed stakeholder values in the elicitation process of requirements. The requirements resulting from this workshop are collected as value-based user stories. In addition to the classical format of user stories (Cohn, 2004), Wanderley et al. (Wanderley et al., 2014) present a visual language for user stories. The visual representation of the user story supports the evaluation of the requirements with users and can be utilized with a User Story Visual Editor. Prototypes are categorized by the studies on different types of fidelity (low, mid and high fidelity). Besides, the authors use the terms prototypes, mock-ups and wireframes. Lucia et al. (Lucia and Qusef, 2010) recommend using paper prototypes to document requirements with the purpose of communication and knowledge sharing between stakeholders and agile teams. In addition, Obendorf et al. (Obendorf and Finck, 2008) state that paper prototypes and informal drawings are very useful in discussions with users. Informal drawings (sketches) were also used in Blomkvist et al. (Blomkvist et al., 2015). In comparison, Rivero et al. (Rivero et al., 2014) applied html mockups to start the modeling process in Model-Driven Web Engineering. HTML-based mockups were used, on one hand, as a foundation to specify features like content, navigation and business logic and, on the other hand, to generate platform-independent UI specifications. Furthermore, Nawrocki et al. (Nawrocki et al., 2014) propose to use mockups to elicit test cases from users to make use cases testable. Use Cases (UC) are often used to describe the behavior of a system from a more technical viewpoint compared to user stories. Issa et al. (Issa and AlAli, 2011) built a UC patterns catalogue that could be used as a feature checklist and to design an initial version of a UC model. Their UC meta-model addresses the environmental, technical, structural, eventual and traceability dimensions of the anticipated system. Besides, Nawrocki et al. (Nawrocki et al., 2014) argued that UCs could be reused for user manual generation, and for effort estimation. Farid (Farid, 2012) divides requirements into functional and non-functional. Functional requirements are described as 41 A Framework for Modeling and Improving Agile Requirements Engineering Agile Use Case, whereas non-functional requirements are presented as Agile Loose Case. Aspect- Oriented “pointcut” operators link functional to non-functional requirements. Scenarios describe how users interact with a system in a specific context. To this end, they are often combined with personas (Maguire, 2013), (Blomkvist et al., 2015). In Bellucci et al. (Bellucci et al., 2015), designers used a journal in order to document scenarios based on user insights, gathered with a user diary and usage try-outs. Moreover, Obendorf et al. (Obendorf and Finck, 2008) used scenarios to connect a design vision with the more technical tasks of programmers in a Scenario-based usability engineering approach. Story Cards present additional information related to user stories. Abdullah et al. (Abdullah et al., 2011) report that story cards allow capturing plans (estimates), history (who worked on the card) and goals. Besides, Farid (Farid, 2012) defined exactly which questions had to be answered with a story card in the W8 User Story Card Model. The eight “W” are: who, what, why, without ignoring, while it is nice to have, within, with a priority of and which may impact. On the contrary, Ramesh et al. (Ramesh et al., 2010) used story cards to document requirements for the next iteration that were elicited through communications with the customer. Artifacts that are mentioned one time: Wall, pin board, Event-driven Process Chain (EPC) models, business process repository, domain models, snapshots, tags, SUI model, index cards, data flow diagrams, user wish list, user journey, UX requirements, design concept, evaluation goals, test specifications, role model, task model, operational model, interaction scenarios, user performance, experience goals, hedonic quality goals, document with Functional Requirements (FR) and Non- Functional Requirements (NFR), realization concept, refined into requirements lists, working software, system model, user model, effect maps, sketches, Product Backlog, user-diaries with picture and videos, sprint goal, delivery roadmap, definition of “done” and sprint burn-down chart. Concerning classical agile artifacts (e.g. Product Backlog, Sprint Goal and Sprint Backlog) we can point out that the included studies rarely mentioned them. This leads to the conclusion that not every used artifact is reported by the studies. Documentation of Requirements The results of the literature review show that there are also some problems regarding documentation of requirements. Blomkvist et al. (Blomkvist et al., 2015) identified that not every artifact was used by developers. They report that developers did not read personas, scenarios and effect maps. They describe this phenomenon as TAGRI principle (They Ain’t Gonna Read It). For this purpose, it is important to find the right combination of artifacts that fit the context of the project and people working in it. Furthermore, Blomkvist et al. (Blomkvist et al., 2015) recommend that HCD specialists should translate their work directly into user stories, otherwise user stories have a strong technical focus. They state that user stories provide a way to translate HCD work into a format that had been already used in ASD. In contrast, Cajander et al. (Cajander et al., 2013) report that they are not well suited to usability work as there are difficulties to describe usability aspects in such a way. To their mind, usability needs to be addressed on a higher level. We also find another aspect concerning the treatment of user stories. Liskin et al. (Liskin et al., 2014) studied the granularity of user stories, specifically the level of functionality an artifact deals with. They verified that there were communication and planning issues for big stories (implementation < 1 week) but they were too vague, thus, they recommend splitting such stories. 48 A Framework for Modeling and Improving Agile Requirements Engineering sub-criteria. Moreover, we do not find groundbreaking new work, which would change the outlined definition of the start of the art of agile RE with focus on stakeholder and user involvement. 2.4. Implications and Gap Analysis The results of the state of the art analysis have several implications for both researchers and practitioners. Based on a qualitative analysis of the included studies, we can conclude that building a shared understanding of the user perspective is not very well established in ASD. It became obvious during the deeper analysis of the identified publications, that only a limited number of papers investigated the presence of the user perspective in ASD. These publications revealed that there were many problems concerning the direct involvement of users and stakeholders. However, we identified four methodologies (Human-Centered Design, Design Thinking, Contextual Inquiry and Participatory Design) that were integrated in ASD with the aim to increase the knowledge about user needs. Furthermore, we identified a broad range of different methods that can be used in ASD to gather data in terms of RE. We identified the following key artifacts for the documentation of requirements that are used in Agile RE: User stories, prototypes, use cases, scenarios and story cards. Industrial practitioners can utilize these findings as recommendations to discover the right combination of artifacts for their development process. With regards to NFRs, we can conclude that on one hand, there are different approaches to deal with NFRs, but on the other hand, we determine an open challenge concerning the estimation and measurement of these requirements (e.g. UX metrics, security policies). General findings. The results of the SLR show that Agile RE has been studied within various research areas (e.g. software engineering, human factors or Participatory Design). We can conclude that this is an important research topic. Furthermore, the heterogeneity of the studied aspects in the reviewed studies shows that this is a complex research field with a lot of different crossfunctional influences. Moreover, we can state that this research field is very close to existing work practices in companies, since most of the included studies analyze the aspects in a real life context. Findings related to RQ1.1. Concerning our first RQ, we can conclude that continuous communication and collaboration is the most frequent used approach to involve stakeholder in the process of RE compatible with ASD. The variety of aspects and methods reported by the reviewed studies point out that there is no common process model for stakeholder involvement in an agile environment. Although a broad range of reviewed studies dealt with the development of systems including a user interface, only half of them involved users directly into the development process. However, it is necessary to engage user in order to validate assumptions during system development; otherwise requirements would be seen as a single point of truth. In light of this, we consider that further research with regard to a structured process model for stakeholder and user involvement is required. Findings related to RQ1.2. Studies that address the user perspective in ASD provide some insights about which methodologies are useful in order to make ASD more user-centric. We find that Human-Centered Design, Design Thinking, Contextual Inquiry and Participatory Design are accepted methodologies. However, the results also show that there are problems with sharing knowledge between stakeholder concerning user requirements and the responsibility for usability/UX. To this end, we can conclude that it is a key to find appropriate methodologies that help to solve these problems. 49 A Framework for Modeling and Improving Agile Requirements Engineering Findings related to RQ1.3. The analysis of the reviewed studies shows that a variety of different artifacts are applied to Agile RE. We have identified user stories, prototypes, use cases, scenarios and story cards as the most frequent used artifacts. Furthermore, we have faced some problems regarding requirements documentation; there are difficulties to identify the right kind of artifacts, which enhance collaboration among stakeholder, developer and agile team. This is a special challenge with regard to the project setting (e.g. co-located or distributed teams). Nevertheless, it is important to create appropriate guidelines for requirements management within ASD. 2.5. Chapter Summary This chapter presented the foundations related to agile methodologies, HCD, and RE, which build the basis for this work. Moreover, we outlined a definition for agile RE, which will be elaborated in the following chapters. Besides, we depict the relation to the work of relevant research groups in the field. Moreover, the most relevant related work regarding the relation among Agile, HCD, and RE has been presented. To this end, we analyzed the state of the art of agile RE with focus on user and stakeholder involvement by means of a SLR. The state of the art study helped us to achieve deep insights into the field of agile RE, in particular in terms of: • presence of the user perspective • stakeholder and user involvement • integration of methodologies to make ASD more human-centered • building a shared understanding • data gathering • non-functional requirements • documentation of requirements by means of artifacts In addition, we identified gaps in existing research related to the iterative involvement of user and stakeholder in the development process, as well as building a shared understanding of the user perspective. We found a variety of methods and artifacts, which are used in ASD for different purposes in a specific context. Guidelines for choosing the right kind of artifacts that enhance the collaboration among all people involved in the development process are missing. Based on the knowledge we gained by analyzing the related work and background in the field of agile RE, we are enabled to define the challenges and objectives of this PhD thesis in the following chapter 3. 50 A Framework for Modeling and Improving Agile Requirements Engineering Chapter 3 Challenges and Objectives n an agile environment Requirements Engineering (RE) is carried out iteratively alongside the whole product development instead of a closed phase in the beginning. To this end, a just-in- time model is often used to refine high level requirements into low level tasks that can be implemented by developers. Therefore, business people, stakeholders, users and developers work together in a collaborative manner. Chapter 2 shows that there is a variety of best practices for agile RE, which are tailored to the context of the environment and often used implicitly by the people working with it. In this chapter, we outline the challenges this PhD thesis pursues (see section 3.1) together with a detailed description of the objectives we want to achieve (see section 3.2). Therefore, we will provide an overall vision expressed by a vision statement. The vision will be split into several objectives and we will describe how we want to achieve each objective by providing an overview of the research methods, which will be used for this work. 3.1. Challenges This section will present in detail the main challenges of this PhD thesis, based on the results of the state of the art analysis, with the aim to explain the research gaps and to justify the need of this work. Traditional process models, such as the waterfall model (Royce, 1970), are plan driven. They consist of sequential phases and start with a big upfront design. In the beginning of these projects, all requirements are consolidated into a specification document. Based on this, schedule and budget are estimated. Requirements are seen as true and changes are made using a heavyweight change request process. In the mid-80s, Takeuchi et al. (Takeuchi et al., 1986) already stated that a sequential phases approach to product development is not well suited due to the lack of flexibility. Since then, new process models have been developed. On one hand, there are iterative process models like Rational Unified Process (Kruchten, 1998), (Kruchten, 2004). On the other hand, there are agile methodologies such as Scrum (Schwaber, 1997), (Schwaber, 2004), Extreme Programming (XP) (Beck, 2000), Feature-Driven Development (Palmer and Felsing, 2001) and Kanban (Anderson, 2004), (Anderson, 2010). Established RE approaches (Sommerville and Sawyer, 1997), (Pohl, 2010) are created in terms of the requirements of sequential approaches to product development. The conditions ASD comes with have an impact on the way RE is carried out in agile environments. By comparison, RE in agile environments is carried out iteratively during the whole development process instead of a closed phase in the beginning. New information is given along the product development by the user and the product itself. This knowledge is processed during further steps and it conditions the decision-making process. Therefore, requirements are treated like assumptions, which are validated continuously. Agile RE is a cross-functional research area comprising areas like Human-Centered Design (HCD), Agile Software Development (ASD), Requirements Engineering (RE) (see Figure 6). Contributing to the body of knowledge of agile RE implies considering research from all of the aforementioned areas and therefore increases the complexity. This complexity needs to be reduced in order to build a strong framework for agile RE. I 51 A Framework for Modeling and Improving Agile Requirements Engineering Organizations facing the challenge to create a value-driven environment, where agile values build the foundation and value delivery has the highest priority. Therefore, people need to focus on outcomes and how they can fulfil human needs through the outputs they produce. The results of the state of the art analysis reveal that building a shared understanding of the user perspective is not very well established in ASD. Furthermore, we identified problems with sharing knowledge among stakeholders and agile teams concerning the user requirements. In light of this, we can conclude that organizations struggle with prioritizing the most valuable requirements. As pointed out in chapter 2 there are related works proposing process models for agile RE (Memmel et al., 2007), (Kautz, 2010), (Maguire, 2013), (Rivero et al., 2014), (Olsson and Bosch, 2015). Analyzing the commonalities of the proposed approaches, they have in common the continuous management of requirements by involving stakeholder and user. Furthermore they are describing the way of working in the context of agile RE by means workflows, role descriptions, and agile techniques. What we can observe is that there is a huge herterogeneity among agile RE process models focusing on user and stakeholder involvement since they are all tailored to specific needs of the environment in which they are applied. Existing research does not provide a commonly accepted process model for involving users and stakeholders and it is shown that there are still open challenges. Furthermore, the related work lacks in providing generic concepts on an abstraction layer. Researchers as well as agile pratitioners facing the challenge to create a collaborative environment, where value delivery for users and customers is the most important aspect. Furthermore, the results of the SLR show that practitioners using a lot of different techniques in terms of agile RE and especially for engaging stakeholders and users. These agile techniques comprise for instance: co-design sessions with users (Bellucci et al., 2015), Participatory Design (Kautz, 2010), Value Story Workshop (Harbers et al., 2015), lead user (Näkki et al., 2011) or Contextual Inquiry (Obendorf and Finck, 2008), (Maguire, 2013). Each of those techniques pursues a specific context. The variety of techniques leads to difficulties choosing the right kind of agile technique, which enhances collaboration among stakeholders, users and agile teams. In terms of agile RE process models is the challenge to understand what is the impact of specific techniques and how these techniques influence the environment. Another challenge caused by the variety of exiting agile techniques is reported by Blomkvist et al. (Blomkvist et al., 2015). They found that not all artifacts are used by the agile teams and summarized this as the TAGRI (They Ain’t Gonna Read It) principle. This is another indicator that shows the importance of choosing the right combination of agile techniques that fit the context. Finding an appropriate combination of agile techniques is nothing done with one trial, instead an organization must continuously reflect on their process models in order to improve them. To sum up, the overview of exiting agile techniques is missing and this makes it hard to find the appropriate agile technique, which solves a specific problem. 3.2. Objectives This section will present the vision of this PhD thesis and discuss the objectives, which we want to achieve with this work. The vision of this PhD thesis is the following: 52 A Framework for Modeling and Improving Agile Requirements Engineering Building a framework for modeling agile RE. Organizations benefit from implementing this framework by increasing their value delivery (organization external) and improving the collaboration (organizational intern). A framework can be defined as “a supporting structure around which something can be built” (Cambridge Dictionary, 2017). Following this definition, we want to provide an abstract view on the complex field of agile RE. To this end, the main part of the framework builds the agile RE metamodel, which is described in chapter 4. The framework enables researchers as well as practitioners to model new process models for agile RE, which are strongly focused on value delivery and therefore can be classified as value-driven process models. Moreover, the framework can be used to evaluate and improve existing process models for agile RE by reflecting on strengths and weaknesses of the approaches in the field. We need to split the vision of this PhD thesis into several objectives, in order to achieve tangible and measurable results. Table 13 highlights the objectives and the corresponding results of this PhD thesis. Table 13: Overview of objectives of this PhD thesis ID Objectives Expected results 0-1 Perform a systematic literature review, in order to capture the current state of the art of the literature related to agile RE with focus on stakeholder and user involvement. State of the art agile RE 0-2 Propose a conceptual approach for agile RE in order to provide an abstract solution for agile RE management. Split into 0-2.1, 0-2.2 0-2.1 Propose a metamodel for agile RE, in order to provide an abstract view on the influencing parameters of agile RE. Agile RE Metamodel 0-2.2 Provide a domain specific model in order to show how the metamodel can be applied in industry. Instantiations of agile RE process models by means of a profile 0-3 Provide techniques to apply the conceptual approach to the enterprise environment. Split into 0-3.1, 0-3.2 0-3.1 Identify the most important problems in agile RE industry has to face today, in order to provide solutions. Agile RE problems 0-3.2 Create agile RE patterns, in order to solve the identified agile RE problems. Agile RE patterns 0-4 Share the knowledge concerning agile RE, in order to empower people to become a value-driven organization. Web application (agileRE.org) Figure 13 displays a conceptual schema of this PhD thesis comprising the vision, the objectives and the results. We started the work in 2015 with a SLR, which builds the foundation of our research. The analysis of the state of the art helps us to understand the existing process models for agile RE in the field and enables us to build an abstract view by means of the agile RE metamodel. Subsequent to this, we iteratively elaborate the agile RE metamodel and learn how to apply it in industry. For this reason, we identify the most important challenges in agile RE industry has to face today. Together with the agile techniques known from our SLR, we can derive agile RE patterns. Moreover, we want to share the acquired knowledge about agile RE with the community by means of a web application (agileRE.org). 53 A Framework for Modeling and Improving Agile Requirements Engineering Figure 13: Conceptual schema representing the objectives to be achieved with the PhD thesis The proposal of this PhD thesis was discussed at the Doctoral Consortium of the CAISE Conference in 2016 (Schön, 2016). We elaborated the vision and the objectives since that presentation. In the following, we will detail the objectives presented in Table 13 and provide a description on how we want to achieve those objectives. 3.2.1. Perform a Systematic Literature Review As mentioned before, the foundation of this PhD thesis builds a deep analysis of the state of the art of the literature related to Agile RE with focus on stakeholder and user involvement (see chapter 2). In particular, we investigate what approaches exist to involve stakeholder in the process, which methodologies are commonly used to present the user perspective and how requirements management is been carried out. This objective will be addressed by the following research questions: • RQ-2.1: What approaches exist, which involve stakeholders in the process of RE and are compatible with ASD? • RQ-2.2: Which agile methodologies, which are capable of presenting the user perspective to stakeholders, can be found? • RQ-2.3: What are the common ways for requirements management in ASD? In order to analyze the state of the art, we carry out a SLR with an extensive quality assessment of the relevant studies (Schön et al., 2017a). The search space comprises databases like Google scholar, Science direct, SpringerLink, Scopus, IEEEXplore, and ACM. At the beginning, 54 A Framework for Modeling and Improving Agile Requirements Engineering the search results showed a high amount of papers. We reduce the results while carrying out a predefined search process to identify relevant papers for further analysis. 3.2.2. Propose a Metamodel for Agile RE The heart of our framework is the agile RE metamodel, which visualizes the big picture of agile RE [Ref.]. It provides generic concepts, which can be used for tailoring domain specific process models for agile RE according to the organizational environment, where the product is developed. On the one hand, the agile RE metamodel can be used to model new process models, on the other hand, it can be utilized to improve existing process models. Up to now there is no metamodel for agile RE, which builds an abstraction layer about the variety of existing process models. In addition, we want to create a visual language for the agile RE metamodel by means of a profile. An profile allows us to extend the metaclasses of the agile RE metamodel in order to tailor domain specific models. Therefore, this objective is guided by: • RQ-4.1: What does a metamodel that covers agile RE approaches look like? • RQ-4.2: What does an profile for the agile RE metamodel look like? In order to describe agile RE on an abstract level, we will analyze existing process models for agile RE. Based on the related work and our experience working with requirements in agile environments, we can create a metamodel for agile RE (see chapter 4). This metamodel for agile RE will be optimized in several iterations by means of discussion with other researchers and practitioners in the field of agile RE as well as further studies. 3.2.3. Provide Domain Specific Models of the Metamodel A metamodel is a model of a model and visualizes in this work an abstract view on the field of agile RE. Compared to a metamodel, a process model describes what actually happens in the field and details the way of working. A process model can be derived by creating an instance of the metamodel by means of an profile. In chapter 7, we present instances of the agile RE metamodel and provide examples how the metamodel can be applied in industry. This objective will be addressed by: • RQ-7.1: How can the metamodel be tailored to different process models? In order to show how the metamodel can be applied in industry, we will provide two examples of instantiations of the metamodel [Ref.]. On the one hand, we exemplify a process model for a Kanban-based process model for developing an internet-based newspaper portal. On the other hand, we derive an instantiation of the agile RE metamodel into a Scrum-based process regarding the development of web applications in the e-government areas. 3.2.4. Identify the Most Important Problems in Agile RE For an agile transformation, organizations need to continuously improve their established approaches to RE as well as their approaches to software development. This is accompanied by some problems in terms of agile RE. The main objective of chapter 5 is to identify the most 55 A Framework for Modeling and Improving Agile Requirements Engineering important problems in agile RE industry has to face today. Thus, the research questions we pose are listed below: • RQ-5.1: What are the key problems in Agile Requirements Engineering? • RQ-5.2: How can we deal with the identified key problems? In order to identify the most important problems in agile RE, we will conduct an iterative expert judgement process with 26 experts in the field of ASD, comprising three complementary rounds (Schön et al., 2017d). The iterative expert judgement process is rooted in a Delphi study and its main benefit is that we can use the learnings from a previous iteration for carrying out the following iterations. The questionnaire of round 1 comprises open questions, whereas the one in round 2 covers closed questions and comments. In addition, in round 3 we combine closed questions, open questions and comments. 3.2.5. Create Agile RE Patterns Patterns are a common way for sharing knowledge concerning a specific topic. A pattern consists of a recurring problem and the core of a correlated solution. We will introduce the concept of agile RE patterns (Schön et al., 2017b) since agile RE is a complex research field, where we can find a reasonable amount of recurring problems as shown in chapter 5. The derived agile RE patterns will cause strong impact on practitioners since we gather the data among experts in the field of ASD. We address this objective by the following research questions: • RQ-6.1: How can we create agile RE pattern? • RQ-6.2: Which agile RE patterns do exist? In order to create the agile RE patterns, we will carry out an iterative pattern mining process comprising three phases (see chapter 6). In the first phase, we identify agile techniques by means of an SLR. In the second phase, we identify agile RE problems and solutions to solve them. Then, in the third phase, we derive the agile RE patterns by mapping agile RE problems with agile techniques. 3.2.6. Share the Knowledge Concerning Agile RE We want to share the acquired knowledge concerning agile RE with the community. To this end, we will provide tool support by means of a web application. The aim of the web application is to support practitioners and researchers improving their agile RE process models as well as enabling them to solve their key problems in agile RE. Our mission is to empower people to become a value-driven organization. This objective will be addressed by the following research questions: • RQ-6.3: How can we share the knowledge concerning agile RE? In order to share the knowledge concerning agile RE with the community, we will create a web application (agileRE.org). The web application will be developed in several iterations starting with a discussion of the ideas, followed by collaborative prototyping sessions and evaluations with potential users. The main part of the content represents the agile RE patterns. 56 A Framework for Modeling and Improving Agile Requirements Engineering 3.3. Chapter Summary In this chapter we presented the challenges and objectives this PhD thesis will address. The conditions of ASD have an impact on the environment where RE process models are applied. As shown previously, there is a variety of existing process models as well as best practices for agile RE considering stakeholder and user engagement. Each one is adapted to a specific context. An overall framework for improving the variety of agile RE process models is still missing. We can use the existing knowledge and accumulate it in order to build our framework, which is the main goal of this PhD work. The framework for modeling agile RE can be used for the one thing to increase the value delivery. The definition of what value means can vary from organization or environment and needs to be defined by people who work on the product. For the other thing the framework allows to improve the collaboration among people working in the environment, where the product development takes place. Figure 14 shows how the vision of this PhD thesis is split into four main objectives. These objectives are mapped to corresponding research questions, which will be addressed in the scope of this research work. Figure 14: Mapping between vision of this PhD thesis, objectives and research questions All the aforementioned aspects close the introduction of the PhD thesis. From now on, we will detail the contribution of this PhD thesis within the following chapters. 57 A Framework for Modeling and Improving Agile Requirements Engineering Chapter 4 Metamodel for Agile Requirements Engineering he previous chapters provided a state of the art analysis of agile RE with focus on stakeholder and user involvement (chapter 2) and outlined the challenges and objectives we want to address with this PhD thesis (chapter 3). It has been mentioned that agile RE is a crossfunctional area where people from different domains work together in order to deliver value to their customers and users. This cross-functionality lead to a certain amount of complexity, which needs to be reduced in order to build a powerful framework for agile RE. In industry, there can be found different process models for agile RE. These process models differ in how the way of working is described. For instance, which methods or artifacts are recommended to use for discovering, refining, prioritizing, reviewing or documenting requirements in an agile environment. In order to gain a better understanding of agile RE and to reduce the complexity of existing process models, we can build an abstract view on this field by means of a metamodel. As the environment where agile is used is complex, due to the number of diverse people working together in different roles to create one system, we need an extremely adaptive metamodel for describing agile RE on an abstract level. In this chapter, we will contribute a conceptual approach for agile RE with the aim to provide an abstract solution for agile RE management. The main part of this conceptual approach builds the agile RE metamodel, which is visualizing the influencing parameters of agile RE and allows us to understand and to learn how to handle the aforementioned complexity [Ref]. The agile RE metamodel will be fostered by an profile, which builds the visual language of our approach. Section 4.1 summarizes the related work, followed by a gap analysis in section 4.2. Then, section 4.3 describes the research approach and the development of the agile RE metamodel and the profile. Section 4.3.3 presents the improved agile RE metamodel, the profile and discusses on the benefits and implications. 4.1. Summary of Related Work In literature, there can be found related studies proposing process models for agile RE. All these process models share the continuous management of requirements by involving stakeholder and user. The most related ones are highlighted below. Memmel et al. (Memmel et al., 2007) develop a XP-based process model called Crossdiscipline User Interface and Software Engineering (CRUISER). The process starts with an Initial Requirements Up-front Phase (IRUP) whose results are agile models that describe user needs by means of agile techniques (e.g. essential use cases, scenarios and prototypes). The gathered information is elaborated and processed during the different phases of CRUISER. Kautz (Kautz, 2010) carried out a case study with the aim to investigate user and customer involvement in ASD. He does not claim to propose a process model but the results of the study present an implicitly applied process model. Kautz integrates Participatory Design activities in XP. The agile team can detect problems concerning misunderstanding of requirements early before they could grow into larger problems by the application of an onsite customer as well as iteratively reviews with users and customers. T I 64 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.2. Metaclass <<Impact>> Description Due to its iterative and incremental character, agile methodologies have an impact on how requirements are managed in an organizational environment. The metaclass Impact is filled during runtime and describes how requirements are managed. For instance in Scrum, requirements are managed by a Product Backlog, whereas in Kanban, requirements are managed by a Kanban board. Generalization no Attributes The metaclass is filled during runtime Operations no Associations • is in: Methodology [1..*] one or many impacts define the manner how requirements are managed in a methodology • requirement [0..*] one or many impacts describe the effect on how requirements are managed. Restrictions no 65 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.3. Metaclass <<OrganizationalEnvironment>> Description The organizational environment describes the surroundings or conditions in which the product development takes place. On the one hand, the organizational environment is changing because of the different people involved in the process (user, stakeholder or agile team). On the other hand, it is changing because of the domain or the agile methodology used. The metaclass OrganizationalEnvironment is also filled during runtime. Moreover, it makes the metamodel very strong and scalable in terms of future extensions. For instance, someone would like to use the agile RE metamodel in an organizational environment where sequential approaches like the waterfall model or the V- model are used. In such environments there are no agile RE problems, but other kind of problems. In this case, we do not have to change the whole metamodel, instead we can add an additional metaclass to the metaclass OrganizationalEnvironment in order to extend the framework with additional features. Generalization no Attributes Metaclass is filled during runtime Operations no Associations • is part of: Methodology [1..*] one or many organizational environments are composed of methodologies, which describe the manner how the system is developed • requirement [1..*] (association of composition) one or many organizational environments have requirements, which are specified for the system to be developed in the organizational environment • is part of: Domain [1..*] (association of composition) one or many organizational environments are embedded in domains • Stakeholder [1..*] (association of composition) organizational environments have one or many stakeholders Restrictions no 66 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.4. Metaclass <<AgileREProblem>> Description The metaclass AgileREProblem describes problems, which occur in an agile environment in terms of RE. Agile RE problems occur during system development in an agile context. Examples of agile RE problems can be “continuous management of requirements” (C4) or “not to lose sight of the big picture during the implementation of complex requirements” (C3) (see Table 21). Generalization Metaclass <<OrganizationalEvironment>> Attributes • name: String [1] precise description which allows to identify the agile RE problem • description: String [1] provides additional information concerning the agile RE problem to make it tangible Operations no Associations • agileTeam: String [1] one agile RE problem can belong to one agile team • is solved by: AgileREPattern [1..*] one or many agile RE patterns can be used in order to solve an agile RE problem Restrictions no 67 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.5. Metaclass <<Domain>> Description The requirements for each system differ due to the diverse domains in which the system is used. The metaclass Domain becomes more important in terms of the era of Industry 4.0. As an example, it can be highlighted that there are a lot of different IoT platforms on the market that need to be customized for a specific domain (e.g. automotive, utilities or wind energy). Generalization no Attributes • name: String [1] precise description of the area in which the system is used • description: String [1] provides additional information concerning the area in which the product is used • condition: String [1..*] description of domain specific constraints, which have an impact on the system, for instance caused by changes in market or technology • regulation [1...*]: String [1..*] description of laws, guidelines or standards, which have an impact on the system Operations no Associations • is part of: OrganizationalEnvironment: String [1] (association of composition) one domain describes in which area the organizational environment is embedded • is used in: System [1..*] one or many systems are used in a domain Restrictions no 68 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.6. Metaclass <<Stakeholder>> Description A stakeholder is an individual or organization having a right, share, claim or interest in a system or in its possession of characteristics that meet their needs and expectations. This can be for instance management, sales, marketing, or customer. The continuous involvement of stakeholders is very important to ASD in order to develop a system, which meets the expectations and needs of customers and users. Generalization no Attributes • name: String [1] precise name of the stakeholder • group: String [1..*] group in which the stakeholder is part of (e.g. marketing, management) • need: String [1..*] related to human needs, which the stakeholder has in terms of system development • priority: String [1..*] describes how relevant the requests are in terms of system development Operations no Associations • organizationalEnvironment: String [1..*] (association of composition) one or many stakeholders are part of an organizational environment • agileTeam: String [0..*] (association of aggregation) no or many stakeholders might be part of an agile team. In this case, we choose the association of aggregation since an agile team exist without having a stakeholder • defined by: Requirement [0..*] nor or many stakeholders define requirements Restrictions no 69 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.7. Metaclass <<AgileTeam>> Description The agile team groups those people who are responsible for system development including roles like developer, UX designer, tester, Agile Coach, Scrum Master or Product Owner. Generalization no Attributes • name: String [1] precise name of the agile Team • member[1...*]: String [1..*] describes one member of the agile team with all skills s/he has • size: String [1] number of team size • role: String [1..*] describes the roles, which the team is composed of Operations no Associations • agileREProblem: String [0..*] an agile team may have agile RE problems, which they need to solve • stakeholder: String [0..*] (association of aggregation) nor or many agile teams can have stakeholders in it. In this case, we choose the association of aggregation since an agile team exist without having a stakeholder • is part of: User [0..*] nor or many agile teams might have users in it Restrictions no 70 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.8. Metaclass <<AgileREPattern>> Description A pattern is composed of a recurring problem and a solution description. In particular, an agile RE pattern is composed of an agile RE problem and one or more agile techniques, which support solving the problem. For instance, the agile RE problem “continuous management of requirements” can be solved by the agile RE patterns “continuous refinement meetings with stakeholders” or “sprint review meetings”. Generalization no Attributes • name: String [1] name of the agile RE pattern • context: String [1..*] preconditions that describe when to use the agile RE pattern • tag: String [1..*] key word that allows a categorization and findability of an agile RE pattern • problem summary: String [1] short description that provide details about the problem • usage description: String [1..*] provides additional information about how to use the agile RE pattern • example[1...*]: String [1..*] shows an application of the solution in order to provide guidance • template[1...*]: String [1..*] additional material, which eases the application of the agile RE pattern Operations no Associations • is solved by: AgileREProblem [0..*] no or many agile RE patterns might be used to solve agile RE problems • is part of: Methodology [0..*] no or many agile RE patterns might be part of methodologies • is used in: AgileREActivity [0..*] no or many agile RE patterns are used in agile RE activities Restrictions no 71 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.9. Metaclass <<System>> Description The system is a combination of hardware, software and/or services, which describe the product. Generalization no Attributes • name: String [1] name of the system to be developed • description: String [1..*] information that describe the system and its purpose according to the readers needs • feature[1...*]: String [1..*] describes part of the system and provides additional information about the characteristics of it • business goal: String [1..*] objective, which will be addressed by the system • infrastructure[1...*]: String [1..*] describes the surroundings in which the system is embedded Operations no Associations • is used in: Domain [1..*] one or many systems are used in a domain • is part of: ContextOfUse [1..*] one or many systems are part of a context of use Restrictions no 72 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.10. Metaclass <<User>> Description The user is a person who interacts with the system. The user is a specialized form of a stakeholder and is valued by an additional metaclass in the agile RE metamodel, since s/he is in the center of product development within a valuedriven organization where HCD (International Organization for Standardization, 2010) plays an important role. Generalization Metaclass <<Stakeholder>> Attributes • name: String [1] name of the user • personal information: String [1..*] information that allows to build a picture of the user and his/her personality • behavior: String [1..*] information that describes typical behavior of the user in daily life • usage behavior: String [1..*] information that describes how s/he interacts with the system • motivation: String [1..*] describes the underlying reason for using the system • pain point[1...*]: String [1..*] information about what causes lead to the usage of the system and additional information why the system is relevant Operations no Associations • is part of: AgileTeam [0..*] nor or many users might be part of an agile team • is in: ContextOfUse [1] one user is in a context of use while using the system Restrictions no 73 A Framework for Modeling and Improving Agile Requirements Engineering 4.4.1.11. Metaclass <<AgileREActivity>> Description An agile RE activity is an action that is carried out in terms of RE in an agile environment. Agile RE activities (see Table 1) describe actions that take place in terms of requirements management. In the context of the agile RE metamodel, agile RE activities can be used to categorize agile RE patterns. Generalization no Attributes • name: String [1] name of the agile RE activity • stage: String [1..*] describes when the activity takes place in terms of phases of project management Operations no Associations • is used in: AgileREPattern [0..*] nor or many agile RE activities can categorize agile RE patterns Restrictions no 80 A Framework for Modeling and Improving Agile Requirements Engineering 4.5. Chapter Summary This chapter contributes a metamodel for agile RE (agile RE metamodel), aiming to build an abstraction layer about the variety of existing process models in the field. The agile RE metamodel allows us to analyze the environment of an organization in terms of value delivery and collaboration. Moreover, we contribute a visual language for the agile RE metamodel by means of a profile. The profile can be used to tailor domain specific models. The initial version of the agile RE metamodel is inferred by an analysis of related process models for agile RE and experiences working in industry projects. We improved the metamodel iteratively by means of discussions with other researchers and further empirical studies. The agile RE metamodel builds the heart of our framework and shapes the conceptual approach for agile RE management. It allows us to reflect the dependencies among people and product development process and supports improvement by inspection and adaptation. The metamodel has several implications for both, researchers and practitioners. On the one hand, the metamodel can be used to model new process models for agile RE that are strongly focused on value delivery and therefore can be classified as value-driven process models. On the other hand, existing process models can be evaluated as well as improved by the metamodel. The following chapters will propose techniques to apply the presented conceptual approach. Thus, we will identify the most important problems in agile RE industry has to face today as well as create agile RE patterns that allow to solve the identified agile RE problems. 81 A Framework for Modeling and Improving Agile Requirements Engineering Chapter 5 Identification of Agile RE Problems n the previous chapters we explained our conceptual approach by both introducing the agile RE metamodel and a profile. The main part of the agile RE metamodel builds the organizational environment, which is composed of the surrounding in an agile organization. Applying agile methodologies lead from time to time to recurring problems in terms of agile RE. As explained by the metamodel agile RE problems can be solved by agile RE patterns. For an agile transformation, organizations need to continuously improve their established approaches to RE as well as their approaches to software development. This is accompanied by some problems in terms of agile RE. The agile RE problems have different rationales. Nevertheless, often they can be abstracted to a similar core. This chapter aims to identify the most important problems in terms of agile RE industry has to face today resulting in a catalogue of agile RE problems. The identification of agile RE problems allows agile practitioners as well as researchers to improve their agile environment since the knowledge supports them to understand and to reflect their problems. In this chapter, we will provide techniques to apply our conceptual approach to the enterprise environment. We present the results of our iterative expert judgement process, comprising three complementary rounds (Schön et al., 2017d). As mentioned before, research about agile RE is very close to existing work practices in companies. Therefore, we conducted this study with a panel consisting of 26 experts working in the field of ASD. Section 5.1 summarizes the reported problems in agile RE by related work and presents what approaches were used in order to identify these problems. Then, section 5.2 discusses on the gaps in existing literature. In section 5.3, we will present our research design by explaining the iterative expert judgement process and the three rounds. Subsequent, in section 5.4 we discuss the results, meaning of findings and limitations of this study. 5.1. Summary of Related Work Performing agile RE can lead to problems organizations have to deal with. In literature, there can be found some studies investigating problems in agile RE by means of different research methods. Table 16 shows an overview of the reported challenges and used research methods. I 82 A Framework for Modeling and Improving Agile Requirements Engineering Table 16: Problems in agile RE reported by related work Authors Research method Reported problems Ramesh, Cao, Baskerville (Ramesh et al., 2010) Multi-case study (16 companies) Problems with cost and schedule estimation; inadequate or inappropriate architecture; neglect of non-functional requirements; customer access and participation; prioritization on a single dimension; inadequate requirements verification; minimal documentation Bjarnason, Wnuk, Regnell (Bjarnason et al., 2011) Case study Planning for agility; weak requirements prioritization; weak effort estimates; quality issues; system completed late; capturing innovation; lack of documented requirements; customer-proxy role; ensuring competence (RE, VV); motivating teams for requirements work; weak requirements at start Inayat, Salim, Marczak, Daneva, Shamshirband (Inayat et al., 2015) Systematic Literature Review Minimal documentation; customer availability; inappropriate architecture; budget and time estimation; neglecting nonfunctional requirements (NFRs); customer inability and agreement; contractual limitations; requirements change and its evaluation Heikkila, Damian, Lassenius, Paasivaara (Heikkilä et al., 2015) Mapping Study Problems with client or customer representatives; insufficiency of user story format; difficulties in prioritization of requirements; growing technical debt; reliance on tacit requirements knowledge; imprecise effort estimates Soares, Alves, Mendes, Mendonca, Spinola (Soares et al., 2015) Systematic Literature Review Requirement prioritization; non-functional requirements identification; lack of information; volatility of requirements; requirements definition; dependence among requirements; prediction of impacts of changes; user dependence; communication and collaboration with users; requirements validation 5.2. Gap Analysis of Related Work Analyzing the related work, we can state that the authors use two different kinds of research approaches in general. On the one hand, Ramesh et al. (Ramesh et al., 2010) and Bjarnason et al. (Bjarnason et al., 2011) utilize case studies to investigate the problems in the field. On the other hand, Inayat et al. (Inayat et al., 2015), Heikkila et al. (Heikkilä et al., 2015) and Soares et al. (Soares et al., 2015) report problems in agile RE by analyzing primary studies with the aim to identify available evidence in existing research. Ramesh et al. (Ramesh et al., 2010) results were published in 2010. However, as ASD is a rapidly changing research area and the body of knowledge has evolved over the last years, we need to clarify whether the reported problems are still relevant today. For instance, NFRs may not be longer a problem for industry since the concept of the Definition of Done and the usage of acceptance criteria are widely spread. Bjarnason et al. (Bjarnason et al., 2011) carry out a case study in only one company, therefore the results may not be applicable to other companies and may not be representative in general. In comparison, Inayat et al. (Inayat et al., 2015), Heikkila et al. (Heikkilä et al., 2015) and Soares et al. (Soares et al., 2015) review primary studies by analyzing existing literature, which is a good approach to get an impression of relevant aspects from a 83 A Framework for Modeling and Improving Agile Requirements Engineering theoretical viewpoint. Nevertheless, one could argue that this is not an appropriate approach to investigate the existing problems in practice. To this end, the aim of this chapter is to identify the most important problems in agile RE industry has to face up today by getting insights from 26 experts in the field. To the best of our knowledge there is no existing study investigating these problems by means of a qualitative study with practicing experts in ASD working for many different companies. 5.3. Research Method We used an iterative expert judgement process rooted in a Delphi study (Dalkey and Helmer, 1963), (Diamond et al., 2014), (Linstone and Turoff, 2002) in order to identify the most important problems industry has to struggle with in terms of agile RE. Figure 22 shows the four phases of a Delphi study and the tasks, which had been carried out. Figure 22: Phases of a Delphi study Delphi studies are used across different domains in order to obtain the opinion of a panel of experts in an iterative structured manner by means of a series of questionnaires. They had been used since the 50s by the RAND Corporation in the military area (Dalkey and Helmer, 1963). Linstone and Turoff (Linstone and Turoff, 2002) define the technique as follows: “Delphi may be characterized as a method for structuring a group communication process so that the process is effective in allowing a group of individuals, as a whole, to deal with a complex problem.” Classically the method aims to produce consensus among the panel of experts concerning the subject under investigation. Therefore, different metrics like Fleiss´ Kappa (Fleiss, 1971) or Kendall´s coefficient of concordance (Legendre, 2005) are utilized. Recent studies reveal that the definition of what constitutes consensus is less clear (Diamond et al., 2014). During our study, we recognized that we were able to achieve more valuable insights by adapting the questionnaire of each round to the objectives of the study. To this end, we applied a modified Delphi study where measuring consensus and stability at group level among several iterations was not the most crucial part. On the contrary, we shifted the focus to applying the valuable features of Delphi for conducting our iterative expert judgement process (Dalkey, 1969): • Anonymity among experts to avoid influence of dominant individuals • Iterative approach • Controlled feedback with statistical group response 84 A Framework for Modeling and Improving Agile Requirements Engineering The main benefit of our modified approach was utilizing the learnings from a previous iteration for carrying out the following ones. 5.3.1. Objectives and Research Questions The study pursues the main objective of identifying the most important problems in agile RE industry has to address today. We aim to build a shared understanding concerning these problems among voices that matter by means of experts in the field of agile RE. Thus, the research questions we pose are listed below: • RQ-5.1: What are the key problems in Agile Requirements Engineering? • RQ-5.2: How can we deal with the identified key problems? 5.3.2. General Study Design The study was performed in three complementary rounds. Figure 23 gives a general overview of the process. At the beginning of each round, we started designing the questionnaire, optimized by a pretest. Once finished, the invitation was sent to the experts via email. In the second and third round, we attached the results of the previous rounds to the invitation in order to share the outcomes among the panel. The experts had two weeks to fill in the questionnaire. During the following two weeks we evaluated the results, created the report, specified the criteria for dropping items for the following round and designed the questionnaire for the next round. We conducted the study in German since most of the experts are native speaker. Since we are aware that the term agile RE is not very accepted in the agile community and some experts understand this as a contradiction in itself, we decided not to ask for problems in agile RE directly. On the contrary, we phrased our questions differently and described the context of our study within the introduction part of each questionnaire. Figure 23: General process of study We used google forms for the first and second round, whereas limesurvey was used for the third round due to the complexity of the questionnaire. In general, we decided to use 7-point Likert items since this has been proven to be the best choice in terms of avoiding interpolations within related research fields (Finstad, 2010). Besides, we adapted the quality criteria proposed by Diamond et al. (Diamond et al., 2014) so as to ensure the quality of our study. Table 17 highlights parameters concerning the three rounds of the study. In the beginning, we sent out 54 invitations in order to recruit our panel. In the first round, we received responses from 26 experts, who will be referred as panel. The panel was quite stable among the second and third round, where we have response rates of 88.5% and 84.6%. Literature recommends to have a 85 A Framework for Modeling and Improving Agile Requirements Engineering minimum of 10-15 panelists, for reliable results of a Delphi study (Lilja et al., 2011). In the last round, we received responses from 22 experts which fit this requirement of reliability. Table 17: Overview of rounds Parameter Round 1 Round 2 Round 3 Period of data gathering 2016-08-16 until 2016-08-29 2016-09-14 until 2016-09-28 2016-10-26 until 2016-11-11 Number of invitations 54 26 26 Responses 26 23 22 Response rate 48.1% 88.5% 84.6% 5.3.3. Panel of Experts We selected our panelists specifically for their knowledge or position regarding the issue under study. As shown previously, the research field of agile RE is very close to existing work practices in industry (see section 2.2). To this end, we defined the reproducible criteria for selecting participants as follows: • Many years of experience as professional in the field of ASD. With this criterion we aim to avoid bias from theoretical experts who may state a consolidated view gained from literature. • Working experience in one or more of the following roles: Product Owner, Scrum Master, Agile Coach, Consultant for Agile Transition, Kanban Expert or Lean Startup Expert. We choose these roles due to their responsibilities they have to cope with problems in terms of Agile RE. We recruited the panel within our network since we wanted to ensure that each expert has the required knowledge to answer our questions and to provide valuable insights into the subject under study. The panel consisted of 26 experts who are working in 19 different companies located in Germany and Switzerland. The companies differ in terms of their employees from small to large organizations and spread over different sectors (e.g. consulting, e-commerce, GIS provider, cyber security, publishing industry, or fiscal). The experts referred the following job titles: Consultant (7x), Scrum Master / Agile Coach (3x), Head of Digital Media (2x), Head of Project Management (2x), Product Owner (2x), Project Manager (2x), Software Engineer (2x), Director Consulting Services (1x), Director of a company (1x), Head of UX (1x), UX Enthusiast (1x), Team Coach (1x), and Solution Architect (1x). Figure 24 displays the type of organization the different panelists are working for. It can be observed that most of the experts (20 out of 26) working for companies in the private sector. No expert is working in for an university or a research institution. 86 A Framework for Modeling and Improving Agile Requirements Engineering Figure 24: Type of organization for which experts work In general, the experts had 2-10 years of experience working in ASD (average = 6.14 years). In comparison, experts have about 0-16 years of experience with RE (average = 6.65 years). Even though one expert stated that he had no experience with RE at all, we decided to include his answers into the study, since he has long experience in ASD and in general there do not exist a specific role of a requirements engineer. Figure 25 shows the kind of process models experts have been working with. It is worth mentioning that most of the experts have experience both with sequential approaches and with agile approaches. Figure 25: Process models used by experts In addition, Table 18 displays the know-how level in terms of ASD rated by experts themselves. In sum, 84.6% of the experts rate their know-how either as high or very high. None of them assess his/her knowledge as poor. Table 18: Know-how of panel (N=26) in terms of ASD know-how very poor 1 2 3 4 5 know-how very high 0.0% 0.0% 15.4% 69.2% 15.4% 87 A Framework for Modeling and Improving Agile Requirements Engineering 5.3.4. Round 1 Questionnaire Design The questionnaire of the first round comprised two sections. The first section queried personal information for the characterization of each expert (summarized in section 5.3.3). The second section contained two open questions, repeated 15 times (see Figure 26). The first question queried the most important challenges with requirements in terms of ASD. The second question asked for a statement for each challenge to clarify why the expert considered this challenge as important. The minimum number of required answers was 3, whereas the maximum was 15. In sum, we received 107 answers (items) from 26 experts. Figure 26: Excerpt of questionnaire of round 1 88 A Framework for Modeling and Improving Agile Requirements Engineering Data Analysis With respect to data analysis, each challenge was categorized by the authors (see (Schön et al., 2017d)) during a workshop. Those items, which could not be categorized easily, were discussed within the group of authors. We used the following categories: stakeholder and user involvement, collaboration within the team, vision and big picture, iteration planning and estimation, granularity of requirements, dependencies of requirements, understanding agile and agile values, continuous delivery of value, roles and responsibilities, need for security, requirement validation, RE methods, format of requirements, clarity of requirements, prioritization, refinement, discovery and transparency. Additionally, the reported challenges were categorized according to their agile RE activity (see Table 1). Results As mentioned before, we carried out the study in German. The results of round 1 are the categorized items stated by the experts, which are mostly written in German. Table 19 shows an example of an item consisting of a challenge and a statement concerning importance. Table 19: Exemplary item in round 1 Question round 1 Answer given by expert What challenge do you perceive with requirements in terms of Agile Software Development? Stakeholders affected by requirements or changing the system are not involved. Why do you consider this challenge as important? In one of my projects, representatives of end users did not really knew the pain of end users. Even the early UI prototypes were tested by incorrect stakeholders, which led to risks of conflicts and failure. For avoiding bias, we did not translate all the statements. Nevertheless, the categorized items can be found in Appendix IV. In addition, the result report of round one can be found in (Schön et al., 2017e). 5.3.5. Round 2 Questionnaire Design We checked each item of round one critically, whether or not it was appropriate for answering our RQs and being queried in the next round. Thus, items of round 1 were consolidated or excluded. Subsequent, we reduced the contextual information given by the expert and merged related items, in order to find the core of the stated issues. In the end, we identified 34 items as relevant for assessing them in round 2. Based on those items, we created the questionnaire for the second round. The resulting questionnaire assessed 34 items related to the following topics: 89 A Framework for Modeling and Improving Agile Requirements Engineering • stakeholder and user involvement (6 items) • understanding agile and agile values (6 items) • RE methods (10 items) • iteration planning and estimation (6 items) • format of requirements (6 items) The items were grouped into sections by the aforementioned topics. After each section, the experts had the opportunity to give additional comments in free text form. They rated each item using 7-point Likert items (see example in Figure 27). Moreover, they could choose giving no statement. Figure 27: Exemplary item of round 2 Data Analysis To sum up, we received responses from 23 experts. For each item we calculated mean, variance and standard deviation. Additionally, we created a diagram showing the distribution of experts’ opinion (see example in Figure 27) and discussed on the meaning of findings. Results Calculation of mean, variance, standard deviation and the diagram of experts’ opinion can be found in Appendix IV. The result report of round two can be found in (Schön et al., 2017f). Stakeholder and User Involvement. Experts agree that it is necessary to have a direct involvement of user and stakeholder so that product development will succeed and user´s needs can be fulfilled (Item 1.1, Item 1.2). In this context, there is a high correlation to both the type of product and the reachability of the target group (see comment expert 9). In addition, lightweight methods need to be applied in order to involve users and stakeholder (see comment expert 15). Furthermore, the continuous communication is an important aspect in this context (Item 1.6). The experts agree that face-to-face communication is an appropriate technique not to miss important expectations from different stakeholders (Item 1.3). Moreover, we can state that the experts differ whether the requirements need to be validated with direct end users before development can start (see Item 1.4). This may correlate with the culture of a company in terms of HCD (International Organization for Standardization, 2010). Moreover, the experts have a different opinion concerning the moment of receiving feedback from the stakeholder (Item 1.5). 96 A Framework for Modeling and Improving Agile Requirements Engineering C1i How important is the coordination, which evolves due to functional or technical dependencies with other teams? Figure 30: Importance key problem C1 (N=13) 13 experts assessed the importance of functional or technical dependencies to other teams (C1i), although C1 is the agile RE problem most expert stated that it is a problem (see Table 21). Compared to other items (C2i-C6i), we can observe a very low standard deviation (0.38), since 84,6% of the experts weigh this item as totally important and 15.4% weigh it as important. In sum, the panel rates this key problem as important. No expert weighs this item as unimportant. Table 22: Calculations of importance concerning key problem C1 ID N Mean Standard deviation Important Unimportant C1i 13 1.15 0.38 100% 0% 97 A Framework for Modeling and Improving Agile Requirements Engineering C2i How important is that stakeholders understand, that the development team can make independent (detailed) decisions? Figure 31: Importance key problem C2 (N=14) With C2i we queried the panel how important they perceive that stakeholders understand that the development team can make independent (detailed) decisions. This has direct impact on the understanding of agile values within an organization. 14 experts assessed this item and the answers range from totally important (57.1%), over important (21.4%), and rather important (14.3%), to neutral (7.1%). Compared to other items, this item has the highest standard deviation (0.99). In sum, the panel rates this key problem as important. No expert weighs this item as unimportant. One expert weighs this problem as neutral. Table 23: Calculations of importance concerning key problem C2 ID N Mean Standard deviation Important Unimportant C2i 14 1.71 0.99 92.9% 0% 98 A Framework for Modeling and Improving Agile Requirements Engineering C3i How important is not to lose sight of the big picture during the implementation of complex requirements? Figure 32: Importance key problem C3 (N=15) 15 experts of the panel assessed the importance of not losing sight of the big picture during the implementation of complex requirements. 60% of those experts rate C3i as totally important, 26.7% weigh the item as important, and 13.3% as rather important. The standard deviation is 0.74. In sum, the panel rates this key problem as important to totally important. No expert weighs this item as unimportant. Table 24: Calculations of importance concerning key problem C3 ID N Mean Standard deviation Important Unimportant C3i 15 1.53 0.74 100% 0% 99 A Framework for Modeling and Improving Agile Requirements Engineering C4i How important is the continuous management of requirements? Figure 33: Importance key problem C4 (N=15) The sample for measuring the importance of the continuous management of the requirements consists of 15 experts. The opinion of experts distributes among totally important (66.7%), important (20%), rather important (6.7%), and neutral (6.7%). Compared to other items, this one has also a relatively high standard deviation (0.92). In sum, the panel rates this key problem as important to totally important. No expert weighs this item as unimportant. Table 25: Calculations of importance concerning key problem C4 ID N Mean Standard deviation Important Unimportant C4i 15 1.53 0.92 93.3% 0% 100 A Framework for Modeling and Improving Agile Requirements Engineering C5i How important is it to work out user requirements and quality of use in cooperation with direct users (end users) of the product? Figure 34: Importance key problem C5 (N=12) This item (C5i) was rated by the fewest number of experts (N=12). We measured how important is to work out user requirements and quality of use in cooperation with direct users of the product. 58.3% of the experts weigh this item as totally important, 25% weigh it as important, and 16.7% as rather important. The standard deviation is 0.79. In sum, the panel rates this key problem as important to totally important. No expert weighs this item as unimportant. Table 26: Calculations of importance concerning key problem C5 ID N Mean Standard deviation Important Unimportant C5i 12 1.58 0.79 100% 0% 101 A Framework for Modeling and Improving Agile Requirements Engineering C6i How important is it to involve stakeholders throughout the whole development process in regular iterations? Figure 35: Importance key problem C6 (N=14) Item C6i queried the importance of the involvement of stakeholders throughout the whole development process in regular iterations. The opinion of the experts divides into totally important (57.1%) and important (42.9%). For item C6i, we observe with 0.26 the lowest standard deviation. In sum, the panel rates this key problem as important to totally important. No expert weighs this item as unimportant. Table 27: Calculations of importance concerning key problem C6 ID N Mean Standard deviation Important Unimportant C6i 14 1.43 0.26 100% 0% 102 A Framework for Modeling and Improving Agile Requirements Engineering 5.4.2. (RQ-5.2) How Can We Deal With The Identified Key Problems? Experts recommend techniques, methods and tools in order to deal with the problems in agile RE. Below, we will list the techniques and methods proposed by the panel for each key problem. C1 In agile software development functional or technical dependencies with other teams are a challenge because a considerable coordination effort is required. More than three experts recommended using scaled frameworks such as LeSS (The LeSS Company B.V., 2017), SAFe (Scaled Agile Inc., 2017) or Scrum of Scrum (Sutherland, 2001). Moreover, they proposed the use of the following techniques: creating a common understanding among all, enhancing continuous communication and collaboration, training the ability to solve dependencies, holding weekly coordination meetings, organizing teams in matrix management, building communities of practices for transcending topics, release planning (SAFe), teamtranscending availability of product und sprint backlogs, involving temporary representatives in other teams, enforcing continuous integration, improving API-driven development and microservices. C2 In agile software development it is a challenge that stakeholders understand that the development team can make independent (detailed) decisions. The following techniques were suggested: continuous coordination and presenting possible solutions to stakeholder, providing transparency about rationales of the decisions, strengthening product owner with competency in decision making and helping stakeholders become aware of the consequences of interfering into detailed decisions. More than three experts recommended providing alternative solutions for one requirement. In addition, it is useful to demonstrate that the recommended solution of a stakeholder is an alternative out of many. In previous rounds, more than one expert stated that product owner and stakeholder altogether decide what to be developed. In contrast, the development team decides how the requirement should be developed. C3 In agile software development it is a challenge not to lose sight of the big picture during the implementation of complex requirements. The following techniques were recommended: creating a shared understanding regarding the meaning of the big picture by means of a product vision, defining epics or sub goals in the beginning, managing the big picture as a responsibility of the product owner, providing transparency concerning changes among all, understanding connections among user stories by means of story mapping, visualizing customer journey in the beginning, involving users continuously in order to focus on the problem to be solved and identifying central contact person for related topics to enable rapid coordination. Moreover, the experts advised to use visualization by means of roadmaps, sketches of the system and processes, and value streams. 103 A Framework for Modeling and Improving Agile Requirements Engineering C4 In agile software development continuous management of requirements is a challenge since not all of them are fixed at the beginning and they may change over the course of the project. The experts proposed the following techniques, methods and tools: collaborating closely with the requesting stakeholder, communicating regularly within the team, refining and prioritizing continuously the Product Backlog, grooming on demand (Kanban), describing in detail the requirements in the sprint backlog, reviewing the results regularly, discussing the maturity level of a requirement with the team, grouping user stories to epics, using Kano analysis, screening and scoring the theme, weighting relatively, utilizing spike stories to evaluate uncertainty in requirements and using ticketing tools (e.g. JIRA). C5 In agile software development it is a challenge to work out user requirements and quality of use in cooperation with direct users (end users) of the product. The experts recommended utilizing the following techniques: prototypes, interviews, observing users by the think aloud method, A/B testing, UX labs, analyzing usage behavior, friendly user tests, alpha/beta/silent launches, improving continuously a released version, utilizing a UX-board for play back user insights and testing hypotheses with real users. In addition, one expert suggested adapting user research to ASD by reducing the methods to the minimal, evaluate within the team without report creation, reducing financial restrictions for user involvement as well as problems of accessing real user by means of panels or a prior recruitment. C6 In agile software development it is a challenge to involve stakeholders throughout the whole development process in regular iterations so that product development will succeed. The following techniques were proposed: defining stakeholders and their involvement in regular iterations, proposing goals instead of prescribing solutions, involving all possible stakeholders in the beginning and reducing the amount of people over time. More than eight experts suggested involving stakeholders by regular planning and review meetings to gather feedback and useful information. In light of this, they recommended clarifying the purpose of the meetings and the importance of the outcomes to be discussed beforehand. 104 A Framework for Modeling and Improving Agile Requirements Engineering 5.4.3. Meaning of Findings Comparing our findings to the identified problems of the related work (see Table 16), we can conclude that 16 out of our 20 problems are not reported by the related studies. Our key problem C5 (user involvement) is reported by all related studies. In addition, three studies (Inayat et al., 2015), (Soares et al., 2015), (Ramesh et al., 2010) report issues with nonfunctional requirements, which is comparable to our problem C13. There is also a relation between the key problem C4 (continuous requirements management) and the problem “requirements change and its evaluation” reported by (Inayat et al., 2015). Moreover, the key problem C1 (technical or functional dependencies to other teams) is reported by (Soares et al., 2015) in a slightly different manner since they phrase it like “dependence between requirements”. Moreover, the results show that the identified problems are often not limited to ASD, but they rather refer to software development in general. Therefore, we can conclude that organizations still struggle with agile transition and understanding agile values, in particular, in terms of stakeholder and user involvement. The identified agile RE problems allow agile practitioners as well as researchers to improve their agile environment since the knowledge supports them to understand and to reflect their problems. The high level description of the key problems can be enriched with contextual information, so that they are adapted to the specific environments. For instance agile practitioners can ask themselves during a retrospective whether they struggle with one of the identified key problems and think about the variation of the problem. 5.4.4. Limitations We are aware that the design of a questionnaire is important for the process of data gathering. To this end, we made several pretests of each questionnaire we used with participants matching our criteria of expert selection. Nevertheless, we observed two experts struggling with the UX of the questionnaire tool (Google Forms) used in round 1. Therefore, we decided to use another tool (LimeSurvey) for the questionnaire in round 3, which was more complex than the previous two. To carry out the study, the group of authors created summaries of the results and made decisions concerning the kind of items they had to query in the following rounds. That may lead to bias in the opinion building process of the panel. We tried to prevent this point by being very accurate in terms of data analysis and by creating the reports. In addition, we selected items for the following rounds through the selection criteria defined earlier. Our panel consists of experts from Germany and Switzerland. Therefore, one could argue that there is a geographical limitation of the applicability of the results. In terms of this, we have to say that most of the experts are working in international environments. Nevertheless, we decided not to tackle this aspect within the scope of this study in order to keep the complexity of this study as low as possible. We started to set up new questionnaires with the aim to assess the identified agile RE problems (see Table 20) by means of an international sample. It could be argued that there are some limitations in terms of the general application of the results due to the usage of an expert validation. However, the experts work for different companies and therefore they have different levels of experience in the agile transformation of an organization. Additionally, the identified key problems (Table 21) only give guidance for the optimization of existing process models for agile RE in industry and do not claim to be fully complete. 105 A Framework for Modeling and Improving Agile Requirements Engineering 5.5. Chapter Summary This chapter has addressed the identification of the most important problems in agile RE industry has to face up today. Moreover, we examined how to deal with those problems. For that purpose, we carried out an iterative expert judgement process comprising three complementary rounds. During these rounds we find the core of a specific problem by abstracting contextual information of the problems. The learnings from previous iterations were used for carrying out the following ones. Our panel consisted of 26 experts in the field of ASD working for 19 different companies. We have identified 20 problems industry has to address at present in terms of agile RE. Six of these problems have been defined as key problems: • technical or functional dependencies to other teams (C1) • understanding of agile values of the stakeholders (C2) • staying focused on the big picture (C3) • continuous requirements management (C4) • refine requirements in collaboration with users (C5) • involve stakeholder iteratively (C6) In addition, we have analyzed options to deal with those key problems by means of agile techniques recommended by the panel. The results show that the identified challenges are often not limited to ASD, but they rather refer to software development in general. Therefore, we can conclude that organizations still struggle with agile transition and understanding agile values, in particular, in terms of stakeholder and user involvement. The agile RE problems and agile techniques allow us to create agile RE patterns. The concept of agile RE patterns will be introduced in the following chapter. 112 A Framework for Modeling and Improving Agile Requirements Engineering Launch of product features Methods x x Lean user research Methods x x Users time is valuable Methods x Co-design Methods x x x Define agile RE process model Methods x x x Transparency of decisions Methods x x x Weighing up various solution proposals Methods x x Coaching Methods x x Lifecycle managements by means of tools Methods x x Shared understanding Methods x x x x Scaling agile Methods x Continuous integration Methods x API-driven Development Methods x Micro services Methods x Community of practices Methods x Pairing Methods x Product Owner Roles x x x x Agile Coach/Scrum Master Roles x x x Development team Roles x x x Expert Roles x x x x x Stakeholder Roles x x An agile RE pattern is composed of dynamic and static parts (see Figure 40). On the one hand, the agile technique Scaling Agile is an activity and therefore represents the dynamic part. On the other hand, the agile technique user story is an artifact and represents the static part of an agile RE pattern. Figure 40: Agile RE patterns are composed of dynamic and static parts Table 29 shows the classification of agile RE patterns according to their dynamic or static parts. Agile RE patterns, which agile technique is an activity are classified as dynamic agile RE pattern, whereas patterns with a static agile technique are classified as static agile RE patterns. Roles do not fit into this classification in a classical sense, since they could be handled as actors in 113 A Framework for Modeling and Improving Agile Requirements Engineering UML. However, in terms of classification of agile RE patterns we classify them as static due to the descriptive nature of a role description. Table 29: Classification of Agile RE patterns in dynamic and static Pattern type Dynamic agile RE patterns Static agile RE patterns Artifacts Minimum Viable Product (MVP), Kanban board, Prototypes, Definition of ready and definition of done, User stories, Product Backlog, Roadmap, System models, Story map, Process models, Value stream, Customer journey map, Product vision, Impact map Meetings Refinement meeting, Planning meeting, Review meeting, Daily standup meeting, Retrospective Methods Evaluation and testing, Launch of product features, Lean user research, Users time is valuable, Co-design, Define agile RE process model, Transparency of decisions, Weighing up various solution proposals, Coaching, Lifecycle managements by means of tools, Shared understanding, Scaling agile, Continuous integration, API- driven Development, Micro services, Community of practices, Pairing Roles Product Owner, Agile Coach/Scrum Master, Development team, Expert, Stakeholder 114 A Framework for Modeling and Improving Agile Requirements Engineering 6.4.2. Examples of Agile RE Patterns Each agile methodology (e.g. Scrum, Kanban, XP) comes with its own requirements that have impact on how RE is carried out (see Figure 17, metaclass Impact). For instance, there is a difference between flow-driven approaches like Kanban or time-boxed approaches like Scrum. To this end, agile RE problems are combined with different agile techniques, resulting in a set of related agile RE pattern. Below, we will present four examples of Agile RE patterns: a) Evaluation and Testing b) Story Map c) Product Owner d) Review Meeting As noticed, we used appropriate guidelines for pattern writing provided by (Wellhausen and Fiesser, 2011) and (Kohls, 2012). We created our own pattern template based on existing templates. The elements of our pattern template are the attributes of the metaclass AgileREPattern (see Figure 41), which is defined by the agile RE metamodel (see Figure 17). Figure 41: Metaclass AgileREPattern The name is a unique identifier for the agile RE pattern, whereas the context set the stage, where the pattern can be applied. The tag is used for classification of the agile RE pattern. The problem summary combines a description of the problem and forces, which explain why the problem is difficult to solve. Moreover, the usage description entails the solution and the consequences that would follow, if the pattern is applied. The section example provide an excerpt of know uses and the section template provides templates, which are used in industry as best practices for applying the agile technique. The agile RE pattern Evaluation and Testing is classified as method, Story Map is classified as artifact, Product Owner is classified as role and Review Meeting is a meeting. We applied the agile RE pattern Evaluation and Testing in one of our case studies related to the integration of HCD and Kanban (see section 7.3.1 and (Schön et al., 2016b)). The project was carried out in a medium-sized IT company, located in Germany, specialized in e-commerce, mobile apps and SAAS tools. We aim to relaunch an internet-based newspaper portal in a period of six months along 2013/2014. 115 A Framework for Modeling and Improving Agile Requirements Engineering Pattern name Evaluation and testing Context Working in a Kanban system make people focus on small tasks. This can cause the problem of leaving out sight of the big picture during the implementation of complex requirements. Hence, it is hard to design a positive User Experience (UX) for the user. Tag Discovery, refinement, review, methods Problem summary In ASD it is a challenge not to lose sight of the big picture during the implementation of complex requirements (see C3, Table 21). • providing a positive UX to the user. • carrying out a release evaluation continuously. • not interrupting the workflow due to scheduling testing activities and organization. • reducing costs for long-term UX testing. Usage description Carrying out a regular release evaluation (Schön et al., 2016b) by means of Usability and UX testing (Hartson and Pyla, 2012), (Schrepp et al., 2014). Therefore, a work in progress (WIP) limit to the last column (“Done”, see Figure 42) of the Kanban board should be introduced. The release evaluation should start, once the WIP limit is reached. The regular release evaluation helps you stay focused on the big picture. In addition, it enables improving the overall UX of the product. Introducing a WIP limit to the last column of the Kanban board reduces the complexity of finding the starting point for a release evaluation. Moreover, it allows carrying out UX testing continuously. Examples Figure 42: Example of a Kanban board The agile RE pattern evaluation and testing is related to the pattern Sprint Zero by Bertholdo et al. (Bertholdo et al., 2014), since both patterns address the same problem in different stages. Sprint Zero is utilized as a stage before starting the project implementation, whereas evaluation and testing is a recurring task during the development of a product. Besides, the solution of the pattern evaluation and testing is related to the pattern Usability Testing Sessions Alongside Agile Development Tests by Salah et al. (Salah et al., 2015) since it may also solve the problem of scheduling Usability and UX testing in an agile environment. Template See Figure 42 116 A Framework for Modeling and Improving Agile Requirements Engineering Pattern name Story map Context User stories let people focus on developing small increments. This leads to the problem of losing sight of the big picture. In addition, prioritizing user stories is difficult due to the open question of what the user really needs. Tag Discovery, refinement, prioritization, documentation, artifacts Problem summary In ASD it is a challenge not to lose sight of the big picture during the implementation of complex requirements (see C3, Table 21). • it is hard to find out what the user needs and what product to build. • agile teams and stakeholder struggle with prioritizing requirements for different releases. • it is tough to identify the scope of the MVP, so it is uncertain when a first version of a product should be released. Usage description Managing user stories by means of a story map. Story Mapping (Patton, 2014) is an agile technique that can be used to manage user stories. A story map tells the story about the product and its usage from a user´s perspective. Dependencies among user stories can be identified easily and prioritizing requirements becomes simple. Story Mapping keep people focused on users and their experiences with the product. In addition, the conversation among agile team members will be more vivid as well as effective due to the storytelling approach. Continuous management of requirements will be a natural activity because of the visual presentation (see Figure 43) of the prioritized user stories. The agile technique Story mapping is introduced by Patton (Patton, 2004), (Patton, 2014). In 2004, Patton (Patton, 2004) presented the idea but he did not call the technique Story Mapping at that time. In the following years, he noticed that other people applied a similar approach to solve problems concerning prioritization of requirements and losing sight of the big picture. Then, Patton (Patton, 2014) started realizing that he discovered a pattern. Example Figure 43: Example of a story map Template See Figure 43 117 A Framework for Modeling and Improving Agile Requirements Engineering Pattern name Product Owner Context Organizations want to maximize the return on invest (ROI) in order to achieve a continuous grow. With regard to product development, stakeholders have different desires in terms of product features or look and feel. These different desires need to be balanced in order to increase the value of a product. Tag Discovery, refinement, prioritization, review, roles Problem summary In ASD it is a challenge to involve stakeholders throughout the whole development process in regular iterations, so that product development will succeed (see C6, Table 21). • managing different interests of stakeholders is a challenge. • objectives of development and objectives of organization are sometimes disconnected. • receiving feedback is necessary for a good expectation management. Usage description Engaging stakeholder into product development iteratively by means of a Product Owner (Schwaber and Sutherland, 2016). The Product Owner is responsible for maximizing the value of the product. Therefore, he manages the Product Backlog by prioritizing requirements according to needs of the users and customers as well as regulatory and market requirements. He collaborates with the development team, stakeholders, users and management. Having an empowered Product Owner in places, facilitates the decision making process in terms of strategic product management. In addition, there is one contact person, who is responsible for managing the different expectations and desires. The agile technique Product Owner has its origin in Scrum (Schwaber, 2004). The idea behind the role is to have one single person, who is in charge of maximizing the value of the product and to be responsible for the overall quality of the product. Compared to the role of an Agile Coach or Scrum Master, who is the person in charge in terms of process quality. Example n.a. Template n.a. 118 A Framework for Modeling and Improving Agile Requirements Engineering Pattern name Review Meeting Context Receiving feedback from stakeholders and users in an iterative manner is very valuable for product development to succeed. Tag Discovery, refinement, prioritization, review, meetings Problem summary In ASD it is a challenge to work out user requirements and quality of use in cooperation with direct users (end users) of the product (see C5, Table 21). • receiving feedback from direct users of the product. • adapt direction of product development according to users´ needs. • refine requirements according to evolving needs and wishes of users. Usage description Holding regular Review Meetings supports receiving feedback from direct users of the product. Review Meetings (Schwaber and Sutherland, 2016) aim to inspect the product and to adapt the direction of product development to new learnings and insights. Attendees are affected people like developments team, stakeholders, users and management who collaborate in order to direct future product development. The Review Meeting can be organized like an exhibition, in order to receive feedback from a large group (Kraus, 2013). Attendees can try out new product features and receive additional information regarding the development progress. For this, every development team presents their objectives on a booth, where user stories with additional material like graphics, flow charts or prototypes are detailed (see example in Figure 44). Regular Review Meetings improve the feedback culture of an organization. The direction of product development can be adapted according to users´ and customers´ needs. Review Meetings provide space to involve users´ feedback into product development and requirements can be refined in cooperation with direct users. The concept of Review Meetings has its origin in Scrum, where the Sprint Review Meeting (Schwaber, 2004) is a necessary feature of the Scrum framework. The intention is to showcase the work of the previous Sprint and to inspect the delivered product increment as well as to adapt Product Backlog according to learnings and feedback. Example Figure 44: Review Meeting in form of an exhibition (Glase, 2016) Template n.a. 119 A Framework for Modeling and Improving Agile Requirements Engineering 6.4.3. Knowledge Sharing by Means of a Web Application The knowledge concerning agile RE presented by agile RE patterns will be distributed by means of a web application. Typically, agile practitioners do not have time to read full books due to their daily business. The knowledge needs to be presented in chunks in order to be communicated in an effective manner. To this end, we decided to share the knowledge regarding agile RE by means of agile RE patterns. The written agile RE patterns will be distributed by means of a web application (agileRE.org) (Schön et al., 2017c). The aim of agileRE.org is to support practitioners and researchers improving their agile RE process models as well as to support them in solving the key problems in agile RE by means of tool support. Figure 45 presents the landing page of agileRE.org. On the one hand, the user can browse through agile RE problems. On the other hand, the user can browse through agile RE patterns that are classified by their pattern type (artifacts, meetings, methods or roles, see Table 28). Figure 45: Landing page agileRE.org The web application is providing tool support for our framework by means of visualizing parts of the agile RE metamodel (see chapter 4) in order to provide techniques to apply the conceptual approach to the enterprise environment. To this end, it bridges the way to playback knowledge from theory to practice. We developed the web application in collaboration with a bachelor student. First, we had several workshops where we discussed the vision and build a shared understanding concerning the objectives of agileRE.org. Then, we analyzed existing patterns and pattern templates in order to get to know best practices in the field. The results were used as input for our collaborative prototyping workshops. Therefore, we used the prototyping tool balsamiq 6 for creating wireframes iteratively. Moreover, we created a template for agile RE patterns in order to present them in the same fashion. This has a positive effect on the readability from a user´s perspective. The analysis of existing pattern templates on different websites allows us to understand what expectations users 6 https://balsamiq.com 120 A Framework for Modeling and Improving Agile Requirements Engineering have and give guidance for designing the pattern template. Figure 46 shows how our agile RE patterns are presented on agileRE.org by means of the example Story Map. Compared to the presentation of the pattern Story Map in section 6.4.2, we enriched the patterns with additional information (e.g. section for downloads, relation to other patterns, agile RE activity). This enrichment of metadata is useful in terms of linking patterns with each other on a website. Figure 46: Example of agile RE pattern (Story Map) presented on agileRE.org 121 A Framework for Modeling and Improving Agile Requirements Engineering The visual design (e.g. colours, typo, icons, and logo) is adapted to requirements of the agile community. We decided to implement the web application on the basis of WordPress 7 since this guarantees a sustainable solution in terms of maintainability and extensibility. We have applied several methods in order to improve the web application continuously (e.g. UX testing or web analytic tools). In addition, our aim is to improve the agile RE patterns continuously. The content is currently written in German, translations will follow. 6.5. Chapter Summary This chapter presents the concept of agile RE patterns. For this purpose, we carried out a pattern mining process comprising three phases by means of empirical research in literature and industry. In the first phase, we identified agile techniques by means of a SLR. Then, we conducted an iterative expert judgement process with 26 experts in the field of ASD to identify the most important problems in agile RE. After that, we derived in sum 41 agile RE patterns. Therefore, we mapped the agile techniques to the agile RE problems. We showed an overview of the 41 identified agile RE patterns and presented examples of agile RE patterns: a) Evaluation and Testing, b) Story Map, c) Product Owner and d) Review Meeting. Moreover, we presented the web application agileRE.org in order to show how the knowledge concerning agile RE will be shared among the agile community. We can conclude that our agile RE patterns are highly relevant for the industry as well as the research community, since we gathered the data from experts in the field of ASD. Agile RE patterns enable practitioners and researchers to implement their agile RE process models and will therefore cause strong impact. With agile RE problems and agile RE patterns we provide techniques to apply our conceptual approach to the enterprise environment. 7 https://en.wordpress.com 128 A Framework for Modeling and Improving Agile Requirements Engineering Figure 50: Example of multiple Kanban boards for pattern Continuous management of requirements by means of tools It is worth mentioning that more than one of the presented agile RE problems occurred along the development of project case b. For instance, it was a problem not to lose sight of the big picture during the implementation of complex requirements since the development with Kanban is focused on small tasks. Therefore, the integration of HCD activities by means of a release evaluation was applied (Schön et al., 2016b). 7.3.2. Case B: Tailoring Within a Scrum-based Environment in Spain Scrum (Beck, 2000) is an agile methodology that can be classified as an time-box approach comprising a set of roles, meetings and artifacts. Since Scrum is described as a framework where people can address complex problems, we need to adapt the framework to the specific context of a project. This case presents an instantiation of the agile RE metamodel in a project using NDT-Agile (based on Scrum and Earned Value Management) to create a web application for e-government. The full description of the case study can be found in Sedeño (Sedeño, 2017). 7.3.2.1. Background of the Project The project was carried out at Junta de Andalucía, the regional government of the Spanish region of Andalusia. The Information and Communication Technologies (ICT) Department of the Ministry of Culture has led the project in liaison with other departments belonging to the regional government. In between 2008-2012, the Ministry of Culture transformed itself into an organization to be able to operate under SOA paradigm (Sedeño et al., 2014). In 2012, they started to adopt agile methodologies (Torrecilla-Salinas et al., 2013), (Torrecilla-Salinas et al., 2015). 7.3.2.2. Instantiation into Scrum-based Process Model In the following paragraphs, we will provide an instantiation of the metamodel (Figure 17). Moreover, we will discuss the way of working within the scope of case b. Figure 51 displays the domain specific model for agile RE in a Scrum-based environment for case b. For the creation of the instance, we used the profile in Enterprise Architect (see section 4.4.3). 129 A Framework for Modeling and Improving Agile Requirements Engineering Figure 51: Domain specific model for agile RE in a Scrum-based environment The methodology, which had been used in case b is NDT-Agile (based on Scrum and Earned Value Management). Scrum as well as NDT have an impact on how requirements are managed in case b. Requirements are managed by a Product Backlog and described in the form of user stories or using NDT. The domain of the organizational environment is related to the public sector and in particular, the area of e-government. The system in case b comprises all information systems involved in these web applications together with the services portfolio that belong to the organization. Moreover, the IT infrastructure is key part of the system. With regard to the stakeholder, we can describe them as people in charge of the different departments (e.g. ministry, IT or business). The users are citizens and public employees as well as the IT responsible or even the agile team. In light of this, the context of use is the Ministry of Culture that delivers egovernment services by means of web applications using agile software developments. An agile team belongs to the IT service and consists of one Scrum Master, agile developers, and one Product Owner, who is the deputy head of IT department. All the aforementioned information describes the organizational environment in terms of the project setting for case a. With regard to this analysis, we detected the agile RE problem technical or functional dependencies to other teams. In particular, the problem is related to the discovery of what services exist in the services portfolio, which are covering the user story proposed for a new web application. Moreover, this problem is related to software reusability in early stages of the software development lifecycle. The detected agile RE problem was solved by applying the agile RE pattern discovering services that covers an agile requirement. This pattern is categorized by the agile RE activities discovery and refinement. In the following, we will detail the pattern discovering services that cover an agile requirement which was applied in case b. 130 A Framework for Modeling and Improving Agile Requirements Engineering Pattern name Discovering services that cover an agile requirement Context Detecting existing services, which can be found in the service portfolio of the organization, that cover user stories functionality identified by the agile team. This pattern is used in early stages of a sprint, directly after the Sprint Planning and it is necessary for the software reutilization. The identification of existing services is a very difficult task since the services area is not connected with the agile techniques that are used in the requirements elicitation phase. Tag Discovery, refinement, methods Problem summary In ASD functional or technical dependencies with other teams are a challenge because a considerable coordination effort is required (see C1, Table 21). • discovery of services in early stages of ASD. • discovering what services exist in the services portfolio, which cover the user story proposed for a new web application. • reusability of software in early stages of the software development lifecycle. Usage description For applying the solution, the process named DS4aRE (Discovering services for agile requirements) by (Sedeño, 2017) is used: 1) Indexing the Services Portfolio. 2) Formalizing the agile requirement by means of the value-based agile requirements metamodel (Sedeño, 2017). 3) Transforming this formalization into a query. 4) Launching this query against the index. 5) Preparing the result of the query. 6) Obtaining the Candidate Services (i.e. a Service that is inside the Services Portfolio and would cover an agile requirement). Example The Ministry needs to develop a new web application to comply with Law 39/2015 that obliges to make any notification by telematics means. The Ministry needs to know how many services of the existing e-government services portfolio could be used in this application to cover the user stories identified by the agile team. Template DS4aRE process (Sedeño, 2017) 131 A Framework for Modeling and Improving Agile Requirements Engineering 7.3.3. Impact and Limitations The application of the agile RE metamodel allows us to reflect the agile RE problems, which occurred in the presented cases (see section 7.3.1 and section 7.3.2) as well as to find appropriate solutions for handling these problems. Analyzing the two cases, we can observe some differences in terms of the elaboration of information related to specific stereotypes (see Figure 49, Figure 51). For instance, the context of use is deeper explored and more elaborated in case a (see Figure 49). This effect occurs because of the integration of HCD. In HCD, the context of use is very important in terms of user research. Taking this as an example, we can discuss whether it makes sense to put some effort in the analysis of the context of use in case b and discuss what impact this analysis might have on the value delivery. Further differences between the two domain specific models can be found in the manner of how people are related to each other. For instance in case b (Figure 51), the user is part of the agile team, since some member of the agile team use the system as citizens or employers of the Ministry of Culture. Another difference between both domain specific models can be observed in the relation between the methodology and the agile RE pattern. In case a, the agile RE pattern is part of the methodology, since the agile technique Kanban board, is rooted in the methodology Kanban. In comparison, the agile technique in case b (DS4aRE process), is a new technique, which is not part of a methodology. The profile is capable of presenting more details in the resulting domain specific models as displayed in Figure 49 and Figure 51. For instance, all attributes of the metaclasses from the agile RE metamodel (see Figure 17) can be displayed with their corresponding values. For both presented cases in this PhD thesis, we decided to hide this feature, because it is not necessary to show the details for understanding the application of the framework. In general, the longer the framework is used, the more mature becomes the organizational environment and more information in terms of details of the domain specific model will be clarified. With regard to the presented cases in this work, there are some shortcomings in terms of the analysis of the organizational environment. We have to state that we were not able to gather all the data the metamodel is capable of due to the time, when the case studies were carried out. The final version of the metamodel and the profile were created in July 2017, whereas the case studies were conducted before this time. This fact causes us some difficulties in applying the whole framework accurately. Nevertheless, we started new case studies, where we are going to apply the full framework in an international environment where the product development takes place with people working in different countries. However, what we have learned from both cases is that the domain specific models allow us to receive a detailed view on complex organizational environments. This detailed view enables us to identify the problems and impediments that occur in terms of people, processes, or tools. In addition, the detailed view supports us to compare specific excerpts of different organizational environment with the aim to reflect what went well and what should be shared. In light of this, the combination of agile RE problem and corresponding solution can be inferred to new agile RE patterns. These agile RE patterns facilitate sharing the experience with the 132 A Framework for Modeling and Improving Agile Requirements Engineering community. The gathered knowledge can help other people to solve their own agile RE problems, which may be similar to the presented ones. Moreover, we were able to demonstrate that the framework for modeling agile RE is applicable on an international level and brings benefits for the industry. 7.4. Chapter Summary In this chapter we showed how the framework for modeling agile RE can be applied in industry and how the different components of the framework collude with each other. Moreover, we explained the research method we followed to receive domain specific models for two cases. The first domain specific model for agile RE (case a) presents a Kanban-based process model for developing an internet-based newspaper portal in the area of e-commerce. The case study was conducted in Germany. In this case, we detected the agile RE problem Continuous requirements management. This problem was solved by applying the agile RE pattern Continuous management of requirements by means of tools. The second domain specific model for agile RE (case b) highlights a process model for a Scrum-based process regarding the development of web applications in the e-government area. This case study was located in Spain. In case b, we detected the agile RE problem Technical or functional dependencies to other teams, which was solved by using the agile RE pattern Discovering services that cover an agile requirement. Moreover, we discussed the benefits and difficulties, which occur while applying the framework for modeling agile RE. What we can conclude is that the framework for modeling agile RE can be applied on an international level. In addition it brings benefits for the industry since it allows to analyze the organizational environment of a company and helps to improve it in terms of value delivery and collaboration. After presenting how the framework for modeling agile RE is applied in industry, the following chapter will summarize the results of this PhD thesis together with the conclusion. 133 A Framework for Modeling and Improving Agile Requirements Engineering Chapter 8 Results, Future Work and Conclusions ith this PhD thesis, we contribute a framework for modeling and improving value delivery in agile environments. The framework allows improving the agile environment within organizations by means of a conceptual approach as well as techniques to apply the conceptual approach. The framework for modeling agile RE has been built by an inductive (bottom up) research approach, where we used the learnings from our research studies in order to create the components of the framework. In particular, we analyzed the state of the art of agile RE with focus on stakeholder and user involvement. Based on this knowledge, we contribute the agile RE metamodel, which allows us to build a high level view on the complex field of agile RE. In addition, we created a profile. Then, we identified the most important problems agile practitioners have to face in industry in terms of agile RE. With regard to our conceptual approach, we analyzed solutions for handling these agile RE problems. Moreover, we created agile RE patterns, which are distributed by means of a web application (agileRE.org) in order to make the framework accessible for the community and to share the knowledge about agile RE. In this chapter we will summarize the results of the PhD thesis. In addition, we will outline the identified future lines of research and the relations to the work of our research group (IWT2). Finally, the main conclusions of this work will be presented. 8.1. Results Agility is increasing throughout organizations. Agile methodologies and agile techniques are used for product development across almost all industries. People in agile environments strive for continuous improvement, which leads to a rapidly evolving knowledgebase in the field of ASD. Agile techniques like continuous delivery (Humble and Farley, 2010) or DevOps (Hüttermann, 2012) are changing the way software is developed and delivered from a technical point of view. However, requirements are often managed in a plan-driven manner although agile techniques like story mapping (Patton, 2014) or impact mapping (Adzic, 2012) are in place. Organizations still struggle with improving their processes systematically in order to increase value delivery and collaboration. The main contribution of this PhD thesis is a framework for modeling agile RE. The framework consists of three main components: • The agile RE metamodel, which allows analyzing the environment of an organization, where the product development takes place. • A catalogue of agile RE problems, which supports detecting common problems that occur in terms of agile RE. • A catalogue of agile RE patterns, which can be used to improve the environment and support solving the identified agile RE problems. W 134 A Framework for Modeling and Improving Agile Requirements Engineering The framework provides several benefits for organizations. On the one hand, organizations can use the framework for increasing their value delivery. Value delivery is an organization external parameter, which is related to the outcome of a product in terms of users´ and customers´ needs. The people working in an organization need to define what value means and need to find appropriate KPIs for measuring the value. In this context, they need to leave room for iterating on the definition of value. On the other hand, organizations can utilize the framework to improve the collaboration among people working in the environment. Collaboration is an organization internal parameter, which is related to how people feel during their work as well as on the effectiveness of product development. The presented framework enables agile practitioners as well as researchers to improve their environments in a systematic manner. In particular, the framework is suitable for complex environments, where a lot of different people with different roles are involved or hybrid models for product development are applied as shown in the case studies. With regard to the vision of this PhD, which was decomposed into four main objectives (see section 3.2), the following results have been achieved: • An analysis of the state of the art of agile RE with focus on stakeholder and user involvement. • A conceptual approach for agile RE management by means of the agile RE metamodel and two examples of domain specific models, which show how the metamodel can be applied in industry. The domain specific models are created by means of a profile for the agile RE metamodel. • Techniques to apply the conceptual approach to the enterprise environment by means of agile RE problems and agile RE patterns. • Tool support by means of a web application (agileRE.org) We analyzed the state of the art of agile RE with focus on user and stakeholder involvement by means of a SLR (Schön et al., 2017a). Therefore, the most relevant related work regarding the relation among Agile, HCD, and RE has been presented. The state of the art study helped us to achieve deep insights into the field of agile RE, in particular in terms of: • presence of the user perspective • stakeholder and user involvement • integration of methodologies to make ASD more human-centered • building a shared understanding • data gathering • non-functional requirements • documentation of requirements by means of artifacts Based on this study, we were able to identify gaps in existing research, which helped us to target our research. The available literature shows that there are still challenges regarding the iterative involvement of user and stakeholder in the development process as well as building a shared understanding of the user perspective. We found a variety of methods and artifacts, which are used in ASD for different purposes in a specific context. However, guidelines for choosing the right kind of artifacts that enhance collaboration among all people involved in the development process are missing. In a next step, we created the agile RE metamodel, aiming to build an abstraction layer about the variety of existing process models for agile RE in the field. The initial version of the agile RE 135 A Framework for Modeling and Improving Agile Requirements Engineering metamodel is inferred by an analysis of related process models for agile RE and experiences working in industry projects (Schön, 2016). We improved the metamodel iteratively by means of discussions with other researchers and further empirical studies [Ref]. The agile RE metamodel builds the heart of our framework and shapes the conceptual approach for agile RE management. It allows us to reflect the dependencies among people and product development process and supports improvement by inspection and adaptation. The metamodel has several implications for both, researchers and practitioners. On the one hand, the metamodel can be used to model new process models for agile RE that are strongly focused on value delivery and therefore can be classified as value-driven process models. On the other hand, existing process models can be evaluated as well as improved by the metamodel. Moreover, we contribute a visual language for the agile RE metamodel by means of a profile. The profile allows us to extend the metaclasses of the agile RE metamodel with the aim to tailor domain specific models. Subsequent to the definition of our conceptual approach, we contribute techniques (a catalogue of agile RE problems and a catalogue of agile RE patterns) that allow us to apply the conceptual approach to the enterprise environment. For creating the catalogue of agile RE problems, we carried out an iterative expert judgement process comprising three complementary rounds (Schön et al., 2017d). Our panel consisted of 26 experts in the field of ASD working for 19 different companies. During these rounds we find the core of a specific problem by abstracting contextual information of the problems. The learnings from previous iterations were used for carrying out the following ones. In sum, we identified 20 problems industry has to address at present in terms of agile RE. Six of these problems have been defined as key problems: • technical or functional dependencies to other teams (C1) • understanding of agile values of the stakeholders (C2) • staying focused on the big picture (C3) • continuous requirements management (C4) • refine requirements in collaboration with users (C5) • involve stakeholder iteratively (C6) A further result of this study is the analysis of options to deal with those key problems by means of agile techniques, which are recommended by the panel. These results supported us in creating the catalogue of agile RE patterns. The agile RE problems and agile techniques allow us to create agile RE patterns. As part of the results of this PhD thesis, we introduced the concept of agile RE patterns (Schön et al., 2017b). For creating the agile RE patterns, we carried out a systematic pattern mining process comprising three phases. In the first phase, we collected the agile techniques identified by our SLR. Then, we combined these results with the ones from our iterative expert judgement process. Finally, we mapped the results in order to create agile RE patterns. An agile RE pattern is composed of an agile RE problem and appropriate agile techniques, which can be utilized to solve the problem. With this mapping, we derived in sum 41 agile RE patterns. In this work, we showed an overview of the 41 identified agile RE patterns and presented examples: a) Evaluation and Testing, b) Story Map, c) Product Owner and d) Review Meeting. 136 A Framework for Modeling and Improving Agile Requirements Engineering The identified agile RE patterns and the knowledge concerning agile RE will be shared among the community by means of a web application (agileRE.org). agileRE.org aims to provide tool support for our framework. For a better understanding of our conceptual approach, we showed how the framework for modeling agile RE can be applied in industry by means of domain specific models. To this end, we provided two examples of instantiations of the metamodel. Firstly, we exemplify a process model for a Kanban-based process model for developing an internet-based newspaper portal within the ecommerce area. This case detected the agile RE problem Continuous requirements management. This problem is solved by applying the agile RE pattern Continuous management of requirements by means of tools. Secondly, we derive an instantiation of the agile RE metamodel into a Scrum-based process regarding the development of web applications in the e-government area. This case detected the agile RE problem Technical or functional dependencies to other teams, which is solved by using the agile RE pattern Discovering services that cover an agile requirement. Summarizing the implications of this PhD thesis, we can state that we identified evidence in industry by means of identifying real world problems in the context of agile RE. The problems were generalized by reducing the contextual information of the environment. With regard to our conceptual approach for agile RE management we can learn to understand the dependencies and to reflect problems in a specific environment. In this context, the agile RE metamodel allows to analyze the environment within an organization and supports adding contextual information according to the specific needs. This is a necessary task in order to understand the conditions for selecting an appropriate agile technique, with which the detected problem can be solved. Besides, agile RE patterns help to share the knowledge and experiences with problem solving in the context of agile RE among the community. 8.2. Future Work This PhD work has opened new lines of future research, which will be outlined in the next paragraphs. In the scope of this PhD thesis, we showed how our framework can be utilized for improving domain specific agile RE process models in the area of e-government and e-commerce (brownfield approach). In the next years, we want to apply the framework on a greenfield approach, where we can use it to model new process models for agile RE according to needs of an organization. In industry, it is not so easy to find an appropriate case, since most of the companies already have a process model in place and it is hard to find an environment where we can start from scratch. Furthermore, we are going to apply the framework to other domains since this will allow us to learn more about the boundaries of the framework in a real world context. In a next step, we want to create a marketable version of the framework so that the value of the framework becomes more handy to industry. First companies indicate that they are highly interested in cooperation and applying the framework. Moreover, future research may specifically focus on integrating further tools that support the semi-automatic analysis of requirements in an agile environment, similar to NDT (Escalona and Aragon, 2008) which is used for automatic analysis of requirements in sequential approaches to RE. Another thing to highlight is that we want to enrich the identified agile RE patterns by means of further empirical research in industry. Some of the patterns had been applied in industry, whereas 137 A Framework for Modeling and Improving Agile Requirements Engineering others are composed during the work of this thesis and therefore are not fully evaluated. Although our agile RE patterns are a combination of best practices known from agile methodologies, we want to evaluate most of them. Those patterns, which have not been evaluated in case studies, can be classified as proto patterns. Our aim is to gain more experience with the agile RE patterns and to learn how they can support organizations in terms of becoming a value-driven culture. Moreover, we want to create a pattern language based on our learnings from the application of the framework for modeling agile RE. We started to carry out further studies in order to learn more about the identified agile RE problems. First workshops among UX experts showed, that the discussions about the agile RE problems enable collaboration in solution finding among cross-functional teams (Schön and Thomaschewski, 2017). The results of our iterative expert judgement process reveal that the identified agile RE problems are not limited to ASD. To this end, another important future line of research is to investigate in what degree the agile RE problems are applicable to RE in general. These results may offer new opportunities to extend our framework. In this context, a cooperation with the research group of the NaPiRE surveys (Méndez Fernández et al., 2017) is planned. Moreover, we learned that patterns are a good way to playback results from research to industry. In future work, we want to investigate what more options do exist to bridge the gap between research and industry and figure out new ways how research can be applied in companies. 8.3. Relation to IWT2 research group At this point, we want to outline the relation of this work to the research work of the IWT2 research group. In general, RE and ASD have been lines of research of the IWT2 group for several years. The PhD work of Dr. D. Jorge Sedeño López (Sedeño, 2017), supervised by Dr. María José Escalona Cuaresma and Dr. Manuel Mejías Risoto, is focused on discovering services in early stages of ASD and contributes a process for the formalization of requirements in an agile environment. The formalization of requirements is carried out by agile techniques like a Product Backlog or user stories within the NDT-Agile framework. The formalized requirement can be mapped to a catalog of existing services in order to evaluate, which services already exist that cover the requirement. To this end, a metamodel is defined, which allows formalizing a requirement. The activity of mapping requirements to an existing catalog of services is an important activity in terms of the reusability of software components. The PhD work of Dr. Carlos J. Torrecilla Salinas (Salinas Torrecilla, 2016), supervised by Dr. María José Escalona Cuaresma and Dr. Manuel Mejías Risoto, contributes a framework named NDT-Agile. The aim of NDT-Agile is to provide a mature agile framework for web development projects. NDT-Agile combines the following components: a lifecycle, a set of complementary techniques, and a governance model. The lifecycle is based on Scrum and additional elements to be compliant to CMMI-DEV. The usage of CMMI-DEV increases process improvement and leads to a mature agile development process. The complementary techniques are for instance project subcontracting or project reporting. The governance model provides information regarding continuous improvement or quantitative-managed project management. After presenting the results of this PhD work, highlighting the future lines of research and outlining the relations to the IWT2 research group, the following section will state the main conclusions. 144 A Framework for Modeling and Improving Agile Requirements Engineering Magues, D.A., Castro, J.W., Acuna, S.T., 2016. HCI usability techniques in agile development, in: 2016 IEEE International Conference on Automatica (ICA-ACCA). IEEE, pp. 1–7. doi:10.1109/ICAACCA.2016.7778513 Maguire, M., 2013. Using human factors standards to support user experience and agile design, in: Proceedings - International Conference, UAHCI 2013, Held as Part of HCI International 2013, Las Vegas, NV, USA. pp. 185–194. doi:10.1007/978-3-642-39188-0_20 Magureanu, G., Gavrilescu, M., Pescaru, D., Jian, I., 2012. UML profile for Cyber-Physical System wireless communication specification, in: 2012 7th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI). IEEE, pp. 383–388. doi:10.1109/SACI.2012.6250034 Mahmud, I., Veneziano, V., 2011. Mind-mapping: An effective technique to facilitate requirements engineering in agile software development, in: 14th International Conference on Computer and Information Technology (ICCIT 2011). IEEE, pp. 157–162. doi:10.1109/ICCITechn.2011.6164775 Memmel, T., Gundelsweiler, F., Reiterer, H., 2007. Agile Human-Centered Software Engineering. Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI...but not as we know it - Volume 1 167–175. doi:10.1177/004057368303900411 Méndez Fernández, D., Lochmann, K., Penzenstadler, B., Wagner, S., 2011. A Case Study on the Application of an Artefact-Based Requirements Engineering Approach. 15th Annual Conference on Evaluation & Assessment in Software Engineering (EASE) 104–113. doi:10.1049/ic.2011.0013 Méndez Fernández, D., Penzenstadler, B., Kuhrmann, M., Broy, M., 2010. A meta model for artefactorientation: Fundamentals and lessons learned in requirements engineering. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 6395 LNCS, 183–197. doi:10.1007/978-3-642-16129-2_14 Méndez Fernández, D., Wagner, S., Felderer, M., Kalinowski, M., 2017. Naming the pain in requirements engineering (NaPiRE) [WWW Document]. URL http://www.mendezfe.org/projects/napire/ (accessed 6.8.17). Méndez Fernández, D., Wagner, S., Kalinowski, M., Felderer, M., Mafra, P., Vetr?, A., Conte, T., Christiansson, M.-T., Greer, D., Lassenius, C., M?nnist?, T., Nayabi, M., Oivo, M., Penzenstadler, B., Pfahl, D., Prikladnicki, R., Ruhe, G., Schekelmann, A., Sen, S., Spinola, R., Tuzcu, A., de la Vara, J.L., Wieringa, R., 2016. Naming the pain in requirements engineering. Empirical Software Engineering 1–41. doi:10.1007/s10664-016-9451-7 Méndez Fernández, D., Wieringa, R., 2013. Improving Requirements Engineering by Artefact Orientation, in: Heidrich J., Oivo M., Jedlitschka A., B.M.T. (Ed.), Product-Focused Software Process Improvement. PROFES 2013. Lecture Notes in Computer Science, Vol 7983. pp. 108–122. doi:10.1007/978-3-642-39259-7_11 Näkki, P., Koskela, K., Pikkarainen, M., 2011. Practical model for user-driven innovation in agile software development, in: Proceedings of the 2011 17th International Conference on Concurrent Enterprising (ICE 2011). IEEE, pp. 1–8. Nawrocki, J., Ochodek, M., Jurkiewicz, J., Kopczyńska, S., Alchimowicz, B., 2014. Agile requirements engineering: A research perspective, in: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). pp. 40–51. doi:10.1007/978-3-319-04298-5-5 Newman, P., Ferrario, M.A., Simm, W., Forshaw, S., Friday, A., Whittle, J., 2015. The Role of Design Thinking and Physical Prototyping in Social Software Engineering, in: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering. IEEE, pp. 487–496. doi:10.1109/ICSE.2015.181 Obendorf, H., Finck, M., 2008. Scenario-based usability engineering techniques in agile development processes, in: Proceeding of the Twenty-Sixth Annual CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI ’08. ACM Press, New York, New York, USA, p. 2159. doi:10.1145/1358628.1358649 Object Management Group Inc. (OMG), 2015. OMG Unified Modeling Language (v2.5). 145 A Framework for Modeling and Improving Agile Requirements Engineering Olsson, H.H., Bosch, J., 2015. Towards Continuous Customer Validation: A Conceptual Model for Combining Qualitative Customer Feedback with Quantitative Customer Observation. LNBIP 210, 154–166. doi:10.1007/978-3-319-19593-3_13 Palmer, S.R., Felsing, M., 2001. A Practical Guide to Feature-Driven Development. Pearson Education. Patton, J., 2014. User Story Mapping: Discover the Whole Story, Build the Right Product, First edit. ed. O´Reilly. Patton, J., 2004. Interaction Design Meets Agility: Practicing Usage Centered Design on Agile Development Projects, in: Zannier, C., Erdogmus, H., Lindstrom, L. (Eds.), Lecture Notes in Computer Science. Berlin, p. 226. Pfeiffer, T., Hellmers, J., Schön, E.-M., Thomaschewski, J., 2016. Empowering User Interfaces for Industrie 4.0. Proceedings of the IEEE 104, 986–996. doi:10.1109/JPROC.2015.2508640 Pohl, K., 2010. Requirements Engineering: Fundamentals, Principles, and Techniques, 1st ed. Springer Publishing Company, Incorporated. Ramesh, B., Cao, L., Baskerville, R., 2010. Agile requirements engineering practices and challenges: an empirical study. Information Systems Journal 20, 449–480. doi:10.1111/j.1365- 2575.2007.00259.x Richter, M.D., Mason, J.D., Alford, M.W., Burns, I.F., Helton, H.A., 1976. Software Requirements Engineering Methodology. Rivero, J.M., Grigera, J., Rossi, G., Robles Luna, E., Montero, F., Gaedke, M., 2014. Mockup-Driven Development: Providing agile support for Model-Driven Web Engineering. Information and Software Technology 56, 670–687. doi:10.1016/j.infsof.2014.01.011 Rolland, C., 1993. Modeling the requirements engineering process, in: 3rd European-Japanese Seminar on Information Modelling and Knowledge Bases Budapest, Hungary. doi:10.1.1.29.8738 Royce, W.W., 1970. Managing the development of large software systems, in: IEEE WESCON. pp. 1–9. doi:10.1016/0378-4754(91)90107-E Salah, D., 2011. A framework for the integration of user centered design and agile software development processes. Proceeding of the 33rd international conference on Software engineering - ICSE ’11 1132. doi:10.1145/1985793.1986017 Salah, D., Paige, R., Cairns, P., 2016. A Maturity Model for Integrating Agile Processes and User Centred Design, in: Communications in Computer and Information Science. pp. 109–122. doi:10.1007/978-3-319-38980-6_9 Salah, D., Paige, R., Cairns, P., 2015. Patterns for integrating agile development processes and user centred design. EuroPLoP ’15 Proceedings of the 20th European Conference on Pattern Languages of Programs 1–10. doi:10.1145/2855321.2855341 Salah, D., Paige, R.F., Cairns, P., 2014. A Systematic Literature Review for Agile Development Processes and User Centred Design Integration, in: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. p. 5:1--5:10. doi:10.1145/2601248.2601276 Salinas Torrecilla, C.J., 2016. A mature Agile approach in Web Engineering contexts. University of Seville. Scaled Agile Inc., 2017. Scaled Agile Framework – SAFe for Lean Software and System Engineering [WWW Document]. URL http://www.scaledagileframework.com/ (accessed 5.15.17). Schön, E.-M., 2016. A Methodology for Agile Requirements Engineering based on a Pattern Approach, in: Proceedings of CAiSE 2016 Doctoral Consortium. Ljubljana, Slovenia, pp. 1–8. Schön, E.-M., Thomaschewski, J., 2017. Werkstattgespräch zum agilen Produktentwicklungsprozess. Hamburg. doi:10.13140/RG.2.2.23627.18728 Schön, E.-M., Thomaschewski, J., Escalona, M.J., 2017a. Agile Requirements Engineering: A Systematic Literature Review. Computer Standards & Interfaces 49, 79–91. doi:http://dx.doi.org/10.1016/j.csi.2016.08.011 146 A Framework for Modeling and Improving Agile Requirements Engineering Schön, E.-M., Thomaschewski, J., Escalona, M.J., 2017b. Identifying Agile Requirements Engineering Patterns in Industry by Means of Empirical Research, in: 22nd European Conference on Pattern Languages of Programs. EuroPLoP. Schön, E.-M., Thomaschewski, J., Escalona, M.J., 2017c. agileRE.org [WWW Document]. URL www.agileRE.org Schön, E.-M., Thomaschewski, J., Escalona, M.J., 2016a. Agile Requirements Engineering - Protocol for a Systematic Literature Review [WWW Document]. doi:10.13140/RG.2.2.16615.85923 Schön, E.-M., Winter, D., Escalona, M.J., Thomaschewski, J., 2017d. Key Challenges in Agile Requirements Engineering, in: Baumeister, H., Lichter, H., Riebisch, M. (Eds.), XP 2017, LNBIP 283. pp. 37–51. doi:10.1007/978-3-319-57633-6_3 Schön, E.-M., Winter, D., Thomaschewski, J., Escalona, M.J., 2017e. Results of “Challenges in Agile Requirements Engineering” (Round 1). doi:10.13140/RG.2.2.34571.28961 Schön, E.-M., Winter, D., Thomaschewski, J., Escalona, M.J., 2017f. Results of “Challenges in Agile Requirements Engineering” (Round 2). doi:10.13140/RG.2.2.32893.56802 Schön, E.-M., Winter, D., Thomaschewski, J., Escalona, M.J., 2017g. Results of “Challenges in Agile Requirements Engineering” (Round 3). doi:10.13140/RG.2.2.16116.35201 Schön, E.-M., Winter, D., Uhlenbrok, J., Escalona, M.J., Thomaschewski, J., 2016b. Enterprise Experience into the Integration of Human-Centered Design and Kanban, in: Proceedings of the 11th International Joint Conference on Software Technologies (ICSOFT 2016). Lisbon, Portugal, pp. 133–140. Schön, E.M., Escalona, M., Thomaschewski, J., 2015. Agile Values and Their Implementation in Practice. International Journal of Interactive Multimedia and Artificial Intelligence 3, 61. doi:10.9781/ijimai.2015.358 Schrepp, M., Hinderks, A., Thomaschewski, J., 2014. Applying the User Experience Questionnaire (UEQ) in Different Evaluation Scenarios, in: Springer International Publishing (Lecture Notes in Computer Science). pp. 383–392. doi:10.1007/978-3-319-07668-3_37 Schwaber, K., 2004. Agile Project Management with Scrum. Microsoft. Schwaber, K., 1997. SCRUM Development Process, in: Sutherland, J., Casanave, C., Miller, J., Patel, P., Hollowell, G. (Eds.), Business Object Design and Implementation. London. doi:10.1007/978-1- 4471-0947-1_11 Schwaber, K., Sutherland, J., 2016. Scrum Guide [WWW Document]. URL http://www.scrumguides.org/scrum-guide.html (accessed 11.22.16). Sedeño, J., 2017. Una propuesta para el descubrimiento sistemático de servicios en fases tempranas de desarrollos ágiles. University of Seville. Sedeño, J., Schön, E.-M., Torrecilla-Salinas, C., Thomaschewski, J., Escalona, M.J., Mejias, M., 2017a. Modeling Agile Requirements using Context-based Persona Stories, in: Proceedings of WEBIST the 13th International Conference on Web Information Systems and Technologies. SCITEPRESS - Science and Technology Publications. pp. 196–203. Sedeño, J., Schön, E.-M., Torrecilla-Salinas, C., Thomaschewski, J., Escalona, M.J., Mejias, M., 2017b. Modeling Agile Requirements using Context-based Persona Stories, in: 13th International Conference on Web Information Systems and Technologies (WEBIST). pp. 196–203. doi:10.5220/0006220301960203 Sedeño, J., Torrecilla-Salinas, C.J., Escalona, M.J., Mejías, M., 2014. An approach to transform public administration into SOA-based organizations. WEBIST 2014 - Proceedings of the 10th International Conference on Web Information Systems and Technologies 1, 135–142. doi:10.5220/0004794701350142 Silva da Silva, T., Martin, A., Maurer, F., Silveira, M., 2011. User-Centered Design and Agile Methods: A Systematic Review, in: 2011 AGILE Conference. IEEE, pp. 77–86. doi:10.1109/AGILE.2011.24 Silva da Silva, T., Selbach Silveira, M., Maurer, F., Hellmann, T., 2012. User Experience Design and Agile 147 A Framework for Modeling and Improving Agile Requirements Engineering Development: From Theory to Practice. Journal of Software Engineering and Applications 5, 743– 751. doi:10.4236/jsea.2012.510087 Silva Da Silva, T., Silveira, M.S., Maurer, F., 2015. Usability evaluation practices within agile development. Proceedings of the Annual Hawaii International Conference on System Sciences 2015–March, 5133–5142. doi:10.1109/HICSS.2015.607 Soares, H.F., Alves, N.S.R., Mendes, T.S., Mendonca, M., Spinola, R.O., 2015. Investigating the Link between User Stories and Documentation Debt on Software Projects, in: 2015 12th International Conference on Information Technology - New Generations. IEEE, pp. 385–390. doi:10.1109/ITNG.2015.68 Sommerville, I., Sawyer, P., 1997. Requirements Engineering: A Good Practice Guide, 1st ed. John Wiley & Sons, Inc., New York, USA. Sowizral, H.A., 1987. Design Methodology for Object-Oriented Programming, in: OOPSLA 87 Panel Session. pp. 91–95. Sutherland, J., 2001. Agile Can Scale: Inventing and Reinventing SCRUM in Five Companies. Cutter IT Journal 14, 5–11. Takeuchi, H., Nonaka, I., Takeuchi, H., 1986. The new new product development game. Harvard Business Review 64, 137–146. doi:10.1016/0737-6782(86)90053-6 The LeSS Company B.V., 2017. LeSS Framework - Large Scale Scrum (LeSS) [WWW Document]. URL https://less.works/less/framework/index.html (accessed 5.15.17). Torrecilla-Salinas, C.J., Sedeño, J., Escalona, M.J., Mejías, M., 2015. Estimating, planning and managing Agile Web development projects under a value-based perspective. Information and Software Technology 61, 124–144. doi:10.1016/j.infsof.2015.01.006 Torrecilla-Salinas, C.J., Sedeño, J., Escalona, M.J., Mejías, M., 2013. Using an Agile framework to deliver e - Government services in public administrations, in: 21nd Annual Software Quality Management (London, 2-5, Sep, 2013). SQM2013. VersionOne, 2016. 10th Annual State of Agile Report. doi:10.1093/jicru/ndl025 VersionOne Inc., 2017. 11th Annual State of Agile Report. Wagner, S., Méndez Fernández, D., Felderer, M., Kalinowski, M., 2017. Requirements Engineering Practice and Problems in Agile Projects: Results from an International Survey. CIbSE - 20 Ibero- American Conference on Software Engineering. Wanderley, F., Silva, A., Araujo, J., Silveira, D.S., 2014. SnapMind: A framework to support consistency and validation of model-based requirements in agile development, in: 2014 IEEE 4th International Model-Driven Requirements Engineering Workshop (MoDRE). IEEE, pp. 47–56. doi:10.1109/MoDRE.2014.6890825 Wellhausen, T., Fiesser, A., 2011. How to write a pattern?: a rough guide for first-time pattern authors. In Proceedings of the 16th European Conference on Pattern Languages of Programs, New York, NY, USA: ACM (p. 5:1-5:9). doi:10.1145/2396716.2396721 Williams, L., 2012. What agile teams think of agile principles. Communications of the ACM 55, 71–76. doi:10.1145/2133806.2133823 148 A Framework for Modeling and Improving Agile Requirements Engineering Appendix I Glossary Word Abbreviation Description Agile - Umbrella term for a mindset which is based on agile values, agile methodologies and agile techniques and tools Agile RE activity - Activity that is carried out in terms of RE in an agile environment (see Table 1) Agile RE pattern - Combination of agile RE problem and agile techniques, with which the problem can be solved Agile RE problem - Problem that occurs in an agile environment in terms of RE Agile team - Product development team including roles like developer, UX designer, tester, Agile Coach, Scrum Master or Product Owner Agile Requirements Engineering Agile RE Requirements Engineering tailored to an agile environment Artifact - Output that is generated while using a method. Context of use - Users, tasks, equipment (hardware, software and materials), and the physical and social environments in which a product is used. (International Organization for Standardization, 2010) Domain - Specific area or subject that the product is being developed for. Functional requirement FR Describing features of an application or a component, which support the user in carrying out tasks. Human-Centered Design HCD Refers to ISO 9241-210:2010 - Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems (International Organization for Standardization, 2010) Impact - Effect or influence resulting from a cause Meeting - People come together for a specific purpose (often timeboxed) Method - Procedure that is used in order to achieve an objective Methodology - A system of methods used in a particular area of study or activity Navigational Development Techniques NDT Model-driven approach to deal with requirements in Web systems (Escalona and Aragon, 2008) Non-Functional requirement NFR Describing constraints of a system, which affect different functional requirements, for instance security, performance, reliability. Organizational environment - Surroundings or conditions in which the product development takes place. Requirements Engineering RE Combination of all activities concerning discovering, documenting, and maintaining a set of requirements Role - Description of tasks and responsibilities that are related to a specific person 149 A Framework for Modeling and Improving Agile Requirements Engineering Stakeholder - Individual or organization having a right, share, claim or interest in a system or in its possession of characteristics that meet their needs and expectations (International Organization for Standardization, 2010), for instance management, sales, marketing, or customer System - Combination of hardware, software and/or services User - Person who interacts with the product. (International Organization for Standardization, 2010) User-Centered Design UCD Refers to ISO 13407:1999 - Human-centred design processes for interactive systems (International Organization for Standardization, 1999) 150 A Framework for Modeling and Improving Agile Requirements Engineering Appendix II Protocol of SLR Objectives Agile Software Development (ASD) is used to cope with increasing complexity in system development. Hybrid development models, with the integration of User-Centered Design (UCD), are applied with the aim to deliver competitive products with a suitable User Experience (UX). Therefore, stakeholder and user involvement during Requirements Engineering (RE) are essential in order to establish a collaborative environment with constant feedback loops. The aim of this Systematic Literature Review (SLR) is to investigate what approaches exist to involve stakeholder in the process, which methodologies are accepted to present the user perspective and how requirements management is been carried out. Our main goal is to clarify the state of the art in the field of Agile Requirements Engineering (RE) in order to identify the existing basis for the research of the PhD thesis. In addition, we want to find out where the research fits into the current body of knowledge. To this end, we are going to conduct an SLR following appropriate guideline by Kitchenham and Charters (Kitchenham and Charters, 2007). Research Questions To define the aims and the scope of this SLR, we created three research questions (RQ) with the following sub-criteria: RQ1: What approaches exist, which involve stakeholders in the process of RE and are compatible with ASD? On one hand, we analyzed whether the existing approaches involve stakeholders and users directly into the development process. On the other hand, our aim was to study whether the approaches apply a process model for the involvement. In addition, we queried what kind of methods they use in order to gather data. With regard to the agility of the existing approaches, we examined whether there are iterations along the development process. RQ2: Which agile methodologies, which are capable of presenting the user perspective to stakeholders, can be found? Concerning this RQ, we analyzed the included studies in terms of methodologies that are used to handle the user perspective within agile environments. Furthermore, we investigated how the knowledge of user requirements is shared among stakeholders. RQ3: What are the common ways for requirements management in ASD? In terms of this RQ, our aim was to investigate what types of artifacts are used and how they are utilized. Moreover, we wanted to discover whether the documentation of requirements is understandable without further knowledge in order to be able to work in a collaborative manner. In addition, we examined the treatment of non-functional requirements. Search Process • Manual search process with search string • Adapt search string for every digital library 151 A Framework for Modeling and Improving Agile Requirements Engineering • Use reference search (Snowballing) o Forward snowballing (search in papers that cited the paper) o Backward snowballing (search in the reference list of the paper) • Search process is divided into the following phases: Figure 52: Search process Search Space The following digital libraries are taken into account: Table 30: Search space Digital library Search strategy Google scholar full text Science direct abstract, title and keywords SpringerLink abstract and keywords Scopus abstract, title and keywords IEEEXplore abstract and keywords ACM abstract and keywords Search Criteria • Define keywords for search • Add keywords from any relevant papers we already have • Identify alternative spellings and synonyms • Since the search process is a critical aspect, we optimized the search keywords iteratively (define  test  optimize) • Full list is shown in Table 31 Snowballing Scan content manually Scan abstract manually Scan title manually Apply selection criteria Apply search strategy Use search string without restrictions 152 A Framework for Modeling and Improving Agile Requirements Engineering Table 31: Full list of search keywords Topic Search Keywords Agile agile, agile development, agile software development, agile method*, agile practice, agile approach, agile project, agile lifecycle, agile software engineering, Scrum, Kanban, extreme programming, lean startup, lean development, feature driven development, feature-driven development, agile unified process, rational unified process, Crystal Clear Human Computer Interaction human computer interaction, hci, human-computer interaction, computerhuman interaction, hmi, human-machine interaction, machine-human interaction, human system interaction, human interaction user centered design, user-centered design, ucd, human centered design, human centered software engineering, usage-centered design, service design, customer experience, user centered development, experience design, interaction design, goal-directed design, design thinking Usability, usability evaluation, usability engineering, usability method, user experience, joy of use User, end user, end-user, consumer, costumer, participation, user involvement, user interface, participatory design, innovation design human factor Requirements Engineering requirements engineering, requirement, user-centered requirements, documentation, requirements documentation, agile documentation Search String Since every digital library has its own characteristics concerning its search engines, we have to adapt the search string for every library. To this end, we create a short and a long form of the search string. ST = [Agile] + [Human Computer Interaction] + [Requirements Engineering] Search String (short form) (agile OR scrum OR kanban OR “extreme programming” OR lean) AND (hci OR hmi OR ucd OR usability OR human OR user) AND (“requirements engineering”) Search String (long form) ((agile OR “agile development” OR “agile software development” OR “agile method*” OR “agile practice” OR “agile approach” OR “agile project” OR “agile lifecycle” OR “agile software engineering” OR scrum OR kanban OR “extreme programming” OR “lean startup” OR “lean development” OR “feature-driven development” OR “agile unified process” OR “rational unified process” OR “Crystal Clear”) AND (“human-computer interaction” OR hci OR “computer-human interaction” OR hmi OR “human-machine interaction” OR “machine-human interaction” OR “human system interaction” OR “human interaction” OR “user-centered design” OR ucd OR “human centered design” OR “human centered software engineering” OR “usage-centered design” OR “service design” OR “customer experience” OR “user centered development” OR “experience design” OR “interaction design” OR “goal-directed design” OR “design thinking” OR usability OR “usability evaluation” OR “usability engineering” OR “usability method” OR “user experience” OR “joy of use” OR user OR “end-user” OR 153 A Framework for Modeling and Improving Agile Requirements Engineering consumer OR costumer OR participation OR “user involvement” OR “user interface” OR “participatory design” OR “innovation design” OR “human factor”) AND (“requirements engineering” OR requirement* OR “user-centered requirements” OR documentation OR “requirements documentation” OR “agile documentation”)) Study Selection Criteria Table 32: Inclusion and exclusion criteria Inclusion Criteria Exclusion Criteria papers written in English no full books papers published in between 1995-2015; papers whose full text were not available papers under peer review papers with results that had been already published papers presenting approaches to integrate user into agile development processes papers that were not focused on agile development papers related to Agile RE papers only presenting ideas, lessons learnt, recommendations or guidelines papers associated with agile requirements documentation. papers introducing tools whose underlying methodology was not comprehensibly described (black box) specific book chapters studies, whose primary focus moved away from agile methodologies Study Selection Process In the following it will be described how the selection criteria will be applied: • Study should be included if the first 3 inclusion criteria can be applied (Yes) • Study should be excluded if any exclusion criteria can be applied (No) • If there are doubts with available information then o Read full text o Ask other researcher for opinion Quality Assessment We develop a quality checklist to assess the individual studies using a 3 point Likert scale (see Table 33).