Variability management and software product line knowledge in software companies

Variability management and software product line knowledge in software companies✩ Antonio M. Gutiérrez-Fernández b,∗,Ana Eva Chacón-Luna a,David Benavides b,Lidia Fuentes c, Rick Rabiser d aUniversidad Estatal de Milagro, Km. 1 1/2 vía km. 26, Milagro, Ecuador bI3US, Universidad de Sevilla, ETSII, Av. Reina Mercedes s/n, Seville, 41012, Sevilla, Spain cITIS Software, Universidad de Málaga, Málaga, Spain dCDL VaSiCS, LIT CPS Lab, Johannes Kepler University Linz, Altenberger Straße 69, Linz, 4040, Austria A R T I C L E I N F O Keywords: Software product lines A B S T R A C T Software product line engineering aims to systematically generate similar products or services within a given domain to reduce cost and time to market while increasing reuse. Various studies recognize the success of product line engineering in different domains. Software variability have increased over the years in many different domains such as mobile applications, cyber–physical systems or car control systems to just mention a few. However, software product line engineering is not as widely adopted as other software development technologies. In this paper, we present an empirical study conducted through a survey distributed to many software development companies. Our goal is to understand their need of software variability management and the level of knowledge the companies have regarding software product line engineering. The survey was answered by 127 participants from more than a hundred of different software development companies. Our study reveals that most of companies manage a catalog of similar products in a way or another (e.g. clone- and-own, common modules that are statically imported,etc.), they mostly document the features of products using text or spreed sheet based documents and more than 66% of companies identify a base product from which they derive other similar products. We also found a correlation between the lack of Software Product Line (SPL) knowledge and the absence of reuse practices. Notably, this is the first study that explore software variability needs regardless of a company’s prior knowledge of SPL. The results encourages further research to understand the reason for the limited knowledge and application of software product line engineering practices, despite the growing demand of variability management. 1. Introduction Software product lines (SPL) enable the systematic reuse of artifacts within an organization. Software product line engineering (SPLE) (Bosch,2000;Pohl et al.,2005;Rabiser et al.,2018;Raatikainen et al.,2019) has gained significant attention in the last 20 years in the development of a portfolio of similar software products. Several studies have reported improvements with respect to cost, quality, and time-to-market (Van der Linden et al.,2007;Schmid and Verlage, 2002). Rabiser et al. showed that academic research in early years intensively investigated SPL adoption but later mostly stopped, while the industry research continued to explore the adoption of SPLs (Rabiser et al.,2018). Despite the existing literature on the use and tool ✩Editor: Laurence Duchien. ∗Corresponding author. E-mail addresses: [email protected] (A.M. Gutiérrez-Fernández), [email protected] (A.E. Chacón-Luna), [email protected] (D. Benavides), support of SPLs (Horcas et al.,2019,2023), industry seems not to pay much attention to the problem of managing variability and reuse, experimenting with its own solutions and approaches. Numerous cases of successful adoption of SPL techniques for variability management exist (Martinez et al.,2017;Van der Linden et al., 2007). Recent research (Berger et al.,2020) reaffirms that the industry still faces many challenges when embracing SPL practices. We thus argue that it is important to understand if companies manage variability regardless the knowledge of SPL terminology or practices. The relationship between explicit variability management and the lack of SPL knowledge remains unexplored (Chacón-Luna et al.,2020). We conjecture that bridging this knowledge gap can strengthen and enhance SPL frameworks. [email protected] (L. Fuentes), [email protected] (R. Rabiser). https://doi.org/10.1016/j.jss.2024.112114 Received 2 March 2023; Received in revised form 6 November 2023; Accepted 24 May 2024 2 A.M. Gutiérrez-Fernández et al. In a previous study (Chacón-Luna et al.,2019), a single company’s practices were studied, revealing that even without defining an SPL, they still derived products from a common development platform. However, they lacked documentation analyzing common or variable aspects of their product portfolio, underscoring the gap between practical variability management and established SPLE practices. Motivated by the recommendation given by Seaman (1999): ‘‘the combination of qualitative and quantitative data often provides a better understanding of the phenomenon studied’’ we conducted a survey (with closed and open questions) to consider more sources of information on the variability management practices and the software product line knowledge companies have, regardless of whether or not they are aware of SPLE. The survey instrument was also motivated by previous work related to the adoption of the SPLE (Pohl et al.,2005;Thüm et al.,2014;El-Sharkawy et al.,2019;Ebert,2014; Bastos et al.,2017;Helferich et al.,2006;Schackmann and Lichter, 2006), survey research (Pfleeger and Kitchenham,2001;Kitchenham and Pfleeger,2002a,b,c,d,2003), and from industrial evidence (case study Chacón-Luna et al. (2019)). This study allowed us to gather a sample or 127 participants allowing us to analyze and generalize the findings based on a more representative dataset. According to Kitchenham et al. (2015), empirical evidence studies allow to create knowledge through academic or industrial experiences. We consider that the results of this survey can serve as a starting point to identify the lack of knowledge about SPLE in practice and support the development of new or adaptation of existing SPLE techniques and models. For our study, we consider a widely accepted differentiation between the activities in SPLE. According to Pohl et al. (2005), SPLE consists of two main processes: First, domain engineering aims to produce the common platform, including application commonality and variability. Second, application engineering aims to achieve the highest possible reuse of domain artifacts when developing a product. Our study focuses on three subprocesses: (i) product management, (ii) domain requirements engineering (from the domain engineering) and (iii) application requirements engineering (from the application engineering). These processes are considered more abstract and technologyindependent (Pohl et al.,2005;Nuseibeh and Easterbrook,2000), making them suitable for studying variability management. We assume that these processes are implicitly or explicitly executed by companies to manage variability, even if they lack deep SPL knowledge. The product management process is in charge of the market strategy. That is, it is responsible for managing company’s product portfolio. First, product management employs scope analysis techniques (product line scoping) to define what is and what is not within the scope of the product line. Second, the domain requirements engineering subprocess comprises all activities to elicit and document common and variable product line requirements. Finally, the application requirements engineering subprocess encompasses all activities performed to specify the application requirements and the amount of reuse of domain artifacts. One of the purposes of this subprocess is the discovery of deltas between application requirements and available platform capabilities. Our assumption is that effectively managing variability and reusing common components is crucial for those companies handling a portfolio of software related products, regardless of their familiarity with SPLE concepts (Berger et al.,2020;Fischer et al.,2023). This paper aims to confirm or refute this assumption and investigate companies’ awareness of SPL concepts. We formulate our research question as follows: RQ. Is there a relationship between the offer of similar or customisable products, the use of variability management mechanisms and the knowledge of SPL concepts? For assessing SPL knowledge, we use a defined set of SPL vocabulary. In addition to detailing companies’ approaches to manage variations in their products and requirements, our study shows a clear correlation between systematic variability management practices and awareness of the SPL vocabulary. Furthermore, our findings underscore a statistically significant correlation between the years of work experience and expertise in Software Product Line (SPL) knowledge. Section 2discusses existing empirical studies of variability management practices. Section 3explains the research method we followed. Section 4presents the results. In empirical studies, statistical analysis is necessary to validate the hypothesis. Therefore, in Section 5, we evaluate the answers’ correlation to validate our hypothesis and the lessons learnt. In Section 6we discuss the results in relation to our research questions and also highlight threats to validity. Section 7 concludes this article. 2. Related work Our research builds upon a set of works that have investigated variability management in practice. The pivotal concept explored in the current research stems from the precedent set in a previous work, where insights were derived from a company’s application of variability management techniques despite lacking prior awareness of Software Product Lines (SPL) concepts (Benavides and Galindo,2014). However, it lacked the rigor and systematic approach necessary for conducting and reporting a comprehensive study. The author suggested future work that would involve structured empirical research in companies unfamiliar with SPLs to gain deeper insights into their real-world practices. Rabiser et al.’s study (Rabiser et al.,2018) influenced the direction of our research. In their paper, they presented a comparative analysis of the trends in SPL research between the academia and industrial sectors. Moreover, this study helped us pinpoint the specific processes and subprocesses within the SPL framework that are of the greatest interest in both academic and industrial research, leading us to focus on three key processes: product management, domain requirements engineering, and application requirements engineering. Rabiser et al.’s research also revealed that a substantial portion of academic work (34%) predominantly showcased artificial or toy examples, while 27% of papers lacked any form of evaluation. These discoveries further emphasized the need for empirical research, inspiring our own study. Chacón-Luna et al. (2019) conducted a case study examining how a company lacking SPL knowledge manages variability. Their findings indicated that the observed company did employ some variability management practices, including the reuse of product assets, among other aspects. However, the study’s limited scope prevented broad generalizations. Empirical methods often serve as complementary tools for gathering diverse sources of information. Hence, we decided to complement previous work with a survey. In a prior publication (Luna et al., 2022), we detailed the survey’s design, with the goal of making the instrument available to the community for feedback before conducting the study. While studies focusing on SPL practices in companies unfamiliar with SPLs are scarce, numerous related studies exist. For example, there are investigations into SPL adoption in small and medium-sized companies (Benavides and Galindo,2014;Bastos et al.,2017), where authors seek to justify the use of agile methods and a multimethod approach, respectively, for SPL adoption. Similarly to our work, Bastos et al. (2017) also aims to define transitional practices for implementing SPL techniques. Song and Runeson (2023) explore various scenarios of collaboration between academia and industry, considering social factors to enhance this collaboration. Several papers discuss software testing practices in the industry in general and the testing of software product lines in particular (Lopez- Herrejon et al.,2015;Engström and Runeson,2011;do Carmo Machado et al.,2014;Lamancha et al.,2010). Recently, Berger et al. (2020) 3 A.M. Gutiérrez-Fernández et al. Fig. 1. Study steps. reported a survey of twelve medium to large companies – from fields such as automotive, aerospace, and railway systems – on their adoption of variability management techniques. Limited evidence suggests the utilization of SPL approaches for testing, reinforcing the assumption that companies approach variability in testing implicitly and nonsystematically, similar to our findings regarding how companies handle variability management concerning domain and application requirements engineering. Mukelabai et al. (2018) use a survey to understand industry practices regarding variability management. This survey shares a common foundational assumption with our research: the recognition that companies exhibit variability management needs independent of their knowledge of formal SPLE techniques. While Mukelabai’s work provides valuable insights into this domain, it is important to note key distinctions between their objectives and the goals of our own research. First, Mukelabai’s research focuses on variability management at the development level, excluding strategic considerations related to the management of a portfolio, in connection with the product management subprocess. Then, they extend the scope to encompass the entire software development life cycle. And most notably, our research distinguishes itself by exploring the relationship between the adoption of variability management practices and the SPLE knowledge. Recent work reports a case study (Fischer et al.,2023) with multiple companies developing software for cyber–physical systems with focus on the product line testing. This work concludes that variability modeling techniques and tools are not applied and that the testing configuration mechanisms used only partially cover the configuration space through automated tests. These findings are aligned with our study, suggesting that companies often do not explicitly model variability in their products. 3. Research method According to Fink (2003), surveys are a method of data collection and should be used under appropriate conditions. Fink (2003) emphasizes that a survey isn’t just about gathering information through a questionnaire or checklist; it’s a comprehensive research method. In our study, we conducted a descriptive survey, which allowed us to collect information and evidence of the distribution of respondents’ statements regarding managing the variability of the products they develop. This research method allows us to describe, compare and explain the knowledge, attitudes and behaviors related to the definition and application of variability management practices in software development companies. To ensure the quality of our survey, we followed the guidelines recommended by Kitchenham et al. (2015), Pfleeger and Kitchenham (2001), Kitchenham and Pfleeger (2002a,b,c,d,2003) and Kitchenham (2004). We planned and executed the survey using the following steps: 1. Study definition. This stage involved defining the research objective and the expected results of the survey (Pfleeger and Kitchenham,2001). 2. Survey design. We determined the best approach to obtain the information needed to address the research objectives and set the appropriate sample size (Kitchenham and Pfleeger,2002a,d). 3. Survey implementation. In this step, we developed all materials required to conduct the survey in accordance with the study planning (Kitchenham and Pfleeger,2002b). 4. Survey execution. We executed the survey as planned, including the collection of data allowing quality control in post-survey execution audits (Kitchenham and Pfleeger,2002c). 5. Data analysis and interpretation. We validated data, partitioned responses, coded and analyzed data. 6. Report of survey results. Finally, we presented the results of the survey, summarizing the key findings. For visual clarity, the steps we followed are represented in Fig. 1. As a result of the first stage, planning the study (three first steps), we generated a questionnaire that is used as a tool for our study. In our previous article (Luna et al.,2022), we presented the details of creating this questionnaire. Below we summarize the questionnaire and provide the information required to understand the data analysis and results presented in this paper. 3.1. Survey definition The designed questionnaire seeks to evaluate the main research question RQ proposed in Section 1: study the variability management practices regarding the 3 above mentioned subprocesses (product management, domain requirements engineering and application requirements engineering) from the SPLE framework and also the knowledge of SPL vocabulary. Therefore with consider the following 4 questions. •RQ1.1. Do companies manage a product portfolio in which similar or related products appear? This question investigates whether the company’s product portfolio includes related products through customization or adaptation of existing products, focusing on Product Management subprocess practices. •RQ1.2. Are product characteristics analyzed so that a family or portfolio of related products is identified? This question explores the extent to which the company addresses the development of related products through feature analysis, common platform derivation, and maintenance, delving into Domain Requirements Engineering practices. •RQ1.3. Are the requirements of each project systematically analyzed so that common attributes and variables can be compared for potential reuse? This question seeks to identify the degree to which the analysis and comparison of common and variable requirements from the product portfolio are conducted to facilitate reuse, thereby addressing Application Requirements Engineering practices. •RQ1.4. Do companies have knowledge of SPL terms? The final question assesses the level of knowledge regarding specific SPL vocabulary, enabling an analysis of the relationship between this knowledge and the management of a portfolio of related products. 4 A.M. Gutiérrez-Fernández et al. 3.2. Survey design Our research team spent several months developing the questionnaire within the context of TASOVA (Network on new trends in software Architecture and variability).1The target population of our survey included any software development company with a software product catalog. To create our sample, we leveraged the TASOVA research network, which is geographically dispersed throughout Spain. We also extended our sample by contacting companies through email, including international participants. The companies under study have registered activities in the software development sector. This approach was influenced by a pilot survey conducted with Ecuadorian companies. In this pilot, we distributed the survey to the entire population sampled from company directories, but the response rate was only about 15% so the total number of responses was very low (about 10). Hence, we decided that a more efficient data collection strategy was to contact companies associated with the TASOVA network. 3.3. Survey implementation The survey consists of a questionnaire that was developed following the steps designed by Fink (2003) and Kitchenham and Pfleeger (2008): search the relevant literature; construct an instrument; evaluate the instrument; and document the instrument. Based on Fink’s research (Fink,2003), different survey designs have specific objectives and can be categorized as experimental or descriptive. Experimental designs allow for comparison between two or more groups, where at least one is experimental (Wohlin et al., 2012). Descriptive designs, on the other hand, provide information about existing groups and phenomena. The current study utilizes a descriptive survey. The introduction of the questionnaire included the motivation of the study (to know the variability management in companies that do not know SPL concepts), the importance of the participation of each individual and how the confidentiality of the collected answers would be implemented. As recommended by Kitchenham et al. (2015) for self-administered questionnaires, the study questionnaire comprises 26 closed questions and 7 open questions, primarily addressing the participant profile and private data. The design of the questionnaire aimed for completion within twenty minutes. Tables 1 and 2show the complete questionnaire. The questionnaire has 7 sections, including 14 questions with answers designed with ordinal frequency scales. The first 3 sections include questions on the profile of the company, the respondent and the products developed by the company, respectively. Specifically, Section 1(Q1 to Q6) asks about the scope, type and size of the company. Section 2(Q7 to Q13) asks about the respondent’s professional situation, such as experience in the software production sector and the relationship with his position within the company. In Section 3of the form (Q14, Q15 and Q16) 3 questions were asked about the profile of the products they develop, the size of the projects (in terms of cost) and the number of products they currently have on the market. Regarding the project size, we considered a multiple answer, as the same company can manage projects with different sizes. Sections 4,5and 6answer the research subquestions RQ1.1, RQ1.2 and RQ1.3, respectively. Specifically, Section 4(Q18 to Q21) corresponds with research question RQ1.1 and includes questions on how companies handle customization, adaptation or extension of an already developed product. Section 5(Q22 to Q25) focuses on managing product variability, RQ1.2. Section 6(Q26 to Q29) includes questions to learn about practices in requirements management to answer RQ1.3. Finally, in Section 7(Q30 to Q33) we inquire about the respondent’s level of knowledge on the topic of SPL concepts, RQ1.4. We asked 1http://tasova.uma.es/. questions to determine the respondents’ level of knowledge about various concepts. For this purpose, we considered a 5-point Likert scale from ‘‘Very high’’ (the respondent is an expert on the subject) to ‘‘Very low’’ (the respondent does not know what it is). We chose the terms ‘‘Software product lines’’, ‘‘Software product families’’, ‘‘Feature models’’ and ‘‘Variability intensive systems’’ after performing a systematic literature review about empirical practices in SPLs (Chacón-Luna et al.,2020), where these were the most used terms. 3.4. Conducting the survey Data collection is an essential part of achieving the survey objectives. Hence, to ensure data collection from the pertinent stakeholder group, we initially attempted to motivate respondents to complete the form. To this end, we communicated the purpose of the study, emphasized the significance of their participation, and detailed how we would maintain the confidentiality of the gathered opinions. Contact information was only collected to keep explicitly interested participants informed. A sample of 135 people was surveyed that were obtained through non-probabilistic sampling of the ‘snowball’ type, i.e., as described above, we used company contacts of the TASOVA network members to find software development managers. The survey was conducted by applying the questionnaire, which was sent in April 2021 via email. 3.5. Data analysis After collecting responses, we validated the data to ensure completeness. We excluded eight responses from our sample due to incomplete answers, resulting in a final sample of 127 respondents. We performed the data analysis using SPSS, a statistical analysis tool, and engaged an external expert in statistical methods. We utilized frequency analysis to process both single and multiple responses and performed cross-tabular analysis to quantitatively explore relationships between variables, thereby addressing our research questions.2 3.6. Report of survey results In the subsequent sections, we present the results obtained from the survey. First, Section 4shows the data collected and the description of the data in relation to the research questions. Section 5delves into the statistical analysis of answer relationships from various questions to assess the link between variability management practices and SPL knowledge. 4. Results The survey was conducted among 127 participants from companies with subsidiaries in Spain. The majority of these companies are privately owned. Roughly two-thirds of the companies involved in the study operated internationally, while approximately one-third had a domestic scope. The precise count remains unknown due to some participants failing to specify the nature of their company. 4.1. Company and product profiles Fig. 2 shows the results of the answers to the questions regarding the company profile. In terms of size, considering the number of workers (Q5, Fig. 2(a)), 45,7% of the companies surveyed are small companies (between 0 and 50 workers), while 31,5% are large companies (more than 250 workers) and 22,8% are medium-sized (between 50 and 2Data available in Spanish, the original language, in https://drive.google. com/drive/folders/1DCzE9hbEzP1rOkSYFGdnrGWGoxRKqrZB?usp=share_ link. 5 A.M. Gutiérrez-Fernández et al. Table 1 Survey questionnaire: company profile sections. Software variability management survey Section ID Question Answers Q1 Company name Open Q2 Company web Open Q3 Scope of the company’s market National/International/Other Q4 Type of company Public/Private Q5 Company size Small (0–50)/Medium (50–250)/Large (+than 250) Company profile Q6 How many years has the company been in business? 3 years or less/Between 4 and 10 years/More than 10 years Q7 Name and Family name Open Q8 Contact telephone Open Q9 Contact email Open Q10 Would you like to be contacted to send you the results or progress of the study? Yes/No Q11 City from where you work Open Q12 Years of work experience in the software production industry 3 years or less/Between 4 and 10 years/More than 10 years Participant profile Q13 Which of the following job descriptions most closely matches your job description? Manager/Consultant/Department head or project manager/Analyst/Programmer Q14 What type of products does the company develop (multiple choice)? Web applications/Mobile applications/Control systems/Critical systems/Internet of things/Data analytics/ERPs/Security/Information systems/Other. Q15 How big are the projects developed by the company? Small (less than 100.000 euros)/Medium (between 100.000 and 500.000 euros)/Large (more than 500.000 euros) Profile of the products developed by the company Q16 How many different products does the company currently have on the market and/or in development? Between 1 and 5/Between 5 and 10/Between 10 and 20/More than 20/Don’t know. Table 2 Survey questionnaire: sections on development management. Software variability management survey Section ID Question Answers Q17 After the execution of a project, another product similar to the one developed in the project is produced. Always/Frequently/Sometimes/Occasionally/Never Q18 A new customer asks us to adapt a similar product from one already developed/offered. Always/Frequently/Sometimes/Occasionally/Never Q19 The company’s current methodology allows us to approach new products by customizing or adapting existing products in such a way that we save time and cost. Always/Frequently/Sometimes/Occasionally/Never Q20 Indicate the methodology you use to successfully approach the adaptation/customization/extension of an already developed product. Open Product customizationadaption Q21 What percentage of projects developed by the company are made from scratch? Between 0 and 25%/Between 25 and 50%/50%/Between 50 and 75%/Between 75 and 100% Q22 We prepare documentation detailing the specifications of the characteristics of each of your products. Always/Frequently/Sometimes/Occasionally/Never Q23 We have identified our own flagship products or platforms from which other products originate for other customers. Always/Frequently/Sometimes/Occasionally/Never Q24 When we perform evolution or maintenance tasks, we come up with ideas to improve the products already deployed in the market. Always/Frequently/Sometimes/Occasionally/Never Product variability Q25 If so, in what format do you store this information about the products and their characteristics? A spreadsheet/A text document/Web format/Using specific software (e.g. DOORs))/Open Q26 We prepare a matrix or similar where the common and variable requirements of each of the company’s similar products are detailed. Always/Frequently/Sometimes/Occasionally/Never Q27 When making a new product, we perform an analysis to determine which modules/components are common with respect to other existing products, and therefore reusable, and which are different or new. Always/Frequently/Sometimes/Occasionally/Never Q28 We develop a diagram or documentation that allows us to describe the interrelation between the characteristics of each product developed in the company. Always/Frequently/Sometimes/Occasionally/Never Analysis of requirements and variability of products Q29 The documentation where we collect the requirements of a new product also includes the analysis of the characteristics of other similar products. Always/Frequently/Sometimes/Occasionally/Never Q30 Software product lines, SPLs Very high/High/Medium/Low/Very low Q31 Software product families Very high/High/Medium/Low/Very low Q32 Feature models, FMs Very high/High/Medium/Low/Very low Knowledge of variability terminology Q33 Variability intensive systems Very high/High/Medium/Low/Very low 6 A.M. Gutiérrez-Fernández et al. Fig. 2. Company profiles. Fig. 3. Product profiles. 250 workers). Regarding the company experience (Q6, Fig. 2(b)), it is important to mention that 75% of the companies surveyed have been in the market for more than 10 years, while 16% are between 4 and 10 years in the market, and 9% are 3 years or less. We consider this result important, because the relation between long experience and good practices and approaches from the SPL paradigm deserves to be further studied.. Specifically in the software production sector (Q12, Fig. 2(c)), 28,3% of the surveyed companies have 3 years or less, 20,5% have between 4 and 10 years inclusive, and 51,2% have more than 10 years of experience. When asked about the type of products they develop, 108 of the respondents indicated that among the products in their portfolio are web applications, followed by mobile applications (76). Also, 72 of the respondents said that the company develops information systems, 58 data analysis, 51 control systems, 50 ERP, 40 security applications, 38 internet of things, 31 critical systems and 19 of the respondents said they develop other systems (such as Ecommerce, banking, or desktop applications), as shown in Fig. 3(a). Regarding the projects, depicted in Fig. 3, when respondents were asked about the number of different products the company currently has on the market and/or in development (Q14, Fig. 3(a)), 28% indicated that the company has between 1 and 5 products. 16% responded that the company has between 5 and 10 products. Six percent stated that it has between 10 and 20 products, and 29% indicated that it has more than 20 products in the market. Finally, 21% of respondents said they did not know the number of products the company handles. As for the size of the projects developed by the company (Q14, Fig. 3(b)), 70 of the respondents indicated that they are small projects (less than e100.000), while 60 said that the projects they develop are medium-sized (between e100.000 and e500.000). Finally, 63 of the respondents said that the company develops large projects (more than e500.000). Note that the number of answers is higher than the 7 A.M. Gutiérrez-Fernández et al. Fig. 4. Answers to RQ1.1. number of respondents because the same company develops project with different sizes. The data collected from the profile of the companies indicates the diversity of companies with different sizes of projects, different types of products developed, different number of years in the software production market, and different size of the company. 4.2. Product management Product management within the SPL framework (Pohl et al.,2005) entails making decisions about which features are shared across multiple products, as well as those that are unique to specific variants. It also involves handling the evolution of individual products within the product line. RQ1.1 Do companies manage a product portfolio in which similar or related products appear? Fig. 4 shows a summary of the answers to RQ1.1. Fig. 4(a) shows that 35,4% of companies, frequently, and 5,5% of companies, always, after the execution of a project, produce some other similar product (Q17). These results are reinforced by the fact that only 17,3% of the respondents indicated that between 75% and 100% of the projects they develop are developed from scratch (Q21), as shown in Fig. 4(d). In addition, Fig. 4(b) shows that when respondents were asked whether customers ask to adapt a similar product from an already developed one (Q18), 55,1% answered that customers frequently and 13,4% answered that customers always ask for it. This is related to the results presented in Fig. 4(c), where it can be seen that at least 59,1% of the respondents indicated that (frequently 31,5% and always 27,6%) the current methodology allows them to create new products by customizing or adapting existing products (Q19). Findings of RQ1.1 •From the results presented, we can infer that companies in our study are adopting some practices that allow reuse during the development of software products, since they are not starting from scratch. Furthermore, for 59% of the respondents (frequently 31,5% and always 27,6%) the methodology they use allows them to create new products by customizing or adapting existing products as shown in Fig. 4(c). •We can also infer that the companies’ products are related because they develop products with similar characteristics, since at least 68,5% (frequently 55,1% and always 13,4%) of the times, when a new customer requests the development of a product, they adapt it from a project already developed/offered, as shown in Fig. 4(b). 4.3. Domain requirements engineering RQ1.2. Are product characteristics analyzed so that a family or portfolio of related products is identified? The answers to the closed questions related to RQ1.2 are depicted in the charts in Fig. 5. Regarding the Q22, Fig. 5(a) shows that more than 62,2% of the respondents said that they frequently prepare documentation detailing the specifications of the characteristics of each of their products (29,1% frequently and 33,1% always). Another important fact is that more than 8 A.M. Gutiérrez-Fernández et al. 66,2% (40,2% frequently and 26,0% always) of the companies stated that they have identified a star product from which other products originate (Q23). Only 3,9% indicated that they do not have a platform or flagship product identified among their products. This data is shown in Fig. 5(b). Fig. 5(c) shows that more than 85,6% (that is, frequently 44,1% and always 31,5%) are concerned about making improvements to products already deployed in the market when performing evolution or maintenance tasks, while 0,8% never and 4,7% occasionally do so (Q24). When asked about the formats they use to document the requirements and features of their products 66 respondents indicated that they use a text document, 42 said they use a spreadsheet. Also 42 respondents said they use a web format, while 34 use a specific software and another 25 respondents said they use another format such as Github, UML, Confluence or Jira. It should be noted that the alternatives in this question overlap each other because respondents could choose or describe more than one option. Therefore, the sum of the answers obtained is not 127 (number of respondents). Findings of RQ1.2 •From the results obtained, we can infer that more than 60% of the companies are aware that their products are related, as they are concerned about analyzing the common characteristics among their products and even have identified a star product or base platform from which products for new customers originate. •It is also evident that most of the respondents use text documents and spreadsheets to store information about products and their characteristics. 4.4. Application requirements engineering RQ1.3. Are the requirements of each project systematically analyzed so that common attributes and variables can be compared for potential reuse? The answers to the questions related to RQ1.3 are depicted in the charts in Fig. 6. Fig. 6(a) shows that while 37% (i.e., frequently 26,0% and always 11,0%) of the respondents indicated that a matrix or similar is prepared detailing the common and variable requirements of each of the products (Q26), 43,3% indicated that they do not (i.e., never 19,7% and occasionally 23,6%) create such a matrix. These figures are related to Fig. 6(c), which shows that while 44% (that is, frequently 28,3% and always 15,7%) of the respondents have indicated that they make some diagram that allows them to describe the interrelationship between the characteristics of each product in the company’s product portfolio (Q28), 30,7%, that is, (never 12,6% and occasionally 18,1%) do not make such diagrams. Fig. 6(b) shows that about 68,5% of the respondents (i.e., frequently 35,4% and always 33,1%) indicated that when creating a new product they perform an analysis to determine which components are reusable because they are common to other existing products (Q27). Only 3,9% indicated that they never perform such an analysis. Fig. 6(d) shows that about 38,6% (that is, often 19,7% and always 18,9%) document common characteristics of a new product (Q29). Findings of RQ1.3 •According to the results obtained, we can infer that few companies document the analyses they carry out in relation to the characteristics of their products. Around 63% (never 19,7%, occasionally 23,6%, sometimes 19,7%) of the companies do not document the common and variable requirements of the company’s similar products, as shown in Fig. 6(a). In the same vein, they also often fail to document the interrelations of their product characteristics, reaching 55,9% (i.e., never 12,6%, occasionally 18,1%, sometimes 25,2%), as shown in Fig. 6(c). •Another inference is that companies do indeed want to reuse and do perform analyses to understand common components. This is evidenced in Fig. 6(b), which suggests that they only develop support for new requirements of their customers, which in SPL is known as application deltas. However, it clearly appears that in practice few companies (less than 50%, Q29, i.e.: often 19,7% and always 18,9%) document the analyses they perform to demonstrate the characteristics of their products or the existing interrelations between their products. 4.5. SPL terms knowledge RQ1.4. Do companies have knowledge of SPL terms? Fig. 7 groups the results of the respondents’ knowledge regarding 4 concepts (software product line, software product family, feature model and variability-intensive systems). In general, the tendency is that less than 30% have high knowledge of these topics. Only 33,9% (high 26,8% and very high 7,1%) of the respondents know about the software product line concept, Q30. These figures are similar to the question about software product families, Q31, where 35,5% indicated that they do know the term (high 27,6% and very high 7,9%). 29,1% answered that they know about the feature model concept, Q32, (high 23,6% and very high 5,5%). When asked about variabilityintensive systems, Q33, only 18,9% indicated knowing the term (high 15,0% and very high 3,9%). Findings of RQ1.4 •According to the figures presented, we can infer that companies know very little about SPL concepts. It stands out that the least known term was variability-intensive system, where respondents indicated 3,9%. •The terms software product line and software product family are slightly more known by the respondents. 7,1% and 7,9% even indicated to have very high knowledge respectively. •The overall picture is that less than 30 percent have high and very high knowledge of SPL topics. 5. Statistical analysis Correlational analysis helps us to demonstrate the relationships or degree of association between the variables under study, which are detailed below. The significant results of the correlational analysis are described below. The specific data items appear in Tables 3 and 4, designated in parentheses. Consider that the 𝑎𝑙𝑝ℎ𝑎 confidence levels of 0.05 and 9 A.M. Gutiérrez-Fernández et al. Fig. 5. Answers to RQ1.2. 0.01 were chosen depending on the explanation of the nature of the data needed from the research data. The higher 𝑎𝑙𝑝ℎ𝑎 level (0.05) is acceptable in exploratory research. The lower 𝑎𝑙𝑝ℎ𝑎 level (0.01) was chosen to strengthen the confidence in some results. For ease of comprehension, the analysis is segregated into two tables, focusing on SPL subprocesses and knowledge. However, some questions overlap between both tables when correlating between different sections. •1. Employee experience vs. level of knowledge. A statistically significant direct correlation can be seen, although not too significant, between the 4 items of SPL knowledge and the years of experience in software production. The greater the work experience in the software production sector, the greater the knowledge of the SPL. It can also be seen that there is a direct correlation at 99%, although not very significant, between the SPL knowledge component as the sum of the items (as a whole) and the years of experience in software production (p =0.007, r =0.24). •2 and 3. Similar products vs. level of knowledge. There are significant direct relationships in respondents who answered that the current methodology sometimes allows addressing new products by customizing or adapting existing products and have knowledge about software product lines (p =0.011, r =0.508) and software product families (p =0.02, r =0.611), as well as feature models (p =0.046, r =0.412). Also, from the respondents who answered that the current methodology ALWAYS allows approaching new products through customization or adaptation, there is another statistically significant relationship. The higher the frequency of ‘after the execution of a project some other product similar to the one developed in the project originates’ (item 3), the greater the correlation observed in their familiarity with the terminology ‘feature models’ (p =0.038, r =0.353). •4. Current methodology vs. level of knowledge. Table 3 shows that respondents who indicated that they have knowledge about high variability systems (p =0.023, r =0.202), perceive that the methodology currently used by the company allows new products by customizing or adapting existing products, thus saving time and cost. •5. Current methodology vs. development from scratch. There is a statistically significant direct correlation (p =0.039, r = 0.184) between the current methodology used by the companies surveyed and whether they do not start from scratch in the development of their products. •6. Development from scratch vs. level of knowledge segmented by methodology success ‘NEVER’. According to the item 6, it clearly appears evident that respondents who indicated that the current methodology never allows addressing new products by customizing or adapting existing products and who in turn stated that most of their products are developed from scratch, have a lower level of knowledge of ‘software product lines’ (p = 0.023, r = −0.82) and software product families’ (p =0.028, r = −0.80), i.e., there is a high degree of inverse correlation in both crossovers. •7. Product customization vs. level of knowledge segmented by methodology success ‘ALWAYS’. For those who consider the methodology to always be successful in addressing new products by customizing or adapting existing products, statistically significant direct relationships are seen in crossovers when more tend to know the term feature models (p =0.044, r =0.343), as well as variability-intensive systems (p =0.044, r =0.342). In other words, respondents who have more knowledge of these terms always consider that the methodology allows them to customize or adapt new products. The degree of direct correlation is not excessively relevant in both crosstabs. •8. Requirements management good practices vs. product customization. There is a direct correlation (although in no case too important) that the respondents who answered that ‘they