A Water Footprint Management Construct in Agri-Food Supply Chains: a Content Validity Analysis

  Citation: Barbosa, M.W.; Cansino, J.M. A Water Footprint Management Construct in Agri-Food Supply Chains: A Content Validity Analysis. Sustainability 2022,14, 4928. https:// doi.org/10.3390/su14094928 Academic Editors: Xiawei Liao, Li Chai and Xi Hu Received: 24 January 2022 Accepted: 17 April 2022 Published: 20 April 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). sustainability Article A Water Footprint Management Construct in Agri-Food Supply Chains: A Content Validity Analysis Marcelo Werneck Barbosa 1,* and JoséM. Cansino 1,2 1Facultad de Administración y Negocios, Universidad Autónoma de Chile, Pedro de Valdivia 425, Providencia 7500912, Santiago, Chile; [email protected] 2 Department of Economic Analysis and Political Economy, Facultad de Ciencias Económicas y Empresariales, Universidad de Sevilla, Avda. Ramón y Cajal 1, 41018 Sevilla, Spain *Correspondence: mar[email protected] Abstract: Common problems when carrying out water footprint (WF) assessments are obtaining specific primary data, dealing with the complexity of its computation, and the availability of quality data. In a supply chain context, inconsistencies are even more exacerbated. In order to fill in this research gap, this study proposes and evaluates the content validity of a survey scale to assess WF management initiatives implemented by companies, with a focus on supply chains and the agriculture industry. In order to do so, a literature review was performed to identify candidate survey items whose content was later validated with experts in terms of their relevance, clarity, and essentiality to measure WF management. Content validity was assessed using several indices (items’ content validity index (I-CVI), Kappa’s coefficient, Aiken’ V coefficient, and content validity ratio (CVR)), which indicated high content validity for the selected items. This study provides a set of measurement survey items that can be used to evaluate WF management initiatives implemented in agri-food supply chains in future empirical studies. Keywords: water footprint management; water footprint; agri-food supply chains; content validity 1. Introduction Water is a fundamental resource for our existence and socio-economic development. The establishment of the United Nations’ Sustainable Development Goal (SDG) number 6, which refers to water quality and withdrawal reduction [ 1 ], corroborates the world’s concern with sustainable water management. In order to support water use and assessment, Hoekstra and Hung introduced the concept of the water footprint (WF), which is defined as the total volume of water resource used, directly or indirectly, to produce goods and services [ 2 ]. It includes not only water directly used to deliver products, but also water indirectly consumed and polluted throughout the production processes [ 3 ]. One of the sectors that is most responsible for water consumption, degradation, and pollution is the agriculture industry [ 4 , 5 ]. Irrigated agriculture accounts for most of the total consumptive water use worldwide [ 6 ]. For these reasons, agri-food water resources management research is a topic of increasing research interest [7]. Despite the existence of varied measurement indicators, WF assessments currently face some challenges, which need to be explored by researchers in this field. The first common problem when adopting WF assessments is obtaining specific data, which is specially a challenge for products with long and complex supply chains [ 8 ]. In fact, due to the difficulty in obtaining local primary data for water assessments, several authors use literature data [ 9 ] and official water accounting data, with few opportunities for direct data collection [ 10 ]. The second challenge is related to difficulties in implementing WF due to the complexity of its computation [ 5 , 11 ]. The third concern is that the quality of WF assessments depends upon the quality of available data. In a supply chain context, inconsistencies are even exacerbated through the chain complexities due to difficulties Sustainability 2022,14, 4928. https://doi.org/10.3390/su14094928 https://www.mdpi.com/journal/sustainability Sustainability 2022,14, 4928 2 of 17 in obtaining the necessary data from partners to carry out WF assessments [ 7 ]. In fact, researchers recognize that there can be significant differences between the real data and official statistics due to the use of distinct methodologies [ 10 ]. In addition, each measurement technique or indicator has its own limitations. For instance, at farm scale, WF assessments do not take into account information about irrigation management, so related data must be incorporated into the WF assessment [ 12 ]. Fourth, WF values of different studies can be substantially uncertain and highly variable [ 13 ]. Sometimes, due to constraints related to data availability and to inconsistencies in databases, the results of assessments need to be considered as best estimates [ 14 ]. Moreover, existing inconsistencies between different evaluation approaches and metrics can lead to uncertainties that might be confusing to decision makers, consumers or manufacturers/retailers, hampering their ability to develop sustainable solutions [ 15 ] and to make decisions. Fifth, results published based on standardized approaches sometimes do not consistently follow the respective established guidelines, which could lead to misinformation whenever results are published without specifying existing limitations [ 16 ]. The reported problems and inconsistencies in the interpretation and application of these indicators have resulted in incomparable and sometimes contradictory results [ 15 ]. Finally, there is also criticism concerning the usage of some water usage-related metrics [ 17 ], among different research communities on the subject. While part of the scholars in the field consider that WF assessment should be based on volumetric indicators, other researchers defend that these measurements should be focused on local impact of water use [18]. Moreover, Lovarelli et al. [ 19 ], through a review study on WF use in agricultural production, found out that the vast majority of studies published so far are aimed at directly quantifying WF. Other measurement mechanisms must be proposed so that studies that do not count with available raw data can be carried out. Finally, water footprint was considered by Kamble et al. [ 20 ] an important dimension to define the concept of environmental sustainability performance of an agri-food supply chain (AFSC), but no measurement items have been provided to define this constructs. As so, quantitative studies aiming to explore WF management practices become difficult to operationalize because no scale that can measure WF management in a reliable and valid manner has been developed until now. The lack of a specific measurement scale for WF management indicates a crucial knowledge gap. In conclusion, the gaps, concerns, and problems previously presented show that other ways of assessing WF management, especially in agri-food supply chain contexts, are necessary. Different methods for scale development have been proposed [ 21 – 23 ]. Although these methods have slightly different procedures, in general, all of them include determining the measurement goals, generating a pool of measurement items, having these items assessed by experts, and finally, administering these items to a larger sample. In order to fill the aforementioned research gaps, this study proposes and evaluates the content validity of a survey scale to assess WF management initiatives implemented by companies, with a focus on supply chains and the agriculture industry. As so, the focus of this study is on the first activities of the scale development process, which include item generation and their judgement by experts for content validity. These activities were carried out with the support of Chilean experts in the field. Content validity is the “degree to which a measure’s items represent a proper sample of the theoretical content domain of a construct” [ 24 ]. It is aimed at assessing that the scale items will properly represent the construct [ 25 ]. Therefore, the scale items proposed in this study fill in the research gaps presented, since they do not require raw water footprint data availability, nor they do not rely on complex data gathering or computation procedures. In order to achieve this objective, a systematic literature review was performed to identify candidate survey items, whose content was later validated by experts. The main contribution of this study is to provide a set of measurement survey items that can be used to evaluate WF management initiatives implemented in AFSCs. It is important to highlight that the measurement items proposed in this study are not a direct measurement method Sustainability 2022,14, 4928 3 of 17 like the techniques used for water footprint assessment, but an alternative method that evaluates WF management initiatives that can be applied when it is not possible to directly measure WF due to the previously presented constraints. This study is grounded on the Natural-Resources-Based View (NRBV), which states that firms can generate unique sources of competitive advantage through proactive engagement with the natural environment [ 26 ] and that competitive advantage will be achieved if firms can manage natural resources and develop capabilities to address environmental changes [ 27 ]. Under the NRBV, competitive advantage is achieved if companies can create profits that are supported by resources that are not easily duplicated by competitors [ 28 ]. Being able of managing WF along the supply chain is a source of competitive advantage. 2. Theoretical Background The WF indicator refers to the use of water resources in products or services and water pollution during production. WF assess the use of water resources throughout all the stages of a product’s life cycle (production, transportation, marketing, consumption, and reuse) [ 29 ]. It is a multidimensional indicator composed of three components: the blue WF and green WF, which measure direct water consumption, and grey WF, which measures the pollution produced [ 2 ]. The blue WF refers to the volume of surface and groundwater consumed to produce a good. The green WF refers to the rainwater consumed during the production process. It addresses the rain water stored in the soil and then absorbed by plants [ 30 ]. Green WF is particularly of interest for agricultural products [ 6 ]. The grey WF of a product refers to the volume of freshwater that is required to assimilate the load of pollutants [31]. It considers the use of pesticides and fertilizers applied in the fields [32]. WF measures water consumed and the impact of the production processes in the quantity and quality of water [ 9 ]. It indicates net water consumption, that is, water that cannot be used for other purposes [ 33 ]. In irrigated agriculture, WF is a key indicator for evaluating sustainability [ 6 ]. Due to its relevance, WF research has been growing and has become one of the predominant topics in the research area of water resources management [ 29 ]. In fact, Zhang et al. [ 34 ] found out that the mainstream research related to WF is its accounting methodologies and application in water resource management. The WF concept is integrative applicable at different levels and along supply chains [ 35 ]. It is usually focused on three different perspectives: the production perspective, the trade perspective, and the consumption perspective [ 8 ] and it has been adopted at national, corporate and product levels [ 30 ]. As so, WF can be calculated for a specific product, for geographical locations or producers [ 36 ]. The assessment of WF also considers the whole supply chain, that is, every relevant stakeholder involved in water management [ 37 ]. Supply chains, besides facing several current challenges (risks, disruptions, lockdowns) due to the COVID-19 pandemic’s effects on production and distribution [ 38 ], also have to be concerned about sustainability issues and customers’ environmental pressures [ 39 ]. In AFSCs, WF management methods and initiatives can be used to assure that supply chain partners comply to sustainability standards and goals [ 40 ]. WF management across industry supply chains is a central topic in the public agenda [ 41 ]. In spite of being a growing research field, WF assessment studies at a supply chain extent are rather limited [30]. The full potential of WF analysis along supply chains needs to be realized still [35]. Water resource management is an effective way to measure the supply and demand of water resources and improve the efficiency of its use [ 42 ]. It includes: monitoring, modelling, exploration, assessment, policy design, performance monitoring, and maintenance [ 43 ]. It is a way to protect food production, reduce poverty, and eliminate waterrelated diseases. However, freshwater supply is a major global concern, particularly in developing countries [36]. In this context, based on the definition of WF and on previous studies [ 30 , 44 ], this study considers WF management to be the use of policies and actions at the strategic, tactical, and operational levels that support in mitigating the WF of a product across its entire supply chain. WF management is essential for promoting environmental sustainability in Sustainability 2022,14, 4928 4 of 17 the agri-food industry [ 11 ] and profitability through increased operational efficiency [ 39 ]. It has been widely used for evaluation of water efficiency in agricultural production since it is an important indicator to manage agricultural water resources. It is significantly affected by water management practices [ 13 ]. Due to its relevance, it has been incorporated into companies’ business strategies [30]. Researchers have been calling for a more holistic management of WF to address sustainable water use in the whole supply chain [ 8 ]. Sustainable water management in the supply chain is critical to the long-term viability of some agricultural organizations [ 45 ]. Chico et al. [ 46 ] consider that strategies to reduce the impact of a product WF need to take into account the WF along the whole chain [ 46 ]. More research is needed in order to determine actions to reduce the impacts of water usage along supply chain echelons [30]. 3. Materials and Methods Survey questionnaires are the most used tool to collect data because they allow data collection from a huge population quickly and at a lower cost. They are also convenient to respondents, can provide anonymous answers, reduce interviewer bias, and allow question standardization. However, some notorious disadvantages of a questionnaire include poor data quality (incomplete responses), wording problems, and inaccurate questions [47]. Establishing content validity involves judgment by experts as to whether scale items reflect the appropriate content to capture the concept being assessed. Content experts are required to assess content validity because they could potentially improve the content validity of measurement items [ 48 ]. Content validity determines whether the items of a survey questionnaire provide adequate coverage of a topic under investigation [49]. The methodology for identifying measurement items and validating their content was based on previous published studies [ 47 , 48 ], which establish rigorous procedures for scale development that include item generation and expert review [ 50 ]. In order to carry out this task, a deductive method was used. First, items were identified from a systematic literature review, then, experts reviewed each item and the construct definition as a whole [ 47 ] to assess their capacity of measuring the construct as well as their readability. Expert judges are highly knowledgeable about one knowledge domain. They evaluate each survey item to determine whether they represent the domain of interest. Their assessments have been quantified using formalized scaling and statistical procedures such as the content validity ratio for quantifying consensus [ 51 ]. They come both from academy and industry. They can be a person with knowledge and expertise in the construct being developed, someone familiar with the target population on whom the questionnaire will be used, questionnaire users, data analysts, or those who make decisions based on the outcomes of the survey [ 47 ]. Expert judges try to eliminate poorly worded questions and revise items’ phrasing so they can be maximally understood [51]. The content validity process is composed of three stages, as displayed in Figure 1. The methodology used in this study will be explained as follows. Sustainability 2022, 14, x FOR PEER REVIEW 4 of 17 In this context, based on the definition of WF and on previous studies [30,44], this study considers WF management to be the use of policies and actions at the strategic, tactical, and operational levels that support in mitigating the WF of a product across its entire supply chain. WF management is essential for promoting environmental sustainability in the agri-food industry [11] and profitability through increased operational efficiency [39]. It has been widely used for evaluation of water efficiency in agricultural production since it is an important indicator to manage agricultural water resources. It is significantly affected by water management practices [13]. Due to its relevance, it has been incorporated into companies’ business strategies [30]. Researchers have been calling for a more holistic management of WF to address sustainable water use in the whole supply chain [8]. Sustainable water management in the supply chain is critical to the long-term viability of some agricultural organizations [45]. Chico et al. [46] consider that strategies to reduce the impact of a product WF need to take into account the WF along the whole chain [46]. More research is needed in order to determine actions to reduce the impacts of water usage along supply chain echelons [30]. 3. Materials and Methods Survey questionnaires are the most used tool to collect data because they allow data collection from a huge population quickly and at a lower cost. They are also convenient to respondents, can provide anonymous answers, reduce interviewer bias, and allow question standardization. However, some notorious disadvantages of a questionnaire include poor data quality (incomplete responses), wording problems, and inaccurate questions [47]. Establishing content validity involves judgment by experts as to whether scale items reflect the appropriate content to capture the concept being assessed. Content experts are required to assess content validity because they could potentially improve the content validity of measurement items [48]. Content validity determines whether the items of a survey questionnaire provide adequate coverage of a topic under investigation [49]. The methodology for identifying measurement items and validating their content was based on previous published studies [47,48], which establish rigorous procedures for scale development that include item generation and expert review [50]. In order to carry out this task, a deductive method was used. First, items were identified from a systematic literature review, then, experts reviewed each item and the construct definition as a whole [47] to assess their capacity of measuring the construct as well as their readability. Expert judges are highly knowledgeable about one knowledge domain. They evaluate each survey item to determine whether they represent the domain of interest. Their assessments have been quantified using formalized scaling and statistical procedures such as the content validity ratio for quantifying consensus [51]. They come both from academy and industry. They can be a person with knowledge and expertise in the construct being developed, someone familiar with the target population on whom the questionnaire will be used, questionnaire users, data analysts, or those who make decisions based on the outcomes of the survey [47]. Expert judges try to eliminate poorly worded questions and revise items’ phrasing so they can be maximally understood [51]. The content validity process is composed of three stages, as displayed in Figure 1. The methodology used in this study will be explained as follows. Figure 1. Content validity methodology. Figure 1. Content validity methodology. 3.1. Item Identification A construct definition demands a careful examination of the literature [ 52 ]. In order to identify candidate measurement items for WF management, a systematic literature review (SLR) was conducted. SLRs search for, appraise and synthesize research evidence [ 53 ]. A SLR should be based on a clear search protocol. In this study, the review protocol used followed the procedures recommended by Tranfield et al. [ 54 ], which include the definition Sustainability 2022,14, 4928 5 of 17 of the search string, the search period, the definition of the databases to retrieve studies from, and the criteria to identify relevant studies. In terms of material collection, this study defines a keyword-based search protocol, but also other approaches, as recommended in previous studies [ 55 ]. A literature search was conducted using the terms [(“water footprint management”) OR (“WF management”)] AND (scale OR initiatives OR measurement OR indicators OR PLS-SEM) in Web of Science and Science Direct, which are databases used in previous studies [ 56 ]. The partial least squares structural equation method (PLS-SEM) has been included as a search parameter since it is a method that attempts to establish and assess cause-effect analysis among constructs, which are usually defined with a set of measurement items [57]. The search was executed, returning a total of 43 articles. Studies were selected based on inclusion and exclusion criteria. The inclusion criteria applied were: (1) peer-reviewed articles and book chapters written in English; and (2) studies that present WF measurement items or initiatives that reflect the adoption of WF management practices; (3) studies that present practices that can be applied with a supply chain focus; (4) studies with practices that can be applied to the agriculture sector. The exclusion criteria applied were: (1) reports and dissertations; and (2) studies that do not present initiatives or practices related to WF management. The application of these criteria was first based on an analysis of the study’s title and abstract, then the introduction and conclusion sections and, finally, the complete text, if necessary. This step led to a final sample of 21 relevant review papers. In order to expand the search, articles referenced by the ones initially identified were scanned (snowball search). This is in accordance with previous studies that stated that informal approaches and snowball methods are also important strategies to complement keyword-based searches [ 58 ]. Govindan and Hasanagic recommend performing manual searches in order to avoid limitations and to eliminate search errors [55]. The final set of papers was carefully read with the objective of identifying potential measurement items for WF management. According to our previously stated definition of this concept, policies and actions at the strategic, tactical, and operational levels that support managing and reducing the WF of a product across its entire supply chain were identified and considered as candidate measurement items. 3.2. Experts Selection The second phase of this study comprised the selection of expert judges to evaluate these candidate measurement items. There is no unique formula for expert selection. In general, researchers should use their common sense to define criteria for respondent selection [ 59 ]. Different criteria exist for selecting experts, such as the respondents’ hierarchical position and level of expertise, qualification and exposure to the problem being investigated [ 49 ], academic publications, professional experience in the field, membership of relevant institutions [ 59 ], personal involvement [ 60 ], respondents’ level of interest in participating in the study, and easiness of access to the experts [ 61 ]. Given the shortcomings implied in the various expert-identification procedures, researchers recommend combining different criteria [ 60 ]. Moreover, the panel of experts should be heterogeneous [ 59 ] to mitigate cognitive biases [61]. Experts were invited to participate based on the following criteria: (1) knowledgeable professionals in WF and water management or engaged in the agricultural industry; (2) experienced professionals with practical experience of at least 10 years in the field; (3) solid educational background and at least a bachelor’s degree on Agronomy, Agricultural Sciences, or related fields; (4) declared familiarity with the topics and concepts evaluated in this study; and (5) voluntary participation in this study. Similar criteria have been adopted in related studies [62–64]. This study was carried out with the support of Chilean experts on WF management and the agriculture industry. Some of the participating experts work in a public-private organization whose purpose is to promote the transformation of Chile towards sustainable development assuring a water transition that guarantees water security and water Sustainability 2022,14, 4928 6 of 17 availability in the country. The other participating experts include a PhD researcher in Environmental Sciences interested in water resources management in the agriculture industry, a senior specialist in hydrometry, founder of a company dedicated to hydrometry and environmental monitoring, and experienced food and agricultural engineers. In Chile, water scarcity problems have been observed for some years now [ 65 ]. Agricultural activities have intensified [ 66 ] to a level that the sector accounts for most of the consumptive extractions of water in the country [ 67 ]. There are foundations and organizations in the country concerned with water protection and management, so Chile counts with several experts in the field. The number of 8 participating experts was considered adequate for content validation, given that the number recommended to establish content validity ranges from a minimum of 3 to a maximum of 10 [47,50,59,68–70]. 3.3. Items’ Assessment The questionnaire sent to experts was introduced by a cover letter in order to explain why experts were invited to participate, along with clear and concise instructions on how to rate each item. A covering letter is an important part when sending a questionnaire for review [ 47 ]. Before the presentation of the candidate measurement items, the definition of the WF management construct was presented to experts, who were asked to evaluate the survey items based on this definition. Measurement items were evaluated according to three perspectives: relevance, clarity and essentiality. To do so, experts were given an appraisal sheet with the following inquiries: (1) the relevance of each item (how important the item is); (2) the clarity of each item (how clear their wording is); (3) the essentiality of each item (how necessary the item is); and (4) recommendations for improvement of each item, as recommended by Rodrigues et al. [ 71 ]. For the relevance scale, a 4-point Likert scale was used, and possible responses were: 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, and 4 = very relevant. A 3-point Likert scale was used for the clarity and essentiality scales. The clarity scale was composed of the following categories: 1 = not clear, 2 = item needs some revision; and 3 = very clear, and for essentiality, the following choices were possible: 1 = not essential; 2 = useful, but not essential; and 3 = essential. Experts could provide additional comments and recommendations for each item in the Comments column of the sheet, as recommended by Rodrigues et al. [ 71 ]. A final open remarks field was provided so experts could give their feedback on anything not covered by the questionnaire format, for example, an underrepresentation of the construct by the items [47]. As there is no statistical test to specifically assess content validity, researchers usually adopt a qualitative approach, through the assessment of expert judges, and then, a quantitative approach using the content validity index (CVI). The CVI measures the proportion or percentage of judges who agree on certain aspects of a tool and its items. Whenever the CVI refers to an item, it is called I-CVI. Ratings of 1 and 2 are considered content invalid while ratings of 3 and 4 are considered content valid, as suggested by Souza et al. [ 57 ]. A CVI of at least 0.80 and higher than 0.90 is accepted [ 47 , 57 ]. This index is simple to calculate, easy to understand, and it provides information about each item which guides their modification or deletion [72]. Despite its frequent use to assess content validity, CVI does not consider the possibility of inflated values due to chance agreement. The Kappa coefficient provides information about the degree of agreement beyond chance, and it is calculated as: K=I−CVI −Pc 1−Pc (1) where Pc =[N!/A!(N−A)!]∗0.5N (2) Pc is the probability of chance agreement; N is the number of experts; and A is the number of experts who agree about the item’s relevance. Kappa values above 0.74 are Sustainability 2022,14, 4928 7 of 17 considered excellent. Values between 0.60 and 0.74 are considered good while values between 0.40 and 0.59 are judged as fair [71–73]. This study has also used another index to calculate content validity—the content validity ration (CVR), which determines how many raters marked an item as essential [ 71 ]. The formula used for calculating this index is: CVR = (Ne −N/2)/N/2 (3) where Ne is the number of experts that rated the item as “essential”, and N is the total number of experts. CVR can range from − 1 to +1 [ 73 ]. A higher score indicates further agreement of experts on the necessity of an item in a questionnaire [ 72 ]. The cutoff values for CVR are determined by Lawshe’s Table [ 74 ]. According to this table, for a number of 8 experts, a CVR greater than 0.75 is considered acceptable. The last index used to assess content validity was Aiken’s V coefficient. The use of this coefficient is recommended because it can not only be used to summarize the magnitude of expert ratings, but also to test specific hypotheses concerning the values of the ratings for the population. This coefficient is calculated as: V = (X −l)/k (4) where X represents the mean of experts’ ratings, l represents the lowest possible rating, and k the range of possible values of scale adopted. This coefficient provides a measurement for rater endorsement, ranging from 0 to 1. A value of 1 is calculated when all experts choose the highest possible rating [ 75 ]. Values greater than 0.75 are in general accepted [ 76 ]. Finally, in order to assess content validity for the overall WF management construct, the average scale-CVI (S-CVI) was used, which is computed as the average of all items’ CVI. It is recommended that S-CVI should be higher than 0.8 to reflect content validity [77]. The content validity analysis was carried out online by experts from 27 October 2021 to 18 November 2021. As previously stated, items were evaluated according to their relevance, clarity and essentiality to represent the construct of WF management. Relevance was assessed with a 4-point Likert scale and clarity and essentiality with a 3-point Liker scale, following recommendations of previous studies. For each of these criteria, some content validity indices were calculated: I-CVI, Kappa’s coefficient and Aiken’s V coefficient. The CVR index was used for essentiality only. Table 1summarizes the content validity process used in this study. Table 1. Content validity process. Stages of the Content Validity Process Details Item identification •Systematic literature review (keyword-based search in scientific databases) and snowball search •Application of inclusion and exclusion criteria •Identification of candidate WF measurement items Expert selection •Definition of criteria for expert selection •Selection and invitation of experts •Calculation of expert’s authority coefficient Items’ assessment •Assessment of candidate measurement items in terms of relevance, clarity, and essentiality by experts •Calculation of several content validity assessment indices 4. Results As previously stated, the first step of this research comprised carrying out a systematic literature review to identify potential measurement items. The final set of papers identified by the SLR was carefully read with the objective of identifying candidate measurement Sustainability 2022,14, 4928 8 of 17 items for WF management. The candidate practices identified at this step, along with their referenced studies are shown in Table 2. Practice WF17 was excluded because it was considered similar to practice WF22. Table 2. Candidate WF measurement items. ID Candidate Measurement Item Reference WF1 Our company promotes and adopts global standards for water footprint accounting and assessment [3] WF2 Our company promotes water accounting over the whole supply chain, cooperating with others along the supply chain to be able to produce full accounts for final products WF3 Our company reports water-related efforts, targets and progress made in annual sustainability report, also covering the supply chain. WF4 Our company reduces the risk of pollution, avoiding or minimizing the use of substances in products that may be harmful when reaching the water WF5 Our company compensates downstream users that are affected by intensive upstream water use in the catchment where the company’s (residual) water footprint is located. WF6 Our company establishes agreements on reduction targets for water use with suppliers. WF7 Our company shifts to other supplier when they do not agree to water footprint standards. WF8 Our company would be willing to change its business model in order to incorporate or get better control over the supply chain in terms of water footprint reduction. WF9 Our company prioritizes the cultivation of crops requiring less water [30] WF10 Our company has converted conventional crops into organic crops WF11 Our company selects and collaborates with water-friendly partners WF12 Our company establishes water auditing and control systems WF13 Our company invests in water-efficient technologies WF14 Our company carries out campaigns for raising consumer awareness about water consumption reduction. WF15 Our company develops water stewardship programs [39] WF16 Our company sets targets to manage freshwater resources efficiently WF17 Our company integrates responsible water use into their business strategy (excluded) WF18 Our company adopts techniques and processes to enhance water retention in the soil. [44] WF19 Our company reuses and recycles wastewater WF20 Our company conducts integrated water assurance assessment of the supply chain. [7] WF21 Our company identifies the local water risks WF22 Our company embeds water stewardship into the corporate strategy WF23 Our company engages with various stakeholders for water stewardship The second phase of this study comprised the evaluation of these candidate measurement items by expert judges. Table 3shows the profile of the experts who participated in this study. Sustainability 2022,14, 4928 9 of 17 Table 3. Experts’ profile. Categories Number of Experts Education Bachelor’s degree 1 Master’s degree 6 Doctor’s degree 1 Position CEO/Director 4 Manager/Coordinator 4 WF familiarity/knowledge High 6 Average 2 The expert’s authority coefficient (Cr) was calculated to express how knowledgeable and familiar with the agriculture industry and the concept of WF the selected experts are. This coefficient was determined by three factors: the expert’s academic level (q1), the hierarchical position the expert occupies in his/her current job (q2), and the degree of familiarity with the concept of WF (q3). Each dimension was divided into three degrees, indicating the lowest and highest classification, as recommended by Shi et al. [ 64 ]. The expert academic level weight assignment considered that a PhD’s degree was weighted as 0.3, a master’s degree or equivalent was weighted as 0.2, and a bachelor’s degree had a weight of 0.1. For the hierarchical positions, CEO and directors were assigned a value of 0.3, for managers and coordinators, a value of 0.2 while 0.1 was given to professionals in operational positions. The familiarity with water footprint was evaluated as high (0.4), average (0.25), and low (0.15). Therefore, in this study, the expert’s authority coefficient was calculated as the average of q1, q2, and q3, therefore, equal to 0.81, indicating that the experts participating in this study were authoritative, since Cr was higher than 0.7 [63]. Table 4shows the values calculated for the indices assessed in this study for each candidate measurement item. Table 4. Content validity indices for candidate items. ID Relevance Clarity Essentiality I-CVI Kappa Aiken’s V I-CVI Kappa Aiken’s V I-CVI Kappa Aiken’s V CVR WF1 1.00 1.00 0.88 0.75 0.72 0.88 0.88 0.87 0.94 0.75 WF2 1.00 1.00 0.79 0.25 0.16 0.63 0.88 0.87 0.88 0.75 WF3 1.00 1.00 0.75 0.88 0.87 0.94 0.38 0.20 0.69 −0.25 WF4 0.88 0.87 0.92 0.75 0.72 0.88 0.88 0.87 0.94 0.75 WF5 0.88 0.87 0.79 0.50 0.31 0.63 0.38 0.20 0.56 −0.25 WF6 1.00 1.00 0.92 0.88 0.87 0.94 0.75 0.72 0.88 0.50 WF7 0.88 0.87 0.83 0.88 0.87 0.88 0.50 0.31 0.69 0.00 WF8 0.88 0.87 0.88 1.00 1.00 1.00 0.75 0.72 0.75 0.50 WF9 0.63 0.52 0.58 0.88 0.87 0.94 0.50 0.31 0.69 0.00 WF10 0.63 0.52 0.50 0.88 0.87 0.94 0.13 0.10 0.25 −0.75 WF11 0.75 0.72 0.67 0.88 0.87 0.94 0.38 0.20 0.63 −0.25 WF12 1.00 1.00 0.96 0.88 0.87 0.94 1.00 1.00 1.00 1.00 WF13 1.00 1.00 0.96 0.88 0.87 0.94 1.00 1.00 1.00 1.00 WF14 0.75 0.72 0.71 0.88 0.87 0.94 0.38 0.20 0.69 −0.25 WF15 1.00 1.00 0.96 0.75 0.72 0.88 0.75 0.72 0.81 0.50 WF16 0.75 0.72 0.79 0.63 0.52 0.69 0.50 0.31 0.69 0.00 WF18 1.00 1.00 0.92 1.00 1.00 1.00 0.75 0.72 0.88 0.50 WF19 1.00 1.00 1.00 1.00 1.00 1.00 0.88 0.87 0.94 0.75 WF20 0.75 0.72 0.63 0.13 0.10 0.38 0.38 0.20 0.69 −0.25 WF21 0.88 0.87 0.92 0.75 0.72 0.88 0.88 0.87 0.94 0.75 WF22 1.00 1.00 0.96 0.88 0.87 0.94 0.75 0.72 0.88 0.50 WF23 1.00 1.00 0.92 0.38 0.20 0.63 0.75 0.72 0.88 0.50 Sustainability 2022,14, 4928 16 of 17 64. Shi, C.; Zhang, Y.; Li, C.; Li, P.; Zhu, H. Using the Delphi method to identify risk factors contributing to adverse events in residential aged care facilities. Risk Manag. Healthc. Policy 2020,13, 523–537. [CrossRef] 65. Novoa, V.; Ahumada-Rudolph, R.; Rojas, O.; Munizaga, J.; Sáez, K.; Arumí, J.L. Sustainability assessment of the agricultural water footprint in the Cachapoal River basin, Chile. Ecol. Indic. 2019,98, 19–28. [CrossRef] 66. Yevenes, M.A.; Arumí, J.L.; Farías, L. Unravel biophysical factors on river water quality response in Chilean Central-Southern watersheds. Environ. Monit. Assess. 2016,188, 264. [CrossRef] 67. Novoa, V.; Ahumada-Rudolph, R.; Rojas, O.; Sáez, K.; de la Barrera, F.; Arumí, J.L. Understanding agricultural water footprint variability to improve water management in Chile. Sci. Total Environ. 2019,670, 188–199. [CrossRef] [PubMed] 68. Li, X.; Su, X.; Du, Y. The environmental sustainability of an exhibition in visitors’ eyes: Scale development and validation. J. Hosp. Tour. Manag. 2021,46, 172–182. [CrossRef] 69. Piha, L.; Papadas, K.; Davvetas, V. Brand orientation: Conceptual extension, scale development and validation. J. Bus. Res. 2021 , 134, 203–222. [CrossRef] 70. Lynn, M.R. Determination and quantification of content validity. Nurs. Res. 1986,35, 382–386. [CrossRef] 71. Rodrigues, I.B.; Adachi, J.D.; Beattie, K.A.; MacDermid, J.C. Development and validation of a new tool to measure the facilitators, barriers and preferences to exercise in people with osteoporosis. BMC Musculoskelet. Disord. 2017,18, 540. [CrossRef] 72. Zamanzadeh, V.; Ghahramanian, A.; Rassouli, M.; Abbaszadeh, A.; Alavi-Majd, H.; Nikanfar, A.-R. Design and implementation content validity study: Development of an instrument for measuring patient-centered communication. J. Caring Sci. 2015 ,4, 165–178. [CrossRef] 73. Jansen, M.; Doornebosch, A.J.; de Waal, M.W.M.; Wattel, E.M.; Visser, D.; Spek, B.; Smit, E.B. Psychometrics of the observational scales of the Utrecht Scale for Evaluation of Rehabilitation (USER): Content and structural validity, internal consistency and reliability. Arch. Gerontol. Geriatr. 2021,97, 104509. [CrossRef] 74. Lawshe, C.H. A quantative approach to content validity. Persistent Psychol. 1975,28, 563–575. [CrossRef] 75. Penfield, R.D.; Giacobbi, P.R. Measurement in physical education and exercise science applying a score confidence interval to Aiken’s item content-relevance index. Meas. Phys. Educ. Exerc. Sci. 2004,8, 213–225. [CrossRef] 76. Acosta-Banda, A.; Aguilar-Esteva, V.; Ortiz, M.P.; Ortiz, J.P. Construction and Validity of an Instrument to Evaluate Renewable Energies and Energy Sustainability Perceptions for Social Consciousness. Sustainability 2021,13, 2333. [CrossRef] 77. Shrotryia, V.K.; Dhanda, U. Content validity of assessment instrument for employee engagement. SAGE Open 2019 ,9, 1–7. [CrossRef] 78. ISO 14046:2014; Environmental Management—Water Footprint—Principles, Requirements and Guidelines. ISO: Geneva, Switzerland, 2014. 79. Bai, X.; Ren, X.; Zheng, N.; Zhou, N.; Hu, M. Comprehensive water footprint assessment of the dairy industry chain based on ISO 14046: A case study in China. Resour. Conserv. Recycl. 2018,132, 369–375. [CrossRef] 80. Gilsbach, L.; Schütte, P.; Franken, G. Applying water risk assessment methods in mining: Current challenges and opportunities. Water Resour. Ind. 2019,22, 100118. [CrossRef] 81. Dania, W.A.P.; Xing, K.; Amer, Y. Collaboration behavioural factors for sustainable agri-food supply chains: A systematic review. J. Clean. Prod. 2018,186, 851–864. [CrossRef] 82. Asian, S.; Hafezalkotob, A.; John, J.J. Sharing economy in organic food supply chains: A pathway to sustainable development. Int. J. Prod. Econ. 2019,218, 322–338. [CrossRef] 83. Adams, D.; Donovan, J.; Topple, C. Achieving sustainability in food manufacturing operations and their supply chains: Key insights from a systematic literature review. Sustain. Prod. Consum. 2021,28, 1491–1499. [CrossRef] 84. Chapagain, A.K.; Network, W.F.; Hague, T. Water Footprint: State of the Art: What, Why, and How? Elsevier: Amsterdam, The Netherlands, 2017; Volume 4, ISBN 9780124095489. 85. Schornagel, J.; Niele, F.; Worrell, E.; Böggemann, M. Water accounting for (agro) industrial operations and its application to energy pathways. Resour. Conserv. Recycl. 2012,61, 1–15. [CrossRef] 86. Sanders, K.T.; Masri, S.F. The energy-water agriculture nexus: The past, present and future of holistic resource management via remote sensing technologies. J. Clean. Prod. 2016,117, 73–88. [CrossRef] 87. Pantazi, X.E.; Moshou, D.; Bochtis, D. Sensors in agriculture. Intell. Data Min. Fusion Syst. Agric. 2020,1, 1–15. [CrossRef] 88. Cazcarro, I.; Hoekstra, A.Y.; Sánchez Chóliz, J. The water footprint of tourism in Spain. Tour. Manag. 2014 ,40, 90–101. [CrossRef] 89. Nishant, R.; Kennedy, M.; Corbett, J. Artificial intelligence for sustainability: Challenges, opportunities, and a research agenda. Int. J. Inf. Manag. 2020,53, 102104. [CrossRef] 90. Elbeltagi, A.; Deng, J.; Wang, K.; Hong, Y. Crop water footprint estimation and modeling using an artificial neural network approach in the Nile Delta, Egypt. Agric. Water Manag. 2020,235, 106080. [CrossRef] 91. Xiang, X.; Li, Q.; Khan, S.; Khalaf, O.I. Urban water resource management for sustainable environment planning using artificial intelligence techniques. Environ. Impact Assess. Rev. 2021,86, 106515. [CrossRef] 92. Klemeš, J.J.; Varbanov, P.S.; Lam, H.L. Water Footprint, Water Recycling and Food-Industry Supply Chains; Woodhead Publishing Limited: Sawston, UK, 2009. 93. Khan, S.A.R.; Ponce, P.; Yu, Z.; Golpira, H.; Mathew, M. Environmental technology and wastewater treatment: Strategies to achieve environmental sustainability eris. Chemosphere 2022,286, 131532. [CrossRef] [PubMed] Sustainability 2022,14, 4928 17 of 17 94. Negreiros, F.; Oliveira, D.; Leiras, A.; Ceryno, P. Environmental risk management in supply chains: A taxonomy, a framework and future research avenues. J. Clean. Prod. 2019,232, 1257–1271. [CrossRef] 95. Barbosa, M.W. Uncovering research streams on Agri-Food Supply Chain Management: A bibliometric study. Glob. Food Sec. 2021 , 28, 100517. [CrossRef] 96. Román, R.; Cansino, J.M.; Rueda-Cantuche, J.M. A multi-regional input-output analysis of ozone precursor emissions embodied in Spanish international trade. J. Clean. Prod. 2016,137, 1382–1392. [CrossRef] 97. Wang, W.; Wang, J.; Cao, X. Water use efficiency and sensitivity assessment for agricultural production system from the water footprint perspective. Sustainability 2020,12, 9665. [CrossRef] 98. Barbosa, M.W.; Ladeira, M.B.; de Oliveira, M.P.V.; de Oliveira, V.M.; de Sousa, P.R. The effects of internationalization orientation in the sustainable performance of the agri-food industry through environmental collaboration: An emerging economy perspective. Sustain. Prod. Consum. 2022,31, 407–418. [CrossRef] 99. Aivazidou, E.; Tsolakis, N. Water Footprint Management in the Fashion Supply Chain: A Review of Emerging Trends and Research Challenges; Elsevier: Amsterdam, The Netherlands, 2019; ISBN 9780081026335.