A Multi-Criteria Decision Support Concept for Selecting the Optimal Contractor

applied sciences Article A Multi-Criteria Decision Support Concept for Selecting the Optimal Contractor Ivan Marovi´c 1,* , Monika Peri´c 1and Tomaš Hanak 2   Citation: Marovi´c, I.; Peri´c, M.; Hanak, T. A Multi-Criteria Decision Support Concept for Selecting the Optimal Contractor. Appl. Sci. 2021, 11, 1660. https://doi.org/10.3390/ app11041660 Academic Editors: Bo˙ zena Hoła and Anna Hoła Received: 30 December 2020 Accepted: 9 February 2021 Published: 12 February 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). 1Faculty of Civil Engineering, University of Rijeka, HR-51000 Rijeka, Croatia; [email protected] 2Faculty of Civil Engineering, Brno University of Technology, 602 00 Brno, Czech Republic; [email protected].cz *Correspondence: ivan.mar[email protected]; Tel.: +385-51-265-921 Abstract: A way to minimize uncertainty and achieve the best possible project performance in construction project management can be achieved during the procurement process, which involves selecting an optimal contractor according to “the most economically advantageous tender.” As resources are limited, decision-makers are often pulled apart by conflicting demands coming from various stakeholders. The challenge of addressing them at the same time can be modelled as a multi-criteria decision-making problem. The aim of this paper is to show that the analytic hierarchy process (AHP) together with PROMETHEE could cope with such a problem. As a result of their synergy, a decision support concept for selecting the optimal contractor (DSC-CONT) is proposed that: (a) allows the incorporation of opposing stakeholders’ demands; (b) increases the transparency of decision-making and the consistency of the decision-making process; (c) enhances the legitimacy of the final outcome; and (d) is a scientific approach with great potential for application to similar decision-making problems where sustainable decisions are needed. Keywords: contractor selection; multi-criteria decision making; decision support concept; AHP; PROMETHEE; construction procurement 1. Introduction Selecting the optimal contractor for construction projects can be seen as the most important strategic decision in such an investment, one which can have long lasting effects that may emerge not only during the particular project, but assuredly during its exploitation phase. At the same time, it is one of the most important decisions made by the clients. Often, decision-making and decision support in civil engineering is solely based on cost-benefit analysis (CBA). However, this has been found to be highly inadequate, both in terms of incorporation and assessment of multiple-criteria like environmental and wider economic issues which are usually essentially difficult to quantify, and because traditional CBA relies heavily on estimating both demand forecasts and construction costs [ 1 – 3 ]. Over the years, various researchers dealt with such aspects of project performance, claiming that demand forecasts and construction cost estimations in particular are subject to a large degree of uncertainty—commonly referred to as optimism bias [4–12]. In order to minimize uncertainty and achieve the best possible project performance, two EU Directives were implemented in 2004 that allowed a codification of rules and procedures across EU countries regarding public procurement—Directives 2004/18 /EC and 2004/17/EC. These directives guided contracting authorities, i.e., clients, to approach their projects in a more strategic and forward-looking way in order to achieve successful and thus sustainable projects. Accordingly, public procurement should be based on disinterested criteria [ 13 ] that ensure compliance with transparency, nondiscrimination, equal treatment, and with guarantees that tenders are evaluated in circumstances of effective competition. Such can be achieved by two approaches: “the lowest price” and “the most economically advantageous tender.” Both approaches are present in EU countries, as each Appl. Sci. 2021,11, 1660. https://doi.org/10.3390/app11041660 https://www.mdpi.com/journal/applsci Appl. Sci. 2021,11, 1660 2 of 17 country often builds in some specificities, but almost as a rule it comes down to a single criterion, i.e., price. Dealing with a number of criteria directly implies the need to use multi-criteria methods that are usually perceived as “difficult to understand.” In the latter approach, the criteria related to the particular public procurement (article 53 of Directive 2004/18/EC and 55 of Directive 2004/17/EC) are in the hands of the clients, the contracting authorities, and therefore vary from one tender to another. As it is a multi-criteria problem, the use of any multi-criteria decision-making method seems to be the right choice if the decision-maker is aiming toward a consistent decision-making process from beginning to end. In general, the field of strategic management is not defined by a particular theoretical paradigm, but rather by its focus on a particular dependent variable—overall organizational performance—and the role of managers in shaping that performance [ 14 ], but also by extending, clarifying and applying such theories in new and interesting ways [ 15 ]. The strategic management process advocated by [ 16 ] has been defined as comprising a sequential set of analyses and choices that can increase the likelihood that a company will choose a strategy that generates competitive advantage. This can also be applied to projects, programs, and portfolios. Similarly, such strategic thinking can be applied to the problem of selecting the optimal contractor for a particular construction investment by applying multi-criteria decision analysis (MCDA) approach and a logic of decision support systems (DSS). Salling and Pryn [ 1 ] proposed a decision support model named SUSTAIN-DSS to bring informed decision support, both in terms of single aggregated estimates, i.e., deterministic calculation, and also in terms of interval results by certainty graphs, i.e., stochastic calculation. Such interaction enabled the analysts to investigate not only the feasibility of risk when assessing investment projects [ 5 ] but also to highlight the importance of expanding the decision-making process beyond the consideration of solely economic factors and point estimates. Various researchers [ 17 – 22 ] proposed different MCDA approaches based on value measurement using qualitative inputs from a ratifying stakeholder group via multiplicative analytic hierarchy process (AHP), which were found to be well suited for group decision-making. An extensive literature review from 2000 to 2018 [ 23 ] led to the identification and classification of commonly used criteria in construction procurement, commonly used decision-making techniques, and the origins of researchers working on this topic. In the last decades there have been a number of papers dealing with outranking methods in construction project management, focusing on AHP and/or PROMETHEE methods [ 24 ]. While some authors focused on the AHP method [ 25 – 30 ], the analytic network process [ 31 ] or PROMETHEE methods [ 20 , 32 – 37 ] the important driver was given by [ 38 ] to use these methods in synergy to achieve the most in a multi-stakeholder environment [ 19 , 39 – 45 ]. To tackle the problem of selecting an appropriate contractor, various authors approach the problem from the stakeholder point-of-view using the AHP or group AHP [ 21 , 29 – 31 , 46 , 47 ], while others consider the problem as an overall approach of managing stakeholders such as the multi-actor multi-criteria analysis methodology, i.e., MAMCA [ 19 , 39 ], or decision support concept, i.e., DSC [32,34,35,42]. Regardless of the approach used, in order to determine “the most economically advantageous tender” it is important to address not only technical aspects, but also economic, social, environmental, and other aspects of the tenderers as well as the long-term impact of the project outcomes as a whole. Therefore, such requirements can be achieved be establishing adequate selection criteria during the procurement process. This has been done by all the previously mentioned researchers, but some focused on defining the main criteria in a more detail way [ 3 , 13 , 48 – 54 ] by using them to select tenderers in competitive tendering systems. In this context, the main objective is to develop a decision support concept for the selecting the optimal contractor based on the synergy effect of the AHP (for the development of the hierarchical goal structure) and PROMETHEE methods (for the pairwise comparison Appl. Sci. 2021,11, 1660 3 of 17 of alternatives, i.e., tenderers/contractors). An additional aim is to define and implement a multi-stakeholder management procedure during the construction procurement process that: (a) allows the incorporation of opposing stakeholders’ demands; (b) increases the transparency of decision-making and the consistency of the decision-making process; ( c ) enhances the legitimacy of the final outcome; and (d) is a scientific approach with great potential to be applied in similar decision-making problems where sustainable decisions are needed. 2. Methods and Methodology To ensure that the construction project can be successfully completed regarding the projects’ scope, time, costs and quality, the client must select the most appropriate contractor, regardless of the type of investment, private or public. This involves a procurement system that comprises several process elements (project packaging, invitation to compete, prequalification, short-listing and bid evaluation). The existing literature on contractor selection mainly deals with how to identify and evaluate the criteria, thus providing the general lists of criteria for managing purposes in civil engineering. A more promising approach that classifies the criteria for contractor selection has been provided by Hatush and Skitmore [ 48 , 49 ], and Cheng and Li [ 31 ]. Taking their approach into account, i.e., focusing exclusively on the elements of prequalification and bid evaluation in construction procurement, served as the basis for the proposed decision support concept. 2.1. Data and Methods In order to address how the existing body of knowledge in civil engineering has developed in the direction of construction procurement, especially the contractor selection problem, a systematic literature review was conducted in this study as well as direct correspondence and collaboration with experts. A systematic literature review was conducted for the purpose of multi-stakeholder analysis and establishing the hierarchical goal structure. The review was conducted in the Scopus and Web of Science databases using selected keywords (group decision-making, multi-criteria, contractor selection, decision support, construction procurement, AHP, PROMETHEE), and their syntax derivatives. To ensure the high quality and novelty of the analyzed knowledge, only papers published in scientific journals between January 2000 and December 2020 were considered. This resulted in a list of seven criteria that are most commonly used to select the optimal contractor. This list of criteria was used in collaboration with two different groups of experts (contractors and clients). The first group of experts, i.e., the contractor group, consisted of eight private contractors selected from the local area. All examinees from this group are experts in the field of construction procurement with 15 (2 examinee), 25 (5 examinees) and 30 (4 examinees) years of experience and work at strategic management levels at their companies. Some contractors were represented by more than one representative, but their opinion was used in the further analysis as a single one, i.e., company point of view. The second expert group, i.e., the client group, consisted of 13 public contractors selected from the local area representing local government (5), government agencies (3), and universities (5). All examinees from this group are experts in the field of construction management and/or construction procurement with 15 (4 examinee), 20 (6 examinees), 25 (5 examinees) and 30 (3 examinees) years of experience and work at tactical and/or strategic management levels. As some clients were represented by more than one representative, their opinion was used in further analysis as a single one, i.e., client point of view. By means of structured interviews as well as workshops, both groups participated in collective decision-making by expressing their view on criteria using the AHP and Saaty scale. This served not only as a participatory process in decision-making where stakeholders adopt the decisions through a majority vote [ 46 ], but also in seeking the agreement of those who participate by generating consensus among them. This resulted in Appl. Sci. 2021,11, 1660 4 of 17 two points-of-view, that of the clients and that of the contractors, which will be discussed further in Section 3. Since the identified criteria are both quantitative and qualitative, another outranking method, PROMETHEE, was used for ranking of the tenderers as an appropriate MCDA method for solving such problems. For this purpose, experts from the client group were asked to evaluate each tenderer in relation to each criterion, resulting in a decision matrix that was used for prioritization. The proposed decision support concept was tested on a case study, a small multistory residential building, while the multi-stakeholder analysis and multi-criteria decision analysis were tested by involving experts from public and private procurement as mentioned. 2.2. Concept Development The proposed decision support concept for selecting the optimal contractor (DSCCONT) consists of several processes, as shown in Figure 1. The focus of the proposed concept is a two-stage procurement procedure: (1) prequalification, and (2) evaluation of tenderers. To achieve the best possible outcome, the DSC-CONT uses the synergy of the AHP and PROMETHEE methods. This approach of using the synergy of the AHP and PROMETHEE has been previously tested in various multi-criteria problems [ 20 , 32 , 35 – 37 ] and showed promising results. This is due to the strength of AHP in creating a hierarchical goal structure and the strength of PROMETHEE in ranking alternatives according to criteria that are evaluated both quantitatively and qualitatively. Creating such operational synergies by strengthening PROMETHEE with AHP gives the robustness and consistency in the decision-making process of the DSC-CONT. This approach is preferred by the authors, based on their own experience with similar methodological approaches, but also because of the research of other authors [19,33,34,39,41–44,55–57]. Figure 1. Decision support concept for selecting optimal contractor. Appl. Sci. 2021,11, 1660 5 of 17 The novelty of the proposed concept is in its robustness and resilience to changes in the decision-making process, especially in allowing stakeholders to express their attitudes and their opposing demands. The methods used provide stakeholders with the opportunity to express their attitudes in a clear way. At the same time, the transparency of decision-making is increased and the legitimacy of the final outcome is strengthened. The advantage of such an approach is that even if there is a change in the structure of the decision-makers, the decision-making procedure itself remains intact and consistent. Moreover, the proposed concept takes into account EU directives and can be easily implemented in all public construction tenders regulated by Directives 2004/18 /EC and 2004/17/EC. The DSC-CONT consists of two processes. During the prequalification process, it is important to compare key contractor-organizational criteria among a group of contractors desirous to tender. Such criteria can be identified in various ways. In general, this concept provides a hierarchical goal structure procedure (Figure 2) and brings stakeholders into the middle of the analysis. This is done by applying the AHP logic and giving stakeholders the opportunity to reach consensus in order to come up with a sustainable solution. The AHP [ 17 , 58 ] is used to determine the importance of the main goal, objectives and criteria of each stakeholder group (client and contractor). Depending on whether the aggregation is performed at the comparison level or at the priorities level, the procedure differs but the result remains the same, i.e., the hierarchical goal structure is formed with all weights. This is done by the multi-stakeholder analysis, while the contractor analysis offers the insight into the alternatives, i.e., contractors/tenderers, which leads to their evaluation according to previously defined criteria. Figure 2. Hierarchical goal structure procedure [37]. The following process is the evaluation of tenderers and here essentially the multicriteria decision analysis is carried out. Since the previously defined criteria can be both qualitative and/or quantitative, the DSC-CONT uses the strengths of the PROMETHEE methods for ranking the alternatives. Here, the PROMETHEE II method [ 59 – 62 ] is used to obtain a complete ranking, but before a final rank-list is produced, it is important to check the results using VisualPROMETHEE [ 63 ] features, such as PROMETHEE Diamond and/or PROMETHEE Network, as well. The rank-list provides the decision-maker with the basis for making a final decision, especially if it is presented graphically. While the evaluation of tenderers process considers specific criteria that can measure the suitability of the tenderers, i.e., contractors, it is not equivalent to the contractor selection process, although in practice it is considered to be one. Since the evaluation of tenderers is the process of investigating or measuring specific project attributes, the contractor selection is referred to as the process of aggregating the results of the evaluation to identify the optimal choice. Cheng and Li [ 31 ] also highlighted this: “In practice, these two processes are always grouped together to represent a single procedure to prioritize the contractors according to the project specific criteria”. Overall, the DSC-CONT provides the decisionmaker with a tool to identify, evaluate, and analyze, but the final decision is always in their hands. Appl. Sci. 2021,11, 1660 6 of 17 2.2.1. Building the Hierarchy The stakeholder management is often seen as the most important part of construction project management [ 37 ], directly affecting the projects’ scope, time, cost and quality. Therefore, to manage them proactively by capturing their attitudes, the hierarchical goal structure (HGS) procedure (Figure 2) is applied. This particular procedure has been used in some previous research [ 20 , 24 , 36 , 37 ] and showed promising results in multi-stakeholder analysis. The main advantages are the clear goal hierarchy by allowing stakeholders, i.e., experts, to participate in the creation of the hierarchical goal structure, but also to express their attitude towards each criterion. Assessing weights, i.e., stating attitudes, often seems to be the weakest element due to the subjective approach, but in the case where the search for consensus on weighting of each criterion is a necessity, this leads to a consensus weighting of all involved stakeholders and can therefore be considered objective. Nevertheless, the responsibility is in the hands of the decision-maker and his ability to involve all relevant stakeholders in the HGS procedure. The proposed HGS procedure ensures insight into the definition of objectives (O) and criteria (C) of the defined main goal (MG). Since stakeholder relationships are not static, but on the contrary dynamic and in constant change [ 42 ], their attitudes and actions may change at different project stages, and endanger the overall performance of the project. Since the hierarchical goal structure procedure is an iterative process that ends when all stakeholders agree, the decision-maker can be sure that if the procedure is followed, all stakeholders’ attitudes are embedded in the criteria, the objectives and the main goal. The result of this procedure is a list of criteria, as shown in Table 1, and gives all stakeholders involved a clear insight into the HGS and how each element is described, evaluated and preferred. One can be assured that by completing and fulfilling each criteria, the main goal will be achieved as an outcome of the process. In addition, this becomes a transparent tool for the weighting phase. Table 1. Criteria with short description, evaluation technique, and preference. Criteria Label Criteria Name Short Description of Criteria and Evaluation Technique Preference (Min/Max) C1 Tender price Tender price that includes construction/reconstruction costs according to bill of quantities and technical documentation; Expressed in 1.000 € Min C2 Expected duration Expected duration of construction/reconstruction according to bill of quantities and dynamic plan; Expressed in weeks Min C3 Quality Expert assessment taking into account quality of the work to be done; Qualitative scale 1 (min) to 5 (max) Max C4 Past relationship Expert assessment taking into account past relationship of the tenderer; Qualitative scale 1 (min) to 5 (max) Max C5 Resources Expert assessment taking into account tenderers’ capabilities of possible allocation of the resources for finishing construction/reconstruction in given project constraints; Qualitative scale 1 (min) to 5 (max) Max C6 WLCC Expert assessment of project’s impact on whole life-cycle costs (WLCC); Qualitative scale 1 (min) to 5 (max) Min C7 Past experience Expert assessment taking into account past experience of the tenderer; Qualitative scale 1 (min) to 5 (max) Max As mentioned earlier, this list of criteria (Table 1) resulted from the systematic literature review and the stakeholders were asked to state their attitudes only about them. It is important to emphasize that due to the differences in construction projects and tenders, the proposed HGS procedure offers the possibility to update this list or to create a completely Appl. Sci. 2021,11, 1660 7 of 17 new, i.e., customized, list of criteria that provides the best results in terms of the projects’ scope, time, cost and quality. 2.2.2. Weighting Phase Once the HGS is made, it is necessary to determine their importance, i.e., their weights. In a multi-stakeholder environment, this can be achieved in various ways. In this particular case, each stakeholder group (contractors and clients) has been given the opportunity to express their point-of-view. Typically, stakeholders think that their own expectations have not been taken properly into account. Therefore, it is of utmost importance that this procedure is transparent and all their attitudes and actions are considered as a part of the collaborative governance [46,64,65]. The AHP method and Saaty scale (1–9) were used for weighting. Since there may be multi-stakeholders in each group, we proposed the weight aggregation at the comparison level of each group. The multiplicative AHP is useful for stakeholders and decision-makers to align common viewpoints and ultimately reach an agreement, i.e., consensus. Each group can be further analyzed as a separate scenario and its consensus as a standalone scenario, if needed. 2.2.3. Ranking Procedure While the AHP was used for the definition of HGS and weighting, the PROMETHEE methods are recommended as appropriate ones for the MCDA of the proposed decision concept. It is supported by the fact that there are different types of criteria which can be both qualitative and quantitative. Such cases are very common when dealing with criteria that involve various technical, economic, social, and environmental aspects. Since the general objective of this process is to rank and compare all alternatives, it is of utmost importance for decision analysts to prepare the results as graphically as possible. In this case, the use of PROMETHEE II results should be supported by graphical representation of PROMETHEE Diamond and/or PROMETHEE Network. The above is explained in more detail in the following section. 3. Results and Discussion Once the HGS is created, it enables collaboration with the identified stakeholders. In this case, two stakeholder groups have been identified; contractors and clients. Stakeholders from both groups were interviewed about the HGS, especially ranking criteria. For the purpose of this study, the proposed concept is tested on a case study of a small multistory residential building. The central issue is to show the possibilities offered by the DSC-CONT, rather than the selection of the contractor in an actual tender. Therefore, in order to present the procedure, the criteria have been defined as previously described. At the same time, the procedure of creating HGS is also presented. This will allow decision-makers the opportunity to create HGS according to the specifics of their tender. To provide insight into DSC-CONT and achieve the defined goals of the study, this section begins with the prequalification process and multi-stakeholder and contractor analysis. The interviews were conducted in one-on-one sessions where each stakeholder had the opportunity to reflect on given criteria and assign the weights. Each stakeholder made the pairwise comparisons for the defined criteria. Different scales were proposed to them to transform their judgments into numbers of the pairwise comparison. The one that was used in the end was Saaty’s linear scale, where the values of comparison range from 1 (indifference) to 9 (extreme preference). This stage corresponds to the collaborative part of the governance process, where all the preferences, likes, and desires, i.e., attitudes of the stakeholders are included in matrices of pairwise comparison. By collecting their judgment in square matrices, the relative dominance of one criterion over the other is generated. Each stakeholder participated in the elaboration of the matrices among the experts who designed HGS and the final result was presented to them at the end. Appl. Sci. 2021,11, 1660 8 of 17 The first group, i.e., contractors, consisted of eight selected contractors from the local area, as we saw in Section 2.1. They were all technical managers and/or general managers in small and medium-sized enterprises (SMEs). This group was asked to weight the criteria as they would like them to be evaluated in future tenders. Their respective weightings are shown in Figure 3. Figure 3. Weights of each criterion—contractors’ point-of-view. As mentioned earlier, some contractors saw certain criteria differently. From their point of view, the three most important criteria were defined as quality, tender price, and past experience. At the same time, three criteria have relative peaks in contrast to certain attitudes. Those criteria are quality (Figure 3, Series 3), whole life-cycle costs, i.e., WLCC (Figure 3, Series 7), and past experience (Figure 3, Series 8). It is interesting to see that even with a small number of experts involved, their attitudes differ significantly. In this case, the reasons can be found in their specializations. Consequently, in Series 3 the experts come from a company specialized in prefabricated buildings, in Series 7 the experts come from a company specialized in Design-Build projects, and in Series 8 the experts come from a company with a 55 year tradition in civil engineering. In summary, even with a small poll of stakeholders, the DSC-CONT provides the opportunity to incorporate opposing stakeholders’ demands while increasing the transparency of the decision-making process. When applying the AHP method, the consistency ratio (CR) must be considered. For all the contractors’ matrices, the inconsistency found was less than 0.1, which means that the weights were calculated correctly. In order to evaluate them as a single group, i.e., as a scenario, the overall matrix was created with aggregated values (Table A1). The aggregation of each pairwise was done using the median, and the final weights are presented in Table 2 . The CR is 0.08. These weights were used for the following evaluation of tenderers and presents Scenario 1—contractor group. To conclude, this approach additionally gives transparency during aggregation as all stakeholders’ demands are included in the decisionmaking process. Table 2. Contractors’ aggregated weights. Tender Price Expected Duration Quality Past Relationship Resources WLCC Past Experience wi21.9 11.6 25.7 13.1 4.2 4.9 18.6 Appl. Sci. 2021,11, 1660 9 of 17 The same approach was carried out for the second group, i.e., clients, which consisted of 13 selected clients from the local area, as we saw in Section 2.1. They were all public sector clients and mandatory users of public procurement. Five examinees represent the university experts’ point of view, five represent local government on city municipality regions, and three at the regional government agencies. This group was asked to weight criteria in order to select the best contractor in the future tenders. Their respective weightings are shown in Figure 4. Figure 4. Weights of each criterion—clients’ point-of-view. As mentioned earlier, some clients see certain criteria differently. Their view resulted in defining the three most important criteria, namely WLCC, quality, and tender price. At the same time, two criteria have relative peaks in contrast to the given attitudes. Those criteria are tender price (Figure 4, Series 2 and 13), and WLCC (Figure 4, Series 1 and 5). It is interesting to see that even with a small number of experts involved, their attitudes differ significantly. In this case, the reasons can be found in their prior experience with construction projects. Consequently, in Series 2 and 13 the experts’ prior experience indicate that they are more oriented towards traditional budgeting and more likely to see WLCC and nontraditional budgeting approaches such as public − private partnership (e.g., Series 1 and 5 ). In summary, even with a small poll of stakeholders, the DSC-CONT provides the opportunity to incorporate opposing stakeholders’ demands at the same time increasing the transparency of the decision-making process. For all their matrices, the determined inconsistency was less than 0.1, which means that the weights were calculated correctly. In order to evaluate them as a single group, i.e., scenario, the overall matrix was created with aggregated values (Table A2). The aggregation of each pairwise was done by the median, and the final weights are presented in Table 3. The CR is 0.05. These weights were used for the following evaluation of tenderers and presents Scenario 2—client group. To conclude, this approach additionally gives transparency during aggregation as all stakeholders’ demands are included in the decision-making process. Appl. Sci. 2021,11, 1660 16 of 17 32. Jajac, N.; Knezi´c, S. Marovi´c, I. Decision support system to urban infrastructure maintenance management. Organ. Technol. Manag. Constr. 2009,1, 72–79. 33. Jajac, N.; Bili´c, I.; Mladineo, M. Application of multicriteria methods to planning of investment projects in the field of civil engineering. Croat. Oper. Res. Rev. 2012,3, 113–124. 34. Jajac, N.; Bili´c, I.; Ajduk, A. Decision Support Concept to Management of Construction Projects—Problem of Construction Site Selection. Croat. Oper. Res. Rev. 2013,4, 235–247. 35. Jajac, N.; Marovi´c, I.; Bauˇci´c, M. Decision support concept for managing the maintenance of city parking facilities. Electron. J. Fac. Civ. Eng. Osijek E Gfos 2014,9, 60–69. [CrossRef] 36. Marovi´c, I.; Car-Puši´c, D.; Hrvatin, Z. Establishing a model to evaluate public administration projects. Electron. J. Fac. Civ. Eng. Osijek E Gfos 2014,5, 56–66. [CrossRef] 37. Marovi´c, I.; Hanak, T. Selection of adequate site location during early stages of construction project management: A multi-criteria decision analysis approach. IOP Conf. Ser. Mater. Sci. Eng. 2017,251, 012044. [CrossRef] 38. Macharis, C.; Springael, J.; De Brucker, K.; Verbeke, A. PROMETHEE and AHP: The design of operational synergies in multicriteria analysis. Strengthening PROMETHEE with ideas of AHP. Eur. J. Oper. Res. 2004,153, 307–317. [CrossRef] 39. Macharis, C.; de Witte, A.; Ampe, J. The multi-actor, multi-criteria analysis methodology (MAMCA) for the evaluation of transport projects: Theory and practice. J. Adv. Transp. 2010,43, 183–202. [CrossRef] 40. Zavadskas, E.K.; Turskis, Z.; Tamošaitiene, J. Contractor selection of construction in competitive environment. J. Bus. Econ. Manag. 2008,9, 181–187. [CrossRef] 41. Rogulj, K.; Jajac, N.; Simic, F. Decision support concept for construction design project—Selecting the type of glass facade. Croat. Oper. Res. Rev. 2017,8, 333–350. [CrossRef] 42. Jajac, N.; Kili´c, J.; Rogulj, K. An integral approach to sustainable decision-making within maritime spatial planning—A DSC for the planning of anchorages on the island of Šolta, Croatia. Sustainability 2018,11, 104. [CrossRef] 43. Kili´c Pamukovi´c, J.; Rogulj, K.; Dumani´c, D.; Jajac, N. A sustainable approach for the maintenance of asphalt pavement construction. Sustainability 2021,13, 109. [CrossRef] 44. Jajac, N.; Marovi´c, I.; Mladineo, M. Planning support concept to implementation of sustainable parking development projects in ancient Mediterranean cities. Croat. Oper. Res. Rev. 2014,5, 345–359. [CrossRef] 45. Semaan, N.; Salem, M. A deterministic contractor selection decision support system for competitive bidding. Eng. Constr. Archit. Manag. 2017,24, 61–77. [CrossRef] 46. Losada Maestre, R.; Sanchez Medero, R.; Berlanga de Jesus, A.; Molina Lopez, J.M. The application of analytic hierarchy process to implement collaborative governance process: The allocation of the urban structural funds in the city of Madrid. J. Multi Criteria Decis. Anal. 2020. [CrossRef] 47. Lakhani, A.; Zeeman, H.; Wright, C.J.; Watling, D.P.; Smith, D.; Islam, R. Stakeholder priorities for inclusive accessible housing: A systematic review and multicriteria decision analysis. J. Multi Criteria Decis. Anal. 2020,27, 5–19. [CrossRef] 48. Hatush, Z.; Skitmore, M.R. Criteria for contractor selection. Constr. Manag. Econ. 1997,15, 19–38. [CrossRef] 49. Hatush, Z.; Skitmore, R.M. Evaluating contractor prequalification data: Selection criteria and project success factors. Constr. Manag. Econ. 1997,15, 129–147. [CrossRef] 50. Russell, J.S.; Skibniewski, M.J. Decision criteria in contractor prequalification. J. Manag. Eng. 1988,4, 148–164. [CrossRef] 51. Marzouk, M. A superiority and inferiority ranking model for contractor selection. Constr. Innov. 2008,8, 250–268. [CrossRef] 52. Hanak, T.; Nekardova, I. Selecting and evaluating suppliers in the Czech construction sector. Period. Polytech. Soc. Manag. Sci. 2020,28, 155–161. [CrossRef] 53. Hopfe, C.J.; Augenbroe, G.L.; Hensen, J.L. Multi-criteria decision making under uncertainty in building performance assessment. Build. Environ. 2013,69, 81–90. [CrossRef] 54. Lesniak, A.; Plebankiewicz, E.; Zima, K. Design and build procurement system-contractor selection. Arch. Civ. Eng. 2012 ,18, 1–14. [CrossRef] 55. Hazir, O.A. A review of analytical models, approaches and decision support tools in project monitoring and control. Int. J. Proj. Manag. 2015,33, 808–815. [CrossRef] 56. Kim, J.I.; Kim, J.; Fischer, M.; Orr, R. BIM-based decision-support method for master planning of sustainable large-scale developments. Autom. Constr. 2015,58, 95–108. [CrossRef] 57. Taticchi, P.; Garengo, P.; Nudurupati, S.S.; Tonelli, F.; Pasqualino, R. A review of decision-support tools and performance measurement and sustainable supply chain management. Int. J. Prod. Res. 2014,53, 6473–6494. [CrossRef] 58. Saaty, T.L. The Analytic Hierarchy Process; McGraw-Hill International: New York, NY, USA, 1980. 59. Brans, J.-P.; De Smet, Y. PROMETHEE Methods: Multiple Criteria Decision Analysis; Springer: New York, NY, USA, 2016. 60. Brans, J.P.; Mareschal, B.; Vincke, P.H. PROMETHEE—A New Family of Outranking Methods in Multicriteria Analysis; Brans, J.P., Ed.; Operational Research IFORS 84: Amsterdam, The Netherlands, 1984; pp. 477–490. 61. Brans, J.P.; Vincke, P.H. A preference ranking organization method, the PROMETHEE method for MCDM. Manag. Sci. 1985 ,31, 647–656. [CrossRef] 62. Behzadian, M.; Kazemzadeh, R.B.; Albadvi, A.; Aghdasi, M. PROMETHEE: A comprehensive literature review on methodologies and applications. Eur. J. Oper. Res. 2010,200, 198–215. [CrossRef] 63. Visual PROMETHEE. Available online: http://www.promethee-gaia.net/software.html (accessed on 15 September 2020). Appl. Sci. 2021,11, 1660 17 of 17 64. Doberstein, C. Designing collaborative governance decision-making in search of a “collaborative advantage”. Public Manag. Rev. 2015,18, 819–841. [CrossRef] 65. Johnston, E.W.; Hicks, D.; Nan, N.; Auer, J.C. Managing the inclusion process in collaborative governance. J. Public Adm. Res. Theory 2011,51, 699–721. [CrossRef]