Document [original]

Vol.:(0123456789)

1 3

Environment Systems and Decisions (2021) 41:633–650

https://doi.org/10.1007/s10669-021-09824-0

A prescriptive framework forrecommending decision attributes

ofinfrastructure disaster recovery problems

MiladZamanifar1· TimoHartmann1

Accepted: 12 July 2021 / Published online: 23 July 2021

Abstract

This paper proposes a framework to systematically evaluate and select attributes of decision models used in disaster risk

management. In doing so, we formalized the attribute selection process as a sequential screening-utility problem by for-

mulating a prescriptive decision model. The aim is to assist decision-makers in producing a ranked list of attributes and

selecting a set among them. We developed an evaluation process consisting of ten criteria in three sequential stages. We used

a combination of three decision rules for the evaluation process, alongside mathematically integrated compensatory and

non-compensatory techniques as the aggregation methods. We implemented the framework in the context of disaster resilient

transportation network to investigate its performance and outcomes. Results show that the framework acted as an inclusive

systematic decision aiding mechanism and promoted creative and collaborative decision-making. Preliminary investigations

suggest the successful application of the framework in evaluating and selecting a tenable set of attributes. Further analyses

are required to discuss the performance of the produced attributes. The properties of the resulting attributes and feedback of

the users suggest the quality of outcomes compared to the retrospective attributes that were selected in an unaided selection

process. Research and practice can use the framework to conduct a systematic problem-structuring phase of decision analysis

and select an equitable set of decision attributes.

Keywords Attributes· Decision-making· Disaster· Environment· Resilience· Problem structuring

1 Introduction

Decision analysis is broadly used for planning and solving

problems concerning contemporary challenges such as dis-

aster risk management, resilience planning and risk assess-

ment, which often integrate multiple objectives and decision

attributes. When responding to risks in environmental sys-

tems such as climate change and hazard-induced disasters,

several objectives and attributes are involved covering mul-

tifaceted characteristics of a modeled problem. Identifying

the underlying decision attributes is an essential, prelimi-

nary step in the decision-making modeling process (Keeney

2007; Belton and Stewart 2012). However, in both practice

and research, systematic approaches towards the selection

of attributes are either rare or inadequately applied which

hinders the identification of contextual, representative, and

complete attribute sets (Dale etal. 2015; Niemeijer and de

Groot 2006; Tiesmeier 2016). Attributes are often selected

without the contextual justification or formal approach

needed to shift speculative intuitions to rational judgments

(Tiesmeier 2016). This issue has been shown in many deci-

sion-making contexts, including Disaster Recovery Plan-

ning of Transportation Network (DRPTN) (Zamanifar and

Hartmann 2020). DRPTN is a decision-making context in

which optimized recovery operation plans are identified to

increase the resilience of transportation systems. Recovery

interventions respond to the disruptive impact of hazards

to restore an expected performance of a transportation net-

work with repair and reconstruction operations. The extent

to which the outcomes of DRPTN decision models are effec-

tive and reliable relies on the quality of attributes integrated

into the decision modeling process. Therefore, selecting ten-

able decision attributes is critical for complex and sensitive

problems such as disaster recovery and resilience planning

due to the socioeconomic and environmental consequences

of decisions (Sandri etal. 2020; Beling 2013). However, a

* Milad Zamanifar

Milad.Zamanif[email protected]

1 Department ofCivil andBuilding Systems, Technical

University ofBerlin, Gustav-Meyer-Allee 25, 13355Berlin,

Germany

634 Environment Systems and Decisions (2021) 41:633–650

1 3

gap in conceptual or systematic support for the selection pro-

cess of DRPTN attributes exists. Based on this premise, the

current paper is a response to the call of several studies for

an approach that allows a systematic and transparent selec-

tion process of contextual decision attributes (Zamanifar and

Hartmann 2020; Ha and Yang 2018; Tiesmeier 2016; Vaidya

and Mayer 2016; Dale etal. 2015). Therefore, we propose

a framework in the form of a decision aid mechanism that

supports and facilitates the selection of attribute sets. In

doing so, we formalize the process of attribute selection as

a screening-utility choice problem, since “the problem of

choosing between various formulations can itself be repre-

sented as a complex decision-making problem” (Mitroff and

Featheringham 1974). As part of the developed framework,

we formulated a prescriptive multi-criteria decision model.

The model incorporates ten criteria as the evaluation fac-

tors based on the literature of Multiple-Criteria Decision

Analysis (MCDA) and a combination of compensatory and

non-compensatory techniques as the aggregation and evalu-

ation method. We used this model in three sequential stages

of evaluation to assist decision-makers (DMs) in assessing

the performance of both attributes in isolation and attrib-

utes in sets. Once the framework was developed, we tested

it in a real-life problem of disaster recovery planning and

analyzed the results together with the application process

of the framework.

Although the framework (supposedly) possesses the

capacity to be generalized to various decision contexts,

mainly those that address complex decision problems with

multiple objectives or criteria, we chose to explore its per-

formance in the context of recovery planning of transporta-

tion networks after natural and socio-natural hazards. On

this ground, we collected decision attributes from the lit-

erature of DRPTN and experts’ opinions as the input of the

model. Then, we held a workshop with experienced emer-

gency managers as the DMs for conducting the evaluation of

inputted attributes following the flowchart of the framework.

Results suggest that the framework is capable of facilitating

a degree of supervision over the selection process and pro-

moting critical and creative thinking. DMs were able to sys-

tematically evaluate attributes and collaboratively produce a

ranked list and a set of attributes. The implementation of the

developed framework revealed satisfactory application from

the users’ point of view. We also employed several analyses,

including typological examination of the set, properties of

the selection process, and the feedback of experts, to inves-

tigate the quality of the attributes while further experimental

evidence is required to discuss the attributes’ performance.

The rest of the paper is structured as follows: in the next

section, we argue the necessity of this research and high-

light the existing knowledge gap. In Sect.3, we discuss the

existing attribute selection approaches and how the pro-

posed framework can situate itself among them, inviting the

readers to understand the innovative aspects and contribu-

tions of this study. Section4 illustrates the evaluation fac-

tors as the criteria of the developed decision model and the

structure of the designed decision environment. Section5

presents the methods that we employed and developed to

construct, implement, and analyze the framework. Addition-

ally, this section describes how users can adopt and apply

the framework to their decision problems. Section6 reports

the results of implementing the framework and offers an

analysis of its application. The Sect.7 provides a discus-

sion as to how results fulfill the objectives of this research

and support the contributions. This section also points out

some limitations involved with the decision modeling and

implementation of the framework. Finally, the paper is con-

cluded in Sect.8.

2 Knowledge gaps andthenecessity

The attribute set of a decision model represents essential

problem-related characteristics and the behavior of the

modeled system (Keeney and Gregory 2005). The degree

to which this representativeness is preserved within attrib-

utes indicates the directness of attributes. A primary pur-

pose for establishing a set of attributes is to disaggregate a

complex decision problem into more analytically tractable

components while maintaining the representativeness and

collectivity of the modeled problem as direct as possible.

Therefore, a well-thought-out attribute set can increase the

likelihood of representativeness, directness, and complete-

ness of a decision model. Despite the critical and fundamen-

tal role of attributes in decision analysis, the inadequacy of

problem structuring and efforts for attributes identification

in the decision modeling process is well documented (see,

e.g., von Winterfeldt and Fasolo 2009; Tiesmeier 2016;

Belton 1999). Maier and Stix (2013), Belton (1999), and

Keeney and Gregory (2005), among others, raised aware-

ness that far too little attention has been paid to the manner

in which a list of attributes and their contextual structures

are obtained. Specifically, much of the literature on deci-

sion analysis neglects the role of problem structuring and

thorough investigations on attributes as the primary task for

structuring a decision-making model (Franco and Montibel-

ler 2010). Similarly, in practice, Girod etal. (2003) observed

that during three workshops involving experts engaged in

the decision-making process of an engineering design, less

than 8% of the time was used to identify the criteria for the

targeted problem (Girod etal. 2003). Thus, it is hardly clear

to what extent the model’s recommended solution holds for

the real system (Corner etal. 2001). This issue is crucial for

Decision Support Systems (DSS) in disaster recovery and

environmental models that seek to present decision-making

methodologies associated with tremendous socioeconomic

635Environment Systems and Decisions (2021) 41:633–650

1 3

loss or gain (Goujon and Labreuche 2015; McDaniels etal.

2015).

A destructive or disruptive event that hinders access

within a transportation network or adversely influences the

safety and efficiency of a network’s mobility, to any extent

or period that exceeds the affected community’s socioeco-

nomic tolerance and coping capacity, can be perceived as a

disaster. Consequently, the concept of resilience is employed

to mitigate, postpone, or eliminate the likelihood of a haz-

ard transforming into a disaster. Resilience is “the ‘shear

zone’ between (dynamic) adaptation and (static) resist-

ance” (Alexander 2013). More specifically, disaster resil-

ience in the transportation network context refers to plans

and actions that improve the recovery potential of network

performance and adapt to the components’ failure during

and after a disaster. Non-resilient transportation infrastruc-

ture leads to significant economic loss, threatens society’s

health and well-being, and exacerbates the consequence of

hazard exposure and vulnerability (Kurth etal. 2020; Koks

etal. 2019). To increase resilience, developing reliable dis-

aster recovery planning is essential to meet the restorative,

rapidity, redundancy, and resourcefulness properties of resil-

ient infrastructures (Bruneau etal. 2003; Liu etal. 2020).

Post-disaster recovery planning of a transportation network

is a fundamental characteristic for a disaster resilient com-

munity, usually formulated as a decision model to rank or

optimize the order of links for recovery operations (Zhang

etal. 2017; Aydin etal. 2018; Rouhanizadeh and Kerman-

shachi 2019). To improve disaster recovery planning, one

must establish a set-up in which DMs can make informed

decisions concerning the attributes integrated into the dis-

aster recovery decision model. Therefore, it is of utmost

importance that disaster recovery models harness the ben-

efit of properties of a desirable attribute set while engaging

with such ever evolving and critical problems (Pearson etal.

2018; Quigley etal. 2019).

Studies have pointed out the role of MCDA in the deci-

sion modeling processes used for risk assessment, resilience,

and recovery planning (e.g., Cegan etal 2017; Rand etal.

2020; Manyaga etal. 2020). Keisler and Linkov (2014) high-

light the utility and favorability of MCDA, such as linear

additive scoring models in decision recommendations for

environment models. Rand etal. (2020) argue that disaster

recovery planning requires decision models and decision

support systems for informed decision-making since the

recovery of infrastructure is coupled to at-risk communities’

resilience. While the application of MCDA in prescriptive

decision models is evident, it has also been vastly employed

for problem structuring, identifying decision values, or rel-

evant metrics under the guide of frameworks (e.g., Fox-Lent

etal. 2015; Convertino etal. 2013; Linkov etal. 2018). For

example, Linkov etal. (2013) developed the “Resilience

Matrix” framework to identify metrics and to calculate the

performance scores for critical functions related to disas-

ter resilience of a defined system. Keeney and McDaniels

(1992) highlight that a key component in decision analysis is

the use of facilitation to identify values and frame the multi-

criteria problem (Keeney and McDaniels 1992). Moreover,

Keisler and Linkov (2014) also argue that the value of inter-

ventions in decision analysis is not only in the scoring and

rating of alternatives but the ability to facilitate discussion

and articulating viewpoints and decision values. They fur-

ther point out a need for tools and approaches to allow ana-

lysts to measure and discuss the desirability of hypothetical

alternatives. This paper seeks to offer a facilitation process

that helps DMs process values in a certain decision context

and to more reliably select decision attributes. The neces-

sity of this facilitation escalates when the decision context

embeds disaster resilience and recovery planning.

While the critical importance of attributes in decision

analysis is generally recognized, it is still insufficiently

addressed in the disaster risk management context. For

instance, DRPTN studies have introduced a wide range of

attributes to optimize or rank recovery operations of trans-

port lifelines. However, only 22.5% of studies have illus-

trated how the problem of DRPTN is structured or decision

attributes are selected (Zamanifar and Hartmann 2020).

Even with highly visible decision processes, insufficient

thought is typically given to the identification and choice

of attributes (Keeney and Gregory 2005), while in the con-

text of DRPTN, variables and factors are often inherently

uncertain. This challenge is not limited to the disaster man-

agement field, but has been shown in other contexts too.

For instance, Desmond (2007) outlines that there is a lack

of methodologies that assist in identifying attributes or

alternative sets in the strategic environmental assessment

field. Similarly, Ha and Yang (2018) share the same point

of view and recognize this gap in the infrastructure perfor-

mance assessment domain. They highlight that studies lack

a systematic approach capable of processing and incorpo-

rating adequate information, such as decision factors, into

the decision problem. Furthermore, Niemeijer and de Groot

(2006) argue that the selection process of attributes is mainly

subject to arbitrary decisions and called for a clear process

for selecting attributes, while Lin etal. (2009) believe that

attribute selection processes in most cases are insufficiently

systematic and transparent. Tiesmeier (2016) identifies the

same shortcoming in the real estate domain, reported incom-

plete lists, as well as high inconsistency across studies while

identifying attributes. He underscores that very few studies

fully justify the adoption of the chosen attribute systems.

Moreover, Ma etal. (2017) highlight that answering the

question of how to select the optimal decision attributes is

a compelling future research direction and a critical pro-

cess for many domains that use decision analysis. Fekete

(2019) takes a similar stance and emphasizes the demand

636 Environment Systems and Decisions (2021) 41:633–650

1 3

for guidance on attribute selection in disaster social vul-

nerability context. Overall, many decision-making models

fall short in benefiting from a reproducible and transpar-

ent model that assists analysts in selecting decision factors

(Tiesmeier 2016). That is, the task of attribute identification

itself remains a challenge that has not been adequately met

(Vaidya and Mayer 2016; Dale etal. 2015) which has led to

a call for a systematic guide as a reliable decision aid frame-

work to select attributes of decision problems.

3 Current approaches towardstheselection

ofattributes

Decision attributes are often chosen based on expert opin-

ions, literature, or a combination of the two. The expert-

based approach refers to drawing out information from

stakeholders, actors, and DMs to articulate important deci-

sion factors in a specific context (e.g., McIntosh and Becker

2020; Elboshy etal. 2019; Mirzaee etal. 2019). The expert-

based approach has the benefit of being based on the expe-

riences of experts who possess the knowledge contextually

related to the values of the decision context (assuming that

the desired properties of stakeholder analysis, interviews,

inclusion criteria of interviewees, and aggregation methods

are met). However, on the one hand, it falls short in includ-

ing existing literature and might fail in providing a com-

plete list of attributes. On the other hand, expert opinions

are assumed reliable sources for providing preferences and

values in a decision-making process as long as they possess

adequate decision-relevant knowledge, experience, or stake

(Bond etal. 2008). Nevertheless, empirical studies suggest

that individuals’ striking inability to understand their objec-

tives, values, and preferences, and their markedly deficiency

in communicating them is a plausible consideration (Barron

and Barret 1996; Kahneman etal. 1982). Thus, expert-driven

attributes could be a product of bias and error-prone efforts

in a limited amount of time and lesser in-depth thinking on

a specific problem (Girod etal. 2003; Tiesmeier 2016). In

order to shift towards a less interview-intensive and intui-

tive approach, some studies used the available literature to

identify and select the attributes of a decision problem (e.g.,

Herrera and Kopainsky 2020; Merad etal. 2013; Yu and

Solvang 2017). Although selecting attributes based on exist-

ing literature is an accepted approach, critique holds that

literature might disregard some aspects of a problem due to

its limitations in accessing comprehensive data, communi-

cation constraints, and simplifying assumptions. When one

adds the challenge of dynamic nature of problems, temporal

limitation of empirical studies, and the contextual inconsist-

ency of literature-recommended attributes to the problem

at hand, therefore, sole reliance on existing literature might

not sufficiently ensure an exhaustive and error minimized

approach for selection of attributes. To shape a more com-

plete, up-to-date, and practical set of attributes, the third

approach is a combination of expert opinion and previous

literature within the field (e.g., Walpole etal. 2020; Caru-

zzo etal. 2020; Kassem etal. 2016). While this approach

maximizes the exhaustiveness of the inclusion of attributes,

it yields a broad list of attributes from which some must

be selected intuitively. Nevertheless, objectively supervise

this intuition to select a viable attribute set remains a chal-

lenge. Thus, a prescriptive model as a decision intervention

is needed to formulate and solve the choice problem of the

selection among a finite number of alternatives.

To overcome this challenge, the model-based approach

has been introduced to systematically select attributes that

cover the concerns and values of the problem under consid-

eration. Accordingly, a few studies provide model-driven

attributes by formulating the selection process of attributes

as a choice problem (Cinelli etal. 2020; Höfer etal. 2020;

Otto etal. 2018; Rossberg etal. 2017; Dale etal. 2015; Con-

vertino etal. 2013). Regardless of the source of the alterna-

tive pool, these studies evaluate candidate attributes based

on properties of the desired attribute and apply a systematic

process to select the attribute set. Properties of the desired

attribute are factors that evaluate the merits of an attrib-

ute, such as unambiguous, operational, and direct (see, e.g.,

Keeney 1992). Our work extends this strand of approaches

with some innovation in the formulation approach, capabil-

ity, and generalizability of the application. First, as a general

guide for constructing a framework that seeks to prescribe

a decision, we followed the recommended structure of the

prescriptive decision analysis. Prescriptive decision analysis

is an intervention process to model a rational choice with

the recommended steps of problem structuring, preference

elicitation, evaluation/aggregation, and solution handling

(Clemen 1996; Keeney 1982). Second, we introduced three

stages of evaluation that led us to three decision regions

and subsequently three decision rules. The multi-stage

evaluation process allows the incorporation of ten evalua-

tion factors without the urge to introduce hierarchy into the

criteria system or increasing the complexity of the modeled

problem. This architect of the evaluation environment also

leads to a thorough yet cognitively manageable evaluation

process. Third, the built-in screening evaluation stage grants

the inclusion of candidate attributes from various sources

such as literature and expert opinions. Fourth, the developed

decision model not only evaluates attributes but also evalu-

ates sets of attributes in the third decision region. Fifth, we

proposed a comprehensive and illustrated framework to sup-

port the implementation of the decision model that can be

used for future research and practice as well as by those who

are not necessarily an expert in decision analysis. Therefore,

the added value of this research is:

637Environment Systems and Decisions (2021) 41:633–650

1 3

1. Formulating the process of selecting attributes of the

DRPTN problem as a prescriptive decision model to

aid the attribute selection process and DMs’ knowledge

acquisition;

2. Exploring the performance of integrating compensatory

and non-compensatory decision rules and introducing a

new application for this integration;

3. Presenting a tractable and user-friendly framework to

assist systematic multi-stage evaluation and selection of

tenable decision attributes of disaster recovery planning

problems.

The framework aims to systematically process the DMs’

inputs and develop contextual decision values and assist

them in selecting an effective, operational, and complete set

of decision attributes. The contribution of this paper is the

proposed framework and the embedded decision model. The

practical implication of results is to help decision analysts to

make an informed choice and tenable decisions within the

construction of a decision-making model. Therefore, schol-

ars who develop multi-objective or multi-attribute decision

models can use this framework in the problem-structuring

phase of their modeling process since “one of the most

important determinations of a problem’s solution is how

that problem has been represented or formulated in the first

place” (Mitroff and Featheringham 1974).

4 Evaluation factors andthedecision

environment

As decision criteria, we adopted existing evaluation factors

of attributes, or “properties of a good attribute” (Keeney

1992), by conducting a review in MCDA problem structur-

ing literature. Based on the recommendation of Franco and

Montibeller (2010), these evaluation factors are adopted to

address whether attributes are operational and relevant to

the decision context, the way they measure the performance

of alternatives, and how they are aligned to the objectives.

In the literature, except for Roy (1996), the properties of

an attribute and a set of attributes are not distinguishably

discussed. Therefore, we took into account the “properties

of a good set of attributes” by identifying the factors that

address the characteristics of an attribute set. We also inter-

preted “measurability” to “certainty of measure” to tailor

a set of evaluation factors for our particular problem, since

we had good reasons to believe that after a disaster, the cer-

tainty of the measuring associated with an attribute reduces

the uncertainty of the value of an objective. Table1 shows

ten adopted evaluation factors, where seven count for mem-

bers of an attribute set while three factors evaluate sets of

attributes.

Evaluation factors are divided into three decision regions

based on whether they evaluate the performance of indi-

vidual attributes or a set of attributes, and whether they

address the property of necessary or sufficient conditions of

the desired attribute. That led us to three decision regions for

which we assigned each a known decision rule: compensa-

tory, non-compensatory, and optimal. The reason for design-

ing a multi-stage evaluation strategy is to take into account

both the properties of evaluation factors and properties of

alternatives of this specific problem. Therefore, while in the

first two regions, attributes are evaluated individually, in the

third region, they are evaluated as a set. For the properties

of evaluation factors, the first region includes factors that

are necessary for attributes to meet. Hence, compromise

among factors is not desired which justifies the use of non-

compensatory decision rules. The second decision region

includes factors that can compensate for each other, which

makes using a compensatory decision rule reasonable. In the

third decision region, the optimal performance of three eval-

uation factors is the target of the evaluation. Additionally,

Table 1 The evaluation factors for members of an attribute set and sets of attributes (Zamanifar and Hartmann 2021)

Evaluation factors For members For set Suggested by

Coherency with objectives ◆ Belton (1999), Majumder (2015), Gregory and Failing (2002)

Operational ◆ Baker etal. (2001), Belton (1999), Dodgson etal. (2009), Belton and Stewart (2012),

Keeney (2007)

Discriminative ◆ Baker etal. (2001), Gregory and Failing (2002)

Understandable ◆ Keeney (2007), Belton and Stewart (2012), Roy (1996)

Direct ◆ Belton and Stewart (2012), Majumder (2015)

Certainty in measure ◆ Gregory and Failing (2002), Majumder (2015)

Representativeness ◆ Roy (1996), Baker etal. (2001), Dodgson etal. (2009), Belton and Stewart (2012), Keeney

(2007)

Completeness ◆ Belton and Stewart (2012), Dodgson etal. (2009), Roy (1996), Baker etal. (2001)

Non-redundant ◆ Belton and Stewart (2012), Dodgson etal. (2009), Baker etal. (2001), Roy (1996), Gregory

and Failing (2002)

Concise ◆ Belton (1999), Majumder (2015), Gregory and Failing (2002), Baker etal. (2001)

638 Environment Systems and Decisions (2021) 41:633–650

1 3

designing separate decision regions allowed us to reduce

the complexity of the evaluation process and avoid possi-

ble errors and biases associated with a hierarchized criteria

structure such as systematic spitting bias (Hämäläinen and

Alaja 2008) influence of the type of asymmetry in a hierar-

chy (Marttunen etal. 2017), and larger variance in weights

(Jacobi and Hobbs 2007). Breaking the evaluation into three

discrete stages resulted in a more cognitively manageable

process when assessing alternatives’ performances with the

maximum number of evaluation factors in each region not

exceeding four (Cowan 2010). Figure1 demonstrates the

decision regions and corresponding decision rules of the

model.

The evaluation begins with the screening region, as the

filtering phase of the decision process with a non-compen-

satory decision rule. Thus, alternatives, which fail to satisfy

the factors of this region, will be either removed or con-

sidered for redefinition. We postulate a genuine interest in

the first three evaluation factors which we call “concrete

factors”. Therefore, the screening region excludes attributes

that; (1) are not relevant to the decision context (coherency

with objective; (2) are not commensurable in a consistent

manner and with a reasonable amount of effort (operational);

and (3) are not clearly distinguishing among all alternatives

to perform a comparison (discriminative). By establishing

a screening region, we ensure that concrete factors are not

ignored in compensation for other evaluation factors. Attrib-

utes that meet the three evaluation factors of the screen-

ing region move into the choice region. The choice region

operates with a compensatory decision rule; therefore, a

compromise among evaluation factors in this decision region

is desired. The choice region evaluates the attributes based

on the four criteria as “choice factor”, including (1) “under-

standable” when it has a clear and unambiguous definition;

(2) “certainty” when it yields a certain measured value for

the objectives; (3) “directness” when it directly measures

the primary objective of the decision problem; and (4) “rep-

resentative” when it represents the essential characteristics

of the system. The third group of factors evaluates “sets of

attributes” based on the optimal decision rule. The optimal

decision rule can be regarded as the optimized outcome for

a set of attributes that capture the maximum key aspects of

all objectives (completeness) with an optimized size of the

attribute set (concise). In addition, the set should not contain

a double-counting attribute (non-redundancy), which can be

expressed as the constraint of the optimal region (Zamanifar

and Hartmann 2021).

5 Methodology andthedeveloped

framework

This section presents the methodology for developing the

attribute selection framework, the framework itself, and

methods used for implementing and evaluating the frame-

work. Table2 demonstrates the adopted methods in the

frame of the prescriptive decision analysis. While the prob-

lem-structuring phase was presented in Sect.4, this section

discusses methods and their applications for the remaining

tasks of our research design.

5.1 Models oftransition andaggregation

Both compensatory and non-compensatory approaches

have their own application and advantages; hence, a con-

trast between them is not meaningful. Nonetheless, there are

decision contexts in which employing either of compensa-

tory or non-compensatory decision rules alone cannot meet

the characteristic of the modeled problem. Non-compensa-

tory aspect-based methods rely on a sequential elimination

approach based on sorted criteria that usually leads to a

straightforward selection of the most preferred alternative.

Non-compensatory methods are widely used in normative

decision theory (see, e.g., Gigerenzer and Goldstein 1996;

Gigerenzer and Gaissmaier 2011) since they are consistent

with the concept of bounded rationality. However, aspect-

based non-compensatory methods overlook the existence of

some criteria and a part of the available information in the

decision context is often regarded as irrelevant (Rothrock

and Yin 2008). This is because most of the information col-

lected on alternatives will not play a role in the evaluation

process (Munda 2005). Therefore, they are limited in appli-

cation to conditions when non-compensatory is the desired

Fig. 1 The structure of evaluation factors, notions, decision regions,

and decision rules for members and sets

639Environment Systems and Decisions (2021) 41:633–650

1 3

rule of the entire decision context. Meanwhile, compensa-

tory methods cannot be applicable when some criteria are

infinitely more important than others. With this preferential

model, compensatory methods could lead to an undesirable

outcome as the choice might fail to meet the minimum level

of desirability in one or more criteria. Consequently, com-

pensatory methods might not be efficient when a part of

the decision context does not accept trade-offs among some

of the criteria. Both compensatory and non-compensatory

methods, when they are used individually, assume that the

same decision rule holds for all criteria. Therefore, for some

problems, they are unrepresentative of the decision strategy

they seek to represent. For the benefit of our framework, we

combined the application of compensatory and non-com-

pensatory methods and designed two decision regions for

appraising isolated attributes within a single decision sys-

tem. Integrating compensatory and non-compensatory deci-

sion rules maximizes the amount of incorporated informa-

tion and allows its specificity within the modeling procedure.

The first decision region performs in a perfectly non-com-

pensatory fashion, while the second decision environment

allows for a compensatory interaction among the factors.

The first decision region is absolutely preferred to the second

decision region which we mathematically formulated as part

of the transition between two regions.

For the non-compensatory region, we adopt the axioms

presented with the lexicographical choice concept (Tversky

1972) and formulated it to an aspect-based screening condi-

tion as it is formalized in Eq.(1). For the choice region, we

directly used the well-known Multi-Attribute Value Theory

(MAVT) (Keeney and Raiffa 1993) and contextualized it to

the local variables of our decision context shown in Eq.(2).

In doing so, the following mathematical expression repre-

sents the integration of compensatory and non-compensa-

tory decision rules. The condition expressed in Eq.(1) indi-

cates that alternative A is preferred to B when two

alternatives have equal performance on a set of factors in a

binary format while there exists at least one factor that alter-

native B is not satisfying. Accordingly, for the first three

evaluation factors (coherency with objective, operational,

and discriminative:

𝛾s

1,𝛾s

2,𝛾s

) of the screening region: Let

Γs

{

𝛾

,…,𝛾

i},

∀i∈{1, …,h}

h≥2

be the finite set of

already known Concrete factors of our screening region (

which is denoted by s. Now, suppose there exist a nonempty

finite set called

𝜗

that

𝜗

𝛾

represents the importance degree

among factor

𝛾s

where

represents the preference of DMs

on a factor and

𝛾s



Moreover, let

{

𝛿s

1,…,𝛿s

}

,∀j∈{1, …,m},m>

and a positive

integer define the set of competing alternatives and

Ψij

denotes the binary value of

ith

factor to

jth

alternative where

Ψij ∈{1, 0}

. Now consider

𝛿s

and

𝛿s

j+n∈Δ,∀n>0

then

𝛿s

lexicographically preferred to

𝛿s

j−

if, and only if:

Meanwhile, for the choice region with compensatory

decision rule, we aggregate four evaluation factors of under-

standable, representative, direct, and certain (

𝛾c

(m)4

𝛾c

(m)5

𝛾c

(m)6

𝛾c

(m)7

) with formulating MAVT for our problem. The

transition from the screening region to the choice region can

be shown: again, let

𝛾c

,…,𝛾c

,∀

{1, …,k},k≥2 an integer

for the new discrete finite set of

Choice factors of the choice utility region (

which is

denoted by c such that . Moreover,

𝛿s

1,…,𝛿s

}

,∀j∈{1, …,m

}

m>2

as the

choice set that already satisfied the set of

Γs

in the screening

region such that

⊆Δ2

. Additionally, assume preference

(1)

⎧

⎪

⎨

⎪

⎩

Ψ𝛿s

ij equals 1 for all Γ=

�

𝛾s

1,…,𝛾s

�

and,

∃𝛾s

i∈Γthat Ψ𝛿s

j−n

ij equals 0.

Table 2 An overview of the adopted methods and task for developing the proposed framework

Purpose Task Method

Problem structuring Adopting evaluation factors Content analysis

Defining decision regions and assigning decision rules Based on the property of factors and alternatives

Preference elicitation Identifying the relative importance of compensatory evaluation factors Cardinal ranked-based weighting approach

Problem solving Combination and extension of compensatory and non-compensatory

MADM techniques

Multi-attribute value theory

Elimination by aspect

Implementation and

solution handling

Collecting data Systematic review

Questionnaire

Implementing methodology Workshop with decision-makers

Analyzing data Proposed methodology

Analyzing results Performance observation and feedback survey

Retrospective comparison

Typology of selected attributes

640 Environment Systems and Decisions (2021) 41:633–650

1 3

weight among factors follow

0>V

𝛾c

1>V

𝛾c

2>…>V

𝛾

then, without loss of generality, the utility of each alternative

is:

where

(

𝛿c

)

is the scaled utility function of alternatives,

denote the performance of attribute

i∈

on alternative

as a single attribute value function,

(

𝛿c

)

is the perfor-

mance of alternative

for attribute

and

𝜔i

is scaling factor

projecting the importance weight of attribute

∑

i=1

𝜔i=

For further investigation on the axiomatic background of the

Eq.(2), one can see the original work of Keeney and Raiffa

(1993).

5.2 Preference elicitation

Imprecise weight elicitation is based on ordinal and car-

dinal values that DMs approximate regarding the relative

importance of criteria. Ordinal information refers to the

rank of criteria based on their importance, while cardinal

information represents the relative range of intervals among

assigned ranks. Ordinal methods such as Rank Ordered

Centroid (ROC), Rank Sum (RS), and Rank Reciprocal

(RR) (for a review, see, e.g., Roszkowska 2013) convert the

rank of criteria to (surrogate) numerical weights. Unlike

the approaches that are based on semantic and numerical

scales, weight approximation methods assume that compel-

ling DM to express their exact perceived values is cogni-

tively demanding and refrain from obtaining viable prefer-

ences (Barron and Barrett 1996; Alfares and Duffuaa 2008).

For example, Barfod and Leleur (2014) argue that DMs are

more comfortable and confident with ranking the attrib-

utes rather than communicating their preferences by verbal

scales. Roszkowska (2013) highlights that for a group of

DM is easier to agree on rank-based sorting of items than to

assigning precise numerical values. However, the critique

also exists that preference elicitation following ordinal meth-

ods encounters information loss since these methods do not

inquire or make use of information regarding the magni-

tude or intensity of preference among sorted items (Daniel-

son and Ekenberg 2016). Rank-based methods, generally,

rely on the centroid of ordered factors and do not require

any further input from DMs on the preference difference

between ordered pairs of factors. Some studies also report

the victimized weight for lower-placed criteria due to large

discrepancy between the highest and lower sorted criteria,

suggesting the need for methods that incorporate cardinal

information into the weight approximation process and

(2)

generate “smoother” weight (Roberts and Goodwin 2002;

Huang etal. 2011; Belton and Stewart 2012).

Building upon the existing weight approximation

approaches (Kárný 2013; Salo and Hamalainen 2001; Bar-

ron and Barret 1996), we designed a rank-based tool to allow

experts to communicate their cardinal and ordinal prefer-

ence related to the relative importance of four compensatory

evaluation factors. The reason that we customized a weight

approximation approach instead of pure ordinal methods was

first to avoid extreme weights that significantly marginalize

the weight of lower factors (Belton and Stewart 2012), sec-

ond to prevent equalizing impact on upper factors (Kunsch

and Ishizaka 2019), and third to utilize the available cardi-

nal information that ordinal methods often do not take into

account (Danielson and Ekenberg 2016).

We used the input of six MCDA experts to suggest trade-

offs among the evaluation factors that represent criteria of a

good attribute. In order to recognize participants as experts,

we considered the fulfillment of three criteria consisting of

research engagement in the MCDA field, current involve-

ment in the field through MCDA-related publications in the

last five years, and post-graduates holding academic research

assistant positions. Since the properties of attributes in the

decision-making context have been widely discussed in deci-

sion analysis and MCDA discipline, we chose to acquire the

input of academic experts within this field. Experts provided

ordinal information for evaluation factors by ranking them

from the most important to the least important on a vertically

2D visualized slider. They then adjust the distance among

the sorted evaluation factors to express their cardinal prefer-

ences. Therefore, while experts can communicate the ordinal

preference by ordering the factors, they can also regulate the

intervals among each pair of sorted factors to indicate the

pairwise preference intensity. No numerical scale has been

presented for experts on the slider and visuospatial scale

was the interface for regulating the distances. The math-

ematical model that interprets the defined rank and intervals

into numerical weights can be formally articulated by the

following:

The input value for an evaluation factor

di∧d1=0

where

is Euclidean distance of factors assigned by experts

to the Cartesian origin

of the vertically visualized slider.

Each evaluation factor is presented to experts as an item on

this slider, while the first factors always remain at the origin

. The assigned location of each item represents experts’

ordinal preferences. Since the most important factor is set

as an item in the highest point of the slider, it is then logi-

cal to assume for

∀i∈n

, 9 ≥

n≥2

where is the revisited distance of factor

to a new origin

that increases as do the preference of factors. This assump-

tion allows for converting the growth of the distance from

the origin equal to the growth of preference. Therefore,

for

= 4 evaluation factors of the choice region, we have

641Environment Systems and Decisions (2021) 41:633–650

1 3

≤

which represents

d1≽d2≽d3≽d4

. Now

following the equations below, the relative importance of

factors can be calculated:

where

is the mean value of all distances from origin

is the virtual origin point of the weight vector as a base for

calculating the revised distance,

is the initial distance for

each interval that experts assign,

is the number of evalua-

tion factor with equal ordinal preference (if there is any),

is the number of factors, is the revised distance of each

interval value to the

, and

is the calculated weight of

factor when

∑

wi=

. We incorporated the exponential

effect in Eq.(4) to prevent equalized weights while allowing

the weight vector to count for both ordinal and cardinal pref-

erence input of DMs. Haung etal. (2011) argue that the

non-linear distances between single dimension scores and

ratios should produce smoother trade-offs. In addition, we

know of no study that suggests preference of individual

related to a sorted list of objects is distributed uniformly (see

Roberts and Goodwin 2002). With this approach, we could

shift the weight vector toward a flatter shape and, in the

meantime, prevent significant discrepancy between the

weight of the first and last factors.

By using the experts’ input regarding the order and inter-

val among four compensatory evaluation factors, the weight

vector was delivered in the format of Eq.(6). Assuming that

the preference for all factors is judgmentally independent,

the preferential model of the four evaluation factors of the

choice region, including understandable, representative,

direct, and certain, (

𝛾c

(m)4

𝛾c

(m)5

𝛾c

(m)6

𝛾c

(m)7

) can be shown as

the weight vector

w=20.68𝛾c

(m)4

36.56𝛾c

(m)5

27.44𝛾c

(m)6+15.32𝛾c

(m)7

5.3 The proposed framework

We designed the framework based on the evaluation fac-

tors, their preferential model, and the decision rules as

well as the adopted transition and aggregation logic. It

consists of core nine steps which acts as a decision aiding

(3)

�∑

�

di−d

�

(n−1)−k

max

(4)

(5)

(6)

=w1𝛾c

(m)4+w2𝛾c

(m)5+w3𝛾c

(m)6+w4𝛾c

(m)7,

∑

wi=1, d∈ℝ,wi≠

toolkit for selecting attributes of DRPTN problems. Fig-

ure2 provides a detailed flowchart and the process of

applying the proposed framework.

The implementation of the framework begins with iden-

tifying the primary objectives of the original problem.

The original problem refers to the problem for which the

framework intends to select attributes. Since the frame-

work embeds a choice model, the task in the second step

is to develop a set of alternatives. The alternative pool

is a set of candidate attributes as the input of the model

for the evaluation process. Regardless of the approach

used in constructing such a set, it is essential to establish

an expanded and complete set of attributes from diverse

sources such as literature, experts, stakeholders, actors,

and decision-makers to obtain a “reasonably complete”

list of attributes (Keeney 2007).

The third step is adopting the ten evaluation factors for

the three evaluation stages as well as the relative impor-

tance among the four evaluation factors of the choice

region presented in 5.1. In step four, Eq.(1) indicates

that alternatives for which the necessary condition of all

three screening factors is not held must be ruled out or

redefined. Within the fifth step, the remainders of alter-

natives enter the choice region in which users can assign

a numerical value for attributes’ performance in relation

to satisfying compensatory factors. Once each alternative

has received the scores of the previous step, in step six,

an additive aggregation method (Eq.2) is recommended

to rank the alternatives under the compensatory decision

rule. During the evaluation, users can redefine or suggest

new attributes to satisfy the screening evaluation factors or

improve the performance on the choice evaluation factors,

under the condition that the evaluation process iterates

from the second step.

Concept mapping of ranked attributes is the aim of the

seventh step, by assigning each ranked attribute from the

previous step to its representing objectives. In accord-

ance with Keeney’s recommendation (Keeney 1992), we

suggest the generic value tree as the concept mapping

approach to aid in identifying the completeness of the

attribute set. The last step inherits the ranked list of attrib-

utes from screening and choice regions that are classified

within the organization of the conceptual value tree of

the defined objectives. In this step, the task is to evaluate

sets of attributes based on completeness, size, and non-

redundancy. Using the value tree, the completeness of the

set can be monitored in terms of covering all objectives.

In the optimal region, the minimum size of the attribute

set that satisfies the completeness of the set and does not

contain a double-counting attribute is the selected attribute

set. If the resulting ranked attributes of step seven could

not cover all the objectives of the value tree, expanding the

alternative pool is necessary.

642 Environment Systems and Decisions (2021) 41:633–650

1 3

5.4 Methods ofimplementation

For obtaining a degree of validation and investigating the

performance of the framework, we conducted an experiment

to test the implementation of the framework in the context

of mid-term disaster recovery planning of the Tehran trans-

portation network. We sought to select an attribute set for

optimizing the performance of the network recovery process

after a major earthquake. In doing so, we collected data from

literature of DRPTN and disaster recovery experts to obtain

a list of candidate attributes as the input of the framework.

A systematic literature review of 46 papers allowed us to

extract 34 attributes from DRPTN publications. Addition-

ally, we collected ten additional unique attributes from 23

decision-makers of crisis management organizations using

a paper-administrated survey. Thereafter, we organized a

workshop and followed the steps of the developed frame-

work that is presented in Sect.5.3. We acquired the input of

a focus group that includes four senior members of a group

of city planners and emergency managers who had previ-

ously developed the disaster recovery planning of transpor-

tation network. Before the evaluation session, we explained

the structure and function of the framework and discussed

the problem by presenting a brief disaster scenario as well

Fig. 2 The consecutive algorithm of the framework for the selection process of an attribute set

643Environment Systems and Decisions (2021) 41:633–650

1 3

as detailed descriptions and definitions of the evaluation fac-

tors. For the sake of consistency with a real-life instance of

the DRPTN, we accepted the previously defined objectives

for the same problem that the focus group had formulated.

The objectives were maximizing accessibility and mobility

as the properties of the network, and maximizing recovery

effectivity and recovery efficiency as the properties of the

recovery process.

At first, DMs evaluated the alternatives based on the three

non-compensatory evaluation factors of the screening region

and assigned a binary value of 1 or 0. Following the non-

compensatory decision rule shown in Eq.(1), alternatives

were screened and transmitted to the choice region. In the

next step, DMs used a direct rating on a local scale of 0 to

10. The group was also free to redefine the attributes that

have not satisfied the evaluation factors of the screening

region. DMs deliberated on the score of each attribute and

communicated it verbally once a consensus was reached.

This process varied for different attributes. Sometimes disa-

greements required more extended discussion to be resolved,

particularly in the presence of an opposite voice, and in other

cases, the value assignment process was relatively fast. In the

screening region, three attributes have been excluded using

a majority vote where no consensus could be reached. We

observed that conflicts mainly occurred in the screening and

optimal region, while the evaluation process in the choice

region encountered some relatively minor disagreements

resulting in less controversial discussion. In both screening

and choice regions, critical issues were resolved by allowing

experts to redefine or improve attributes that failed to meet

non-compensatory evaluation factors. As subject-matter

experts, participants had prior experience analyzing possi-

ble attributes for disaster recovery planning of transportation

network and emergency network planning. Once the scores

had been documented, we used the weight vector presented

in Sect.5.2 and Eq.(2) to aggregate DMs inputs and solve

a utility choice problem that resulted in a ranked list of

attributes. Consequently, we used a conceptual value tree of

objectives to assist DMs in assigning each ranked attribute to

the representative objectives. In the last step, DMs selected

a set among ranked attributes which satisfied the optimal

region’s three factors and constituted the recommended set.

The evaluation session took 3:07min, while the total time

of the subject-relevant discussion was 2:42min.

Finally, to analyze the performance of the framework

and the resulting attributes of the case study, we began by

investigating the typology of the selected attributes accord-

ing to Keeney’s recommendations (Keeney 1992). Keeney

characterized attributes as three different types: Natural,

Proxy, and Constructed. Natural attributes directly measure

the degree to which an objective is met and can be counted

or physically measured. Proxy attributes share features of

natural attributes but are less informative and do not directly

indicate the achievement of an objective. Constructed attrib-

utes are developed when there are no natural attributes for

the objective of the concern. The certainty and accuracy of

these attributes might be less than Natural attributes with

respect to measuring the objective, but their presence is

essential in the absence of natural attributes (Keeney 2007).

Secondly, we determined the source of inclusion of attrib-

utes in different stages of the evaluation process and in the

ranked list of attributes to understand how the population of

the attributes in each stage is distributed. Thirdly, we com-

pared the resulting attributes of the framework to the list of

working attributes previously selected by the participating

experts for the same planning in the case area. Lastly, we

obtained the focus group’s feedback to two questions: (1) as

users, to what degree are you satisfied with the application of

the framework, and (2) to what degree do you agree with the

improved quality of the framework’s outcome compared to

the previously selected attributes. We used an anonymously

printed-format survey based on a 5-point Likert scale (Joshi

etal. 2015) two days after the workshop. Additionally, we

performed an unstructured group discussion with open-

ended questions that lasted approximately 30min immedi-

ately after the workshop, allowing DMs to openly commu-

nicate the experience of using the framework.

6 Result andsynthase

6.1 Performance oftheframework

intheimplementation process

We followed the steps of the framework and applied it to

the Tehran DRPTN problem. Having 57 candidate attrib-

utes as the input of the framework, the screening region

filtered 42% of alternatives that failed to meet at least one

of the non-compensatory evaluation factors. Therefore, the

compensatory evaluation in the choice region began with

33 attributes that formed 58% of the initial alternative pool.

Table3 highlights the number of alternatives in each stage

of the evaluation process.

The outcome of the choice region yielded a ranked list

of attributes that was preliminary to organizing a generic

value tree based on the primary objectives. In the opti-

mal region, the distribution of eight first ranked attributes

to the objectives was required the way that all objectives

receive at least one representative attribute. Similarly,

DMs assigned the first 11 attributes to the value tree

to provide each objective with no less than two attrib-

utes. To be able to provide three available choices for all

four objectives, assigning 16 first ranked attributes was

required. That supports the transmission of 16 attributes

to the optimal region (approx. 26% of the alternative

644 Environment Systems and Decisions (2021) 41:633–650

1 3

pool). Table4 shows the first 16 attributes assign to the

four objectives until each objective receives at least three

attributes.

Based on the arrangement of the rank attributes on the

value tree, the DMs selected six attributes as the recom-

mended set of decision attributes for the DRPTN problem.

The set contains the first five ranked attributes as (1)access

level to Service Providing (SP) nodes; (2) product of link

travel delay and traffic flow; (3) travel time improvement

per recovery duration; (4) travel time improvement per

resources, and (5) centrality measures plus Link capacity

that is ranked as the eighth attribute. The recommended set

covers the main concerns of the decision problem based on

the primary objectives and is supposed to be complete and

non-redundant with an optimized size of the set. Table5

provides a brief description of the selected attributes.

Table6 demonstrates the properties of each selected

attribute of the recommended set and their representing

objectives. The calculated utility based on the compensa-

tory factors as well as the rank of each attribute irrespec-

tively indicates the score and ordinal importance of attrib-

utes. The selected set consists of four natural attributes,

one constructed, and one proxy attribute. The range of the

assigned utility of attributes was between 14.22 and 29.67

while the best utility could ideally be 30.3, and 3.03 for the

worst utility.

Table 3 Number and share of attributes in decision regions of the evaluation process

Number Share (%) Examples

Proceed into the screening region 57 100 Full list available at: https:// doi. org/ 10. 14279/ depos itonce- 10019

Filtered in the screening region 24 42.1 Lifeline interaction, traveler convince, link geometry, damage complexity

Proceed into the choice region 33 58 Depot and need points, traffic redundancy, centrality measures, redundancy

Proceed into the optimal region 16 26 Link topology, capacity, social vulnerability* link delay, recovery efficiency

Table 4 The generic distribution of the first 16 attributes to the respective primary objectives

Objective Rank 1 Rank 2 Ranke 3 Rank 4 Ranke 5

Maximize recovery

effectivity

Travel delay of link * link

flow

Impact on Total Network

Travel Time

Social vulnerability* link

flow

Social vulner-

ability*

zone travel

demand

Maximize recovery

efficiency

Travel time improvement

per resources

Travel time improve-

ment/recovery duration

Recovery efficiency

Maximize accessibility Access level to the ser-

vice providing nodes

Centrality measures East–west and north–

south connectivity

Connectivity

to other traf-

fic zones

Network topology

Maximize mobility Link capacity Annual average weekly

traffic

Annual average daily

traffic

Traffic density

Table 5 A brief description of attributes of the selected set

Selected attributes Brief description

Access to service providing nodes The capability of links in providing access to the location of critical facilities and service

providing nodes on the network

Link capacity The ability of each link to carry the traffic as a measure of mobility performance of a link

Travel time improvement (TTI) per resources The amount of machinery or monetary resources that have to be assigned to achieve a certain

improvement in travel time on the network

Travel time improvement per recovery duration The amount of required unit of time to achieve a defined travel time improvement in the network

Travel delay * link flow The travel delay time that closure of a link imposes to a specific O-D trip combined to the

amount of traffic volume that the link sustains

Centrality measure Topological importance of a link in a network graph regardless of the traffic flow

645Environment Systems and Decisions (2021) 41:633–650

1 3

6.2 Synthesis oftheframework implementation

andfeedback ofparticipating DMs

Table7 shows that the literature of DRPTN contributes to

59.64% of the initial alternative pool while the survey with

experts formed 17.54% of attributes. Additionally, 13 attrib-

utes (22.8%) were added to the alternative pool during the

evaluation process as by-products of the framework. Similar

to the initial population of the alternative pool, in the first

10 and 20 ranked attributes, literature-based attributes have

the largest share. In the selected set, half of the attributes

belong to the literature review’s output, while the rest were

introduced during the workshop. Results show that while

collected attributes from 23 disaster managers DMs occupy

22.8% of the population of the initial alternative pool, they

contribute the least in all ranked attribute classes and have

no representative within the selected set.

Table8 shows the retrospective attributes which had been

selected by participating DMs, for the same problem and

the same geographical context as a set of DRPTN work-

ing attributes. Model-driven attributes refer to the attributes

selected by DMs following the proposed framework of the

current paper. Two sets share two identical attributes based

on equal serving objectives. The size of the model-driven

attribute set is six members and the working attribute set

contains nine attributes.

With respect to the users of the framework, the results of

the survey communicate a “moderately to strongly satisfac-

tory” application of the framework while the majority of

DMs were “strongly agree” with the quality of the frame-

work’s outcome as the selected attribute set of the DRPTN

problem. Table9 shows the response of the participants to

the question addressing to what degree users were satisfied

with the application of the framework and Table10 is the

response to the question that to what degree do they agree

with the improved quality of the framework’s outcome com-

pared to the previously selected attributes.

During the open discussion after the workshop, DMs con-

firmed that it was not foreseen for them to select an attribute

set that significantly differed from the one they had previ-

ously selected. All participants agreed that disciplined and

structured evaluating of candidate attributes could lead them

to revisit the current working set of attributes. Two DMs

were not completely satisfied with the evaluation process

in the choice region due to the number of alternatives. One

DM expressed that the evaluation process in the choice

region was not as easy as it was for the screening region

with non-compensatory decision rules. Another DM took a

similar stance and suggested a mechanism to reduce the size

of the alternative set in the choice region, while two other

participants found the evaluation process of the framework

relatively easy to use. Finally, DMs responded positively to

whether the resulting attribute set fairly reflects a complete

range of their interests and values concerning the objectives

of the planning.

7 Discussion

Based on the characteristics of the framework and analy-

sis of its outcome, the following argument discusses the

reasons why we believe the application of the framework

was successful. First, since the screening region allows

for pre-evaluation, the framework is inclusive and open to

alternatives suggested by diverse sources such as experts’

opinions and literature. The framework also accepts the

redefined attributes during the evaluation process which

not only increases the likelihood of reaching a complete

set of attributes, but also provides a basis for brainstorm-

ing, critical thinking, and creative input into the model.

Harnessing the benefit of integration of compensatory and

non-compensatory techniques, alternatives are evaluated

in a thorough yet manageable manner. Therefore, although

the modeled process is flexible in accepting alternatives

as inputs, it is rigorous in evaluating them since only 26%

of attributes from the alternative pool proceeded into the

optimal region and above 42% of attributes were filtered

in the screening region. Additionally, 100% of alternatives

were evaluated at least once, while at least two stages of

evaluation took place for 52% of alternatives, and 26%

of alternatives have been assessed three times. Filtering

42% of attributes in the screening region could suggest a

rigorous screening process due to the number of attributes

in the alternative pool. Pre-evaluation or size of the alter-

native pool could impact this process, while the trade-off

Table 6 Rank, calculated utility, representing objectives, and type of attributes of the selected set

Attribute Access level to SP nodes Travel delay of

link * link flow

TTI/resources TTI/recovery duration Link centrality index Link capacity

Rank (1) (2) (3) (4) (5) (8)

Utility 29.675 29.375 28.912 28.448 27.942 25.878

Type Natural Constructed Natural Natural Proxy Natural

Objective Accessibility Effectivity Efficiency Efficiency Accessibility Mobility

646 Environment Systems and Decisions (2021) 41:633–650

1 3

between the completeness of the input attribute list and the

rigor of the process should be taken into account.

Secondly, the framework remains contextual and prob-

lem-dependent because the evaluation process relies on the

primary objectives of the original problem as the bench-

mark for the selection of candidate attributes (steps 1 and 2),

evaluation (steps 4 and 5), and identifying the generic class

of attributes for the final selection (steps 7 and 8). Further-

more, the dependency of the framework on the decision con-

text resulted in the presence of four natural attributes in the

selected set, which suggests the directness of the attribute set

and context-centric performance of the framework. Moreo-

ver, the model not only evaluates individual attributes but

also appraises the properties of desirable sets of attributes

of the targeted problem, ensuring the non-redundancy and

completeness of the set based on the primary objectives.

Thirdly, the framework does not impose a significant

cognitive burden because the evaluation factors are divided

into three independent regions with a maximum size of four

factors in a flat hierarchy format. It allows that individual

judgments, in both articulating preferences among four com-

pensatory evaluation factors and value assessments, remain

in a relatively reasonable state (Bond etal. 2008; Marttunen

etal. 2017; Cowan 2010). Furthermore, it was evident from

the implementation process and participants’ feedback that

the structured framework promotes an amount of supervi-

sion over the inevitable subjectivity associated with the

attribute selection by allowing to track and locate where

subjectivity might influence the evaluation process. Hence,

the selected attributes are less likely to be prone to bias and

error than attributes selected without a systematic, tracta-

ble, and transparent procedure. Based on the post-workshop

survey, the model-driven attributes meaningfully integrated

the concerns, values, and interests of the DMs into the deci-

sion analysis with a reduced size of the set compared to the

previously selected attributes. However, we cannot dismiss

the possibility that the positive feedback of the DMs could

have originated from availability heuristic, courtesy, etc.,

and future research must re-implement this methodology in

different settings.

Finally, the share of the redefined attributes during the

workshop in the selected set, first 10, and 20 list of attrib-

utes suggests that the framework is likely to promote crea-

tive and critical thinking. Additionally, we observed that

Table 10 Satisfaction degree for using the framework set

Strongly

satisfied

Moderately

satisfied

Neutral Moderately

dissatisfied

Strongly

dissatis-

fied

DM1 ✕

DM2 ✕

DM3 ✕

DM4 ✕

Table 7 Distribution of

attributes based on their source

in the initial pool, first 5, 10,

and 20 ranked attributes

Selected set The first

5 ranked

attributes

The first

10 ranked

attributes

The first

20 ranked

attributes

33 attributes

of the choice

region

The initial

alternative

pool

Redefined 3 50% 3 60% 3 30% 6 30% 10 30.3% 13 22.8%

Experts Survey 0 0% 0 0% 3 30% 4 20% 7 21.2% 10 17.54%

Literature 3 50% 2 40% 4 40% 10 50% 16 48.5% 34 59.64%

Table 8 An overview of the selected attributes by the framework and previously selected attributes by the DMs in an unaided process

Attribute sets Objectives Set size

Accessibility Effectivity Efficiency Mobility

Model-driven Attribute Access level to SP nodes,

Link centrality index Travel delay of link * link

flow TTI/resources, TTI/recov-

ery duration Link capacity 6

Working attributes Access level to critical

nodes, access to main

highways, west–east:

north–south connectivity

Proximity population, Peak

hour traffic flow Traffic flow improvement/

resource, Recovery

duration

Link capacity, Density 9

Table 9 Agreement degree to the quality of the selected

Strongly agree Mod-

erately

agree

Neutral Moderately

disagree

Strongly

disagree

DM1 ✕

DM2 ✕

DM3 ✕

DM4 ✕

647Environment Systems and Decisions (2021) 41:633–650

1 3

the evaluation factors and the evaluation process framed the

discussion and provided a ground for brainstorming and col-

laborative decision-making. Moreover, according to Table7,

while the academic literature provided 50% of attributes of

the selected set, DMs’ local knowledge contributes to the

other half of the attribute set which indicates the perfor-

mance of the framework with regard to balance incorpora-

tion of available knowledge sources.

Different preference elicitation techniques deliver differ-

ent weights and the same holds for different experts. While

weighting vectors mainly depend on the method and experts

that generate them, uncertainty regarding the used prefer-

ence elicitation method in this study still stands. Using any

preference elicitation method due to the absence of a known

solution or “true weight” cannot claim superiority. More

evidence from the application of the employed method is

needed to provide a level of confidence in the trustworthi-

ness of the generated weights. The preference elicitation pro-

cess could be subjected to re-implementation, uncertainty

analysis, or a wider domain of analysis to increase confi-

dence in the robustness of weights. There are two types of

uncertainty involved within the process of the framework.

First is the uncertainty related to the application of the attrib-

utes in terms of measuring DRPTN objectives, and second is

the uncertainty of the selection process. Uncertainty within

the selection process can be related to the individual sub-

jective value assignment, group dynamics, and preference

determination. Uncertainty of attributes application is due

to the lack of experimental investigation on the quality of

attributes. Further evidence is required from (1) the appli-

cation of this framework in different settings and (2) the

application of the produced attributes in real-life problems or

modeled disaster scenarios. For the former, we invite studies

to employ the proposed framework, while the latter, we aim

to address it in future research.

The developed framework can offer a practical appli-

cation in the disaster resilient infrastructure context as it

can support the problem structuring of these problems by

facilitating the identification of problem-relevant decision

attributes. The same process can (supposedly) be applied to

other similar contexts, since selecting decision attributes is

the primary and critical step of decision analysis and mod-

eling in general, particularly when the popularity of MCDA

in environmental, engineering, and management studies is

growing (Keisler and Linkov 2014; Bruen 2021). Employing

a systematically selected set of attributes for decision models

could reduce the uncertainty related to decision factors in

multi-objective or multi-criteria decision-making models.

This study suggests that in the problem-structuring phase

of decision modeling, analysts employ the suggested frame-

work or any other systematic attribute selection process to

increase the likelihood of achieving a viable attribute set.

8 Conclusion

While social and infrastructure systems encounter an

unprecedented risk of climate change and natural hazards,

it is of paramount significance to develop DSSs that pave the

way for informed collaborative decision-making. Doing so,

modelers should feed the decision models with equitable and

plausible attributes that are the result of a tenable systematic

selection framework. This study sought to bridge the iden-

tified knowledge gap in the problem structuring of multi-

criteria decision problems by proposing a choice-screening

model to assist in the evaluation of decision attributes and

prescribe a set. We illustrated the developed framework

with a case study to select decision attributes of a disaster

recovery planning problem. The innovative integration of

compensatory and non-compensatory aggregation meth-

ods within a newly designed sequential, 3-stage evaluation

process constituted the developed framework. The formal-

ized attribute selection process facilitated harnessing DMs’

knowledge and consequently led to a set of attributes of

the case problem. DMs were able to systematically evalu-

ate attributes and collaboratively produce a ranked list of

attributes as well as the final selected set. We investigated

the performance of the framework based on the typology

of the produced results, the discussed characteristics of the

framework, and the feedback from users. Observing the

development of the discussion in the workshop and position

of the redefined attributes in the final rank, it is not implau-

sible to conclude that the evaluation mechanism within the

framework facilitates critical and creative brainstorming,

thus fostering the incorporation of the available knowledge

sources. Therefore, the preliminary investigations suggest

the successful application of the framework in evaluating

and selecting an equitable and tenable set of attributes. How-

ever, further evidence from field experiments or simulated

implementation is required to support the quality of the

selected set.

This question of the extent to which decision aid inter-

ventions are successful in controlling the subjectivity

and guiding the intuitive feelings to rational judgments

has been discussed widely in other disciplines. However,

data-driven, systematic, or evidence-based approaches do

not always make a decision-making process immune to

epistemological errors (Power etal. 2019). Therefore, we

cannot rule out a possible implication of common cog-

nitive biases prevalent in many decision processes. Nev-

ertheless, it is reasonable to assume that the systematic

attribute selection process could allow analysts to track

and locate where subjectivity might influence the evalu-

ation process. For further use of this framework, we sug-

gest that a moderator oversees the evaluation session and

acts as an opposite voice, if necessary, to facilitate the

648 Environment Systems and Decisions (2021) 41:633–650

1 3

extraction and formalization of DMs’ knowledge. Using

the proposed framework, one must take into account the

size and completeness of the alternative set. A so-called

diverse complete set of alternatives is required since the

result will be as complete as the alternative pool. Never-

theless, analysts and further users of the framework must

establish a balance between the desired completeness and

the complexity of the model. One should also note that

the effectiveness of the selected set remains dependent

on the recognition of the right problem and consequently

defining the right objectives, since the framework remains

true to the defined primary objectives of the decision prob-

lem. Research and practice can both use the proposed

framework for establishing an equitable set of attributes

of decision problems, and even one who is not necessar-

ily an expert in decision analysis. While dealing with any

decision context involving multiple criteria, a systematic

approach towards the identification and selection of deci-

sion attributes need to be employed. More time and effort

must be dedicated to a formal problem-structuring phase

before formulating a decision problem, specifically with

regard to complex and critical problems, such as those

which address environmental challenges, risk assessment,

and disaster resilience planning.

Acknowledgements We would like to acknowledge the contribution of

three anonymous reviewers (especially “reviewer 2”) for their valuable

insights and comments towards improving this paper.

Funding Open Access funding enabled and organized by Projekt

DEAL.

Declarations

Conflict of interest The authors have no conflict of interest with regard

to the content of this manuscript.

Open Access This article is licensed under a Creative Commons Attri-

bution 4.0 International License, which permits use, sharing, adapta-

tion, distribution and reproduction in any medium or format, as long

as you give appropriate credit to the original author(s) and the source,

provide a link to the Creative Commons licence, and indicate if changes

were made. The images or other third party material in this article are

included in the article’s Creative Commons licence, unless indicated

otherwise in a credit line to the material. If material is not included in

the article’s Creative Commons licence and your intended use is not

permitted by statutory regulation or exceeds the permitted use, you will

need to obtain permission directly from the copyright holder. To view a

copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.

References

Alexander DE (2013) Resilience and disaster risk reduction: an ety-

mological journey. Nat Hazards Earth Syst Sci 13:2707–2716.

https:// doi. org/ 10. 5194/ nhess- 13- 2707- 2013

Alfares HK, Duffuaa SO (2008) Assigning cardinal weights in multi-

criteria decision making based on ordinal ranking. J Multi-Crit

Decis Anal 15:125–133. https:// doi. org/ 10. 1002/ mcda. 420

Aydin NY, Duzgun H, Heinimann H, Wenzel F, Gnyawali KR (2018)

Framework for improving the resilience and recovery of trans-

portation networks under geohazard risks. Int J Disaster Risk

Reduct 31:832–843

Baker D, Bridges D, Hunter R etal (2001) Guidebook to decision-

making methods. US Department of Energy Washington, DC,

WSRC-IM-2002-00002

Barfod MB, Leleur S (eds) (2014) Multi-criteria decision analysis

for use in transport decision making. 2 edn, DTU Transport,

DTU Lyngby

Barron FH, Barrett BE (1996) Decision quality using ranked attrib-

ute weights. Manag Sci 42:1515–1523

Beling PA (2013) Multi-scale decision making: challenges in engi-

neering and environmental systems. Environ Syst Decis 33:323–

325. https:// doi. org/ 10. 1007/ s10669- 013- 9469-y

Belton V (1999) Multi-criteria problem structuring and analysis in

a value theory framework. In: Gal T, Stewart T, Hanne T (eds)

Multicriteria decision making, advances in MCDM—models,

algorithms, theory, and applications. Kluwer Academic Publish-

ers, Dordrecht, pp 12–132

Belton V, Stewart T (2012) Multiple criteria decision analysis: an

integrated approach. Springer, New York

Bond SD, Carlson KA, Keeney RL (2008) Generating objectives:

can decision makers articulate what they want? Manag Sci

54(1):56–70. https:// doi. org/ 10. 1287/ mnsc. 1070. 0754

Bruen M (2021) Uptake and dissemination of multi-criteria decision

support methods in civil engineering—lessons from the litera-

ture. Appl Sci 11:2940. https:// doi. org/ 10. 3390/ app11 072940

Bruneau M, Chang SE, Eguchi RT etal (2003) A Framework to

quantitatively assess and enhance the seismic resilience of

communities. Earthq Spectra 19(4):733–752. https:// doi. org/

10. 1193/1. 16234 97

Caruzzo A, Blanco CMR, Joe P (2020) Developing a multi-

attribute decision aid model for selection of a weather

radar supplier. Environ Syst Decis. https:// doi. org/ 10. 1007/

s10669- 020- 09770-3

Cegan JC, Filion AM, Keisler JM etal (2017) Trends and applications

of multi-criteria decision analysis in environmental sciences: lit-

erature review. Environ Syst Decis 37:123–133. https:// doi. org/

10. 1007/ s10669- 017- 9642-9

Cinelli M, Spada M, Kim W etal (2020) MCDA Index Tool: an interac-

tive software to develop indices and rankings. Environ Syst Decis.

https:// doi. org/ 10. 1007/ s10669- 020- 09784-x

Clemen RT (1996) Making hard decisions: an introduction to decision

analysis, 2nd edn. Duxbury Press, Pacific Grove

Convertino M, Baker KM, Vogel JT etal (2013) Multi-criteria decision

analysis to select metrics for design and monitoring of sustainable

ecosystem restorations. Ecol Ind 26:76–86

Corner J, Buchanan J, Henig M (2001) Dynamic decision problem

structuring. J Multi-Crit Decis Anal 10:129–141. https:// doi. org/

10. 1002/ mcda. 295

Cowan N (2010) The magical mystery four: how is working memory

capacity limited, and why? Curr Dir Psychol Sci 19(1):51–57.

https:// doi. org/ 10. 1177/ 09637 21409 359277

Dale VH, Efroymson RA, Kline KL, Davitt MS (2015) A framework

for selecting indicators of bioenergy sustainability. Biofuels Bio-

prod Biorefin 9(4):435–446. https:// doi. org/ 10. 1002/ bbb. 1562

Danielson M, Ekenberg L (2016) The CAR method for using pref-

erence strength in multi-criteria decision making. Group Decis

Negot 25:775–797. https:// doi. org/ 10. 1007/ s10726- 015- 9460-8

Desmond M (2007) Decision criteria for the identification of alter-

natives in strategic environmental assessment. Impact Assess

649Environment Systems and Decisions (2021) 41:633–650

1 3

Project Apprais 25(4):259–269. https:// doi. org/ 10. 3152/ 14615

5107X 269067

Dodgson J, Spackman M, Pearman AD, Phillips LD (2009) Multi-

criteria analysis: a manual. Department of the Environment, Lon-

don: Department of the Environment, Transport and the Regions

Elboshy B, Kanae S, Gamaleldin M etal (2019) A framework for plu-

vial flood risk assessment in Alexandria considering the coping

capacity. Environ Syst Decis 39:77–94. https:// doi. org/ 10. 1007/

s10669- 018- 9684-7

Fekete A (2019) Social vulnerability (Re-)assessment in context to

natural hazards: review of the usefulness of the spatial indicator

approach and investigations of validation demands. Int J Disaster

Risk Sci 10:220. https:// doi. org/ 10. 1007/ s13753- 019- 0213-1

Fox-Lent C, Bates ME, Linkov I (2015) A matrix approach to com-

munity resilience assessment: an illustrative case at Rockaway

Peninsula. Environ Syst Decis 35:209–218

Franco AL, Montibeller G (2010) Problem structuring for multi-criteria

decision analysis interventions. Working Paper 09(115), pp 1–25

Gigerenzer G, Gaissmaier W (2011) Heuristic decision making.

Annu Rev Psychol 62(1):451–482. https:// doi. org/ 10. 1146/ annur

ev- psych- 120709- 145346

Gigerenzer G, Goldstein DG (1996) Reasoning the fast and frugal way:

models of bounded rationality. Psychol Rev 103:650–669

Girod M etal (2003) Decision making in conceptual engineering

design: an empirical investigation. Proc Inst Mech Eng Part B

217(9):1215–1228

Goujon B, Labreuche C (2015) Use of a multi-criteria decision support

tool to prioritize reconstruction projects in a post-disaster phase.

In: ICTDM 2015, Rennes

Gregory R, Failing L (2002) Using decision analysis to encourage

sound deliberation: water use planning in British Columbia,

Canada. J Pol Anal Manag 21:492–499. https:// doi. org/ 10. 1002/

pam. 10059

Ha MH, Yang Z (2018) Modelling interdependency among attributes in

MCDM: its application in port performance measurement. In Lee

PT-W, Yang Z (eds), Multi-criteria decision making in maritime

studies and logistics, International Series in Operations Research

& Management Science 260

Hämäläinen R, Alaja S (2008) The threat of weighting biases in envi-

ronmental decision analysis. Ecol Econ 68:556–569

Herrera H, Kopainsky B (2020) Using system dynamics to support a

participatory assessment of resilience. Environ Syst Decis. https://

doi. org/ 10. 1007/ s10669- 020- 09760-5

Höfer S, Ziemba A, El Serafy GA (2020) Bayesian approach to ecosys-

tem service trade-off analysis utilizing expert knowledge. Environ

Syst Decis 40:67–83. https:// doi. org/ 10. 1007/ s10669- 019- 09742-2

Huang IB, Keisler J, Linkov I (2011) Multi-criteria decision analysis in

environmental sciences: ten years of applications and trends Sci-

ence of the Total Environment. Sci Total Environ 409:3578–3594

Jacobi SK, Hobbs B (2007) Quantifying and mitigating the splitting

bias and other value tree-induced weighting biases. Decis Anal

4:194–210

Joshi A, Saket K, Satish K etal (2015) Likert scale: explored and

explained. Br J Appl Sci Technol 7:396–403

Kahneman D, Slovic P, Tversky A (1982) Judgment under uncertainty:

heuristics and biases. Cambridge University Press, New York

Kárný M (2013) Automated preference elicitation for decision making.

In: Guy T, Karny M, Wolpert D (eds) Decision making and imper-

fection studies in computational intelligence, vol 474. Springer,

Berlin

Kassem A, Al-Haddad K, Komljenovic D, Schiffauerova A (2016) A

value tree for identification of evaluation criteria for solar ther-

mal power technologies in developing countries. Sustain Energy

Technol Assess 16:18–32

Keeney R (1982) Decision analysis: an overview. Oper Res

30(5):803–838

Keeney R (1992) Value focused thinking: a path to creative decision

making. Harvard University Press, Cambridge

Keeney R (2007) Developing objectives and attributes. In: Edwards W,

Miles RF, von Winterfeldt D (eds) Advances in decision analysis.

Cambridge University Press, New York

Keeney R, Gregory R (2005) Selecting attributes to measure the

achievement of objectives. Oper Res 53:1–11

Keeney R, Raiffa H (1993) Decisions with multiple objectives: prefer-

ences and value trade-offs, 2nd edn. Cambridge University Press,

Cambridge

Keeney RL, McDaniels TL (1992) Value-focused thinking about strate-

gic decisions at BC hydro. INFORMS J Appl Anal 22(6):94–109

Keisler J, Linkov I (2014) Models, decisions and environment. Environ

Syst Decis 34:369–372

Koks E, Rozenberg J, Zorn C etal (2019) A global multi-hazard risk

analysis of road and railway infrastructure assets. Nat Sustain

10:1–11

Kunsch PL, Ishizaka A (2019) A note on using centroid weights

in additive multi-criteria decision analysis. Eur J Oper Res

277(1):391–393

Kurth M, Kozlowski W, Ganin A, Mersky AC, Leung B, Dykes J,

Kitsak M, Linkov I (2020) Lack of resilience in transportation

networks: economic implications. Transp Res Part D 86:102419

Lin T, Lin JY, Cui SH, Cameron S (2009) Using a network frame-

work to quantitatively select ecological indicators. Ecol Indic

9:1114–1120

Linkov I, Eisenberg DA, Bates ME, Chang D, Convertino M, Allen

JH, Seager TP (2013) Measurable resilience for actionable policy.

Environ Sci Technol 47(18):10108–10110

Linkov I, Fox-Lent C, Read L etal (2018) Tiered approach to resilience

assessment. Risk Anal 38(9):1772–1780

Liu Y, McNeil S, Hackl J, Adey B (2020) Prioritizing transportation

network recovery using a resilience measure. Sustain Resilient

Infrastruct. https:// doi. org/ 10. 1080/ 23789 689. 2019. 17081 80

Ma W, Luo X, Jiang Y (2017) Multicriteria decision making with

cognitive limitations: a ds/ahp-based approach. Int J Intell Syst

32(7):686–721

Maier K, Stix V (2013) A semi-automated approach for structuring

multi criteria decision problems. Eur J Oper Res 225(3):487–496

Majumder M (2015) Impact of urbanization on water shortage in

face of climatic aberrations, pp 35–48. https:// doi. org/ 10. 1007/

978- 981- 4560- 73-3

Manyaga F, Nilufer N, Hajaoui Z (2020) A systematic literature review

on multi-criteria decision making in disaster management. Int J

Bus Ecosyst 2(2):1–7. https:// doi. org/ 10. 36096/ ijbes. v2i2. 197

Marttunen M, Belton V, Lienert J (2017) Are objectives hierarchy

related biases observed in practice? A meta-analysis of environ-

mental and energy applications of Multi-Criteria Decision Analy-

sis. Eur J Oper Res

McDaniels TL, Chang SE, Hawkins D etal (2015) Towards disaster-

resilient cities: an approach for setting priorities in infrastructure

mitigation efforts. Environ Syst Decis 35:252–263. https:// doi. org/

10. 1007/ s10669- 015- 9544-7

McIntosh RD, Becker A (2020) Applying MCDA to weight indicators

of seaport vulnerability to climate and extreme weather impacts

for US North Atlantic ports. Environ Syst Decis. https:// doi. org/

10. 1007/ s10669- 020- 09767-y

Merad M, Dechy N, Marcel F etal (2013) Multiple-criteria decision-

aiding framework to analyze and assess the governance of sustain-

ability. Environ Syst Decis 33:305–321. https:// doi. org/ 10. 1007/

s10669- 013- 9447-4

Mirzaee S, Fannon D, Ruth M (2019) A comparison of preference elici-

tation methods for multi-criteria design decisions about resilient

and sustainable buildings. Environ Syst Decis 39:439–453. https://

doi. org/ 10. 1007/ s10669- 019- 09726-2

650 Environment Systems and Decisions (2021) 41:633–650

1 3

Mitroff II, Featheringham TR (1974) On systemic problem solving and

the error of the third kind. Behav Sci 19:383–393

Munda G (2005) Multi-criteria analysis. In: Proops J, Safonov P

(eds) Modelling in ecological economics. Edward Elgar, NERA

(National Economic Research Associates)

Niemeijer D, de Groot RS (2006) A conceptual framework for selecting

environmental indicator sets. Ecol Indic 8:14–25

Otto SA, Kadin M, Casini M etal (2018) A quantitative framework for

selecting and validating food web indicators. Ecol Indic 84:619–

631. https:// doi. org/ 10. 1016/j. ecoli nd. 2017. 05. 045

Pearson J, Punzo G, Mayfield M etal (2018) Flood resilience: con-

solidating knowledge between and within critical infrastructure

sectors. Environ Syst Decis 38:318–329. https:// doi. org/ 10. 1007/

s10669- 018- 9709-

Power DJ, Cyphert DR, Roth RM (2019) Analytics, bias, and evidence:

the quest for rational decision making. J Decis Syst 28(2):120–

137. https:// doi. org/ 10. 1080/ 12460 125. 2019. 16235 34

Quigley MC, Bennetts LG, Durance P etal (2019) The provision and

utility of science and uncertainty to decision-makers: earth sci-

ence case studies. Environ Syst Decis 39:307–348. https:// doi. org/

10. 1007/ s10669- 019- 09728-0

Rand K, Kurth M, Fleming CH, Linkov I (2020) A resilience matrix

approach for measuring and mitigating disaster-induced popula-

tion displacement. Int J Disaster Risk Reduct 42:e101310

Roberts R, Goodwin P (2002) Weight approximations in multi-attribute

decision models. J Multi-Crit Decis Anal 11:291–303. https:// doi.

org/ 10. 1002/ mcda. 320

Rossberg AG, Uusitalo L, Berg T, Zaiko A, Chenuil A, Uyarra MC,

Borja A, Lynam CP (2017) Quantitative criteria for choosing tar-

gets and indicators for sustainable use of ecosystems. Ecol Indic

72:215–224. https:// doi. org/ 10. 1016/j. ecoli nd. 2016. 08. 005

Roszkowska E (2013) Rank ordering criteria weighting methods—

a comparative overview. Optimum Studia Ekonomiczne

5(65):14–33

Rothrock L, Yin J (2008) Integrating compensatory and noncompensa-

tory decision-making strategies in dynamic task environments.

In: Kugler T, Smith JC, Connolly T, Son YJ (eds) Decision mod-

eling and behavior in complex and uncertain environments, vol

21. Springer, New York

Rouhanizadeh B, Kermanshachi S (2019) Investigating the rela-

tionships of socioeconomic factors delaying post-disaster

reconstruction. Comput Civil Eng 33–40. https:// doi. org/ 10. 1061/

97807 84482 445. 005

Roy B (1996) Multicriteria methodology for decision aiding. Kluwer,

Dordrecht

Salo AA, Hamalainen RP (2001) Preference ratio in multiattribute

evaluation (PRIME)- elicitation and decision procedures under

incomplete information. IEEE Trans Syst Man Cybern Part A

31(6):533–545

Sandri O, Hayes J, Holdsworth S (2020) Regulating urban development

around major accident hazard pipelines: a systems comparison of

governance frameworks in Australia and the UK. Environ Syst

Decis. https:// doi. org/ 10. 1007/ s10669- 020- 09785-w

Tiesmeier DK (2016) MCDM problem-structuring framework and

a real estate decision support model. University of Manchester,

Manchester

Tversky A (1972) Elimination by aspects: a theory of choice. Psychol

Rev 79:77–94

Vaidya A, Mayer LA (2016) Criteria and indicators for a bioenergy

production industry identified via stakeholder participation. Int J

Sustain Dev World Ecol 23(6):526–540

Von Winterfeldt D, Fasolo B (2009) Structuring decision problems:

a case study and reflections for practitioners. Eur J Oper Res

199(3):857–866

Walpole EH, Toman E, Stidham M etal (2020) The science and prac-

tice of ecological restoration: a mental models analysis of resto-

ration practitioners. Environ Syst Decis. https:// doi. org/ 10. 1007/

s10669- 020- 09768-x

Yu H, Solvang WD (2017) A multi-objective location-allocation

optimization for sustainable management of municipal solid

waste. Environ Syst Decis 37:289–308. https:// doi. org/ 10. 1007/

s10669- 017- 9632-y

Zamanifar M, Hartmann T (2020) Optimization-based decision-mak-

ing models for disaster recovery and reconstruction planning of

transportation networks. Nat Hazards. https:// doi. org/ 10. 1007/

s11069- 020- 04192-5

Zamanifar M, Hartmann T (2021) Decision attributes for disaster

recovery planning of transportation networks: a case study. Transp

Res Part D 93:102771. https:// doi. org/ 10. 1016/j. trd. 2021. 102771

Zhang W, Wang N, Nicholson C (2017) Resilience-based post-disaster

recovery strategies for road-bridge networks. Struct Infrastruct

Eng. https:// doi. org/ 10. 1080/ 15732 479. 2016. 12718 13