This version is available at https://doi.org/10.14279/depositonce-8202
© © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for
all other uses, in any current or future media, including reprinting/republishing this material for
advertising or promotional purposes, creating new collective works, for resale or redistribution to
servers or lists, or reuse of any copyrighted component of this work in other works.
Terms of Use
Amorim, T.; Vogelsang, A.; Pudlitz, F.; Gersing, P.; Philipps, J. (2019): Strategies and Best Practices for
Model-based Systems Engineering Adoption in Embedded Systems Industry. In: 2019 ACM/IEEE 41st
International Conference on Software Engineering. IEEE.
Tiago Amorim, Andreas Vogelsang, Florian Pudlitz, Peter Gersing, Jan
Philipps
Strategies and Best Practices for Model-
based Systems Engineering Adoption in
Embedded Systems Industry
Accepted manuscript (Postprint)Conference paper |
Strategies and Best Practices for Model-based
Systems Engineering Adoption in Embedded
Systems Industry
Tiago Amorim
Technische Universit¨
at Berlin
Berlin, Germany
Peter Gersing
GPP Communication GmbH & Co. KG
Munich, Germany
Andreas Vogelsang
Technische Universit¨
at Berlin
Berlin, Germany
andreas.v[email protected]
Jan Philipps
foqee GmbH
Munich, Germany
Florian Pudlitz
Technische Universit¨
at Berlin
Berlin, Germany
Abstract—[Context] Model-based Systems Engineering
(MBSE) advocates the integrated use of models throughout
all development phases of a system development life-cycle. It is
also often suggested as a solution to cope with the challenges
of engineering complex systems. However, MBSE adoption is
no trivial task and companies, especially large ones, struggle
to achieve it in a timely and effective way. [Goal] We aim to
discover what are the best practices and strategies to implement
MBSE in companies that develop embedded software systems.
[Method] Using an inductive-deductive research approach, we
conducted 14 semi-structured interviews with experts from
10 companies. Further, we analyzed the data and drew some
conclusions which were validated by an on-line questionnaire in
a triangulation fashion. [Results] Our findings are summarized
in an empirically validated list of 18 best practices for MBSE
adoption and through a prioritized list of the 5 most important
best practices. [Conclusions] Raising engineers’ awareness
regarding MBSE advantages and acquiring experience through
small projects are considered the most important practices to
increase the success of MBSE adoption.
Keywords—Model-based Systems Engineering, Process adop-
tion, Best Practices, Embedded systems, Empirical research
I. INTRODUCTION
Model-based Systems Engineering (MBSE) is the formal-
ized application of modeling to support system requirements,
design, analysis, verification and validation activities. It be-
gins in the conceptual design phase and continues throughout
development and later life cycle phases [1]. MBSE is part of
a long-term trend towards model-centric approaches adopted
by other engineering disciplines including mechanical, elec-
trical, and software.
In this approach, models (as opposed to document-centric
approaches) serve as blueprints for developers to write code,
provide formalization, tackle complexity, and enhance the
understanding of the system. Specialized tools automate
much of the non-creative work (which translates to gains
in productivity and quality) and generate code based on
the models. MBSE foster artifact reuse, improves product
quality, and shortens time to market [2].
Despite the aforementioned benefits, adopting MBSE is a
complex task, especially for larger and established compa-
nies [3]. Process changes are required in all system life-cycle
phases as well as a shift in the development paradigm (i.e.,
abstract thinking [4]) and application of new tools. Projects
are not likely to meet their cost and delivery target when
adoption is carried out poorly.
Our goal was to find out what was tried, what worked,
what did not work, how the problems were solved, what
can be recommended, and what should be avoided when
adopting MBSE in organizations that develop embedded
systems. For this purpose, we conducted 14 semi-structured
interviews with experts from embedded systems organiza-
tions. From these interviews, we extracted 18 best practices
fitted for tackling MBSE adoption challenges. Sequentially,
we validated and prioritized the best practices with the help
of an on-line questionnaire which was answered by MBSE
practitioners.
Our findings provide input for planning MBSE adoption
based on the knowledge of practitioners that went through
the experience of implementing MBSE in already established
embedded systems development organizations. Our paper
makes the following contributions:
•A granular set of MBSE adoption best practices derived
from real field experience and validated by practitioners
through a questionnaire.
•A prioritized list of the five most important best prac-
tices.
•The findings shed light on MBSE adoption issues and
how they were overcome by practitioners.
We expect that this knowledge can help organizations to
adopt MBSE in a cost-effective and timely manner.
II. STUDY APPROACH
A. Research Objective
The research objective is summarized in terms of the
Goal-Question-Metric [5] (a.k.a. GQM) template in Table I.
B. Research Design
For this study, we devised an inductive-deductive research
approach in a triangulation fashion [6] composed of three
main activities as depicted in Fig. 1. Inductive reasoning is
used to build hypotheses and theories. However, inductive
reasoning allows for the conclusion to be false [7], thus we
applied deductive reasoning to our findings. Such measure
also addresses some threats to validity (cf. Section II-C). The
activities are described in detail in the following paragraphs.
TABLE I
RESEARCH OBJECTIVE
Interview
with
Practitioners
Qualitative
Coding
Analysis
Verbatim of
Interviews
Adoption
Strategies and
Best Practices
Application of
Validation
Questionnaire
Validated
Results
Activities Outputs
Inductive Reasoning
Deductive
Reasoning
Fig. 1. Research work-flow
1) Inductive Reasoning: At the inductive phase, we fol-
lowed an exploratory research approach [8] with the goal to
identify MBSE adoption strategies and best practices from
the experience of experts. The method provides insights
into the examined topic and gives essential information to
understand the phenomenon in its real context [9], [10].
Semi-structured interviews were used to collect the data,
qualitative coding analysis [11] was used to analyze the data.
Interview with Practitioners. We conducted 14 face-
to-face interviews, each taking around 60 minutes. We
developed an interview guide [12] that was structured along
a funnel model [10]. It started with general questions about
the participant’s context and the understanding of MBSE.
Afterwards going into detail about specific topics such as
employee training, process and tooling selection, and adop-
tion experiences. The interviewee selection was based on two
criteria: First, the interviewee should have work experience
of several years. Second, the interviewee should work in
an environment where MBSE adoption is a realistic option.
Therefore, we restricted the group of interviewees to people
working on embedded systems or within this context. The
participants were screened from work contacts of the authors
and industrial partners that participated in a research project1
with focus on MBSE adoption in practice. Table II provides
an overview of the participants and respective demographic
1https://spedit.in.tum.de/
TABLE II
INTERVIEW PARTICIPANTS
ID Industry
Sector
Type of
Company
Role of
Participant
P1 Tool vendor OEM Technical Sales
P2 Tool vendor Academic Professor
P3 R&D services SME Manager
P4 Automotive OEM Head of Development
P5 Automotive OEM Systems Engineer
P6 Medical SME Head of SW Development
P7 Automotive Supplier Function Architect
P8 Automotive OEM SW Architect
P9 Research Academic Professor
P10 Automotive Supplier Developer
P11 Avionics OEM Team Lead
P12 Electronics OEM Head of SW Development
P13 Avionics SME Head of System Engineering
P14 Robotics OEM Team Lead
data. In consent with the interviewee, the interviewer took
notes for detailed analysis. The outcome of this activity is the
Verbatim of Interviews, which is input for the next activity.
Qualitative Coding Analysis. Three researchers analyzed
the interviews using qualitative coding [11] and managed
using the qualitative data analysis tool ATLAS.ti2. Neither
of them participated in the interview phase. The analysis
started with all three researchers working on the same five
interviews. The results were later discussed and merged
in a meeting. We used the discussions to homogenize the
understanding of the codes among the researchers [13]
(i.e., what/how to look for). The remaining interviews were
tackled in a cross-analysis fashion. In a round with all three
researchers, the unresolved conflicts were ironed out. The
outcome of this activity was a list of Adoption Strategies
and Best Practices for MBSE adoption.
2) Deductive Reasoning: Rather than obtaining new in-
formation, our goal in this phase was to validate our previous
finding through triangulation [6]. For this mean, a question-
naire was built based on the findings of the previous activity
and distributed among practitioners.
Application of Validation Questionnaire. The question-
naire respondents were screened from two research projects
mailing list, CrESt3and SPEDiT4, and two MBSE dis-
cussion groups from a business and employment-oriented
social network5. Our anonymous questionnaire had three
parts. In the first part, we asked for demographic data
(organization size (cf. Fig. 2), industry sector (cf. Fig. 3))
and added a yes/no question whether the respondent has ever
participated or observed some endeavor to introduce Model-
based Engineering in an organization. We used this question
to exclude respondents without proper experience. We had
40 respondents in total, 3 were excluded. In the second part,
we asked the respondents to express their agreement with
each best practice (BP) using a Likert scale [14] with four
levels: Strongly Disagree, Disagree, Agree, Strongly Agree.
In the last part, we asked the respondents to select up to 5
BPs that they considered most important.
2http://atlasti.com
3https://crest.in.tum.de/
4https://spedit.in.tum.de/
5https://www.linkedin.com
13,5%
13,5%
10,8%
62,2%
1-20
21-100
101-1000
> 1000
Fig. 2. Organization size of the questionnaire respondents (i.e., number of
employees)
35,1%
29,7%
21,6%
8,1%
5,4%
Automotive
Others
Academia
Energy
Manufacturing
Fig. 3. Industry sector of the questionnaire respondents
Section III contains the output of all activities and respec-
tive discussion.
C. Threats to Validity
The validity of our results is subject to the following
threats:
Sampling bias. All interview subjects work in Germany
(omission bias). They were selected from a convenience
sample of project partners and professional contacts (in-
clusion bias). To mitigate these issues, we created the
questionnaire and distributed it to a broader audience (i.e.,
triangulation).
Researcher bias. To mitigate this threat, the interviews
were conducted by two researchers who took notes indepen-
dently. Further, the interpretation presented in this paper was
validated through a questionnaire.
Research method. To minimize misunderstandings be-
tween interviewees and the researchers, the study goal was
explained to the participants prior to the interview. Steps
taken to improve the reliability of the interview guide
included a review and a pilot test. We followed several
strategies as proposed by Maxwell [15] to mitigate threats.
External validity. We expect that our results are repre-
sentative for the embedded systems industry, thus we cannot
generalize the results to other types of industry.
D. Availability of Data.
Due to unreasonable effort necessary for anonymizing the
interview transcripts, we do not disclose them. However, we
disclose the interview guideline, the questionnaire, and the
results of the questionnaire6.
III. RESULTS AND DISCUSSION
In this section, we present and discuss our findings, which
are shaped in the form of Best Practices and are classified
in four groups:
•Piloting - what to care about at the first attempts
•Tool and process - selection and adoption decisions
•Knowledge building - development of team competence
•Management - human resources motivation and support
This section is divided as following. Next subsection
presents all best practices, the following four subsections
describe the groups and associated best practices. We present
quotes as evidence and the result of the questionnaire. We
also discuss related existing work. In the last subsection, we
present the list of Best Practices that are the top five most
selected best practices.
A. Summary of Best Practices
The identified best practices and their respective groups
are summarized below:
Piloting
BP01: The organization should start adopting MBSE with new
projects.
BP02: The pilot project should create real value for the orga-
nization (i.e., no didactic project).
BP03: The pilot project should have enough budget and time
allocated to bear the overhead of adoption.
BP04: No translation of old artifacts, exception is when con-
sidering reusable artifacts.
BP05: Start small in terms of project and team size in order
to acquire some experience.
Tools and Process
BP06: Tools with open interfaces and homogeneous work-flow
are preferred.
BP07: All engineers should have access to the tools.
BP08: Tool acquisition is very costly therefore should be
thoroughly planned.
BP09: Have the new MBSE processes well documented so you
better understand what tool you will need.
Knowledge building
BP10: All engineers should get, at least, basic training in
MBSE.
BP11: Using examples that are familiar to the domain of
the organization eases the understanding. Model some
existing artifacts for using as examples.
BP12: Many strategies can be used to build knowledge of an
organization, the context should be taken into consid-
eration.
BP13: There should be a planned form of later evaluation to
fill eventual gaps.
Management
BP14: Make the advantages of MBSE clear.
BP15: Have technically prepared people to support your engi-
neers (i.e., not sales personnel).
BP16: Bring everyone to adoption (i.e., avoid creating castes).
6https://figshare.com/s/e5e2c5535282ae60fb29
51%
43%
41%
62%
51%
38%
19%
54%
24%
41%
11%
32%
5%
11%
8%
0% 20% 40% 60% 80% 100%
BP05: Start small in terms of project and
team size in order to acquire some…
BP04: No translation of old artifacts except
for reusable artifacts.
BP03: The pilot project should have enough
budget and time allocated to bear the…
BP02: The pilot project should create real
value for the organization (i.e., no didactic…
BP01: The organization should start
adopting MBSE with new projects.
Strongly Disagree Disagree Agree Strongly Agree
Fig. 4. Questionnaire results for Piloting group.7
BP17: If you have good engineers let them do the work for
you, it is cheaper and they will engage more (i.e.,
empowering).
BP18: Management should unify all employees towards adop-
tion.
B. Piloting
This subsection describes best practices related to the
initial attempt to implement MBSE and how to harvest
benefits from it. The results of the questionnaire for the best
practices of this group are depicted in Fig. 4. In general,
the respondents agreed with most of the BPs. BP03 got
the most agreement (95%) among its peers. On the other
hand, 38% of the participants disagreed with BP04. Our
hypothesis is that respondents that have no reuse culture in
their organizations are likely to disagree with this BP.
BP01: The organization should start adopting MBSE
with new projects.
“New projects with MBSE” (P14)
“Do not recycle the past” (P9)
“No migration of old projects” (P8)
“New start in new project preferred” (P1)
“Set up own project for method development instead of
accompanying from series development” (P3)
“Remove [team] from existing organization, set up your
own project, the rest continues as before. Within a few years,
new product generation has higher market share.” (P10)
Introducing MBSE through new projects was a good
practice pointed out by many interviewees. By doing
so, effort would not be wasted with re-work of existing
artifacts that might just be archived in a near future.
Starting from scratch also prevent developers to shortcut the
work-flow with information that is already documented thus
compromising the learning of the method (i.e., there is no
document version for referencing the possible incomplete
information). Additionally, already running projects have
fixed budget and deadlines that do not consider such
overhead. Meeting those constraints gets harder when the
engineers need to concentrate on learning MBSE.
“Legacy stay where they are” (P14)
7BP02 and BP04 received 3% and 6% respectively of ‘Strongly Dis-
agree’.
“Existing projects remained in old surroundings and were
gradually archived” (P1)
“Backwards compatibility, so that only changes need to
be redesigned (includes not only code, but also specs, tests,
test infrastructure)” (P10)
As MBSE became more pervasive in the organizations,
the old document-based projects were naturally phased-out.
Exceptions for this are Product Line intensive organizations
or organizations with high reuse levels. In this case,
translating the existing document-based artifacts that
are meant to be reused to model-based artifacts is a
recommended practice.
BP02: The pilot project should create real value for the
organization (i.e., no didactic project).
“Start with a concrete project” (P9)
Learning MBSE through a real project with real deadlines
and milestones is reportedly better than learning through
mock up projects and generic examples. The real-life setting
boosts engineers’ motivation since their work is already
producing something useful. Additionally, real projects are
related to the domain of the organization.
BP03: The pilot project should have enough budget and
time allocated to bear the overhead of adoption.
“The first projects must be able to bear the burden.” (P9)
“Business case because of investment hurdle in the begin-
ning” (P12)
The effort needed by engineers to learn should be
considered when deciding for the pilot project, which must
be able to bear the burden i.e., managers should avoid using
critical (time-wise, business-wise, technology-wise) projects
for this matter. By doing so, managers can mitigate the risk
of a project failing to meet its constraints or MBSE being
not properly implemented.
BP04: No translation of old artifacts except for reusable
ones.
“Parts with re-use are taken over” (P14)
“It started with reverse engineering. The existing system
was clustered according to the information model” (P7)
Despite the support for using new projects, converting
the existing document-based artifacts that are prone to
re-use was also a strategy presented by some respondents.
Such artifacts are well detailed thus allowing the developer
to concentrate in the methodology rather than the content.
Moreover, the newly created models can be compared to
the existing document-based version for verification and
validation of the models. It is important to chose artifacts
that are going to be reused in future MBSE projects
otherwise the effort will just be used for learning purposes.
BP05: Start small in terms of project and team size in
order to acquire some experience.
“Only 1-2 people, then the rest of the team” (P14)
“Start in some areas that you can better oversee [...]
Best to start in individual areas according to company
guidelines” (P6)
“Often pilot installation to gain experience, then expan-
sion companies-wide” (P1)
“It is advisable to start with a partial introduction first,
later through a consistent MBE. - needed breath” (P2)
“Introduction as ”submarine project” by dedicated devel-
oper, then application in own department, now overarching
roll-out.” (P5)
“If necessary implementation strategy with 3-4 months
observation of what the developer does, and then decide
how to improve the process with MBE” (P2)
Starting with a small, highly motivated group of
employees in a grassroots strategy fashion is likely to
increase the chances of success. Such practice allows the
training the implementation in an easily controlled setting
and identification of characteristics that are exclusive to
the organization (current processes and auxiliary tools)
and its domain (development of DSL). Thus, issues can
be identified with more precision and addressed promptly.
Once the small group develops acquaintance with the new
tasks and tools, the implementation should be rolled out to
the rest of the team.
Related work. Investments in training, tools and modeling
environment should be considered in the projects selected
for initially applying MBSE. Expectations should be handled
so proper time is given for the effort to pay dividends [16].
The experience of Fosse and Bayer [16] made the authors
support the “learning by doing” strategy in real life projects
by keeping the focus on project deliverables and modeling
as needed. According to the authors, the pressure to deliver
real engineering products forces discovery and resolution of
problems not likely encountered in a didactic-only project.
The authors stated that it is possible to yield communica-
tion and understanding improvement at an earlier stage by
focusing at description first, then analysis. Hutchinson et al.
[4] state that “starting small” when adopting MDE and then
growing, thus committing more resources on a wider scale
is a helpful response towards MDE adoption.
C. Tools and process
Tools are an important cornerstone of MBSE. Traceability,
simulation, automatization of task are among the things
that are only possible with the use of specialized tools.
However, tool licenses are expensive and new tools require
learning investment. Rolling back tool acquisition is very
costly, thus wise decisions can save time and resources.
Proper tool selection requires identifying what capabilities
the organization would like to develop by implementing
MBSE and acquire the tools that will help it fulfills its goal.
Figure 5 shows the survey results for Tools and process
best practices. We can see a fairly amount of agreement
among the respondents and very few disagreement
responses.
BP06: Tools with open interfaces and homogeneous
work-flow are preferred.
“Customers want homogeneous tool chain but open inter-
faces” (P1)
“Often the entire tool chain is replaced” (P1)
Because MBSE is so complex and encompasses the
complete system development life-cycle, supporting tool
chains are often equally complex. Currently, several tools
are required to do the job because existing ones are not
developed enough to cover the whole life-cycle. For this
matter, import and export features must allow the work to
continue in further design phases without much ado, thus
interoperability through open interfaces becomes a very
desirable characteristic. By seeking open interfaces tools,
the organization protects himself of many problems that
using exclusive proprietary tools can bring.
BP07: All engineers should have access to the tools.
“The worker level must also get the tools” (P9)
“Tool usage for all developers who use the tools” (P6)
MBSE tools should be widely available for all engineers.
Failing to do so can create two classes of engineers and this
could jeopardize the adoption. However, some engineers
are mainly producing models while others are mostly
consuming them for some activity. Therefore, understanding
how the tools and models are going to be used can lower
acquisition costs.
BP08: Tool acquisition is very costly therefore should be
thoroughly planned.
“Develop an adoption strategy” (P8)
“Development of a holistic approach/methodology” (P1)
Defining well and in advance the new process is highly
recommended. Planning should be done considering full
implementation of the processes and tools. This measure
avoids realizing at a later point in time that the decisions
taken when planning the MBSE adoption (tools to be
acquired, processes to adopt, training contents) are not fitted
for further intended process maturity. The consequences of
this mis-happen can be manifold: from extra effort to work
out tool inter-interoperability, until costs with redundant
tool acquirement and waste of time with training.
BP09: Have the new MBSE processes well documented
so you better understand what tool you will need.
“If the standard process is well documented, the MBE
implementation will work easier.” (P1)
Organizations that have their current development process
described and documented as it is performed have less
trouble when implementing MBSE. The tasks and artifacts
to be replaced can be straightforwardly identified and
consequences of the change are easily identified.
“First the process was set, then the tool decision” (P7)
In MBSE, tool and process are highly intertwined. It is
not uncommon that tool manufacturers provide also the
process work-flow specially designed to be used with their
tool. Thus, tool selection is also a contributing factor when
choosing which MBSE process to adopt.
“Tool support for automation of work” (P12)
Automation of work is one of the best features provided
by MBSE. The formality of models allow tools to
automatically verify many properties that otherwise would
require intensive human labor. While some capabilities
are indispensable, others might be just nice to have. By
understanding which features bring the best benefits to the
organization (now and in the future) a better cost benefit
51%
54%
57%
68%
43%
32%
24%
22%
3%
14%
16%
11%
0% 20% 40% 60% 80% 100%
BP09: Have the new MBSE processes well
documented so you better understand…
BP08: Tool acquisition is very costly
therefore should be thoroughly planned.
BP07: All engineers should have access to
the tools.
BP06: Tools with open interfaces and
homogeneous work-flow are preferred.
Strongly Disagree Disagree Agree Strongly Agree
Fig. 5. Questionnaire results for Tools and process group.8
relation can be achieved when deciding for tool adoption.
Related work. In the Jet Propulsion Laboratory of
NASA [16], MBSE teams were organized in a 3-tier fashion,
with a small set of core modelers, a larger set of modeling-
savvy SEs, within larger set of project personnel. In this sort
of team arrangement, everyone needs some sort of training,
but not to the same depth. The authors of [2] cite three
points that must be taken into account: (1) the existing
process should be well documented and should cover the
whole life-cycle otherwise is quite complicated to understand
what needs to be adapted, (2) the MBSE model management
processes (i.e., creation, update, and maintenance of models)
should be defined thereto derive engineering artifacts, (3)
investment in full-scale MBSE tool (i.e., not RE management
tools) that are accessible to all team members is necessary.
Whittle et al. [17] stress the importance of social and
organizational factors for the selection and development
of tools. They found in their study that it is better to
“Match tools to people, not the other way around” and
to “more focus on processes, less on tools”. The provided
answers in our interviews reflect the opinion that exchange
of information (i.e., models) between tools is a problem
that must be addressed by proper import/export mechanisms
and open interfaces. In a previous study, we have seen
that tool incompatibility is one of the major forces that
hinder companies to adopt MBSE solutions [3]. Research
has developed alternatives that may fit the MBSE paradigm
better. Seamless MBSE may be supported by multiple tools
manipulating models in a common model repository instead
of importing/exporting models (cf. projectional editors [18]).
D. Knowledge building
Training can be carried out in many ways. In this
subsection we will present different views and strategies
which sometimes can also be contradictory. This does
not invalidate them as the reader should realize that the
context of the organization influences the best approach.
The questionnaire answers had a good agreement rate with
the best practices, the smallest being 78% rate of ’Agree’.
The exception is BP12 which received 22% ‘Disagree’s.
(cf. Figure 6).
8BP07 and BP09 received 3% of ‘Strongly Disagree’ each.
9BP13 received 3% of ‘Strongly Disagree’.
68%
56%
51%
41%
24%
22%
43%
49%
5%
22%
5%
11%
0% 20% 40% 60% 80% 100%
BP13: There should be a planned form of
later evaluation to fill eventual gaps.
BP12: Many strategies can be used to build
knowledge of an organization, the context
should be taken into consideration.
BP11: Using examples that are familiar to
the domain of the organization eases the
understanding. Model some existing…
BP10: All engineers should get, at least,
basic training in MBSE.
Strongly Disagree Disagree Agree Strongly Agree
Fig. 6. Questionnaire results for Knowledge building group.9
BP10: All engineers should get, at least, basic training
in MBSE.
“Training, training, training!” (P1)
“Basic training for all users of MBSE” (P6)
“Everyone who uses MBSE should be trained in the
methodology” (P8)
“Broad basic training of all employees - Everyone should
have the same understanding” (P9)
Basic training should be provided to all employees.
Nevertheless, the required knowledge will vary among team
members (i.e., not everyone will require the same type of
knowledge or in the same depth). For instance, developing
complete models require much deeper knowledge then only
understanding them.
BP11: Using examples that are familiar to the domain
of the organization eases the understanding. Model some
existing artifacts for using as examples.
“Examples from similar industry help” (P9)
“Catching up with experiences / feedback from the net-
work (from the same industry)” (P13)
“[Training team] has modeled examples of clients and
then presented them [sic] to the client (we’ll show you how
we do it)” (P2)
The overall understanding can be enhanced by modeling
some requirements that are familiar to the developers. The
effort required to understand what is being modeled (i.e.,
domain) is diminished thus trainees can concentrate solely
in the modeling concepts. If an organization has no prior
knowledge in MBSE or modeling in general, it helps when
some experts model small parts of existing systems from
the organization as examples to transfer and demonstrate
the idea of MBSE.
BP12: Many strategies can be used to build knowledge
of an organization, the context should be taken into
consideration.
“F2F, except perhaps for basics” (P6)
“Face-to-Face Seminar with exercises” (P7)
“Deeper training of individual employee, who then pass
on their knowledge in the team” (P9)
Depending on the dynamics of the team, it is possible
to concentrate the training on a small group of participants
and later those people will be responsible to disseminate
the knowledge in the bigger group. This strategy fits very
well when the piloting is performed also with a small part
of the team. Other interviewee said that training should be
done with face-to-face seminars followed by exercises. As
we can see both approaches seem to work therefore the
context should drive such type of decision.
“Per tool training e.g. Doors10” (P7)
“Model should be reference. To do this, connect other
tools (e.g., importing Doors10 for initial filling, then only
update exports back to Doors, generated code frames force
interface fidelity)” (P5)
Tool and process are very intertwined in MBSE thus
training should mind the tools that will be used. The
training should have emphasis in the model, showing
there should not be co-existing two artifacts with the
same purpose, namely a document-based version and a
model-based version. The verbatim above gives the example
of a requirements modeling tool (Doors).
“Strategic cooperation, introductory consulting [...]
Coaching during the introduction has proven very success-
ful” (P1)
Hiring external consultancy to provide training guarantees
a minimal homogeneous level of knowledge throughout
all employees. Experienced consultants are very efficient
and can transmit the knowledge very efficiently. However
one should have in mind that participant P1 works with
technical sales.
BP13: There should be a planned form of later evaluation
to fill eventual gaps.
“Continuous explanation of the methodology” (P1)
“Accompanying technical monitoring of work results; is
not happening systematically.” (P5)
Training is an ongoing process, which includes returning
to training concepts at a further point in time and assessing
the engineers’ knowledge (e.g., through the quality of the
artifacts being produced). This is necessary to be put into
practice and is not happening systematically.
Related work. The training activities should use real
examples since they are much more effective at conveying
understanding and building support [16]. According to [2],
the organization should nurture a cadre of trained systems
engineers with at least moderate skill in employing
MBSE tools and techniques, and whose MBSE roles are
clearly delineated from the more traditional roles. For the
engineering staff a basic level of training in the MBSE
processes (so that they understand the value of the models
and what to expect from the systems engineers) and in
how to read MBSE artifacts (so that they can interpret
information provided from the MBSE processes). Although
the awareness towards training, 62% of “MBSE active
users” surveyed by [19] never received official training from
their organization and just 2% had a complete technical
course sponsored by their organization. In line with the
reported best practices, several works exist that report on
efficient transfer of MBSE methodologies by remodeling
10https://www.ibm.com/us-en/marketplace/rational-doors
30%
32%
22%
68%
59%
38%
49%
58%
32%
32%
30%
14%
19%
8%
0% 20% 40% 60% 80% 100%
BP18: Management should unify all
employees towards adoption.
BP17: If you have good engineers let them
do the work for you, it is cheaper and they…
BP16: Bring everyone to adoption (i.e.,
avoid creating castes).
BP15: Have technically prepared people to
support your engineers (i.e., not sales…
BP14: Make the advantages of MBSE clear.
Strongly Disagree Disagree Agree Strongly Agree
Fig. 7. Questionnaire results for Management group.11
parts of existing systems from companies (e.g., [20]). Tool
vendors usually provide training for the methodology and
work-flow to be used with their own products [21].
E. Management
Although current state of the practice has achieved some
degree of automatization in systems engineering, its tasks
are still human-intensive. Thus, introducing process change
in an already running and established organization is not
frictionless [22]. In an ideal world everyone would jump
in the adoption boat without hesitating and fully motivated,
the reality says that management should care for the team
vibe towards the adoption. Micromanagement is required to
tackle hurdles and psychological barriers, specially when it
comes to such a complex change as MBSE adoption [3].
This section is about strategies to manage the overall mood
and expectations of the ones affected by MBSE adoption.
The questionnaire results showed that BP16, B17, and
BP18 received many disagreement votes. Although most of
the respondents still agreed with them (i.e., at least 68%
of Agreement). One hypothesis is that they are context
dependent. In the other hand, BP14 and BP15 got almost
no disagreeing votes. (cf. Figure 7)
BP14: Make the advantages of MBSE clear.
“Advantages of simulability were made visible” (P11)
“Good examples show: project for new methods tools,
then show that it works (utility and acceptance).” (P13)
“Representation as a means of unique selling proposition.
Works, but each developer must be convinced individually.”
(P10)
The way people perceive the MBSE introduction can
become an obstacle, thus it is very important to manage the
mood of those involved, especially in the very beginning,
when everything is experimental. The advantages of the
approach should be made clear, also emphasizing benefits
for the employees. By doing so, organizations can foster
engineers acceptance and collaboration towards MBSE
adoption.
“Every few years new product generation, e.g. because of
Platform changes through technical advances (multicore ...)
or new system approaches that affect many functions. Then
there is a break in the system concepts, many have to be
11BP17 and BP18 received 5% and 2% respectively of ‘Strongly Dis-
agree’ each.
taken in hand, because you can introduce new methods.”
(P10)
As the technology evolves, new paradigms need to be
adopted to keep up with the market. Pairing such changes
with MBSE adoption improves its acceptance by surfing on
the already needed change atmosphere. However, waiting
for a paradigm shift may take a while, thus, this is very
specific strategy and is not fit for all situations.
BP15: Have technically prepared people to support your
engineers (i.e., not sales personnel).
“Tool sellers sometimes only send sales personnel, can
not answer technical questions, does not make a good
impression.” (P13)
The engineers will have very specific and technical
doubts about the modeling environment. Therefore, having
prepared people to support them is fundamental. Sales
personnel are usually not enough prepared for this task and
failing to do so might raise skepticism towards the adoption.
BP16: Bring everyone to adoption (i.e., avoid creating
castes).
“Avoid living apart, everyone has to go!!!” (P9)
Other best practices enforced that all employees should
receive access to the tools (BP07) as well as basic training
(BP10). The problem of not doing so is the creation of two
classes of engineers (i.e., those working with MBSE and
those that aren’t) and this can jeopardize their motivation.
Albeit, it is not uncommon teams working in different
technologies in the same organization. Another exception
is when the strategy is to train few engineers which will
pass the knowledge to the others. This point should be
considered with reservations.
BP17: If you have good engineers let them do the work
for you, it is cheaper, and they will engage more (i.e.,
empowering).
“Information model developed itself” (P7)
“[The knowledge was created by the] developers available
or self-built (i.e. only UML and the DSLs)” (P4)
Some organizations had their developers learning about
models on their own (i.e, the knowledge was organically
developed). That is not so surprising since in technological
areas engineers need to be learning new technologies all
the time.
BP18: Management should unify all employees towards
adoption.
“Guidance from management important to bring everyone
on the same platform” (P6)
Implementing MBSE involves engineers from different
phases of the software development and from different
fields of engineering. These professionals are usually not
working together in an everyday basis. Filling this gap
is the duty of management, thus uniting the whole team
towards MBSE adoption.
Related work. In a previous work [3], we identified forces
that work towards hindering or fostering the adoption of
MBSE and their origin. Hindering forces can be classified
64%
59%
51%
41%
36%
0% 10% 20% 30% 40% 50% 60% 70%
BP14: Make the advantages of MBSE clear
to the users.
BP05: Start small in terms of project and
team size in order to acquire some
experience
BP02: The pilot project should create real
value for the organization (i.e., no didactic
project).
BP03: The pilot project should have enough
budget and time allocated to bear the
overhead of adoption.
BP10: All engineers should get, at least,
basic training in MBSE.
Fig. 8. Top-5 most important best practices given by the percentage of
respondents that have rated them as “most important”.
as either Inertia or Anxiety. The former is triggered by the
feeling that the current solution is “good enough” and habits
that keep people from trying out something new. The latter
is triggered by fears that MBSE adoption will not pay-off,
mainly caused by uncertainties and perception flaws [3].
Motamedian [19] had in his survey questioned subjects about
barriers using MBSE. The second and third reasons most
voted were “the lack of perceived value of MBSE” and
“resistance to change”, therefore the need to micromanage
the engineers expectations and motivation toward the process
of adoption. Some ideas are just normal process changing
measures with MBSE flavor. This was something also found
by Hutchinson et al. [4] Nevertheless, the emphasis and
alignment with other phenomena increase their relevance
thus are worth mentioning. The challenges that arise [3] are
due to the nature of the current context, the shift to MBSE
(e.g., compile development skills vs abstraction skills), and
the human-in-the-loop.
F. Most Important Best Practices
Besides validating our interview findings, we used the
questionnaire to discover which best practices are considered
the most important. For this means, we asked the respon-
dents to select among all BPs the five most important when
adopting MBSE. The results can be seen in Fig. 8. From the
5 most important, three were from the Piloting group, one
from the Knowledge building group and the most voted is
from the Management group. The BPs are discussed in the
following under this light:
BP14 The most important BP is related to increasing the
engineers’ motivation towards MBSE adoption by
making them understand its benefits. Engineers are
less likely to withstand the hurdles of adoption
if they cannot perceive its benefits [3]. Systems
Engineering is a human-intensive activity thus the
collaboration of the engineers is crucial for adop-
tion success.
BP05 is about using a small setting to understand the best
way to introduce MBSE considering the organiza-
tion context. Through experimenting, managers can
understand which tools, languages, and styles are
best fitted for the organization and its respective
domains and current processes.
BP02 is about making the adoption efforts meaningful
and relevant right from the beginning. By working
into something that will be used in production
setting, the employees have to learn and employ
MBSE. If it is something that is just for learning,
there are no consequences if the project is incom-
plete or not well done thus making the learning also
incomplete. It also gives room for procrastination
(i.e., learning later when it is necessary).
BP03 This BP relates to the effort required by the engi-
neers to develop their MBSE skills. The project
selected to pilot the adoption should have its
budget planned to cover the learning curve costs
as well as the delivery of its artifacts should be
planned accordingly. Time-critical projects should
be avoided. If the project selected is not fit, the
engineers will drop the MBSE techniques in favor
of already established development methods in
order to achieve celerity gains and meet deadlines.
The adoption is very likely to fail.
BP10 In a system project, different skill sets are mas-
tered by different professionals which specialize
in specific parts of the life-cycle (e.g., develop-
ment, testing, design). Despite their skill focus, all
professionals have an overall system development
knowledge which helps them to understand the big
picture. This is similar to what should happen with
MBSE. Although not all engineers require deep
modeling skills, everyone should be able to, at
least, read and understand the models.
As for the least important best practices, BP04: No transla-
tion of old artifacts except for reusable artifacts was not
selected as “most important” by any of the respondents.
BP12: Many strategies can be used to build knowledge of an
organization, the context should be taken into consideration
was selected only once.
IV. RELATED WORK
In this section, we list and comment on similar studies.
The intersections with each of our findings are discussed in
detail in Section III.
Motamedian [19] reports on an on-line survey among
MBSE practitioners. The author’s goals were: (1) to high-
light the position of MBSE in real projects, (2) to assess
the popularity rate of MBSE concept among engineers, and
(3) the usage besides the advantages (i.e., barriers and con-
cerns of using “modeling language” and “modeling tools”)
in MBSE efforts among various industries. In an MBSE
workshop, the lessons learned while implementing MBSE
at the Jet Propulsion Laboratory of the American National
Space Agency (NASA) and the Mars2020 project [16] were
presented. Carroll and Marlins [2] describe the benefits of
MBSE adoption extracted from a series of case studies.
In the end of the study, the authors present a list of
implementation-lessons drawn from the findings. For them,
the cultural changes necessary to implement an MBSE
approach successfully (roles, rewards, behavior, and support
at all levels) were described as the most difficult challenges
to overcome. In a previous study [3], we identified forces
that foster or hinder MBSE adoption in practice. The focus
TABLE III
HUTCHINSON’SRESPONSES AND RELATED BEST PRACTICES
Response Best Practice
Adaptive BP14, BP16, BP17
Business led -
Committed BP03, BP07, BP10, BP18
Iterative BP05
Progressive BP05
was on listing characteristics without analyzing possible
solutions.
A. Hutchinson et al.
Hutchinson et al. published a number of papers that focus
on adoption of Model Driven Engineering (MDE) in the
context of software development [4], [17], [23]–[25]. Their
research comprise case studies built around semi-structured
interviews and on-line surveys devised to gather information
about MDE usage through closed questions (e.g., diagrams
used for modeling, modeling languages, MDE purpose of
use, etc). In [4], their findings are summarized into 10
dimensions of organizational attitude to MDE adoption, half
being helpful responses and the other half being unhelpful.
The responses are described in a very broad and general
manner. Table III provides the relation between their helpful
responses and the best practices describe by us. In [24], [25],
the authors state that MDE should be tried on projects that
‘can not fail’. We also reported similar finding as presented
by BP02. In [17], they present a taxonomy of MDE tool
related issues; ‘Chaining tools together’ and ‘Flexibility of
tools’ are covered by BP06, ‘Sustainability of tools over the
long term’ and ‘How to select tools’ are related to BP08.
Albeit the intersection of our findings with the work of the
aforementioned authors, our contribution goes further:
•Their focus was MDE (a software development method-
ology which encompasses models and code generation)
whilst our focus is on MBSE (a systems engineering
methodology devised for covering mechatronic, elec-
tronic and software parts). MDE is a subset of MBSE
(i.e., MDE ⊆MBSE).
•Their case study findings are based on anecdotal ev-
idence, whilst we used questionnaires to empirically
validate and generalize our conclusions with practition-
ers in a triangulation [6] fashion (i.e., the questionnaire
was built based on the conclusions we derived from the
interviews (cf. Section II)).
•Their findings are laid out within discussions of case
studies whilst we provide a crisp list of best practices
and a prioritized list of the five most important ones,
aiming to help practitioners to know where to focus.
V. CONCLUSIONS AND FUTURE WORK
The complexity of MBSE and its pervasiveness creates
challenges that when not properly addressed can jeopardize
its implementation in an organization. In this work, we
identified best practices of MBSE adoption through a series
of interviews. These practices were then classified in four
big groups: piloting, knowledge building, tools and process,
and management. Further, they were discussed, compared
with related work, and summarized at the end of each
group subsection. These summaries were used to build a
questionnaire in which each respondent could express their
level of agreement with the best practice.
Adopting MBSE is no trivial task. However, companies
are gaining experience in MBSE adoption and there are
success stories as well [26]. There is no need to reinvent
the wheel, thus learning from the experience of others and
applying proven best practices can save effort and enhance
the chances of success. In the long run, benefits outweigh
the costs and hurdles, however, it is necessary to identify
success factors and share best practices enabling efficient
and effective MBSE adoption in industry.
As future work, we could investigate the Best Prac-
tices that received many disagreement votes to understand
whether context plays a role in such phenomenon.
ACKNOWLEDGMENT
This work was partly funded by the German Federal Min-
istry of Education and Research (BMBF), grant “SPEDiT,
01IS15058”.
REFERENCES
[1] INCOSE, “Systems engineering vision 2020,” 2007.
[2] E. R. Carroll and R. J. Malins, “Systematic literature review: How is
model-based systems engineering justified?” 2016.
[3] A. Vogelsang, T. Amorim, F. Pudlitz, P. Gersing, and J. Philipps,
Should I Stay or Should I Go? On Forces that Drive and Prevent
MBSE Adoption in the Embedded Systems Industry. Cham: Springer
International Publishing, 2017, pp. 182–198. [Online]. Available:
https://doi.org/10.1007/978-3-319-69926-4\14
[4] J. Hutchinson, J. Whittle, and M. Rouncefield, “Model-driven engi-
neering practices in industry: Social, organizational and managerial
factors that lead to success or failure,” Science of Computer Pro-
gramming, vol. 89, pp. 144 – 161, 2014, special issue on Success
Stories in Model Driven Engineering.
[5] V. R. Basili, “Software modeling and measurement: the
goal/question/metric paradigm,” Tech. Rep., 1992.
[6] N. Denzin, Sociological Methods: A Sourcebook, ser. Methodological
perspectives. Aldine Transaction, 2006.
[7] I. Copi, Essentials of Logic. Pearson Prentice Hall, 2003.
[8] P. Shields and N. Rangarjan, A Playbook for Research Methods: Inte-
grating Conceptual Frameworks and Project Management. Stillwater,
OK: New Forums, 2013.
[9] A. Dresch, D. P. Lacerda, and J. A. V. Antunes, Design Science
Research. Springer International Publishing, 2015.
[10] P. Runeson and M. H¨
ost, “Guidelines for conducting and reporting
case study research in software engineering,” Empirical Software
Engineering, vol. 14, no. 2, 2008.
[11] W. L. Neuman, Social Research Methods: Qualitative and Quantita-
tive Approaches, 7th ed. Alpha Books, 2010.
[12] A. Bryman, Social research methods. Oxford university press, 2015.
[13] C. Weston, T. Gandell, J. Beauchamp, L. McAlpine, C. Wiseman,
and C. Beauchamp, “Analyzing interview data: The development and
evolution of a coding system,” Qualitative Sociology, vol. 24, no. 3,
2001.
[14] R. Likert, “A technique for the measurement of attitudes.” Archives
of Psychology, vol. 22, no. 140, pp. 1–55, 1932.
[15] J. A. Maxwell, Qualitative research design: An interactive approach.
Sage publications, 2012, vol. 41.
[16] E. Fosse and T. Bayer, “Model-based systems engineering mbse
101,” 2016. [Online]. Available: http://www.omgwiki.org/MBSE/lib/
exe/fetch.php?media=mbse:2016\iw-mbse\101.pdf
[17] J. Whittle, J. E. Hutchinson, M. Rouncefield, H. Burden, and R. Hel-
dal, “A taxonomy of tool-related issues affecting the adoption of
model-driven engineering,” Software and System Modeling, vol. 16,
no. 2, 2017.
[18] M. Voelter, J. Siegmund, T. Berger, and B. Kolb, “Towards user-
friendly projectional editors,” in Software Language Engineering,
B. Combemale, D. J. Pearce, O. Barais, and J. J. Vinju, Eds. Cham:
Springer International Publishing, 2014, pp. 41–61.
[19] B. Motamedian, “MBSE applicability analysis,” International Journal
of Scientific and Engineering Research, 2013.
[20] W. B¨
ohm, M. Junker, A. Vogelsang, S. Teufl, R. Pinger, and K. Rahn,
“A formal systems engineering approach in practice: An experience
report,” in 1st International Workshop on Software Engineering Re-
search and Industrial Practices (SER&IPs). New York, NY, USA:
ACM, 2014, pp. 34–41.
[21] H.-P. Hoffmann, “Deploying model-based systems engineering with
ibm ; rational; solutions for systems and software engineering,” in
2012 IEEE/AIAA 31st Digital Avionics Systems Conference (DASC),
Oct 2012, pp. 1–8.
[22] D. R. Conner, Managing at the Speed of Change. Random House,
1993.
[23] J. Hutchinson, M. Rouncefield, and J. Whittle, “Model-driven engi-
neering practices in industry,” in Proceedings of the 33rd International
Conference on Software Engineering, ser. ICSE ’11. New York, NY,
USA: ACM, 2011, pp. 633–642.
[24] J. Hutchinson, J. Whittle, M. Rouncefield, and S. Kristoffersen,
“Empirical assessment of mde in industry,” in 33rd International
Conference on Software Engineering (ICSE), May 2011, pp. 471–480.
[25] J. Whittle, J. E. Hutchinson, and M. Rouncefield, “The state of
practice in model-driven engineering,” IEEE Software, vol. 31, no. 3,
pp. 79–85, May 2014.
[26] D. D. Ruscio, R. F. Paige, and A. Pierantonio, “Guest editorial to the
special issue on success stories in model driven engineering,” Science
of Computer Programming, vol. 89, pp. 69 – 70, 2014.
