energies
Article
Illustrating the Benefits of Openness: A Large-Scale
Spatial Economic Dispatch Model Using the
Julia Language
Jens Weibezahn * and Mario Kendziorski
Workgroup for Infrastructure Policy (WIP), Technische Universität Berlin, H 33, Straße des 17. Juni 135,
*Correspondence: [email protected]; Tel.: +49-30-314-27500
Received: 19 December 2018; Accepted: 19 March 2019; Published: 25 March 2019
Abstract:
In this paper we introduce a five-fold approach to open science comprised of open data,
open-source software (that is, programming and modeling tools, model code, and numerical solvers),
as well as open-access dissemination. The advantages of open energy models are being discussed.
A fully open-source bottom-up electricity sector model with high spatial resolution using the Julia
programming environment is then being developed, describing source code and a data set for
Germany. This large-scale model of the electricity market includes both generation dispatch from
thermal and renewable sources in the spot market as well as the physical transmission network,
minimizing total system costs in a linear approach. It calculates the economic dispatch on an hourly
basis for a full year, taking into account demand, infeed from renewables, storage, and exchanges
with neighboring countries. Following the open approach, the model code and used data set are
fully publicly accessible and we use open-source solvers like ECOS and CLP. The model is then being
benchmarked regarding runtime of building and solving against a representation in GAMS as a
commercial algebraic modeling language and against Gurobi, CPLEX, and Mosek as commercial
solvers. With this paper we demonstrate in a proof-of-concept the power and abilities, as well as the
beauty of open-source modeling systems. This openness has the potential to increase the transparency
of policy advice and to empower stakeholders with fewer financial possibilities.
Dataset: 10.5281/zenodo.1044463
Keywords:
power systems modeling; open source; algebraic modeling language; optimization;
benchmarking; Julia; JuMP
MSC: 90-04; 90C05; 90C90; 91B26
JEL Classification: C61; D41; L94; O34; Q41
1. Introduction
In the wake of the strenuous efforts to reduce the effects of climate change, electricity systems
worldwide have undergone profound transformations over the last decades from mostly centralized
conventional power generation using carbon-intense fossil fuels towards more decentralized renewable
power plants. Nevertheless, the goals of climate protection demand for further action and massive
changes in the upcoming decades. In order to achieve a better comprehension of electricity systems,
assess and optimize operation and investment decisions, but also to generate insights for policy
making, electricity sector models are being used. These models are usually large-scale, complex
techno-economic models describing the behavior of an electricity system in operation. The rapid
Energies 2019,12, 1153; doi:10.3390/en12061153 www.mdpi.com/journal/energies
Energies 2019,12, 1153 2 of 21
change of the electricity sector, driven by vast extensions of renewable installations and an increase in
sector coupling with heat and transportation, make them even more relevant for a consistent energy
transition in the present and coming years.
Historically, most of these models have acted as proprietary black-box solutions, written in
commercial systems and operated by organizations without the opportunity for other researchers to
reproduce and validate results and for the public to fully understand and use these models, leading
to a lack of transparency in the modeling community. One example is the European Commission’s
strategic long-term vision for a climate neutral economy by the year 2050. The policy package laid out
here is based on insights gained using an energy sector model that cannot be directly reproduced since
neither the model source code nor the data sets have been published.
Against this backdrop, more and more voices are advocating for open source, open data, and open
access in energy system modeling [
1
]. Some initiatives already have published their models for an
open-source use (Examples are the energy modeling system OSeMOSYS [
2
] or the power system
analysis tool PyPSA [
3
] and the open energy modeling framework (oemof, www.oemof.org), both
written in Python. Those two models can be used fully open source, and oemof even provides a library
for output visualization.).
With this paper we are presenting a new tool set for electricity and energy system modeling:
the rather new programming language Julia, developed at MIT specifically for the needs of scientific
computing, in combination with its algebraic modeling library JuMP. In a benchmark study with a
proof-of-concept (PoC) for a fully ‘open’ electricity system model we are presenting a quantitative
comparison with regard to computation time of the new Julia/JuMP with the conventional proprietary
General Algebraic Modeling System (GAMS).
We argue that using an open-source language like Julia, the modeler’s efficiency and productivity
can even be enhanced, since the whole modeling workflow from data pre-processing to visualization
can be implemented within the same system at only very low start-up costs. Embedded into a
broader open concept, this would lead to an increase in transparency but also to a strengthening of the
modeling community.
On the other hand, our benchmark study also shows one deficit of open-source tools: for the
time being at least, very complex models are still dependent on proprietary software in the form of
the numerical solvers required since open-source alternatives can mostly not keep pace with their
commercial counterparts.
With this paper we also introduce Joulia.jl, an open-source package for large-scale spacial
economic dispatch problems written in Julia/JuMP, solely using open data and—where complexity
allows—making use of open-source numerical solvers.
The remainder of this paper is structured as follows: Section 2describes the benefits of open
science. Section 3provides a short introduction into the Julia programming language and the algebraic
modeling language JuMP. Section 4explains the model and gives an overview of the used input data.
In Section 5the implementation in Julia/JuMP and the results of the benchmark tests are discussed.
The paper then concludes with a summary and outlook in Section 6.
2. The Benefits of Openness
In their manifesto (www.openmod-initiative.org/manifesto), the Open Energy Modeling Initiative
(openmod) advocates for more openness in energy modeling:
«Energy models are widely used for policy advice and research. They serve to help answer
questions on energy policy, decarbonization, and transitions towards renewable energy
sources. [...] We believe that more openness in energy modeling increases transparency
and credibility, reduces wasteful double-work and improves overall quality. This allows
the community to advance the research frontier and gain the highest benefit from energy
modeling for society.»
Energies 2019,12, 1153 3 of 21
The Open Definition 2.1 (www.opendefinition.org) states: “Knowledge is open if anyone is free
to access, use, modify, and share it—subject, at most, to measures that preserve provenance and
openness.” More specifically, the openness of a modeling project can affect different dimensions. We
define five dimensions of openness that will be described in the following section: open data, an
open-source modeling language, open-source model code, open-source solvers, and finally open-access
publications. This follows recommendations by DeCarolis et al. [
4
] and Morrison [
5
]. These dimensions
can be aggregated to three major topics: the availability and usage as well as publication of input and
output data, the software part, and the scientific publications. See Figure 1for a schematic overview of
the workflow and the dimensions of openness in the energy modeling process.
input data
data processing
energy model
modeling tool
publica�on
dissemina�on
numerical solver
model formula�on output data
data processing
open source
open access
open data
Figure 1.
Schematic workflow and dimensions of openness in the energy modeling process. Source:
own depiction based on [6].
The difference between open-source software (OSS) and closed-source software (CSS) generally
lies in the availability of the source code to the general public. OSS, as promoted by the Open
Software Initiative (OSI) (www.opensource.org), comes with a license with minimal or no restrictions
on the (re-)distribution, use, and modification of the software. The Free Software Foundation (FSF)
(www.fsf.org) and the GNU Project promote a rather similar approach:
«The word “free” in our name does not refer to price; it refers to freedom. First, the freedom
to copy a program and redistribute it to your neighbors, so that they can use it as well as you.
Second, the freedom to change a program, so that you can control it instead of it controlling
you; for this, the source code must be made available to you.»(GNU’s Bulletin Volume 1
No. 1, 1986)
CSS or proprietary software, on the other hand, is always distributed under a very restrictive
license and as a ‘black box’ with no possibility to view the source code in order to determine
the functionality.
The advantages and disadvantages of OSS vs CSS can be described using the categories
customizability and control (is the software a ‘black box’ or can the user check what it does ‘under
the hood’?; can the user change the source code in order to adapt the software to her specific needs?),
security (can the software be corrupted by hackers?), reliability (does the software do what it promises?
does it come with a huge number of bugs?), and maintenance (will bugs be fixed with a short lead
time? will new features be implemented?).
Famous examples of OSS are Python as an established programming language, Julia as a rather
new programming language, and OSeMOSYS [
2
] as an energy sector model. Infamous examples of
CSS, on the other hand, are GAMS as an algebraic modeling language, and PRIMES [
7
] as an energy
sector model.
Energies 2019,12, 1153 4 of 21
2.1. Open Data
Energy sector models in general and electricity sector models in particular are mostly not rocket
science (To quote an expert in the field: “The whole world can be modeled as linear programs!”) but are
largely data-driven (hence: ‘large-scale models’). In the past, most of this data was not available to the
general public, hidden in commercial databases or not accessible at all due to trade secrets and matters
of ‘national security’. Over the last few years, stakeholders in the electricity sector started to open up
and publish data online, in most cases because of legislation obligating them to a certain transparency.
Nevertheless, while many of those data sources can now be openly viewed, it is—according to
copyright law and licensing—mostly not legally possible to use, process, and redistribute this data
why more and more initiatives are calling for improved legislation [
8
]. Nevertheless, there is and
always will be a certain portion of data that will not be available to the public.
Another aspect is the structure of publication and quality of the published data. Projects like
Open Power System Data (OPSD) (www.open-power-system-data.org) [
9
] try to tackle this issue by
providing Python scripts to download and pre-process commonly needed power system data for
modelers. This also increases the productivity of modelers since not everyone has to go through the
same tedious process of data collection and pre-processing again [1].
2.2. Open-Source Programming & Modeling Tool
An algebraic modeling language is a modeling tool to formulate an optimization (or simulation)
problem in a high-level language and then pass the generated matrix on to a so-called solver—an
independent software—for calculating the numerical solution to the problem rather than writing the
input directly in low-level code.
When it comes to the decision what modeling tool should be used for a project, there is a quasi
standard at least for the academic and industry energy community: GAMS, the General Algebraic
Modeling Language. Aside from this, AMPL can be used but there is also a whole range of viable
open-source alternatives with major advantages over their commercial competitors. One of them
is R, a language originally designed for statistical computing. A more general solution is Python
in combination with Pyomo as an optimization library, which could be considered to be the open
source standard. In this paper we propose the usage of the Julia Language in combination with the
optimization package JuMP. For details on Julia and JuMP see Section 3.
The major advantage of proprietary software in the context of the used modeling language is the
ease of use. Usually the software comes as an out-of-the-box solution with an IDE, ready to be used.
The software is being maintained on a regular basis and everything should work reliably including
the links to the solvers to be used. On the contrary, it is less customizable for example for the usage
of alternative solver packages. Only supported solvers can be used with the proprietary software.
Furthermore, established OSS solutions like Python with a long history and a substantial developer
community come with at least the same level of reliability.
Table 1gives an overview of the software considered in this paper and some of its characteristics.
2.3. Open-Source Model Formulation
The most important part of the process is the model formulation in the form of source code.
Assessment models should not be a ‘black box’, just delivering numbers as results that are used
for policy implications and that might on the way become perceived as facts by policymakers.
Bazilian et al. [10]
and Pfenninger et al. [
1
,
11
] all argue that energy scientist must show what happens
‘under the hood’ of their models.
This transparency is the only way other researchers, but also the general public, can replicate and
validate the results and fully understand and challenge the models in the peer-review of publications
but also in the context of policy advice. This is also the only way to fulfill the standards of open science.
Energies 2019,12, 1153 5 of 21
Furthermore, it increases the quality of models since developers are forced to decrease the number of
errors or at least errors can be found by others. The models therefore become more robust.
Table 1. Considered software packages and their characteristics.
Software Functionality Type Website
GAMS Algebraic modeling language (AML) CSS www.gams.com
AMPL Algebraic modeling language (AML) CSS www.ampl.com
Python Programming language OSS www.python.org
Julia Programming language OSS julialang.org
R Programming language OSS www.r-project.org
Pyomo Algebraic modeling library (Python-based) OSS www.pyomo.org
JuMP.jl Algebraic modeling library (Julia-based) OSS www.juliaopt.org
lpSolve Algebraic modeling library (R-based) OSS lpsolve.r-forge.r-project.org
CPLEX Solver CSS www.cplex.com
Gurobi Solver CSS www.gurobi.com
MOSEK Solver CSS www.mosek.com
CLP Solver OSS www.coin-or.org/Clp
GLPK Solver OSS www.gnu.org/software/glpk
ECOS Solver OSS www.embotech.com/ecos
Especially when it comes to the usage of model results as arguments for certain policy implications
or recommendations, the credibility and legitimacy of those results increases significantly if everyone
is able to check them and to see the underlying assumptions.
Last but not least, publishing models according to open standards grants access to anyone
and therefore also to stakeholders with less financial means like non-governmental organizations or
developing countries, enabling them to produce their own analyses. It also fosters the interoperability
of different models [12].
Pfenninger et al. [
13
] supply a guideline of strategies on how to open models up, while
Hülk et al. [14]
provide a transparency checklist for models. Several meta studies describe the current
questions and challenges of electricity and energy sector modeling [12,15,16].
2.4. Open-Source Numerical Solver
For numerically solving large-scale problems—like electricity sector models—commercial solvers
are usually the product of choice for most modelers. Commonly known solver packages are the
CPLEX Optimizer by IBM and Gurobi by Gurobi Optimization but also less well-known products like
MOSEK by Mosek ApS can be used. For academics at universities these products are usually free of
charge under academic licenses, while research institutes, government agencies, non-governmental
organizations, and commercial users must purchase commercial licenses.
Open-source solvers can—under certain circumstances—be an alternative. In this paper we are
benchmarking a number of open projects against the commercial ones. The advantages in those cases
are similar to the ones for the modeling tool: publicly available and therefore controllable (no ‘black
box’) source-code as well as cost savings for license fees. Standard open-source solvers for linear
programs are CLP by COIN-OR and GLPK by the GNU Project. Another promising product is ECOS
by embotech, a spin-off of ETH Zurich.
Since CLPEX and Gurobi are, by now, well established products with high license fees and
therefore bigger resources than open projects, their performance in solving problems is usually many
times better. Nevertheless we wanted to use open-source solvers as a proof-of-concept: it is possible
and—depending on the size of the project—worthwhile to cover the complete modeling workflow
with open solutions. This point is especially important for users or stakeholders with very little budget
like non-governmental organizations or even developing countries.
Loading more pages...