Analysis of Fuel Cells Utilizing
Mixed Reality and IoT Achievements
Burkhard Hoppenstedt1, Michael Schmid2, Klaus Kammerer1, Joachim
Scholta2, Manfred Reichert1, and R¨udiger Pryss1
1Institute of Databases and Information Systems, Ulm University, Ulm, Germany
2Zentrum f¨ur Sonnenenergie- und Wasserstoff-Forschung Baden-W¨urttemberg, Ulm,
Germany
Abstract. Recent advances in the development of smart glasses enable
new interaction patterns in an industrial context. In the field of Mixed
Reality, in which the real world and virtual objects fuse, new develop-
ments allow for advanced procedures of condition monitoring. Hereby,
the smart glasses serve as a mobile display and inspection station. In
this work, we focus on the applicability of Mixed Reality to monitor
data of the spatially resolved current density distribution of a fuel cell.
To be more specific, we implemented an IoT approach based on the
Message Queuing Telemetry Transport protocol (MQTT) to enable the
aforementioned monitoring. The realized solution, in turn, provides a live
monitoring as well as an overview feature.
Keywords: Fuel Cells ·Mixed Reality ·IoT ·MQTT
1 Introduction
In the context of the industrial internet of things (IIoT), also denoted as In-
dustry 4.0 [18], the collection of sensor values becomes more and more crucial.
These values are then used, e.g., for condition monitoring [23], process control
[11], or advanced analytics (e.g., Predictive Maintenance [14]). The overall goal
of a production setting connected through sensors is to increase the production
efficiency by a) reducing downtimes through predictive methods, b) increasing
the production transparency to discover bottlenecks, and c) enabling data-driven
approaches for a self-diagnostics plant. Hereby, machine communication proto-
cols, such as Open Platform Communications Unified Architecture (OPC UA)
[13] or the Message Queuing Telemetry Transport protocol (MQTT) [16], are
an essential part to exchange data in the needed distributed architectures [4].
These protocols implement features to ensure the guaranteed delivery of mes-
sages and required encryption needs. In this work, an IoT approach based on
(1) the MQTT communication protocol and (2) the Microsoft HoloLens smart
glass was realized to test its feasibility for the monitoring of current density
distribution data of a fuel cell.
2 B. Hoppenstedt et al.
Research into alternative energy sources is particularly important nowadays
as the impact of greenhouse gases on the environment through the use of fossil
fuels continues to increase as these resources become more and more scarce
[3]. One way to overcome these problems could be the use of fuel cells and
the expansion of the hydrogen infrastructure. Such energy conversion devices
generate electricity using hydrogen and oxygen in an electrochemical process for
which water is the only remaining waste product [2]. Therefore, in recent years,
a variety of scientific research has been conducted to optimize fuel cells and
minimize their manufacturing costs [22].
In our use case, the fuel cell represents a machine that delivers sensor values,
whereas a HoloLens is the monitoring application. We connect these two devices
via the MQTT protocol for a quick and trustworthy connection. The connection
allows to supervise the progress of the fuel cell’s sensor values as well as to
automatically generated alarms, which, in turn, can be sent to various recipients
(e.g., process control staff).
The remainder of the paper is structured as follows: in Section 2 related work
is discussed, while Section 3 introduces the backgrounds on fuel cells, Mixed Real-
ity, and the MQTT protocol. In Section 4, the developed prototype is presented,
in which the data set, the Graphical User Interface (GUI), and the backend
system are presented. Threats to validity are presented in Section 5, whereas
Section 6 concludes the paper with a summary and an outlook.
2 Related Work
The first part of the related work refers to fuel cells. According to the United
States Department of Energy (DOE) [7], fuel cells with polymer electrolyte mem-
branes (PEMs) have been developed for use in automobiles since the late 1980’s
and steady progress has been made to date. Fundamental studies of electro-
chemical properties are particularly important for improving PEM fuel cells as
they can generate current density distribution inhomogeneities due to different
reactions and activities in the active cell region. These are also influenced by
parameters such as temperature and humidity of the membrane and have a fun-
damental influence on the life cycle and performance of a fuel cell. By visualizing
the current density distribution within the fuel cell as shown in Fig. 1, corre-
sponding information can be obtained [8]. Concerning the second part of related
work, augmented reality is used in various use cases to monitor aspects of the real
world. In [10], 3D models are compared to real world objections for the purpose
of construction supervision. Wireless sensor networks, in turn, are monitored
by [9] using an augmented reality interface. The HoloLens, which represents a
smart glass of the category Mixed Reality, is often utilized in the medical con-
text (cf. [17] or [15]). However, to the best of our knowledge, a combination of
technologies as shown in this work, has not been presented in other works so far.
Analysis of Fuel Cells Utilizing Mixed Reality and IoT Achievements 3
Fig. 1. Visualization of the current density distribution inside a fuel cell
3 Fundamentals
3.1 Fuel Cells
Hydrogen (H2) is introduced on the anode side and air containing oxygen (O2)
on the cathode side. At the anode, the molecular hydrogen is split into hydrogen
nuclei (H+), also called protons and electrons (e–), with the help of a catalyst.
The protons migrate through the electrolyte membrane - which is permeable
only to them - to the cathode side. The electrons travel from the anode through
an electrical conductor to the cathode. The resulting current flow, in turn, can
be exploited. On the cathode side, two electrons reduce oxygen which then com-
bines with two H+-ions to form water (H2O) [5], as shown in Equation 1. This
electrochemical process is schematically shown in Fig. 2.
2 H++ O –
2H2O (1)
3.2 Mixed Reality
The HoloLens is a device to realize mixed-reality applications. Mixed Reality is
known to have the highest intersection of reality and virtual environment of all
augmented reality approaches [20] due to a concept named spatial mapping. This
procedure creates a model of the environment in the augmented reality device.
Therefore, interactions of holograms and real-world objects become possible.
Mixed Reality is basically used to display 3D models (1) for which a real-world
model would be too large or small (e.g., in the domains automotive or architec-
ture), (2) in medical use cases (assistance during a surgery), or (3) in industrial
maintenance support setting. The HoloLens is equipped with various cameras
and sensors, such as a depth sensor, a RGB camera, and an ambient light sensor.
The holograms can be anchored to real-life objects, but infinite projections are
4 B. Hoppenstedt et al.
Anode Cathode
+
H⁺
H₂+
+
O₂
H₂O
2e
2e
Rload
Fig. 2. Schematic of the electrochemical reaction inside a fuel cell
not possible, neither to the distance nor to the proximity. With a weight of 579g,
the HoloLens should not be used for a long period of time due to an unnatural
head positioning. A HoloLens case study [25] found out that the heavy weight
of the device degrades the user’s comfort level. Finally, in an intensive use case,
the battery lasts for about 2.5 hours, which also inhibits a long-time usage.
3.3 Message Queuing Telemetry Transport Protocol
The Message Queuing Telemetry Transport protocol (MQTT) is a light-weight
machine to machine communication protocol. It uses a publish-subscribe [6] pat-
tern, including the use of topics. According to [16], publish/subscribe systems
are wide-spread in distributed computing. Hereby, a topic can be considered as
a black board for messages. Subscribers are informed about changes to these
topics and new messages (e.g., sensor values) can be pushed to these topics.
A distribution server, denoted as broker, is responsible to forward messages to
subscribed clients. MQTT offers a Quality of Service (QoS) level [19], for which
the delivery of a specific message is guaranteed at most once, at least once, or
exactly once. MQTT has, in contrast to OPC UA, no semantic structure and
can therefore transport any kind of message. All these mechanisms make MQTT
a suitable communication protocol for IoT use cases.
Analysis of Fuel Cells Utilizing Mixed Reality and IoT Achievements 5
4 Prototype
The realized prototype shall allow for the monitoring of the current density dis-
tribution of a fuel cell in Mixed Reality. More specifically, the Microsoft HoloLens
is used as MQTT client to display the current state of the fuel cell. The user, in
turn, shall be enabled to interact with the holographic visualization through the
MQTT interface. Following this, for example, the values of the fuel cell can be
monitored and evaluated in real time. As a first step of the developed prototype,
the fuel cell was digitized using a 3D modeling tool (i.e., Blender, see also Fig.
3). Hereby, the arrows are animated to indicate input and output of the fuel
cell gases. A cell grid represents all measuring points in the fuel cell. As a next
step, the blender model can be attached with interaction logic. Therefore, we
implemented a tap to place method, so that the model can be placed anywhere
in the real world.
Fig. 3. 3D rendering of the fuel cell end plates with space for the current density
distribution values for spatial visualization with the HoloLens device
Then, the values of the fuel cell are sent to the model for monitoring. We
implemented the use cases live monitoring and loading of a data set. The main
difference constitutes a possible replay and change of playing speed for the sec-
ond use case. The control for the replay is provided by MQTT. As we integrated
a MQTT client into the HoloLens application, MQTT can be used as a remote
control to set the current frame or frame rate. Using the replay mode, it is possi-
ble to get a quick overview of the temporal behavior by viewing the sensor values
in a time-lapse mode. In contrast, when using the live mode, it is possible to be
alerted via a sound or sending the alert to any IoT device that can implement
6 B. Hoppenstedt et al.
the MQTT protocol (e.g., smartphones, machines, or computers). The realized
architecture of the prototype is shown in Fig. 4
HoloLens
Broker
Topics
/framerate
/currentTimeStamp
/play&pause
/alert
/values
Iot Device
User
Fuel Cell
publish
interact
subscribe publish
subscribe
Fig. 4. Prototype Architecture
5 Threats to Validity
The following limitations need to be considered for the work at hand. First, the
weight of the HoloLens smart glass cannot be neglected. Intensive use might
cause headaches or dizziness. Second, the further connectivity of the HoloLens
might be a problem. The options for data science analyses in Mixed Reality are
not sufficiently evolved so far. However, the possibilities of data analytics in aug-
mented reality, denoted as immersive analytics, are more and more investigated
[12]. Third, a general problem of distributed systems is the network security [26]
and network stability. In our approach, we solely rely on the connectivity and
security features implemented by MQTT. Finally, new interaction methods, pro-
vided by the HoloLens, also result in new challenges. When using the HoloLens
via voice commands, it is essential to be aware of a user bias. For the voice com-
mands, studies have shown that speech recognition performs worse for women
compared to men (cf. [24], [21]). Despite these limitations, the strength of our
prototype was that we combined a fuel cell, an IoT protocol, and a contemporary
smart glass to an interactive visualization approach.
6 Summary and Outlook
We presented a prototype for the monitoring of the current density distribution
of a fuel cell in Mixed Reality. The architecture incorporates an IoT message
Analysis of Fuel Cells Utilizing Mixed Reality and IoT Achievements 7
protocol (MQTT) for a light-weight communication to any device that supports
this protocol. The Microsoft HoloLens, which represents a mixed-reality smart
glass, is used as MQTT client to display the current state of the fuel cell. The
user can interact with the holographic visualization using voice commands or by
sending commands via the MQTT interface. The values of the fuel cell can be
either inspected in real time or by using a preloaded data set. The latter offers
a quick inspection, e.g., by using a time-lapse function. Currently, the field of
view for the HoloLens is limited to a small window. However, upcoming types
of mixed-reality glasses will fix this limitation and offer new user interaction
patterns. Moreover, the analytic part of our approach is currently limited to
alerts. By including stream analytic approaches [1], we could provide a more
powerful online analytic tool. In the presented approach, solely the current den-
sity distribution was used. However, the spatially resolved visualization of the
temperature and humidity would be also promising targets. Moreover, other im-
portant values may be presented in an interactive head-up display in order to
keep an eye on the condition of the fuel cell in an even more efficient manner.
On top of this, it is conceivable to adapt the operating parameters by means of
gestures or voice controls. Altogether, this work has shown that Mixed Reality
can play an important role in different domains that are less considered so far.
7 Acknowledgements
The authors want to thank the German Federal Ministry for Economic Affairs
and Energy for funding part of the presented work within the project SoHMuS-
DaSS (FKZ: 03ET6057C).
References
1. Andrade, H.C., Gedik, B., Turaga, D.S.: Fundamentals of stream processing: ap-
plication design, systems, and analytics. Cambridge University Press (2014)
2. de Beer, C., Barendse, P.S., Pillay, P., Bullecks, B., Rengaswamy, R.: Classification
of High-Temperature PEM Fuel Cell Degradation Mechanisms Using Equivalent
Circuits. IEEE Transactions on Industrial Electronics 62(8), 5265–5274 (aug 2015)
3. Chandan, A., Hattenberger, M., El-kharouf, A., Du, S., Dhir, A., Self, V., Pollet,
B.G., Ingram, A., Bujalski, W.: High temperature (HT) polymer electrolyte mem-
brane fuel cells (PEMFC) A review. Journal of Power Sources 231, 264–278 (jun
2013)
4. Datta, S.K., Bonnet, C., Nikaein, N.: An iot gateway centric architecture to provide
novel m2m services. In: Internet of Things (WF-IoT), 2014 IEEE World Forum on.
pp. 514–519. IEEE (2014)
5. Ehret, O.: Wasserstoff und Brennstoffzellen: Antworten auf wichtige Fragen. NOW
GmbH - Nationale Organisation Wasserstoff- und Brennstoffzellentechnologie,
Berlin (2018)
6. Eugster, P.T., Felber, P.A., Guerraoui, R., Kermarrec, A.M.: The many faces of
publish/subscribe. ACM computing surveys 35(2), 114–131 (2003)
7. Fuel Cell Technologies Office: Multi-Year Research, Development, and Demonstra-
tion Plan. Tech. rep., U.S. Department of Energy (2012)
8 B. Hoppenstedt et al.
8. Geske, M., Heuer, M., Heideck, G., Styczynski, Z.A.: Current density distribution
mapping in PEM fuel cells as an instrument for operational measurements. Energies
3(4), 770–783 (2010)
9. Goldsmith, D., Liarokapis, F., Malone, G., Kemp, J.: Augmented reality environ-
mental monitoring using wireless sensor networks. In: 12th International Confer-
ence Information Visualisation. pp. 539–544. IEEE (2008)
10. Golparvar-Fard, M., Pe˜na-Mora, F., Savarese, S.: D4ar–a 4-dimensional augmented
reality model for automating construction progress monitoring data collection,
processing and communication. Journal of information technology in construction
14(13), 129–153 (2009)
11. Gonzaga, J., Meleiro, L.A.C., Kiang, C., Maciel Filho, R.: Ann-based soft-sensor for
real-time process monitoring and control of an industrial polymerization process.
Computers & chemical engineering 33(1), 43–49 (2009)
12. Gracia, A., Gonz´alez, S., Robles, V., Menasalvas, E., Von Landesberger, T.:
New insights into the suitability of the third dimension for visualizing multivari-
ate/multidimensional data: A study based on loss of quality quantification. Infor-
mation Visualization 15(1), 3–30 (2016)
13. Hannelius, T., Salmenpera, M., Kuikka, S.: Roadmap to adopting opc ua. In:
6th IEEE International Conference on Industrial Informatics. pp. 756–761. IEEE
(2008)
14. Hoppenstedt, B., Pryss, R., Stelzer, B., Meyer-Br¨otz, F., Kammerer, K., Treß,
A., Reichert, M.: Techniques and emerging trends for state of the art equipment
maintenance systemsa bibliometric analysis. Applied Sciences 8, 1–29 (2018)
15. Hoppenstedt, B., Schneider, C., Pryss, R., Schlee, W., Probst, T., Neff, P., Simoes,
J., Treß, A., Reichert, M.: Holoview: Exploring patient data in mixed reality. In:
TRI / TINNET Conference 2018 (2018)
16. Hunkeler, U., Truong, H.L., Stanford-Clark, A.: Mqtt-sa publish/subscribe proto-
col for wireless sensor networks. In: 3rd international conference on Communication
systems software and middleware and workshops. pp. 791–798. IEEE (2008)
17. Hurter, C., McDuff, D.: Cardiolens: remote physiological monitoring in a mixed
reality environment. In: ACM SIGGRAPH 2017 Emerging Technologies. p. 6. ACM
(2017)
18. Lasi, H., Fettke, P., Kemper, H.G., Feld, T., Hoffmann, M.: Industry 4.0. Business
& Information Systems Engineering 6(4), 239–242 (2014)
19. Lee, S., Kim, H., Hong, D.k., Ju, H.: Correlation analysis of mqtt loss and delay
according to qos level. In: International Conference on Information Networking.
pp. 714–717. IEEE (2013)
20. Milgram, P., Takemura, H., Utsumi, A., Kishino, F.: Augmented reality: A class of
displays on the reality-virtuality continuum. In: Telemanipulator and telepresence
technologies. vol. 2351, pp. 282–293. International Society for Optics and Photonics
(1995)
21. Nicol, A., Casey, C., MacFarlane, S.: Children are ready for speech technology-but
is the technology ready for them. Interaction Design and Children, Eindhoven, The
Netherlands (2002)
22. Orfanidi, A.: Electrocatalytic Investigation of High Temperature PEM Fuel Cells
Alin Orfanidi (December) (2014)
23. Rangwala, S., Dornfeld, D.: Sensor integration using neural networks for intelligent
tool condition monitoring. Journal of Engineering for Industry 112(3), 219–228
(1990)
Analysis of Fuel Cells Utilizing Mixed Reality and IoT Achievements 9
24. Rodger, J.A., Pendharkar, P.C.: A field study of the impact of gender and user’s
technical experience on the performance of voice-activated medical tracking appli-
cation. International Journal of Human-Computer Studies 60(5-6), 529–544 (2004)
25. Zhang, L., Dong, H., El Saddik, A.: Towards a qoe model to evaluate holographic
augmented reality devices: A hololens case study. IEEE MultiMedia (2018)
26. Zhang, Z.K., Cho, M.C.Y., Wang, C.W., Hsu, C.W., Chen, C.K., Shieh, S.: Iot se-
curity: ongoing challenges and research opportunities. In: 7th International Confer-
ence on Service-Oriented Computing and Applications. pp. 230–234. IEEE (2014)
