Citation: Pernat, N.; Gathof, A.K.;
Herrmann, J.; Seitz, B.; Buchholz, S.
Citizen Science Apps in a Higher
Education Botany Course: Data
Quality and Learning Effects.
Sustainability 2023,15, 12984.
https://doi.org/10.3390/
su151712984
Academic Editors: Kerstin Kremer
and Maria Peter
Received: 26 May 2023
Revised: 10 August 2023
Accepted: 11 August 2023
Published: 29 August 2023
Copyright: © 2023 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
sustainability
Article
Citizen Science Apps in a Higher Education Botany Course:
Data Quality and Learning Effects
Nadja Pernat 1,2,*, Anika Kristin Gathof 1, Johann Herrmann 3, Birgit Seitz 4and Sascha Buchholz 1,2
1Institute of Landscape Ecology, University of Münster, 48149 Münster, Germany;
agathof@uni-muenster.de (A.K.G.); saschabuchholz@uni-muenster.de (S.B.)
2Centre for Integrative Biodiversity Research and Applied Ecology (CIBRA), University of Münster,
48149 Münster, Germany
3Institute for Bee Protection, Julius Kühn Institute (JKI), Federal Research Centre for Cultivated Plants,
4Department of Ecology, Technische Universität Berlin, 12165 Berlin, Germany; [email protected]
*Correspondence: nadjapernat@uni-muenster.de
Abstract:
Although species identification apps are becoming increasingly popular in citizen science,
they are hardly used in university courses on biodiversity literacy. In this study, we investigated
whether the use of a plant identification app by students provides similar data quality to the use
of scientific keys and whether it improves the process of knowledge acquisition. To this end, dry
grassland plots were monitored in Berlin to record plant species diversity by two groups, Bachelor’s
and Master’s students, with different experience in plant identification. Both groups were asked
to survey the plots once in April and once in June, the first time with the app Pl@ntNet, and the
second time with scientific keys commonly used in teaching. To evaluate their performance and the
respective tools, the results were compared with those of experts from the same plots. The students
identified, on average, only half of the plants per plot and misidentified or failed to identify a high
proportion of species compared with the experts, regardless of the identification tool. In addition, the
number of plants identified that did not occur at all in the region or in the considered habitat was
alarmingly high. In view of the worldwide loss of species knowledge, it is becoming clear that apps
can trigger the study of a species group, but do not solve the fundamental problem of neglecting
biodiversity courses at universities.
Keywords:
species literacy; biodiversity literacy; citizen science; artificial intelligence; higher
education
1. Introduction
Citizen science (CS), recently also discussed under the name community science [
1
],
is the active involvement of the public in the whole or parts of the research process. The
bipartite promise of CS for science and society is that CS can both contribute to science,
for example, by providing valuable scientific data, and increase the knowledge and the
social capital of participants from the public. The scientific contribution of CS programs,
with data quality studies being in the foreground, has been the focus of the scientific
community for a long time. Due to the advent of smartphone and internet technologies [
2
],
especially in biodiversity-related programs, the amount of data collected by amateur
naturalists has rapidly increased and, with it, investigations on the usability of CS data.
Particularly in projects appropriate for the masses, like eBird, eButterfly, or Mückenatlas,
bias originates from the diverse recording behavior and level of identification skills of
the citizen scientists [
3
–
5
]. Over the last few years, strategies to mitigate bias and to
increase data quality have been developed. These strategies target either project design
and protocol [
6
–
8
] or statistical analysis [
9
–
12
], and are now widely used by the scientific
community, such that there is already talk of CS becoming mainstream [13].
Sustainability 2023,15, 12984. https://doi.org/10.3390/su151712984 https://www.mdpi.com/journal/sustainability
Sustainability 2023,15, 12984 2 of 15
The recognition of the direct dependence of CS biodiversity data on the observation
performance of the citizen scientists led to the research focus of the science of citizen science
slowly shifting from the data aspect to the participants—also in the light of the second part
of the CS promise of benefitting society. Expectations are high that public involvement
will result in better dialogue between the public and scientists, a research agenda that is
more relevant to the public, a better understanding of scientific methods and timeframes,
an increase in scientific literacy, and, with regard to biodiversity-related projects, raised
awareness of environmental issues or changes in attitudes or behavior [
14
]. In order to
investigate these expectations, studies have been conducted on participants of biodiversity-
related programs, focusing on their motivation [
15
–
17
], their behavior [
18
–
20
], and their
demographic background [21,22].
Of growing interest in recent years has also been the question of the educational
power of CS and whether or what people get out of their participation (e.g., an increase
in knowledge or skills). Most studies report gains in knowledge and skills related to the
project topics and, moreover, improved quality of life [
21
,
23
–
26
], but less so in relation to
the understanding of scientific processes [
27
]. However, most studies relate to informal
learning environments. The literature on the implementation of CS in formal education is
sparse; hence, little is known about the integration practices and beneficial outcomes of
applying CS in higher education.
A review by Vance-Chalcraft and colleagues [
28
] found, in 15 papers and 79 instructor
survey responses, that CS projects are mostly used in smaller classes (fewer than 30 students)
for collecting and submitting data for ecological and environmental studies. They reported
that students benefit from positive feelings toward being engaged in CS, improved scientific
practices, and increased knowledge and engagement. Paradise and Bartkovich [
29
] found
that the majority of students in an undergraduate entomology course perceived CS tools
as helpful for identifying insect specimens and that their comprehension and valuation of
biodiversity improved, as well as their species literacy.
The challenges recognized are the different levels of engagement with CS, logistical and
data quality problems, and difficulties when referencing the scientific literature and other
sources [
28
,
30
,
31
]. Interestingly, the COVID-19 pandemic facilitated or even accelerated the
inclusion of CS approaches and tools in many higher education courses to engage students
working from home [
28
]. For example, free apps were used in some courses to investigate
specific research questions [
32
–
34
], there was a shift toward CS methods and tools for
biodiversity monitoring and literacy courses [
35
,
36
], and with BotanizeR [
37
], even an R
package was developed to teach botany skills remotely.
The successful application of CS in biodiversity and species literacy classes is of par-
ticular interest, given the low knowledge about native species among students [
38
,
39
],
also in Germany, be they plants [
40
], vertebrates [
41
], or insects [
42
,
43
]. The same trend is
measurable among adults, who are becoming more and more getting disconnected from
nature [
44
], and lay experts, for example, hobby naturalists, who overage without a new
generation coming along [
45
]. At the same time, professional experts from institutional
science are becoming older and fewer [
46
,
47
], with the same problem of replacement by
younger researchers. This is an extremely negative development in the face of the biodiver-
sity crisis [
48
], in which taxonomic expertise is urgently needed to assess global biodiversity
data, which can only be sufficiently collected by involving the entire public [
49
]. Scientists
can only cover a certain number of places and only have the resources to do so. The idea of
increasing species literacy and, therefore, also of laying the foundation for civic engagement
through CS in higher education is highly relevant to a future sustainable society.
CS platforms and apps that support species observation and identification (e.g., iNatu-
ralist) have found their way into higher education in North American colleges and univer-
sities [
50
,
51
], because these easy-to-use apps are both adequate for first and advanced users
from the digital native generations [
31
]. In contrast, mastering the skills to identify species
with field guides and scientific literature is challenging and takes years, if not decades. The
process of using scientific identification keys and the terminology knowledge it requires
Sustainability 2023,15, 12984 3 of 15
can make frustrated students reluctant to enter the field of taxonomy in the first place [
52
].
To overcome this initial barrier, image-based species identification via deep learning for
plants, fungi, insects, and other taxa has constantly improved in accuracy and application
area [
53
–
55
]. Corresponding smartphone and web apps have become very practical for
beginners and advanced users, and are growing in number. However, a test with nine apps
for plant identification showed very different results in the performance of the respective
tool, so incorrect determinations can occur, especially with beginners, and thus lead to a
counterproductive learning effect [56].
It is slow for apps to find their way into university seminars and courses, and whether
they actually help to build species literacy in higher education—and provide useful data
in the process—has not yet been sufficiently investigated. In this comparative analysis,
Bachelor’s and Master’s students with different levels of experience and knowledge in
plant identification recorded plant species diversity in urban dry grassland sites. To assess
the students’ identification accuracy, their results were compared with those of expert
botanist collections for the same plots. The students worked alternating with both a plant-
specific identification app (Pl@ntNet) and standard identification keys, and before and after
classical botany identification courses. The study aims to answer the following questions:
1.
Does the use of a plant identification app by students provide useful data in comparison
with conventional species identification resources (field guides, scientific literature)?
2.
Does a plant identification app enhance the process of knowledge acquisition for
species identification?
2. Methodology
2.1. Study Context
The long tradition of the Technische Universität Berlin in plant ecology, especially in
the context of urban ecology, is reflected in the botany courses offered by the university.
Within two separate courses during the COVID-19 pandemic in 2021, two groups of
students from the study programs Ecology and Environmental Planning (Bachelor’s) and
Urban Ecosystem Sciences (Master’s conducted rounds of vegetation surveys as part of
their training in botany. As an innovative element, the Pl@ntNet app for plant identification
was employed for the first time in both of these courses for mapping flora in the field.
In order to evaluate its effectiveness, it was only applied in the first round of vegetation
surveys, and in the second round, the scientific keys by Jäger and Rothmaler [
57
,
58
] were
used—both classic field guides for teaching. To assess the results from the two survey
rounds and to compare the performance of both the identification tools and the differently
experienced student groups, their observations were compared with those of an expert
botanist. In the following, we provide background information on the tools used for plant
identification, the student participants and the corresponding university courses, the data
collection protocols of the students and the expert, and the data analysis methods.
2.1.1. Pl@ntNet App
One of the earliest and most widely used apps dedicated to plant species identification
is Pl@ntNet (https://plantnet.org/en/ (accessed on 20 April 2022)), referred to as PlantNet
or simply the App in the following. It was developed in France in 2009, and 320 million
identification requests have since been identified via the App. PlantNet is among the
most accomplished plant identification apps on the market [
56
] and performs especially
well when provided with more than one picture of the same plant—a feature that only
a few other apps contain. It is also free to use, even for non-registered users, and works
in two steps. First, one or more images of the same plant are uploaded, labeled leaf,
flower, etc., and the artificial intelligence provides a list of plant species candidates. Each
candidate comes with a score that can be interpreted, like confidence levels or probabilities.
According to the programmers of the App, the accuracy measured for the first five species
suggested for each image ranges from 89% to 63%, depending on the difficulty of the tested
datasets [
59
]. If the user is registered, in the second step, the picture can be saved with the
Sustainability 2023,15, 12984 4 of 15
selected species suggestion, which is then confirmed or substituted with the right species
name by the PlantNet community. In return, the images labeled with the correct species
name are regularly fed back into the training database to improve the algorithm.
2.1.2. Scientific Keys for Plant Determination
The “Exkursionsflora von Deutschland” (excursion flora of Germany) is a classic guide
to plant identification and, along with another popular book [
60
], the most commonly used
textbook in botany courses. The book is already in its 22nd edition at the time of writing
and works, like many other scientific identification books, via a step-by-step determination
from the main groups to the family and genus and then to the species via a dichotomous
key system with numbers. Although the book provides many illustrations, an extensive
glossary, and a very good index, it can be a challenge for beginners, especially in field work,
with its several hundred pages (depending on the edition).
2.2. Study Participants
2.2.1. Students
Both Bachelor’s and Master’s students participated in the study within their two dif-
ferently designed botany courses in the summer semester of 2021. The Master’s course was
offered online due to social contact restrictions during the COVID-19 pandemic. In addition
to learning plant identification, the course aimed to provide a broader understanding of
urban habitats and their monitoring, and of plants in the urban environment in general.
Twenty students were enrolled at the beginning, and nineteen completed the program.
These Master’s students also had previous experience acquired in botanical identification
exercises. In addition to the theoretical content, an important part of the course was the
application of knowledge during vegetation surveys in the field. The survey protocol
(2.3.2) was explained in detail in a course handout. The use of the PlantNet app was not
explained in this handout, but reference was made to the corresponding website where
all the necessary information was summarized and students were expected to acquire the
knowledge on their own. Students received a course script and a video on how to use the
scientific identification key. However, they were asked to provide feedback and to contact
the instructor with questions if they were unclear. In addition, fixed office hours and a
forum on ISIS (Information System for Instructors and Students) were offered to answer
questions, but both were hardly used. All materials were provided through ISIS.
In the Bachelor’s course, the same instructions for conducting the vegetation survey
were given. Similarly, 20 students enrolled, but the course structure was different compared
with that of the Master’s class. Instead of a continuous course sequence, there was a
block unit between the first and the second vegetation surveys in which the basics of
plant identification were taught. As a result, the students approached the first and second
rounds of data collection with different levels of knowledge. However, in contrast with
the Master’s students, when entering the course, these Bachelor’s students were at the
beginning of their second semester, with no previous experience in vegetation science
or botany.
2.2.2. Experts
Due to contact restrictions during the COVID-19 pandemic, botanical identification
exercises could not take place as usual at the university. This gave rise to the idea of
qualitatively testing the practical use of identification apps in teaching by using a data
set of botany experts as a reference, which had been recorded the year before as part of a
research project. In our case, the expert in botany is the co-author B.S., who is one of the
leading botanists for the Berlin and Brandenburg region (e.g., [
61
]) and has authored key
baseline studies, such as the Red List [62] and the atlas of Berlin’s flora [63].
Sustainability 2023,15, 12984 5 of 15
2.3. Data Collection
2.3.1. Study Sites
The study area was Berlin, the largest city in Germany, with a surface area of 891 km
2
and a population of 3.8 million inhabitants in 2021. About 59% of Berlin is developed with
built-up areas and streets, whereas green and blue spaces cover 41%, including forests
(18%) and grassland (5%) [
64
]. We selected 20 study sites from the dry grassland plots
that were established within the CityScapeLab Berlin, an experimental research platform
to untangle urbanization’s effects on biodiversity and biotic interactions [
65
]. These sites
extend across the outskirts of Berlin and were developed on sandy soils on ruderal sites, on
roadsides, in forest clearings, or near forests. They are extensively managed by mowing a
maximum of once per year, without fertilization or irrigation. All patches belong to the
same phytosociological vegetation type (Sedo–Scleranthetea communities [
66
]). Each site
encompasses one randomly located plot with a standardized size (4 m
×
4 m), which was
calibrated with GPS and marked with four colored marker points to facilitate recognition
during the different vegetation periods.
2.3.2. Task Description and Vegetation Surveys
Students and experts followed the Braun–Blanquet method [
67
] for vegetation map-
ping. All participants recorded the species community and species richness, and assessed
the degree of coverage of all and single species on the 4 m ×4 m plot at the site.
The students worked alone or in teams of two and were responsible for one study plot.
The students organized themselves in a way that every plot was only surveyed by one team
or individual, so that no duplication occurred. In both courses, the students surveyed their
dedicated plot(s) twice, once in April and once in June. In the first round in April, they
were only allowed to use the PlantNet app and internet sources (in the analysis referred
to as Bachelor I and Master I); in the second round in June, they carried out the survey
with scientific literature and expert field guides (Bachelor II and Master II). The expert also
surveyed the 20 sites and corresponding 20 plots in 2020 two times: once between April
and May and a second time in August. The outcomes provide the benchmark to validate
the students’ findings. The expert recorded all vascular plants, and species coverage was
visually estimated in 10% increments [
68
] following the same protocol as the students; the
nomenclature followed Jäger and Rothmaler [
57
,
58
]. Table 1provides an overview of the
student and expert participants of the study.
Table 1. Overview of the research design.
Bachelor Students Master Students Expert
Study program Ecology and Environmental
Planning Urban Ecosystem Sciences -
No. students enrolled 20 20 -
Course form
Block (1st vegetation survey,
two-week identification course, 2nd
vegetation survey)
Continuous
(weekly plus 1st and 2nd
vegetation surveys)
-
Survey plots (one plot per site)
Working form (no. of plots)
Survey year
20
Individually (1)/pairs (2)
2021
19
Individually (1)/pairs (2)
2021
20
Individually (20)
2020
1st vegetation survey in
1st survey identification tools
2nd vegetation survey in
2nd survey identification tools
April
PlantNet app
June
Scientific keys [57,58]
April
PlantNet app
June
Scientific keys [57,58]
April/May
Scientific keys [57,58]
June
Scientific keys [57,58]
2.4. Data Analysis
In the first step, the species numbers obtained with the different experience levels
were compared with those of the expert mapping (Bachelor I vs. Bachelor II vs. Master I
vs. Master II vs. Expert). After testing for normal distribution and variance homogeneity,
Sustainability 2023,15, 12984 6 of 15
an ANOVA (analysis of variance) was performed. Pairwise multiple comparison was
performed afterward using Holm–Sidak post hoc tests.
In the second step, we pooled the surveys of the Bachelor’s students and those of the
Master’s students and analyzed possible differences using ANOVA and Holm Sidak post
hoc tests (Bachelor’s vs. Master’s vs. Expert). To compare the species compositions of the
different study sites, we performed non-metric multidimensional scaling (NMDS). The
Bray–Curtis Index was used as a distance measure. The vegetation data were square-root-
transformed beforehand. To determine the best possible model (local optimum of the stress
value), a total of 100 computational runs for the NMDS were preset.
In the final step, we calculated the number of misidentifications by checking whether
the identified plants were known for Berlin [
62
,
63
] and for dry grassland habitats [
69
]. In
addition, the percentage of missed species and misidentifications compared with the expert
surveys was calculated to be the benchmark (100%). These two error rates were used as
proxies to answer our research questions because we intended to quantitatively evaluate
the usability and usefulness of PlantNet and of scientific keys based on the results of the
plant identifications. We took the first error rate, i.e., the proportion of incorrectly identified
plants because they did not occur in Berlin or in the corresponding habitat, as a proxy for
whether the App or the scientific field guides had an influence on data quality. Should this
error rate change significantly between the two survey rounds within the groups, we could
infer differences in the students’ performance in handling the corresponding identification
tool (e.g., to double-check with available information on distribution and habitat). The
second proxy applies to the students’ overall ability to identify plants and results from
measuring the students’ error compared with the species identified by the expert. If the
proportion of overlooked and incorrectly determined species did not change between the
two survey rounds within the groups, there would be no learning effect from using the
App. t-tests and chi-square tests were performed to reveal differences in the error rates
between the first and second runs for both Bachelor’s and Master’s students, or just both
groups, respectively.
All statistical and multivariate analyses were performed in R version 4.1.2 [
70
] using
the ‘vegan’ package [
71
] for multivariate analysis. The statistical analyses were accom-
panied by a descriptive analysis of how many of the plant species found by the student
groups did not occur in Berlin or which plant species did not normally occur in the habitat
studied [62,63] (Table S1).
3. Results
Considering the total of 20 plots, 330 vascular plant species were recorded. Of
this total number of plant species on all plots, the Bachelor’s student surveys yielded
182 species, whereas the Master’s students recorded 188 species. The expert surveys re-
sulted in 185 plant species in total. We found significant differences in the number of plant
species recorded by those with different levels of experience. Students recorded signifi-
cantly fewer species per site than the expert (F = 44.47, p< 0.001, ANOVA) (Figure 1A).
The Bachelor’s student surveys recorded 6 to 22 species per plot (mean = 13
±
1 SEM) in
the first round and 5 to 21 species (12
±
1) per plot in the second round. The Master’s
students recorded 5 to 28 species per plot during their first survey (14
±
1) and 6 to 24 per
plot in the second one (
14 ±2)
. The expert recorded 17 to 59 species (37
±
3) per plot.
Differences among experience levels were also significant when combining the Bachelor’s
and Master’s surveys (F = 23.67, p< 0.001, ANOVA) (Figure 1B). The ranges were 9 to 30
species
(20 ±1)
for Bachelor’s students and 10 to 40 species (20
±
2) for Master’s students
per plot, combined for the first and second runs.
The species composition varied among the study sites (Figure 2). The differences
in species composition were lower in the expert surveys than those in the Bachelor’s
and Master’s surveys. Both Bachelor’s and Master’s students recorded several species
that neither occurred in Berlin. nor in the habitat type dry grassland (Figure 3). The
numbers of species that did not occur in Berlin was 34 species (18.7%) for Bachelor’s
Sustainability 2023,15, 12984 7 of 15
and 24 (12.7%) for Master’s students. The numbers of species that did not occur in dry
grasslands were higher, with 59 species (32.4%) and 53 species (28.2%) for Bachelor’s and
Master’s surveys, respectively.
Sustainability 2023, 15, x FOR PEER REVIEW 7 of 15
Master’s surveys (F = 23.67, p < 0.001, ANOVA) (Figure 1B). The ranges were 9 to 30 species
(20 ± 1) for Bachelor’s students and 10 to 40 species (20 ± 2) for Master’s students per plot,
combined for the first and second runs.
Figure 1. The number of plant species recorded by those with different levels of experience (A),
which differed significantly between the students and the expert (F = 44.47, p < 0.001, ANOVA). The
numbers of species also differed significantly (F = 23.67, p < 0.001, ANOVA) when comparing Bach-
elor’s students, Master’s students, and the expert (B). Different lower-case letters indicate significant
differences among groups, with p < 0.001.
The species composition varied among the study sites (Figure 2). The differences in
species composition were lower in the expert surveys than those in the Bachelor’s and
Master’s surveys. Both Bachelor’s and Master’s students recorded several species that nei-
ther occurred in Berlin. nor in the habitat type dry grassland (Figure 3). The numbers of
species that did not occur in Berlin was 34 species (18.7%) for Bachelor’s and 24 (12.7%)
for Master’s students. The numbers of species that did not occur in dry grasslands were
Figure 1.
The number of plant species recorded by those with different levels of experience (
A
),
which differed significantly between the students and the expert (F = 44.47, p< 0.001, ANOVA).
The numbers of species also differed significantly (F = 23.67, p< 0.001, ANOVA) when comparing
Bachelor’s students, Master’s students, and the expert (
B
). Different lower-case letters indicate
significant differences among groups, with p< 0.001.
The Bachelor’s students who had no prior knowledge, but attended a plant identi-
fication course between the two data collection dates, had significantly lower error rates
with regard to species not known in Berlin or in dry grassland habitats in the second
round (Berlin: t = 3.14, p< 0.003/habitat type: t = 3.43, p< 0.001, t-test). No significant
changes in error rates were reported for the Master’s students, who had experience in
plant species identification prior to data collection (Berlin: t = 1.34, p< 0188/habitat type:
t = 1.31,
p< 0.198,
t-test). Compared with the data from the expert surveys, the students
overlooked or misidentified a large number of species (Table S1). Between 77.5% and 81.2%
of the species determined by the expert were not detected by the students in all runs, and
between 12.4% and 22.6% of the species identified by the students were incorrect (Figure 4).
These error rates were not significantly different between student groups.
Sustainability 2023,15, 12984 8 of 15
Sustainability 2023, 15, x FOR PEER REVIEW 8 of 15
higher, with 59 species (32.4%) and 53 species (28.2%) for Bachelor’s and Master’s surveys,
respectively.
Figure 2. The comparison of the plant species composition revealed a wider dispersion among the
results of the Bachelor’s (square) and Master’s (circles) surveys than that of the expert (triangle)
(NMDS, stress = 0.19).
Figure 2.
The comparison of the plant species composition revealed a wider dispersion among the
results of the Bachelor’s (square) and Master’s (circles) surveys than that of the expert (triangle)
(NMDS, stress = 0.19).
Sustainability 2023, 15, x FOR PEER REVIEW 8 of 15
higher, with 59 species (32.4%) and 53 species (28.2%) for Bachelor’s and Master’s surveys,
respectively.
Figure 2. The comparison of the plant species composition revealed a wider dispersion among the
results of the Bachelor’s (square) and Master’s (circles) surveys than that of the expert (triangle)
(NMDS, stress = 0.19).
Figure 3.
Both Bachelor’s and Master’s students recorded several species that neither occurred
in Berlin (
A
), nor were found in the habitat type dry grassland (
B
). No significant differences (
χ
2
(1, N = 173) = 0.56, p> 0.05) were found between groups.
Sustainability 2023,15, 12984 9 of 15
Sustainability 2023, 15, x FOR PEER REVIEW 9 of 15
Figure 3. Both Bachelor’s and Master’s students recorded several species that neither occurred in
Berlin (A), nor were found in the habitat type dry grassland (B). No significant differences (χ2 (1, N
= 173) = 0.56, p > 0.05) were found between groups.
The Bachelor’s students who had no prior knowledge, but attended a plant identifi-
cation course between the two data collection dates, had significantly lower error rates
with regard to species not known in Berlin or in dry grassland habitats in the second
round (Berlin: t = 3.14, p < 0.003/habitat type: t = 3.43, p < 0.001, t-test). No significant
changes in error rates were reported for the Master’s students, who had experience in
plant species identification prior to data collection (Berlin: t = 1.34, p < 0188/habitat type: t
= 1.31, p < 0.198, t-test). Compared with the data from the expert surveys, the students
overlooked or misidentified a large number of species (Table S1). Between 77.5% and
81.2% of the species determined by the expert were not detected by the students in all
runs, and between 12.4% and 22.6% of the species identified by the students were incorrect
(Figure 4). These error rates were not significantly different between student groups.
Figure 4. Percentage of missed and misidentified species by Bachelor’s and Master’s students in
relation to the expert surveys for both the first and second rounds of surveys.
4. Discussion
In this study, we compared the data quality and potential learning effects of the use
of different plant species identification tools by Bachelor’s and Master’s students using
expert knowledge for reference. With respect to our first question, we started by looking
at species richness. The expert recorded, on average, twice as many species per site as the
students. Likely due to their greater experience, experts can identify vegetation very
quickly and less time is needed to consult a scientific identification key. In contrast, the
different levels of performance of the students, that is to say, not all students have the
same skills or experience, may have led to a lower average and a wider range in the
Figure 4.
Percentage of missed and misidentified species by Bachelor’s and Master’s students in
relation to the expert surveys for both the first and second rounds of surveys.
4. Discussion
In this study, we compared the data quality and potential learning effects of the use
of different plant species identification tools by Bachelor’s and Master’s students using
expert knowledge for reference. With respect to our first question, we started by looking
at species richness. The expert recorded, on average, twice as many species per site as
the students. Likely due to their greater experience, experts can identify vegetation very
quickly and less time is needed to consult a scientific identification key. In contrast, the
different levels of performance of the students, that is to say, not all students have the same
skills or experience, may have led to a lower average and a wider range in the number of
species identified per site. Without expert knowledge, it is, in general, harder to identify
species and, in addition, students overlooked or misidentified many species (Table S1).
The misidentified and overlooked species were mainly grasses or non-flowering
specimens that were difficult to recognize from photos or without expert knowledge (e.g.,
rosettes and seedlings). The level of student commitment to completing the task may have
also been a factor influencing the species richness. Students’ intrinsic motivation has been
shown to correlate positively with performance in species identification [
72
]. Lastly, when
accumulating the plant species across sites, there was nearly agreement on the total number
identified between the expert and student groups. However, this result was misleading,
because the students made a high proportion of incorrect determinations.
The first indication of possible misidentifications in the students’ data collection
was the higher dissimilarities in species composition between sites shown in the NMDS
(Figure 2). The dispersion in the plant assemblages between sites was much wider in the
students’ vegetation collections compared with the expert’s. A difference in the species
composition between the sites could be expected, for example, to be affected by the different
degrees of urbanization per sample location [
73
,
74
] or correlations between phenology and
microclimatic conditions [
75
]. However, these factors applied to all groups and could not
Sustainability 2023,15, 12984 10 of 15
account for the large difference in dispersion between student and expert surveys, because
species composition would not change that dramatically from 2020 to 2021.
A more direct measure, but independent of the expert’s result, is whether the identified
species had already been documented for Berlin or in urban dry grasslands. This measure
would represent how accurately the students worked and what local species knowledge
they had. Between 12.7% and 32.4% of the plants identified by the students did not fall
into these two categories. These values could be interpreted like an error rate, given the
excellent documentation of the flora of Berlin [
62
,
69
] and the vegetation of dry grasslands in
this region [
76
–
79
]. Consequently, a change in the error rate in plant identification between
the students’ first and second data collections would provide indications of whether the
use of the App or the scientific literature had any influence on data quality.
We considered the Master’s students to be the control group and the Bachelor’s
students to be the experimental group, because the latter attended an additional botanical
identification course between the two sampling rounds. A significant decrease in the
error rate between the samplings was demonstrated for the Bachelor’s students. The
Master’s students had more experience in general and, therefore, performed better than
the Bachelor’s students, but there was no change in the error rate between the two data
collections. Presumably, the traditionally taught plant identification course, where they
learned to use scientific determination keys and to identify the characteristics of selected
plant families, was responsible for the Bachelor’s students’ improved identification skills.
They probably had a better understanding of the terminology and plant characteristics
after the course.
Finally, by measuring the error of the students in comparison with the species identi-
fied by the expert, the results revealed high percentages of missed and incorrectly deter-
mined species. This measure can be interpreted like an error rate representing the students’
overall ability to identify plants. The visible, but non-significant, decrease in misidenti-
fications between the two survey rounds for both the Bachelor’s and Master’s courses
could be due to the Bachelor’s botany course or the use of scientific literature. However,
considering that the supposedly more experienced Master’s students fared even worse
than the Bachelor’s students, the overall performance of the students was humbling.
To use the data for further research beyond the teaching obligation, misclassifications
have to be ruled out. One possibility would be to filter student’s data by matching them
with already existing, excellent databases (as performed here), but also, other approaches
that have been developed in the context of citizen science in recent years could be applied,
for example, automated filtering, consensus methods, or expert validations [
80
,
81
]. In
fact, PlantNet has an advantage over the handed field guides in that users can view a
world map to explore whether the species has already been recorded for this place or
region, or find out information about its distribution via direct links to further information,
for example, Wikipedia or GBIF (Global Biodiversity Information Facility). In addition,
students should be made aware that PlantNet has set its own accuracy threshold above 0.9
(one criterion to be automatically uploaded to GBIF) and that they should take extra care
when adopting the App’s suggestions if the score is lower. It is possible that the students
were not aware of these functions, which would have helped them to assess the correctness
of their determination themselves. For future field work with students using PlantNet or
another mobile application that specifically maps the German flora, for example, Flora
Incognita [82], a more thorough introduction to the functionalities seems necessary.
Considering not only the first, but also our second research question, whether the
App may enhance knowledge acquisition, our results show two consequences, firstly for
the use of plant identification apps in teaching and beyond, and secondly for species
identification courses in general. With regard to identification apps in teaching, Thomas
and Fellowes [
83
] reported similar performances in bird species identification, whereas
Jeno and colleagues [
72
] reported higher performance in identifying plant species with a
corresponding mobile application compared with traditional field guides. Noncomparative
studies have shown that students using citizen science apps, like iNaturalist, can generate
Sustainability 2023,15, 12984 11 of 15
a high percentage of high-quality data [
31
,
50
]. Apps for species identification in higher
education, therefore, have their justification, given that they seem to provide useful data
for different applications, even when users are novice botanists. In our case, the proportion
of incorrectly identified species does not fully support the idea that species identification
apps, in the way that they were used in our course, help to develop better skills, or generate
high-quality data. However, it can be assumed that the students would have been able
to identify far fewer plants without PlantNet and that their performance and learning
progress—in contrast with other studies—were assessed on the basis of expert knowledge
and in the context of a very specific type of biotope with species that are difficult to identify,
for example, grasses.
Even though we could not demonstrate in our investigation that the use of PlantNet
led to immediate learning effects, other cases showed positive impacts by allowing the
students to use of apps instead of books or printed handouts. Mobile applications have
been found to increase motivation to complete species identification tasks [
72
,
83
]. This
is also true for applications that do not contain automated image recognition, but only
a digital version of a scientific key that appears to be more user-friendly than turning
pages in a book [
84
,
85
]. In general, students in higher education found the use of a citizen
science-based mobile application to support species identification motivating [
30
,
51
]. The
not significant, but still higher average number of species identified in the first round, when
Bachelor’s and Master’s students were allowed to use PlantNet, could be an indication
that this was true for our student groups as well, although many plants could also have
dried out until June due to the limited rainfall in spring 2021. However, it cannot be
ruled out that the higher number was caused by the faster and more convenient use of an
app that immediately identified the species, perhaps even without double-checking with
further resources.
In assessing the usefulness of courses for species identification in general, the error
rates highlighted how difficult the task of identifying plant species is and how little can be
taught at university. Developing these skills takes time and patience, and often requires
not only experience in field data collection and working with scientific literature, but also
an expert mentor to receive feedback and improve [
86
,
87
]. Apps can stimulate interest in
learning about plants more easily and straightforwardly, and help to achieve initial success
more quickly. This is also one of the core tasks of these apps, which is to motivate the
general population to concern themselves with the environment and its protection [
59
].
But learning one or more species groups is a lengthy process that cannot be shortened via
digitization either. Human experts are extremely important, also for the capabilities of the
apps, for example, for validating or labeling images to improve accuracy [
53
]. Therefore,
the teaching of species knowledge should be given a higher priority at university, not to
mention in the context of global biodiversity loss. Given the possibilities and limitations of
plant identification apps, combining them with scientific literature and digital identification
keys would probably be the best way to improve the accuracy of students’ performance
and, at the same time, avoid frustration.
The results of this study should only be understood to be preliminary, because they
are subject to some limitations. To assess the learning effect through the use of an app,
it would be advisable to separate the groups further, for example, into teams working
only with PlantNet or only with the field guides before and after the botany course. A
control group that does not participate in the botany course is not feasible, because the
species identification course is mandatory for all students and would also contradict the
educational mission. In addition, it would be better to measure learning effects with quizzes
or surveys before and after the course, in addition to estimating error rates. Simultaneously,
other factors, like the impacts of the App use on motivation, attitudes toward the task and
the environment, and effects on social engagement, could be examined with accompanying
questionnaires or interviews [
83
]. It is not clear to what extent students were familiar with
the various features of the App. There was a possibility that not all helpful information
was exploited to assess whether the identified plant was a realistic selection. Including a
Sustainability 2023,15, 12984 12 of 15
community-validated digital herbarium or collection that could be accessed as a registered
user of PlantNet could have helped students build the knowledge base needed for the
second round of sampling. Last, but not least, this study was also limited by the random-
ness of the field work, which confronted both the students and the expert with different
conditions—the respective weather patterns during the recording, the phenology, and other
unknown factors influencing the occurrence and seasonal appearance of plant species.
Supplementary Materials:
The following supporting information can be downloaded at: https:
//www.mdpi.com/article/10.3390/su151712984/s1, Table S1: Top misidentified and missed species.
Author Contributions:
Conceptualization, S.B. and N.P.; formal analysis, S.B.; investigation, J.H.,
A.K.G., B.S. and S.B.; resources, J.H., A.K.G., B.S. and S.B.; writing—original draft preparation, N.P.;
writing—review and editing, all authors; project administration, J.H., A.K.G., B.S. and S.B. All authors
have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement:
The data presented in this study are available on request from the
corresponding author. The data are not publicly available due to ongoing research.
Conflicts of Interest: The authors declare no conflict of interest.
References
1.
Cooper, C.B.; Hawn, C.L.; Larson, L.R.; Parrish, J.K.; Bowser, G.; Cavalier, D.; Dunn, R.R.; Haklay, M.; Gupta, K.K.; Jelks, N.T.O.;
et al. Inclusion in citizen science: The conundrum of rebranding. Science 2021,372, 1386–1388. [CrossRef]
2.
Newman, G.; Wiggins, A.; Crall, A.; Graham, E.; Newman, S.; Crowston, K. The future of citizen science: Emerging technologies
and shifting paradigms. Front. Ecol. Environ. 2012,10, 298–304. [CrossRef]
3.
Sullivan, B.L.; Aycrigg, J.L.; Barry, J.H.; Bonney, R.E.; Bruns, N.; Cooper, C.B.; Damoulas, T.; Dhondt, A.A.; Dietterich, T.;
Farnsworth, A.; et al. The eBird enterprise: An integrated approach to development and application of citizen science. Biol.
Conserv. 2014,169, 31–40. [CrossRef]
4.
Prudic, K.L.; McFarland, K.P.; Oliver, J.C.; Hutchinson, R.A.; Long, E.C.; Kerr, J.T.; Larrivee, M. eButterfly: Leveraging massive
online citizen science for butterfly conservation. Insects 2017,8, 53. [CrossRef]
5.
Pernat, N.; Kampen, H.; Ruland, F.; Jeschke, J.M.; Werner, D. Drivers of spatio-temporal variation in mosquito submissions to the
citizen science project ‘Muckenatlas’. Sci. Rep. 2021,11, 1356. [CrossRef] [PubMed]
6.
Lewandowski, E.; Specht, H. Influence of volunteer and project characteristics on data quality of biological surveys. Conserv. Biol.
J. Soc. Conserv. Biol. 2015,29, 713–723. [CrossRef] [PubMed]
7.
Kelling, S.; Johnston, A.; Bonn, A.; Fink, D.; Ruiz-Gutierrez, V.; Bonney, R.; Fernandez, M.; Hochachka, W.M.; Julliard, R.; Kraemer,
R.; et al. Using semistructured surveys to improve citizen science data for monitoring biodiversity. Bioscience
2019
,69, 170–179.
[CrossRef]
8.
Locke, C.M.; Anhalt-Depies, C.M.; Frett, S.; Stenglein, J.L.; Cameron, S.; Malleshappa, V.; Peltier, T.; Zuckerberg, B.; Townsend,
P.A. Managing a large citizen science project to monitor wildlife. Wildl. Soc. Bull. 2019,43, 4–10. [CrossRef]
9.
Isaac, N.J.B.; van Strien, A.J.; August, T.A.; de Zeeuw, M.P.; Roy, D.B.; Anderson, B. Statistics for citizen science: Extracting signals
of change from noisy ecological data. Methods Ecol. Evol. 2014,5, 1052–1060. [CrossRef]
10.
Kelling, S.; Fink, D.; La Sorte, F.A.; Johnston, A.; Bruns, N.E.; Hochachka, W.M. Taking a ‘Big Data’ approach to data quality in a
citizen science project. Ambio 2015,44 (Suppl. 4), 601–611. [CrossRef]
11.
Isaac, N.J.B.; Jarzyna, M.A.; Keil, P.; Dambly, L.I.; Boersch-Supan, P.H.; Browning, E.; Freeman, S.N.; Golding, N.; Guillera-Arroita,
G.; Henrys, P.A.; et al. Data integration for large-scale models of species distributions. Trends Ecol. Evol.
2020
,35, 56–67. [CrossRef]
[PubMed]
12.
Johnston, A.; Hochachka, W.M.; Strimas-Mackey, M.E.; Ruiz Gutierrez, V.; Robinson, O.J.; Miller, E.T.; Auer, T.; Kelling, S.T.; Fink,
D.; Fourcade, Y. Analytical guidelines to increase the value of community science data: An example using eBird data to estimate
species distributions. Divers. Distrib. 2021,27, 1265–1277. [CrossRef]
13.
Callaghan, C.T.; Rowley, J.J.L.; Cornwell, W.K.; Poore, A.G.B.; Major, R.E. Improving big citizen science data: Moving beyond
haphazard sampling. PLoS Biol. 2019,17, e3000357. [CrossRef]
14.
Vohland, K.; Land-Zandstra, A.; Ceccaroni, L.; Lemmens, R.; Perelló, J.; Ponti, M.; Samson, R.; Wagenknecht, K. The Science of
Citizen Science; Springer Nature: Cham, Switzerland, 2021. [CrossRef]
15.
Domroese, M.C.; Johnson, E.A. Why watch bees? Motivations of citizen science volunteers in the Great Pollinator Project. Biol.
Conserv. 2017,208, 40–47. [CrossRef]
Sustainability 2023,15, 12984 13 of 15
16.
Larson, L.R.; Cooper, C.B.; Futch, S.; Singh, D.; Shipley, N.J.; Dale, K.; LeBaron, G.S.; Takekawa, J.Y. The diverse motivations
of citizen scientists: Does conservation emphasis grow as volunteer participation progresses? Biol. Conserv.
2020
,242, 108428.
[CrossRef]
17.
West, S.; Dyke, A.; Pateman, R. Variations in the motivations of environmental citizen scientists. Citiz. Sci. Theory Pract.
2021
,6,
14. [CrossRef]
18.
Boakes, E.H.; Gliozzo, G.; Seymour, V.; Harvey, M.; Smith, C.; Roy, D.B.; Haklay, M. Patterns of contribution to citizen science
biodiversity projects increase understanding of volunteers’ recording behaviour. Sci. Rep. 2016,6, 33051. [CrossRef] [PubMed]
19.
August, T.; Fox, R.; Roy, D.B.; Pocock, M.J.O. Data-derived metrics describing the behaviour of field-based citizen scientists
provide insights for project design and modelling bias. Sci. Rep. 2020,10, 11009. [CrossRef]
20.
Sturm, U.; Straka, T.M.; Moormann, A.; Egerer, M. Fascination and joy: Emotions predict urban gardeners’ pro-pollinator
behaviour. Insects 2021,12, 785. [CrossRef]
21.
Mac Domhnaill, C.; Lyons, S.; Nolan, A. The citizens in citizen science: Demographic, socioeconomic, and health characteristics of
biodiversity recorders in Ireland. Citiz. Sci. Theory Pract. 2020,5, 16. [CrossRef]
22.
Pateman, R.; Dyke, A.; West, S. The diversity of participants in environmental citizen science. Citiz. Sci. Theory Pract.
2021
,6, 9.
[CrossRef]
23.
Haywood, B.K.; Parrish, J.K.; Dolliver, J. Place-based and data-rich citizen science as a precursor for conservation action. Conserv.
Biol. J. Soc. Conserv. Biol. 2016,30, 476–486. [CrossRef]
24.
Schuttler, S.G.; Sears, R.S.; Orendain, I.; Khot, R.; Rubenstein, D.; Rubenstein, N.; Dunn, R.R.; Baird, E.; Kandros, K.; O’Brien, T.;
et al. Citizen science in schools: Students collect valuable mammal data for science, conservation, and community engagement.
BioScience 2019,69, 69–79. [CrossRef]
25.
Peter, M.; Diekötter, T.; Kremer, K. Participant outcomes of biodiversity citizen science projects: A systematic literature review.
Sustainability 2019,11, 2780. [CrossRef]
26.
Peter, M.; Diekötter, T.; Höffler, T.; Kremer, K. Biodiversity citizen science: Outcomes for the participating citizens. People Nat.
2021,3, 294–311. [CrossRef]
27.
Bonney, R.; Phillips, T.B.; Ballard, H.L.; Enck, J.W. Can citizen science enhance public understanding of science? Public Underst.
Sci. 2016,25, 2–16. [CrossRef] [PubMed]
28.
Vance-Chalcraft, H.D.; Hurlbert, A.H.; Styrsky, J.N.; Gates, T.A.; Bowser, G.; Hitchcock, C.B.; Reyes, M.A.; Cooper, C.B. Citizen
science in postsecondary education: Current practices and knowledge gaps. Bioscience 2022,72, 276–288. [CrossRef]
29.
Paradise, C.; Bartkovich, L. Integrating citizen science with online biological collections to promote species and biodiversity
literacy in an entomology course. Citiz. Sci. Theory Pract. 2021,6, 28. [CrossRef]
30.
Lichti, D.; Mosley, P.; Callis-Duehl, K. Learning from the trees: Using project budburst to enhance data literacy and scientific
writing skills in an introductory biology laboratory during remote learning. Citiz. Sci. Theory Pract. 2021,6, 32. [CrossRef]
31.
Stevenson, R.; Merrill, C.; Burn, P. Useful biodiversity data were obtained by novice observers using iNaturalist during college
orientation retreats. Citiz. Sci. Theory Pract. 2021,6, 27. [CrossRef]
32.
Van Haeften, S.; Milic, A.; Addison-Smith, B.; Butcher, C.; Davies, J.M. Grass Gazers: Using citizen science as a tool to facilitate
practical and online science learning for secondary school students during the COVID-19 lockdown. Ecol. Evol.
2021
,11,
3488–3500. [CrossRef] [PubMed]
33.
Oberbauer, A.M.; Lai, E.; Kinsey, N.A.; Famula, T.R. Enhancing student scientific literacy through participation in citizen science
focused on companion animal behavior. Transl. Anim. Sci. 2021,5, txab131. [CrossRef] [PubMed]
34.
Schirmel, J. COVID-19 pandemic turns life-science students into “citizen scientists”: Data indicate multiple negative effects of
urbanization on biota. Sustainability 2021,13, 2992. [CrossRef]
35. Baker, B. Biodiversity collections, data, and COVID. BioScience 2020,70, 841–847. [CrossRef]
36.
Gerhart, L.M.; Jadallah, C.C.; Angulo, S.S.; Ira, G.C. Teaching an experiential field course via online participatory science projects:
A COVID-19 case study of a UC California Naturalist course. Ecol. Evol. 2021,11, 3537–3550. [CrossRef] [PubMed]
37.
Weigelt, P.; Denelle, P.; Brambach, F.; Kreft, H. BotanizeR: A flexible R package with Shiny app to practice plant identification for
online teaching and beyond. Plants People Planet 2021,4, 122–127. [CrossRef]
38.
Genovart, M.; Tavecchia, G.; Enseñat, J.J.; Laiolo, P. Holding up a mirror to the society: Children recognize exotic species much
more than local ones. Biol. Conserv. 2013,159, 484–489. [CrossRef]
39.
Yli-Panula, E.; Matikainen, E. students and student teachers’ ability to name animals in ecosystems: A perspective of animal
knowledge and biodiversity. J. Balt. Sci. Educ. 2014,13, 559–572. [CrossRef]
40.
Buck, T.; Bruchmann, I.; Zumstein, P.; Drees, C. Just a small bunch of flowers: The botanical knowledge of students and the
positive effects of courses in plant identification at German universities. PeerJ 2019,7, e6581. [CrossRef]
41.
Gerl, T.; Randler, C.; Jana Neuhaus, B. Vertebrate species knowledge: An important skill is threatened by extinction. Int. J. Sci.
Educ. 2021,43, 928–948. [CrossRef]
42.
Sieg, A.K.; Teibtner, R.; Dreesmann, D. Don’t know much about bumblebees?—A study about secondary school students’
knowledge and attitude shows educational demand. Insects 2018,9, 40. [CrossRef] [PubMed]
43.
Christ, L.; Dreesmann, D.C. SAD but true: Species awareness disparity in bees is a result of bee-less biology lessons in Germany.
Sustainability 2022,14, 2604. [CrossRef]
Sustainability 2023,15, 12984 14 of 15
44.
Hooykaas, M.J.D.; Schilthuizen, M.; Aten, C.; Hemelaar, E.M.; Albers, C.J.; Smeets, I. Identification skills in biodiversity
professionals and laypeople: A gap in species literacy. Biol. Conserv. 2019,238, 108202. [CrossRef]
45. Frobel, K.; Schlumprecht, H. Erosion der Artenkenner. Naturschutz Landschaftsplanung 2016,48, 105–113.
46. Hochkirch, A. The insect crisis we can’t ignore. Nature 2016,539, 141. [CrossRef] [PubMed]
47.
Wägele, J.W.; Bodesheim, P.; Bourlat, S.J.; Denzler, J.; Diepenbroek, M.; Fonseca, V.; Frommolt, K.-H.; Geiger, M.F.; Gemeinholzer,
B.; Glöckner, F.O.; et al. Towards a multisensor station for automated biodiversity monitoring. Basic Appl. Ecol.
2022
,59, 105–138.
[CrossRef]
48.
Dirzo, R.; Young, H.S.; Galetti, M.; Ceballos, G.; Isaac, N.J.; Collen, B. Defaunation in the Anthropocene. Science
2014
,345, 401–406.
[CrossRef]
49.
Pocock, M.J.O.; Chandler, M.; Bonney, R.; Thornhill, I.; Albin, A.; August, T.; Bachman, S.; Brown, P.M.J.; Cunha, D.G.F.; Grez, A.;
et al. A vision for global biodiversity monitoring with citizen science. In Advances in Ecological Research; Bohan, D.A., Dumbrell,
A.J., Woodward, G., Jackson, M., Eds.; Academic Press: Cambridge, MA, USA, 2018; Volume 59, pp. 169–223. [CrossRef]
50.
Hitchcock, C.; Vance-Chalcraft, H.; Aristeidou, M. Citizen science in higher education. Citiz. Sci. Theory Pract.
2021
,6, 22. [CrossRef]
51.
Rokop, M.; Srikanth, R.; Albert, M.; Radonic, C.; Vincent, R.; Stevenson, R. Looking more carefully: A successful Bioblitz
orientation activity at an urban public university. Citiz. Sci. Theory Pract. 2022,7, 1. [CrossRef]
52.
Chu, M.; Leonard, P.; Stevenson, F. Growing the base for citizen science. In Citizen Science: Public Participation in Environmental
Research, 1st ed.; Dickinson, J.L., Bonney, R., Eds.; Cornell University Press: Ithaca, NY, USA, 2012; pp. 69–81.
53.
Wäldchen, J.; Rzanny, M.; Seeland, M.; Mader, P. Automated plant species identification—Trends and future directions. PLoS
Comput. Biol. 2018,14, e1005993. [CrossRef]
54.
Raphael, A.; Dubinsky, Z.; Iluz, D.; Netanyahu, N.S. Neural network recognition of marine benthos and corals. Diversity
2020
,
12, 29. [CrossRef]
55.
Hoye, T.T.; Arje, J.; Bjerge, K.; Hansen, O.L.P.; Iosifidis, A.; Leese, F.; Mann, H.M.R.; Meissner, K.; Melvad, C.; Raitoharju, J. Deep
learning and computer vision will transform entomology. Proc. Natl. Acad. Sci. USA
2021
,118, e2002545117. [CrossRef] [PubMed]
56.
Jones, H.G. What plant is that? Tests of automated image recognition apps for plant identification on plants from the British flora.
AoB Plants 2020,12, plaa052. [CrossRef] [PubMed]
57. Jäger, E. Exkursionsflora von Deutschland, 20th ed.; Springer Spektrum: Heidelberg, Germany, 2011.
58.
Jäger, E.; Rothmaler, J. Exkursionsflora von Deutschland, Gefäßpflanzen: Atlasband, 12th ed.; Springer Spektrum: Heidelberg,
Germany, 2013.
59.
Bonnet, P.; Joly, A.; Faton, J.M.; Brown, S.; Kimiti, D.; Deneu, B.; Servajean, M.; Affouard, A.; Lombardo, J.C.; Mary, L.; et al. How
citizen scientists contribute to monitor protected areas thanks to automatic plant identification tools. Ecol. Solut. Evid.
2020
,
1, e12023. [CrossRef]
60. Schmeil, O.; Fitschen, J. Flora von Deutschland und Angrenzender Länder; Quelle & Meyer Verlag: Heidelberg, Germany, 1993.
61.
Seitz, B. Mapping the urban flora of Berlin. In The Botanical City, 1st ed.; Gandy, M., Jasper, S., Eds.; Jovis: Berlin, Germany, 2020;
pp. 293–300.
62.
Seitz, B.; Ristow, M.; Meißner, J.; Machatzi, B.; Sukopp, H. Rote Liste und Gesamtartenliste der etablierten Farn- und
Blütenpflanzen von Berlin. In Der Landesbeauftragte für Naturschutz und Landschaftspflege, Rote Listen der Gefährdeten Pflanzen, Pilze
und Tiere von Berlin; Senatsverwaltung für Umwelt, Klima und Verkehr: Berlin, Germany, 2018; pp. 6–94.
63.
Seitz, B.; Ristow, M.; Prasse, R.; Machatzi, B.; Klemm, G.; Böcker, R.; Sukopp, H. Verhandlungen des Botanischen Vereins von
Berlin und Brandenburg. In Der Berliner Florenatlas; Beiheft 7; Natur+Text GmbH: Rangsdorf, Germany, 2012; pp. 1–533.
64.
Senatsverwaltung für Stadtentwicklung und Wohnen Berlin. Luftbilder 2016 (Orthophotos). Digitale Farbige Orthophotos 2016
(DOP20RGB). Available online: https://fbinter.stadt-berlin.de/fb/?loginkey=showMap&mapId=k_luftbild2016_rgb@senstadt
(accessed on 2 March 2020).
65.
von der Lippe, M.; Buchholz, S.; Hiller, A.; Seitz, B.; Kowarik, I. CityScapeLab Berlin: A research platform for untangling
urbanization effects on biodiversity. Sustainability 2020,12, 2565. [CrossRef]
66.
Leuschner, C.; Ellenberg, H. Ecology of Central European Non-Forest Vegetation: Coastal to Alpine, Natural to Man-Made Habitats;
Springer: Cham, Switzerland, 2017.
67. Braun-Blanquet, J. Pflanzensoziologie, 3rd ed.; Springer: Wien, Austria; New York, NY, USA, 1964.
68. van der Maarel, E.; Franklin, J. Vegetation Ecology, 2nd ed.; Blackwell: Wiley, Austria, 2012.
69. Jäger, E. Exkursionsflora von Deutschland. Gefäßpflanzen: Grundband, 21st ed.; Springer Spektrum: Heidelberg, Germany, 2016.
70.
R Core Team. A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2018.
71.
Oksanen, J.; Blanchet, F.G.; Friendly, M.; Kindt, R.; Legendre, P.; McGlinn, D.; Minchin, P.R.; O’Hara, R.B.; Simspon, G.L.; Solymos,
P.; et al. Vegan: Community Ecology Package, R Package; version 2.5–4; 2019. Available online: https://cran.r-project.org/
package=vegan (accessed on 6 May 2022).
72.
Jeno, L.M.; Grytnes, J.-A.; Vandvik, V. The effect of a mobile-application tool on biology students’ motivation and achievement in
species identification: A Self-Determination Theory perspective. Comput. Educ. 2017,107, 1–12. [CrossRef]
73.
Albrecht, H.; Haider, S. Species diversity and life history traits in calcareous grasslands vary along an urbanization gradient.
Biodivers. Conserv. 2013,22, 2243–2267. [CrossRef]
74.
Cochard, A.; Pithon, J.; Jagaille, M.; Beaujouan, V.; Pain, G.; Daniel, H. Grassland plant species occurring in extensively managed
road verges are filtered by urban environments. Plant Ecol. Divers. 2017,10, 217–229. [CrossRef]
Sustainability 2023,15, 12984 15 of 15
75.
Christmann, T.; Kowarik, I.; Bernard-Verdier, M.; Buchholz, S.; Hiller, A.; Seitz, B.; von der Lippe, M. Phenology of grassland
plants responds to urbanization. Urban Ecosyst. 2023,26, 261–275. [CrossRef]
76. Krausch, H.-D. Die Sandtrockenrasen in Brandenburg. Mitt. Florist. Soziologischen Arb. (Neue Folge) 1968,13, 71–100.
77.
Bornkamm, R. Zu den Standortbedingungen einiger Sand-Therophytenrasen in Berlin West. Verhandlungen Bot. Ver. Prov.
Brandenbg. 1977,113, 27–39.
78.
Sukopp, H. Sandmagerrasen auf urban-industriellen Sekundärstandorten. Beobachtungen im Berliner Gebiet 1952–1998. Verh.
Bot. Ver. Berl. Brandenbg. 1999,132, 221–252.
79.
Hüllbusch, E.; Brandt, L.M.; Ende, P.; Dengler, J. Little vegetation change during two decades in a dry grassland complex in the
Biosphere Reserve Schorfheide-Chorin NE Germany. Tuexenia 2016,36, 395–412. [CrossRef]
80.
Kosmala, M.; Wiggins, A.; Swanson, A.; Simmons, B. Assessing data quality in citizen science. Front. Ecol. Environ.
2016
,14,
551–560. [CrossRef]
81.
Soroye, P.; Ahmed, N.; Kerr, J.T. Opportunistic citizen science data transform understanding of species distributions, phenology,
and diversity gradients for global change research. Glob. Chang. Biol. 2018,24, 5281–5291. [CrossRef]
82.
Wäldchen, J.; Mäder, P.; Cooper, N. Machine learning for image based species identification. Methods Ecol. Evol.
2018
,9, 2216–2225.
[CrossRef]
83.
Thomas, R.L.; Fellowes, M.D.E. Effectiveness of mobile apps in teaching field-based identification skills. J. Biol. Educ.
2016
,51,
136–143. [CrossRef]
84.
Stagg, B.C.; Donkin, M.E. Apps for angiosperms: The usability of mobile computers and printed field guides for UK wild flower
and winter tree identification. J. Biol. Educ. 2016,51, 123–135. [CrossRef]
85.
Finger, A.; Groß, J.; Zabel, J. Plant identification in the 21st century—What possibilities do modern identification keys offer for
biology lessons? Educ. Sci. 2022,12, 849. [CrossRef]
86. Noss, R.F. The naturalists are dying off. Conserv. Biol. 1996,10, 1–3. [CrossRef]
87.
Pearson, D.L.; Hamilton, A.L.; Erwin, T.L. Recovery plan for the endangered taxonomy profession. BioScience
2011
,61, 58–63.
[CrossRef]
Disclaimer/Publisher’s Note:
The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.
