An evaluation of accessibility of Covid-19 statistical charts of governments and health organisations for people with low vision

e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 1 An evaluation of accessibility of Covid-19 statistical charts of governments and health organisations for people with low vision Rubén Alcaraz-Martínez; Mireia Ribera-Turró How to cite this article: Alcaraz-Martínez, Rubén; Ribera-Turró, Mireia (2020). “An evaluation of accessibility of Covid-19 statistical charts of governments and health organisations for people with low vision”. Profesional de la información, v. 29, n. 5, e290514. https://doi.org/10.3145/epi.2020.sep.14 Manuscript received on April 28th 2020 Accepted on June 11th 2020 Rubén Alcaraz-Martínez * https://orcid.org/0000-0002-7185-0227 Universitat de Barcelona Departament de Biblioteconomia, Documentació i Comunicació Audiovisual Melcior de Palau, 140 08014 Barcelona, Spain [email protected] Mireia Ribera-Turró https://orcid.org/0000-0003-1455-1869 Universitat de Barcelona Departament de Matemàtiques i Informàtica Gran Via de les Corts Catalanes, 585 08007 Barcelona, Spain [email protected] Funding This research has been done in the framework of the PhD Programme in Engineering and Information Technology of the Universitat de Lleida (UdL) and has been partially supported by the Spanish project PID2019-105093GB-I00 (Mineco/Feder, UE); Cerca Programme/Generalitat de Catalunya; and by Mineco Grant RTI2018-095232-B-C21 and SGR 1742. Abstract During the Covid-19 pandemic, people rely on the Internet in order to obtain information that can help them understand the coronavirus crisis. This situation has exposed the need to ensure that everyone has access to essential information on equal terms. During this situation, statistical charts have been used to display data related to the pandemic, and have had an important role in conveying, clarifying and simplifying information provided by governments and health organisations. Scientific literature and the guidelines published by organizations have focused on proposing solutions to make charts accessible for blind people or people with very little visual rest. However, the same efforts are not made towards people with low vision, despite their higher prevalence in the population of users with visual impairment. This paper reviews the accessibility of the statistical charts about the Covid-19 crisis for people with low vision that were published by the Brazilian, British, Russian, Spanish, European Union, and the United States’ governments and also by the World Health Organization and Johns Hopkins University, relating to the countries most severely affected by the pandemic. The review is based on specific heuristic indicators, with a mixed quantitative and qualitative approach. Overall, the reviewed charts offer a reasonable level of accessibility, although there are some relevant problems affecting many of the low vision profiles that remain to be solved. The main problems identified are: poor text alternatives in both, raster images and SVG charts; the incompatibility with a keyboard interface; insufficient non-text contrast against adjacent colours (in chart elements such as bars, lines or areas), no customization options; and the lack of an optimized print version for users for whom reading on screen is challenging. Keywords Covid-19; SARS-CoV-2; 2019-nCoV; Coronavirus; Pandemics; Statistical charts; Information visualization; Low vision; Colour vision deficiency; Colour blindness; Web accessibility; Health information; Heuristic evaluation. Rubén Alcaraz-Martínez; Mireia Ribera-Turró e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 2 1. Introduction The Internet has become the main source of information on a wide range of topics, including health. It has acquired an even greater role for citizens in countries where confinement is enforced to prevent the spread of Covid-19; regrettably, the spread of the pandemic has also generalized the dissemination of false information about the virus that can affect health (Salaverría et al., 2020) and can exacerbate the amount of contagion (Pérez-Dasilva et al., 2020); as a consequence, official institutions had an important role to play in communicating reliable data. It is known that adequate dissemination of information about Covid-19 through the Internet can help decrease the spread of the pandemic, and the associated anxiety among the population (Cugelman; Thelwall; Dawes, 2011) and it helps the population better understand the crisis. It is therefore crucial to ensure that everyone has access to rigorous and reliable information on equal terms. A recent report by Nielsen et al. (2020) shows an increase in news consumption in 2020 March compared to previous reports on the same topic (Newman et al., 2019). Among the people reading news, people with disabilities are also avid consumers of Internet news. In particular, ‘low vision users’ refers to people with any visual impairment other than blindness. There are multiple categories of low vision, e.g., the ones defined by governments and organisations such as International Blind Sports Federation (IBSA), and the ones based on different levels of visual acuity (clarity or sharpness of vision) and field of vision (the area in which objects can be perceived with peripheral vision when the eyes are focused on a central point) (National Library of Medicine, 2020). Low vision can also manifest itself in problems related to low sensitivity to light or to contrast as well as related to the so-called colour-blindness or colour vision deficiency (CVD), responsible for the inability to differentiate certain colour combinations. Low vision is the visual impairment with the highest prevalence in the world, affecting 246 million people (World Health Organization, 2012), and this prevalence is higher among the elderly, those most vulnerable to the Covid-19 pandemics. This is a large number of people, much higher than the number of blind people (39 million) and worth taking into account. In Nielsen’s report, the analysis of sources of information chosen by the audience from all surveyed countries reveal multiple information providers, with a significant presence of national governments (41%), national health organisations (35,3%) and global health organisations (29,5%), above other sources of information such as peers, politicians, and only surpassed by news organisations (64,16%). In every country covered by the survey, except for Spain and the United States, a majority of people rate their national government as relatively trustworthy. Additionally, in all the surveyed countries, people rate health authorities and expert sources as highly trusted. There are well known rules on how to make information accessible for low-vision users. For example, the most referred guidelines related to digital accessibility, WCAG 2.1 (Kirkpatrick et al., 2018a), cover different requirements related to low vision. Likewise, the Low Vision Task Force of the W3C has worked on a list of specific accessibility requirements for such user profiles (Allan; Kirkpatrick; Henry, 2019). Although neither of these two documents address statistical charts in particular, their requirements can be easily applied to this type of content. Among the organizations that focus on statistical charts accessibility, the Diagram Center (2015) stands out with its Image description guidelines. However, these guidelines focus accessibility efforts on the needs of blind or severe low vision users. Evergreen (2018) created a data visualization checklist, relying on design principles collected by Evergreen and Metzner (2013), which covers many relevant aspects to accessibility despite doing so with a business and marketing point of view, and without specifically focusing on the accessibility of statistical charts. This checklist has been tested later by Sanjines (2018). Regarding the scientific literature published so far, it is mainly focused on the accessibility of statistical charts for blind people (Alcaraz-Martínez; Ribera-Turró; Granollers-Saltiveri, 2020a). Suggested recommendations focus on one or more of the following four approaches to achieve the accessibility of statistical charts: - the use of textual alternatives, - the sonification of data, - the generation of tactile alternatives, and - the creation of multimodal alternatives. Regarding the use of textual alternatives, but not oriented specifically to charts, but to a broader set of image types, the work of Splendiani (2015) focuses on how to textually describe non-text content for scientific articles. Previously, the analysis of computer science journals conducted by Splendiani and Ribera-Turró (2014) had already shown a deficit in the use of text alternatives, safe colour combinations on the marks of the charts, an insufficient font size, or the use of images with a minimum resolution and dimensions. Simon et al. (2019) show that the most common problem with charts and figures in the proceedings published by the Innovation and Technology in Computer Science Education (ITiCSE) are captions that do not adequately describe the figure and the use of font sizes too small to be readable. On the other hand, authors such as Corio and Lapalme (1999); Chester and Elzer (2005); Elzer et al. (2008); Ferres et al. (2010); Greenbacker (2011); Gao; Zhou and Barner (2012); Nazemi and Murray (2013) or Low vision is the visual impairment with the highest prevalence in the world, affecting 246 million people, and this prevalence is higher among the elderly, those most vulnerable to the Covid-19 pandemics An evaluation of accessibility of Covid-19 statistical charts of governments and health organisations for people with low vision e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 3 De (2018) suggest different methods for the automated generation of textual alternatives from the information available in a chart. For their part, authors such as Elzer et al. (2007) or Yu et al. (2009) have studied the importance of captions for the understanding of a chart as “it often concisely summarizes a paper’s most important results” (Cohen; Wang; Murphy, 2003). In the field of sonification, defined by Kramer (1994) as the representation of data as sound, the mapping of charts to musical tones (Cohen et al., 2005) and vibrations (Evreinova et al., 2008) have been explored, as it has the use of sounds to communicate trends (Alty; Rigas, 2005; Walker; Nees, 2005) or the use of volume, timbre and position, to represent quantitative and qualitative data (Franklin; Roberts, 2003) (Treviranus; Mitchell; Clark, 2018). For its part, the creation of tactile versions of charts and maps has an important tradition, and there are even specific guidelines for its design (Braille Authority of North America, 2012). In the literature we find different approaches for its semi-automated generation. The works of Ladner et al. (2005), Miele and Marston (2005), and Watanabe et al. (2014) are some examples. Finally, other authors opt for multimodality, combining haptic solutions with data sonification and other stimuli (Fritz; Barner, 1999; Yu; Ramloll; Brewster, 2000; Roth et al., 2002; Yu; Brewster, 2003; Iglesias et al., 2004; McGookin; Brewster, 2006; Doush et al., 2009; Goncu; Marriott; Hurst, 2010). To fill in the existing low-vision gap for this type of content, our research team has developed a list of heuristics to cover the needs of low vision users in relation to statistical charts, successfully tested in previous jobs (Alcaraz-Martínez et al., 2020b). This paper reviews the accessibility of the statistical charts about the Covid-19 crisis for people with low vision published by the Brazilian, Russian, Spanish, British, European Union and the United States’ governments. This one complemented by the dashboard of Johns Hopkins University, which has become a de facto “official” information source for the pandemic in the United States, and the World Health Organization (WHO). 2. Research method For this research a list of heuristics created in previous work (Alcaraz-Martínez; Ribera-Turró; Granollers-Saltiveri, in review) is used to evaluate the charts. In this study, the heuristic H15 (without disturbing elements) has not been applied, as the websites analysed do not contain advertising, which was the most common disturbing element. Previously, the heuristics were scored with the 7-point Likert scale, but in this study the scale has been replaced by a 5-point Likert scale to ease the evaluation process. The criteria used to weight up scores, the definition of the heuristic indicators, and the scores themselves are shown in Tables 1, 2, and 3 respectively. Table 1. Weighting criteria Criteria Weight If the chart fails the heuristic, one or more user profiles will not have a satisfactory user experience with the chart, mildly compromising its accessibility. If the chart succeeds at the heuristic the chart’s accessibility slightly improves. x1 If the chart fails the heuristic, one or more user profiles will have serious difficulties to perceive the chart information, severely compromising its accessibility. If the chart succeeds at the heuristic the chart’s accessibility considerably improves. x2 If the chart fails the heuristic, one or more user profiles will not be able to perceive the chart information, totally compromising its accessibility. This heuristic is key to provide access to the chart for one or more user profiles. x3 The final score for each chart is represented as a number between 0 and 10 out of 10. It is calculated by grading each indicator between 0 and 4 except the not applicable indicators or those where failure is not a problem, according to the Likert scale. The grade is multiplied by the weight of the indicator. All the weighted indicators are summed and multiplied by 10 and the result is divided by the maximum possible score. The formula is shown below: Authors evaluated the selected charts based on the above-mentioned list of heuristics. The evaluation of each statistical chart is recorded in an ad hoc-made template1 which automatically calculates the final score. The template includes a field for the evaluator to comment on each indicator, in each evaluated chart. Comments are of special interest since, far from seeking to give a final score, heuristic evaluations pursue, above all, the identification of the accessibility problems. An adequate dissemination of information about Covid-19 through the Internet can help decrease the spread of the pandemic, and the associated anxiety among the population (Cugelman; Thelwall; Dawes, 2011) and it helps the population better understand the crisis. It is therefore crucial to ensure that everyone has access to this important information on equal terms Rubén Alcaraz-Martínez; Mireia Ribera-Turró e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 4 The selection of charts was done selecting the five countries most affected by the Covid-19, that is, with the highest number of infections and evaluating all different available charts in each source: USA (2 charts), Brazil (16 charts), Russia (6 charts), United Kingdom (7 charts) and Spain (4 charts) in descending order. Official government sources in these countries were identified, and the European Union (9 charts), as a supra governmental entity covering Spain and UK was also included. Finally, the study also covered the World Health Organization (WHO) (6 charts) for its relevance in global health. Priority was given to official and government entities according to the preferences of the citizens collected in the report by Nielsen et al. (2020), however, since Johns Hopkins University (4 charts) has become a de facto “official” information source in the USA and it is highly referenced in many Covid-19 sources (as in Cobarsí-Morales, 2020), it has also been included. In total, the sample has 54 charts. The URLs of the analysed charts and a web gallery with screenshots of each one to guarantee their consultation over time are offered in Annex 1. The evaluation process took place between April 13 and April 21, 2020 (Spain, Europe, WHO, USA and Johns Hopkins), and June 3 to June 5, 2020 (Brazil, Russia, and United Kingdom). Both evaluators worked independently, and a spreadsTable 2. Heuristic indicators ID Name Heuristic Target user profile Weight H1 Title Does the chart have a brief and descriptive title that helps users identify it among others appearing on the same page, as well as navigate between them? Any user x1 H2 Legend If the chart uses shapes, colour or patterns encodings is there a legend to decodify them? Any user x1 H3 Axes If the chart needs axes, are they visible and have appropriate, concise and clear labels and titles? Any user x1 H4 Caption Does the chart have a caption helping understand it? Any user x1 H5 Abbreviations Are all the abbreviations in the chart expanded? Any user x1 H6 Data source Does the chart include information about its source (institution, date and URL of dataset)? Any user x1 H7 Print version Is there an optimized version for printing available? Any user x1 H8 Short text alternative Does the chart provide a text alternative that briefly informs about its contents and helps users decide if they want more information? Screen reader users x1 H9 Long description In case the text alternative does not adequately convey the information provided by the chart, does the chart provide a textual long description containing complete and structured information about the data? Screen reader users x3 H10 Safe colours If the chart uses colours to provide information, is the colour scheme safe for the different types of colour vision deficiencies, including achromatopsia (total absence of colour vision)? Low vision and CVD users x3 H11 Contrast Does the visual presentation of text and background have a contrast ratio of at least 4.5:1, and the non-text elements of the chart a contrast ratio of at least 3:1? Low contrast sensitivity users x3 H12 Legibility Is the text included in the chart legible (sans-serif font, font size of at least 16px or 12pt, line spacing of at least 1.5, no abuse of capital letters, bold or italics)? Low vision users x2 H13 Image quality If the chart is provided as a bitmap image, does the image have sufficient quality for a clear visualization and does it support a zoom of at least 200% without blurring or pixelation? Low vision users x3 H14 Resize Can the chart be zoomed up to 200% without an assistive tool and without loss of content or functionality? Low vision users x2 H16 Focus visible When an element of the chart (lines, bars, points...) receives the focus, is there a visual indication of it? Low vision users x1 H17 Device independent navigation Is it possible to navigate between the marks and elements of the chart with keyboard, mouse and gestures? Low vision and screen reader users x3 H18 Customization Is it possible to customize the chart (colour scheme, contrast, typography...) with assistive technologies or with a resource-specific customization system? Low vision, low contrast sensitivity and screen reader users x2 Table 3. Scores Likert scale Score Level of compliance - Not applicable (NA) - Failure is not a problem (NP) 0 No compliance 1 Low compliance 2 Acceptable compliance 3 High compliance 4 Excellent compliance An evaluation of accessibility of Covid-19 statistical charts of governments and health organisations for people with low vision e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 5 heet was used to compare the individual scores and to compute standard deviation between the two evaluators. Scores with a standard deviation above 2 (in total 14 with 2.12, and 10 with 2.82 out of 918 total scores) were discussed in depth in order to ensure that both evaluators shared an understanding of the criteria and had not overlooked any aspect. Figure 1 shows a diagram of the process. The authors of this research have expanded on the general accessibility problems found per provider to indicate their severity, with a qualitative heuristic evaluation as described in discount usability methods, by Nielsen (1995). To determine the severity of each of the problems, Nielsen suggests considering the frequency at which the problem occurs, and the impact of the problem if it occurs. This evaluation complements the above-mentioned scores and includes contextual information which provides a broader insight into the problem, which is difficult to include in a chart by chart evaluation. Qualitative comments were also tabulated to compute their impact and frequency (see tables 4 and 5). For each detected accessibility problem, a description, a recommendation for improvement, the list of related heuristics and the impact and frequency values of each organization are provided. The same practice is carried out with the detected accessibility features. The authors have used the criteria shown in Table 4 to rate the frequency of occurrence and the impact of the accessibility problems of each provider evaluated in the qualitative evaluation. Table 5. Impact rating scale Impact value Formula Criteria No impact 0 The problem does not affect the use of the chart Low impact 0.25 x heuristic indicator weight value2The problem affects the ease of use of the chart Moderate impact 0.5 x heuristic indicator weight value The problem hinders the use of the chart, making it cumbersome to interact with it. Serious impact 0,75 x heuristic indicator weight value The problem prevents the user to access some of the functionality / information in the chart Critical impact 1 x heuristic indicator weight value The problem prevents the user to access the chart The quantitative and qualitative results are presented in the results section and can be accessed online.3 3. Results Overall, the reviewed charts offer a reasonable level of accessibility. In particular, the contextual information related to the charts, such as titles, axes, and the associated data are sufficient in most cases. All charts have a title, although a small percentage of the titles (3 of 25) are not informative enough. The non-informative title problem has only been detected in the charts made by the Spanish Government and the World Health Organization, and with a frequency of no more than 25% in both cases. Titles help to quickly navigate through the charts of a page or a dashboard in order to focus on the item most relevant to the user, and are particularly relevant to low-vision users, who strive to visually process each chart. However, they do not have a great impact on the final accessibility and ease of use of the analysed charts, since there are many contextual clues available on the web pages featuring the charts. Moreover, the contextual clues help diminish the negative impact of specific captions missing from all of the charts. The information provided in the axes is clear and complete in all but one of the charts. Unfortunately, the axes labels in some of the charts have a vertical or diagonal orientation which hinders the readability, especially with magnified screens. This problem occurs in all (100% frequency) charts published by the Centers for Disease Control and Prevention Table 4. Frequency rating scale Frequency value Criteria 0 The problem never occurs 1 The problem occurs in up to 25% of the charts 2 The problem occurs in up to 50% of the charts 3 The problem occurs in up to 75% of the charts 4 The problem occurs in 75% or more of the charts Determine the set of heuristics Perform the heuristic evaluation Selection of the sample Analyse results Quantitative results Detected accessibility problems Figure 1. Diagram of the process Rubén Alcaraz-Martínez; Mireia Ribera-Turró e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 6 of the United States of America, by the Brazilian Government and by the European Commission. The overall impact of the axes labels has been considered moderate. Regarding the incorporation of the data source, the reviewed charts are made by the same organizations that create or collect the visualized data, and therefore identifying the source is a simple task. In many cases data is also offered as an HTML table or available in XLSX or CSV formats. This allows low-vision users to analyse the specific numbers in detail with their preferred tools or to conduct searches for a datum. Also, since in this particular case the publication date is of vital importance, all of the analysed charts include it. Yet, there are some remaining shortcomings in the charts, which may affect users belonging to several of the low vision profiles. The most important shortcomings include colour and contrast, the lack of keyboard navigation, textual alternatives or print version. None of the interactive charts can be navigated with a keyboard and some present certain difficulties with touch gestures on mobile devices. This severely limits accessibility for low vision users who do not use the mouse as the main interaction device, but also for other groups of users who rely on keyboards when navigating web pages. For instance, all the Spanish charts are bitmaps and thus no interaction with them is possible. People with CVD will have problems distinguishing values on charts with more than 6 colour categories, and people with achromatopsia (able to see light variations, but not hue) will have problems with almost all Covid-19 charts. For those who have lost acute perception of light and need enhanced contrast (as many elderly people do), only 4 of the 25 analysed charts are perceivable; this barrier also applies to those intending to print the chart in black and white. A low nontext contrast is a problem that occurs in up to 50% of the charts published by Centers for Disease Control and Prevention of the North American Government and those by Johns Hopkins University, in up to 75% of the charts published by the Spanish Government and by the Russian Government, 100% of the charts by WHO, by the Brazilian Government, by the UK Government, and the European Commission The above-mentioned limitations related to colour schemes could be solved through customization, but no customization options were offered in any of the charts. Only those users relying in assistive technology would be able to partially solve this issue. Those users complementing their access with a screen reader will feel unsupported by the majority of charts as there are no textual alternatives to the visual information. In fact, the evaluation results show a frequency of 100% regarding the absence of sufficiently informative and useful short textual alternatives. Only the charts from the Spanish Government (3 out of 4) offer a short alternative, but very unhelpful and uninformative. Interactive charts don’t offer WAI-ARIA (Accessible Rich Internet Applications) attributes or any other textual help. The impact has been classified as low since, after all, a complex image such as a chart requires long descriptions as a complete text alternative, with short text alternatives being only a complement to the first ones (W3C, 2019). As discussed above, some charts (24 out of 54) provide a data table that could partially fulfil the purpose of a long description; this is not a complete solution for some users, as for example severe low vision users, because there is no direct link from the chart to the table and therefore the impact has been classified as high. Concerning the print version, none of the analysed charts offered a quality print version and the default print options cut them between pages, or partially hide them. A remarkable exception is found in the Spanish Government charts, which offer a high-resolution version that can be printed with assurances. The lack of a quality print version is not a consequence of the authoring tools, as it is demonstrated by the complementary infographics offered by Johns Hopkins University for every state. Reading on screen may introduce additional difficulties for some low vision users; it is common for these users to read from a very short distance from the screen which means a very harsh position causing fatigue. However, the impact of this problem is classified as low because the absence of this alternative only affects the ease of use of the chart in certain contexts and for some user profiles, but does not prevent access to any of the profiles, not even those that have a more severe low vision. The two organizations obtaining best results are from the United States (Centers for Disease and Prevention and Johns Hopkins University), followed by the European Centre for Diseases Prevention and Control, all with above-average results. The governments of the United Kingdom, Spain, Russia and Brazil, along with the World Health Organization, are below average. Exact values obtained by each government and organization are shown in figure 2. It is worth mentioning that the overall accessibility of the charts depends also largely on the accessibility of the web page containing them, and in this aspect the United Kingdom Government websites stand out among the rest. Finally, the reader must take into account that although the weighted results of the heuristic evaluation are shown over 10, an average score of 5 does not imply that a chart is accessible. The great variability within low vision profiles means that, for example, not fulfilling the requirements of contrast or safe colours results in not perceivable charts for many users. For this reason, the qualitative approach needs to complement the quantitative analysis. In the case of accessibility, some simple solutions that are already existing must be implemented urgently An evaluation of accessibility of Covid-19 statistical charts of governments and health organisations for people with low vision e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 7 Qualitative results show that the main problems are the use of images of text (critical impact); insufficiently descriptive titles (low impact); axes without titles (low impact); charts that do not have a caption (low impact); no versions optimized for printing (low impact) and print versions that are cut off (low impact); charts without short alternative texts (low impact); no long descriptions (serious impact); the use of unsafe colour palettes for people with CVD (critical impact); the use of adjacent colours with insufficient contrast ratios (critical impact); the lack of customization options (critical impact); charts not operable to a keyboard interface (moderate impact), the use of a font size below the recommended (low impact) and the inclusion of text with a vertical or diagonal orientation (low impact). The use of images of text only occurs in the charts of the Spanish Government and with a frequency of 75% or more. The use of insufficiently descriptive titles only happens in the case of the Spanish Government, the WHO and Russian Government, with a frequency that does not reach 25% of the cases. The axes without titles are given in the cases of the Spanish Government (75% or more), European Commission (up to 25%) and Russia and the United Kingdom governments (75% or more). The use of unsafe colour combinations for people with CVD occurs in up to 25% of cases in charts from Spain and Johns Hopkins University, in up to 50% of cases in the charts from the WHO, USA, Brazil, Russia and UK, and in up to 75% of cases in the charts of the European Union. Insufficient colour contrast ratios are given with up to 50% frequency in the case of USA, Johns Hopkins University and Russia, with up to 75% frequency in the case of Spain, and with a frequency of 75% or more in the cases of the WHO, the European Union, Brazil and the UK. The use of diagonal or vertical texts occurs in up to 75% of cases in Brazil, and in 75% or more in the USA, the European Union and the UK. Finally, the lack of captions, lack of alternative texts and long descriptions, not having print optimized versions, the use of size below the recommended, the inaccessibility through a keyboard interface and the impossibility of customization occurs in all the analysed dashboards, 75% or more of the times. On the other hand, some positive accessibility features have also been observed, which stand out for their impact on the final accessibility of the charts: the use of SVG charts, which can be resized without losing quality; the inclusion of downloadable structured data allowing users to review it with their favourite software applications; a structured data table complementing the chart that allows detailed data consultation; the possibility to enlarge the graphics to full screen; and the use of popups that provide additional information about the values of each mark (bar, point, etc.) when they receive the focus. The full results of the qualitative analysis are available online, together with the details of the quantitative results.2 4. Discussion While all the charts analysed here have been published by large and mainly public institutions, which are required to comply with legislation on accessibility, most of them still fail to comply with the legal requirements. One reason for this could be that those charts were created in a hurry –due to the rapid spread of the pandemic–, and therefore complying with accessibility rules was not prioritized. However, the analysed organizations and governments were already equipped with a powerful infrastructure to generate visualizations and were using them regularly, so the Covid-19 charts did not require many alterations to the existing procedures. Another possible explanation is that the charts were created with ad hoc software or with authoring tools not committed to accessibility (Richards; Spellman; Treviranus, 2015). This explanation is not likely either, since all the organisations created dynamic visualizations with established JavaScript libraries except for Spain. Some examples of the software are Qlik4 (European Union), Leaflet5 (WHO), Epi Info6 (CDC), plotly.js7 (Johns Hopkins University) or Chart.js (Brazil and United Kingdom)8 and most of this software states it can create accessible charts. Accessible charts should at least include accessible colour combinations, keyboard navigation and it can easily cover customization options, however, many of these features were not implemented, thus preventing many users from benefiting from the charts’ information. Two factors seem to influence the accessibility of the analysed charts: on the one hand, technology and on the other, the chart creators’ knowledge of accessibility requirements. Both need to go hand in hand to obtain charts with built-in Figure 2. Average score by organism of the evaluations carried out Rubén Alcaraz-Martínez; Mireia Ribera-Turró e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 8 quality and accessibility. If these factors don’t work together we may find paradoxes such as the following one: WHO, with a very advanced visualization infrastructure and using web and JavaScript technologies with the potential to create very versatile charts for all needs, gets a lower score, on average, than the Spanish Government, which uses bitmap static charts, less flexible in origin. Some flaws identified in the heuristic evaluation, such as the absence of useful text alternatives, the non-use of safe colours and sufficient colour contrast, or the absence of captions that help the reader to better understand the charts, coincide with those detected by other authors such as Splendiani and Ribera-Turró (2014) or Simon et al. (2019) in their respective investigations. Others are not described in any previous research and represent a big contribution to the existing literature, this is due to that low vision users’ needs have been largely overlooked in accessibility studies, as mentioned in the introduction. In any case, this research has exposed the lack of interest in guaranteeing universal access to information from some of the most important organisations. Accessibility has not been prioritized in the design of these charts, despite significant effort was clearly made to solve other (often more challenging) problems related to Covid-19 such as privacy, where many resources have been destined to create privacy-friendly tracing apps. In the case of accessibility, some simple solutions that are already existing must be implemented urgently. These solutions include, but are not limited to, the use of textures or patterns, to solve problems caused by colours in line and bar charts; the use of darker outlines (Campbell; Cooper; Kirkpatrick, 2018) to discern the edges of the slices in pie charts or the bars in bar charts, a solution to achieve the required contrast level, keyboard access, which is a programmatic issue with well-known and documented solutions, and customization of the appearance of the charts, easily achieved with CSS or filters. 5. Limitations The average values per organization shown in the results must be taken with caution as there were different number of charts available per organization, ranging from 16 in Brazil to 2 in USA. Although the selected sample is still limited and the evaluation was made only by two evaluators, the results of the heuristic evaluation gives a broad view of common practices and the most frequent accessibility problems in the visualizations created by governments and international organizations in the field of health. 6. Conclusions The pandemic evolves every day so showing updated data is very relevant and necessary to better understand the evolution of the Covid-19 crisis. Making this information accessible is critical to ensure the rights of the citizens with disabilities and of elders, taking into account that especially elders are among the vulnerable groups. While the focus of this work is low vision users, any improvement in digital accessibility brings benefits to everyone, particularly elderly people, users with contextual disabilities or users with situational limitations. In particular, many of the suggested heuristic indicators also benefit other user profiles such as blind people (H8 alt text, H9 long descriptions and H17 device independence), people with motor disabilities (H17 device independence), or people with learning or cognitive disabilities (H4 caption, H5 abbreviations, H7 print version, H11 contrast, H12 legibility, H16 visible focus and H18 customization). This article has highlighted issues never included in other studies. Low vision users deserve more attention in accessibility studies. We still have a long way to go in digital accessibility, and it is mainly related to awareness and responsibility. 7. Notes 1. http://www.ub.edu/adaptabit/covid-research/template.xlsx 2. If several heuristic indicators are involved, the weight is computed as the average. 3. Quantitative results available at: http://www.ub.edu/adaptabit/covid-research/heuristic_evaluation_quantitative_analysis.xlsx Qualitative results available at: http://www.ub.edu/adaptabit/covid-research/heuristic_evaluation_qualitative_analysis.xlsx 4. https://www.qlik.com 5. https://leafletjs.com 6. https://www.cdc.gov/epiinfo 7. https://plotly.com/javascript 8. https://www.chartjs.org We have a long way to go in digital accessibility, and it is mainly due to awareness and responsibility An evaluation of accessibility of Covid-19 statistical charts of governments and health organisations for people with low vision e290514 Profesional de la información, 2020, v. 29, n. 5. e-ISSN: 1699-2407 9 8. References Alcaraz-Martínez, Rubén; Ribera-Turró, Mireia; Granollers-Saltiveri, Toni (2020a). “La accesibilidad de los gráficos estadísticos para personas con baja visión y visión cromática deficiente: revisión del estado del arte y perspectivas”. Interacción: revista digital de AIPO, v. 1, n. 1. http://revista.aipo.es/index.php/INTERACCION/article/view/9 Alcaraz-Martínez, Rubén; Ribera-Turró, Mireia; Granollers-Saltiveri, Toni; Pascual, Afra (2020b). “Accesibilidad para personas con baja visión de los gráficos estadísticos en la prensa digital: una propuesta metodológica basada en indicadores heurísticos”. Profesional de la información, v. 29, n. 5, e290515. https://doi.org/10.3145/epi.2020.sep.15 Alcaraz-Martínez, Rubén; Ribera-Turró, Mireia; Granollers-Saltiveri, Toni (in review). “Methodology for heuristic evaluation of the accessibility of statistical charts for people with low vision andcolor vision deficiency” (in review). Allan, Jim; Kirkpatrick Andrew; Henry, Shawn-Lawton (2019). Accessibility requirements for people with low vision, W3C. https://w3c.github.io/low-vision-a11y-tf/requirements.html Alty, James L.; Rigas, Dimitrios (2005). “Exploring the use of structured musical stimuli to communicate simple diagrams: the role of context”. International journal of human-computer studies, v. 62, n. 1, pp. 21-40. https://doi.org/10.1016/j.ijhcs.2004.08.003 Braille Authority of North America (2012). Guidelines and standards for tactile graphics. http://brailleauthority.org/tg/web-manual Campbell, Alastair; Cooper, Michael; Kirkpatrick, Andrew (2018). “Understanding success criterion 1.4.11: non-text contrast”. In: Understanding WCAG 2.1. W3C. https://www.w3.org/WAI/WCAG21/Understanding Chester, Daniel; Elzer, Stephanie (2005). “Getting computers to see information graphics so users do not have to”. In: Proceedings of the 15th Int’l Symposium on methodologies for intelligent systems (Ismis), pp. 660-668. https://doi.org/10.1007/11425274_68 Cobarsí-Morales, Josep (2020). “Covid 19: fuentes de información cuantitativa”. Anuario ThinkEPI, v. 14, e14d02. https://doi.org/10.3145/thinkepi.2020.e14d02 Cohen, William W.; Wang, Richard; Murphy, Robert F. (2003). “Understanding captions in biomedical publications”. In: Proceedings of the 9th ACM SIGKDD International conference on knowledge discovery and data mining, pp. 499-504. https://wwcohen.github.io/postscript/ismb-2003.pdf Cohen Robert F.; Yu, Rui; Meacham, Arthur; Skaff, Joelle (2005). “Plumb: displaying graphs to the blind using an active auditory interface”. In: Proceedings of the 7th international ACM sigaccess conference on computers and accessibility, pp. 182-183. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.487.5932&rep=rep1&type=pdf Corio, Marc; Lapalme, Guy (1999). “Generation of texts for information graphics”. In: Proceedings of the 7th European workshop on natural language generation (Ewnlg’99), pp. 49-58. http://rali.iro.umontreal.ca/rali/sites/default/files/publis/CorioEWNLG.pdf Cugelman, Brian; Thelwall, Mike; Dawes, Philip L. (2011). “Online interventions for social marketing health behavior change campaigns: a meta-analysis of psychological architectures and adherence factors”. Journal of medical internet research, v. 13, n. 1, e17. https://doi.org/10.2196/jmir.1367 De, Paramita (2018). “Automatic data extraction from 2D and 3D pie chart images”. In: Proceedings of the 8th International advance computing conference (IACC 2018), pp. 20-25. https://doi.org/10.1109/IADCC.2018.8692104 Diagram Center (2015). Image description guidelines. http://diagramcenter.org/table-of-contents-2.html Doush, Iyad-Abu; Pontelli, Enrico; Simon, Dominic; Son, Tran-Cao; Ma, Ou (2009). “Making Microsoft Excel™ accessible: multimodal presentation of charts”. In: Proceedings of the 11th international ACM Sigaccess conference on computers and accessibility. New York, NY: ACM, pp. 147-154. https://doi.org/10.1145/1639642.1639669 Elzer, Stephanie; Carberry, Sandra; Chester, Daniel; Demir, Seniz; Green, Nancy; Zukerman, Ingrid; Trnka, Keith (2007). “Exploring and exploiting the limited utility of captions in recognizing intention in information graphics”. In: Proceedings of the 43rd Annual meeting of the Association for Computational Linguistics (ACL’05), pp. 223-230. https://www.researchgate.net/publication/220874350_Exploring_and_Exploiting_the_Limited_Utility_of_Captions_ in_Recognizing_Intention_in_Information_Graphics