The Vaccination Concerns in COVID-19 Scale (VaCCS) : Development and validation

This is a self-archived version of an original article. This version may differ from the original in pagination and typographic details. Author(s): Title: Year: Version: Copyright: Rights: Rights url: Please cite the original version: CC BY 4.0 https://creativecommons.org/licenses/by/4.0/ The Vaccination Concerns in COVID-19 Scale (VaCCS) : Development and validation © 2022 the Authors Published version Hamilton, Kyra; Hagger, Martin S. Hamilton, K., & Hagger, M. S. (2022). The Vaccination Concerns in COVID-19 Scale (VaCCS) : Development and validation. PLoS ONE, 17(3), Article e0264784. https://doi.org/10.1371/journal.pone.0264784 2022 RESEARCH ARTICLE The Vaccination Concerns in COVID-19 Scale (VaCCS): Development and validation Kyra Hamilton 1,2,3 *, Martin S. HaggerID 1,3,4,5 * 1School of Applied Psychology, Griffith University, Brisbane, Australia, 2Menzies Health Institute Queensland, Griffith University, Gold Coast, Australia, 3Health Sciences Research Institute, University of California, Merced, California, United States of America, 4Psychological Sciences, University of California, Merced, California, United States of America, 5Faculty of Sport and Health Sciences, University of Jyva ¨skyla ¨, Jyva ¨skyla ¨, Finland *[email protected] (MSH); [email protected] (KH) Abstract Vaccines are highly effective in minimizing serious cases of COVID-19 and pivotal to managing the COVID-19 pandemic. Despite widespread availability, vaccination rates fall short of levels required to bring about widespread immunity, with low rates attributed to vaccine hesitancy. It is therefore important to identify the beliefs and concerns associated with vaccine intentions and uptake. The present study aimed to develop and validate, using the AMEE Guide, the Vaccination Concerns in COVID-19 Scale (VaCCS), a comprehensive measure of beliefs and concerns with respect to COVID-19 vaccines. In the scale development phase, samples of Australian (N= 53) and USA (N= 48) residents completed an initial open-response survey to elicit beliefs and concerns about COVID-19 vaccines. A concurrent rapid literature review was conducted to identify content from existing scales on vaccination beliefs. An initial pool of items was developed informed by the survey responses and rapid review. The readability and face validity of the item pool was assessed by behavioral science experts (N= 5) and non-experts (N= 10). In the scale validation phase, samples of Australian (N= 522) and USA (N= 499) residents completed scaled versions of the final item pool and measures of socio-political, health beliefs and outcomes, and trait measures. Exploratory factor analysis yielded a scale comprising 35 items with 8 subscales, and subsequent confirmatory factor analyses indicated acceptable fit of the scale structure with the data in each sample and factorial invariance across samples. Concurrent and predictive validity tests indicated a theoretically and conceptually predictable pattern of relations between the VaCCS subscales with the socio-political, health beliefs and outcomes, and trait measures, and key subscales predicted intentions to receive the COVID-19 vaccine. The VaCCS provides a novel measure to assess beliefs and concerns toward COVID-19 vaccination that researchers and practitioners can use in its entirety or select specific subscales to use according to their needs. PLOS ONE PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 1 / 33 a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Citation: Hamilton K, Hagger MS (2022) The Vaccination Concerns in COVID-19 Scale (VaCCS): Development and validation. PLoS ONE 17(3): e0264784. https://doi.org/10.1371/journal. pone.0264784 Editor: Camelia Delcea, Bucharest University of Economic Studies, ROMANIA Received: November 30, 2021 Accepted: February 16, 2022 Published: March 14, 2022 Peer Review History: PLOS recognizes the benefits of transparency in the peer review process; therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. The editorial history of this article is available here: https://doi.org/10.1371/journal.pone.0264784 Copyright: ©2022 Hamilton, Hagger. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability Statement: Data files and analysis output related to the research are available online: https://osf.io/k96bn/. Funding: The author(s) received no specific funding for this work. Introduction COVID-19 infections have had substantive global social, economic, and health impacts, and contributed considerably to excess deaths worldwide [1–3]. In response, governments have imposed restrictions on movement and gatherings, and introduced other preventive measures (e.g., mandatory face mask wearing, imposing physical distancing rules, encouraging hand hygiene practices) to minimize infection transmission [4,5]. The rapid development of highly efficacious COVID-19 vaccines and mass mobilization to distribute them has assisted in reducing infection rates, cases needing hospitalization, and bringing the pandemic under control [6,7]. However, success of COVID-19 vaccination programs is highly dependent on high uptake rates among the population to build widescale immunity. Modeling data suggests that rates of vaccination of above 80% are required for exponential reductions in infection rates and, particularly, numbers of serious cases, hospitalizations, and deaths due to COVID-19 [8, 9]. Based on these data, some governments have begun to ease coronavirus restrictions and reduce requirements to engage in preventive measures contingent on high rates of vaccination uptake [10–12]. While there has been considerable uptake of the COVID-19 vaccine in countries where it has been offered to all adults such as Australia, Singapore, UK, and many European Union countries, the world population rate of fully vaccinated individuals is less than 40%. In addition, countries such as USA, Russia, and South Africa are witnessing a slowing of vaccine uptake [13], and many programs are running out of individuals willing to be vaccinated as a consequence [14,15]. One contributing factor may be vaccine hesitancy, which represents a psychological state of indecision with respect to getting vaccinated [16]. Research and media reports have indicated considerable hesitancy with respect to COVID-19 vaccines in many populations and it has been identified as a salient contributor to reduced rates of vaccine uptake [17–20]. The issue is compounded by increased attention being given to ‘antivax’ groups and conspiracy theorists in the popular media who propagate misinformation and misperceptions about the efficacy, safety, and side effects of the COVID-19 vaccines [21–23]. Vaccine hesitancy in general, and specific concerns and false beliefs with respect to the COVID-19 vaccines, have considerable potential to stymie vaccination program effectiveness. In addition to vaccine hesitancy in general, other beliefs (e.g., social and moral norms) [24, 25] and concerns (e.g., trust in government) [26] specific to COVID-19 may also contribute to intentions to receive, and actual uptake of, COVID-19 vaccines. Furthermore, vaccine hesitancy itself is likely to have multiple belief-based determinants [27]. This has compelled research examining these beliefs and the extent to which they account for unique variance in COVID-19 vaccination intentions. Knowledge of such beliefs, and their links to vaccination intentions, is valuable because it will inform the development of messaging included in interventions developed by organizations tasked with maximizing vaccine coverage of COVID-19 vaccine programs. This is especially important given the emergence of new strains of the SARS-CoV-2 virus that causes COVID-19, and the potential of the need for people to receive vaccine booster shots in the face of waning immunity, both of which will necessitate ongoing vaccination programs and accompanying advocacy [28–30]. One limitation of research examining relations between vaccine hesitancy, other vaccinerelated beliefs and concerns, and vaccine intentions in the context of the COVID-19 pandemic is the relative dearth of measures with good psychometric properties and adequate construct and concurrent validity. While some measures of vaccine hesitancy and intentions, and other vaccine-related beliefs more broadly, have been developed in the context of COVID-19 or adapted from other measures [29], there is, to date, no validated measure that captures a broad range of beliefs and concerns that would be expected to be associated with COVID-19 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 2 / 33 Competing interests: The authors have declared that no competing interests exist. vaccination intentions and uptake. Current measures tend to focus on one or a narrow range of factors such as confidence, trust, or vaccine literacy, and neglect the fact that vaccine hesitancy is likely to be a function of multiple beliefs and intrapersonal factors [27,31]. With respect to vaccine hesitancy, conceptual work and theory suggests that it has multiple determinants such as socio-political beliefs, particularly generalized distrust in authority and medical science [32,33]; generalized traits which include personality and individual difference constructs that reflect generalized tendencies that impact behavior and decision-making [30, 34–36]; and health-related beliefs and side effect concerns, particularly in terms of the rapid development of COVID-19 vaccines and their authorization for emergency use [37,38]. Effects of these determinants on COVID-19 vaccine hesitancy and vaccine intentions, are likely to be heightened or exacerbated by contextual factors including the rapid development of the vaccine and the high profile of vaccine scepticism in media outlets particularly those with populist, right-leaning political perspectives. Specifically, the expedited process of vaccine development, a process more rapid than any vaccine in history [39–41], is likely to contribute to concerns among the general public with respect to safety. Concerns over a perceived lack of rigor or extensiveness of trials to test long-term effectiveness, or even a perception that developers ‘cut corners’, may contribute to vaccine concerns. This is likely to be exacerbated by misinformation regarding vaccine development and as well as high profile publicised cases of side effects or associations with nosocomial conditions [42,43]. Alongside this, the rise of a populist political agenda that is generally sceptical of science and perceives vaccination as governmental interference and overreach, is also likely to heighten concerns relating to COVID-19 vaccine safety and rigor in development [41,44]. In addition, high-profile vaccine-sceptic personalities and influencers in right-wing media outlets and social media platforms also model vaccine hesitancy and convey an aura of credibility to misinformation on the COVID-19 vaccine to a wide spectrum of the general public [22,45]. These contextual factors are likely to magnify concerns among individuals pre-disposed to be sceptical and to whom right-wing populist beliefs have the most appeal, particularly those with traits such as right-wing authoritarianism and social dominance orientation. Taken together, these factors specific to COVID-19 make it a special case with respect to beliefs and concerns about the vaccine, potentially contributing to the slowing rates of vaccination uptake observed in this context. Given the multifactorial nature of concerns surrounding the COVID-19 vaccines, there is a need for a comprehensive measure, which also demonstrates good psychometric properties with sets of constructs representing these specific issues (i.e., socio-political beliefs, personality and individual difference constructs, and health-related beliefs and outcomes), that can inform future intervention design and evaluation for COVID- 19 vaccination uptake. The present study To date, there is no evidence-based measure that captures sets of beliefs and concerns that are expected to relate to COVID-19 vaccination intentions. This study addresses this evidence gap by developing and validating a comprehensive measure that captures these concerns, the Vaccination Concerns in COVID-19 Scale (VaCCS). The VaCCS will provide researchers and practitioners with a novel measure to assess beliefs and concerns toward COVID-19 vaccination and assist in identifying the determinants of vaccination intention and uptake, informing the development of messaging and interventions that may promote vaccination. Importantly, our approach to validation is intended to produce a scale that is flexible to use such that researchers and practitioners can use the scale in its entirety or select specific sub-scales to use according to their needs. PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 3 / 33 We followed a systematic, multi-step process to develop and validate the VaCCS [46], which was expected to yield a final scale comprising of sets of items representing multiple COVID-19 vaccine concerns and belief dimensions. We expect concurrent and predictive validity tests to indicate a conceptually predictable pattern of relations between the VaCCS subscales with the socio-political, trait measures, and health beliefs and outcomes. Specifically, we anticipate that the scale will capture beliefs and concerns including, but not limited to: concerns with respect to side effects and concerns over safety attributed to a perceived lack of testing and the rapid development of COVD-19 vaccines, conspiracy beliefs relating to this and other vaccines and about how COVID-19 emerged, generalized fear of vaccines, distrust in the government and particularly the pharmaceutical companies that have developed the vaccines, uncertainty over the outcomes of the COVID-19 pandemic, and general knowledge and understanding of how vaccines work. We therefore expect individuals with higher concerns in these areas to be more likely to hold political views and express general distrust in government, to have higher concerns about vaccines in general and concerns over medicines, lower risk perceptions with respect to COVID-19, and also greater trait-levels of neuroticism and lower conscientiousness. Method Participants and recruitment We followed a systematic, multi-step design process using the AMEE Guide [46] to develop the VaCCS. The AMEE Guide presents a seven-step survey scale design process broadly consisting of a development phase (steps 1–6) and a validation phase (step 7) (see [46] for details). In the scale development phase, samples of Australian (N= 53; M Age = 44.45, SD Age = 19.57, 36% female) and USA (N= 48; M Age = 36.95, SD Age = 12.64, 58% female) residents were recruited via a research panel company to complete an online open response survey. To be eligible for inclusion, participants were required to be aged 18 years or older and have not received a COVID-19 vaccine. Participants received modest compensation for their participation based on expected completion time consistent with the panel company’s published rates. In the final phase of scale development, to assess readability and face validity of scale items, a convenience sample of 10 laypersons (M Age = 39.00, SD Age = 18.86, 50% female) and 5 experts (4 females; comprising clinical and health psychologists, behavior change scientists, and a vaccination researcher) were recruited. For the subsequent scale validation phase, samples of Australian (N= 522, 60.7% female) and USA (N= 499, 70.7% female) residents were recruited via an online research panel company. To be eligible for inclusion, participants were required to be aged 18 years or older and not having received a COVID-19 vaccine. Participants were not stratified by specific demographic variables as our primary goal was to recruit participants who had not yet been vaccinated so we opted to be more inclusive in the recruitment phase. Data were collected between May 14 and May 28, 2021. At the time of data collection, only Australian residents aged 50 years and over, Aboriginal and Torres Strait Islander peoples aged 18 years and older, those with underlying medical conditions, and those whose employment placed them at high risk of contracting or spreading COVID-19 were eligible to get the COVID-19 vaccine [47]. By contrast, USA residents aged 18 years and older were eligible. We also collected data from an additional sample of vaccinated USA residents (N= 479, 56.8% female) between June 3 and June 7, 2021, which we used to replicate the validation procedures. Participants in this sample were required to have received both doses of an FDA-approved two-dose vaccine (i.e., Pfizer, Moderna) or the one-dose vaccine (i.e., Johnson and Johnson). Participants received modest compensation for their participation based on expected completion time consistent with the panel company’s published rates. Sample characteristics are presented in Table 1. PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 4 / 33 Design and procedure Approval for study procedures was granted prior to data collection from the Griffith University Human Research Ethics Committee (#2021/108). A systematic, multi-step design process using the AMEE Guide, which broadly consists of a development phase (steps 1–6) and a Table 1. Sample characteristics and descriptive statistics. Variable Sample Difference tests Australia USA (unvaccinated) USA (vaccinated) Participants 522 499 479 Age, Myears (SD) 47.40 (14.83) 55.36 (14.36) 52.14 (14.55) F(2,1497) = 1.313, p= .269 Gender, n(%) χ 2 (2) = 21.285, p<.001 b Female 317 (60.7) 353 (70.7) 272 (56.8) Male 202 (38.7) 144 (28.9) 203 (42.4) Non-binary 0 (0.0) 1 (0.2) 2 (0.4) Not specified/prefer not to answer 3 (0.06) 1 (0.2) 2 (0.4) Employment status, n(%) χ 2 (8) = 98.585, p<.001 Currently unemployed/full-time caregiver 122 (23.4) 116 (23.3) 78 (16.3) Part-time/casual employed 121 (23.2) 55 (11.0) 44 (9.2) Currently employed full-time 181 (34.7) 149 (29.9) 229 (47.8) Leave without pay/furloughed 2 (0.04) 2 (0.4) 1 (0.2) Retired 96 (18.4) 177 (35.6) 127 (26.5) Race, n(%) χ 2 (10) = 105.950, p<.001 Black 3 (0.6) 36 (7.2) 19 (4.0) Caucasian/White 411 (78.7) 433 (86.8) 421 (87.9) Asian (South-East Asia/South Asia) 75 (14.4) 13 (2.6) 22 (4.6) Middle-Eastern 8 (1.5) 2 (0.4) 1 (0.2) Other 12 (2.3) 13 (2.6) 13 (2.7) Prefer not to answer 13 (2.5) 2 (0.4) 3 (0.6) Income, n(%) a χ 2 (2) = 10.353, p= .006 b Low income (�US$30,000/AU$40,000) 61 (11.7) 59 (11.8) 78 (16.4) High income (>US$30,000/AU$40,000) 150 (28.7) 135 (27.1) 315 (66.0) Prefer not to answer 311 (59.6) 305 (61.1) 84 (17.6) Education level, n(%) χ 2 (8) = 105.11, p<.001 Completed junior/lower/primary school 23 (4.4) 13 (2.6) 4 (0.8) Completed senior/high/secondary school 158 (30.3) 192 (38.5) 122 (25.5) Post-school vocational qualification/diploma 144 (27.6) 120 (24.1) 66 (13.8) Undergraduate University degree 140 (26.8) 130 (26.1) 171 (35.7) Postgraduate University degree 57 (10.9) 44 (8.8) 116 (24.2) Previous diagnosis for COVID-19 χ 2 (2) = 35.221, p<.001 b Yes 6 (1.2) 47 (9.4) 38 (7.9) No 516 (98.8) 448 (89.8) 440 (91.9) Prefer not to say 0 (0.0) 4 (0.8) 1 (0.2) Current COVID-19 χ 2 (8) = 31.415, p<.001 Yes 1 (0.2) 1 (0.2) 18 (3.8) No 520 (99.6) 496 (99.4) 460 (96.0) Prefer not to say 1 (0.2) 2 (0.4) 1 (0.2) Note. a Participants were given the choice of opting out of reporting their income. b Analysis based on a binary dependent variable. https://doi.org/10.1371/journal.pone.0264784.t001 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 5 / 33 validation phase (step 7), was adopted to develop an initial set of items for the VaCCS [46]. In the first few steps of the developmental phase (steps 1–4; [46]), participants from the initial Australia and the USA samples completed the open response survey comprising five structured questions [48] and reported their demographic information. This step was conducted to learn how the population of interest conceptualizes and describes a range of beliefs and concerns with respect to COVID-19 vaccines [46]. Responses were subjected to qualitative content analysis using NVivo 10 qualitative analysis software, which resulted in the extraction of 10 themes: behavior, protection, barriers, emotions, normality, safety, trust, development, efficacy, and social influences (see S1 File). Next, and concurrent with the open response survey, a rapid review of the literature was conducted followed by a synthesis of the open response survey and literature review data to develop a comprehensive set of survey items for further evaluation. Specifically, these steps were conducted to ensure that the beliefs and concerns raised with respect to COVID-19 vaccines aligned with relevant prior research and theory, to identify existing survey scales or items that might be used or adapted, and to ensure survey items were written in accordance with current best practice and the language used was appropriate to the population of interest [46]. The search was conducted in February and March 2021 using OVID, Medline, Web of Science, and Embase, and also encompassed meta-analyses and systematic reviews investigating the measurement of vaccine beliefs (see S2 File for the search syntax). The search resulted in the identification of 448 records after removal of duplicates. After abstract and full text screening, 10 systematic reviews and meta-analyses were included, from which 63 measures of vaccine beliefs were extracted. We also included the Oxford Coronavirus Explanation, Attitudes, and Narratives survey (making it 64 measures), which was published subsequent to the included reviews [29]. A flowchart of the screening and inclusion process is presented in S3 File and a list of the extracted scales and their sources is presented in S4 File. A pool of 561 items was extracted from the 64 measures identified in the review process, with a total of 480 items after duplicate items were removed. Items were then subjected to a thematic analysis using NVivo 10 qualitative analysis software and subsequently sorted into one of 11 themes and entered into an excel spreadsheet. The themes were largely similar to the themes extracted from the open response survey: behavior, efficacy-protection, barriers, emotions, normality, safety, trust, development, importance, knowledge, and alternative medicine. The items were then examined by the lead investigators for relevance and appropriateness, resulting in the identification of 159 items considered eligible for inclusion in the VaCCS. These items were then further reviewed for similarity of content and expression and cross-referenced with the qualitative open response survey data. This process resulted in the retention of 63 items for further evaluation. Items were adjusted where necessary to make reference to COVID-19 vaccination (see Table 2). In the final steps of the development phase (steps 5–6; [46]), to ensure the readability and face validity of the scale, a convenience sample of behavioral science experts (N= 5) and laypersons (N= 10) and were recruited in May 2021. After providing demographic information and informed consent, participants were asked to complete the 63 potential items, and then rate them for readability (“The items are easy to understand for the average person”), relevance (“The measure is relevant to assessing people’s COVID-19 vaccine beliefs and concerns”), and suitability (“The measure is suitable for assessing individuals’ COVID-19 vaccine beliefs and concerns”) on a 5-point Likert type scale (1 = strongly disagree to 5 = strongly agree) developed by the authors. Participants were also given the opportunity to provide further comment in an open response format, which resulted in a small number of edits to item wording to improve clarity. Results indicated that both expert and lay-person groups found the scale easy to understand (M= 4.80, SD = 0.42; M= 4.25, SD = 0.50, respectively), relevant (M= 5.00, SD = 0.00; PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 6 / 33 Table 2. Results of exploratory factor analysis of candidate pool of items of the Vaccination Concerns in COVID-19 Scale (VaCCS). # Item Factor 182357946 29 If I get the COVID-19 vaccine it will help to protect my family and friends against the coronavirus .740 27 The COVID-19 vaccine will protect me from the coronavirus .734 30 The COVID-19 vaccine will strengthen the immune system against the coronavirus .733 26 The COVID-19 vaccine will stop the spread of the coronavirus .720 25 The COVID-19 vaccine is effective .711 31 If individuals like me get the COVID-19 vaccine it will save a large number of lives .678 32 The COVID-19 vaccine is likely to work for almost everyone .676 28 The COVID-19 vaccine will reduce the severity of symptoms if I get the coronavirus .612 33 The COVID-19 vaccine is likely to work for me .573 53 Getting the COVID-19 vaccine will help to get things back to normal .542 54 Getting the COVID-19 vaccine will help to ensure people can freely travel again .533 55 Getting the COVID-19 vaccine will help to ensure people can freely go out again .529 42 It is important to get the COVID-19 vaccine so that outbreaks do not occur .492 43 Getting the COVID-19 vaccine is important for the health of others in my community .483 41 It is important to get the COVID-19 vaccine to prevent coronavirus spreading in the community .478 .304 52 Getting the COVID-19 vaccine will give me complete freedom to get on with life just as before .472 44 Getting the COVID-19 vaccine is important for my health .426 .316 40 Getting the COVID-19 vaccine is important .413 46 Getting the COVID-19 vaccine makes me feel relieved .357 4 It is safe for a person to get the COVID-19 vaccine .315 18 I trust the Government to give me reliable information on the benefits and risks of the COVID-19 vaccine .838 19 I trust Healthcare Providers and Health Professionals to give me reliable information on the benefits and risks of the COVID-19 vaccine .817 12 I trust the Government’s conclusions that the COVID-19 vaccine is safe .816 20 I trust Scientists to give me reliable information on the benefits and risks of the COVID-19 vaccine .809 14 I trust Scientists’ conclusions that the COVID-19 vaccine is safe .798 13 I trust Healthcare Providers’ and Health Professionals’ conclusions that the COVID-19 vaccine is safe .788 24 I trust vaccine manufacturers to give me reliable information on the benefits and risks of the COVID-19 vaccine .607 11 The COVID-19 vaccine was proven safe before it was approved for use .519 3 I am concerned about the side effects of the COVID-19 vaccine .882 2 I am worried about the side effects of the COVID-19 vaccine .867 1 I fear that the COVID-19 vaccine will cause side effects .853 6 I am concerned about the safety of the COVID-19 vaccine .833 5 I am worried about the safety of the COVID-19 vaccine .801 7 The COVID-19 vaccine is risky .573 10 The COVID-19 vaccine can cause the coronavirus in some people .939 9 The COVID-19 vaccine can give you a serious case of the very same virus you’re trying to avoid .922 8 I can get the coronavirus from the COVID-19 vaccine .855 16 The COVID-19 vaccine safety data is often made up .613 17 People have been deceived about the safety of the COVID-19 vaccine .606 22 The COVID-19 vaccine is promoted mainly because of manufacturers’ profit .568 23 The main reason for promoting the COVID-19 vaccine is for drug companies to make money .527 15 The COVID-19 vaccine safety data is untrustworthy .484 49 I am opposed to the COVID-19 vaccine because it goes against freedom of choice .475 (Continued) PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 7 / 33 M= 4.75, SD = 0.50, respectively), and suitable (M= 4.20, SD = 1.14; M= 4.75, SD = 0.50, respectively). In the scale validation phase (step 7; [46]), participants completed an anonymized online survey comprising study measures alongside measures of socio-demographic variables and characteristics (age, gender, employment status, education level, race, income). This last step in the VaCCS design process was to check for adequate item variance, reliability, and convergent and discriminant validity with respect to other measures [46]. Study measures comprised the candidate VaCCS items identified in the development phase alongside measures of constructs and variables used to test the concurrent and predictive validity of the VaCCS. Specifically, participants completed sets of measures tapping socio-political, personality and individual difference constructs, and health beliefs and outcomes. Where a ‘target’ behavior was mentioned in the study measures (e.g., intention), items in the unvaccinated Australian and USA samples made reference to getting the COVID-19 vaccine, while measures in the USA vaccinated sample referred to getting a booster vaccine in future to control for emerging variants of the virus. Measures In the scale validation phase, we assessed concurrent and predictive validity of the VaCCS by including measures of constructs and variables expected to exhibit a characteristic pattern of Table 2. (Continued) # Item Factor 182357946 21 A lot of important information about the COVID-19 vaccine is not shared with the public .471 48 I am afraid of getting the COVID-19 vaccine .900 47 I am fearful about getting the COVID-19 vaccine .868 45 Getting the COVID-19 vaccine makes me feel anxious .586 61 Overall, I am hesitant about getting the COVID-19 vaccine .306 37 I am more likely to trust the COVID-19 vaccine once it has been around for a while .561 35 The COVID-19 vaccine is too new so I should wait before deciding to get it .553 34 More time is needed to be able to fully investigate the true effects of the COVID-19 vaccine .482 39 I am afraid that the COVID-19 vaccine has not been successfully tested on enough people .474 38 I am concerned that the COVID-19 vaccine has not been tested adequately .311 .467 63 I am uncertain whether or not I will get the COVID-19 vaccine when it is offered to me .339 36 The speed of developing and testing the COVID-19 vaccine means it will be unsafe .302 56 I have access to all the information I need to make good decisions about getting the COVID-19 vaccine .830 57 Information about the COVID-19 vaccine is easy to understand .699 58 I don’t have enough information about the COVID-19 vaccine to decide .593 51 Getting the COVID-19 vaccine should be on a strictly voluntary basis -.444 50 Individual rights are more important than requirements to get the COVID-19 vaccine .373 -.433 60 When the COVID-19 vaccine is offered to me, I will get it straight away .394 .428 59 I will get the COVID-19 vaccine when it is offered to me -.425 62 When the COVID-19 vaccine is available to me I will refuse to get it .371 .425 Proportion of Variance Explained .112 .088 .075 .045 .044 .038 .030 .030 .030 Cumulative Variance Explained .112 .200 .275 .320 .363 .401 .431 .461 .491 Note. Factor 1 = Efficacy; Factor 2 = Worry; Factor 3 = Cause; Factor 4 = Literacy; Factor 5 = Scepticism; Factor 7 = Fear; Factor 8 = Trust; Factor 9 = Uncertainty. Coefficients are standardized structure factor loadings after oblimin rotation. Loadings are presented in order of size and factors presented in order of variance explained. Loadings <.300 are suppressed for clarity. Items in bold font were selected for the final 35-item VaCCS scale. https://doi.org/10.1371/journal.pone.0264784.t002 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 8 / 33 Confirmatory factor analyses and invariance analyses Next, we fit the 8-factor, 35-item VaCCS model to data from the Australia and USA samples separately using confirmatory factor analysis. Pending adequate fit of the model with the data in each sample, we subsequently tested the invariance of the model factor loadings and intercepts across the two samples using multi-group confirmatory factor analysis. We also replicated the confirmatory factor analytic model in the additional USA sample of vaccinated participants. Goodness-of-fit statistics for the initial confirmatory factor analysis model in each sample are presented in Table 3. Overall, the model exhibited fit statistics that approached acceptable cut-off values for the multiple adopted indices of good fit. However, modification indices (MI) indicated that redundancy across some items from within the same subscale was primarily responsible for the misspecification, and identified pairs of within-factor indicator error covariances that would resolve the misspecification if freely estimated. The error variances associated with the largest misspecification in the model were identical across the two samples. Examination of the content of these items indicated that they tapped similar conceptual content in the measured factor. We therefore set error covariances free between items with the five largest expected parameter change (EPC) statistics identified by the MIs (EPC >50.000) in each sample. The practice of including error covariances is only advocated when justified by the content of the items involved in the redundancy [73]. In the case of the inclusion of the five error covariances for these models, there was justification given the closeness in the item content. We expected the inclusion of these error covariances would have little consequence on model integrity and we did not want reject a tenable model on trivial grounds. The final model including these error covariances exhibited acceptable goodness-of-fit statistics in both samples (Table 3). Solution estimates of the final model in each sample are presented in Table 4. Factor loadings and R-squared values for each item on its respective factor approached or exceeded the .70 and .50 expected values, respectively, with narrow confidence intervals for the factor loadings. Table 3. Model fit statistics for single-sample and multi-group confirmatory factor analyses with comparisons. Samples and model χ 2 df CFI TLI RMSEA 90% CI RMSEA SRMSR Δχ 2 Δdf ΔCFI ΔTLI LB UB Australia sample Initial model 2561.737��� 532 0.904 0.892 .087 .084 .091 .048 − − − − Final model 1699.744��� 526 0.947 0.940 .065 .062 .069 .043 − − − − USA sample (unvaccinated) Initial model 2083.983��� 532 0.915 0.905 .076 .073 .080 .053 − − − − Final model 1379.883��� 526 0.953 0.947 .057 .053 .061 .046 − − − − USA sample (vaccinated) Initial model 2687.983��� 532 0.880 0.865 .092 .089 .095 .047 − − − − Final model 1599.621��� 526 0.940 0.932 .065 .062 .069 .040 − − − − Invariance analysis Configural 3211.064��� 1056 0.946 0.940 .063 .061 .066 .044 − − − − Weak invariance 3275.088��� 1083 0.946 0.940 .063 .061 .065 .047 64.025��� 27 .000 .000 Strong invariance 3520.574��� 1110 0.940 0.936 .065 .063 .068 .050 245.486��� 27 .006 .004 Note. χ 2 = Model goodness-of-fit chi-square; df = Degrees of freedom of model goodness of fit chi square; CFI = Comparative fit index; TLI = Tucker-Lewis Index; RMSEA = Root mean square error of approximation; 90% CI RMSEA; 90% confidence interval of the RMSEA; LB = Lower bound of the 90% confidence interval of the RMSEA; UB = Upper bound of the 90% confidence interval of the RMSEA; SRMSR = Standardized root mean square of the residuals; Δχ 2 = Change in goodness-of-fit chi-square; Δχ 2 = Change in degrees of freedom of the goodness of fit chi-square; ΔCFI = Change in CFI; ΔTLI = Change in TLI. https://doi.org/10.1371/journal.pone.0264784.t003 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 15 / 33 Table 4. Results of confirmatory factor analysis of the Vaccination Concerns in COVID-19 Scale (VaCCS) in each sample. Scale and Item# Sample Australia USA (unvaccinated) USA (vaccinated) λ95% CI R 2 AVE λ95% CI R 2 AVE λ95% CI R 2 AVE LB UB LB UB LB UB Efficacy .735 .739 .593 29 .886 .865 .906 .784 .868 .844 .892 .753 .839 .807 .870 .703 27 .878 .856 .900 .771 .912 .895 .930 .833 .756 .714 .799 .572 26 .846 .820 .873 .716 .881 .859 .903 .777 .762 .720 .804 .580 25 .881 .859 .902 .776 .882 .860 .903 .777 .821 .787 .855 .674 28 .770 .733 .807 .593 .793 .759 .828 .629 .766 .725 .807 .586 53 .828 .800 .857 .686 .824 .794 .855 .680 .765 .724 .807 .586 42 .876 .853 .898 .767 .865 .840 .889 .748 .726 .679 .773 .527 43 .877 .854 .899 .768 .850 .824 .877 .723 .750 .706 .794 .562 Trust .819 .793 .705 18 .892 .872 .911 .795 .855 .830 .881 .732 .781 .744 .818 .610 19 .911 .894 .928 .830 .841 .814 .868 .707 .881 .858 .903 .776 12 .944 .932 .955 .891 .910 .893 .927 .828 .861 .836 .887 .742 20 .904 .886 .921 .817 .897 .878 .916 .805 .910 .892 .928 .829 14 .928 .915 .942 .862 .960 .951 .969 .921 .931 .916 .946 .866 13 .937 .925 .950 .879 .939 .927 .951 .882 .878 .855 .901 .771 24 .792 .758 .825 .627 .812 .781 .844 .660 .668 .617 .720 .446 Worry .864 .845 .741 3 .953 .941 .964 .907 .957 .944 .970 .916 .857 .827 .886 .734 1 .940 .927 .953 .883 .915 .898 .933 .838 .783 .744 .823 .614 5 .895 .876 .915 .802 .887 .865 .909 .786 .940 .918 .961 .883 Cause .813 .800 .781 10 .950 .935 .966 .903 .941 .922 .959 .885 .898 .877 .920 .807 9 .945 .930 .961 .893 .915 .894 .936 .837 .965 .951 .979 .932 8 .809 .776 .841 .654 .823 .791 .855 .677 .796 .760 .831 .633 Scepticism .651 .618 .700 16 .799 .763 .836 .639 .753 .708 .798 .567 .815 .781 .849 .664 17 .823 .789 .857 .677 .844 .807 .881 .713 .840 .810 .870 .706 22 .806 .771 .842 .650 .814 .777 .852 .663 .831 .800 .862 .690 23 .811 .777 .846 .658 .773 .730 .816 .598 .846 .817 .876 .716 49 .795 .759 .832 .633 .753 .709 .796 .566 .851 .823 .880 .725 Fear .841 .769 .832 48 .962 .951 .973 .925 .957 .941 .972 .915 .949 .936 .962 .900 47 .948 .936 .960 .899 .960 .945 .975 .922 .950 .937 .963 .902 45 .830 .802 .859 .690 .677 .627 .726 .458 .832 .803 .862 .693 Uncertainty .766 .676 .659 35 .894 .872 .917 .800 .826 .791 .861 .682 .832 .800 .864 .692 34 .788 .751 .824 .620 .849 .817 .882 .721 .657 .602 .711 .431 39 .918 .898 .938 .842 .799 .761 .838 .639 .924 .903 .944 .853 Literacy .602 .541 .736 56 .854 .819 .888 .729 .769 .710 .829 .592 .829 .796 .863 .688 57 .874 .840 .908 .764 .794 .733 .856 .631 .851 .821 .882 .725 (Continued) PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 16 / 33 Given the acceptable fit of the single-sample models, we conducted the invariance analysis of the proposed model according to our a priori invariance routine. The configural model exhibited adequate goodness-of-fit with the data, indicating that the proposed model was tenable across the two samples (Table 3). Estimating models that constrained the factor loadings (weak invariance) and intercepts (strong invariance) revealed minimal differences in model fit across each model in the invariance routine according to Cheung and Rensvold’s criterion for change in the CFI and TLI. We subsequently concluded that there was minimal variance in parameters of the proposed VaCCS model across the two samples. Finally, we replicated the proposed confirmatory factor analytic model of the VaCCS in the USA vaccinated sample. Consistent with the previous analyses, the model exhibited acceptable goodness-of-fit statistics (Table 3) and solution estimates (Table 4) in this additional sample. Descriptive statistics and reliability estimates Descriptive statistics and reliability estimates for the VaCCS subscales and study measures are presented in Table 5, and factor and manifest variable correlations among the subscales presented in Table 6. Reliability estimates indicated acceptable internal consistency for all VaCCS subscales. Reliability estimates for the variables included to test concurrent and criterion validity of the VaCCS were also acceptable, with few exceptions. The most notable exceptions were the two-item scales tapping the five-factor personality constructs, which all exhibited statistically significant inter-item correlations, but each was relatively modest in size, a trend noted elsewhere [56]. Table 4. (Continued) Scale and Item# Sample Australia USA (unvaccinated) USA (vaccinated) λ95% CI R 2 AVE λ95% CI R 2 AVE λ95% CI R 2 AVE LB UB LB UB LB UB 58 .626 .566 .686 .392 .654 .586 .722 .428 .900 .875 .925 .810 Note. Efficacy = Beliefs in efficacy and prevention VaCCS subscale; Trust = Trust in authorities VaCCS subscale; Worry = Worry about safety and side effects VaCCS subscale; Cause = Beliefs vaccine causes COVID-19 VaCCS subscale; Scepticism = Scepticism and mistrust in the vaccine VaCCS subscale; Fear = Fear of vaccine VaCCS subscale; Uncertainty = Uncertainty and hesitation getting vaccinated VaCCS subscale; Literacy = Vaccine literacy VaCCS subscale; λ= Standardized factor loading; 95% CI = 95% Confidence interval of the standardized factor loading; LB = Lower bound of 95% confidence interval; UB = Upper bound of 95% confidence interval; AVE = Average variance extracted for the factor. https://doi.org/10.1371/journal.pone.0264784.t004 Table 5. Descriptive statistics and reliability estimates of study measures. Scale # Items Range Australia sample USA sample (unvaccinated) USA sample (vaccinated) M SD ωM SD ωM SD ω VaCCS Efficacy 8 1−7 4.914 1.379 .974 3.707 1.495 .971 5.856 0.946 .955 VaCCS Trust 7 1−7 4.554 1.604 .981 3.095 1.620 .977 5.551 1.207 .969 VaCCS Worry 3 1−7 4.884 1.694 .950 5.611 1.481 .943 3.101 1.681 .931 VaCCS Cause 3 1−7 2.943 1.572 .950 3.744 1.643 .943 2.429 1.583 .931 VaCCS Scepticism 5 1−7 3.312 1.566 .943 4.491 1.559 .955 2.593 1.534 .960 VaCCS Fear 3 1−7 4.270 1.785 .939 5.037 1.682 .905 2.743 1.657 .938 VaCCS Uncertainty 3 1−7 4.775 1.657 .904 5.557 1.406 .866 3.379 1.541 .857 VaCCS Literacy 3 1−7 4.466 1.426 .833 3.906 1.589 .792 5.621 1.204 .897 Oxford scale 7 1−5 3.487 1.238 .981 2.292 1.194 .976 4.421 0.778 .960 (Continued) PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 17 / 33 Concurrent validity Correlations between VaCCS subscales and sets of socio-political variables, personality and individual difference constructs, and health beliefs and outcomes are presented in Table 7, Table 8 and Table 9, respectively. Associations between VaCCS subscales and concurrent validity measures formed characteristic patterns of associations consistent with expectations. Table 5. (Continued) Scale # Items Range Australia sample USA sample (unvaccinated) USA sample (vaccinated) M SD ωM SD ωM SD ω Gov. trust 9 1−7 4.522 1.636 .989 3.042 1.778 .991 5.125 1.494 .986 Polit. orient. 2 0−100 49.330 19.694 .652 a 63.685 23.866 .704 a 49.289 25.383 .806 a Free will 5 1−7 5.179 1.094 .907 5.190 1.260 .921 5.339 1.080 .915 Polit. trust 6 1−7 5.039 0.974 .883 5.538 1.050 .895 5.236 0.929 .856 COVID-19 CB 6 1−7 3.005 1.369 .916 4.242 1.471 .906 2.788 1.292 .916 Vaccine CB 7 1−7 3.229 1.521 .974 4.452 1.544 .969 2.736 1.551 .978 General CB 5 1−11 6.829 2.066 .894 8.011 1.888 .888 6.612 2.002 .895 Vaccine hesitancy 1 1−5 3.163 1.501 −4.036 1.317 −2.238 1.507 − Vaccine denial Intention 3 1−7 4.568 1.973 .991 2.674 1.878 .991 5.995 1.209 .976 Risk perception 2 1−7 4.063 1.716 .767 a 5.039 1.581 .801 a 2.753 1.588 .768 a BMQ 8 1−5 2.767 0.811 .914 3.046 0.817 .903 2.580 0.842 .919 Vaccine confidence 5 1−5 2.707 0.653 .609 2.856 0.604 .456 2.243 0.696 .766 Vaccine knowledge b 9 0−18 10.082 4.621 .852 7.964 4.493 .826 10.795 4.176 .824 Vulnerable people b 3 1−6 5.100 1.128 .834 5.018 1.268 .842 4.835 1.078 .790 SRH 1 1−5 3.634 0.955 −3.593 1.057 −3.818 .959 − E 2 1−7 3.333 1.312 .429 a 3.645 1.503 .414 a 3.722 1.475 .383 a A 2 1−7 5.066 1.080 .230 a 5.252 1.120 .183 a 5.168 1.158 .199 a C 2 1−7 5.383 1.136 .457 a 5.743 1.086 .511 a 5.676 1.141 .419 a N 2 1−7 3.467 1.385 .542 a 3.127 1.360 .451 a 3.204 1.417 .518 a O 2 1−7 4.455 1.174 .291 a 4.670 1.230 .317 a 4.694 1.201 .216 a IUS-12 12 1−6 3.692 0.822 .741 a 3.500 0.868 .742 a 3.637 0.814 .687 a Note a Reliability estimate for two-item scales is the Spearman rank-order inter-item correlation b Scales comprises items with responses made on dichotomous (1 = yes, 0 = no) scales. ω= Revelle’s Omega reliability coefficient; Efficacy = Beliefs in efficacy and prevention VaCCS subscale; Trust = Trust in authorities VaCCS subscale; Worry = Worry about safety and side effects VaCCS subscale; Cause = Beliefs vaccine causes COVID-19 VaCCS subscale; Scepticism = Scepticism and mistrust in the vaccine VaCCS subscale; Fear = Fear of vaccine VaCCS subscale; Uncertainty = Uncertainty and hesitation getting vaccinated VaCCS subscale; Literacy = Vaccine literacy VaCCS subscale; Oxford scale = Oxford COVID-19 Vaccine Hesitancy Scale (Freeman et al., 2021); Gov. trust = Trust in government organizations in handling COVID-19 (Grimmelikhuijsen & Knies, 2017); Polit. orient. = Political orientation (Kroh, 2007); Free will = Free will beliefs; Polit. trust = Trust in politicians handling of the COVID-19 pandemic; COVID-19 CB = COVID-19 conspiracy beliefs scale (Imhoff & Lamberty 2020); Vaccine CB = Vaccine conspiracy beliefs scale (Shapiro et al., 2016); General CB = General conspiracy beliefs scale (Bruder et al., 2013); Vaccine hesitancy = Single-item vaccine hesitancy measure; Vaccine denial = Single-item vaccine denial measure; Intention = COVID-19 vaccination intentions; Risk perception = Beliefs in risk of COVID-19; BMQ = Beliefs about medicines questionnaire; Vaccine confidence = Vaccine confidence scale (Betsch et al., 2018); Vaccine knowledge = Knowledge of COVID-19 vaccine scale; Vulnerable people = Close contact with people known to be vulnerable to COVID-19; SRH = Single-item selfreported health; E = Extroversion personality trait; A = Agreeableness personality trait; C = Conscientiousness personality trait; N = Neuroticism personality trait; O = Openness to experience personality trait; IUS-12 = Intolerance of uncertainty scale short form (Carleton et al., 2007). ���p<.001 ��p<.01 �p<.05. https://doi.org/10.1371/journal.pone.0264784.t005 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 18 / 33 Socio-political beliefs. Focusing on correlations with socio-political beliefs, the VaCCS efficacy, trust, and literacy subscales were statistically significantly and positively correlated with the Oxford vaccine hesitancy scale and government trust in all three samples with medium-to-large effect sizes. These scales were also negatively correlated with COVID-19, vaccine, and general conspiracy beliefs, and vaccine hesitancy with medium-to-large effect sizes, although effects for the literacy subscale were smaller. In keeping with this pattern, the VaCCS worry, cause, scepticism, fear, and uncertainty subscales were negatively correlated with the Oxford scale and government trust, and positively related to all conspiracy beliefs variables and vaccine hesitancy, again with medium-to-large effect sizes and in all samples. The efficacy, trust, and literacy subscales were also positively correlated with free will, but only in the Australia and vaccinated USA samples. In addition, the efficacy and trust subscales were negatively correlated with vaccine denial in all samples, and negatively correlated with political trust except in the vaccinated USA sample, all with small effect sizes. The worry, cause, scepticism, fear, and uncertainty subscales were positively related to political trust and vaccine hesitancy in all samples with small effect sizes. Table 6. Correlations among the Vaccination Concerns in COVID-19 Scales (VaCCS) subscales. Subscale 1 2 3 4 5 6 7 8 1. Efficacy .870 -.589 -.510 -.758 -.483 -.577 .533 −.802 -.420 -.430 -.741 -.232 -.512 .268 .794 -.403 -.411 -.548 -.462 -.553 .695 2. Trust .889 -.673 -.467 -.745 -.571 -.654 .592 .837 −-.486 -.332 -.730 -.299 -.624 .348 .842 -.436 -.350 -.554 -.470 -.563 .733 3. Worry -.598 -.698 .505 .657 .766 .766 -.533 -.425 -.503 −.416 .506 .634 .699 -.267 -.417 -.454 .641 .701 .799 .684 -.353 4. Cause -.550 -.518 .538 .648 .475 .476 -.380 -.455 -.366 .436 −.557 .286 .350 -.160 -.461 -.432 .678 .771 .689 .674 -.255 5. Scepticism -.777 -.766 .679 .684 .563 .660 -.490 -.748 -.743 .509 .585 −.339 .624 -.291 -.587 -.605 .715 .834 .775 .793 -.441 6. Fear -.511 -.606 .787 .539 .587 .715 -.554 -.248 -.312 .647 .327 .366 −.537 -.262 -.491 -.514 .812 .755 .830 .736 -.376 7. Uncertainty -.620 -.714 .821 .534 .724 .750 -.609 -.559 -.689 .758 .395 .687 .576 −-.374 -.630 -.648 .743 .774 .900 .838 -.521 8. Literacy .606 .646 -.540 -.392 -.520 -.549 -.630 .346 .425 -.296 -.202 -.336 -.286 -.441 − .770 .790 -.368 -.313 -.495 -.420 -.604 Note. The matrix presented above the principal diagonal comprises manifest (averaged) variable correlations, and the matrix below the principal diagonal comprises latent variable correlations. Correlations presented on the upper, center, and lower lines are for the Australian, USA (unvaccinated), and USA (vaccinated) samples, respectively. VaCCS = Vaccination Concerns in COVID-19 Scale; Efficacy = Beliefs in efficacy and prevention VaCCS subscale; Trust = Trust in authorities VaCCS subscale; Worry = Worry about safety and side effects VaCCS subscale; Cause = Beliefs vaccine causes COVID-19 VaCCS subscale; Scepticism = Scepticism and mistrust in the vaccine VaCCS subscale; Fear = Fear of vaccine VaCCS subscale; Uncertainty = Uncertainty and hesitation getting vaccinated VaCCS subscale; Literacy = Vaccine literacy VaCCS subscale. All correlations are statistically significant (p<.001). https://doi.org/10.1371/journal.pone.0264784.t006 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 19 / 33 Personality and individual difference constructs. No clear pattern of statistically significant correlations emerged between the VaCCS subscales and the personality and individual difference constructs across the samples. Across samples, only the fear subscale was positively Table 7. Correlations of Vaccine Concerns in COVID-19 Scale (VaCCS) subscales with socio-political beliefs. Scale VaCCS subscales Efficacy Trust Worry Cause Sceptic. Fear Uncertain. Literacy Oxford scale .829��� .800��� -.665��� -.506��� -.726��� -.574��� -.675��� .513��� .780��� .760��� -.453��� -.357��� -.678��� -.303��� -.572��� .270��� .741��� .719��� -.495��� -.434��� -.550��� -.540��� -.561��� .594��� Gov. trust .618��� .676��� -.344��� -.238��� -.477��� -.274��� -.337��� .384��� .683��� .737��� -.305��� -.247��� -.585��� -.161�-.446��� .279��� .619��� .721��� -.252��� -.207��� -.351��� -.287��� -.343��� .577��� Polit. orient. -.159�� -.130 .141�.212��� .222��� .102 .140�-.052 -.237��� -.236��� .158�.019 .250��� .063 .193�� -.032 -.217��� -.252��� .333��� .363��� .425��� .324��� .362��� -.154� Free will .195��� .190��� -.028 -.027 -.076 -.056 -.043 .219��� .042 .027 .103 .170�� .080 .057 .137 .049 .295��� .254��� -.040 -.002 -.001 -.045��� -.033 .267��� Polit. trust -.153�-.218��� .169�� .132�.274��� .163�� .228��� -.123 -.371��� -.430��� .199��� .177�� .442��� .138 .327��� -.097 .035 -.068 .197��� .183�� .234��� .154�.193�� -.010 COVID-19 CB -.642��� -.611��� .473��� .581��� .760��� .408��� .472��� -.360��� -.650��� -.612��� .282��� .383��� .704��� .162�.386��� -.104 -.515��� -.533��� .496��� .579��� .745��� .566��� .633��� -.460��� Vaccine CB -.681��� -.670��� .566��� .613��� .828��� .510��� .577��� -.450��� -.675��� -.679��� .453��� .502��� .810��� .299��� .552��� -.213��� -.537��� -.547��� .633��� .710��� .851��� .688��� .772��� -.456��� General CB -.458��� -.504��� .445��� .406��� .538��� .368��� .457��� -.321��� -.474��� -.537��� .334��� .280��� .544��� .215��� .450��� -.156� -.277��� -.363��� .391��� .356��� .464��� .398��� .469��� -.265��� Vaccine hesitancy -.405��� -.480��� .517��� .332��� .465��� .577��� .578��� -.475��� -.255��� -.280��� .383��� .142�.299��� .315��� .360��� -.161� -.337��� -.320��� .488��� .495��� .543��� .524��� .525��� -.254��� Vaccine denial -.354��� -.331��� .221��� .159�� .327��� .189��� .176�� -.116 -.257��� -.322��� .217��� .158�.297��� .124 .282��� -.101 -.180�� -.176�� .297��� .258��� .277��� .342��� .314��� -.104 Note. Correlations presented on the upper, center, and lower lines are for the Australian, USA (unvaccinated), and USA (vaccinated) samples, respectively. p-values are adjusted for multiple tests. VaCCS = Vaccination Concerns in COVID-19 Scale; Efficacy = Beliefs in efficacy and prevention VaCCS subscale; Trust = Trust in authorities VaCCS subscale; Worry = Worry about safety and side effects VaCCS subscale; Cause = Beliefs vaccine causes COVID-19 VaCCS subscale; Scepticism = Scepticism and mistrust in the vaccine VaCCS subscale; Fear = Fear of vaccine VaCCS subscale; Uncertainty = Uncertainty and hesitation getting vaccinated VaCCS subscale; Literacy = Vaccine literacy VaCCS subscale; Oxford scale = Oxford COVID-19 Vaccine Hesitancy Scale (Freeman et al., 2021); Gov. trust = Trust in government organizations in handling COVID-19 (Grimmelikhuijsen & Knies, 2017); Polit. orient. = Political orientation (Kroh, 2007); Free will = Free will beliefs; Polit. trust = Trust in politicians handling of the COVID-19 pandemic; COVID-19 CB = COVID-19 conspiracy beliefs scale (Imhoff & Lamberty 2020); Vaccine CB = Vaccine conspiracy beliefs scale (Shapiro et al., 2016); General CB = General conspiracy beliefs scale (Bruder et al., 2013); Vaccine hesitancy = Single-item measure of vaccine hesitancy; Vaccine denial = Single-item measure of vaccine denial. ���p<.001 ��p<.01 �p<.05. https://doi.org/10.1371/journal.pone.0264784.t007 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 20 / 33 associated with intolerance of uncertainty in all three samples with small effect sizes. The only other patterns of correlations were confined to specific samples. The VaCCS worry, cause, scepticism, fear, and uncertainty subscales were negatively correlated with conscientiousness and agreeableness, and positively related to neuroticism, in the USA (vaccinated) sample with small-to-medium effect sizes. Health beliefs and outcomes. Turning to correlations with health beliefs and outcomes, the VaCCS efficacy, trust, and literacy subscales were statistically significantly and positively correlated with intentions to get the vaccine, and negatively related to risk perceptions, beliefs about medicines, and vaccine confidence with medium-to-large sized effects in all samples. Analogously, the VaCCS worry, cause, scepticism, fear, and uncertainty subscales were negatively correlated with intentions, and positively correlated with risk perceptions, beliefs about medicines, and vaccine confidence with medium-to-large sized effects and in all samples. The VaCCS efficacy and trust subscales were positively correlated, and the VaCCS worry, cause, scepticism, fear, and uncertainty subscales negatively correlated, with vaccine knowledge except in the vaccinated USA sample. A similar pattern of correlations was exhibited with receiving the influenza vaccine and influenza vaccine regularity, but only in the Australia sample. The VaCCS efficacy and trust subscales were positively correlated with receiving an influenza vaccine and influenza vaccine regularity in all three samples. Table 8. Correlations of Vaccine Concerns in COVID-19 Scale (VaCCS) subscales with personality and traits. Scale VaCCS subscales Efficacy Trust Worry Cause Sceptic. Fear Uncertain. Literacy Extroversion -.073 -.080 .069 .107 .090 .067 .012 -.043 .009 -.023 -.108 -.052 -.017 -.112 -.064 .020 .090 .039 -.083 .012 .000 -.086 -.036 .058 Agreeableness .187�� .152�-.053 -.149�-.187�� -.032 -.048 .100 .053 .041 .017 -.113 -.071 .037 .012 .035 .068 .111 -.108 -.227��� -.234��� -.178�� -.250��� .106 Conscientiousness .128 .072 -.014 -.140 -.169�� -.059 -.026 .128 -.003 -.076 .093 -.140 -.004 .072 .145 .054 .160�.112 -.324��� -.366��� -.385��� -.404��� -.351��� .193�� Neuroticism -.028 -.009 .008 .073 .066 .100 .048 -.139 -.039 -.007 .061 .108 .065 .178�� .062 -.066 -.090 -.125 .202��� .170�� .188�� .303��� .219��� -.129 Openness to experience .144�.127 -.036 -.094 -.110 -.082 -.111 .144� .079 .062 -.030 -.008 -.071 -.056 -.018 .043 .176�� .208��� -.127 -.117 -.181�� -.205��� -.172�� .182�� IUS-12 -.034 -.044 .107 .101 .123 .176�� .135 -.214��� .036 .018 .120 .173�� .049 .230��� .124 -.108 -.038 -.058 .152�.146�.191�� .232��� .173�� -.085 Note. Correlations presented on the upper, center, and lower lines are for the Australian, USA (unvaccinated), and USA (vaccinated) samples, respectively. p-values are adjusted for multiple tests. VaCCS = Vaccination Concerns in COVID-19 Scale; Efficacy = Beliefs in efficacy and prevention VaCCS subscale; Trust = Trust in authorities VaCCS subscale; Worry = Worry about safety and side effects VaCCS subscale; Cause = Beliefs vaccine causes COVID-19 VaCCS subscale; Scepticism = Scepticism and mistrust in the vaccine VaCCS subscale; Fear = Fear of vaccine VaCCS subscale; Uncertainty = Uncertainty and hesitation getting vaccinated VaCCS subscale; Literacy = Vaccine literacy VaCCS subscale; IUS-12 = Intolerance of uncertainty scale short form (Carleton et al., 2007). ���p<.001 ��p<.01 �p<.05. https://doi.org/10.1371/journal.pone.0264784.t008 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 21 / 33 Predictive validity We also examined the unique effects of the VaCCS subscales on intentions to get the COVID-19 vaccine, or, in the case of the USA vaccinated sample, intentions to get a COVID-19 ‘booster’ vaccine, alongside socio-demographic covariates, risk perceptions, and vaccine hesitancy. Results of the linear regression of intentions on the VaCCS subscales, risk perceptions, vaccine hesitancy, and socio-demographic variables in each sample are presented in Table 10. Across the samples, Table 9. Correlations of Vaccine Concerns in COVID-19 Scale (VaCCS) subscales with health-related beliefs and outcomes. Scale VaCCS subscales Efficacy Trust Worry Cause Sceptic. Fear Uncertain. Literacy Intention .807��� .803��� -.673��� -.476��� -.683��� -.571��� -.684��� .536��� .701��� .716��� -.416��� -.280��� -.600��� -.300��� -.548��� .278��� .769��� .747��� -.462��� -.390��� -.518��� -.506��� -.514��� .651��� Risk perceptions -.589��� -.641��� .712��� .469��� .631��� .692��� .678��� -.555��� -.569��� -.576��� .622��� .350��� .578��� .479��� .614��� -.283��� -.459��� -.471��� .697��� .669��� .753��� .768��� .707��� -.379��� BMQ -.458��� -.426��� .443��� .509��� .562��� .424��� .435��� -.315��� -.361��� -.338��� .281��� .381��� .473��� .204��� .342��� -.149� -.282��� -.238��� .459��� .561��� .615��� .540��� .581��� -.202��� Vaccine confidence a -.476��� -.425��� .440��� .510��� .585��� .442��� .404��� -.299��� -.293��� -.262��� .333��� .333��� .423��� .307��� .373��� -.187�� -.425��� -.379��� .576��� .613��� .739��� .656��� .676��� -.328��� Vaccine knowledge .300��� .267��� -.286��� -.329��� -.358��� -.283��� -.267��� .291��� .048 .096 -.073 -.149�-.118 -.109 -.156�.221��� .360��� .360��� -.272��� -.263��� -.303��� -.282��� -.348��� .389��� Vulnerable people -.107 -.104 -.001 -.007 .050 -.062 -.010 .005 -.128 -.082 -.031 .071 .089 -.103 .036 .002 .007 .019 -.173�� -.141 -.148 -.146 -.148 -.010 SRH .135 .123 -.075 -.070 -.047 -.094 -.082 .149� -.059 -.069 -.023 .047 .103 -.124 .043 .061 .164�.217��� -.039 .048 .014 -.051 -.020 .236�� Flu shot .357��� .328��� -.253��� -.225��� -.314��� -.218��� -.298��� .258��� .145 .109 -.070 -.114 -.161�-.040 -.098 .016 .181�� .147 -.078 -.045 -.048 -.129 -.111 .177�� s Regular flu shot .447��� .411��� -.323��� -.267��� -.358��� -.268��� -.347��� .304��� .237��� .215��� -.092 -.128 -.181�� -.077 -.151�.058 .262��� .219��� -.079 -.038 -.059 -.129 -.130 .218��� Note. a High scores on this scale represent lower confidence in vaccines. Correlations presented on the upper, center, and lower lines are for the Australian, USA (unvaccinated), and USA (vaccinated) samples, respectively. p-values are adjusted for multiple tests. VaCCS = Vaccination Concerns in COVID-19 Scale; Efficacy = Beliefs in efficacy and prevention VaCCS subscale; Trust = Trust in authorities VaCCS subscale; Worry = Worry about safety and side effects VaCCS subscale; Cause = Beliefs vaccine causes COVID-19 VaCCS subscale; Scepticism = Scepticism and mistrust in the vaccine VaCCS subscale; Fear = Fear of vaccine VaCCS subscale; Uncertainty = Uncertainty and hesitation getting vaccinated VaCCS subscale; Literacy = Vaccine literacy VaCCS subscale; Intention = COVID-19 vaccination intentions; Risk perceptions = Beliefs in risks of COVID-19; BMQ = Beliefs about medicines questionnaire; Vaccine confidence = Vaccine confidence scale (Betsch et al., 2018); Vaccine knowledge = Knowledge of COVID-19 vaccine scale; Vulnerable people = Close contact with people known to be vulnerable to COVID-19; SRH = Single-item self-reported health; Flu shot = Received influenza vaccine in the past year; Regular flu shot = Regular recipient of influenza vaccine. ���p<.001 ��p<.01 �p<.05. https://doi.org/10.1371/journal.pone.0264784.t009 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 22 / 33 the VaCCS efficacy and trust subscales were statistically significant positive predictors of intentions with small-to-medium effect sizes, alongside a significant negative effect of vaccine hesitancy with a small effect size. The VaCCS uncertainty subscale was also a significant predictor across the samples, with small effect sizes, although the effect was negative in the unvaccinated samples, but positive in the vaccinated sample. This positive, statistically significant effect was unexpected given the large, statistically significant and negative correlation between intention and the uncertainty subscale (r= -.514, p<.001; Table 9). This is likely to be a suppressor effect caused by the substantive correlation between the uncertainty subscale and the other VaCCS subscales in this sample, particularly the scepticism and uncertainty subscales (Table 6). Further, risk perception was also a significant negative predictor of intentions in the unvaccinated samples, with a small effect size, but not in the vaccinated sample. In addition, the VaCCS fear and literacy subscales were significant predictors of intentions in the latter sample. Overall, these constructs accounted for between 62.1% and 76.2% of the variance in vaccine intentions across the samples. Discussion The present study reports on the development and initial validation of the Vaccine Concerns in COVID-19 Scale (VaCCS), a psychometric instrument designed to measure individuals’ Table 10. Results of linear multiple regression analysis of effects of Vaccine Concerns in COVID-19 Scale (VaCCS) subscales, risk perceptions, and socio-demo- graphic variables on vaccine intentions. Variable Australia sample USA unvaccinated sample USA vaccinated sample β95% CI Β95% CI β95% CI LB UB LB UB LB UB Age .036 -.009 .082 .039 -.016 .093 -.011 -.069 .047 Gender -.030 -.074 .014 -.057�-.112 -.001 -.003 -.057 .050 Education .026 -.019 .070 .019 -.036 .074 .008 -.046 .062 Employment status .007 -.037 .051 -.068�-.123 -.012 .025 -.029 .079 Race -.039 -.085 .007 -.022 -.077 .033 -.004 -.057 .049 COVID history -.047�-.089 -.004 -.019 -.073 .036 -.020 -.075 .036 Risk perceptions -.129��� -.198 -.060 -.221��� -.300 -.142 -.066 -.156 .023 Vaccine hesitancy -.100��� -.156 -.045 -.123��� -.183 -.063 -.117��� -.180 -.054 VaCCS Efficacy .464��� .374 .554 .316��� .215 .416 .410��� .322 .498 VaCCS Trust .179��� .080 .278 .317��� .214 .421 .274��� .177 .370 VaCCS Worry -.098�-.179 -.017 .112�.023 .202 -.067 -.156 .022 VaCCS Cause -.016 -.073 .041 .031 -.038 .100 -.008 -.095 .079 VaCCS Scepticism .069 -.012 .150 .029 -.071 .129 .070 -.044 .183 VaCCS Fear .048 -.027 .122 -.016 -.090 .057 -.114�-.220 -.009 VaCCS Uncertainty -.173��� -.249 -.096 -.093�-.184 -.002 .115�.019 .212 VaCCS Literacy -.044 -.102 .014 .002 -.057 .062 .132�� .052 .213 Note. Model R 2 values for intention were .762, .621, and .681 for the Australia, USA (vaccinated), and USA (unvaccinated) samples, respectively. β= Standardized regression coefficient; 95% CI = 95% Confidence interval of β; LB = Lower bound of 95% CI; UB = UB of 95% CI; Risk perceptions = Beliefs in risks of COVID-19; Vaccine hesitancy = Single-item measure of vaccine hesitancy; VaCCS = Vaccination Concerns in COVID-19 Scale; Efficacy = Beliefs in efficacy and prevention VaCCS subscale; Trust = Trust in authorities VaCCS subscale; Worry = Worry about safety and side effects VaCCS subscale; Cause = Beliefs vaccine causes COVID-19 VaCCS subscale; Scepticism = Scepticism and mistrust in the vaccine VaCCS subscale; Fear = Fear of vaccine VaCCS subscale; Uncertainty = Uncertainty and hesitation getting vaccinated VaCCS subscale; Literacy = Vaccine literacy VaCCS subscale. ���p<.001 ��p<.01 �p<.05. https://doi.org/10.1371/journal.pone.0264784.t010 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 23 / 33 beliefs and concerns with respect to COVID-19 vaccines. The measure was developed in a multi-stage procedure from first principles comprising a developmental stage, in which an initial item pool was provided via an evidence synthesis of previous vaccine scales and an openended survey, and a validation stage in which a final pool of items was selected via rigorous construct and concurrent validity assessment in samples of Australian and the USA residents. The procedure produced a final 35-item scale with eight subscales. The scale exhibited strong psychometric integrity with a coherent factor structure that was invariant across samples. Subscale scores exhibited a predictable pattern of correlations with salient measures of socio-polit- ical beliefs, health beliefs and outcomes, and some selected personality and individual difference constructs. In addition, the VaCCS efficacy and trust subscales were consistent positive predictors of intentions to get the COVID-19 vaccine across samples, alongside vaccine hesitancy and risk perceptions. In addition, the uncertainty VaCCS subscale was negatively associated with COVID-19 vaccine intentions in the unvaccinated samples, but positively associated with intentions in the vaccinated sample. A comprehensive measure of COVID-19 vaccine beliefs and concerns Our development process yielded an instrument that captures the wide-ranging sets of beliefs likely to be of relevance to individuals’ decisions to get the COVID-19 vaccine. Concerns about the vaccine featured prominently in the scale and were captured by a number of its subscales. For example, the worry and uncertainty subscales reflect safety concerns regarding the rapid development of the vaccines and a lack of long-term studies of their effects, and concerns that the vaccine may lead to adverse side effects. These concerns have also been identified in research examining safety concerns in the context of COVID-19 [74,75]. In addition, the fear subscale captures anxiety and fear over getting a vaccine, which may also be linked with safety concerns, or stem from a generalized fear of medicines or medical procedures, or with the process of vaccine administration such as a fear of injections or syringes [76]. Further concerns are captured by the scepticism and trust subscales. The trust subscale taps into individuals’ general mistrust of the government, scientists, and pharmaceutical corporations responsible for developing and administering the vaccines, and likely captures generalized “anti-vax” beliefs. Similarly, vaccines have been equated as an instrument of governmental control, particularly with recent mandates that require vaccinations among essential workers [77–79], beliefs captured by the scepticism subscale. There have also been concerns over the efficacy of the vaccines, particularly in light of highly-publicized, albeit rare, cases where vaccinated individuals have been hospitalized with severe COVID-19, and reports of high infection rates among the vaccinated as new, highly contagious variants of the virus spread. The efficacy subscale, in particular, captures these beliefs, particularly concerns that vaccines may not be sufficiently effective or may do more harm to health than good. There is also evidence that individuals may harbor beliefs focused on the vaccine itself and that it may infect individuals with COVID-19 [80,81]–the cause subscale taps into these beliefs. Such beliefs likely stem from a lack of understanding of the vaccine and how it works, and a perceived lack of clear information on the vaccine and its effects, both of which may be related to low levels of health literacy. Such beliefs identified in the literacy subscale. Taken together, the VaCCS represents a comprehensive measure that captures a range of beliefs individuals may hold with respect to getting a COVID-19 vaccine, and are likely to be implicated in their future decisions to get the vaccine. Concurrent validity Given the breadth of beliefs captured by the VaCCS, examination of relations between the subscales and measures of conceptually-related beliefs and constructs was important to provide PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 24 / 33 31. Dube ´E, Laberge C, Guay M, et al.: Vaccine hesitancy. Hum Vaccin Immunther. 2013; 9:1763–1773. https://doi.org/10.4161/hv.24657 PMID: 23584253 32. Larson HJ, Clarke RM, Jarrett C, et al.: Measuring trust in vaccination: A systematic review. Hum Vaccin Immunther. 2018; 14:1599–1609. https://doi.org/10.1080/21645515.2018.1459252 PMID: 29617183 33. Smith LE, Amlo ˆt R, Weinman J, Yiend J, Rubin GJ: A systematic review of factors affecting vaccine uptake in young children. Vaccine. 2017; 35:6059–6069. https://doi.org/10.1016/j.vaccine.2017.09.046 PMID: 28974409 34. Hughes S, Machan L: It’s a conspiracy: Covid-19 conspiracies link to psychopathy, Machiavellianism and collective narcissism. Pers Ind Diff. 2021; 171:110559. https://doi.org/10.1016/j.paid.2020.110559 PMID: 33867616 35. Johnson MO: Personality correlates of HIV vaccine trial participation. Pers Ind Diff. 2000; 29:459–467. https://doi.org/10.1016/S0191-8869(99)00206-8 36. Lee CHJ, Duck IM, Sibley CG: Personality and demographic correlates of New Zealanders’ confidence in the safety of childhood vaccinations. Vaccine. 2017; 35:6089–6095. https://doi.org/10.1016/j. vaccine.2017.09.061 PMID: 28965999 37. Kreps S, Prasad S, Brownstein JS, et al.: Factors associated with US adults’ likelihood of accepting COVID-19 vaccination. JAMA Net Open. 2020; 3:e2025594–e2025594. https://doi.org/10.1001/ jamanetworkopen.2020.25594 PMID: 33079199 38. Guidry JPD, Laestadius LI, Vraga EK, et al.: Willingness to get the COVID-19 vaccine with and without emergency use authorization. Am J Infect Con. 2021; 49:137–142. https://doi.org/10.1016/j.ajic.2020. 11.018 PMID: 33227323 39. Dror AA, Eisenbach N, Taiber S, et al.: Vaccine hesitancy: The next challenge in the fight against COVID-19. Eur J Epidemiol. 2020; 35:775–779. https://doi.org/10.1007/s10654-020-00671-y PMID: 32785815 40. Kashte S, Gulbake A, El-Amin Iii SF, Gupta A: COVID-19 vaccines: rapid development, implications, challenges and future prospects. Hum Cell. 2021; 34:711–733. https://doi.org/10.1007/s13577-021- 00512-4 PMID: 33677814 41. Lin C, Tu P, Beitsch LM: Confidence and receptivity for COVID-19 vaccines: A rapid systematic review. Vaccines. 2021; 9:16. https://doi.org/10.3390/vaccines9010016 42. Islam MS, Kamal A-HM, Kabir A, et al.: COVID-19 vaccine rumors and conspiracy theories: The need for cognitive inoculation against misinformation to improve vaccine adherence. PLoS ONE. 2021; 16: e0251605. https://doi.org/10.1371/journal.pone.0251605 PMID: 33979412 43. Lockyer B, Islam S, Rahman A, et al.: Understanding COVID-19 misinformation and vaccine hesitancy in context: Findings from a qualitative study involving citizens in Bradford, UK. Health Exp. 2021; 24:1158–1167. https://doi.org/10.1111/hex.13240 PMID: 33942948 44. Murphy J, Vallières F, Bentall RP, et al.: Psychological characteristics associated with COVID-19 vaccine hesitancy and resistance in Ireland and the United Kingdom. Nat Comm. 2021; 12:29. https://doi. org/10.1038/s41467-020-20226-9 PMID: 33397962 45. Brennen JS, Simon F, Howard PN, Nielsen RK: Types,sources,and claims of COVID-19 misinformation. Retrieved September 29, 2021 from https://reutersinstitute.politics.ox.ac.uk/types-sources-and- claims-covid-19-misinformation 46. Artino AR, La Rochelle JS, Dezee KJ, Gehlbach H: Developing questionnaires for educational research: AMEE Guide No. 87. Med Teach. 2014; 36:463–474. https://doi.org/10.3109/0142159X.2014.889814 PMID: 24661014 47. Australian Government: Australia’s COVID-19 vaccine national roll-out strategy. Retrieved November 2021, 2021 from https://www.health.gov.au/sites/default/files/documents/2021/01/covid-19- vaccination-australia-s-covid-19-vaccine-national-roll-out-strategy.pdf 48. Martin LR, Petrie KJ: Understanding the dimensions of anti-vaccination attitudes: the vaccination attitudes examination (VAX) scale. Ann Behav Med. 2017; 51:652–660. https://doi.org/10.1007/s12160- 017-9888-y PMID: 28255934 49. Grimmelikhuijsen S, Knies E: Validating a scale for citizen trust in government organizations. Int Rev Admin Sci. 2017; 83:583–601. https://doi.org/10.1177/0020852315585950 50. Kroh M: Measuring left–right political orientation: The choice of response format. Pub Opin Quar. 2007; 71:204–220. https://doi.org/10.1093/poq/nfm009 51. Nadelhoffer T, Shepard J, Nahmias E, Sripada C, Ross LT: The free will inventory: Measuring beliefs about agency and responsibility. Consc Cog. 2014; 25:27–41. https://doi.org/10.1016/j.concog.2014. 01.006 PMID: 24561311 52. Akkerman A, Mudde C, Zaslove A: How populist are the people? Measuring populist attitudes in voters. Comp Pol Stud. 2014; 47:1324–1353. https://doi.org/10.1177/0010414013512600 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 31 / 33 53. Imhoff R, Lamberty P: A Bioweapon or a Hoax? The Link Between Distinct Conspiracy Beliefs About the Coronavirus Disease (COVID-19) Outbreak and Pandemic Behavior. Soc Psychol Pers Sci. 2020; 11:1110–1118. https://doi.org/10.1177/1948550620934692 54. Shapiro GK, Holding A, Perez S, Amsel R, Rosberger Z: Validation of the vaccine conspiracy beliefs scale. Papillomavirus Res. 2016; 2:167–172. https://doi.org/10.1016/j.pvr.2016.09.001 PMID: 29074176 55. Bruder M, Haffke P, Neave N, Nouripanah N, Imhoff R: Measuring individual differences in generic beliefs in conspiracy theories across cultures: Conspiracy mentality questionnaire. Front Psychol. 2013; 4:225. https://doi.org/10.3389/fpsyg.2013.00225 PMID: 23641227 56. Gosling SD, Rentfrow PJ, Swann WB Jr: A very brief measure of the Big-Five personality domains. J Res Pers. 2003; 37:504–528. https://doi.org/10.1016/S0092-6566(03)00046-1 57. Carleton RN, Norton MAPJ, Asmundson GJG: Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. J Anx Dis. 2007; 21:105–117. https://doi.org/10.1016/j.janxdis.2006.03.014 58. Ajzen I: Constructing a TPB questionnaire:Conceptual and methodological considerations. Retrieved September 1, 2002, from http://people.umass.edu/~aizen/pdf/tpb.measurement.pdf 59. Horne R, Weinman J, Hankins M: The beliefs about medicines questionnaire: The development and evaluation of a new method for assessing the cognitive representation of medication. Psychol Health. 1999; 14:1–24. https://doi.org/10.1080/08870449908407311 60. Betsch C, Schmid P, Heinemeier D, et al.: Beyond confidence: Development of a measure assessing the 5C psychological antecedents of vaccination. PLoS ONE. 2018; 13:e0208601. https://doi.org/10. 1371/journal.pone.0208601 PMID: 30532274 61. Zingg A, Siegrist M: Measuring people’s knowledge about vaccination: Developing a one-dimensional scale. Vaccine. 2012; 30:3771–3777. https://doi.org/10.1016/j.vaccine.2012.03.014 PMID: 22445808 62. Subramanianm SV, Huijts T, Avendano M: Self-reported health assessments in the 2002 World Health Survey: how do they correlate with education? Bull World Health Org. 2010; 88:131–138. https://doi. org/10.2471/BLT.09.067058 PMID: 20428370 63. Revelle W: psych:Procedures for psychological,psychometric,and personality research. Retrieved May 6, 2018 from https://cran.r-project.org/web/packages/psych/index.html 64. Bernaards C, Jennrich R: GPArotation. Retrieved September 29, 2021 from https://cran.r-project.org/ web/packages/GPArotation/GPArotation.pdf 65. Marsh HW, Hau KT, Wen Z: In search of golden rules: Comment on hypothesis testing approaches to setting cutoff values for fit indexes and dangers in overgeneralising Hu & Bentler’s (1999) findings. Struc Eq Mod. 2004; 11:320–341. https://doi.org/10.1207/s15328007sem1103_2 66. Hu L, Bentler PM: Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Eq Mod. 1999; 6:1–55. https://doi.org/10.1080/10705519909540118 67. Diamantopoulos A, Sigauw JA: Introducing LISREL. Thousand Oaks, CA: Sage, 2000. 68. Fornell C, Larcker DF: Structural equation models with unobservable variables and measurement error. J Mark Res. 1981; 18:39–50. 69. Cheung GW, Rensvold RB: Evaluating goodness-of-fit indexes for testing measurement invariance. Struc Eq Mod. 2002; 9:233–255. https://doi.org/10.1207/S15328007SEM0902_5 70. Rosseel Y: lavaan: An R package for structural equation modeling. J Stat Soft. 2012; 48:1–36. https:// doi.org/10.18637/jss.v048.i02 71. Na ´jera Catala ´n HE: Reliability, population classification and weighting in multidimensional poverty measurement: A Monte Carlo study. Soc Ind Res. 2019; 142:887–910. https://doi.org/10.1007/s11205-018- 1950-z 72. Gwet KL: Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008; 61:29–48. https://doi.org/10.1348/000711006X126600 PMID: 18482474 73. Brown TA: Confirmatory factor analysis for applied research ( 2nd Ed.). Abingdon, UK: Routledge, 2015. 74. Harrison EA, Wu JW: Vaccine confidence in the time of COVID-19. Eur J Epidemiol. 2020; 35:325– 330. https://doi.org/10.1007/s10654-020-00634-3 PMID: 32318915 75. Karlsson LC, Soveri A, Lewandowsky S, et al.: Fearing the disease or the vaccine: The case of COVID- 19. Pers Ind Diff. 2021; 172:110590. https://doi.org/10.1016/j.paid.2020.110590 76. Freeman D, Lambe S, Yu L-M, et al.: Injection fears and COVID-19 vaccine hesitancy. Psychol Med. 2021. https://doi.org/10.1017/S0033291721002609 PMID: 34112276 77. Debus M, Tosun J: Political ideology and vaccination willingness: Implications for policy design. Pol Sci. 2021; 54:477–491. https://doi.org/10.1007/s11077-021-09428-0 PMID: 34149102 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 32 / 33 78. Lasco G: Medical populism and the COVID-19 pandemic. Glob Pub Health. 2020; 15:1417–1429. https://doi.org/10.1080/17441692.2020.1807581 PMID: 32780635 79. Eberl J-M, Huber RA, Greussing E: From populism to the “plandemic”: Why populists believe in COVID- 19 conspiracies. J Elec Pub Op Part. 2021; 31:272–284. https://doi.org/10.1080/17457289.2021. 1924730 80. Stein RA, Ometa O, Pachtman Shetty S, et al.: Conspiracy theories in the era of COVID-19: A tale of two pandemics. Int J Clin Prac. 2021; 75:e13778. https://doi.org/10.1111/ijcp.13778 PMID: 33480171 81. Geana MV, Anderson S, Ramaswamy M: COVID-19 vaccine hesitancy among women leaving jails: A qualitative study. Pub Health Nurs. 2021; 38:892–896. https://doi.org/10.1111/phn.12922 PMID: 33973268 82. Scherr S, Leiner D: The populist hotbed: How political attitudes, resentment, and justice beliefs predict both exposure to and avoidance of specific populist news features in the United States. PLoS ONE. 2021; 16:e0258220. https://doi.org/10.1371/journal.pone.0258220 PMID: 34618837 83. Stecula DA, Pickup M: How populism and conservative media fuel conspiracy beliefs about COVID-19 and what it means for COVID-19 behaviors. Res Polit. 2021; 8:2053168021993979. https://doi.org/10. 1177/2053168021993979 84. Rogers RW: A protection motivation theory of fear appeals and attitude change. J Psychol. 1975; 91:93–114. https://doi.org/10.1080/00223980.1975.9915803 PMID: 28136248 85. Floyd DL, Prentice-Dunn S, Rogers RW: A meta-analysis of research on protection motivation theory. J Appl Soc Psychol. 2000; 30:407–429. https://doi.org/10.1111/j.1559-1816.2000.tb02323.x 86. Milne S, Sheeran P, Orbell S: Prediction and intervention in health-related behavior: A meta-analytic review of protection motivation theory. J Appl Soc Psychol. 2000; 30:106–143. https://doi.org/10.1111/ j.1559-1816.2000.tb02308.x 87. Guillon M, Kergall P: Factors associated with COVID-19 vaccination intentions and attitudes in France. Pub Health. 2021; 198:200–207. https://doi.org/10.1016/j.puhe.2021.07.035 PMID: 34481275 88. Malik AA, McFadden SM, Elharake J, Omer SB: Determinants of COVID-19 vaccine acceptance in the US. E Clin Med. 2020; 26. https://doi.org/10.1016/j.eclinm.2020.100495 PMID: 32838242 89. Sherman SM, Smith LE, Sim J, et al.: COVID-19 vaccination intention in the UK: results from the COVID-19 vaccination acceptability study (CoVAccS), a nationally representative cross-sectional survey. Hum Vaccin Immunther. 2021; 17:1612–1621. https://doi.org/10.1080/21645515.2020.1846397 90. Caserotti M, Girardi P, Rubaltelli E, et al.: Associations of COVID-19 risk perception with vaccine hesitancy over time for Italian residents. Soc Sci Med. 2021; 272:113688. https://doi.org/10.1016/j. socscimed.2021.113688 PMID: 33485215 91. Ruiz JB, Bell RA: Predictors of intention to vaccinate against COVID-19: Results of a nationwide survey. Vaccine. 2021; 39:1080–1086. https://doi.org/10.1016/j.vaccine.2021.01.010 PMID: 33461833 92. Jennings W, Stoker G, Bunting H, et al.: Lack of trust, conspiracy beliefs, and social media use predict COVID-19 vaccine hesitancy. Vaccines. 2021; 9:593. https://doi.org/10.3390/vaccines9060593 PMID: 34204971 93. Schernhammer E, Weitzer J, Laubichler MD, et al.: Correlates of COVID-19 vaccine hesitancy in Austria: Trust and the government. J Pub Health. 2021. https://doi.org/10.1093/pubmed/fdab122 PMID: 33948665 94. Gu T, Mack JA, Salvatore M, et al.: Characteristics associated with racial/ethnic disparities in COVID- 19 outcomes in an academic health care system. JAMA Net Open. 2020; 3:e2025197–e2025197. https://doi.org/10.1001/jamanetworkopen.2020.25197 95. Muñoz-Price LS, Nattinger AB, Rivera F, et al.: Racial disparities in incidence and outcomes among patients with COVID-19. JAMA Net Open. 2020; 3:e2021892–e2021892. https://doi.org/10.1001/ jamanetworkopen.2020.21892 PMID: 32975575 96. Little C, Alsen M, Barlow J, et al.: The impact of socioeconomic status on the clinical outcomes of COVID-19; a retrospective cohort study. J Comm Health. 2021; 46:794–802. https://doi.org/10.1007/ s10900-020-00944-3 97. Larson HJ, Jarrett C, Eckersberger E, Smith DMD, Paterson P: Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: A systematic review of published literature, 2007–2012. Vaccine. 2014; 32:2150–2159. https://doi.org/10.1016/j.vaccine.2014.01.081 PMID: 24598724 PLOS ONE The vaccination concerns in COVID-19 scale PLOS ONE | https://doi.org/10.1371/journal.pone.0264784 March 14, 2022 33 / 33