Functional health literacy : psychometric properties of the newest vital sign for Portuguese adolescents (NVS-PTeen)

Nutrients 2021, 13, 790. https://doi.org/10.3390/nu13030790 www.mdpi.com/journal/nutrients Article Functional Health Literacy: Psychometric Properties of the Newest Vital Sign for Portuguese Adolescents (NVS-PTeen) Osvaldo Santos 1,2,3,*, Miodraga Stefanovska-Petkovska 1, Ana Virgolino 1,3, Ana Cristina Miranda 1, Joana Costa 1, Elisabete Fernandes 3, Susana Cardoso 4,5 and António Vaz Carneiro 1,3,6,7 1 Instituto de Saúde Ambiental, Faculdade de Medicina, Universidade de Lisboa, 1649-028 Lisboa, Portugal; mpetkovsk[email protected]lisboa.pt (M.S.-P.); [email protected] (A.V.); [email protected] (A.C.M.); [email protected] (J.C.); [email protected] (A.V.C.) 2 Unbreakable idea Research, Lda., 2550-426 Painho, Portugal 3 Faculdade de Medicina, Universidade de Lisboa, 1649-028 Lisboa, Portugal; [email protected] 4 MARE, Escola Superior de Turismo e Tecnologia do Mar, Instituto Politécnico de Leiria, 2520-614 Peniche, Portugal; [email protected] 5 CiTechCare, Instituto Politécnico de Leiria, 2410-541 Leiria, Portugal 6 Institute for Evidence Based Healthcare, 1649-028 Lisboa, Portugal 7 Cochrane Portugal, 1649-028 Lisboa, Portugal * Correspondence: [email protected]; Tel.: +351-936-103-168 Abstract: Self-management of health requires skills to obtain, process, understand, and use healthrelated information. Assessment of adolescents’ functional health literacy requires valid, reliable, and low-burden tools. The main objective of this study was to adapt and study the psychometric properties of the Newest Vital Sign for the Portuguese adolescents’ population (NVS-PTeen). Classic psychometric indicators of reliability and validity were combined with item response theory (IRT) analyses in a cross-sectional survey, complemented with a 3-month test-retest assessment. The NVS-PTeen was self-administered to students enrolled in grades 8 to 12 (12 to 17 years old) in a school setting. Overall, 386 students (191 girls) from 16 classes of the same school participated in the study (mean age = 14.5; SD = 1.5). Internal reliability of the NVS-PTeen was α = 0.60. The NVSPTeen total score was positively and significantly correlated with Portuguese (r = 0.28) and mathematics scores (r = 0.31), school years (r = 0.31), and age (r = 0.19). Similar to the original scale (for the U.S.), the NVS-PTeen is composed of two dimensions, reading-related literacy and numeracy. Temporal reliability is adequate, though with a learning effect. IRT analyses revealed differences in difficulty and discriminative capacity among items, all with adequate outfit and infit values. Results showed that the NVS-PTeen is valid and reliable, sensible to inter-individual educational differences, and adequate for regular screening of functional health literacy in adolescents. Keywords: functional health literacy; questionnaire; psychometrics; item response theory; adolescents 1. Introduction Over the past decades, health literacy has become a flourishing field of research. This concept has evolved from a rather simple one, mainly focused on specific healthor disease-related contents, toward a complex construct [1]. Despite the variations concerning its definition, health literacy is generally understood as a set of individual capacities to obtain, process, and understand basic health information and services, which supports appropriate healthor disease-related decision making [2]. Health literacy relates to general literacy, as it involves the combination of verbal (reading and writing) and numerical skills with specific health literacy skills to access, understand, appraise, Citation: Santos, O.; StefanovskaPetkovska, M.; Virgolino, A.; Miranda, A.C.; Costa, J.; Fernandes, E.; Cardoso, S.; Carneiro, A.V. Functional Health Literacy: Psychometric Properties of the Newest Vital Sign for Portuguese Adolescents (NVS-PTeen). Nutrients 2021, 13, 790. https://doi.org/10.3390/ nu13030790 Academic Editor: Antoni Sureda Received: 13 January 2021 Accepted: 23 February 2021 Published: 27 February 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses /by/4.0/). Nutrients 2021, 13, 790 2 of 16 and apply the information gathered in order to make decisions and engage in health behaviors [1,3]. The extent to which an individual is autonomous and empowered to selfmanage health, following the chain of access–understand–appraise–apply health-related information, is broadly indicated by his/her levels of functional, interactive, and critical literacy [4]. Individuals possessing basic reading, writing, and numeracy skills that are necessary for them to function effectively in everyday situations, including managing their health or disease more easily, show adequate levels of functional health literacy [1,4]. Increased autonomy and empowerment in self-management of health come from advanced cognitive, social, and literacy skills that allow individuals to gather, interpret, and apply health-related information to changing conditions (interactive literacy) [4] and, on top of this, critically appraise health-related information (critical literacy), promoting the individual’s control over his/her own health [4,5]. The ultimate consequences of adequate health literacy levels include better life course health outcomes and reduced burden of healthcare service utilization (e.g., costs, frequency, and length of hospitalizations) [1,6]. The potential positive effects of health literacy promotion have been increasingly debated and are increasingly influential as a policy priority by decision makers across countries (e.g., [7]). A critical step, prior to the design and implementation of sustainable and (costand time-) effective health literacy policies, is to measure citizens’ health literacy. The European Health Literacy Survey (HLS-EU), an eight-country effort to measure health literacy, reported inadequate health literacy for 12.4% of the adult population from the eight European countries that took part in the survey (Austria, Bulgaria, Germany, Greece, Ireland, the Netherlands, Poland, and Spain) [8]. Furthermore, 35% of the participants in the HLS-EU had limited health literacy. In the particular case of Portugal, two nationwide studies using the same HLS-EU instrument estimated that limited functional health literacy ranges between 49% [9] and 61% [10] for the Portuguese adult population. Efforts to estimate health literacy have been mainly focused on clinical and community samples of adults (≥18 years old), whereas health literacy among children and adolescents has been sparsely measured [11–15]. Despite this, increased attention has been given to the association between parents’ health literacy levels and health outcomes of their dependents (e.g., [16,17]). At this level, the existing evidence suggests that parents’ limited health literacy is associated with a higher number of non-urgent visits to the pediatric emergency department [17] and hinders both their engagement in shareddecision making and their children’s therapeutic adherence [16]. As such, empowering children and adolescents to actively participate in health decision making in a meaningful manner is fundamental to achieving better health outcomes from a life course perspective. Health outcomes and behaviors in adolescence and adulthood are strongly associated. Thus, the lack of research in health literacy during adolescence is somehow striking. Adolescence is a life transition period characterized by numerous developmental changes (physical, cognitive, and emotional) (e.g., [18–21]), which are inherently linked to (health) behavioral change [20,22] and habit formation, as well as to an increasing level of individuals’ autonomy in many spheres of their lives, including health decision making. At this stage, several levels of influence—individual traits, peer and family influences, school and neighborhood environments—collectively determine individual development and the adoption of health behaviors [12], namely in relation to physical activity, healthy eating, substance abuse, and sexual risk-taking behaviors, these being major determinants of later life health outcomes and inequities [23–25]. Under this scenario, effective health literacy programs are expected to be implemented within the environments in which adolescents are embedded [12]. Schools are particularly relevant toward this end [26–28], as also emphasized by Manganello [12], because they provide the resources and supportive environment that facilitate the development of general literacy skills, i.e., reading, writing, and numeracy, which are also required for health literacy. Nutrients 2021, 13, 790 3 of 16 Given the undisputable role of health literacy during adolescence for health gains in a life span perspective, why are data scarce for this life stage? The lack of validated instruments for measuring health literacy, and, most specifically, functional health literacy, during adolescence is perhaps the main contributor to this knowledge gap (e.g., [14]). Much of the measurement instruments currently under use are of weak or moderate validity [29] and heterogeneous concerning their scope, which also shows the lack of consensus regarding the definition and conceptualization of health literacy, as previously mentioned [11,29,30]. Nevertheless, adaptations of health literacy measurement instruments developed for adults, such as the Test of Functional Health Literacy in Adults (TOFHLA) [31], the Rapid Estimate of Adult Literacy in Medicine (REALM) [32], and the Newest Vital Sign (NVS) [33] (just to name some of the most widely used tools in this area), have been used to assess functional health literacy in children and adolescents (e.g., [13,15,29]), although their psychometric properties have not always been studied. These three instruments measure different domains of functional health literacy: TOFHLA is a 50-item reading comprehension and numerical ability test that takes approximately 22 min to complete [31]; REALM is a reading recognition test that takes approximately 2–3 min to complete [32]; and, finally, the NVS is a 6-item functional health literacy instrument that assesses reading comprehension and numeracy [33]. The main objective of this study was to adapt the NVS instrument to the adolescent Portuguese population (NVS-PTeen) and examine its psychometric properties. The NVS instrument was selected because it is a brief, easy-to-administer scale [33–35] and has revealed adequate psychometric properties for assessing functional health literacy, including good internal consistency, for adult populations from different cultures, as well as for some adolescent populations (see Table S1, in Supplementary Material, for an extensive list of studies presenting the psychometric properties of the NVS). The original instrument consists of a food nutrition label with six associated questions scored on a dichotomous scale [33]. Although it has been previously adapted for the Portuguese adult population [36,37], its psychometric properties have not been evaluated yet for the adolescent Portuguese population (12 to 17 years old). As such, this is a pioneering, relevant, and timely contribution to the assessment of functional health literacy among Portuguese adolescents. 2. Materials and Methods 2.1. Study Design and Setting A cross-sectional observational study was conducted to assess the internal reliability, as well as the construct and convergent validity, of the NVS-PTeen. Regarding temporal reliability, data collection took place at two different moments, with a 3-month interval. This rather long time interval was set up for minimizing the potential learning effect after applying the instrument to the baseline. Self-administered pencil-and-paper questionnaires of the NVS-PTeen were applied in a school setting by a trained researcher. At the beginning of selected lectures, students were asked to fill in the questionnaires, i.e., collective application, with no time restriction. Questionnaires were collected by the same researcher immediately after completion. Due to the collective application of the instrument, the time required for the adolescents to complete the NVS-PTeen was not assessed. 2.2. Sampling and Participants To evaluate the psychometric properties of the NVS-PTeen, adolescents aged 12 to 17 years and enrolled in grades 8 to 12 in a Portuguese public school were invited to participate in this study (census approach), in a total of 16 classes. Exclusion criteria were (a) being Portuguese native speakers and/or (b) having special educational needs (e.g., due to cognitive impairment). Nutrients 2021, 13, 790 4 of 16 There are no clear guidelines about the minimum sample size that is required for assessing psychometric properties of psychosocial scales. Anyway, taking into account the type of statistical analyses considered for this psychometric approach (Spearman’s rank and simple logistic regression, intra-class correlations, exploratory factorial analysis, and item response theory (IRT); further details in the Statistical Analysis section below), a minimum sample size of 250 adolescents was defined. This sample size also corresponds to the median sample size found in a systematic literature review for determining the sample size for validating patient-reported outcome measures [38]. A sub-sample of students participated in the test-retest component of the study. The minimum sample size was settled at 100 students (attending to the nature of the statistical test, without stratification for test-retest analysis). The selection of these participants was done through a random sampling of the classrooms, one class per grade. All students from each randomly selected class were invited to participate in the retest assessment. 2.3. Instruments of Data Collection Table S1 in Supplementary Material provides the psychometric properties of various adapted and validated versions of the NVS questionnaire for use in different languages and countries. Contents of the NVS-PTeen and its scoring system are available in Table S2 (Supplementary Material). The original version of the NVS instrument has been already cross-culturally adapted and validated for the Portuguese adult population [37,39,40], revealing adequate psychometric properties. The authors of the two already mentioned Portuguese versions of the NVS [37,40] followed the standard method for cross-cultural adaptation of instruments by the Scientific Advisory Committee of the Medical Outcomes Trust for linguistic adaptation [41]. In both cases, the authors assumed that cultural issues regarding food labels (typically available on the back of food packages) are not substantially different between the U.S. and Portugal, therefore ensuring conceptually equivalent versions. In the case of the NVS-PTeen, the Portuguese linguistic and cultural adaptation, directly from the original English version, was initially performed for the adult version (same instrument used by Paiva et al. [37]), as follows: (a) two bilingual native Portuguese speakers independently translated the original version of the NVS from English to Portuguese, (b) these two translated versions were merged into a single consensus Portuguese version, (c) the Portuguese consensus version was then backtranslated by two bilingual native English speakers, and (d) the two independent English versions were merged into a single consensus version, which was compared to the original NVS instrument. The research team agreed that the two versions did not differ in any relevant way. The NVS-PTeen mainly differs from the adult version by adopting the second-person singular, thus promoting a rather colloquial style. A pre-test was conducted involving three women and three men of different ages (age range: 20–65 years) and with different levels of education (high school and university education). In addition to the NVS-PTeen, adolescents were asked to provide sociodemographic information, i.e., sex, age, and school year, as well as their final classification results for mathematics and Portuguese disciplines from the previous year. Each questionnaire form included a pre-stamped random individual code, which guaranteed respondents’ anonymity, while ensuring the longitudinal component of the project (only for the testretest sub-sample of students). 2.4. Statistical Analysis Descriptive statistics (i.e., mean, median, standard deviation, and frequency) were calculated for sociodemographic indicators (sex, age, and education level). Data normality was assessed using the Kolmogorov–Smirnov non-parametric test (Lilliefors corrected KS test), complemented with analyses of kurtosis and skewness of the distributions. Since the dataset of the total NVS-PTeen score was not normally distributed, comparisons between sexes and among educational levels were performed using Mann– Nutrients 2021, 13, 790 5 of 16 Whitney U and Kruskal–Wallis tests, respectively. The total score for the NVS-PTeen was recoded according to the cutoff points proposed by the authors of the original American English version of the instrument [33]: likelihood of inadequate health literacy (0 to 1 correct answers), limited health literacy (2 to 3 correct answers), and adequate health literacy (4 to 6 correct answers). The percentage of correct/non-correct answers to the NVS-PTeen items and health literacy levels were compared between sexes and among educational levels using chi-square tests. Psychometric properties of the NVS-PTeen were evaluated using two different approaches, reliability and validity properties (classic psychometry) and item response theory (IRT). NVS-PTeen reliability was assessed by calculating its internal consistency reliability and its reproducibility (temporal/test-retest reliability). Due to the dichotomous nature of NVS-PTeen items, the internal consistency reliability of this instrument was measured through the Kuder–Richardson 20 (KR20) coefficient [42]. Spearman’s rank correlation and pairwise odds ratios were used for assessing inter-item and item-total associations. A reliability coefficient of 0.70 and a corrected item-total subscale correlation of 0.30 or higher were considered good cutoffs for internal reliability [43]. Test-retest reliability was conducted to assess reproducibility of the NVS-PTeen instrument; as such, the two-way mixed, single-measure intraclass correlation coefficient (ICC) was used. Concerning validity of the NVS-PTeen, convergent validity and construct validity were assessed. Convergent validity was studied using bivariate correlation analysis (Spearman’s r correlation coefficient) between its global score and five theoretically related variables: age, school years, previous-year final classifications for Portuguese and mathematics, and the average final classification of Portuguese and mathematics. Construct validity was studied through exploratory factorial analysis (EFA) with direct oblimin rotation, following the same procedure as used in previous studies of the NVS (e.g., [24,44]). The Kaiser–Meyer–Olkin (KMO) test and Bartlett’s test of sphericity were performed to determine assumptions of EFA and sampling adequacy for principal component analysis. The correlation matrix of all six items and mean inter-item correlation were verified to evaluate the strength of association between the items. An eigenvalue higher than 1 and a screen plot were used to determine the number of factors. After oblimin rotation, items with a factor loading of 0.40 or greater were considered adequate for measuring a factor. Finally, IRT was used for estimating item difficulty, discrimination, and fit. Item difficulty refers to the level of health literacy required to meet at least 50% chance of correctly answering an item; item discrimination refers to the capacity of an item to differentiate students with high health literacy from students with low health literacy (items with discrimination values below 1 indicate less discriminating efficacy); and item fit refers to the degree to which observed responses to an item correspond to expected responses, given the difficulty of the item and the respondent’s level of health literacy. Values above 0.8 indicate an adequate item fit [45]. Statistical analyses were performed with IBM Statistical Package for the Social Sciences (SPSS), version 24.0, and with jMetrik, version 4.0.6, for IRT analysis. Statistical significance was set to α = 0.05. 2.5. Ethical Considerations Authorization for adapting and validating the NVS instrument for the Portuguese population was granted from Pfizer, Inc., the company that holds its copyright, and approved by the Ethical Committees of the Universidade do Porto and Centro Académico de Medicina de Lisboa. The assessment of the psychometric properties of the Portuguese version of the NVSPTeen followed the guidelines laid down in the Declaration of Helsinki, amended in Fortaleza [46]. Data collection was approved by the direction board of the school where data collection took place. Prior to study enrollment, the adolescents were informed of the study objectives, of its disassociation from the curricular activities, and that their Nutrients 2021, 13, 790 6 of 16 participation was voluntary, with no impact on their academic activities and/or results. Furthermore, they were explained that the filling in of the questionnaires was part of a research study, not a school test/exam and also that teachers would not have access to the results of the NVS-PTeen. Only the adolescents whose parents signed a consent form (with detailed information about the goals and tasks of the project) and who confirmed their willingness to participate were involved in the study. Anonymity was not possible for the students enrolled in the longitudinal component of the study. However, it was explained that only members of the research team would have access to the data collected, which would be kept confidential, and that no personal data allowing their identification (i.e., name) would be recorded in the main database (a random code was attributed to each student for test-retest matching). 3. Results 3.1. Sample Characterization Overall, 386 students (48.8% female) from 16 different classes participated in the study (Table 1). Regarding upper secondary education (grades 10 to 12), 28 students were not enrolled in mathematics courses and, therefore, only classification marks from Portuguese classes were used. Participants were aged, on average, 14.4 (SD = 1.4) years; no statistically significant differences between boys and girls were found (p = 0.79). In addition, no significant association was found between being a male or a female and the grade in which the students were enrolled; about 60% of the sample was enrolled in lower secondary education (grades 8 to 9; Table 1). About 10% of the students had negative scores in Portuguese in the previous academic year, whereas 20% of them scored negative in mathematics (normal distributions for students in grades 8 to 9; non-normal, leftskewed distribution for students in grades 10 to 12; data not provided). Table 1. Sample characterization: age and educational level of the participants by sex. Female (n = 191) Male (n = 195) Total (N = 386) p-value 1 Age (years) 12 14 (7.3%) 14 (7.2%) 28 (7.3%) 0.672 13 49 (25.7%) 45 (23.1%) 94 (24.4%) 14 39 (20.4%) 54 (27.7%) 93 (24.1%) 15 35 (18.3%) 30 (15.4%) 65 (16.8%) 16 33 (17.3%) 34 (17.4%) 67 (17.4%) 17 21 (11.0%) 18 (9.2%) 39 (10.1%) Mean (±SD) 14.46 (1.49) 14.41 (1.43) 14.43 (1.46) 0.791 Median 14.00 14.00 14.00 Educational level Grade 8 66 (34.6%) 60 (30.8%) 126 (32.6%) Grade 9 43 (22.5%) 61 (31.3%) 104 (26.9%) Grade 10 29 (15.2%) 21 (10.8%) 50 (13.0%) 0.321 Grade 11 35 (18.3%) 36 (18.5%) 71 (18.4%) Grade 12 18 (9.4%) 17 (8.7%) 35 (9.1%) 1 p-value calculated using the chi-square test for categorical variables and the Mann–Whitney U test for continuous variables. 3.2. Functional Health Literacy among Adolescents Girls failed slightly more NVS-PTeen questions than boys, except for items 5 and 6 (Table 2). However, statistically significant differences between boys and girls were only found for item 1 (p = 0.014). The prevalence of adequate health literacy was high for both sexes (80.6% for girls and 86.7% for boys), with no statistically significant differences (p = Nutrients 2021, 13, 790 7 of 16 0.26). In addition, 44.0% of the total sample answered the six questions of the NVS-PTeen correctly, thus obtaining the maximum score (score = 6; 45.0% for girls, 43.1% for boys). Table 2. The Newest Vital Sign for the Portuguese adolescents’ population (NVS-PTeen) score comparison for (a) sex and (b) educational level. NVS-PTeen Items (I) NVS Total Score (Range: 0–6; Mean ± SD) Health Literacy Level I1 (%) I2 (%) I3 (%) I4 (%) I5 (%) I6 (%) Inadequate (Score 0–1 ; %) Limited (Score 2–3 ; %) Adequate (Score 4–6; %) Sex Female ( n = 191) 84.3 71.2 83.2 71.2 90.6 79.1 4.72 ± 1.41 3.7 15.7 80.6 Male ( n = 195) 92.3 71.3 88.7 75.4 88.2 79.0 4.86 ± 1.27 2.1 11.3 86.7 p - value 1 0.014 0.987 0.121 0.353 0.450 0.984 0.526 0.259 Educational level Grade 8 ( n = 126) 84.1 61.1 86.5 57.1 84.9 63.5 4.27 ± 1.47 4.8 23.8 72.2 Grade 9 ( n = 104) 85.6 73.1 76.9 64.4 86.5 78.8 4.50 ± 1.31 3.8 16.3 79.8 Grade 10 ( n = 50) 94.0 94.0 96.0 96.0 96.0 92.0 5.68 ± 0.89 2.0 2.0 96.0 Grade 11 ( n = 71) 90.1 73.2 88.7 88.7 95.8 91.0 5.28 ± 1.03 0.0 5.6 94.4 Grade 12 ( n = 35) 100.0 65.7 91.4 97.1 91.4 94.3 5.29 ± 0.83 0.0 2.9 97.1 p - value 2 0.053 0.001 0.013 <0.001 0.060 <0.001 <0.001 <0.001 Total ( N = 386) 88.3 71.2 86.0 73.3 89.4 79.0 4.79 ± 1.34 2.8 13.5 83.4 1 p-value calculated using the chi-square test for percentages and the Mann–Whitney U test for continuous variables; 2 pvalue calculated with the chi-square test for percentages and the Kruskal–Wallis test for continuous variables. Statistically significant values are in bold. The educational level (i.e., number of school years) was significantly associated with all NVS-PTeen items (p < 0.05), except for items 1 and 5 (Table 2). The prevalence of compromised health literacy was higher among students in grade 8 (4.8% of the students had inadequate health literacy and 23.8% had limited health literacy) and grade 9 (3.8% of the students had inadequate health literacy and 16.3% had limited health literacy) than students in grades 10 to 12 (p < 0.001). Indeed, adequate health literacy was detected among more than 90% of the students in grades 10 to 12. 3.3. Internal Consistency and Test-Retest Reliability The overall internal consistency reliability of the self-administered NVS-PTeen was KR20 = 0.61 (95% CI = 0.54–0.66). The inter-item Spearman’s rank correlation coefficients and odds ratios are provided in Table 3. All inter-item correlation and odds ratio were statistically significant, except for the pairs of items 2 and 3, and 2 and 6. The highest inter-item Spearman’s rank correlation coefficients and odds ratios were obtained for the pair of items 5 and 6 (r = 0.59). Regarding the item-total correlation, which is an item discrimination indicator, Spearman’s rank correlation coefficients ranged from r = 0.49 (item 3) to r = 0.67 (item 4) (Table 3). Table 3. Inter-item and item-total NVS-PTeen scores (Spearman’s rank correlation coefficients matrix and inter-item pairwise odds ratio (OR) with 95% confidence intervals (95% CI); N = 386). Item 2 Item 3 Item 4 Item 5 Item 6 NVS - PTeen Total Score rho OR (95% CI) rho OR (95% CI) rho OR (95% CI) rho OR (95% CI) rho OR (95% CI) rho Item 1: If you eat the whole container of ice cream, how many calories are you going to consume? 0.197 ** 3.33 (1.77–6.29) 0.179 ** 3.40 (1.67–6.93) 0.201 ** 3.41 (1.81–6.44) 0.163 ** 3.35 (1.54–7.29) 0.170 ** 2.94 (1.53–5.67) 0.445 ** Item 2: If you could eat 60 g of carbohydrates, what quantity of ice cream would you be able to eat? 0.057 1.42 (0.77–2.60) 0.315 ** 4.37 (2.69–7.09) 0.115 * 2.12 (1.10–4.11) 0.094 1.63 (0.97–2.73) 0.596 ** Item 3: Your doctor has advised you to reduce the amount of saturated fat in your diet. You generally eat 42 g of saturated fat per day, which includes one portion of ice cream. If you were to stop eating ice cream, how many grams of saturated fat would you be consuming per day? 0.213 ** 3.37 (1.86–6.09) 0.176 ** 3.44 (1.65–7.17) 0.196 * 3.18 (1.73–5.86) 0.447 ** Nutrients 2021, 13, 790 8 of 16 Item 4: If you generally eat 2500 calories per day, what percentage of the daily value of calories would you be consuming if you ate one portion of ice cream? 0.210 ** 3.77 (1.95–7.32) 0.264 ** 3.75 (2.24–6.28) 0.662 ** Item 5: Is it safe for you to eat this ice cream? 0.586 * 63.28 (21.51– 186.15) 0.498 ** Item 6 (asked if the participant answered no to item 5): Why not? 0.599 ** * p < 0.05; ** p < 0.01. The test-retest reliability was acceptable (ICC = 0.605; 95% CI = 0.54–0.66). The majority of deviations were toward improved health literacy from the first to the second observation. Item 2 had low consistency (Table 4). Table 4. Internal consistency (coefficient α) and reproducibility of the NVS-PTeen given as temporal reliability. Coefficient α If Item Deleted (N = 386) Item Difficulty: % of Students Answering the Item Correctly (N = 386) Temporal Reliability % of Test-Retest Accuracy (n = 127) % of Test-Retest Score Improvement (n = 127) Item 1 0.58 88.3% 96.9% 0.8% Item 2 0.60 71.2% 66.1% 27.6% Item 3 0.59 86.0% 78.7% 7.1% Item 4 0.52 73.3% 81.9% 6.3% Item 5 0.53 89.4% 92.9% 3.1% Item 6 0.53 79.0% 86.6% 10.2% 3.4. Convergent Validity of the NVS-PTeen As presented in Table 5, a weak correlation was found between age and the NVSPTeen total score. The educational level, given as the number of school years the student was enrolled for at the time of data collection, and final classifications of Portuguese and mathematics from the previous academic year were moderately correlated with the NVSPTeen final score. Table 5. Convergent validity of the NVS-PTeen assessed as the correlation between theoretically related variables and the NVS-PTeen total score. Pearson’s r Age (N = 386) 0.19 * Educational level (N = 386) 0.31 ** Final classification: Portuguese 1 (N = 386) 0.28 ** Final classification: mathematics 1 (N = 358) 0.31 ** Average final classification: Portuguese and mathematics 1 (N = 358) 0.39 ** 1 Final classifications from the previous academic year; * p < 0.05; ** p < 0.001. 3.5. Construct Validity-Dimensionality The Kaiser–Meyer–Olkin (KMO) test suggested an adequate fit (KMO = 0.64) of the dataset for factorial analysis. A similar result was obtained after Barlett’s test of sphericity, which indicated that the correlation matrix was significantly different from zero (p < 0.001) and, thus, suitable for factorial analysis. Two factors were obtained by means of factorial analysis with direct oblimin rotation, eigenvalues above 1 and factor loading above 0.4. The eigenvalues for these two factors were 2.09 and 1.13, with 34.82% and 18.89% of the explained variance, respectively. The two factors, comprising the six items, explained 53.71% of the total variance (Figure 1). Factor 1 was associated with reading-related Nutrients 2021, 13, 790 9 of 16 literacy, while factor 2 was associated with numeric skills. Item 3 revealed to be weakly associated with both factors. Factor Loadings (Pattern Matrix) (a) Factor 1 Factor 2 NVS-PTeen Item 1 0.073 0.576 NVS-PTeen Item 2 −0.205 0.815 NVS-PTeen Item 3 0.321 0.265 NVS-PTeen Item 4 0.142 0.673 NVS-PTeen Item 5 0.873 −0.047 NVS-PTeen Item 6 0.879 −0.024 Explained variance (%) 34.82 18.89 Alpha coefficient 0.58 0.48 Kaiser–Meyer–Olkin (KMO) = 0.64 Bartlett’s test of sphericity = 297.03, p < 0.001 Overall/cumulative variance (%) = 53.71 (a) In bold are indicated factor loads higher than 0.40. Figure 1. Exploratory factor analysis after direct oblimin rotation (with the screen test plot of eigenvalues) for the NVSPTeen (N = 386). 3.6. Item Response Theory (IRT): Item Difficulty, Discrimination, and Fit After adjusting for spuriousness, i.e., after removing the variance attributable to the NVS-PTeen total score due to the item-specific variance, IRT curves (Figure 2) revealed that items 4, 5, and 6 were the most discriminative ones (a = 0.42), whereas items 2 and 3 were the least discriminative ones (a = 0.26). Concerning item difficulty, items 3 and 6 scored as the easiest ones (b = −0.37 and b = 0.24, respectively), whereas item 2 was the most difficult one (b = 0.82). Finally, the fit between observed and expected responses was adequate for all items (UMS, unweighted mean squares, and WMS, weighted mean squares, values ranged between 0.8 and 1.2), except for item 5. Figure 2. NVS-PTeen items: characteristic and information curves (N = 386). Nutrients 2021, 13, 790 16 of 16 48. MacLeod, S.; Musich, S.; Gulyas, S.; Cheng, Y.; Tkatch, R.; Cempellin, D.; Bhattarai, G.R.; Hawkins, K.; Yeh, C.S. The impact of inadequate health literacy on patient satisfaction, healthcare utilization, and expenditures among older adults. Geriatr. Nurs. (Minneap). 2017, 38, 334–341, doi:10.1016/j.gerinurse.2016.12.003. 49. Shieh, C.; Halstead, J.A. Understanding the impact of health literacy on women’s health. J. Obstet. Gynecol. Neonatal Nurs. 2009, 38, 601–612, doi:10.1111/j.1552-6909.2009.01059.x. 50. Baker, D.W. The meaning and the measure of health literacy. J. Gen. Intern. Med. 2006, 21, 878–883, doi:10.1111/j.15251497.2006.00540.x. 51. Weiss, B.D. The use of the Newest Vital Sign in children. HLRP Heal. Lit. Res. Pract. 2019, 3, e19, doi:10.3928/24748307-2019012202. 52. Keating, D.P. Cognitive and brain development in adolescence. Enfance 2012, 3, 267–279, doi:10.4074/S0013754512003035. 53. Rosa, A.; Loureiro, L.; Sequeira, C. Literacia em saúde mental sobre depressão: um estudo com adolescentes portugueses. Rev. Port. Enferm. Saúde Ment. 2019, 21, 40–46, doi:10.19131/rpesm.0236. 54. Manganello, J.A.; DeVellis, R.F.; Davis, T.C.; Schottler-Thal, C. Development of the Health Literacy Assessment Scale for Adolescents (HAS-A). J. Commun. Healthc. 2015, 8, 172–184, doi:10.1179/1753807615Y.0000000016doi.org/10.1179/1753807615Y.0000000016. 55. Público Comunicação Social SA Ranking das Escolas 2017 Available online: https://www.publico.pt/ranking-das-escolas2017/lista (accessed on Dec 30, 2020). 56. Driessnack, M.; Chung, S.; Perkhounkova, E.; Hein, M. Using the “Newest Vital Sign” to assess health literacy in children. J. Pediatr. Heal. Care 2014, 28, 165–171, doi:10.1016/j.pedhc.2013.05.005. 57. Kogure, T.; Sumitani, M.; Suka, M.; Ishikawa, H.; Odajima, T.; Igarashi, A.; Kusama, M.; Okamoto, M.; Sugimori, H.; Kawahara, K. Validity and reliability of the Japanese version of the Newest Vital Sign: a preliminary study. PLoS One 2014, 9, e94582, doi:10.1371/journal.pone.0094582. 58. Mansfield, E.D.; Wahba, R.; Gillis, D.E.; Weiss, B.D.; L’Abbé, M. Canadian adaptation of the Newest Vital Sign©, a health literacy assessment tool. Public Health Nutr. 2018, 21, 2038–2045, doi:10.1017/S1368980018000253. 59. Tseng, H.-M.; Liao, S.-F.; Wen, Y.-P.; Chuang, Y.-J. Adaptation and validation of a measure of health literacy in Taiwan: the Newest Vital Sign. Biomed. J. 2018, 41, 273–278, doi:10.1016/j.bj.2018.07.001. 60. Welch, V.L.; VanGeest, J.B.; Caskey, R. Time, costs, and clinical utilization of screening for health literacy: a case study using the Newest Vital Sign (NVS) instrument. J. Am. Board Fam. Med. 2011, 24, 281–9, doi:10.3122/jabfm.2011.03.100212. 61. Hoffman, S.; Marsiglia, F.F.; Lambert, M.C.; Porta, M. A psychometric assessment of the Newest Vital Sign among youth in Guatemala city. J. Child Adolesc. Behav. 2015, 3, 190, doi:10.4172/2375-4494.1000190. 62. Warsh, J.; Chari, R.; Badaczewski, A.; Hossain, J.; Sharif, I. Can the Newest Vital Sign be used to assess health literacy in children and adolescents? Clin. Pediatr. (Phila). 2014, 53, 141–144, doi:10.1177/0009922813504025 63. Tavakol, M.; Dennick, R. Making sense of Cronbach’s alpha. Int. J. Med. Educ. 2011, 2, 53–55, doi:10.5116/ijme.4dfb.8dfd. 64. Schmitt, N. Uses and abuses of coefficient alpha. Psychol. Assess. 1995, 8, 350–353, doi:10.1037/1040-3590.8.4.350. 65. Cruvinel, A.F.P.; Méndez, D.A.C.; Chaves, G.C.; Gutierres, E.; Lotto, M.; Oliveira, T.M.; Cruvinel, T. The Brazilian validation of a health literacy instrument: the Newest Vital Sign. Acta Odontol. Scand. 2018, 1–8, doi:10.1080/00016357.2018.1484511. 66. Zotti, P.; Cocchi, S.; Polesel, J.; Mis, C.C.; Bragatto, D.; Cavuto, S.; Conficconi, A.; Costanzo, C.; De Giorgi, M.; Drace, C.A.; et al. Cross-cultural validation of health literacy measurement tools in Italian oncology patients. BMC Health Serv. Res. 2017, 17, 410, doi:10.1186/s12913-017-2359-0. 67. Fransen, M.P.; Leenaars, K.E.F.; Rowlands, G.; Weiss, B.D.; Maat, H.P.; Essink-Bot, M.L. International application of health literacy measures: Adaptation and validation of the Newest Vital Sign in The Netherlands. Patient Educ. Couns. 2014, 97, 403– 409, doi:10.1016/j.pec.2014.08.017. 68. Fransen, M.P.; Van Schaik, T.M.; Twickler, T.B.; Essink-Bot, M.L. Applicability of internationally available health literacy measures in the Netherlands. J. Health Commun. 2011, 16, 134–149, doi:10.1080/10810730.2011.604383. 69. Rodrigues, R.; De Andrade, S.M.; González, A.D.; Birolim, M.M.; Mesas, A.E. Cross-cultural adaptation and validation of the Newest Vital Sign (NVS) health literacy instrument in general population and highly educated samples of Brazilian adults. Public Health Nutr. 2017, 20, 1907–1913, doi:10.1017/S1368980017000787. 70. Frost, M.H.; Reeve, B.B.; Liepa, A.M.; Stauffer, J.W.; Hays, R.D.; Sloan, J.A. What is sufficient evidence for the reliability and validity of patient-reported outcome measures? Value Heal. 2007, 10, S94–S105, doi:10.1111/j.1524-4733.2007.00272.x. 71. Kipping, R.R.; Campbell, R.M.; MacArthur, G.J.; Gunnell, D.J.; Hickman, M. Multiple risk behaviour in adolescence. J. Public Health (Bangkok). 2012, 34, i1–i2, doi:10.1093/pubmed/fdr122. 72. Kupersmidt, J.B.; Scull, T.M.; Austin, E.W. Media literacy education for elementary school substance use prevention: study of media detective. Pediatrics 2010, 126, 525–31, doi:10.1542/peds.2010-0068. 73. Winkelman, T.N.A.; Caldwell, M.T.; Bertram, B.; Davis, M.M. Promoting health literacy for children and adolescents. Pediatrics 2016, 138, e20161937, doi:10.1542/peds.2016-1937.