Cognitive and linguistic skills assessment protocol: adaptation and validation for European Portuguese

COGNITIVE AND LINGUISTIC SKILLS ASSESSMENT PROTOCOL: ADAPTATION AND VALIDATION FOR EUROPEAN PORTUGUESE Inês Martins1, Cristiane Lima Nunes1, Simone Aparecida Capellini2, & Graça S. Carvalho1 1University of Minho – Child Studies Centre CIEC (Portugal) 2São Paulo State University LIDA (Brazil) Abstract Linguistic and Cognitive skills play an essential role in the development of communication, language and literacy. Therefore, their assessment of school children is crucial since it allows the child's cognitive and linguistic profile characterisation, according to the school year she attends. This study intended to describe the adaptation and validation process of the instrument – Cognitive and Linguistic Skills Assessment Protocol. This Protocol was adapted from a Brazilian (Portuguese) version to a European Portuguese version to evaluate the cognitive-linguistic skills of school children (1st to 5th grade). It consists of two versions, the collective version and the individual version. The collective version consists of writing, arithmetic, auditory processing and visual processing skills; the individual version consists of reading, metalinguistic, auditory processing, visual processing and processing speed skills. After adapting the linguistic aspects (morphosyntactic and semantic), a pilot study was carried out to verify whether the instrument was well-adapted and easy to understand for the study’s target population. The sample consisted of a class for each school year, excluding children with special educational needs or intellectual/auditory deficits. A total of 75 children were evaluated: 12 children of the 1st grade; 18 of the 2nd grade; 15 of the 3rd grade; 15 of the 4th grade; and 15 of the 5th grade. The results showed that the protocol was, in general, well adapted. The instrument was then applied to a larger sample (2 classes per school year) in a total of 157 children (without special educational needs or intellectual/auditory deficits), and the data were processed in the statistical program IBM SPSS. In general, the mean values were the expected ones in all subtests of the Protocol Collective Version and some tests of the Individual Version, from the 1st to the 4th grade, but not the 5th grade, which showed non-expected mean values. This work provided the possibility for developing the subsequent phase of the study, where percentiles will be calculated to obtain the standard/normalised values to classify children’s performance as standard, above average or lower than expected. Keywords: Cognitive-linguistic skills, validating, protocol, school children. 1. Introduction The development of cognitive-linguistic competencies is fundamental for literacy development (Su et al., 2017). So, it is important that children, with the transition to the early years of schooling, have acquired the necessary skills to reach all stages of the school process, from the acquisition of reading and writing. Children should develop skills that allow them to associate, memorize, classify, interpret, compare elements (Ferreira & Tabaquim, 2017) and develop phonological awareness, knowledge of alphabet letters (Salvador & Martins, 2017; Caravolas et al., 2012) and the rapid appointment (Caravolas et al., 2012). Phonological awareness is essential in the literacy process because children have to associate a phoneme (auditory) with a grapheme (visual) in order to be able to read/write (Andrade, Andrade & Capellini, 2014; Germano & Capellini, 2015; Oliveira, Germano & Capellini, 2016). Pre-school education plays a vital role in developing these skills (Moniz, 2009; Ferreira & Tabaquim, 2017). It should be noted that during primary school, children learn to read/write but, from then on, reading/writing are the skills that allow the individual to learn new contents throughout their life (Oliveira & Capellini, 2016) and promote their insertion in society (Bandini et al., 2014). The identification of difficulties in cognitive-linguistic competencies, particularly in the first years of schooling, is important. After identifying difficulties, a specialized intervention can be done Education and New Developments 2021 549 before significant perturbations can be manifested in the student's school performance (Navas, 2011; Nicolau & Navas, 2015). The choice of adaptation of this protocol is justified because there is a shortage of instruments that systematically and rapidly assess the cognitive-linguistic competencies involved in the learning process in Portugal. 2. Objectives and instrument description This study intended to describe the adaptation and validation process of the instrument – Cognitive and Linguistic Skills Assessment Protocol (CLSAP) (Capellini, Smythe & Silva, 2012). This Protocol was adapted from a Brazilian (Portuguese) version to a European Portuguese version to evaluate the cognitive-linguistic skills of school children (1st to 5th grade). It consists of two versions, the collective version and the individual version. The collective version consists of 5 subtests: (S1) writing the alphabet in sequence; (S2) copy shapes; (S3) mathematical calculation; (S4) dictation of words and pseudowords; and (S5) repetition of numbers in random order. The individual version consists of 13 subtests: (S1) word reading; (S2) non-word reading; (S3) alliteration; (S4) rhyme; (S5) repetition of words; (S6) repetition of non-words; (S7) rhythm; (S8) syllabic segmentation; (S9) quick naming of figures; (S10) fast digit naming; (S11) visual memory for shapes (2 shapes to 5 shapes); (S12) discrimination of sounds; and (S13) repetition of numbers in reverse order. Data were analysed using the statistical program IBM SPSS 25. 3. Methods The CLSAP was adapted from Brazilian Portuguese, the original Protocol, to European Portuguese. In the adaptation of the CLSAP, author authorization was requested. Two types of adaptation were made: adaptation of the linguistic aspects of the text (direct adaptation of verb tenses, organization and sentence structure) and a lexical adaptation in specific tests. Given the differences between the vocabulary of Brazil and Portugal, it was necessary to do lexical exchanges in some specific tests. Some words were replaced by others, keeping, whenever possible, all phonetic and syllabic characteristics (number and syllabic constitution) (International Test Commission, 2017). After completion of the instrument, a pilot study (pre-test) was carried out to verify that they were correctly adapted and easily understood by the target population (Almeida, & Freire, 2008; Canhota, 2008). After concluding the changes in the CLPAS, the protocol was used to assess children of a larger sample to determine the protocol standard normative values for the target population. All data collection was carried out in two schools in a school group in the northern region of Portugal with convenience samples. The samples included children without special educational needs or intellectual/auditory deficits. It should be noted that written permission was requested from the director of the school group to carry out all data collection. The children ‘s parents/guardians were also informed about the investigation, authorizing their participation of their student in the study. 3.1. Pilot study The CLPAS adapted version was applied to a class of each school year (from the 1st to the 5th year) to check whether the instrument was correctly adapted or if it would be necessary to make some improvement changes. A total of 75 children were evaluated: 12 children of the 1st grade; 18 of the 2nd grade; 15 of the 3rd grade; 15 of the 4th grade; and 15 of the 5th grade. The results showed that the Protocol was, in general, well adapted, and only three words were replaced in the whole Protocol. The adaptation of the protocol underwent some vocabulary adjustments since most students made changes in their reading (1 word in subtest 1) or were unaware of the meaning of the words presenting difficulties in their repetition (1 word in subtest 4 and 1 word in subtest 8). The substituting words keep the same number of syllables and syllabic constitution of the original word (disyllabic word: consonant (C) vowel (V) - consonant (C) vowel (V) vowel (V); trisyllabic word: CV-CV -CVC; disyllabic word: CV-CVV). One must be careful to maintain the structure of the original words to ensure that the adaptation is as reliable as possible. With these small changes, the CLSAP was completed and ready for the next phase of data collection. 3.2. Main study The CLSAP final version was applied to a larger sample (2 classes per school year, from the 1st to the 5th year). Of the 205 evaluated children, six were excluded since they were not present in the joint p-ISSN: 2184-044X e-ISSN: 2184-1489 ISBN: 978-989-54815-8-3 © 2021 550 evaluation session. Of the remaining 199 children, other 42 children were excluded because they had special educational needs or intellectual / hearing deficits. Of the 157 children included in the study, 79 were girls and 78 boys, aged between 6 and 10 years. They were distributed as follows: 31 children from the 1st year, 34 from the 2nd, 31 from the 3rd, 31 from the 4th and 30 from the 5th year. Mean, minimum and maximum values were calculated. Table 1 shows that all subtests of the Protocol Collective Version, mean values are increasing from the 1st to the 4th grade, as expected, but not on the 5th grade, which has a lower mean value. Table 1. Results of Cognitive Skills Assessment Protocol (CLSAP) - Collective Version. Grades Numbers (N) and Scores Cognitive and Linguistic Skills Assessment Protocol (CLSAP) Collective Version S1 S2 S3 S4 S5 1st N 31 31 31 31 31 Min. 10 2 2 4 4 Max. 26 4 8 31 7 Mean 23,19 3,29 5,16 15,10 5,35 2nd N 34 34 34 34 34 Min. 9 2 4 16 4 Max. 26 4 18 37 9 Mean 23,91 3,65 9,65 26,18 6,06 3rd N 31 31 31 31 31 Min. 20 3 8 21 2 Max. 26 4 19 37 10 Mean 25,42 3,77 15,32 31,42 7,19 4th N 31 31 31 31 31 Min. 22 3 10 21 6 Max. 26 4 20 37 10 Mean 25,71 3,87 18,06 31,97 7,84 5th N 30 30 30 30 30 Min. 21 3 13 20 5 Max. 26 4 20 38 9 Mean 25,07 3,77 17,47 31,83 7,20 Total N 157 157 157 157 157 Similarly, through the subtests 1 to subtest 8 of the Protocol Individual Version, mean values are increasing from the 1st to the 4th grade, as expected, but not the 5th grade, which has a lower mean value in subtests 2, 6 and 7. Table 2. Results of Cognitive Skills Assessment Protocol (CLSAP) - Individual Version: Subtest 1-Subtest8. Grades Numbers (N) and Scores Cognitive and Linguistic Skills Assessment Protocol (CLSAP) Individual Version S1 S2 S3 S4 S5 S6 S7 S8 1st N 31 31 31 31 31 31 31 31 Min. 0 2 3 5 2 4 2 12 Max. 17 8 10 20 5 7 7 20 Mean 7,58 4,42 6,42 13,26 3,58 5,42 4,39 17,77 2nd N 34 34 34 34 34 34 34 34 Min. 5 2 3 6 2 3 2 10 Max. 70 10 10 20 7 7 8 20 Mean 37,44 7,24 7,38 15,41 4,38 6,09 4,68 18,85 3rd N 31 31 31 31 31 31 31 31 Min. 29 5 6 8 3 5 1 14 Max. 69 12 10 20 7 7 9 20 Mean 49,71 8,19 8,90 17,35 4,58 6,29 5,81 18,90 4th N 31 31 31 31 31 31 31 31 Min. 37 6 5 9 3 4 4 13 Max. 69 10 10 20 7 7 9 20 Mean 56 8,23 9 17,97 5,26 6,61 6,52 19,26 5th N 30 30 30 30 30 30 30 30 Min. 35 5 4 15 3 4 2 17 Max. 70 10 10 20 7 7 10 20 Mean 61,63 8,20 9,03 18,50 5,27 6,33 6,23 19,53 Total N 157 157 157 157 157 157 157 157 Education and New Developments 2021 551 Table 3 shows that in subtest 9 and 10, de mean values decrease from 1st to the 5th grade, because the results of these subtests are counted in seconds. In subtests 11 and 12, there are no significant differences between the five grades. In general, in subtest 13, the mean value increases with schooling. Table 3. Results of Cognitive Skills Assessment Protocol (CLSAP) - Individual Version: Subtest 9-Subtest13. Grades Mean (M) Cognitive and Linguistic Skills Assessment Protocol (CLSAP) Individual Version S9 (seconds) S10 (seconds) S11 S12 S13 2 shapes 3 shapes 4 shapes 5 shapes 1st N 31 31 31 31 31 31 31 31 Min. 29 72 1 0 0 0 17 2 Max. 67 179 2 2 1 1 20 4 Mean 45,55 106,85 1,71 0,97 0,29 0,13 19,58 3 2nd N 34 34 34 34 34 34 34 34 Min. 25 58 0 0 0 0 15 2 Max. 51 129 2 2 2 1 20 6 Mean 36,32 82,44 1,53 1,03 0,29 0,09 19,21 3,47 3rd N 31 31 31 31 31 31 31 31 Min. 25 48 1 0 0 0 17 2 Max. 55 110 2 2 1 2 20 7 Mean 36,77 71,90 1,61 1,06 0,52 0,42 19,58 4,55 4th N 31 31 31 31 31 31 31 31 Min. 24 43 0 0 0 0 17 2 Max. 44 80 2 2 2 2 20 9 Mean 32,42 63 1,70 1,10 0,77 0,23 19,67 5,03 5th N 30 30 30 30 30 30 30 30 Min. 22 43 1 0 0 0 19 3 Max. 42 81 2 2 2 2 20 7 Mean 29,43 54,80 1,93 1,33 0,53 0,40 19,93 5,03 Total N 157 157 157 157 157 157 157 157 4. Discussion/ conclusions After data analyses, it is possible to conclude that these results are, in general, expected. Mean values increase from the 1st to the 4th grade, excepting subtests 9 and 10 of the CLSAP Individual Version. In those subtests mean values decrease with schooling because the results of these subtests are counted in seconds instead of counting points. These results are justified by the fact of children at primary school are maturing, developing, improving their linguistic, cognitive (Nippold, 2006) and auditory processing skills (Krizman, et al., 2015) that allows them to develop literacy. The exception is the 5th grade, which has lower mean values in the CLSAP Collective Version and some subtests of the CLSAP Individual version (subtests 2, 6 and 7). Several reasons can explain these non-expected results: subtests with lower sensitivity to evaluate children of the 5th year; heterogeneous classes (more students with lower results); unidentified students with special educational needs or intellectual/auditory deficits, among others. Despite the non-expected results of the 5th grade, the CLSAP Protocol seems to be a reliable instrument that allows a screening evaluation of children from 1st to 4th grades, allowing the identification of children at risk of developing learning problems. A more exhaustive and individualized assessment can then be initiated, referrals to different health professionals, promoting a multidisciplinary investigation, confirming or ruling out diagnoses, and promoting an adjusted intervention to the child’s needs. The next phase of the study percentiles will be calculated to obtain the standard/normalized values that define a children’s cognitive-linguistic profile, identify areas with normative values, and identify problematic areas in which the children obtained values lower than expected. References Almeida, L. S., & Freire, T. (2008). Metodologia da investigação em psicologia e educação (5ª ed.). Braga: Psiquiíibrios Edições. Andrade, O. V. C. A., Andrade, P. E., & Capellini, S. A. (2014). Caracterização do Perfil Cognitivo-Linguístico de Escolares com Dificuldades de Leitura e Escrita. Psicologia: Reflexão e Crítica, 27(2), 358-367. doi:10.1590/1678-7153.201427217 p-ISSN: 2184-044X e-ISSN: 2184-1489 ISBN: 978-989-54815-8-3 © 2021 552 Bandini, C. S. M., Bandini, H. H., Motta, S. A. C., & Souza, D. G. (2014). Emergence of Reading and Writing in Illiterate Adults After Matching-to-Sample Tasks. Paidéia (Ribeirão Preto), 24(57), 75-84. doi:10.1590/1982-43272457201410 Canhota, C. (2008). Qual a importância do estudo piloto? In E. E. Silva (Org.). Investigação passo a passo: perguntas e respostas para investigação clínica (pp. 69-72). Lisboa: APMCG. Capellini, S. A., Smythe, I., & Silva, C. (2012). Protocolo de avaliação de habilidades cognitivo-linguísticas. Marília: Fundep, Ed. Caravolas, M., Lervåg, A., Mousikou, P., Efrim, C., Litavsky, M., Onochie-Quintanilla, E., Salas, N., Schöffelová, M., Defior, S., Mikulajová, M., Seidlová-Málková, G., & Hulme, C. (2012). Common patterns of prediction of literacy development in different alphabetic orthographies. Psychological Science, 23(6), 678–686. doi:10.1177/0956797611434536 Ferreira, F. R., & Tabaquim, M. L. M. (2017). Habilidades cognitivas de escolares do ensino público e privado: estudo comparativo de pré-competências para a aprendizagem acadêmica. Revista Psicopedagogia, 34(104), 126-136. International Teste Commission. (2018). ITC Guidelines for Translating and Adapting Tests (Second Edition). International Journal of Testing, 18(2), 101-134. doi: 10.1080/15305058.2017.1398166 Krizman, J., Tierney, A., Fitzroy, A.B., Skoe, E., Amar, J., & Kraus, N. (2015). Continued maturation of auditory brainstem function during adolescence: A longitudinal approach. Clinical Neurophysiology, 126, 2348-2355. Nicolau, C. C., & Navas, A. L. G. P. (2015). Avaliação das habilidades preditoras do sucesso de leitura em crianças de 1º e 2º anos do ensino fundamental. Revista CEFAC, 17(3), 917-926. Doi:10.1590/1982-021620157214 Nippold, M. (2006). Later language development: School-age children, adolescents, and young adults (3ª ed.). Austin: Pro Ed. Oliveira, A. M., & Capellini, S. A. (2016). E-LEITURA II: banco de palavras para leitura de escolares do Ensino Fundamental II. CoDAS, 28(6), 778-817. doi:10.1590/2317-1782/20162016049 Oliveira, A. M., Germano, G. D., & Capellini, S. A. (2016). Desempenho de escolares em provas de processo de identificação de letras e do processo léxico. Revista CEFAC, 18(5), 1121-1132. doi:10.1590/1982-0216201618523315 Salvador, L., & Martins, M. A. (2017). Práticas de literacia familiar, competências linguísticas e desempenho em leitura no primeiro ano de escolaridade. Análise Psicológica, 1(34), 1-12. doi:10.14417/ap.1172 Su, M., Peyre H., Song, S., McBride, C., Tardif, T., Li, H., Zhang, Y., Liang, W., Zhang, Z., Ramus, F., & Shu, H. (2017). The influence of early linguistic and family factors on literacy acquisition in Chinese children: Follow-up from age 3 to age 11. Learning and Instruction, 49, 54-63. doi:10.1016/j.learninstruc.2016.12.003 Education and New Developments 2021 553 Education and New Developments 2021 Edited by Mafalda Carmo Edited by Mafalda Carmo, World Institute for Advanced Research and Science (WIARS), Portugal Published by inScience Press, Rua Tomas Ribeiro, 45, 1º D, 1050-225 Lisboa, Portugal Copyright © 2021 inScience Press All rights are reserved. Permission is granted for personal and educational use only. Commercial copying, hiring and lending is prohibited. The whole or part of this publication material cannot be reproduced, reprinted, translated, stored or transmitted, in any form or means, without the written permission of the publisher. The publisher and authors have taken care that the information and recommendations contained herein are accurate and compatible with the generally accepted standards at the time of publication. The individual essays remain the intellectual properties of the contributors. ISSN (electronic version): 2184-1489 ISSN (printed version): 2184-044X ISBN: 978-989-54815-8-3 Legal Deposit: 428062/17 Printed in Lisbon, Portugal by GIMA - Gestão de Imagem Empresarial, Lda.