Human-made news vs AI-generated news: a comparison of Portuguese and Spanish journalism students’ evaluations

ARTICLE Human-made news vs AI-generated news: a comparison of Portuguese and Spanish journalism students’evaluations João Pedro Baptista1,2✉, Rubén Rivas-de-Roca 3, Anabela Gradim4& Concha Pérez-Curiel5 This study examines how university students in Portugal and Spain perceive the quality of journalistic news produced by generative artificial intelligence (AI). As AI—particularly generative models like ChatGPT-3—becomes increasingly integrated into news distribution and newsroom operations, it offers the promise of greater efficiency and the reduction of non-value-added tasks. However, its rise also raises concerns about potential risks, ethical implications, and the evolving relationship between human journalists and AI-driven content creation. The study focuses on the perceptions of journalism students, addressing key research questions related to their ability to discern AI-generated news, comparing the quality between AI-generated news and news written by journalists, and the influence of news topics in assessing quality. We applied two surveys to 444 participants, evaluating dimensions of news quality, such as readability, informativeness and style. The findings indicate that students generally rated news generated by ChatGPT-3 as being of higher quality than news written by journalists. Students in Spain gave higher ratings to AIgenerated news. The findings suggest a potential shift in perceptions of news quality, promoting discussions about the evolving role of AI in journalism education and its broader implications for the industry. https://doi.org/10.1057/s41599-025-04872-2 OPEN 1Department of Letters, Arts and Communication, University of Trás-os-Montes and Alto Douro, 5000 Vila Real, Portugal. 2CICANT—The Centre for Research in Applied Communication, Culture, and New Technologies, Lusófona University, 1749 Lisbon, Portugal. 3Department of Communication Sciences, Faculty of Communication Sciences, University of Santiago de Compostela, 15782 Santiago de Compostela, Spain. 4Labcom, University of Beira Interior, 6200 Covilhã, Portugal. 5Department of Journalism II, Faculty of Communication, University of Seville, 41092 Seville, Spain. ✉email: [email protected] HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 1 1234567890():,; Introduction Algorithms play a crucial role in the contemporary media ecosystem, significantly influencing the distribution of news content. The use of artificial intelligence (AI) is one of the main technological innovations in journalism, being understood as the peak of automation in information (Diakopoulos 2020) and the latest example of the Fourth Industrial Revolution (Sixto-García et al. 2024). While journalists traditionally held exclusive authority in news creation, generative artificial intelligence (AI) could now play a pivotal role not only in distribution but also in the day-to-day operations of newsrooms. Humans are no longer solely responsible for selecting and writing news; the digital evolution has given rise to robot journalists (Lemelshtrich Latar 2018). The use of AI causes tensions in the news industry as there are several nuances in scope and use (Moran and Shaikh 2022). Some newsrooms are skeptic of this technology, but at the same time AI tools are entering into the work routines of different news media outlets worldwide (Gutiérrez-López et al. 2023; Munoriyarwa et al. 2023). In the field of journalism, automation is not new as different initiatives have resorted to this technology to assist newsroom practices (Clerwall 2014). Besides that, AI is a term that involves different technologies such as a machine learning or natural language processing, which have a long history in shaping a hi-tech journalism (Anderson 2018). However, the launch of ChatGPT at the end of 2022 has opened a new era for AI, making it possible to automate some journalistic processes (Gutiérrez-Caneda et al. 2023). ChatGPT is a generative AI tool developed by OpenAI company. It is based on a simple user interface, which is available in different languages. As generative AI is a sort of AI that can produce several types of content, its possibilities in journalism and education are vast (Pavlik 2023). It should be noted that automation implies to set up pre-defined rules, whereas AI makes its own decisions through machine-learning models. Although the use of robots has been experimental and limited to some actors of the news media sector (de-Lima-Santos and Ceron 2021), early studies reveal that AI improves journalistic coverage by reducing reporters’non-value-added tasks, especially in the areas of Sport and Finance (Canavilhas 2022). Portraits of AI in media outlets in recent years point to a news coverage that normalizes this kind of technology, mixing optimistic and pessimistic approaches (Cools et al. 2024). Nevertheless, the literature also detected how generative AI is little used to create texts, together with a lack of AI experts in the newsrooms (Quian and Sixto-García 2024). The analyses of this hi-tech journalism are usually marked by discussions on their risks, even in the news media. There are criticisms of the inability to distinguish true and false content due to the proliferation of AI-enabled disinformation products (Whyte 2020) and the lack of information about sources (González-Arias and López-García 2023), together with ethical issues, since AI brings a new relationship between humans and machines. Moreover, it is important to acknowledge the difference between ‘fake news’, as stories written to convey false or misleading information, and the incidental hallucinations introduced by generative AI. Both phenomena cause problems in news credibility. As a result, the debate surrounding AI in journalism has sparked varying opinions in literature. Some argue that AI serves as a valuable tool for news organizations, freeing journalists from repetitive tasks and enabling the creation of more complex reports or increased investment in investigative journalism. Others express pessimism towards the future of the sector. Academia and the profession itself tend to emphasize instrumental approaches to AI, fueling computer-assisted reporting (Deuze and Beckett 2022). Along with these debates, Journalism education has also been affected by the rise of AI. According to Westlund and Lewis (2017), students need to learn algorithm management with AI in higher education. These technological transformations introduce changes in news production, based on news routines and quantitative contents that may require instruction in prompt engineering. Journalism professors underline the role of robots as a helping hand in the newsroom (Calvo-Rubio and Ufarte-Ruiz 2020), but prior scholarship has paid little attention to how less human editing is being taken advantage to promote disinformation messages and low-quality news. Our research examines the perceived quality of news generated by Chat GTP-3 among journalism students in Portugal and Spain. This research follows up on another study focused exclusively on Portugal (Baptista and Gradim 2024). AI and the training of journalists in Spain and Portugal Algorithmic journalism provides audience-oriented content, since the patterns of the public are seen as quantifiable through big data (Anderson 2013). The use of AI to assist reporting is another example of the quantitative turn that is now prevalent in professional journalism (Coddington 2015). The massive use of this technology requires a proper training on algorithms and journalism, with a special focus on technical and ethical competencies (Lopezosa et al. 2023). Nevertheless, some evidence points out the scant presence of AI in the curricula of communication Bachelor and Master degrees in Spain (Gómez-Diago 2022). Our study addresses Spain and Portugal as both countries belong to the same media system, which explains some previous comparative analysis of the Iberian news landscape in terms of disinformation (Baptista et al. 2023). The literature has identified some successful AI initiatives in these countries, using case studies and surveys as methods. These practices on automated journalism are still exploratory but seem to be more developed in Spain. According to research carried out by Calvo-Rubio and UfarteRuiz (2020) with heads of news media, journalists, university professors, and students of Communication Studies in Spain, these stakeholders are convinced AI will not have a negative impact on the journalistic sector. These findings contrast with the previously identified lack of knowledge about AI among Spanish journalists (Túñez-López et al. 2018) and highlight the need for solid training in automated journalism (Calvo-Rubio and UfarteRuiz 2020). The concern over limited AI knowledge is also evident in Portugal, where surveys of sports journalists indicate that economic constraints prevent its adoption in newsrooms (Canavilhas 2022). This lack of financial resources is a pattern in other Portuguese-speaking countries such as Brazil, but the Brazilian media resort more to automated journalism, particularly in the field of sports (Canavilhas and Giacomelli 2023). Unsurprisingly, AI facilitates efficient work when reporting routine activities like sports scores. Beyond specific areas, the application of questionnaires to journalists of the main daily newspapers in Portugal (Correio da Manhã, Diário de Notícias, Jornal de Notícias, and Público) highlighted not only the poor knowledge of AI and the required training on the subject, but also that Portuguese journalists are open to using this technology (Gonçalves and Melo 2022). Comparative studies on the use of AI in the Iberian Peninsula are few, but already show automated journalism is more available within Spanish newsrooms; meanwhile, in Portugal the higher education institutions or Google’s Digital News Innovation Fund play a key role in fostering computer-assisted reporting (PérezSeijo et al. 2023). ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-025-04872-2 2HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 Another key difference between Portugal and Spain is the use of AI-driven tools in Spain to produce personalized content for local communities (Noain-Sánchez, 2022). Despite globalization, people still seek information about their neighborhoods and villages. In this context, the automatic generation of news from datasets can help meet information needs that traditional newsrooms have overlooked, as happened with the project of automated news coverage by RTVE (Spanish public broadcasting service) for the 2023 municipal elections (Aramburú Moncada et al. 2023). Algorithmic journalism encompasses various AI techniques that enable computational news discovery (Diakopoulos 2020). These AI-driven journalistic insights have implications for concepts such as newsworthiness and quality, underscoring the need for proper training in AI management. In Spain, journalists began to receive training on how to apply automation, but the instruction on this matter is still at early stages (Ufarte et al. 2020a). As a solution, they propose the inclusion of generative AI on Journalism degrees curricula. Even though automated journalism has gained ground, the educational training in AI at Spanish universities is insufficient, with only few proposals that tackle the technological and ethical competences required for future journalists (Ufarte et al. 2020b). Teaching initiatives to incorporate AI into university courses remain limited but follow two main approaches: a critical perspective, which examines the ethical and social consequences of automation, and an applied perspective, which focuses on data processing and content creation (Gómez-Diago 2022), in which instruction in prompt engineering could also be needed. The literature has not considered the absence of AI training in bachelor’s and master’s degrees in communication in Portugal, where the focus is much more oriented to media literacy, combining the action of primary and secondary educators and journalists to fight disinformation at the local level. In these initiatives, journalists act as trainees and trainers (Foà et al. 2023), for instance sharing verification protocols. In times of platformization, continuous training is essential for both journalists and students. Media literacy requires strengthening knowledge of both traditional and digital technologies (Tomé et al. 2023). However, as AI is increasingly used to generate false content (Karinshak and Jin 2023), its role in disinformation is reshaping the journalism industry. Despite this, a research gap remains in addressing AI within university education, particularly regarding students’perceptions. Accordingly, our study poses the following research questions: RQ1 - How well can journalism students distinguish AIgenerated news from human-written news? RQ2 - Does the news subject influence the quality attributed to both news? Materials and methods This study compares how university students in Portugal and Spain evaluate the textual quality of news produced by ChatGPT3 or created by journalists. Two classroom surveys were distributed separately in the respective countries and at different periods during the first semester of the academic year 2023–2024. All participants responded voluntarily and anonymously to the surveys. The exploratory sample consisted of a total of 444 students, 99 in Portugal and 345 in Spain, considering that the Spanish university system has much more students than the Portuguese. The subjects are aged between 18 and 65 years old, 57.9% are female and 41.9% are male; whereas the response rate was 100% since the activity took place in the classrooms and all the target students participated. In the Iberian countries around 4000 journalism students graduate each year (Ministerio de Ciencia, Innovación y Universidades 2024; Canavilhas and Di Fátima 2025). Our research focuses on this sample, and not a wider population, since these students are future journalists and their perceptions on generative AI may have implications in journalistic practices, including key concepts such as ethics or news quality. Spain and Portugal were selected because they are countries with a cultural proximity and belong to the same media system (Hallin and Mancini 2004), even though the Spanish political sphere is much more polarized, which led us to expect some differences. Prior comparative studies have shown how some Portuguese news media are more likely to adopt technological tools (Rivas-de-Roca 2022), in which AI might be included. The study is based on three procedures. First, there was a selection and collection of news written by journalists. Second, a set of prompts were extracted from the news for ChatGPT-3 to create the same news. The prompts consisted of six keywords extracted from the original news, following the answer to the traditional journalistic Lead questions, the six W’s (who, what, when, where, why, how). Third, two surveys were created with different languages, in Portuguese and Spanish. Procedures. 1) Selection and collection of news We chose to select news from two newspapers with national coverage in both countries. In Portugal, the sample was collected from the website of the newspaper ‘Correio da Manhã’(https:// www.cmjornal.pt) and, in Spain, from the digital newspaper ‘OKdiario’(https://okdiario.com). We chose these two newspapers because they emphasize the same news values, such as celebrities, crimes, accidents and culture. Particularly, they were selected due to their sensationalist approach, based on the assumption that biased information and clickbait practices could be easily replaced by AI. Both media outlets are characterized by a sensationalist agenda, but Correio da Manhã is a long-established newspaper (founded in 1979) that has a print version, while OKdiario is a digital native created in 2015 by Eduardo Inda, a well-known controversial journalist in Spain. Despite some failed attempts, the Spanish media landscape does not have printed tabloid press, which is not the case in most European countries. Print news related to celebrities, crime, accidents and culture were selected and then replicated on ChatGPT-3. These are topics associated with soft news and are often less complex and less subject to opinion, criticism and judgment. Topics such as crime and accidents result in more factual news reporting and were thought more suitable to create prompts for ChatGPT-3. For the questionnaire, we randomly chose one news item, in Portuguese and Spanish, for each topic. The prompts were created based on keywords closely related to the writing technique of news leads. The prompt keywords answered to the six classic journalistic questions (who, what, where, why, how, when). Figure 1is a screenshot that exemplifies how the prompts were presented to ChatGPT-3, on a news item in the ‘accident’category. We asked the chat the following: “can you write a news story with the following keywords: man, 40 years old [who?], Santa Maria da Feira [where?], work accident [why], forklift [how?] and Friday [when?]. We replicated the same procedure for the remaining news. ChatGPT created a news story about a 40-yearold man who suffered an accident at work in Santa Maria da Feira, a town in Portugal, generating the following headline: “40-year-old man suffers an accident at work in a cork industry company in Santa Maria da Feira”. In Table 1we can find the headlines of the news written by journalists and generated by AI and the respective online media. HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-025-04872-2 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 3 2) Survey Both surveys have three sections. First, participants were asked to answer sociodemographic questions. Secondly, the survey presented questions to understand students’news consumption habits and practices. Third, participants were exposed to four news stories created by journalists and the same four news stories generated by ChatGPT-3. Each news item corresponds to a category: accident, crime, celebrity and culture (survey can be consulted in the Supplementary Materials). Students did not receive any alert about the existence of news generated by AI, to ensure the neutrality of their assessment, as other studies have done when measuring, for example, students’ ability to distinguish fake news from real news (Baptista et al. 2022). At the start of the survey, participants were informed that the study aimed to assess how audiences evaluate the quality of online news. Furthermore, both news created by journalists and AI were presented in an identical template, designed exclusively for the research (Fig. 2). As Fig. 2shows, the same event was presented in two different news stories, one created by ChatGPT and the other created by journalists. In the survey, distributed in Portugal, the news generated by ChatGPT were presented as having been published by the fictitious newspaper called “Noticiário”. News written by journalists was attributed to the fictitious newspaper called “Notícias”. In Spain, Okdiario news was presented by the fictional newspaper ‘Periódico’and ChatGPT news was attributed to the fictional newspaper ‘Informativo’. To ensure a textual evaluation of the news and that no other factors interfered with the students’ evaluation, the news was not only presented in the same template, but no author was attributed to the text, as it is well known that the name of the author can influence the credibility of the news (Ruffo and Semeraro 2022). The same image was also always used for all news on the same subject. The image that illustrates each news item, from each category, was taken from the news published by the newspapers Correio da Manhã and Okdiario. Regarding the assessment of the textual quality of news, we used a quality indicator, adapted from Arapakis et al. (2016), composed of three dimensions: readability, informativeness and style. As we intended for students to evaluate the news based exclusively on the article itself, without considering other contexts or factors, the survey questions were designed according to these criteria. A 7-point Likert scale was used (1—Strongly Disagree; Fig. 1 Screenshot: example of prompts posted to ChatGPT-3. Source: Own elaboration. Table 1 News headlines written by journalists and news headlines generated by ChatGPT by each newspaper. Newspaper News headline ChatGPT headline Correio da Manhã Joana Duarte became a mother for the first time. Actress Joana Duarte gives birth for the first time at Hospital da Luz and preserves her father’s anonymity Man seriously injured in work accident in Santa Maria da Feira 40-year-old man suffers an accident at work in a cork industry company in Santa Maria da Feira Man kills woman with rear-naked choke in Lisbon Woman is murdered in her home by her partner with a rear-naked choke in Lisbon Painting “Banana Woman”by Paula Rego sold at auction for 266 thousand euros Painting ‘Banana Woman’by Paula Rego sells for 266 thousand euros at modern and contemporary art auction OKdiario What is Miley Cyrus’new Mullet haircut like. Miley Cyrus causes a stir on social networks by debuting a daring mullet on Instagram Tragic accident at a party in Israel: the floor of a swimming pool gives way and ‘swallows’two people Tragic accident in Israel during summer party leaves one man dead and another injured in pool collapse Prison for the detainee for killing the boy Mohamed in Ceuta for crimes of murder and sexual assault A man arrested in Ceuta accused of murder and sexual assault against a six-year-old boy Ariana Grande received this large amount of money for her ‘Coachella’concert Ariana Grande received $8 million for her spectacular performance at the Coachella festival Source: own elaboration. ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-025-04872-2 4HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 7—Strongly Agree), where a higher score always means higher quality. We considered that the readability of a text is associated, above all, with the ease of reading a text. This may be related to the length, positioning and organization of sentences, which implies the fluency and conciseness of the text (Zamanian and Haydari 2012; Arapakis et al. 2016). Considering this dimension, students evaluated the news according to the following parameters: (a) easy to read and interpret; (b) consistent and accurate presentation of facts. In the informativeness dimension, Arapakis et al. (2016) integrates several fundamental aspects, particularly at the level of description. The degree of description, for example, of a title in line with the content of the news body is a quality parameter. For example, clickbait titles are not very descriptive and do not correspond to the expectations created by users, offering little information (García Orosa et al. 2017). In addition to descriptiveness, Arapakis et al. (2016) also consider completeness and referencing as parameters of the informativeness dimension. On the one hand, news articles must cover a topic in detail and with enough information to support the allegations. On the other hand, the journalistic text must contain sources and quotations that are easily understood by readers. Thus, in terms of informativeness, university students classified the title of each news item, namely on how well it reflects its main content. The way in which a certain subject or topic is reported was also evaluated. We questioned students about whether the news provides enough information about the story. When evaluating informativeness, students also expressed their opinion on the use of news sources in an appropriate and balanced way. As for the third dimension of textual quality, style, as Arapakis et al. (2016) we consider aspects such as the use of punctuation, journalistic writing guidelines and the use of grammatical rules. Results Our results show that Iberian university students considered, on average, news generated by ChatGPT to be of higher quality (M =4.85, SD =0.82) than news written by journalists (M =4.30, SD =0.84). The differences between the two groups are statistically significant [t(444) =11.919, p< 0.001, d =0.566]. Even dividing the sample by country of participants, we found that both groups generally considered the news created by ChatGPT-3 to be of higher quality. However, Spanish university students gave a higher rating to ChatGPT-3 news (M =4.92, SD =0.80) compared to the average rating given by Portuguese students (M =4.59, SD.84). Regarding the perceived quality of news written by journalists, students from Spain gave on average the lowest rating (M =4.33, SD =0.79). The univariate analysis of variance (ANOVA) revealed statistically significant differences only in relation to the perceived quality of ChatGPT news, when comparing the two countries (F(1,444) =12.238, p< 0.001). Across the entire sample, our findings indicate that ChatGPT3-generated news received, on average, higher perceived quality ratings across all news categories: celebrity, crime, accident and culture (Table 2). As shown in Table 2, the news generated by ChatGPT that obtained the highest score were about culture (M =4.92, SD =1.01). This may be related to the fact that the piece about culture written by journalists received one of the lowest ratings (M =4.21, SD =1.06). However, it was crime news that received the lowest score. In fact, we can see that ChatGPT news received significantly higher values in all categories. Analyzing the results by country (Table 3), we was found that, in Portugal, the news that obtained the highest score were also created by ChatGPT and belong to the “accident”category (M =4.76, SD =1.09). The news written by journalists that Fig. 2 Example of news templates used in surveys. Source: own elaboration. HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-025-04872-2 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 5 received the best rating are consistently below the ChatGPT news ratings. In Spain, the average ChatGPT news ratings are higher than the ChatGPT news ratings in Portugal. The ChatGPT news story about crime scored the highest (M =4.95, SD=0.99). Interestingly, crime news was the one with the lowest rating among news written by journalists (M =4.10, SD =1.18). Considering news quality indicators, we found that, overall, journalists performed poorly in presenting sources or references and in their use of punctuation in the news they wrote (Table 4). These were the quality indicators that obtained the lowest scores, namely the use of a correct and adequate punctuation in the news (M =4.03, SD =1.11). In this indicator, ChatGPT news presented a significantly higher quality (M =4.73, SD =0.99) [t(444)=–10.899, p< 0.001, d =–0.517]. News written by journalists received its highest rating for use of clear and precise language (M =4.64, SD =1.16), but this was still significantly lower than ChatGPT news (M =4.83, SD =1.20), [t (444) =– 3.611, p< 0.001,d =–0.171]. In general, ChatGPT news has a higher perceived quality in all indicators analyzed. ChatGPT news received the highest score in terms of readability, presentation of facts, creation of titles, citation of sources, use of punctuation and understandable language (Table 4). Focusing on each country, we note that there are no significant differences that stand out when comparing the quality of news indicators and ChatGPT news. However, there are statistically significant differences in relation to readability when evaluating ChatGPT news [F(1,444) =13.717, p< 0.001], with students from Spain giving a higher rating. There are also statistically significant differences in relation to the perceived quality of the ChatGPT news language [F(1,444) = 26.294, p< 0.001], in which students from Spain recognize higher quality. Even so, comparing the responses from the two countries, most indicators do not show statistically significant differences, which reveals that there is no significant influence of the country of residence and that students reveal similar abilities. Association between variables. Through a multivariate analysis of variance (MANOVA) we noticed that variables such as sex and year of schooling do not have any significant effect on the assessment of news quality [Fsex(2,444) =0.743, p=0.476 x Fyear(6.444) = 1.900, p=0.079]andintheevaluationofChatGPTnews [Fsex(2,444) =0.813, p=0.444 x Fyear(6,444) =0.743, p=0.476]. To measure the effect of (1) trust in news, (2) interest in news and (3) frequency of access to news, we first recoded the variables into categories (1=little, 2=somewhat, 3=a lot). Several MANOVAS did not show statistically significant differences in relation to the effect of variables on quality assessment. Discussion and conclusions Our study investigated the perceived quality of news generated by ChatGPT-3 compared to news written by human journalists among university students in the Iberian Peninsula. The results revealed a significant preference for ChatGPT-generated news, with higher ratings for readability, informativeness, and textual structure across all news categories. Notably, Spanish students tended to rate ChatGPT news even more favorably than their Portuguese counterparts, which may reveal that this tool follows a writing structure for news that is more like the pattern that is taught in Spanish faculties. The findings also indicated that the quality indicators assessed, such as presentation of sources, use of punctuation, and overall language quality, consistently favored ChatGPT-generated news over human-written news. Interestingly, the perceived quality of news created by journalists was lower, particularly in terms of presenting sources and references, indicating potential concerns about journalistic practices. Thus, journalism students struggle to discern newswriting generated by AI from newswriting created by Table 2 Perceived quality of news by category. Celebrity (N=444) Accident (N=444) Crime (N=444) Culture (N=444) News. GPT News. GPT News. GPT News. GPT 4.39 ± 1.14 4.77 ± 1.00 4.47 ± 1.09 4.83 ± 1.07 4.10 ± 1.15 4.86 ± 1.01 4.21 ± 1.06 4.92 ± 1.01 p<.001 p <.001 p <.001 p <.001 t=-5.303 t =-5.189 t =-10.266 t =-10.986 Source: own elaboration. Table 3 Analyzing by country: perceived quality of news by category. Country Celebrity Accident Crime Culture News. GPT News. GPT News. GPT News. GPT Spain (N=345) 4.36 ± 1.18 4.85 ± 1.00 4.53 ± 1.10 4.85 ± 1.06 4.10 ± 1.18 4.95 ± .99 4.14 ± 1.11 5.02 ± 1.01 Portugal (N=99) 4.51 ± 1.02 4.50 ± .092 4.25 ± 1.02 4.76 ± 1.07 4.11 ± 1.05 4.55 ± 1.02 4.44 ± 0.85 4.57 ± 0.95 p=0.262 p =0.002 p =0.021 p =0.434 p =0.940 p <0.001 p =0.015 p < 0.001 F=1.252 F =9.749 F =5.360 F=0.613 F=0.006 F=12.068 F =5.984 F =15.638 Source: own elaboration. Table 4 Score assigned by quality indicators. Indicators News (N=444) News. GPT pt Readability 4.77 ± 1.02 5.13 ± 1.04 <0.001 –9.046 Factual 4.16 ± 1.09 4.87 ± .98 <0.001 –11.600 Headline 4.30 ± 1.04 4.95 ± 1.01 <0.001 –11.553 Coverage 4.27 ± 1.07 4.83 ± .97 <0.001 –9.848 Sources 4.08 ± 1.04 4.59 ± .95 <0.001 –8.780 Punctuation 4.03 ± 1.11 4.73 ± .99 <0.001 –10.899 Language 4.64 ± 1.16 4.83 ± 1.20 <0.001 –3.611 Source: own elaboration. ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-025-04872-2 6HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 human journalists (RQ1), challenging the conventional notion of human journalists as the gold standard in news creation. Our research is a first approach to the perceived quality of news generated by ChatGPT-3, a generative artificial intelligence language model, in the context of the Iberian Peninsula. Our findings showed, from a general perspective, that news created by ChatGPT-3 were rated as having better quality than news written by human journalists. In none of the quality dimensions assessed did university students rate journalist-created news as superior, highlighting implications for the role of humans in the context of AI use. This trend was consistently observed among students from both Spain and Portugal. These results seem to be in line with other studies that analyzed the capacity and quality of automated journalism and found that readers tend not to be able to distinguish between texts generated by AI or humans (Jia 2020; Clerwall 2014; Wölker and Powell 2021; Van der Kaa and Krhamer 2014). Still, our study clearly shows that students considered the articles created by ChatGPT to be better written, even though they were presented in different languages. Similar to our study, Clerwall (2014) also examined the evaluation of university media and communication students, who attributed greater credibility to the automated version in relation to the version written by journalists. However, unlike our study, in Clerwall (2014) students attributed greater readability to articles written by humans. Likewise, Graefe et al. (2018) and Haim and Graefe (2017) found that articles written by humans received a higher readability rating. Given these data, we can discuss two hypotheses. On the one hand, the textual quality of written news may have lost quality over the years. The pressure to publish, the accelerated digitalization of journalism, the attention economy, social networks and growing sensationalism may have had a negative effect on the textual quality of news. On the other hand, the computer’s ability to write news has improved, particularly at the level of natural human language (Caswell and Dorr 2018), which justifies the significantly higher score in terms of readability of ChatGPT-3 in relation to news written by humans. In fact, the ability and language model of ChatGPT-3 has been recognized for the quality of the texts it generates (Leiter et al. 2024), although the same does not apply to the writing of texts of a scientific nature (Gao et al. 2022). In some studies, automated journalism appears more credible for specific categories, such as Sports (Wölker and Powell 2021). In our study, news created by ChatGPT had higher quality scores across all topics: accident, celebrity, culture and crime. This means that the subject of the news influence the perceived quality (RQ2). Interestingly, the quality indicator for news created by humans that obtained the lowest average score was the indicator related to the presentation of sources and references in the text, and the use of correct and appropriate punctuation. This study highlights a new issue in the ongoing debate about AI’s future and its growing impact on journalism—specifically, the quality of human-written journalism at the level of key criteria. Students assigned lower scores to the presentation of sources and references, indicators that extend beyond the textual quality of the news. In addition, in our discussion we also consider the characteristics of the participants who evaluated the news. It is well known that young people today read little news and struggle with attention deficits. Moreover, they generally show low interest in news, even among those studying journalism. Research also indicates that young people often have difficulty distinguishing real news from fake news or from advertisements. Our study prompts considerations regarding the potential impact of AI on journalism, raising questions about the current state of human-written news quality. The higher ratings for ChatGPT-3, even across different languages, suggest a possible decline in the textual quality of human-written news over the years. However, it is important to note that the study focused on university students, who may have specific reading habits and preferences, including a potential human bias towards AIgenerated content. Another limitation is that headlines of current journalism could not include all the basic questions due to editorial or economic constraints, but we appraise that the six classic journalism questions further our understanding on a good journalistic practice. Also, as mentioned, Correio da Manhã in Portugal and OKdiario in Spain are both sensationalist tabloid news media that operate with professional questionable standards. Our decision to evaluate the perceived quality level through these newspapers rested on the assumption that low-quality journalism may be easily replaced by AI-generated news. This points to the possibility of using AI for non-serious topics, but the selection of the sample is also a limitation as it might have shaped perceptions in a negative way. Since these constraints or editorial guidelines may result in the stories being less clear to reach a larger audience, a venue to future research is to approximate these guidelines and require ChatGPT to follow them to see if the differences remain. In the context of the ongoing debate about the future of AI in journalism, our findings highlight not only the capabilities of ChatGPT-3 but also potential concerns about the quality of human-written news that can be considered, especially in aspects beyond pure text quality. As the field of AI continues to advance, further research and discussions should explore the implications for journalism and how it may shape the evolving landscape of news consumption. Data availability Data from the surveys used to carry out this research are available in the Supplementary Data. Received: 12 September 2024; Accepted: 3 April 2025; References Anderson CW (2013) Towards a sociology of computational and algorithmic journalism. N. Media Soc 15(7):1005–1021. https://doi.org/10.1177/ 1461444812465137 Anderson CW (2018) Apostles of Certainty: Data Journalism and the Politics of Doubt. New York: Oxford University Press Aramburú Moncada LG, López Redondo I, López Hidalgo A (2023) Inteligencia artificial en RTVE al servicio de la España vacía. Proyecto de cobertura informativa con redacción automatizada para las elecciones municipales de 2023. Rev Lat de Comunicación Soc 81:1–16. https://doi.org/10.4185/RLCS2023-1550 Arapakis I, Peleja F, Berkant B, Magalhaes J (2016) Linguistic benchmarks of online news article quality. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Long Papers) (pp. 1893–1902) Baptista JP, Gradim A (2024) Are journalists no longer needed? Comparative analysis of the perceived quality of real news and ChatGPT news. In: Proceedings TEEM 2023: Technological Ecosystems for Enhancing Multiculturality Baptista JP, Gradim A, Correia E (2022) A relação entre a crença nas fake news e as estratégias para procurar informação dos jovens portugueses. Observatório (OBS*), 16(3). https://doi.org/10.15847/obsOBS16320222082 Baptista JP, Rivas-de-Roca R, Gradim A, Loureiro M (2023) The Disinformation Reaction to the Russia–Ukraine War: An Analysis through the Lens of Iberian FactChecking. KOME 11(2):27–48. https://doi.org/10.17646/ KOME.of.2 Calvo-Rubio L, Ufarte-Ruiz M (2020) Percepción de docentes universitarios, estudiantes, responsables de innovación y periodistas sobre el uso de inteligencia artificial en periodismo. Profesional de la Inf ón 29(1):e290109. https:// doi.org/10.3145/epi.2020.ene.09 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-025-04872-2 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 7 Canavilhas J (2022) Artificial Intelligence and Journalism: Current Situation and Expectations in the Portuguese Sports Media. Journalism Media 3(3):510–520. https://doi.org/10.3390/journalmedia3030035 Canavilhas J, Di Fátima B (2025) Reporteros del futuro: consumo de noticias entre estudiantes de periodismo en Portugal. Doxa Comunicación Rev Interdisciplinar De Estudios De Comunicación Y Cienc Soc 40:305–319. https:// doi.org/10.31921/doxacom.n40a2392 Canavilhas J, Giacomelli F (2023) Inteligencia artificial en el periodismo deportivo: estudio en Brasil y Portugal. Rev de Comunicación 22(1):53–69. https://doi. org/10.26441/RC22.1-2023-3005 Caswell D, Dörr K (2018) Automated Journalism 2.0: Event-driven narratives: From simple descriptions to real stories. Journalism Pr 12(4):477–496. https://doi.org/10.1080/17512786.2017.1320773 Clerwall C (2014) Enter the robot journalist. Journalism Pr 8(5):519531. https://doi. org/10.1080/17512786.2014.883116 Coddington M (2015) Clarifying journalism’s quantitative turn: A typology for evaluating data journalism, computational journalism, and computer-assisted reporting. Digital Journalism 3(3):331–348. https://doi.org/10.1080/ 21670811.2014.976400 Cools H, Van Gorp B, Opgenhaffen M (2024) Where exactly between utopia and dystopia? A framing analysis of AI and automation in US newspapers. Journalism 25(1):3–21. https://doi.org/10.1177/146488492211226 de-Lima-Santos MF, Ceron W (2021) Artificial Intelligence in News Media: Current Perceptions and Future Outlook. Journalism Media 3(1):13–26. https:// doi.org/10.3390/journalmedia3010002 Deuze M, Beckett C (2022) Imagination, algorithms and news: developing AI literacy for journalism. Digital Journalism 10(10):1913–1918. https://doi.org/ 10.1080/21670811.2022.2119152 Diakopoulos N (2020) Computational news discovery: towards design considerations for editorial orientation algorithms in journalism. Digital Journalism 8(7):945–967. https://doi.org/10.1080/21670811.2020.1736946 Foà C, Tomé V, Margato D, Paisana M, Crespo M, Cardoso G (2023) Roles of journalists in media literacy initiatives: trainees and trainers. Continuity, collaboration, and sustainability of media literacy trainings to mitigate disinformation in Portugal. Profesional de la Inf ón 32(6):e320621. https://doi. org/10.3145/epi.2023.nov.21 Gao CA, Howard FM, Markov NS, Dyer EC, Ramesh S, Luo Y, Pearson AT (2022) Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. BioRxiv. https://doi.org/10.1101/2022.12.23. 521610 García Orosa B, Gallur Santorum S, López García X (2017) El uso del clickbait en cibermedios de los 28 países de la Unión Europea. Rev Lat de Comunicación Soc 72:1261–1277. https://doi.org/10.4185/RLCS-2017-1218 Gómez-Diago G (2022) Perspectivas para abordar la inteligencia artificial en la enseñanza de Periodismo. Una revisión de experiencias investigadoras y docentes. Rev Lat de Comunicación Soc 80:29–46. https://doi.org/10.4185/ RLCS-2022-1542 Gonçalves A, Melo PV (2022) Inteligencia artificial y periodismo: una aproximación al contexto portugués. Fonseca-J Commun 25:23–34. https://doi.org/10. 14201/fjc.29682 González-Arias C, López-García X (2023) ChatGPT: Stream of opinion in five newspapers in the first 100 days since its launch. Profesional de la Inf ón 32(5):e320524. https://doi.org/10.3145/epi.2023.sep.24 Graefe A, Haim M, Haarmann B, Brosius HB (2018) Readers’perception of computer-generated news: Credibility, expertise, and readability. Journalism 19(5):595–610. https://doi.org/10.1177/1464884916641269 Gutiérrez López M, Porlezza C, Cooper G, Makri S, MacFarlane A, Missaoui S (2023) A Question of Design: Strategies for Embedding AI-Driven Tools into Journalistic Work Routines. Digital Journalism 11(3):484–503. https://doi. org/10.1080/21670811.2022.2043759 Gutiérrez-Caneda B, Vázquez-Herrero J, López-García X (2023) AI application in journalism: ChatGPT and the uses and risks of an emergent technology. Profesional de la Inf ón 32(5):e320514. https://doi.org/10.3145/epi. 2023.sep.14 Haim M, Graefe A (2017) Automated news: Better than expected? Digital Journalism 5(8):1044–1059. https://doi.org/10.1080/21670811.2017.1345643 Hallin DC, Mancini P (2004) Comparing Media Systems: Three Models of Media and Politics. New York: Cambridge University Press Jia C (2020) Chinese automated journalism: A comparison between expectations and perceived quality. Int J Commun 14:22 van der Kaa HA, Krahmer EJ (2014) Journalist versus news consumer: The perceived credibility of machine written news. In Computation+Journalism Symposium 2014 Karinshak E, Jin Y (2023) AI-driven disinformation: a framework for organizational preparation and response. J Commun Manag 27(4):539–562. https:// doi.org/10.1108/JCOM-09-2022-0113 Leiter C, Zhang R, Chen Y, Belouadi J, Larionov D, Fresen V, Eger S (2024) Chatgpt: A meta-analysis after 2.5 months. Mach Learn Appl 16:100541. https://doi.org/10.1016/j.mlwa.2024.100541 Lemelshtrich Latar N (2018) Robot journalism: Can human journalism survive? New York: World Scientific Publishing Lopezosa C, Codina L, Pont-Sorribes C, Vállez M (2023) Use of generative artificial intelligence in the training of journalists: challenges, uses and training proposal. Profesional de la Inf ón 32:e320408. https://doi.org/10. 3145/epi.2023.jul.08 Ministerio de Ciencia, Innovación y Universidades (2024) Datos y cifras del Sistema Universitario Español, publicación 2023-2024. Madrid: Administración General del Estado. https://www.universidades.gob.es/wp-content/uploads/2024/ 05/SIIU_DatosCifras2024.pdf Moran RE, Shaikh SJ (2022) Robots in the News and Newsrooms: Unpacking Meta-Journalistic Discourse on the Use of Artificial Intelligence in Journalism. Digital Journalism 10(10):1756–1774. https://doi.org/10.1080/21670811. 2022.2085129 Munoriyarwa A, Chiumbu S, Motsaathebe G (2023) Artificial Intelligence Practices in Everyday News Production: The Case of South Africa’s Mainstream Newsrooms. Journalism Pr 17(7):1374–1392. https://doi.org/10.1080/ 17512786.2021.1984976 Noain-Sánchez A (2022) Addressing the Impact of Artificial Intelligence on Journalism: the perception of experts, journalists and academics. Commun Soc 35(3):105–121. https://doi.org/10.15581/003.35.3.105-121 Pavlik JV (2023) Collaborating With ChatGPT: Considering the Implications of Generative Artificial Intelligence for Journalism and Media Education. Journalism Mass Commun Educator 78(1):84–93. https://doi.org/10.1177/ 10776958221149577 Pérez-Seijo S, Barbosa S, Vicente PN (2023) Artificial Intelligence in Journalism: Case Study of the Spanish, Portuguese and Brazilian News Media Systems. In: Negreira-Rey MC, Vázquez-Herrero J, Sixto-García J, LópezGarcía X (eds). Blurring Boundaries of Journalism in Digital Media. Studies in Big Data,140.Cham:Springer.https://doi.org/10.1007/978-3031-43926-1_18 Quian A, Sixto-García J (2024) Inteligencia artificial en la prensa: estudio comparativo y exploración de noticias con ChatGPT en un medio tradicional y otro nativo digital. Rev de Comunicación 23(1):457–483. https://doi.org/10. 26441/RC23.1-2024-3374 Rivas-de-Roca R (2022) Calidad y modelos de negocio en los medios de proximidad: Estudio de casos en Alemania, España y Portugal. Estudos em Comun ção 34:8196 Ruffo G, Semeraro A (2022) FakeNewsLab: Experimental study on biases and pitfalls preventing us from distinguishing true from false news. Future internet 14(10):283 Sixto-García J, Quian A, Rodríguez-Vázquez A-I, Silva-Rodríguez A, Soengas-Pérez X (2024) Journalism, Digital Media and the Fourth Industrial Revolution. Cham: Palgrave Macmillan Tomé V, Crespo M, Branco S, Nery I (2023) Portuguese journalists training teachers in the new media education landscape during Covid-19. In: Friesem Y, Raman U, Kanizaj I, Choi GY (eds.). The Routledge handbook of media education futures post-pandemic. London: Routledge Túñez-López J-M, Toural-Bran C, Cacheiro-Requeijo S (2018) Uso de bots y algoritmos para automatizar la redacción de noticias: percepción y actitudes de los periodistas en España. Profesional de la Inf ón 27(4):750–758. https:// doi.org/10.3145/epi.2018.jul.04 Ufarte MJ, Calvo LM, Murcia FJ (2020a) Las tecnologías hi-tech en los grados en Periodismo. Planes de estudios, formación de los periodistas y propuestas de inserción curricular. adComunica 20:43–66. https://doi.org/10.6035/21740992.2020.20.3 Ufarte MJ, Fieiras C, Túñez M (2020b) La enseñanza-aprendizaje del periodismo automatizado en instituciones públicas: estudios, propuestas de viabilidad y perspectivas de impacto de la IA. àlisi: Quad de Comunicaciói Cult, 62:131–146. https://doi.org/10.5565/rev/analisi.3289 Westlund O, Lewis SL (2017) Reconsidering News Production: How Understanding the Interplay of Actors, Actants, and Audiences Can Improve Journalism Education (409-428). In: Goodman, R. S., & Steyn, E. (eds.). Global Journalism Education in the 21st Century: Challenges and Innovations World Journalism Education Council, pp. 409-428. Austin: Knight Center for Journalism in the Americas, University of Texas at Austin Whyte C (2020) Deepfake news: AI-enabled disinformation as a multi-level public policy challenge. J Cyber Policy 5(2):199–217. https://doi.org/10.1080/ 23738871.2020.1797135 Wölker A, Powell TE (2021) Algorithms in the newsroom? News readers’perceived credibility and selection of automated journalism. Journalism 22(1):86–103. https://doi.org/10.1177/1464884918757072 Zamanian M, Heydari P (2012) Readability of Texts: State of the Art. Theory & Practice in Language Studies, 2(1) ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-025-04872-2 8HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 Author contributions All authors were equally involved in the application of the research methodology. The corresponding author, João Pedro Baptista, and the author Rúben Rivas-de-Roca carefully wrote the article, under the supervision of the authors Anabela Gradim and Concha Pérez-Curiel. Competing interests The authors declare no competing interests. Ethical statements This study adhered to the ethical guidelines and research policies of the University of Beira Interior (UBI), which align with the highest international standards for academic integrity and responsible research. At UBI, these ethical principles are not merely recommendations but mandatory requirements for all research involving human participants. The study was conducted using the principles of the Declaration of Helsinki, which ensures the protection of participants’rights, dignity, and confidentiality. Among the regulations followed, we strictly adhered to the Academic Integrity Code of UBI, issued in 2018 (https://www.ubi.pt/Ficheiros/Entidades/91363/codigo_integridade.pdf), and the Regulation for Performing Research Surveys, issued in 2022 (https://www.ubi.pt/ Ficheiros/Entidades/91051/Despacho%202022-RT-18.pdf). Our research was carried out in a classroom, where participation is voluntary, non-invasive and does not represent a risk to participants. Therefore, our study did not require formal ethical approval according to these institutional guidelines. Informed consent All participants were fully informed about the objectives, methodology and confidentiality measures of the study before their voluntary participation. The informed consent process took place on 08-05-2023, before data collection, and was documented in the survey itself, which was sent as supporting material. Additional information Supplementary information The online version contains supplementary material available at https://doi.org/10.1057/s41599-025-04872-2. Correspondence and requests for materials should be addressed to João Pedro Baptista. Reprints and permission information is available at http://www.nature.com/reprints Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/ licenses/by-nc-nd/4.0/. © The Author(s) 2025 HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | https://doi.org/10.1057/s41599-025-04872-2 ARTICLE HUMANITIES AND SOCIAL SCIENCES COMMUNICATIONS | (2025) 12:567 | https://doi.org/10.1057/s41599-025-04872-2 9