How ChatGPT performs in Oral Medicine: The case of oral potentially malignant disorders

Oral Diseases. 2023;00:1–7. | 1wileyonlinelibrary.com/journal/odi Received: 17 April 2023 | Revised: 15 August 2023 | Accepted: 13 September 2023 DOI: 10.1111/odi.14750 SHORT COMMUNICATION How ChatGPT performs in Oral Medicine: The case of oral potentially malignant disorders M. Diniz- Freitas | B. Rivas- Mundiña | J. R. García- Iglesias | E. García- Mato | P. Diz- Dios Medical- Surgical Dentistry Research Group (OMEQUI), Health Research Institute of Santiago de Compostela (IDIS), University of Santiago de Compostela (USC), A Coruña, Spain Correspondence B. Rivas- Mundiña, Department of Surgery and Medical- Surgical Specialties, Faculty of Medicine and Odontology, University of Santiago de Compostela, c/Entrerríos s/n, 15782- Santiago de Compostela, Spain. Email: ber[email protected] Keywords: artificial intelligence, ChatGPT, oral leukoplakia, oral potentially malignant disorders The potential of artificial intelligence (AI) in the setting of Dentistry and, in particular, Oral Medicine is gaining importance (Patil et al., 2022). The recently launched ChatGPT, a freeaccess AI tool developed by the OpenAI company, consists of a model trained with large quantities of data, capable of understanding and generating human language with high precision and consistency (OpenAI, https://openai.com/gpt/). The advantage of ChatGPT in the medical setting in terms of navigating web browsers is that it quickly provides appropriate responses to the proposed questions and facilitates decision making based on recent research and guidelines, summarizing the available information and obviating the need for the interviewer to review the original sources (Ahn, 2023). Since its release, ChatGPT has generated considerable expectations regarding its potential utility in the health sciences setting, particularly in terms of education, research and practice for the various medical disciplines (Sallam, 2023). To our knowledge, however, no studies have been published to date on the use of ChatGPT in the setting of Oral Medicine. A particularly concerning topic is the identification of oral potentially malignant disorders (OPMDs). The importance of developing AI tools that facilitate the clinical detection of OPMDs has therefore been indicated (de Souza et al., 2023). The aim of this study is to assess the performance of ChatGPT in responding to questions on the diagnosis and management of OPMDs. We started by selecting guidelines and consensus documents disseminated by reputable scientific societies on the definition, classification, diagnosis, evaluation, malignant transformation and management of OPMDs (Birur et al., 2022; Chen et al., 2021; Lingen et al., 2017; Stojanov & Woo, 2018; Warnakulasuriya et al., 2021). Based on these statements, we formulated a series of “primary questions” that were entered into ChatGPT (version 23 of March 2023) using the function “New chat”. We started by selecting guidelines and consensus documents disseminated by reputable scientific societies on the definition, classification, diagnosis, evaluation, malignant transformation and management of OPMDs. From each guideline, we chose those statements that had an overwhelming positioning (the panel does/does not recommend). Additionally, the GRADE grid (Jaeschke et al, 2008) was specified when this was available. The review and qualification of the responses were performed independently by two reviewers, and the discrepancies were resolved by a third reviewer. The accuracy of each response was qualified with the following score: 1. Complete; 2. Correct but insufficient; 3. Includes correct and incorrect/outdated data; and 4. Completely incorrect (Yeo et al., 2023). When the responses to the primary questions were qualified as correct but insufficient, “secondary questions” were created to determine whether ChatGPT could recover the lost information. The secondary questions were created within the same conversation thread as the corresponding primary question. Lastly, we analyzed the authenticity of the references included in the responses. The formulated questions and the responses obtained using ChatGPT, as well as their verification with the guidelines and selected consensus documents, are detailed in Tables 1– 3. In general, ChatGPT showed a moderate base of knowledge on OPMDs, although this result could have been overestimated by the inaccuracy of the guidelines when deciding on certain aspects of these lesions that are still controversial (Stojanov & Woo, 2018). This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made. © 2023 The Authors. Oral Diseases published by Wiley Periodicals LLC. 2 | D I N I Z - F R E I T A S et al. In total, 7 of the 12 (58%) responses to the primary questions and 7 of the 12 (58%) responses to the secondary questions included correct and incorrect/outdated data or were completely incorrect (scores 3 and 4). Regarding the primary questions on the classification of OPMDs, ChatGPT's responses did not include proliferative verrucous leukoplakia (PVL), oral lichenoid lesion (OLL) or oral graftversus- host disease (OGVHD), unlike the guidelines (Warnakulasuriya et al., 2021). TABLE 1 The result of a conversation about definition and classification of oral potentially malignant disorders through ChatGPT (available on https://chat.openai.com/chat) on April 4, 2023, compared to information obtained from clinical guidelines. ChatGPT WHO Collaborating Centre for Oral Cancer (Warnakulasuriya et al., 2021) 1. What oral conditions are considered oral potentially malignant disorders (OPMD)? Leukoplakia Erythroplakia Oral submucous fibrosis Lichen planus Actinic cheilitis (Score 2) Oral leukoplakia Oral eritroplakia Proliferative verrucous leukoplakia (PVL) Oral lichen planus Oral submucous fibrosis Actinic keratosis (Actinic cheilitis) Oral lichenoid lesion (OLL) Oral graftversus- host disease (OGVHD) 1.a. Is proliferative verrucous leukoplakia (PVL) an oral potentially malignant disorder (OPMD)? Yes, PVL is considered an OPMD. In fact, studies have shown that up to 70% of PVL cases will eventually progress to squamous cell carcinoma, a type of oral cancer (Score 1) PVL is associated with the highest proportion of oral cavity cancer development compared with other OPMDs (Cabay et al., 2007; Iocca et al., 2020). A recent systematic review estimated the proportion to be 49.5% (CI 26.7%– 72.4%) (Iocca et al., 2020) 1.b. Are oral lichenoid lesions (OLLs) an oral potentially malignant disorder (OPMD)? OLLs are not necessarily malignant, but they are considered an OPMD. This means that while most cases of OLLs are benign and not cancerous, there is a small chance that they may develop into oral cancer over time (Score 1) The evidence from their systematic review suggests that patients with OLLs, have more or less similar malignant potential to oral lichen planus (Gonzalez- Moles et al., 2019) 1.c. Is oral graftversus- host disease (OGVHD) an oral potentially malignant disorder (OPMD)? OGVHD is not generally considered a OPMD. While OGVHD can cause changes in the oral tissues that may resemble precancerous lesions, such as white or red patches or ulcerations, these changes are typically a result of chronic inflammation and damage to the tissues rather than a true precancerous condition (Score 3) Since our previous Workshop Report on OPMDs, progression to cancer in OGVHD- related oral lichenoid lesions has subsequently been reported in several case studies (Demarosi et al., 2005; Frydrych et al., 2019; Hashimoto et al., 2019; Mawardi et al., 2011). Multisystem GVHD was a significant risk factor for the development of all solid tumors, significantly higher for oral cancer among adult recipients of an allogeneic stem cell transplantation (Atsuta et al., 2014). 1.d. But is oral graftversus- host disease (OGVHD) considered an oral potentially malignant disorder (OPMD) or not? In summary, while there is some debate on the matter, OGVHD may be considered an OPMD due to the increased risk of cancer in people who have had a bone marrow or stem cell transplant, but this is not a universally accepted view (Score 3) 2. Is chronic hyperplastic candidosis (CHC) an oral potentially malignant disorder? Yes, CHC is considered an OPMD. These lesions have a high potential for malignant transformation and are associated with an increased risk of developing oral squamous cell carcinoma (Score 4) Disorder with insufficient epidemiological evidence. At present, this is not recommended for inclusion within the OPMD group of disorders. 2.a. Are you sure chronic hyperplastic candidosis (CHC) is an oral potentially malignant disorder (OPMD)? I apologize for the confusion in my previous response. CHC, also known as candidal leukoplakia, is a controversial condition and its classification as an OPMD is still a matter of debate (Score 2) 3. Is exophytic verrucous hyperplasia (EVH)/oral verrucous hyperplasia (OVH) an oral potentially malignant disorder (OPMD)? EVH/OVH are considered OPMDs. Although EVH/OVH have a low risk of malignant transformation compared to other OPMDs, they still have the potential to progress to verrucous carcinoma or squamous cell carcinoma (Score 4) Disorders with insufficient epidemiological evidence. At present, these are not recommended for inclusion within the OPMD group of disorders. Note: Bold: primary questions; Italic: secondary questions (explanation in the text); Score: Accuracy of the responses (1. Complete; 2. Correct but insufficient; 3. Includes correct and incorrect/outdated data; and 4. Completely incorrect). 16010825, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/odi.14750 by Universidade de Santiago de Compostela, Wiley Online Library on [12/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License | 3 D I N I Z - F R E I T A S et al. In contrast, ChatGPT considered that chronic hyperplastic candidosis and exophytic verrucous hyperplasia/oral verrucous hyperplasia were OPMDs. This result could be due to a significant limitation in ChatGPT: the pace of updating, given that the data employed for its training is dated from before 2022 (OpenAI). In response to the corresponding secondary questions, ChatGPT considered PVL and OLL as OPMDs; when we insisted on OGVHD, the chat admitted that “… while there is some debate on the matter, OGVHD may be considered an OPMD.” This result shows that ChatGPT uses natural language processing techniques, which allow it to understand the meaning and intent of words and phrases used in a conversation, as well as the context of the discussion, generating responses that not only are relevant for the subject at hand but also follow the flow of the conversation (OpenAI). TABLE 2 The result of a conversation about the diagnosis and evaluation of oral potentially malignant disorders through ChatGPT (available on https://chat.openai.com/chat) on April 4, 2023, compared to information obtained from clinical guidelines. ChatTGPT American Dental Association (Lingen et al., 2017) 1. Among adults with clinically evident, nonsuspicious lesions, or other symptoms, should we recommend the use of adjuncts to identify oral potentially malignant disorders (OPMDs) or oral malignant disorders? Based on the available evidence, the use of adjuncts to identify OPMDs or oral malignant disorders should be considered in adults with clinically evident, nonsuspicious lesions, or other symptoms. Ultimately, the decision to use adjuncts should be made on an individual basis, taking into consideration the patient's overall health status, risk factors, and clinical presentation, … (Score 3) Clinicians should followup periodically with the patient. If the lesion has not resolved and the clinical diagnosis of an OPMD cannot be ruled out, then clinicians should perform a biopsy of the lesion or refer the patient a specialist Evidence level: low Strength of recommendation: conditional 2. Among adults with clinically evident, suspicious lesions, or other symptoms, should we recommend the use of adjuncts to identify oral potentially malignant disorders (OPMDs) or oral malignant disorders? Yes, adjuncts should be considered and recommended for use in adults with clinically evident, suspicious lesions, or other symptoms that raise suspicion for OPMDs or oral malignant disorders (Score 3) The panel suggests that for adult patients with a clinically evident oral mucosal lesion considered to be suspicious of an OPMD or malignant disorder, or other symptoms, clinicians should perform a biopsy of the lesion or provide immediate referral to a specialist Evidence level: low Strength of recommendation: conditional 3. Among adults with clinically evident, seemingly innocuous, or suspicious lesions, or other symptoms, should we recommend the use of cytologic adjuncts for the evaluation of oral potentially malignant disorders (OPMDs)? The decision to use adjuncts should be based on individual patient's risk factors, clinical presentation, and the availability of these adjuncts in the clinical setting. Nonetheless, adjuncts can be valuable tools in the evaluation and management of clinically evident, suspicious lesions, or other concerning symptoms in the oral cavity (Score 3) The panel does not recommend cytologic adjuncts for the evaluation of OPMDs among adult patients with clinically evident, seemingly innocuous, or suspicious lesions Evidence level: low Strength of recommendation: conditional The panel does not recommend autofluorescence, tissue reflectance, or vital staining adjuncts for the evaluation of OPMDs among adult patients with clinically evident, seemingly innocuous, or suspicious lesions Evidence level: low to very low Strength of recommendation: conditional 3.a. Is it necessary to perform a biopsy to reach a definitive diagnosis of oral leukoplakia (OL)? Yes, a biopsy is typically necessary to reach a definitive diagnosis of OL…a definitive diagnosis of OL requires a histopathological examination of a tissue sample obtained through a biopsy (Score 1) Clinicians should followup periodically with the patient. If the lesion has not resolved and the clinical diagnosis of an OPMD cannot be ruled out, then clinicians should perform a biopsy of the lesion or refer the patient a specialist The panel suggests that for adult patients with a clinically evident oral mucosal lesion considered to be suspicious of an OPMD or malignant disorder, or other symptoms, clinicians should perform a biopsy of the lesion or provide immediate referral to a specialist aEvidence level: low Strength of recommendations: conditional 3.b. Are cytologic adjuncts a useful method in the diagnosis of OL? Cytologic adjuncts, such as exfoliative cytology and brush biopsy, can be useful methods in the diagnosis of OL, but they should be used in combination with other diagnostic methods to ensure an accurate diagnosis and appropriate management (Score 3) The panel does not recommend cytologic adjuncts for the evaluation of OPMDs among adult patients with clinically evident, seemingly innocuous, or suspicious lesions Evidence level: low Strength of recommendation: conditional Note: Bold: primary questions; Italic: secondary questions (explanation in the text); Score: Accuracy of the responses (1. Complete; 2. Correct but insufficient; 3. Includes correct and incorrect/outdated data; and 4. Completely incorrect). 16010825, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/odi.14750 by Universidade de Santiago de Compostela, Wiley Online Library on [12/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 4 | D I N I Z - F R E I T A S et al. TABLE 3 The result of a conversation about malignant transformation and management of oral potentially malignant disorders through ChatGPT (available on https://chat.openai.com/chat) on April 4, 2023, compared to information obtained from clinical guidelines. ChatGPT Society of Oral Medicine and Chinese Stomatological Association (Chen et al., 2021)a Indian Oral Cancer task force and Indian Dental Association (Birur et al., 2022) American Academy of Oral Medicine (Stojanov & Woo, 2018) 1. Should areca nut cessation be recommended for all patients with OL? Given the strong association between areca nut use and the development of OL, it is generally recommended that patients with OL should cease areca nut use (Score 1) Evidence level: extremely low Strength of recommendation: strongly recommended Areca nut cessation is an essential treatment Risk reduction strategies, such as tobacco, alcohol and areca nut habit cessation, should be advocated and implemented whenever possible 2. What are the factors identified as high risk for malignant transformation of OL? Larger size OL located on the floor of the mouth, tongue, and soft palate The longer the duration of OL, the higher the risk Dysplasia and carcinoma in situ Tobacco smoking Heavy alcohol consumption, particularly when combined with smoking Certain types of human papillomavirus (HPV) Patients with immunodeficiency, such as those with HIV/AIDS or who have undergone an organ transplant (Score 3) Lesion size >200 mm2 Non homogeneous appearance Lesions on the tongue and the floor of the mouth (commissure and gingiva– buccal region in India) Age > 40 years. Female gender Family history of cancer Presence of invasive Candida albicans Long duration of OL OL in nonsmokers Increased keratosis Verrucous, nodular, or erythematous components Induration or ulceration 2.a. What lesion size is considered a risk factor for malignant transformation of OL? The size of OL that is considered high risk for malignant transformation can vary depending on the location and other risk factors. Generally, lesions that are larger than 1 centimeter (about 0.4 inches) in diameter are more likely to be associated with an increased risk of malignant transformation (Score 3) 2.b. Is nonhomogeneous clinical appearance a risk factor for malignant transformation of OL? Yes, a nonhomogeneous clinical appearance of OL is considered a risk factor for malignant transformation. Lesions with a nodular or verrucous surface texture, or irregular margins, have a higher likelihood of containing areas of dysplasia or carcinoma in situ (Score 1) 2.c. Is gender a relevant factor for the malignant transformation of OL? Yes, gender is a relevant factor in the malignant transformation of OL. Studies have shown that men have a higher incidence of oral cancer, including malignant transformation of OL, compared to women (Score 4) 2.d. Is the presence of Candida albicans a risk factor for malignant transformation of OL? It is important to note that while Candida albicans infection is not a direct risk factor for malignant transformation of OL, it is still important to treat any underlying infection or inflammation, as this can help to prevent the development of oral cancer (Score 3) 16010825, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/odi.14750 by Universidade de Santiago de Compostela, Wiley Online Library on [12/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License | 5 D I N I Z - F R E I T A S et al. ChatGPT Society of Oral Medicine and Chinese Stomatological Association (Chen et al., 2021)a Indian Oral Cancer task force and Indian Dental Association (Birur et al., 2022) American Academy of Oral Medicine (Stojanov & Woo, 2018) 2.e. Is tobacco a risk factor for malignant transformation of OL? Yes, tobacco use is a wellestablished risk factor for the malignant transformation of OL (Score 4) 3. Should palpation be recommended in all patients for monitoring the malignant transformation of OL? Palpation may be particularly useful in patients with highrisk OL or those with lesions in certain areas of the mouth, where changes may be more difficult to detect visually. However, in patients with lowrisk OL or small, stable lesions in lowrisk areas of the mouth, visual inspection alone may be sufficient for monitoring the condition (Score 4) Evidence level: low. Strength of recommendation: strongly recommended 4. Should surgery be recommended in all patients with Ol? For small lesions with no dysplasia or cancer, a conservative approach such as close monitoring and behavior modification may be recommended. In some cases, topical therapies such as retinoids, bleomycin, or photodynamic therapy may also be considered For larger or more severe lesions, surgical therapy may be recommended, such as excision, laser surgery, or cryotherapy In cases where dysplasia or cancer is present, surgery is generally recommended (Score 1) Highrisk OL: surgery is the essential treatment. OL with moderate/severe epithelial dysplasia or carcinoma in situ: surgical excision with or without grafting; for larger homogeneous lesions (>4 cm) wait and watch through rigid surveillance OL with moderate/severe epithelial dysplasia: complete removal should be offered. OL with mild epithelial dysplasia: remove or follow depending on clinical appearance and location Treatment options include surgical excision, laser excision, cryotherapy and photodynamic therapy 5. In a patient with an OL without specific risk factors for malignant transformation what is the recommended followup interval? The frequency of followup appointments may range from every 3 to 6 months to once a year, depending on the size, location and severity of the lesion (Score 2) Followup every 3 months Evidence level: extremely low Strength of recommendation: strongly recommended To reexamine the oral potentially malignant disorders every 3 months for the first year…, then followup every 6 months Optimal followup periods are 3– 6 months 6. In a patient with an OL with highrisk factors for malignant transformation what is the recommended followup interval? Patients with highrisk OL may require more frequent followup appointments, such as every 3 to 6 months or even more frequently, to closely monitor for any changes in the lesion or the development of new symptoms (Score 1) Followup every 1– 3 months Evidence level: extremely low Strength of recommendation: strongly recommended To reexamine the oral potentially malignant disorders every 3 months for the first year…, then followup every 6 months Optimal followup periods are 3– 6 months Note: Bold: primary questions; Italic: secondary questions (explanation in the text); Score: Accuracy of the responses (1. Complete; 2. Correct but insufficient; 3. Includes correct and incorrect/outdated data; and 4. Completely incorrect). a Definition of quality of the evidence and strength of recommendations in Chen et al. (2021). TABLE 3 (Continued) 16010825, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/odi.14750 by Universidade de Santiago de Compostela, Wiley Online Library on [12/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License 6 | D I N I Z - F R E I T A S et al. Regarding the results of the diagnosis and evaluation of OPMDs, ChatGPT insisted on the use of adjuncts to identify OPMDs or oral malignant disorders in adults with clinically evident, seemingly innocuous, or suspicious lesions, or other symptoms. This proposal conflicts with the guidelines of the American Dental Association that indicate that “The panel does not recommend cytologic adjuncts, autofluorescence, tissue reflectance, or vital staining adjuncts for the evaluation of OPMDs…,” although the level of evidence of these recommendations is low and the strength of the recommendation is conditional (Lingen et al., 2017). In the results of a conversation on the malignant transformation of OPMDs, ChatGPT considered tobacco smoking, heavy alcohol consumption, human papillomavirus and immunodeficiency as factors identified as high risk for the malignant transformation of oral leukoplakia (OL). ChatGPT also did not consider invasive Candida albicans as a risk factor, made no reference to OL in nonsmokers and, paradoxically, attributed greater risk to men than to women, confirming significant differences with respect to the guidelines of the Indian Oral Cancer Task Force and the Indian Dental Association (Birur et al., 2022). In terms of the management of OPMDs, we found significant differences between ChatGPT and the proposals of the Society of Oral Medicine and Chinese Stomatological Association (Chen et al., 2021; e.g., regarding the importance of palpating lesions and the frequency of followup appointments). All of these discrepancies, in addition to emphasizing the need to update ChatGPT's contents, demonstrate geographical and cultural differences (e.g., “Areca nut cessation is an essential treatment” in India) and possibly a selection bias in the information sources that should be investigated more deeply. When asking about the references from which ChatGPT obtained the responses, we detected a number of inaccuracies in the citations regarding the authorship, the article titles and the names of the scientific journals, including some nonexistent journals. Although ChatGPT can be a useful tool in the health setting, it has numerous potential limitations such as ethical— legal problems, risk of bias, plagiarism, lack of originality, inaccurate contents, limited knowledge, incorrect citations, cybersecurity problems and the risk of infodemia (Sallam, 2023). ChatGPT's reported limitations include topics such as interpretability, reproducibility and the management of uncertainty, which could have harmful consequences in the healthcare setting (Sallam, 2023). In particular, it has been suggested that the lack of reproducibility in ChatGPT's responses could represent a highly significant limitation in the medical practice (Holzinger et al., 2023). In any case, and given that this is a recently introduced technology, a standardized methodology for assessing its reproducibility is, to our knowledge, still unavailable. This study was based on the methodology employed in previous studies (Yeo et al., 2023). One of ChatGPT's key characteristics is its ability to interact (i.e., maintain multiturn conversations). In this preliminary study, we considered the scenario of multiturn conversations (primary and secondary questions). It has been suggested that the ability to maintain multiturn conversations would allow it to, for example, provide numerous pieces of evidence, request a binary decision rather than an uncertain position and clarify aspects of the response that might be unclear (Zuccon & Koopman, 2023). As in other health settings, ChatGPT can be a valuable resource in the field of Oral Medicine, particularly in clinical decision making and the optimization of clinical work flows; however, one has to be cautiously enthusiastic because the intrinsic value of the knowledge and experience of healthcare practitioners in research and clinical practice are, for now, irreplaceable (Stokel- Walker & Van Noorden, 2023). The incorporation of ChatGPT into the field of Oral Medicine has the potential to significantly accelerate the decisionmaking processes for the diagnosis, treatment and care strategies for patients. In the health setting, however, it is vitally important to recognize the value of the accumulated knowledge and experience of clinicians. The expertise of health practitioners acquired through years of rigorous training, research and clinical practice cannot be underestimated. These professionals provide a deep understanding that transcends mere data and algorithms, encompassing nuanced judgment, contextual understanding and the innate ability to address complex and unpredictable situations. The current limitations of AI technology, including the likelihood of providing incorrect or misleading information, highlights the need for a collaborative approach. Accordingly, AI- inspired tools (such as ChatGPT), combined with the experience and knowledge of healthcare practitioners, have the potential to achieve more reliable and effective results for patients in the Oral Medicine setting. AUTHOR CONTRIBUTIONS M. Diniz- Freitas: Conceptualization; writing – original draft; investigation; methodology. B. Rivas- Mundiña: Conceptualization; writing – review and editing; methodology. J. R. García- Iglesias: Investigation; methodology. E. García- Mato: Writing – original draft. P. Diz- Dios: Conceptualization; writing – review and editing; methodology. ACKNO WLE DGE MENTS None. DATA AVAILABILITY STATEMENT The data that support the findings of this study are available from the corresponding author upon reasonable request. ORCID M. Diniz- Freitas https://orcid.org/0000-0001-5349-1091 B. Rivas- Mundiña https://orcid.org/0000-0003-4976-395X P. Diz- Dios https://orcid.org/0000-0002-1483-401X REFERENCES Ahn, C. (2023). Exploring ChatGPT for information of cardiopulmonary resuscitation. Resuscitation, 185, 109729. https://doi.org/10.1016/j. resus citat ion.2023.109729 Atsuta, Y., Suzuki, R., Yamashita, T., Fukuda, T., Miyamura, K., Taniguchi, S., Kato, K., Kawa, K., Nagamura- Inoue, T., Morishima, Y., Sakamaki, H., Kodera, Y., & Japan Society for Hematopoietic Cell Transplantation. (2014). Continuing increased risk of oral/esophageal cancer after 16010825, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/odi.14750 by Universidade de Santiago de Compostela, Wiley Online Library on [12/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License | 7 D I N I Z - F R E I T A S et al. allogeneic hematopoietic stem cell transplantation in adults in association with chronic graftversus- host disease. Annals of Oncology, 25(2), 435–441. https://doi.org/10.1093/annon c/mdt558 Birur, P. N., Patrick, S., Warnakulasuriya, S., Gurushanth, K., Raghavan, S. A., Rath, G. K., Chaturvedi, P., Chandru, V., Mathew, B., Prabhash, K., Gurudath, S., Mukhia, N., Sunny, S. P., Mehrotra, R., Vivek, V., Patil, S., Kumar, G. S., Fasalkar, S., Pratima, R., & Kuriakose, M. A. (2022). Consensus guidelines on management of oral potentially malignant disorders. Indian Journal of Cancer, 59(3), 442–453. https://doi.org/10.4103/ijc.IJC_128_21 Cabay, R. J., Morton, T. H. J., & Epstein, J. B. (2007). Proliferative verrucous leukoplakia and its progression to oral carcinoma: A review of the literature. Journal of Oral Pathology & Medicine, 36(5), 255–261. https://doi.org/10.1111/j.1600- 0714.2007.00506.x Chen, Q., Dan, H., Pan, W., Jiang, L., Zhou, Y., Luo, X., & Zeng, X. (2021). Management of oral leukoplakia: A position paper of the society of oral medicine, chinesestomatological association. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, 132(1), 32–43. https:// doi.org/10.1016/j.oooo.2021.03.009 de Souza, L. L., Fonseca, F. P., Araujo, A. L. D., Lopes, M. A., Vargas, P. A., Khurram, S. A., Kowalski, L. P., Santos, H. T. D., Warnakulasuriya, S., Dolezal, J., Pearson, A. T., & Santos- Silva, A. R. (2023). Machine learning for detection and classification of oral potentially malignant disorders: A conceptual review. Journal of Oral Pathology & Medicine, 52(3), 197–205. https://doi.org/10.1111/jop.13414 Demarosi, F., Soligo, D., Lodi, G., Moneghini, L., Sardella, A., & Carrassi, A. (2005). Squamous cell carcinoma of the oral cavity associated with graft versus host disease: Report of a case and review of the literature. Oral Surgery, Oral Medicine, Oral Pathology, Oral Radiology, and Endodontics, 100(1), 63–69. https://doi.org/10.1016/j.tripl eo.2004.12.008 Frydrych, A. M., Kujan, O., & Farah, C. S. (2019). Chronic disease comorbidity in patients with oral leukoplakia. Oral Cancer, 3(1), 17–26. h t t p s : / / d o i . o r g / 1 0 . 1 0 0 7 / s 4 1 5 4 8 - 0 1 8 - 0 0 1 5 - y Gonzalez- Moles, M. A., Ruiz- Avila, I., Gonzalez- Ruiz, L., Ayen, A., Gil- Montoya, J. A., & Ramos- Garcia, P. (2019). Malignant transformation risk of oral lichen planus: A systematic review and comprehensive metaanalysis. Oral Oncology, 96, 121–130. https://doi. org/10.1016/j.oralo ncolo gy.2019.07.012 Hashimoto, K., Nagao, T., Koie, S., Miyabe, S., & Saito, T. (2019). Secondary squamous cell carcinoma of the tongue is complicated with bronchiolitis obliterans as a manifestation of graftversus- host disease following peripheral blood stem cell transplantation. Case Reports in Hematology, 2019, 6015803. https://doi. org/10.1155/2019/6015803 Holzinger, A., Keiblinger, K., Holub, P., Zatloukal, K., & Müller, H. (2023). AI for life: Trends in artificial intelligence for biotechnology. New Biotechnology, 74, 16–24. https://doi.org/10.1016/j. nbt.2023.02.001 Iocca, O., Sollecito, T. P., Alawi, F., Weinstein, G. S., Newman, J. G., De Virgilio, A., Di Maio, P., Spriano, G., López, S. P., & Shanti, R. M. (2020). Potentially malignant disorders of the oral cavity and oral dysplasia: A systematic review and metaanalysis of malignant transformation rate by subtype. Head & Neck, 42(3), 539–555. https://doi.org/10.1002/hed.26006 Jaeschke, R., Guyatt, G. H., Dellinger, D. P., Schunemann, H., Levy, M. M., Kunz, R., Norris, S., Bion, J., & GRADE Working Group. (2008). Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. BMJ, 337, a744. https://doi. org/10.1136/bmj.a744 Lingen, M. W., Abt, E., Agrawal, N., Chaturvedi, A. K., Cohen, E., D'Souza, G., Gurenlian, J., Kalmar, J. R., Kerr, A. R., Lambert, P. M., Patton, L. L., Sollecito, T. P., Truelove, E., Tampi, M. P., Urquhart, O., Banfield, L., & Carrasco- Labra, A. (2017). Evidencebased clinical practice guideline for the evaluation of potentially malignant disorders in the oral cavity: A report of the American Dental Association. Journal of the American Dental Association, 148(10), 712–727.e10. https://doi. org/10.1016/j.adaj.2017.07.032 Mawardi, H., Elad, S., Correa, M. E., Stevenson, K., Woo, S., Almazrooa, S., Haddad, R., Antin, J. H., Soiffer, R., & Treister, N. (2011). Oral epithelial dysplasia and squamous cell carcinoma following allogeneic hematopoietic stem cell transplantation: Clinical presentation and treatment outcomes. Bone Marrow Transplantation, 46(6), 884–891. https://doi.org/10.1038/bmt.2011.77 Patil, S., Albogami, S., Hosmani, J., Mujoo, S., Kamil, M. A., Mansour, M. A., Abdul, H. N., Bhandi, S., & Ahmed, S. S. S. J. (2022). Artificial intelligence in the diagnosis of oral diseases: Applications and pitfalls. Diagnostics, 12(5), 1029. https://doi.org/10.3390/diagn ostic s1205 1029 Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Health, 11(6), 887. https://doi.org/10.3390/healt hcare 11060887 Stojanov, I. J., & Woo, S. (2018). AAOM clinical practice statement subject: Leukoplakia. Oral Surgery, Oral Medicine, Oral Pathology and Oral Radiology, 126(4), 331–334. https://doi.org/10.1016/j. oooo.2018.06.006 Stokel- Walker, C., & Van Noorden, R. (2023). What ChatGPT and generative AI mean for science. Nature, 614(7947), 214–216. https://doi. o r g / 1 0 . 1 0 3 8 / d 4 1 5 8 6 - 0 2 3 - 0 0 3 4 0 - 6 Warnakulasuriya, S., Kujan, O., Aguirre- Urizar, J. M., Bagan, J. V., Gonzalez- Moles, M. A., Kerr, A. R., Lodi, G., Mello, F. W., Monteiro, L., Ogden, G. R., Sloan, P., & Johnson, N. W. (2021). Oral potentially malignant disorders: A consensus report from an international seminar on nomenclature and classification, convened by the WHO collaborating Centre for oral cancer. Oral Diseases, 27(8), 1862–1880. https://doi.org/10.1111/odi.13704 Yeo, Y. H., Samaan, J. S., Ng, W. H., Ting, P., Trivedi, H., Vipani, A., Ayoub, W., Yang, J. D., Liran, O., Spiegel, B., & Kuo, A. (2023). Assessing the performance of ChatGPT in answering questions regarding cirrhosis and hepatocellular carcinoma. Clinical and Molecular Hepatology, 29, 721–732. https://doi.org/10.3350/cmh.2023.0089 Zuccon, G., & Koopman, B. (2023). Dr ChatGPT, tell me what I want to hear: How prompt knowledge impacts health answer correctness. arXiv:2302.13793 [cs.CL]. https://doi.org/10.48550/ arXiv.2302.13793 How to cite this article: Diniz- Freitas, M., Rivas- Mundiña, B., García- Iglesias, J. R., García- Mato, E., & Diz- Dios, P. (2023). How ChatGPT performs in Oral Medicine: The case of oral potentially malignant disorders. Oral Diseases, 00, 1–7. https://doi.org/10.1111/odi.14750 16010825, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/odi.14750 by Universidade de Santiago de Compostela, Wiley Online Library on [12/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License