A Bibliometric Analysis of Research on ChatGPT in Education

Hamza Polat; Arif Cem Topuz; Mine Yıldız; Elif Taşlıbeyaz; Engin Kurşun

doi:10.46328/ijte.606

A Bibliometric Analysis of Research on ChatGPT in Education

Hamza Polat, Arif Cem Topuz, Mine Yıldız, Elif Taşlıbeyaz, Engin Kurşun

575 176

Abstract

ChatGPT has become a prominent tool for fostering personalized and interactive learning with the advancements in AI technology. This study analyzes 212 academic research articles indexed in the Scopus database as of July 2023. It maps the trajectory of educational studies on ChatGPT, identifying primary themes, influential authors, and contributing institutions. By employing bibliometric indicators and network analysis, the study explores collaboration patterns, citation trends, and the evolution of research interests. The findings show the exponential growth of interest in leveraging ChatGPT for educational purposes and provide insights into the specific educational domains and contexts that have garnered the most attention. Furthermore, the study reveals the collaborative dynamics and intellectual foundations shaping the field by examining co-authorship and citation networks. This bibliometric analysis contributes to a comprehensive understanding of the current state of ChatGPT research in education, offering researchers and practitioners valuable insights into evolving trends and potential future directions for this innovative aspect of AI and learning.

Keywords

ChatGPT, Educational technologies, Bibliometric analysis, Generative AI

Full Text:

PDF

References

AalSaud, A. B. F. (2021). Artificial intelligence and children’s education: Review study. International Research in Higher Education, 6(2). https://doi.org/10.5430/irhe.v6n2p1

Ali, J., Shamsan, M., Hezam, T., & Mohammed, A. (2023). Impact of ChatGPT on learning motivation: Teachers and students’ voices. Journal of English Studies in Arabia Felix, 2(1), 41-49. https://doi.org/10.56540/jesaf.v2i1.51

Alzahrani, A. (2022). A systematic review of artificial intelligence in education in the Arab world. Revista Amazonia Investiga, 11(54). https://doi.org/10.34069/ai/2022.54.06.28

Aria, M., & Cuccurullo, C. (2017). Bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007

Arif, T., Munaf, U., & Ul-Haque, I. (2023). The future of medical education and research: Is ChatGPT a blessing or blight in disguise?. Medical Education Online, 28. https://doi.org/10.1080/10872981.2023.2181052.

Atlas, S. (2023). ChatGPT for higher education and professional development: A guide to conversational AI. https://digitalcommons.uri.edu/cba_facpubs/548/

Baker, T., & Smith, L. (2019). Educ-AI-tion rebooted? Exploring the future of artificial intelligence in schools and colleges. https://media.nesta.org.uk/documents/Future_of_AI_and_education_v5_WEB.pdf

Boucher, E. M., Harake, N. R., Ward, H. E., Stoeckl, S. E., Vargas, J., Minkel, J., ... & Zilca, R. (2021). Artificially intelligent chatbots in digital mental health interventions: A review. Expert Review of Medical Devices, 18(1), 37-49. https://doi.org/10.1080/17434440.2021.2013200

Buchanan, C., Howitt, M. L., Wilson, R., Booth, R. G., Risling, T., & Bamford, M. (2021). Predicted influences of artificial intelligence on nursing education: Scoping review. JMIR Nursing, 4(1). https://doi.org/10.2196/23933

Bommineni, V., Bhagwagar, S., Balcarcel, D., Bommineni, V., Davatzikos, C., & Boyer, D. (2023). Performance of ChatGPT on the MCAT: The road to personalized and equitable premedical learning. https://doi.org/10.1101/2023.03.05.23286533.

Cascella, M., Montomoli, J., Bellini, V., & Bignami, E. (2023). Evaluating the feasibility of ChatGPT in healthcare: An analysis of multiple clinical and research scenarios. Journal of Medical Systems, 47(33). https://doi.org/10.1007/s10916-023-01925-4

Celik, I., Dindar, M., Muukkonen, H., & Järvelä, S. (2022). The promises and challenges of artificial intelligence for teachers: A systematic review of research. TechTrends, 66(4), 616-630. https://doi.org/10.1007/s11528-022-00715-y

Chan, K. S., & Zary, N. (2019). Applications and challenges of implementing artificial intelligence in medical education: Integrative review. JMIR Medical Education, 5(1). https://doi.org/10.2196/13930

Chu, S. T., Hwang, G. J., & Tu, Y. F. (2022). Artificial intelligence-based robots in education: A systematic review of selected SSCI publications. Computers and Education: Artificial Intelligence. https://doi.org/10.1016/j.caeai.2022.100091

Cobo, M. J., López-Herrera, A. G., Herrera-Viedma, E., & Herrera, F. (2011). An approach for detecting, quantifying, and visualizing the evolution of a research field: A practical application to the Fuzzy Sets Theory field. Journal of Informetrics, 5(1), 146–166. https://doi.org/10.1016/j.joi.2010.10.002

Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 1–12. https://doi.org/10.1080/14703297.2023.2190148

Cooper, G. (2023). Examining science education in Chatgpt: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32(3), 444-452. https://doi.org/10.1007/s10956-023-10039-y

Crescenzi-Lanna, L. (2023). Literature review of the reciprocal value of artificial and human intelligence in early childhood education. Journal of Research on Technology in Education, 55(1), 21-33. https://doi.org/10.1080/15391523.2022.2128480

Donthu, N., Kumar, S., Mukherjee, D., Pandey, N., & Lim, W. M. (2021). How to conduct a bibliometric analysis: An overview and guidelines. Journal of Business Research, 133, 285–296. https://doi.org/10.1016/j.jbusres.2021.04.070

Durso, S. de O., & Arruda, E. P. (2022). Artificial intelligence in distance education: A systematic literature review of Brazilian studies. Problems of Education in the 21st Century, 80(5), 679-692. https://doi.org/10.33225/pec/22.80.679

Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71. https://doi.org/10.1016/j.ijinfomgt.2023.102642

Eggmann, F., Weiger, R., Zitzmann, N. U., & Blatz, M. B. (2023). Implications of large language models such as ChatGPT for dental medicine. Journal Of Esthetic and Restorative Dentistry, 1-5. https://doi.org/10.1111/jerd.13046

Eysenbach, G. (2023). The role of ChatGPT, generative language models and artificial intelligence in medical education: A conversation with ChatGPT - and a call for papers. JMIR Medical Education, 9. https://doi.org/10.2196/46885.

Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2023). A SWOT analysis of ChatGPT: Implications for educational practice and research. Innovations in Education and Teaching International. https://doi.org/10.1080/14703297.2023.2195846

Fidan, M., & Gencel, N. (2022). Supporting the instructional videos with chatbot and peer feedback mechanisms in online learning: The effects on learning performance and intrinsic motivation. Journal of Educational Computing Research, 60(7), 1716-1741. https://doi.org/10.1177/07356331221077901

Francisco, R. E., & de Oliveira Silva, F. (2022). Intelligent tutoring system for computer science education and the use of artificial intelligence: A literature review. international conference on computer supported education, CSEDU - Proceedings, 1. https://doi.org/10.5220/0011084400003182

Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023). How does ChatGPT perform on the United States Medical Licensing Examination? The implications of large language models for medical education and knowledge assessment. JMIR Medical Education, 9, e45312. https://doi.org/10.2196/45312

Guan, C., Mou, J., & Jiang, Z. (2020). Artificial intelligence innovation in education: A Twenty-year data-driven historical analysis. International Journal of Innovation Studies, 4(4), 134–147. https://doi.org/10.1016/j.ijis.2020.09.001

Haleem, A., Javaid, M., & Singh, R. P. (2022). An era of ChatGPT as a significant futuristic support tool: A study on features, abilities, and challenges. BenchCouncil transactions on benchmarks, standards and evaluations, 2(4), 100089. https://doi.org/10.1016/j.tbench.2023.100089.

Harmon, J., Pitt, V., Summons, P., & Inder, K. J. (2021). Use of artificial intelligence and virtual reality within clinical simulation for nursing pain education: A scoping review. Nurse Education Today, 97, 1-9. https://doi.org/10.1016/j.nedt.2020.104700

Heersmink, R., van den Hoven, J., van Eck, N. J., & van Berg, J. den. (2011). Bibliometric mapping of computer and information ethics. Ethics and Information Technology, 13(3), 241–249. https://doi.org/10.1007/s10676-011-9273-7

Hood, W. W., & Wilson, C. S. (2001). The literature of bibliometrics, scientometrics, and informetrics. Scientometrics, 52(2), 291–314. https://doi.org/10.1023/A:1017919924342

Huang, Y., Gomaa, A., Weissmann, T., Grigo, J., Tkhayat, H. B., Frey, B., ... & Putz, F. (2023). Benchmarking chatgpt-4 on acr radiation oncology in-training exam (txit): Potentials and challenges for ai-assisted medical education and decision making in radiation oncology. arXiv preprint arXiv:2304.11957.

Huang, J., & Tan, M. (2023). The role of ChatGPT in scientific communication: writing better scientific review articles. American Journal of Cancer Research, 13(4), 1148–1154.

Huang, X., Zou, D., Cheng, G., Chen, X., & Xie, H. (2023). Trends, research issues and applications of artificial intelligence in language education. Educational Technology & Society, 26(1), 112-131. https://www.jstor.org/stable/48707971

Huh, S. (2023). Are ChatGPT’s knowledge and interpretation ability comparable to those of medical students in Korea for taking a parasitology examination?: A descriptive study. Journal of Educational Evaluation for Health Professions, 20(1). https://doi.org/10.3352/jeehp.2023.20.01

Hwang, G. J., & Tu, Y. F. (2021). Roles and research trends of artificial intelligence in mathematics education: A bibliometric mapping analysis and systematic review. Mathematics, 9 (6),584. https://doi.org/10.3390/math9060584

Imran, M., & Almusharraf, N. (2023). Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature. Contemporary Educational Technology, 15(4), 464. https://doi.org/10.30935/cedtech/13605

Iqbal, S., Ahmad, S., Akkour, K., Wafa, A. N. A., AlMutairi, H. M., & Aldhufairi, A. M. (2021). Review article: Impact of Artificial Intelligence in Medical Education. MedEdPublish, 10(1). https://doi.org/10.15694/mep.2021.000041.1

Jeblick, K., Schachtner, B., Dexl, J., Mittermeier, A., Stüber, A., Topalis, J., Weber, T., Wesp, P., Sabel, B., Ricke, J., & Ingrisch, M. (2022). ChatGPT makes medicine easy to swallow: An exploratory case study on simplified radiology reports. ArXiv, abs/2212.14882. https://doi.org/10.48550/arXiv.2212.14882.

Jeon, J., Lee, S., & Choi, S. (2023). A systematic review of research on speech-recognition chatbots for language learning: Implications for future directions in the era of large language models. Interactive Learning Environments,1-19. https://doi.org/10.1080/10494820.2023.2204343

Jeon, J. (2021) Chatbot-assisted dynamic assessment (CA-DA) for L2 vocabulary learning and diagnosis, Computer Assisted Language Learning, https://doi.org/10.1080/09588221.2021.1987272.

Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for Language Teaching and Learning. RELC JournaL. https://doi.org/10.1177/00336882231162868

Kuhail, M. A., Alturki, N., Alramlawi, S., & Alhejori, K. (2023). Interacting with educational chatbots: A systematic review. Education and Information Technologies, 28(1), 973-1018. https://doi.org/10.1007/s10639-022-11177-3

Lancichinetti, A., & Fortunato, S. (2009). Community detection algorithms: A comparative analysis. Physical Review E, 80(5). https://doi.org/10.1103/PhysRevE.80.056117

Liu, S., Wright, A., Patterson, B., Wanderer, J., Turer, R., Nelson, S., McCoy, A., Sittig, D., & Wright, A. (2023). Assessing the value of ChatGPT for clinical decision support optimization. medRxiv : the preprint server for health sciences. https://doi.org/10.1101/2023.02.21.23286254.

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4). https://doi.org/10.3390/educsci13040410

Macdonald, C., Adeloye, D., Sheikh, A., & Rudan, I. (2023). Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis. Journal of Global Health, 13(01003), 1–7. https://doi.org/10.7189/JOGH.13.01003

Mhlanga, D. (2023). Open AI in education, the responsible and ethical use of chatgpt towards lifelong learning. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4354422

Mojadeddi, Z. M., & Rosenberg, J. (2023). The impact of AI and ChatGPT on research reporting. New Zealand Medical Journal, 136(1575), 60–64. Retrieved from https://www.proquest.com/openview/f65736b6d518d9f07a93f47b381aae95/1?cbl

Nadarzynski T, Miles O, Cowie A, Ridge D.(2019). Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: A mixed-methods study. Digital Health, 5. https://doi.org/10.1177/2055207619871808

Nguyen, P. T., Di Rocco, J., Di Sipio, C., Rubei, R., Di Ruscio, D., & Di Penta, M. (2023). Is this Snippet Written by ChatGPT? An Empirical Study with a CodeBERT-Based Classifier. arXiv preprint arXiv:2307.09381. https://doi.org/10.48550/arXiv.2307.09381

Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2. ttps://doi.org/10.1016/j.caeai.2021.100033

Ouyang, F., Zheng, L., & Jiao, P. (2022). Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020. Education and Information Technologies, 27(6). https://doi.org/10.1007/s10639-022-10925-9

Pavlik, J. V. (2023). Collaborating with ChatGPT: Considering the implications of generative artificial intelligence for journalism and media education. Journalism & Mass Communication Educator, 78(1), 84–93. https://doi.org/10.1177/10776958221149577

Pradana, M., Elisa, H. P. ,& Syarifuddin, S. (2023). Discussing ChatGPT in education: A literature review and bibliometric analysis, Cogent Education. https://doi.org/10.1080/2331186X.2023.2243134

Polat, H. (2023). Transforming education with artificial intelligence: Shaping the path forward. In A. Kaban, & A. Stachowicz-Stanusch (Eds.), Empowering Education: Exploring the Potential of Artificial Intelligence (pp. 3-20). ISTES Organization. Retrieved from https://book.istes.org/index.php/ib/article/view/26

Qureshi, R., Shaughnessy, D., Gill, K. A. R., Robinson, K. A., Li, T., & Agai, E. (2023). Are ChatGPT and large language models ``the answer{‘‘} to bringing us closer to systematic review automation? Systematic Reviews, 12(1). https://doi.org/10.1186/s13643-023-02243-z

Río-Rama, M., Maldonado-Erazo, C., & Álvarez‐García, J. (2018). State of the art of research in the sector of thermalism, thalassotherapy and spa: A bibliometric analysis. European Journal of Tourism Research. https://doi.org/10.54055/ejtr.v19i.325

Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1). https://doi.org/10.37074/jalt.2023.6.1.9

Ruksakulpiwat, S., Kumar, A., & Ajibade, A. (2023). Using ChatGPT in medical research: Current status and future directions. Journal of Multidisciplinary Healthcare, 16, 1513–1520. https://doi.org/10.2147/JMDH.S413470

Sakirin, T., & Said, R. (2023). User preferences for ChatGPT-powered conversational interfaces versus traditional methods. Mesopotamian Journal of Computer Science, 24-31. https://doi.org/10.58496/mjcsc/2022/002

Salas-Pilco, S. Z., Xiao, K., & Hu, X. (2022). Artificial intelligence and learning analytics in teacher education: A systematic review. Education Sciences, 12 (8). https://doi.org/10.3390/educsci12080569

Salas-Pilco, S. Z., & Yang, Y. (2022). Artificial intelligence applications in Latin American higher education:A systematic review. International Journal of Educational Technology in Higher Education, 19(1). https://doi.org/10.1186/s41239-022-00326-w

Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Healthcare, 11(6). https://doi.org/10.3390/healthcare11060887

Salvagno, M., Taccone, F. S., & Gerli, A. G. (2023). Can artificial intelligence help for scientific writing? Critical Care, 27(1), 75. https://doi.org/10.1186/s13054-023-04380-2

Sharma, S., Pajai, S., Prasad, R., Wanjari, M. B., Munjewar, P. K., Sharma, R., & Pathade, A. (2023). A critical review of ChatGPT as a potential substitute for diabetes educators. Cureus Journal of Medical Science, 15(5). https://doi.org/10.7759/cureus.38380

Singh, V. K., Singh, P., Karmakar, M., Leta, J., & Mayr, P. (2021). The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis. Scientometrics, 126(6), 5113–5142. https://doi.org/10.1007/s11192-021-03948-5

Song, P., & Wang, X. (2020). A bibliometric analysis of worldwide educational artificial intelligence research development in recent twenty years. Asia Pacific Education Review, 21(3), 473–486. https://doi.org/10.1007/s12564-020-09640-2

Swathi, M. M., & Gopalreddy, K. (2023). Ai-based chatbot with gpt-3. Journal of Engineering Sciences, 14(08). https://jespublication.com/uploads/2023-V14I8081.pdf

Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments, 10(1), 15. https://doi.org/10.1186/s40561-023-00237

Wollny, S., Schneider, J., Di Mitri, D., Weidlich, J., Rittberger, M., & Drachsler, H. (2021). Are we there yet?-a systematic literature review on chatbots in education. Frontiers in Artificial Intelligence, 4, https://doi.org/10.3389/frai.2021.654924

Wu, R., & Yu, Z. (2023). Do AI chatbots improve students learning outcomes? Evidence from a meta‐analysis. British Journal of Educational Technology,1-24. https://doi.org/10.1111/bjet.13334

Xia, Q., Chiu, T. K., Zhou, X., Chai, C. S., & Cheng, M. (2022). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence,4. https://doi.org/10.1016/j.caeai.2022.100118.

Xu, W., & Ouyang, F. (2022). The application of AI technologies in STEM education: a systematic review from 2011 to 2021. International Journal of STEM Education, 9(1). https://doi.org/10.1186/s40594-022-00377-5

Yang, S., & Evans, C. (2019, November). Opportunities and challenges in using AI chatbots in higher education. In Proceedings of the 2019 3rd International Conference on Education and E-Learning (pp. 79-83). https://doi.org/10.1145/3371647.3371659

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators?. International Journal of Educational Technology in Higher Education, 16(1), 1-27. https://doi.org/10.1186/s41239-019-0171-0

Zhai, X., Chu, X., Chai, C. S., Jong, M. S. Y., Istenic, A., Spector, M., Liu, J. B., Yuan, J., & Li, Y. (2021). A Review of Artificial Intelligence (AI) in Education from 2010 to 2020. Complexity. https://doi.org/10.1155/2021/8812542

Zhang, J., Yu, Q., Zheng, F., Long, C., Lu, Z., & Duan, Z. (2016). Comparing keywords plus of WOS and author keywords: A case study of patient adherence research. Journal of the Association for Information Science and Technology, 67(4), 967–972. https://doi.org/10.1002/asi.23437

Zhang, R., Zou, D., & Cheng, G. (2023). A review of chatbot-assisted learning: pedagogical approaches, implementations, factors leading to effectiveness, theories, and future directions. Interactive Learning Environments, 1-29. https://doi.org/10.1080/10494820.2023.2202704

Zhu, J., & Liu, W. (2020). A tale of two databases: the use of Web of Science and Scopus in academic papers. Scientometrics, 123(1), 321–335. https://doi.org/10.1007/s11192-020-03387-8

DOI: https://doi.org/10.46328/ijte.606