Benefits, Challenges, and Methods of Artificial Intelligence (AI) Chatbots in Education: A Systematic Literature Review

Şahin Gökçearslan; Cansel Tosun; Zeynep Gizem Erdemir

doi:10.46328/ijte.600

Benefits, Challenges, and Methods of Artificial Intelligence (AI) Chatbots in Education: A Systematic Literature Review

Şahin Gökçearslan, Cansel Tosun, Zeynep Gizem Erdemir

1232 223

Abstract

In many fields, AI chatbots continue to be popular with new tools and attract the attention of universities, K12 schools, educational organizations, and researchers. The aim of this research is to review the research on AI chatbots by restricting it to the category of education and to examine this research from a methodological point of view. Therefore, we performed a systematic literature review with a sample of 37 SSCI articles published in the educational context. Within the scope of the selected studies, the advantages and disadvantages of AI chatbots in education for students and educators, as well as the types of chatbots used, year, keywords, and method were analyzed. According to the research results, increased motivation to learn and language skill development are advantages for students, while cost-effectiveness and reduced workload are advantages for educators. Limited interaction, misleading answers for learners, originality, and plagiarism are the most common disadvantages for educators. The study also includes research results and recommendations related to the methodological review.

Keywords

AI chatbots, education, systematic literature review, education stakeholders, method

Full Text:

PDF

References

Adamopoulou, E., & Moussiades, L. (2020). Chatbots: History, technology, and applications. Machine Learning with Applications, 2, 100006. 10.1016/j.mlwa.2020.100006

Akçayır, M., & Akçayır, G. (2017). Advantages and challenges associated with augmented reality for education: A systematic review of the literature. Educational Research Review, 20, 1-11. 10.1016/j.edurev.2016.11.002

Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning. Available at SSRN 4337484. 10.2139/ssrn.4337484

Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1). Retrieved from http://www.asianjde.com/ojs/index.php/AsianJDE/article/view/718

Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877-1901.

Chang, C. Y., Hwang, G. J., & Gau, M. L. (2022). Promoting students' learning achievement and self‐efficacy: A mobile chatbot approach for nursing training. British Journal of Educational Technology, 53(1), 171-188. https://doi.org/10.1111/bjet.13158

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8, 75264-75278. 10.1109/access.2020.2988510

Chen, X., Cheng, G., Zou, D., Zhong, B., & Xie, H. (2023). Artificial intelligent robots for precision education: A topic modeling based bibliometric analysis. Educational Technology and Society, 26(1), 171-186. https://doi.org/10.30191/ETS.202301_26(1).0013

Cooper, G. (2023). Examining science education in ChatGPT: An exploratory study of generative artificial intelligence. Journal of Science Education and Technology, 32(3), 444-452. https://doi.org/10.1007/s10956-023-10039-y

Cotton, D. R., Cotton, P. A., & Shipway, J. R. (2023). Chatting and cheating: Ensuring academic integrity in the era of ChatGPT. Innovations in Education and Teaching International, 1-12. https://doi.org/10.1080/14703297.2023.2190148

Delgado-von-Eitzen, C., Anido-Rifón, L., & Fernández-Iglesias, M. J. (2021). Blockchain applications in education: A systematic literature review. Applied Sciences, 11(24), 11811. 10.3390/app112411811

Deveci Topal, A., Dilek Eren, C., & Kolburan Geçer, A. (2021). Chatbot application in a 5th grade science course. Education and Information Technologies, 26(5), 6241-6265. https://doi.org/10.1007/s10639-021-10627-8

Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. Arxiv Preprint, arXiv:1810.04805. 10.48550/arXiv.1810.04805

Dogan, M. E., Goru Dogan, T., & Bozkurt, A. (2023). The use of artificial intelligence (AI) in online learning and distance education processes: A systematic review of empirical studies. Applied Sciences, 13(5), 3056. 10.3390/app13053056

Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., ... & Wright, R. (2023). So what if ChatGPT wrote it?: Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. 10.1016/j.ijinfomgt.2023.102642

Educause (2023). 2023 Educause horizon report: Teahing and learning edition. https://library.educause.edu/resources/2023/5/2023-educause-horizon-report-teaching-and-learning-edition

Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1), 57. https://doi.org/10.1186/s41239-022-00362-6

Godwin-Jones, R. (2023). Emerging spaces for language learning: AI bots, ambient intelligence, and the metaverse. Language Learning & Technology, 27(2), 6–27. https://hdl.handle.net/10125/73501

Goswami, R. (2023). OpenAI changed its plans and won’t train on customer data, Sam Altman says. https://www.cnbc.com/2023/05/05/sam-altman-openai-wont-tap-into-customerapis.html?utm_campaign=DonanimHaber&utm_medium=referral&utm_source=DonanimHaber

Guo, K., Wang, J., & Chu, S. K. W. (2022). Using chatbots to scaffold EFL students’ argumentative writing. Assessing Writing, 54, 100666. https://doi.org/10.1016/j.asw.2022.100666

Gursoy, D., & Sandstrom, J. K. (2016). An updated ranking of hospitality and tourism journals. Journal of Hospitality & Tourism Research, 40(1), 3-18. 10.1177/1096348014538054

H. A. Abu-Alsaad, "Agent applications in e-learning systems and current development and challenges of adaptive e-learning systems," 2019 11th International Conference on Electronics, Computers and Artificial Intelligence (ECAI), Pitesti, Romania, 2019, pp. 1-6, doi: 10.1109/ECAI46879.2019.9042015.

Han, J. W., Park, J., & Lee, H. (2022). Analysis of the effect of an artificial intelligence chatbot educational program on non-face-to-face classes: a quasi-experimental study. BMC Medical Education, 22(1), 830. https://doi.org/10.1186/s12909-022-03898-3

Hsu, T. C., Huang, H. L., Hwang, G. J., & Chen, M. S. (2023). Effects of incorporating an expert decision-making mechanism into chatbots on students’ achievement, enjoyment, and anxiety. Educational Technology & Society, 26(1), 218-231. https://www.jstor.org/stable/48707978

Hsu, Y. C., Hung, J. L., & Ching, Y. H. (2013). Trends of educational technology research: More than a decade of international research in six SSCI-indexed refereed journals. Educational Technology Research and Development, 61, 685-705.10.1007/s11423-013-9290-9

Hwang, G. J., & Chang, C. Y. (2021). A review of opportunities and challenges of chatbots in education. Interactive Learning Environments. 10.1080/10494820.2021.1952615

Ji, H., Han, I., & Ko, Y. (2023). A systematic review of conversational AI in language education: Focusing on the collaboration with human teachers. Journal of Research on Technology in Education, 55(1), 48-63. https://doi.org/10.1080/15391523.2022.2142873

Kassab, M., DeFranco, J., & Laplante, P. (2020). A systematic literature review on Internet of things in education: Benefits and challenges. Journal of Computer Assisted Learning, 36(2), 115-127. 10.1111/jcal.12383

Kavanagh, S., Luxton-Reilly, A., Wuensche, B., & Plimmer, B. (2017). A systematic review of virtual reality in education. Themes in Science and Technology Education, 10(2), 85-119.

Lin, Y. & Yu, Z. (2023), A bibliometric analysis of artificial intelligence chatbots in educational contexts, Interactive Technology and Smart Education, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/ITSE-12-2022-0165

Liu, C. C., Liao, M. G., Chang, C. H., & Lin, H. M. (2022). An analysis of children’ interaction with an AI chatbot and its impact on their interest in reading. Computers & Education, 189, 104576. https://doi.org/10.1016/j.compedu.2022.104576

Loafers (2023). Chat GPT 4 release date: OpenAI to launch latest version of AI chatbot next week. https://opchatgptai.com/chat-gpt-4-release-date/

Luor, T., Johanson, R. E., Lu, H. P., & Wu, L. L. (2008). Trends and lacunae for future computer assisted learning (CAL) research: An assessment of the literature in SSCI journals from 1998-2006. Journal of the American Society for Information Science and Technology, 59(8), 1313-1320.10.1002/asi.20836

Miles, M.B. & Huberman, A.M. (1994). Qualitative data analysis. California: SAGE Publications Inc.

Moldt, J. A., Festl-Wietek, T., Madany Mamlouk, A., Nieselt, K., Fuhl, W., & Herrmann-Werner, A. (2023). Chatbots for future docs: exploring medical students’ attitudes and knowledge towards artificial intelligence and medical chatbots. Medical Education Online, 28(1). https://doi.org/10.1080/10872981.2023.2182659.

Molnár, G., & Szüts, Z. (2018, September). The role of chatbots in formal education. In 2018 IEEE 16th International Symposium on Intelligent Systems and Informatics (SISY) (pp. 000197-000202). IEEE. 10.1109/sisy.2018.8524609

Nee, C. K., Rahman, M. H. A., Yahaya, N., Ibrahim, N. H., Razak, R. A., & Sugino, C. (2023). Exploring the Trend and Potential Distribution of Chatbot in Education: A Systematic Review. International Journal of Information and Education Technology, 13(3), 516-525. 10.18178/ijiet.2023.13.3.1834

Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. 10.1016/j.caeai.2021.100033

OpenAI (2023). Optimizing language models for dialogue-OpenAI. Available online: https://openai.com/blog/chatgpt/

Paek, S., & Kim, N. (2021). Analysis of worldwide research trends on the ımpact of artificial ıntelligence in education. Sustainability, 13(14), 7941. MDPI AG. http://dx.doi.org/10.3390/su13147941

Pérez, J. Q., Daradoumis, T., & Puig, J. M. M. (2020). Rediscovering the use of chatbots in education: A systematic literature review. Computer Applications in Engineering Education, 28(6), 1549-1565. 10.1002/cae.22326

Pham, X. L., Pham, T., Nguyen, Q. M., Nguyen, T. H., & Cao, T. T. H. (2018, November). Chatbot as an intelligent personal assistant for mobile language learning. In Proceedings of the 2018 2nd International Conference on Education and E-Learning (pp. 16-21). 10.1145/3291078.3291115

Rooein, D. (2019, May). Data-driven EDU chatbots. In Companion proceedings of the 2019 world wide web conference (pp. 46-49). 10.1145/3308560.3314191

Rose, R., Holmes, W., Griffiths, M., Forcier, L. B. (2016). Intelligence unleashed. An Argument for AI in Education, Pearson: London, UK.

Rospigliosi, P. A. (2023). Artificial intelligence in teaching and learning: what questions should we ask of ChatGPT?. Interactive Learning Environments, 31(1), 1-3. 10.1080/10494820.2023.2180191

Sabzalieva, E., & Valentini, A. (2023). ChatGPT and artificial intelligence in higher education: Quick start guide. https://eduq.info/xmlui/handle/11515/38828

Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. 10.1016/j.compedu.2020.103862

Tesch, R. (2013). Qualitative research: Analysis types and software. Routledge. 10.4324/9781315067339

UNESCO (2022). K-12 AI curricula: A mapping of government-endorsed AI curricula.https://unesdoc.unesco.org/ark:/48223/pf0000380602

Vázquez-Cano, E., Mengual-Andrés, S., & López-Meneses, E. (2021). Chatbot to improve learning punctuation in Spanish and to enhance open and flexible learning environments. International Journal of Educational Technology in Higher Education, 18(1), 1-20. https://doi.org/10.1186/s41239-021-00269-8

Xia, Q., Chiu, T. K., Chai, C. S., & Xie, K. (2023). The mediating effects of needs satisfaction on the relationships between prior knowledge and self‐regulated learning through artificial intelligence chatbot. British Journal of Educational Technology. https://doi.org/10.1111/bjet.13305

Xu, Y., Wang, D., Collins, P., Lee, H., & Warschauer, M. (2021). Same benefits, different communication patterns: Comparing Children's reading with a conversational agent vs. a human partner. Computers & Education, 161, 104059. https://doi.org/10.1016/j.compedu.2020.104059

Yang, H., Kim, H., Lee, J., & Shin, D. (2022). Implementation of an AI chatbot as an English conversation partner in EFL speaking classes. ReCALL, 34(3), 327-343. https://doi.org/10.1017/S0958344022000039

Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R. R., & Le, Q. V. (2019). Xlnet: Generalized autoregressive pretraining for language understanding. Advances in Neural Information Processing Systems (Edited by: H. Wallach and H. Larochelle and A. Beygelzimer and F. d'Alché-Buc and E. Fox and R. Garnett), 32.

Zhang, K., & Aslan, A. B. (2021). AI technologies for education: Recent research & future directions. Computers and Education: Artificial Intelligence, 2, 100025. 10.48550/arXiv.1906.08237

Zhang, M., & Li, J. (2021). A commentary of GPT-3 in MIT technology review 2021. Fundamental Research, 1(6), 831-833. 10.1016/j.fmre.2021.11.011

DOI: https://doi.org/10.46328/ijte.600