Identifying Whether a Short Essay was written by a University Student or ChatGPT

Christopher Saarna
427 137

Abstract


This study seeks to clarify whether teachers are able to distinguish between essays written by English L2 students or generated by ChatGPT. 47 instructors who hold experience teaching English to native speakers of Japanese in universities or other higher education institutions were tested on whether they could identify between human written essays and ChatGPT generated essays. The ICNALE written corpus (Ishikawa, 2013) was used to find and randomly select the essays of four Japanese university students’ written work who studied English at roughly CEFR A2 level. The AI chatbot, ChatGPT, was used to generate four essays utilizing prompts which directed the chatbot to mimic grammar mistakes common to nonnative speakers of English. Teachers were requested to identify which of the eight essays they believed to be human written or ChatGPT generated. On average, the teachers were able to identify 54.25% of items accurately. This result is slightly better than random chance, and implies that most teachers cannot make an accurate assessment on a ChatGPT generated essay when ChatGPT is prompted to make grammar mistakes.


Keywords


ChatGPT, Academic dishonesty, EFL, Short essay, Survey of teachers

Full Text:

PDF

References


Saarna, C. (2024). Identifying whether a short essay was written by a university student or ChatGPT. International Journal of Technology in Education (IJTE), 7(3), 611-633. https://doi.org/10.46328/ijte.773




DOI: https://doi.org/10.46328/ijte.773

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Abstracting/Indexing

Web of Science (ESCI) Index                        

                                    

  

International Journal of Technology in Education (IJTE) - ISSN:2689-2758

affiliated with

International Society for Technology, Education and Science (ISTES)

www.istes.org


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.