Research Article

The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis

Thada Jantakoon 1 , Thiti Jantakun 2 * , Kitsadaporn Jantakun 2 , Weerapa Pongpanich 3 , Rungfa Pasmala 4 , Panita Wannapiroon 4 , Prachyanun Nilsook 4
More Detail
1 Rajabhat Maha Sarakham University, Mahasarakham, THAILAND2 Roi Et Rajabhat University, Roi Et, THAILAND3 Rangsit University, Pathum Thani, THAILAND4 King Mongkut’s University of Technology North Bangkok, Bangkok, THAILAND* Corresponding Author
Contemporary Educational Technology, 17(4), October 2025, ep596, https://doi.org/10.30935/cedtech/17310
Published: 20 October 2025
OPEN ACCESS   3413 Views   3871 Downloads
Download Full Text (PDF)

ABSTRACT

The increasing integration of artificial intelligence (AI) in education has raised significant questions about its pedagogical value, especially in language learning. This meta-analysis examines the extent to which AI contributes to the development of English-speaking and listening skills. A systematic review of the literature was conducted by the preferred reporting items for systematic reviews and meta-analyses guidelines, utilizing peer-reviewed studies indexed in Scopus, ERIC, and EBSCOhost from 2017 to 2024. Nineteen studies met the inclusion criteria, all of which utilized experimental or quasi-experimental designs with measurable learning outcomes. The analysis reveals a substantial overall effect of AI-enhanced instruction (standardized mean difference [SMD] = 0.981, 95% confidence interval [0.571, 1.391], p < .001), with particularly notable improvements in speaking proficiency (SMD = 1.033). Although listening outcomes showed a positive trend (SMD = 0.714), the effect did not attain statistical significance. Considerable heterogeneity was noted across the studies, reflecting variations in learner populations, instructional contexts, and AI applications. Quality appraisal using the risk of bias in non-randomized studies of interventions framework indicated a predominantly low to moderate risk of bias. Publication bias analysis, including funnel plot symmetry and fail-safe N, further confirmed the reliability of the results. These findings highlight the advantages of AI in enhancing speaking skills within English instruction and underscore the need for further empirical studies to investigate its impact on listening comprehension. Collectively, the results provide timely, evidence-based guidance for educators and policymakers aiming to integrate AI effectively into language education. Highlight the advantages of AI in enhancing speaking skills within English instruction and underscore the need.
Keywords: artificial intelligence, English instruction, speaking skill, listening skill, meta-analysis

CITATION (APA)

Jantakoon, T., Jantakun, T., Jantakun, K., Pongpanich, W., Pasmala, R., Wannapiroon, P., & Nilsook, P. (2025). The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis. Contemporary Educational Technology, 17(4), ep596. https://doi.org/10.30935/cedtech/17310

REFERENCES

  1. Adipat, S. (2023). An artificial intelligence-enhanced phenomenon-based learning approach for interdisciplinary understanding and speaking skills. International Journal of Instruction, 16(3), 531–550. https://doi.org/10.29333/iji.2023.16329a
  2. Agnes, D., & Srinivasan, R. (2024). Fostering vocabulary memorization: Exploring the impact of AI-generated mnemonic keywords on vocabulary learning through Anki flashcards. World Journal of English Language, 14(2), 434–445. https://doi.org/10.5430/wjel.v14n2p434
  3. Ahmed Ali, S. G. (2020). Using an artificial intelligence application for developing primary school pupils’ oral language skills. Journal of Education Sohag UNV, 75(75), 67–110. https://doi.org/10.21608/edusohag.2020.97643
  4. Alsowat, H. H. (2020). Evidence-based practices of English language teaching: A meta-analysis of meta-analyses. English Language Teaching, 13(11), 75–94. https://doi.org/10.5539/elt.v13n11p75
  5. Amin, M. Y. M. (2023). AI and Chat GPT in language teaching: Enhancing EFL classroom support and transforming assessment techniques. International Journal of Higher Education Pedagogies, 4(4), 554–567. https://doi.org/10.33422/ijhep.v4i4.554
  6. Bao, R., & Sile, L. (2024). Empowering English vocabulary instruction through generative artificial intelligence: The AEFC method. Journal of Educational Research Progress, 4(4). https://doi.org/10.55375/joerp.2024.4.4
  7. Caldwell, A. R. (2022). SimplyAgree: An R package and jamovi module for simplifying agreement and reliability analyses. Journal of Open Source Software, 7(71), Article 4148. https://doi.org/10.21105/joss.04148
  8. Chang, H. (2023). The effect of AI chatbot-based tourism English instruction on intercultural communicative competence. STEM Journal, 24(2), 15–32. https://doi.org/10.16875/stem.2023.24.2.15
  9. Chen, C. H., Koong, C. S., & Liao, C. (2022). Influences of integrating dynamic assessment into a speech recognition learning design to support students’ English speaking skills, learning anxiety, and cognitive load. Educational Technology & Society, 25(1), 1–14.
  10. Choe, S., Lee, K. A., & So, Y. (2020). The effects of phonemic awareness instructions on L2 listening comprehension: A meta-analysis. The Journal of AsiaTEFL, 17(4), 1294–1316. https://doi.org/10.18823/asiatefl.2020.17.4.9.1294
  11. Chu, J., & Szlagor, M. (2023). Multi-intelligence English teaching model based on distance and open education. International Journal of Web-Based Learning and Teaching Technologies, 18(1), 1–15. https://doi.org/10.4018/ijwltt.325617
  12. Daweli, T. W., & Mahoub, R. A. M. (2024). Exploring EFL learners’ perspectives on using AI tools and their impacts in reading instruction: An exploratory study. Arab World English Journal, 10(11). https://doi.org/10.24093/awej/call10.11
  13. Duisenova, M. M., & Zhorabekova, A. N. (2023). The effectiveness of gamification and artificial intelligence in increasing the motivation and effectiveness of students in learning English in elementary school. Eurasia Journal of Mathematics, Science and Technology Education, 19(10), Article em2330. https://doi.org/10.29333/ejmste/13670
  14. Eyding, D., Lelgemann, M., Grouven, U., Härter, M., Kromp, M., Kaiser, T., Kerekes, M. F., Gerken, M., & Wieseler, B. (2010). Reboxetine for acute treatment of major depression: Systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ, 341, Article c4737. https://doi.org/10.1136/bmj.c4737
  15. Fathi, J., Rahimi, M., & Derakhshan, A. (2024). Improving EFL learners’ speaking skills and willingness to communicate via artificial intelligence-mediated interactions. System, 121, Article 103254. https://doi.org/10.1016/j.system.2024.103254
  16. Feng, T., Deng, J., Liu, Y., Gao, Y., Peng, L., & Fu, J. (2023). A study on application of artificial intelligence in college English teaching management and evaluation. In Proceedings of the 2023 International Conference on Emerging Information Technology for Smart Applications (pp. 128–132). ACM. https://doi.org/10.1145/3641343.3641447
  17. Ferrer, R. L. (1998). Graphical methods for detecting bias in meta-analysis. Family Medicine, 30, 579–583.
  18. Hao, T., Wang, Z., & Ardasheva, Y. (2021). Technology-assisted vocabulary learning for EFL learners: A meta-analysis. Journal of Research on Educational Effectiveness, 14(4), 834–862. https://doi.org/10.1080/19345747.2021.1917028
  19. Higgins, J. P., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186
  20. Ho, P. X. P. (2024). Using ChatGPT in English language learning: A study on I.T. students’ attitudes, habits, and perceptions. International Journal of TESOL & Education, 4(1). https://doi.org/10.54855/ijte.24414
  21. Hwang, W. Y., Luthfi, M. I., & Liu, Y. F. (2024). AI-enhanced video drama-making for improving writing and speaking skills of students learning English as a foreign language. Innovation in Language Learning and Teaching. https://doi.org/10.1080/17501229.2024.2437655
  22. Ibrahim, K. A. A. A., Kassem, M. A. M., & Lami, D. (2024). Intelligent computer-assisted language assessment (ICALA) in philosophy-based language instruction: Unraveling the effects on critical thinking, self-evaluation, academic resilience, and speaking development. Language Testing in Asia, 14(1), Article 46. https://doi.org/10.1186/s40468-024-00320-1
  23. Jerusha, A., & Rajakumari, R. (2024). Longitudinal analysis of student progress in AI-integrated English language courses. In Proceedings of the 2024 International Conference on Knowledge Engineering and Communication Systems (pp. 1–6). IEEE. https://doi.org/10.1109/ickecs61492.2024.10616489
  24. John, E., & Yunus, M. M. (2021). A systematic review of social media integration to teach speaking. Sustainability, 13(16), Article 9047. https://doi.org/10.3390/su13169047
  25. Kemelbekova, Z., Degtyareva, X., Yessenaman, S., Ismailova, D., & Seidaliyeva, G. (2024). AI in teaching English as a foreign language: Effectiveness and prospects in Kazakh higher education. XLinguae, 17(1), 69–83. https://doi.org/10.18355/XL.2024.17.01.05
  26. Kim, N. Y. (2022). AI-integrated mobile-assisted language learning: Is it an effective way of preparing for the TOEIC test in classroom environments? English Teaching, 77(3), 79–102. https://doi.org/10.15858/engtea.77.3.202209.79
  27. Knapp, G., & Hartung, J. (2003). Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine, 22(17), 2693–2710. https://doi.org/10.1002/sim.1482
  28. Konyrova, L. (2024). The evolution of language learning: Exploring AI’s impact on teaching English as a second language. Eurasian Science Review: An International Peer-Reviewed Multidisciplinary Journal, 2(2), 133–138. https://doi.org/10.63034/esr-42
  29. Lee, J. H., Lee, H., & Lo, Y. Y. (2023). Effects of EMI-CLIL on secondary-level students’ English learning: A multilevel meta-analysis. Studies in Second Language Learning and Teaching, 13(3), 439–467. https://doi.org/10.14746/ssllt.38277
  30. Lee, S., Kuo, L., Xu, Z., & Hu, X. (2020). The effects of technology-integrated classroom instruction on K-12 English language learners’ literacy development: A meta-analysis. Computer Assisted Language Learning, 35(1–2), 1–29. https://doi.org/10.1080/09588221.2020.1774612
  31. Li, Y. (2024). A deep learning multimodal feature fusion approach for assessing English study. In Proceedings of the 2024 4th Asia-Pacific Conference on Communications Technology and Computer Science (pp. 0047–0052). IEEE. https://doi.org/10.1109/acctcs61748.2024.00047
  32. Lin, C. J., & Mubarok, H. (2021). Learning analytics for investigating the mind map-guided AI chatbot approach in an EFL flipped speaking classroom. Educational Technology & Society, 24(4), 16–35.
  33. Lin, H., & Liu, X. (2024). InVideo AI: Creating instructional audiovisual input for English learning. RELC Journal. https://doi.org/10.1177/00336882241267773
  34. Ling, W. (2023). Artificial intelligence in language instruction: Impact on English learning achievement, L2 motivation, and self-regulated learning. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1261955
  35. Liu, P.-L., & Chen, C.-J. (2023). Using an AI-based object detection translation application for English vocabulary learning. Educational Technology & Society, 26(3), 5–20. https://doi.org/10.30191/ETS.202307_26(3).0002
  36. Madhavi, E., Sivapurapu, L., Koppula, V., Sreehari, V., & Rani, P. (2023). Developing learners’ English-speaking skills using ICT and AI tools. Journal of Advanced Research in Applied Sciences and Engineering Technology, 32(2), 142–153. https://doi.org/10.37934/araset.32.2.142153
  37. Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group. (2010). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. International Journal of Surgery, 8(5), 336–341. https://doi.org/10.1016/j.ijsu.2010.02.007
  38. Murdoch, Y. D., & Lin, Y. (2023). Factors mediating the link between engagement and satisfaction among online English-mediated instruction learners during COVID-19. Electronic Journal of e-Learning, 21(3). https://doi.org/10.34190/ejel.21.3.3031
  39. Ni, A., Cheung, A., & Shi, J. (2022). Effects of educational technology on reading achievement for Chinese K-12 English second language learners: A meta-analysis. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.1025761
  40. Peng, J., & Xie, X. (2021). English-medium instruction as a pedagogical strategy for the sustainable development of EFL learners in the Chinese context: A meta-analysis of its effectiveness. Sustainability, 13(10), Article 5637. https://doi.org/10.3390/su13105637
  41. Qian, M., & Kong, C. (2024). Exploring the gap: The challenge of achieving human-like generalization for concept-based translation instruction using large language models. In Proceedings of the AAAI Spring Symposia. AAAI. https://doi.org/10.1609/aaaiss.v3i1.31283
  42. Qiao, H., & Zhao, A. (2023). Artificial intelligence-based language learning: Illuminating the impact on speaking skills and self-regulation in Chinese EFL context. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1255594
  43. Rahmati, J., Izadpanah, S., & Shahnavaz, A. (2021). A meta-analysis on educational technology in English language teaching. Language Testing in Asia, 11(1), Article 37. https://doi.org/10.1186/s40468-021-00121-w
  44. Rosenthal, R. (1979). The ‘file drawer problem’ and tolerance for null results. Psychological Bulletin, 86(3), 638–641. https://doi.org/10.1037/0033-2909.86.3.638
  45. Sharifi, M., Seyedagha Sharifi, M. B., Rostami AbuSaeedi, A., Jafarigohar, M., & Zandi, B. (2018). Retrospect and prospect of computer assisted English language learning: A meta-analysis of the empirical literature. Computer Assisted Language Learning, 31(4), 413–436. https://doi.org/10.1080/09588221.2017.1412325
  46. Song, F., Khan, K. S., Dinnes, J., & Sutton, A. J. (2002). Asymmetric funnel plots and publication bias in meta-analyses of diagnostic accuracy. International journal of epidemiology, 31(1), 88–95. https://doi.org/10.1093/ije/31.1.88
  47. Sonsaat-Hegelheimer, S., & Kurt, Ş. (2024). The impact of generative AI-powered chatbots on L2 comprehensibility. Journal of Second Language Pronunciation, 10(3), 339–374. https://doi.org/10.1075/jslp.24053.son
  48. Tang, J. L., & Liu, J. L. (2000). Misleading funnel plot for bias detection in meta-analysis. Journal of Clinical Epidemiology, 53(5), 477–484. http://doi.org/10.1016/s0895-4356(99)00204-8
  49. Um, H., Kim, H., Choi, D., & Oh, H. (2023). An AI-based English education platform during the COVID-19 pandemic. Universal Access in the Information Society, 23, 1233–1248. https://doi.org/10.1007/s10209-023-01046-2
  50. Wang, C., Zou, B., Du, Y., & Wang, Z. (2024a). The impact of different conversational generative AI chatbots on EFL learners: An analysis of willingness to communicate, foreign language speaking anxiety, and self-perceived communicative competence. System, 127, Article 103533. https://doi.org/10.1016/j.system.2024.103533
  51. Wang, X., Zhang, Q., Chen, H., Neitzel, A. J., & Davis, M. H. (2024b). The effects of language learning strategy instruction on college students’ English achievement and learner autonomy in mainland China: A meta-analysis. International Journal of Educational Research, 124, Article 102442. https://doi.org/10.1016/j.ijer.2024.102442
  52. Wang, Y., & Zheng, L. (2022). A novel deep framework for English communication based on educational psychology perspective. Frontiers in Public Health, 10. https://doi.org/10.3389/fpubh.2022.916101
  53. Wu, Y. (2024). Study on the impact of utilizing ChatGPT and other AI tools for feedback in EAP writing classrooms on the discursive writing performance of English major students. Transactions on Social Science, Education and Humanities Research, 4, 143–150. https://doi.org/10.62051/4se95x52
  54. Xatamova, N. (2024). The future of legal English learning: Integrating AI into ESP education. SPAST Reports, 1(7). https://doi.org/10.52773/tsuull.conf.teach.foreign.lang.2024.8.5/pnxs6025
  55. Yang, T. (2024). Impact of artificial intelligence software on English learning motivation and achievement. SHS Web of Conferences, 193, Article 02011. https://doi.org/10.1051/shsconf/202419302011
  56. Yaseen, M. G. (2023). Exploring the evolution of AI integration in English as a foreign language education: A Scopus-based bibliometric analysis (1997-2023). Mesopotamian Journal of Computer Science, 2(2), 19–31. https://doi.org/10.58496/mjcsc/2023/019
  57. Zhang, C., Meng, Y., & Ma, X. (2024). Artificial intelligence in EFL speaking: Impact on enjoyment, anxiety, and willingness to communicate. System, 121, Article 103259. https://doi.org/10.1016/j.system.2024.103259
  58. Zhao, Y. (2013). Recent developments in technology and language learning: A literature review and meta-analysis. CALICO Journal, 21(1), 7–27. https://doi.org/10.1558/cj.v21i1.7-27
  59. Zheng, C., Chen, X., Zhang, H., & Chai, C.S. (2024). Automated versus peer assessment: Effects on learners’ English public speaking. Language Learning & Technology, 28(2), 210–228.
  60. Zou, B., Guan, X., Shao, Y., & Chen, P. (2023). Supporting speaking practice by social network-based interaction in artificial intelligence (AI)-assisted language learning. Sustainability, 15(4), Article 2872. https://doi.org/10.3390/su15042872