Research Article

Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research

Mariam Hejab Alshaibani 1 , Waleed Mugahed Al-Rahmi 2 * , Ragad M. Tawafak 3
More Detail
1 Department of Psychology, College of Arts, Taif University, Taif, SAUDI ARABIA2 Department of Management Information System, College of Business Administration, Dar Al Uloom University, Riyadh, SAUDI ARABIA3 Department of Information Technology, AlBuraimi College, Al Buraimi, OMAN* Corresponding Author
Contemporary Educational Technology, 17(4), October 2025, ep595, https://doi.org/10.30935/cedtech/17309
Published: 20 October 2025
OPEN ACCESS   3107 Views   3197 Downloads
Download Full Text (PDF)

ABSTRACT

This study introduces and validates the artificial intelligence anxiety scale (AIAS), a novel instrument designed to measure researchers’ anxieties when employing artificial intelligence (AI) tools in academic writing. As AI technologies rapidly infiltrate scholarly work, however, the primary concern grows about their ethical implications, impact on traditional research skills, and the lack of institutional readiness issues that remain underexplored in existing literature. Addressing this critical gap, the AIAS offers a novel framework grounded in real-world academic concerns. Using an inductive approach, data were collected from 219 faculty members and graduate students at Taif University, Saudi Arabia, revealing four core dimensions of AI-related anxiety: (1) concerns about the accuracy of AI outputs, (2) fear of committing plagiarism, (3) lack of institutional guidelines, and (4) fear of losing research skills. Exploratory and confirmatory factor analyses confirmed the existence of these factors, and reliability testing indicated robust internal consistency. By offering the first validated tool specifically tailored to measure AI-related anxiety in academic contexts, this study provides a significant resource for researchers and institutions. Its application can guide universities in devising training and support initiatives to ensure the ethical and practical integration of AI, thereby sustaining the integrity of traditional research competencies and enhancing the overall quality and credibility of academic endeavors.
Keywords: AI anxiety, confirmatory factor analysis, academic research, psychological assessment

CITATION (APA)

Alshaibani, M. H., Al-Rahmi, W. M., & Tawafak, R. M. (2025). Psychometric validation of the artificial intelligence anxiety scale: A confirmatory factor analysis for academic research. Contemporary Educational Technology, 17(4), ep595. https://doi.org/10.30935/cedtech/17309

REFERENCES

  1. Acemoğlu, D., & Restrepo, P. (2018). Artificial intelligence, automation, and work. Journal of Economic Perspectives, 33(2), 193–210. https://doi.org/10.3386/w24196
  2. AlFarsi, G., Tawafak, R. M., ElDow, A., Malik, S. I., Jabbar, J., & Al Sideiri, A. (2021). Smart classroom technology in artificial intelligence: A review paper. In Proceedings of the International Conference on Culture Heritage, Education, Sustainable Tourism, and Innovation Technologies (pp. 229–235). https://doi.org/10.5220/0010306502290235
  3. Al-Obaydi, L. H., Rahul, D. R., & Pikhart, M. (2024). The effect of online oral reading on reading comprehension, reading anxiety, and classroom anxiety among EFL learners. Education and Information Technologies, 29(3), 2841–2855. https://doi.org/10.1007/s10639-023-11950-y
  4. Al-Obaydi, L. H., Shakki, F., & Pikhart, M. (2025). How does emergency remote teaching affect EFL students’ writing skills? Cogent Education, 12(1), Article 2500001. https://doi.org/10.1080/2331186X.2025.2500001
  5. Alqahtani, M. A., Alamri, M. M., Sayaf, A. M., & Al-Rahmi, W. M. (2022). Exploring student satisfaction and acceptance of e-learning technologies in Saudi higher education. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.939336
  6. Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, Article 149. https://doi.org/10.3389/fpubh.2018.00149
  7. Brown, T. A. (2015). Confirmatory factor analysis for applied research (2nd ed.). The Guilford Press.
  8. Chan, C. K. Y. (2023). Is AI changing the rules of academic misconduct? An in-depth look at students’ perceptions of “AI-giarism.” arXiv. https://doi.org/10.48550/arxiv.2306.03358
  9. Cheng, S. M. S., Mok, C. K. P., Leung, Y. W. Y., Ng, S. S., Chan, K. C. K., Ko, F. W., Chen, C., Yiu, K., Lam, B. H. S., Lau, E. H. Y., Chan, K. K. P., Luk, L. L. H., Li, J. K. C., Tsang, L. C. H. Poon, L. L. M., Hui, D. S. C., & Peiris, M. (2022). Neutralizing antibodies against the SARS-CoV-2 Omicron variant BA.1 following homologous and heterologous CoronaVac or BNT162b2 vaccination. Nature Medicine, 28(3), 486–489. https://doi.org/10.1038/s41591-022-01704-7
  10. Chua, S. L., Chen, D.-T., & Wong, A. F. L. (1999). Computer anxiety and its correlates: A meta-analysis. Computers in Human Behavior, 15(5), 609–623. https://doi.org/10.1016/S0747-5632(99)00039-4
  11. Chubb, J., Cowling, P., & Reed, D. (2021). Speeding up to keep up: Exploring the Use of AI in the research process. Springer Nature, 37(4), 1439–1457. https://doi.org/10.1007/s00146-021-01259-0
  12. Clustering, A. S. (2019). International Arab Conference on Information Technology (ACIT). Higher Education, 165, Article 170.
  13. Dahri, N. A., Al-Rahmi, W. M., Almogren, A. S., Yahaya, N., Vighio, M. S., Al-maatuok, Q., Al-Rahmi, A. M., & Al-Adwan, A. S. (2023). Acceptance of mobile learning technology by teachers: Influencing mobile self-efficacy and 21st-century skills-based training. Sustainability, 15(11), Article 8514. https://doi.org/10.3390/su15118514
  14. DeVellis, R. F. (2016). Scale development: Theory and applications (4th ed.). SAGE.
  15. DiStefano, C., Liu, J., Jiang, N., & Shi, D. (2018). Examination of the weighted root mean square residual (WRMR) in evaluating model fit for structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 25(3), 453–466. https://doi.org/10.1080/10705511.2017.1390394
  16. Eldow, A., Shakir, M., Talab, M. A., Muttar, A. K., & Tawafak, R. M. (2006). Literature review of authentication layer for public cloud computing: A meta-analysis. Journal of Theoretical and Applied Information Technology, 97, Article 12.
  17. Evelyn, T. (2021). Surfacing anger and anxiety in graduate research writing. Journal of Multi-Disciplinary Studies, 45(3), 55–70.
  18. Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 18(1), 39–50. https://doi.org/10.1177/002224378101800104
  19. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.
  20. Habes, M., Jacobson, A. M., Braffett, B. H., Rashid, T., Ryan, C. M., Shou, H., Cui, Y., Davatzikos, C., Luchsinger, J. A., Biessels, G. J., Bebu, I., Gubitosi-Klug, R. A., Bryan, R. N., Nasrallah, I. M., & DCCT/EDIC Research Group. (2023). Patterns of regional brain atrophy and brain aging in middle- and older-aged adults with type 1 diabetes. JAMA Network Open, 6(6), Article e2316182. https://doi.org/10.1001/jamanetworkopen.2023.16182
  21. Hafsa, S., Majid, M. A., & Tawafak, R. M. (2021). Learnability factors of AR usage performance: Validating through survey. In Proceedings of the 2021 International Conference on Software Engineering & Computer Systems and 4th International Conference on Computational Science and Information Management (pp. 371–376). IEEE. https://doi.org/10.1109/ICSECS52883.2021.00074
  22. Hair Jr, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.). Cengage Learning.
  23. Hamari, J., Koivisto, J., & Sarsa, H. (2014, January). Does gamification work? -- A literature review of empirical studies on gamification. 2014 47th Hawaii International Conference on System Sciences, 3025–3034. https://doi.org/10.1109/HICSS.2014.377
  24. Hinkin, T. R. (1995). A review of scale development practices in the study of organizations. Journal of Management, 21(5), 967–988. https://doi.org/10.1177/014920639502100509
  25. Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118
  26. Iqbal Malik, S., Mathew, R., Tawafak, R. M., & Alfarsi, G. (2021). A web-based model to enhance algorithmic thinking for novice programmers. E-Learning and Digital Media, 18(6), 616–633. https://doi.org/10.1177/20427530211026988
  27. Jain, R., & Jain, A. (2023). Generative AI in writing research papers: A new type of algorithmic bias and uncertainty in scholarly work. arXiv. https://doi.org/10.48550/arxiv.2312.10057
  28. Jang, W., & Byon, K. K. (2020). Antecedents and consequence associated with esports gameplay. International Journal of Sports Marketing and Sponsorship, 21(1), 1–22. https://doi.org/10.1108/IJSMS-01-2019-0013
  29. Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). The Guilford Press.
  30. Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
  31. Lei, P.-W., & Shiverdecker, L. K. (2020). Performance of fit indices in multiple-group confirmatory factor models with categorical outcomes. Structural Equation Modeling: A Multidisciplinary Journal, 27(3), 439–461.
  32. Li, L., Huang, Q., Wang, D. C., Ingbar, D. H., & Wang, X. (2020). Acute lung injury in patients with COVID‐19 infection. Clinical and Translational Medicine, 10(1), 20–27. https://doi.org/10.1002/ctm2.16
  33. Liu, J., & Jagadish, H. V. (2024). Institutional efforts to help academic researchers implement generative AI in research. The MIT Press. https://doi.org/10.1162/99608f92.2c8e7e81
  34. Mohammadkarimi, E. (2023). Teachers’ reflections on academic dishonesty in EFL students’ writings in the era of artificial intelligence. Canadian Philosophy of Education Society, 6(2). https://doi.org/10.37074/jalt.2023.6.2.10
  35. Nilsson, N. J. (2010). The quest for artificial intelligence: A history of ideas and achievements. Cambridge University Press.
  36. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). McGraw-Hill.
  37. Perkins, M., & Roe, J. (2024). Generative AI tools in academic research: Applications and implications for qualitative and quantitative research methodologies. arXiv. https://doi.org/10.48550/arxiv.2408.06872
  38. Perkins, M., Roe, J., Postma, D., McGaughran, J., & Hickerson, D. (2023). Game of tones: Faculty detection of GPT-4 generated content in university assessments. arXiv. https://doi.org/10.48550/arxiv.2305.18081
  39. Pikhart, M., Klimova, B., & Al-Obaydi, L. H. (2024). Exploring university students’ preferences and satisfaction in utilizing digital tools for foreign language learning. Frontiers in Education, 9. https://doi.org/10.3389/feduc.2024.1412377
  40. Rosen, L. D., & Weil, M. M. (1995). Computer anxiety: A cross-cultural comparison of university students in ten countries. Computers in Human Behavior, 11(1), 45–64. https://doi.org/10.1016/0747-5632(94)00021-9
  41. Russell, S. J., & Norvig, P. (2020). Artificial intelligence: A modern approach (4th ed.). Pearson.
  42. Tatnall, A. (2019). Editorial for EAIT issue 2, 2019. Education and Information Technologies, 24, 953–962. https://doi.org/10.1007/s10639-019-09874-7
  43. Tawafak, R. M., Alfarsi, G., AlNuaimi, M. N., Eldow, A., Malik, S. I., & Shakir, M. (2020). Model of faculty experience in e-learning student satisfaction. In Proceedings of the 2020 International Conference on Computer Science and Software Engineering (pp. 83–87). IEEE. https://doi.org/10.1109/CSASE48920.2020.9142071
  44. Tawafak, R. M., Al-Rahmi, W. M., Alshimai, A., Alyoussef, I. Y., & Aldaijy, A. (2025). Exploring the impact of digital gameplay on behavioral intentions in learning graphics programming courses. Contemporary Educational Technology, 17(2), Article ep576. https://doi.org/10.30935/cedtech/16115
  45. Tawafak, R. M., Romli, A., & Arshah, R. A. (2019). E-learning prospect on improving academic performance in Omani universities. IOP Conference Series: Materials Science and Engineering, 551(1), Article 012033. https://doi.org/10.1088/1757-899X/551/1/012033
  46. Teng, M. F. (2023). Scientific writing, reviewing, and editing for open-access TESOL journals: The role of ChatGPT. International Journal of TESOL Studies, 5(1), 87–91. https://doi.org/10.58304/ijts.20230107
  47. Ullah, N., Alnumay, W. S., Al-Rahmi, W. M., Alzahrani, A. I., & Al-Samarraie, H. (2020). Modeling cost saving and innovativeness for blockchain technology adoption by energy management. Energies, 13(18), Article 4783. https://doi.org/10.3390/en13184783
  48. Wang, Y. Y., & Wang, Y. S. (2019). Development and validation of an artificial intelligence anxiety scale: An initial application in predicting motivated learning behavior. Interactive Learning Environments, 30(4), 619–634. https://doi.org/10.1080/10494820.2019.1674887
  49. Wen, B., Xu, C., Wolfe, R., Wang, L. L., & Howe, B. (2024). Mitigating overconfidence in large language models: A behavioral lens on confidence estimation and calibration. In NeurIPS 2024 Workshop on Behavioral Machine Learning.
  50. Yusuf, A., Pervin, N., & González, M. R. (2024). Generative AI and the future of higher education: A threat to academic integrity or reformation? Evidence from multicultural perspectives. Springer Nature, 21, Article 21. https://doi.org/10.1186/s41239-024-00453-6
  51. Zhou, X., Smith, L., & Patel, R. (2023). Exploring ethical and psychological dimensions of AI in academic writing. Technology and Society Review, 12(4), 76–89.