Research Article

AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices

Trang Phan 1 * , Mariya A. Yukhymenko-Lescroart 1 , Curtis J. Bonk 2
More Detail
1 Department of Advanced Studies in Education, Kremen School of Education and Human Development, California State University, Fresno, Fresno, CA, USA2 Department of Learning, Design, and Adult Education, Indiana University, Bloomington, IN, USA* Corresponding Author
Contemporary Educational Technology, 18(3), July 2026, ep672, https://doi.org/10.30935/cedtech/18936
Published: 07 July 2026
OPEN ACCESS   1222 Views   362 Downloads
Download Full Text (PDF)

ABSTRACT

The research study aimed to explore how preservice teachers engaged with an artificial intelligence (AI)-based tool called Ask.SMILE to develop higher-order level of questions based on the criteria of Bloom’s taxonomy. Specifically, we evaluated students’ questioning techniques, motivation, satisfaction with Ask.SMILE, and assessed their aspirations to use the tool in future classrooms. Using a mixed-methods approach, 178 participants generated over 50 questions each, received feedback from Ask.SMILE, and reflected on their experiences through written responses. The results revealed that Ask.SMILE supported participants in creating questions aligned with Bloom’s taxonomy principles, encouraging iterative improvement and fostering metacognitive awareness. Participants reported enhanced understanding of inquiry-based learning and recognized the importance of using structured questioning strategies to promote critical thinking. While the findings highlight the educational potential of AI tools like Ask.SMILE, limitations include reliance on self-reported data, the absence of longitudinal tracking, and varying levels of AI proficiency among participants. It is suggested that future research should focus on the long-term impact of AI-supported questioning strategies and practices in diverse classroom contexts, and professional development to enhance teacher proficiency with AI tools.
Keywords: question-generating, techniques, strategies, inquiry-based learning, engagement

CITATION (APA)

Phan, T., Yukhymenko-Lescroart, M. A., & Bonk, C. J. (2026). AI and critical thinking: A mixed-method exploration of students’ uses of Ask.SMILE for question-generating practices. Contemporary Educational Technology, 18(3), ep672. https://doi.org/10.30935/cedtech/18936

REFERENCES

  1. AACTE. (2022). Colleges of education: A national portrait. American Association of Colleges for Teacher Education. https://aacte.org/wp-content/uploads/2022/03/Colleges-of-Education-A-National-Portrait-Executive-Summary.pdf
  2. Aflalo, E. (2021). Students generating questions as a way of learning. Active Learning in Higher Education, 22(1), 63-75. https://doi.org/10.1177/1469787418769120
  3. Aguilar, O. G., Mortimer, E. F., & Scott, P. (2010). Learning from and responding to students’ questions: The authoritative and dialogic tension. Journal of Research in Science Teaching, 47, 174-193. https://doi.org/10.1002/tea.20315
  4. Ali, F., Choy, D., Divaharan, S., Tay, H., & Chen, W. (2023). Supporting self-directed learning and self-assessment using TeacherGAIA, a generative AI chatbot application: Learning approaches and prompt engineering. Learning: Research and Practice, 9(2), 135-147. https://doi.org/10.1080/23735082.2023.2258886
  5. Anderson L, W., & Krathwohl D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Pearson Longman.
  6. Anistyasari, Y., Ekohariadi, E., Yani, M. T., Wiriawan, O., & Hidayati, S. C. (2024). An artificial intelligence-based tool for student-generated questions. E3S Web of Conferences, 513, Article 040018. https://doi.org/10.1051/e3sconf/202451302003
  7. Becker, R. R. (2000). The critical role of students’ questions in literacy development. The Educational Forum, 64(3), 261-271. https://doi.org/10.1080/00131720008984763
  8. Bozkurt, A. (2023). Generative artificial intelligence (AI) powered conversational educational agents: The inevitable paradigm shift. Asian Journal of Distance Education, 18(1), 198-204.
  9. Buckner, E., & Kim, P. (2014). Integrating technology and pedagogy for inquiry-based learning: The Stanford mobile inquiry-based learning environment (SMILE). Prospects, 44(1), 99-118. https://doi.org/10.1007/s11125-013-9269-7
  10. Chin, C., & Brown, D. E. (2002). Student generated questions: A meaningful aspect of learning in science. International Journal a/Science Education, 24(5), 521-549. https://doi.org/10.1080/09500690110095249
  11. Chin, C., & Osborne, J. (2008). Students’ questions: A potential resource for teaching and learning science. Studies in Science Education, 44(1), 1-39. https://doi.org/10.1080/03057260701828101
  12. Chiu, T. K.F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023). Systematic literature review on opportunities, challenges, and future research recommendations of artificial intelligence in education. Computers and Education: Artificial Intelligence, 4, Article 100118. https://doi.org/10.1016/j.caeai.2022.100118
  13. Crompton, H., & Burke, D. (2024). The educational affordances and challenges of ChatGPT: State of the field. TechTrends, 68, 380-392. https://doi.org/10.1007/s11528-024-00939-0
  14. Davey, B., & McBride, S. (1986). Effects of question-generation training on reading comprehension. Journal of Educational Psychology, 78(4), 256-262. https://doi.org/10.1037/0022-0663.78.4.256
  15. Digital Education Council. (2024). AI or not AI: What students want. Digital Education Council global AI student survey 2024. Digital Education Council. https://www.digitaleducationcouncil.com/post/digital-education-council-global-ai-student-survey-2024
  16. Ennis, R. H. (1985). The logical basis for measuring CT skills. Educational Leadership, 43(2), 44-48.
  17. Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. B. Baron, & R. J. Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26). W.H. Freeman.
  18. Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction: Executive summary, The Delphi report. California Academic Press.
  19. Holbeck, R. (2025). Helping students develop AI prompting skills for critical thinking. Faculty Focus. https://www.facultyfocus.com/articles/teaching-with-technology-articles/helping-students-develop-ai-prompting-skills-for-critical-thinking/
  20. Hou, X., Chang, Z., Lu, Z., Wenli, C., Wan, L., Feng, W., Hu, H., & Guo, Q. (2025). EduThink4AI: Bridging educational critical thinking and multi-agent LLM Systems. arXiv.
  21. Hsu, H. Y., & Kim, P. (2016). Preservice teachers’ uses of SMILE to enact student-generated questioning practices. International Journal of Innovation in Education, 3(2-3), 110-121. https://doi.org/10.1504/IJIIE.2016.081547
  22. IBM Corp. (2023). IBM SPSS statistics for Macintosh, version 29.0.1.0. IBM Corp. https://www.ibm.com/support/pages/downloading-ibm-spss-statistics-2901
  23. Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., Krusche, S., Kutyniok, G., Michaeli, T., Nerdel, C., Pfeffer, J., Poquet, O., Sailer, M., Schmidt, A., Seidel, T., ... Kasneci, G. (2023). ChatGPT for good? On opportunities and challenges of large language models for education. Learning and Individual Differences, 103, Article 102274. https://doi.org/10.1016/j.lindif.2023.102274
  24. Khasawneh, E., Hodge-Zickerman, A., York, C. S., Smith, T. J., & Mayall, H. (2023). Examining the effect of inquiry-based learning versus traditional lecture-based learning on students’ achievement in college algebra. International Electronic Journal of Mathematics Education, 18(1), Article em0724. https://doi.org/10.29333/iejme/12715
  25. Kim, M., & Adlof, L. (2024). Adapting to the future: ChatGPT as a means for supporting constructivist learning environments. TechTrends, 68, 37-46 (2024). https://doi.org/10.1007/s11528-023-00899-x
  26. Kim, P. (2024). Trends of AI in education and new competencies for the future at the exploring the ethical use of AI TIP Conference [Video]. YouTube. https://youtu.be/DsUEhxlNku8?si=h2vXfm-P2zNfdg5o
  27. Kim, P., & An, J. Y. (2016). New evaluation vector through the Stanford mobile inquiry-based learning environment (SMILE) for participatory action research. Healthcare Informatics Research, 22(3), 164-171. https://doi.org/10.4258/hir.2016.22.3.164
  28. Kim, P., & Lee, J. (2020). Evolution of online learning environments and the emergence of intelligent MOOCs. In K. Zhang, C. J. Bonk, T. C. Reeves, & T. H. Reynolds (Eds.), MOOCs and open education in the Global South: Challenges, successes, and opportunities (pp. 329-341). Routledge.
  29. Kim, P., Miranda, T., & Olaciregui, C. (2008). Pocket school: Exploring mobile technology as a sustainable literacy education option for underserved indigenous children in Latin America. International Journal of Educational Development, 28(4), 435-445. https://doi.org/10.1016/j.ijedudev.2007.11.002
  30. Kim, P., Wang, W., & Bonk, C. J. (2025). Generative AI as a coach to help students enhance proficiency in question formulation. Journal of Educational Computing Research, 63(3), 565-586. https://doi.org/10.1177/07356331251314222
  31. King, A. (1997). ASK to THINK-TEL WHY: A model of transactive peer tutoring for scaffolding higher level complex learning. Educational Psychologist, 32(4), 221-235. https://doi.org/10.1207/s15326985ep3204_3
  32. Li, Z., Wang, C., & Bonk, C. J. (2024). Exploring the utility of ChatGPT for self-directed online language learning. Online Learning, 28(3), 157-180. https://doi.org/10.24059/olj.v28i3.4497
  33. McGivney, E., & Kim, P. (2016). Using technology to teach the art of asking questions. Stanford Social Innovation Review. https://doi.org/10.48558/89AT-W123
  34. Mollick, E. (2024). Co-intelligence: Living and working with AI. Penguin.
  35. Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1(2), 117-175. https://doi.org/10.1207/s1532690xci0102_1
  36. Paul, R., & Elder, L. (2006). Critical thinking: Tools for taking charge of your learning and your life. Pearson Prentice Hall.
  37. Phan, T., & Zhu, M. (2026). Asking better questions with AI: Exploring student inquiry, critical thinking growth, and the interpretive role of Ask.SMILE. TechTrends. https://doi.org/10.1007/s11528-026-01192-3
  38. Phan, T., Zheng, X., & Zhu, M. (2026). Comparative analysis of AI-generated and peer feedback on student-generated questions in teacher education. AI Enhanced Learning, 2(1), 99-121. https://doi.org/10.70725/493067ulyywm
  39. Quintana, C., Reiser, B., Davis, E. A., Krajcik, J., Golan, R., Kyza, E., Edelson, D., & Soloway, E. (2002). Evolving a scaffolding design framework for designing educational software. In P. Bell, R. Stevens, & T. Satwicz (Eds.), Proceedings of the 5th International Conference for the Learning Sciences. Erlbaum.
  40. Rogoff, B. (1990). Apprenticeship in thinking: Cognitive development in a social context. Oxford Academic. https://doi.org/10.1093/oso/9780195059731.001.0001
  41. Rudolph, J., Tan, S., & Tan, S. (2023). ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? Journal of Applied Learning & Teaching, 6(1), 342-363. https://doi.org/10.37074/jalt.2023.6.1.9
  42. Schroeder, R. (2024). The AI augmented professor of 2024. Inside Higher Ed. https://www.insidehighered.com/opinion/blogs/online-trending-now/2024/05/08/ai-augmented-professor-2024
  43. Seeds of Empowerment. (2025). SMILE LLM project brief: Phi-2 model deployment on Raspberry Pi 5 using llama.cpp. Seeds of Empowerment. https://www.seedsofempowerment.org/smile-llm/
  44. Shi, Z., Castellucci, G., Filice, S., Kuzi, S., Kravi, E., Agichtein, E., Rokhlenko, O., & Malmasi, S. (2025). Ambiguity detection and uncertainty calibration for question answering with large language models. In Proceedings of the 5th Workshop on Trustworthy NLP (pp. 41-55). ACL Anthology. https://doi.org/10.18653/v1/2025.trustnlp-main.4
  45. Singer, J., Marx, R. W., Krajcik, J., & Chambers, J. C. (2000). Constructing extended inquiry projects: Curriculum materials for science education reform. Educational Psychologist, 35(3), 165-178. https://doi.org/10.1207/S15326985EP3503_3
  46. Terada, Y. (2020). Using student-generated questions to promote deeper thinking. Edutopia.
  47. United States Census Bureau. (2025). Back to school: August 2025. United States Census Bureau. https://www.census.gov/newsroom/stories/back-to-school.html
  48. Vendrell, M., & Johnston, S. K. (2026). Scaffolding critical thinking with generative AI: Design principles for integrating large language models in higher education. Computers and Education: Artificial Intelligence, 10, Article 100572. https://doi.org/10.1016/j.caeai.2026.100572
  49. Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
  50. Wang, C., Li, Z., & Bonk, C. J. (2024). Understanding self-directed learning in AI-assisted writing: A mixed methods study of postsecondary learners. Computers & Education: Artificial Intelligence, 6, Article 100247. https://doi.org/10.1016/j.caeai.2024.100247
  51. Wertsch, J. V. (1991). Voices of the mind: A sociocultural approach to mediated action. Harvard University Press.
  52. Wu, L., Liu, Y., How, M. L., & He, S. (2023). Investigating student-generated questioning in a technology-enabled elementary science classroom: A case study. Education Sciences, 13(2), Article 158. https://doi.org/10.3390/educsci13020158
  53. Yu, F. Y., & Wu, W. S. (2020). Effects of student-generated feedback corresponding to answers to online student-generated questions on learning: What, why, and how? Computers & Education, 145, Article 103. https://doi.org/10.1016/j.compedu.2019.103723
  54. Yu, F. Y., Chang, Y. L., & Wu, H. L. (2015). The effects of an online student question-generation strategy on elementary school student English learning. Research and Practice in Technology Enhanced Learning, 10, Article 24. https://doi.org/10.1186/s41039-015-0023-z
  55. Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – Where are the educators? International Journal of Educational Technology in Higher Education, 16(1), Article 39. https://doi.org/10.1186/s41239-019-0171-0
  56. Zhang, K., Bonk, C. J., Reeves, T. C., & Reynolds, T. H. (Eds.). (2020). MOOCs and open education in the Global South: Challenges, successes, and opportunities. Routledge. https://doi.org/10.4324/9780429398919
  57. Zhi, J., Kumar, H., & Lee, M. (2026). Investigating the effects of LLM use on critical thinking under time constraints: Access timing and time availability. In Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (pp. 1-21). ACM. https://doi.org/10.1145/3772318.3791796