Research Article

Students’ Perspectives on the Use of Differentiated Assessment Tool: Results from an Explanatory Sequential Mixed-Method Pilot Study

Christye Majuddin 1 , Mas Nida Md. Khambari 1 * , Su Luan Wong 1 , Norliza Ghazali 1 , Noris Mohd. Norowi 1
More Detail
1 Universiti Putra Malaysia, Malaysia* Corresponding Author
Contemporary Educational Technology, 14(2), April 2022, ep358, https://doi.org/10.30935/cedtech/11667
Published: 02 February 2022
OPEN ACCESS   2263 Views   1819 Downloads
Download Full Text (PDF)

ABSTRACT

While educators worldwide are moving towards the universal design for learning, it is also essential to ensure learners are suitably assessed. Assessments and learning profoundly reciprocate one another as assessment may inform the learning practices, and vice versa. Resonating the aforesaid view, PutraPacer was developed as a customizable tool to empower instructors in embracing differentiated assessment. The objective of this pilot study is to elicit feedback on the use of PutraPacer as a differentiated assessment tool among a group of undergraduate students at a higher education institution. Drawing on the UTAUT model, this study employs an explanatory sequential mixed-method design to gather both quantitative and qualitative data. The quantitative findings show that the mean values for performance expectancy, effort expectancy, social influences, and behavioural intention to use PutraPacer are ranged between 3.56 and 3.67. Based on the Pearson’s correlation coefficient, there are strong association between performance expectancy, effort expectancy, social influences with behavioural intention to use PutraPacer as a differentiated assessment tool. The qualitative findings reveal that the students perceived PutraPacer as a user-friendly and a learning tool that promotes individualized learning experience and supports students with different abilities, and iii) a good platform for practices, quizzes, and revision.
Keywords: alternative assessment, differentiated assessment, learners’ diversity, explanatory sequential mixed method

CITATION (APA)

Majuddin, C., Md. Khambari, M. N., Wong, S. L., Ghazali, N., & Mohd. Norowi, N. (2022). Students’ Perspectives on the Use of Differentiated Assessment Tool: Results from an Explanatory Sequential Mixed-Method Pilot Study. Contemporary Educational Technology, 14(2), ep358. https://doi.org/10.30935/cedtech/11667

REFERENCES

  1. Adnan, N. L., Mohd Sallem, N. R., Muda, R., & Wan Abdullah, W. K. (2019). Is current formative assessment still relevant in turning students into deep learners? TEM Journal, 8(1), 298-304. https://doi.org/10.18421/TEM81-41
  2. Ajideh, P., & Nourdad, N. (2012). The effect of dynamic assessment on EFL reading comprehension in different proficiency levels. Language Testing in Asia, 2(4), 101-122. https://doi.org/10.1186/2229-0443-2-4-101
  3. Algozzine, B., & Anderson, K. M. (2007). Tips for teaching: Differentiating instruction to include all students. Preventing School Failure: Alternative Education for Children and Youth, 51(3), 49-54. https://doi.org/10.3200/psfl.51.3.49-54
  4. Ali, H. I. H. (2015). Toward differentiated assessment in a public college in Oman. English Language Teaching, 8(12), 27-36. https://doi.org/10.5539/elt.v8n12p27
  5. Alias, A., & Osman, K. (2015). Assessing oral communication skills in science: A rubric development. Asia Pacific Journal of Educators and Education, 30, 107-122. http://eprints.usm.my/34769/1/APJEE_30_Art_7_(105_-_122).pdf
  6. Al‐Mahrooqi, R., & Denman, C. (2018). Alternative assessment. In J. I. Liontas & M. DelliCarpini (Eds.), The TESOL encyclopedia of English language teaching (pp. 1-6). John Wiley & Sons, Inc. https://doi.org/10.1002/9781118784235.eelt0325
  7. Anggraeni, C. W. (2018). Promoting education 4.0 in English for survival class: What are the challenges? The Journal of English Language and Literature, 2(1), 12-24. https://doi.org/10.31002/metathesis.v2i1.676
  8. Baumann, C., & Harvey, M. (2018). Competitiveness vis-à-vis motivation and personality as drivers of academic performance. International Journal of Educational Management, 32(1), 185-202. https://doi.org/10.1108/IJEM-10-2017-0263
  9. Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7-74. https://doi.org/10.1080/0969595980050102
  10. Burke, K. (2005). How to assess authentic learning (4th ed.). Corwin Press.
  11. Caliskan, H. & Kasikci, Y. (2010). The application of traditional and alternative assessment and evaluation tools by teachers in social studies. Procedia Social and Behavioural Sciences, 2(2010), 4152-4156. https://doi.org/10.1016/j.sbspro.2010.03.656
  12. Catherine, N., Geofrey, K. M., Moya, M. B., & Aballo, G. (2017). Effort expectancy, performance expectancy, social influence and facilitating conditions as predictors of behavioural intentions to use ATMs with fingerprint authentication in Ugandan banks. Global Journal of Computer Science and Technology: E Network, Web & Security, 17(5), 5-22. https://computerresearch.org/index.php/computer/article/view/1622
  13. Chan, T. W., Looi, C. K., Chen, W., Wong, L. H., Chang, B., Liao, C. C. Y., Cheng, H., Chen, Z. H., Liu, C.C., Kong, S.C., Jeong, H., Mason, J., So, H. J., Murthy, S., Yu, F. Y., Wong, S.L., King, R. B., Gu, X., Wang, M., … Ogata, H. (2018). Interest-driven creator theory: Towards a theory of learning design for Asia in the twenty-first century. Journal of Computers in Education, 5, 435-461. https://doi.org/10.1007/s40692-018-0122-0
  14. Chan, Y. F., & Sidhu, G. K. (2010). Authentic assessment and pedagogical strategies in higher education. Journal of Social Sciences, 6(2), 153-161. https://doi.org/10.3844/jssp.2010.153.161
  15. Charmaz, K. (2014). Constructing grounded theory. SAGE.
  16. Chin, H., Thien, L. M., & Chew, C. M. (2019). The reforms of national assessments in Malaysian education system. Journal of Nusantara Studies, 4(1), 93-111. https://doi.org/10.24200/jonus.vol4iss1pp93-111
  17. Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (4th ed.). Pearson Education, Inc.
  18. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika 16(3), 297-334. https://doi.org/10.1007/BF02310555
  19. Dikli, S. (2003). Assessment at a distance: Traditional vs. alternative assessments. The Turkish Online Journal of Educational Technology, 2(3), 13-19. https://files.eric.ed.gov/fulltext/EJ1101956.pdf
  20. Fisk, P. (2017). Education 4.0 … the future of learning will be dramatically different, in school and throughout life. https://www.peterfisk.com/2017/01/future-education-young-everyone-taught-together/
  21. Gardner, H. (1992). Assessment in context: The alternative to standardized testing. In B. R. Gifford & M. C. O’Connor (Eds.), Changing assessments: Alternative views of aptitude, achievement, and instruction (pp. 77-119). Springer. https://doi.org/10.1007/978-94-011-2968-8_4
  22. Ghani, I. B. A., Ibrahim, N. H., Yahaya, N. A., & Surif, J. (2017). Enhancing students’ HOTS in laboratory educational activity by using concept map as an alternative assessment tool. Chemistry Education Research and Practice, 18(4), 849-874. https://doi.org/10.1039/C7RP00120G
  23. Gozuyesil, E., & Tanriseven, I. (2017). A meta-analysis of the effectiveness of alternative assessment techniques. Eurasian Journal of Educational Research, 70, 37-56. https://doi.org/10.14689/ejer.2017.70.3
  24. Gregory, G. H., & Chapman, C. (2013). Differentiated instructional strategies: One size doesn’t fit all. Corwin Press.
  25. Gulicheva, E., Lisin, E., Osipova, M., & Khabdullin, A. (2017). Leading factors in the formation of innovative education environment. Journal of International Studies, 10(2), 129-137. https://doi.org/10.14254/2071-8330.2017/10-2/9
  26. Hariharasudan, A., & Kot, S. (2018). A scoping review on digital English and education 4.0 for industry 4.0. Social Sciences, 7(11), 227. https://doi.org/10.3390/socsci7110227
  27. Hashemian, M., & Azadi, G. (2011). Arguing for the use of portfolio in L2 classrooms. Theory and Practice in Language Studies, 1(5), 501-506. https://doi.org/10.4304/tpls.1.5.501-506
  28. Ho, E., Bin, J., & Chang, J. (2012). Survey of middle school student learning: Saving the generation of unmotivated. http://topic.parenting.com.tw/issue/2013/futurelearning/article2-1-2.aspx
  29. Hoogland, K., & Tout, D. (2018). Computer-based assessment of mathematics into the twenty-first century: Pressures and tensions. ZDM, 50(4), 675-686. https://doi.org/10.1007/s11858-018-0944-2
  30. Ibrahim, N., Mohd Ayub, A. F., & Md. Khambari, M. N. (2016, December 17). Students’perspectives on the use of mobile phone in learning activities [Paper presentation]. Graduate Research in Education (GREduc) 2016 Seminar, Selangor, Malaysia. http://spel3.upm.edu.my/max/dokumen/GREDUC_GREduc2016_E-proceedings.pdf
  31. Isen, A. M., Daubman, K. A., & Nowicki, G. P. (1987). Positive affect facilitates creative problem solving. Journal of Personality and Social Psychology, 52(6), 1122-1131. https://doi.org/10.1037//0022-3514.52.6.1122
  32. Janisch, C., Liu, X., & Akrofi, A. (2007). Implementing alternative assessment: Opportunities and obstacles. Educational Forum, 71(3), 221-230. https://doi.org/10.1080/00131720709335007
  33. Johanson, G. A., & Brooks, G. P. (2009). Initial scale development: Sample size for pilot studies. Educational and Psychological Measurement, 70(3), 394-400. https://10.1177/0013164409355692
  34. John, M. (2018). Assessment reform in Malaysia: Policy into practice in primary schools [Doctoral dissertation, University of Stirling]. STORRE: Stirling Online Research Repository. http://hdl.handle.net/1893/29915
  35. Kaur, A., Noman, M., & Awang-Hashim, R. (2018). Exploring and evaluating differentiated assessment practices of in-service teachers for components of differentiation. Teaching Education, 30(2), 160-176. https://doi.org/10.1080/10476210.2018.1455084
  36. Koshy, S. (2013). Differentiated assessment activities: Customising to support learning. In P. Bartholomew, N. Courtney, & C. Nygaard (Eds.), Quality enhancement of university teaching and learning (pp. 37). Libri Publishing.
  37. Lawrence, R., Ching, L. F., & Abdullah, H. (2019). Strengths and weaknesses of education 4.0 in the higher education institution. International Journal of Innovative Technology and Exploring Engineering, 9(2S3), 511-519. https://doi.org/10.35940/ijitee.b1122.1292s319
  38. Lawrence-Brown, D. (2004). Differentiated instruction: Inclusive strategies for standards-based learning that benefit the whole class. American Secondary Education, 32(3), 34-62. https://www.jstor.org/stable/41064522
  39. Lawson-Body, A., Willoughby, L., Lawson-Body, L., & Tamandja, E. M. (2018). Students’ acceptance of e-books: An application of UTAUT. Journal of Computer Information Systems, 60(3), 256-267. https://doi.org/10.1080/08874417.2018.1463577
  40. Letina, A. (2015). Primjena tradicionalnih i alternativnih oblika vrednovanja učeničkih postignuća u nastavi prirode i društva. [Application of traditional and alternative assessment in science and social studies teaching]. Croatian Journal of Education, 17(1), 137-152. https://doi.org/10.15516/cje.v17i0.1496
  41. Lin, J. W., & Lai, Y. C. (2019). User acceptance model of computer-based assessment: Moderating effect and intention-behaviour effect. Australasian Journal of Educational Technology, 35(1). https://doi.org/10.14742/ajet.4684
  42. Llewellyn, D. (2002). Inquiry within: Implementing inquiry-based science standards. Corwin Press.
  43. Loh, C. Y. R., & Teo, T. C. (2017). Understanding Asian students learning styles, cultural influence and learning strategies. Journal of Education & Social Policy, 7(1), 194-210. http://jespnet.com/journals/Vol_4_No_1_March_2017/23.pdf
  44. Ministry of Education. (2013). Malaysia education blueprint 2013-2025 (Preschool to post-secondary education). Ministry of Education Malaysia. https://www.moe.gov.my/menumedia/media-cetak/penerbitan/dasar/1207-malaysia-education-blueprint-2013-2025/file
  45. Ministry of Education. (2015). Malaysia education blueprint 2015-2015: Higher education. Kementerian Pendidikan Malaysia. https://www.kooperation-international.de/uploads/media/3._Malaysia_Education_Blueprint_2015-2025__Higher_Education__.pdf
  46. Mohtar, T. M. T. (2010). The use of alternative assessment to sustain teaching and learning. Penerbit UPSI.
  47. Moon, R. T., Brighton, C. M., & Tomlinson, C. A. (2020). Using differentiated classroom Assessment to enhance student learning [eBook edition]. Routledge. https://doi.org/10.4324/9780429452994
  48. Nasri, N., Roslan, S. N., Sekuan, M. I., Bakar, K. A., & Puteh, S. N. (2010). Teachers’ perception on alternative assessment. Procedia-Social and Behavioural Sciences, 7(C), 37-42. https://doi.org/10.1016/j.sbspro.2010.10.006
  49. Newstead, S. E., & Findlay, K. (1997). Some problems with using examination performance as a measure of teaching ability. Psychology Teaching Review, 6, 23-30.
  50. Nguyen, T. T. M., & Nguyen, T. D. (2010). Determinants of learning performance of business students in a transitional market. Quality Assurance in Education, 18(4), 304-316. https://doi.org/10.1108/09684881011079152
  51. Noguera, P., Darling-Hammond, L., & Friedlaender, D. (2015). Equal opportunity for deeper learning. Jobs for the Future. https://files.eric.ed.gov/fulltext/ED560802.pdf
  52. Noman, M., & Kaur, A. (2014). Differentiated assessment: A new paradigm in assessment practices for diverse learners. International Journal of Education and Applied Sciences, 1(4), 167-174.
  53. Norazilawati, A., Noorzeliana, I., Mohd Sahandri, G. H., & Saniah, S. (2015). Planning and implementation of school-based assessment (SBA) among teachers. Procedia-Social and Behavioural Sciences, 211, 247-254. https://doi.org/10.1016/j.sbspro.2015.11.031
  54. NSW Education Standards Authority. (n.d.) Differentiated assessment. https://syllabus.nesa.nsw.edu.au/support-materials/differentiated-assessment/
  55. Pallant, J. (2016). SPSS survival manual: A step by step guide to data analysis using SPSS program. McGraw-Hill Education.
  56. Saldaña, J. (2009). The coding manual for qualitative researchers. SAGE.
  57. Shute, V. J., & Rahimi, S. (2017). Review of computer‐based assessment for learning in elementary and secondary education. Journal of Computer Assisted Learning, 33(1), 1-19. https://doi.org/10.1111/jcal.12172
  58. Suki, N. M., & Suki, N. M. (2017). Determining students’ behavioural intention to use animation and storytelling applying the UTAUT model: The moderating roles of gender and experience level. The International Journal of Management Education, 15(3), 528-538. https://doi.org/10.1016/j.ijme.2017.10.002
  59. Suprayogi, M. N., Valcke, M., & Godwin, R. (2017). Teachers and their implementation of differentiated instruction in the classroom. Teaching and Teacher Education, 67, 291-301. https://doi.org/10.1016/j.tate.2017.06.020
  60. Swaran Singh, C. K., & Abdul Samad, A. (2012). The use of portfolio as an assessment tool in the Malaysian L2 classroom. International Journal of English Language Education, 1(1), 94-108. https://doi.org/10.5296/ijele.v1i1.2851
  61. Tomlinson, C. A. (2001). How to differentiate instruction in mixed-ability classrooms. Association for Supervision and Curriculum Development.
  62. Tomlinson, C. A. (2015). Teaching for excellence in academically diverse classrooms. Society, 52(3), 203-209. https://doi.org/10.1007/s12115-015-9888-0
  63. Tomlinson, C. A., & Moon, T. R. (2013). Assessment and student success in a differentiated classroom. Association for Supervision and Curriculum Development.
  64. Varsavsky, C., & Rayner, G. (2013). Strategies that challenge: Exploring the use of differentiated assessment to challenge high-achieving students in large enrolment undergraduate cohorts. Assessment and Evaluation in Higher Education, 38(7), 789-802. https://doi.org/10.1080/02602938.2012.714739
  65. Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User acceptance of information technology: Toward a unified view. MIS Quarterly, 27(3), 425-478. https://doi.org/10.2307/30036540
  66. Whitehead, A. L., Julious, S. A., Cooper, C. L., & Campbell, M. J. (2016). Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Statistical Methods in Medical Research, 25(3), 1057-1073. https://doi.org/10.1177/0962280215588241
  67. Wilson, D. M., & Narasuman, S. (2020). Investigating teachers’ implementation and strategies on higher order thinking skills in school based assessment instruments. Asian Journal of University Education, 16(1), 70-84. https://doi.org/10.24191/ajue.v16i1.8991
  68. Zitlow, C. S., & Kohn, A. (2001). The case against standardized testing: Raising the scores, ruining the schools. The English Journal, 91(1), 112-114. https://doi.org/10.2307/821673