Comparing Paper-Based and Computer-Based Testing: A Systematic Literature Review on Their Effectiveness in Performance Assessment

Meili Yanti; Irmawati M; Dea Santika Rahayu; Niken Yuni Astiti

doi:10.31764/paedagoria.v17i4.39449

Comparing Paper-Based and Computer-Based Testing: A Systematic Literature Review on Their Effectiveness in Performance Assessment

Meili Yanti, Irmawati M, Dea Santika Rahayu, Niken Yuni Astiti

Abstract

The rapid advancement of technology has significantly influenced various aspects of human life. While these developments offer substantial convenience, they also present certain challenges, particularly during the early stages of adoption. In the field of education, technology integration is not only evident in instructional processes but also in assessment practices. One prominent example is the implementation of Computer-Based Tests (CBT), which provide several advantages; however, their effectiveness in evaluating student performance requires further examination. Accordingly, this study aims to compare the traditional Paper-Based Test (PBT) with CBT in terms of their effectiveness in performance assessment within the learning process. This study employs a Systematic Literature Review (SLR) approach guided by two research questions: (P1) How does the effectiveness of Paper-Based Test (PBT) compare with Computer-Based Test (CBT)? and (P2) How do student performances differ between PBT and CBT? The findings from the SLR indicate mixed results. Several studies reveal that students tend to show a higher preference for CBT over PBT, accompanied by increased motivation. Nevertheless, both assessment methods appear to have no significant impact on students’ performance outcomes. Additional findings and analyses are further elaborated in this study.

Keywords

Evaluation; CBT; Motivation; PBT; Performance Assesment.

Full Text:

PDF

References

Awofala, A. O. A. (2020). I ndonesian J ournal of I nformatics E ducation Are the Keyboards Weightier than the Biros ? The Effect of Computer- Based Testing on Students ’ Achievement and Anxiety in Mathematics. Indonesian Journal of Informatics Education, 4(1), 1–8.

Bhoola-Patel, A., & Laher, S. (2011). The Influence of Mode of Test Administration on Test Performance. Journal of Psychology in Africa, 21(1), 139–144. https://doi.org/10.1080/14330237.2011.10820440

Brown, G. T. L. (2019). Technologies and infrastructure: costs and obstacles in developing large-scale computer–based testing. Education Inquiry, 10(1), 4–20. https://doi.org/10.1080/20004508.2018.1529528

Budiastuti, P., Soenarto, S., Muchlas, M., & Ramndani, H. W. (2021). Analysis of Learning Objectives and Basic Competencies in the Lesson Plan for Basic Electricity and Electronics at Vocational High Schools. Jurnal Edukasi Elektro, 5(1), 39–48. https://doi.org/10.21831/jee.v5i1.37776

Carpenter, R., & Alloway, T. (2019). Computer Versus Paper-Based Testing: Are They Equivalent When it Comes to Working Memory? Journal of Psychoeducational Assessment, 37(3), 382–394. https://doi.org/10.1177/0734282918761496

Ferdiansyah, S. (2016). Moving from Paper-Based Testing (PBT) to Computer-Based Testing (CBT) for Classroom Use: Exploring the Opportunities and Challenges. Prosiding ICTTE FKIP UNS 2015, 1, 514–519.

García-Pérez, L., García-Garnica, M., & Olmedo-Moreno, E. M. (2021). Skills for a working future: How to bring about professional success from the educational setting. Education Sciences, 11(1), 1–25. https://doi.org/10.3390/educsci11010027

Hakim, B. M. (2018). Comparative Study on Validity of Paper-Based Test and Computer-Based Test in the Context of Educational and Psychological Assessment among Arab Students. 8(2), 85–91. https://doi.org/10.5539/ijel.v8n2p85

Hassler Hallstedt, M., & Ghaderi, A. (2018). Tablets instead of paper-based tests for young children? Comparability between paper and tablet versions of the mathematical Heidelberger Rechen Test 1-4. Educational Assessment, 23(3), 195–210. https://doi.org/10.1080/10627197.2018.1488587

Hosseini, M., Jafre, M., Abidin, Z., & Baghdarnia, M. (2014). Comparability of Test Results of Computer Based Tests ( CBT ) and Paper and Pencil Tests ( PPT ) among English Language Learners in Iran. Procedia - Social and Behavioral Sciences, 98, 659–667. https://doi.org/10.1016/j.sbspro.2014.03.465

Jalali, S., Zeinali, M., & Nobakht, A. (2014). Effect of Goal Orientation on EFL Learners ’ Performances in CBT and PBT Across Gender. Procedia - Social and Behavioral Sciences, 98, 727–734. https://doi.org/10.1016/j.sbspro.2014.03.474

Jamiludin, Darnawati, & Uke, W. A. S. U. (2017). Students ’ Perception Towards National Examination 2017 : Computer-Based Test or Paper-Based Test. Mediterranean Journal of Social Sciences, 8(4), 139–144.

Karay, Y., Schauber, S. K., Stodch, C., & Sch¨uttpelz-Brauns, K. (2015). Computer Versus Paper — Does It Make Any Difference in Test Performance ? Teaching and Learning inMedicine, 27(1), 57–62. https://doi.org/10.1080/10401334.2014.979175

Khoshsima, H., & Toroujeni, S. M. H. (2017). Comparability of Computer-Based Testing and Paper- Based Testing : Testing Comparability of Computer-Based Testing and Paper- Based Testing : Testing Mode Effect , Testing Mode Order , Computer Attitudes and Testing Mode preference. International Journal of Computer, 24(1), 80–99.

Kitchenham, B., Brereton, O. P., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering – A systematic literature review. Information and Software Technology, 51(1), 7–15. https://doi.org/10.1016/j.infsof.2008.09.009

Munna, A. S., & Kalam, M. A. (2021). Teaching and learning process to enhance teaching effectiveness: literature review. International Journal of Humanities and Innovation (IJHI), 4(1), 1–4. https://doi.org/10.33750/ijhi.v4i1.102

Piaw, C. Y. (2012). Replacing paper-based testing with computer-based testing in assessment : Are we doing wrong ? Procedia-Social and Behavioral Sciences, 64, 655–664. https://doi.org/10.1016/j.sbspro.2012.11.077

Sobremisana, V. S., & Aragon, R. C. (2016). Computer-Based Assessment : Its Effects On Students ’ Performance And Computer-Based Assessment : Its Effects On Students ’ Performance And Attitude. The International Journal Research Publication’s, 5(9), 10–18.

Torres-Carrión, Vicente, P., Aciar, Silvana, González-González, Soledad, C., Rodríguez-Morales, & Germania. (2018). Methodology for Systematic Literature Review applied to Engineering and Education. IEEE Global Engineering Education Conference (EDUCON), 1364–1373.

Uukkivi, A., Labanova, O., Retshnoi, V., Safiulina, E., & Latõnina, M. (2020). Students ’ Testing Preferences in Mathematics. 2nd International Conference on Research in Teaching and Education, 63–73.

Valentine, A., Belski, I., Hamilton, M., Valentine, A., Belski, I., Developing, M. H., & Valentine, A. (2017). Developing creativity and problem-solving skills of engineering students : a comparison of web- and pen-and-paper-based approaches. European Journal of Engineering Education, 0(0), 1–21. https://doi.org/10.1080/03043797.2017.1291584

Yanti, M., Riandi, & Suhandi, A. (2019). Analyzing Tpack Ability of Science Teacher Based on Experience for Teaching Global Warming in Secondary Level. Unnes Science Education Journal, 8(2), 130–138.

Yarahmadzehi, N., & Goodarzi, M. (2020). Investigating The Role of Formative Mobile Based Assessment in Vocabulary Learning of Pre-Intermediate EFL Learners in Comparison with Paper Based Assessment. Turkish Online Journal of Distance Education, 21(1), 181–196.

Zoghi, M. (2013). Computer-based vs . Paper-based Writing Tests : A Comparative Study of Test Taking Strategies. 1(3), 43–56. https://doi.org/10.5296/ijele.v1i3.3559

DOI: https://doi.org/10.31764/paedagoria.v17i4.39449