Development of a Numeracy Assessment Instrument for Vocational High School (VHS) Students

Jumini Jumini, Kana Hidayati

Abstract


Numeracy  skills  are  essential  components  that  must  be  mastered  by  Vocational  High  School (VHS)  graduates  to  enter  the  workforce.  Developing a  quality  instrument  based  on  the  Item Response Theory (IRT) approach is necessary to measure numeracy skills. Since the development process involved VHS students, the instrument is developed based on the characteristics of the users. This study aims to (1) develop a quality numeracy assessment instrument for VHS students and (2) describe the numeracy skill profile of VHS eleven graders in Sleman Regency based on the assessment result. The subjects of the wide-scale trial were 374 students from nine VHSs in the Sleman  Regency  of  Indonesia,  assessed  in  nine  study  areas.  Data  analysis  used  the  Item Response Theory (IRT) of the Quest program with the outputs: (1) An instrument consisting of 35  questions  involving multiple-choice,  complex  multiple-choice,  short answer,  and  essay.  All items are valid qualitatively based on expert judgment and quantitatively based on infit and outfit MNSQ value and unidimensionality. The test item reliability value is 0.96, including in the special category,  and  the  person  reliability estimateis  0.89,  categorized  as  good.  Based  on  Total Information Function (TIF) and Standard Error of Measurement (SEM), the instrument is reliable in estimating the numeracy skills of the students with the ability ranging from -3.2 to 3.3 logit or covering the ability categories of low, medium, and high. The difficulty level of all items is in the range  of -2.0  to  2.0,  falling  into  the  good  category.  (2)  Most  students,  covering  67%  of  VHS students in Sleman Regency, have numeracy skills at the Basic level.


Keywords


assessment; instrument; numeracy; vocational high school

Full Text:

PDF

References


Adams, R. J., & Khoo, S. T. (1996). Quest: The interactive test analysis system version 2.1.The Australian Council for Educational Research.

Balitbang Kemdikbud. (2019). Laporan Nasional PISA 2018 Indonesia. Kemdikbud RI.

Doig, B., McCrae, B., & Rowe, K. (2003). A Good Start to Numeracy. COmmonwealth of Australia.

Futri, V. I., & Rosnawati, R. (2022). Pengembangan Instrumen Penilaian Kemampuan Numerasi Peserta Didik SMP pada Pembelajaran Matematika[Universitas Negeri Yogyakarta]. https://eprints.uny.ac.id/73262/

Green, D. A., & Riddell, W. C. (2012). Understanding Educational Impacts: The Role of Literacy and Numeracy Skills. 11th IZA/SOLE Transatlantic Meeting of Labor Economists.

Hambleton, R. K., & Swaminathan, H. (1985a). Item Response Theory Principles and Aplications. Kluwer Nijhoff Publishing.

Hambleton, R. K., & Swaminathan, H. (1985b). Items Response Theory: Principles and Application. Kluwer-Nijjhoff Publish.

Hooper, D., Couglan, J., & Mullen, M. R. (2008). Structural Equation Modelling: Guidelines for Determining Model Fit. Electronic Journal of Business Research Methods, 6(1), 53–60. https://www.researchgate.net/publication/254742561_Structural_Equation_Modeling_Guidelines_for_Determining_Model_Fit

Istiyono, E. (2018). Pengembangan Instrumen Penilaian dan Analisis Hasil Belajar Fisika Dengan Teori Tes Klasik dan Modern(First). UNY Press.

Jelatu, S., Mon, E. M., & San, S. (2019). Relasi Antara Kemampuan Numerik Dengan Prestasi Belajar Matematika. Lectura:Jurnal Pendidikan, 10(1). https://doi.org/10.31849/lectura.v10i1.2390

Keeves, J. P., & Alagumalai. (1999). New Approach to Measurement. Pergamon, An Imprint of Elsevier Science.

Kemdikbud RI. (2017). Materi Pendukung Literasi Numerasi. Kemdikbud RI.

Kemdikbud RI. (2020). AKM dan Implikasinya pada Pembelajaran. Pusat Asesmen dan Pembelajaran Badan Penelitian dan Pengembangan dan Perbukuan Kementerian Pendidikan dan Kebudayaan Republik Indonesia.

Kurniawan, A. P., Budiarto, M. T., & Ekawati, R. (2022). Pengembangan Soal Numerasi Berbasis Konteks Nilai Budaya Primbon Jawa. JRPM: Jurnal Review Pembelajaran Matematika, 7(1), 20–34. https://doi.org/10.15642/jrpm.2022.7.1.20-34

Mardapi, D. (2012). Pengukuran Penilaian dan Evaluasi Pendidikan. Nuha Medika.

Myszkowski, N. (2019). Development of the R library “jrt” : Automated Item-Response Theory procedures for judgment data and their application with the Consensual Assessment Technique. Psychology of Aesthetics Creativity and The Arts. https://doi.org/10.1037/aca0000287

Naga, D. S. (1992). Pengantar Teori Skor Pada Pengukuran Pendidikan. Penerbit Gunadharma.

Ojerinde, D. (2013). Classical Test Theory (CTT) VS Item Response Theory (IRT): An Evaluation of The Comparability of Item Analysis Result. Semantic Scholar. https://www.semanticscholar.org/paper/Classical-Test-Theory-(-CTT-)-VS-Item-Response-(-)-Ojerinde/aa27637d699b7e26ee1f7578d0fe2b8173ad769f

Oriondo, L. L., & Antonio, D. E. M. (1998). Evaluation Educational Outcomes. Rex Printing Compagny.

Perdana, S. A. (2018). Analisis Kualitas Instrumen Pengukuran Pemahaman Konsep Persamaan Kuadrat Melalui Teori Tes Klasik Dan Rasch Model. Jurnal Kiprah, 6(1), 41–48.

Pulungan, D. A. (2014). Pengembangan Instrumen Tes Literasi Matematika Model PISA. Journal of Educational Research and Evaluation, 3(2).

Purnama, D. N., & Alfarisa, F. (2020). Karakteristik Butir Soal Try Out Teori Kejuruan Akuntansi SMK Berdasarkan Teori Tes Klasik dan Teori Respons Butir. Jurnal Pendidikan Akuntansi Indonesia, 18(1), 36–46. https://doi.org/10.21831/jpai.v18i1.31457

Reise, S. P. (1990). A comparison of item-and person-fit methods of assessing model-data fit in IRT. Applied Psychological Measurement, 14(2), 127–137. https://doi.org/10.1177/2F014662169001400202

Retnawati, H. (2014). Teori Respons Butir dan Penerapannya. Parama Publishing.

Retnawati, H. (2015). Perbandingan Estimasi Kemampuan Laten antara Metode Maksimum Likelihood dan Metode Bayes. Jurnal Penelitian Dan Evaluasi Pendidikan, 19(2). https://doi.org/10.21831/pep.v19i2.5575

Retnawati, H. (2016). Validitas Reliabilitas & Karakteristik Butir. Parama Publishing.

Setyawarno, D. (2017). Upaya Peningkatan Kualitas Butir Soal dengan Analisis Aplikasi quest. Fakultas Matematika dan Ilmu Pengetahuan Alam Universitas Negeri Yogyakarta.

Suciati, Munadi, S., Sugiman, Ratna, W. D., & Febriyanti. (2020). Design and Validation of Mathematical Literacy Instruments for Assessment for Learning in Indonesia. European Journal of Educational Research, 9(2), 865–875. https://doi.org/10.12973/eu-jer.9.2.865

Sudjana, N. (2012). Penilaian hasil Proses Belajar Mengajar. Remaja Rosdakarya.

Sumintono, B., & Widhiarso, W. (Eds.). (2014). Aplikasi Model Rasch untuk Penelitian Ilmu-ilmu Sosial (Edisi Revisi). Trim Komunikata Publishing House.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi Pemodelan Rasch pada Assessment Pendidikan. Trim Komunikata Publishing House.

Suprawata, I. G. (2022). Pengembangan Tes Numerasi Guru (Tugu) untuk Mengukur Kemampuan Numerasi Guru SD dengan Konteks Lingkungan Sekolah[Universitas Pendidikan Ganesha]. https://repo.undiksha.ac.id/10663/

Tukiran, M. (2020). Filsafat Manajemen Pendidikan. Kanisius.

Wiberg, M. (2004). Classical Test Theory vs. Item Response Theory. Umea Universitet.

Wright, B., & Stone, M. (1999). Measurement Essentials(2nd ed.). Wide Range.




DOI: https://doi.org/10.46517/seamej.v14i1.306

Refbacks

  • There are currently no refbacks.



Indexed by:

      


Southeast Asian Mathematics Education Journal
SEAMEO Regional Centre for QITEP in Mathematics
Jl. Kaliurang Km 6, Sambisari, Condongcatur, Depok, Sleman
Yogyakarta, Indonesia
Telp. +62 274 889955
Email: seamej@qitepinmath.org


p-ISSN: 2089-4716 | e-ISSN: 2721-8546


Southeast Asian Mathematics Education Journal is licensed under a Creative Commons Attribution 4.0 International License

View My Stats

 Supported by: