Analyzing the validity and reliability of an assessment tool for senior high school students
Abstract
This study analysed the validity and reliability of an assessment tool designed for the 1st semester of eleventh-grade students in a senior high school setting, employing a descriptive research approach. Based on the principles of validity and reliability, the analysis inspected content, face, and construct validity, along with reliability. Content validity was affirmed due to the alignment of the test items with the intended instructional objectives, face validity was ascertained through the test's apparent measurement of its designed purpose, and construct validity was assessed by determining how effectively the tool measured the inherent characteristics it intended to measure. Using the Guttman Split-Half Coefficient, the reliability showcased a high consistency in the test scores. A detailed analysis using the Statistical Package for the Social Sciences (SPSS) confirmed the validity of the tool as all item correlation coefficients exceeded the critical value. The reliability test revealed that each instrument had scores surpassing established standards, thereby confirming the tool's reliability. In conclusion, this study meticulously eval_uated and confirmed the content, face, construct validity, and reliability of an assessment tool aimed at senior high school students. By demonstrating alignment with the subject's curriculum objectives and effectively measuring students' proficiency in narrative texts, the results underscore the tool's capability to provide educators with accurate and reliable measures of student understanding. The urgency of these findings lies in their potential to enhance assessment practices, ensuring that educational eval_uations not only reflect students' skills and knowledge but also support their continued academic development.
References
Allen, M. J., & Yen, W. M. (2002). Introduction to Measurement Theory. Illinois: Waveland Press.
Brown, J. D. (1996). Testing in Language Programs. New Jersey: Prentice Hall Regents.
Brown, J. D. (2011). Statistics corner: What is a construct? It's the idea in my question, right? JALT Testing & eval_uation SIG Newsletter, 15(3), 8-16.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/10.1007/BF02310555
Cronbach, L. J., & Meehl, P. E. (1971). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281–302. https://doi.org/10.1037/h0040957
De Vaus, D. A. (2002). Surveys in Social Research. London: Routledge.
Downing, S. M. (2006). Twelve steps for effective test development. Handbook of Test Development, 3-25. Mahwah. https://doi.org/10.1080/15305050701813433
George, D., & Mallery, M. (2016). IBM SPSS Statistics 23 Step by Step: A Simple Guide and Reference. New York: Routledge.
Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255-282. https://doi.org/10.1007/BF02288892
Haladyna, T. M., & Downing, S. M. (2004). Construct-irrelevant variance in high-stakes testing. Educational Measurement: Issues and Practice, 23(1), 17–27. https://doi.org/10.1111/j.1745-3992.2004.tb00149.x
Haynes, S. N., Richard, D., & Kubany, E. S. (1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238–247. https://doi.org/10.1037/1040-3590.7.3.238
Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1): 104-121. https://doi.org/10.1177/109442819800100106
Hughes, D. (2017). Ensuring questionnaire reliability. Practice Nursing, 28(3), 125-127.
Kane, M. T. (2006). Validation. Educational Measurement, 4(2), 17-64.
Kline, P. (2005). An Easy Guide to Factor Analysis. New York: Routledge.
Marianti, S., Rufaida, A., Hasanah, N., & Nuryanti, S. (2023). Comparing item-total correlation and item-theta correlation in test item selection: A simulation and empirical study. Jurnal Penelitian dan eval_uasi Pendidikan, 27(2), 133-145. https://doi.org/10.21831/pep.v27i2.61477
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741-749. https://doi.org/10.1037/0003-066X.50.9.741
Novick, M. R. (1966). The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3(1), 1-18. https://doi.org/10.1016/0022-2496(66)90002-2
Nunnally, J. C. (1978). Psychometric Theory (2nd ed.). New York: McGraw-Hill.
Smith, J., & Jones, M. (2010). Assessing face validity: Principles and practices. Journal of Educational Assessment, 25(2), 47-59.
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
Trochim, W. M. (2006). The Research Methods Knowledge Base. Ohio: Atomic Dog Publishing.
