TY  - JOUR
TI  - Comparison of Inter-Rater Reliability Techniques in Performance-Based Assessment
AB  - The aim of this study is to analyse the importance of the number of raters and compare the results obtained by techniques based on Classical Test Theory (CTT) and Generalizability (G) Theory. The Kappa and Krippendorff alpha techniques based on CTT were used to determine the inter-rater reliability. In this descriptive research data consists of twenty individual investigation performance reports prepared by the learners of the International Baccalaureate Diploma Programme (IBDP) and also five raters who rated these reports. Raters used an analytical rubric developed by the International Baccalaureate Organization (IBO) as a scoring tool. The results of the CTT study show that Kappa and Krippendorff alpha statistical techniques failed to provide information about the sources of the errors causing incompatibility in the criteria. The studies based on G Theory provided comprehensive data about the sources of the errors and increasing the number of raters would also increase the reliability of the values. However, the raters raised the idea that it is important to develop descriptors in the criteria in the rubric.
AU  - Gülleroğlu, Hamide Deniz
AU  - Arslan Mancar, Sinem
DO  - 10.21449/ijate.993805
PY  - 2022
JO  - International Journal of Assessment Tools in Education
VL  - 9
IS  - 2
SN  - 2148-7456
SP  - 515
EP  - 533
DB  - TRDizin
UR  - http://search/yayin/detay/1125652
ER  -