Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory

GÜLER, Neşe; Tasdelen Teker, Gulsen; UYANİK, Gulden Kaya

Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory

GÜLŞEN TAŞDELEN TEKER, (Sakarya Üniversitesi, Sakarya, Türkiye)

Neşe GÜLER, (Sakarya Üniversitesi, Sakarya, Türkiye)

Gulden Kaya UYANİK (Sakarya Üniversitesi, Sakarya, Türkiye)

Kuram ve Uygulamada Eğitim Bilimleri

4 0

Yıl: 2015 Cilt: 15 Sayı: 3 Sayfa Aralığı: 635 - 645 Metin Dili: İngilizce İndeks Tarihi: 29-07-2022

Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory

Öz:

Generalizability theory (G theory) provides a broad conceptual framework for social sciences such as psychology and education, and a comprehensive construct for numerous measurement events by using analysis of variance, a strong statistical method. G theory, as an extension of both classical test theory and analysis of variance, is a model which can deal with multiple sources of error. In conducting the analysis of the G theory, there are several software programs that can be used such as GENOVA, SPSS, SAS, EduG, and G-String. In this study, the general perspectives of G theory are first explained broadly. Then, the SPSS and EduG software programs are used to conduct generalizability analyses on the data obtained from the answers of 30 students (p) to nine open-ended questions (i) as rated by three raters (r). There are three different designs in the study. Two of them are random effects designs, pxixr and pxi:r, and the last one is pxixr design using a fixed rater . According to the findings from the study, SPSS and EduG give the same results for variance component estimates as well as for G (Generalizability) and D (Decision) studies of all designs, as expected. Besides comparing the program outputs, their weaknesses and strengths were also discussed regarding different designs and data sets in this study

Anahtar Kelime:

Konular: Eğitim, Eğitim Araştırmaları

Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Bibliyografik

Brennan, R. L. (1992). Elements of generalizability theory (rev. ed.). Iowa City, IA: American College Testing.
Brennan, R. L. (2000). Performance assessment from the perspective of generalizability theory. Applied Psychological Measurement, 24(4), 339-353.
Brennan, R. L. (2001a). Generalizability theory. New York, NY: Springer-Verlag.
Brennan, R. L. (2001b). Manual for mGENOVA Version 1. Iowa Testing Programs Occasional Papers Number 50.
Iowa City, IA: Iowa Testing Programs, University of Iowa. Brennan, R. L. (2001c). Manual for urGENOVA Version 1. Iowa Testing Programs Occasional Paper Number 49.
Iowa City, IA: Iowa Testing Programs, University of Iowa. Brennan, R. L. (2011). Generalizability theory and classical test theory. Applied Measurement in Education,. 24, 1-21.
Cardinet, J., Johnson, S., & Pini, G. (2010). Applying generalizability theory using EduG. New York, NY: Routledge - Taylor & Francis Group.
Crick J. E., & Brennan R. L. (1983). Manual for GENOVA:A generalized analysis of variance system. Iowa City, IA: The American College Testing Program.
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York, NY: Harcourt Brace Javanovich College Publishers.
Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York, NY: Wiley.
Derstine, P. (2007, August). Which software program for generalizability studies is best? Comparing G_String II and EduG. Paper presented at the eighth Annual Master of Health Professions Education Summer Conference, College of Medicine West, USA.
Guler, N. (2009). Generalizability theory and comparison of the results of G and D studies computed by SPSS and GENOVA packet programs. Education and Science, (154), 93-103.
Guler, N., Kaya Uyanik, G., & Tasdelen Teker, G. (2012). Generalizability theory. Ankara: PegemAkademi Publishing.
Hsu, L. (2012). Applications of generalizability theory to estimate the reliability of EFL learners' performance-based assessment: A preliminary study. Educational Research, (2), 145-154.
Kretchmar, J. (2006). Assessing the reliability of ratings used in undergraduate admission decision. Journal of College Admission, 192, 10-15.
Mushquash, C., & O'Connor, B. P. (2006). SPSS and SAS programs for generalizability theory analysis. Behavior Research Methods, 38(3), 542-547.
Nalbantoglu Yilmaz, F. (2014, June). Which program suitable to use G Theory analysis? Paper presented at the fourth annual Conference of Measurement and Evaluation in Education, Hacettepe University, Turkey.
Ogretmen, T., & Acar, T. (2014). Estimation of generalizability coefficients: An application of structural equation modeling. Journal of Education and Practice, (14), 113-119.
Shavelson, J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
Yelboga, A. (2011, July). Investigation of generalizability theory analysis results with different statistical programs. Poster presented at the XII. European Congress of Psychology, Istanbul, Turkey.
Yin, Y., & Shavelson, J. (2008). Application of generalizability theory to concept map assessment research. Applied Measurement in Education, 21, 273-291.

APA	Tasdelen Teker G, GÜLER N, UYANİK G (2015). Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. , 635 - 645.
Chicago	Tasdelen Teker Gulsen,GÜLER Neşe,UYANİK Gulden Kaya Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. (2015): 635 - 645.
MLA	Tasdelen Teker Gulsen,GÜLER Neşe,UYANİK Gulden Kaya Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. , 2015, ss.635 - 645.
AMA	Tasdelen Teker G,GÜLER N,UYANİK G Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. . 2015; 635 - 645.
Vancouver	Tasdelen Teker G,GÜLER N,UYANİK G Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. . 2015; 635 - 645.
IEEE	Tasdelen Teker G,GÜLER N,UYANİK G "Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory." , ss.635 - 645, 2015.
ISNAD	Tasdelen Teker, Gulsen vd. "Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory". (2015), 635-645.

APA	Tasdelen Teker G, GÜLER N, UYANİK G (2015). Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. Kuram ve Uygulamada Eğitim Bilimleri, 15(3), 635 - 645.
Chicago	Tasdelen Teker Gulsen,GÜLER Neşe,UYANİK Gulden Kaya Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. Kuram ve Uygulamada Eğitim Bilimleri 15, no.3 (2015): 635 - 645.
MLA	Tasdelen Teker Gulsen,GÜLER Neşe,UYANİK Gulden Kaya Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. Kuram ve Uygulamada Eğitim Bilimleri, vol.15, no.3, 2015, ss.635 - 645.
AMA	Tasdelen Teker G,GÜLER N,UYANİK G Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. Kuram ve Uygulamada Eğitim Bilimleri. 2015; 15(3): 635 - 645.
Vancouver	Tasdelen Teker G,GÜLER N,UYANİK G Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory. Kuram ve Uygulamada Eğitim Bilimleri. 2015; 15(3): 635 - 645.
IEEE	Tasdelen Teker G,GÜLER N,UYANİK G "Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory." Kuram ve Uygulamada Eğitim Bilimleri, 15, ss.635 - 645, 2015.
ISNAD	Tasdelen Teker, Gulsen vd. "Comparing the Effectiveness of SPSS and EduG using Different Designs for Generalizability Theory". Kuram ve Uygulamada Eğitim Bilimleri 15/3 (2015), 635-645.