Usability Evaluation of the Online Skill Assessment Tool

Merve Yıldız, (Karadeniz Teknik Üniversitesi, Uzaktan Eğitim Uygulama ve Araştırma Merkezi, Trabzon, Türkiye)
MUHAMMET BERIGEL, (Karadeniz Teknik Üniversitesi, Yönetim Bilişim Sistemleri Daire Başkanlığı, Trabzon, Türkiye)
Furkan Kalyoncu, (Karadeniz Teknik Üniversitesi, Teknoloji Transferi Uygulaması Daire Başkanlığı Araştırma Merkezi, Trabzon, Türkiye)
Ozlem Ozgenc (Karadeniz Teknik Üniversitesi, Orman Ürünleri Mühendisliği Bölümü, Trabzon, Türkiye)
Acta Infologica
11 2
Yıl: 2022 Cilt: 6 Sayı: 1 Sayfa Aralığı: 19 - 32 Metin Dili: İngilizce DOI: 10.26650/acin.1077400 İndeks Tarihi: 23-08-2022

Usability Evaluation of the Online Skill Assessment Tool

Öz:
This study was conducted to evaluate the usability of the online skill assessment tool which was developed to evaluate the literacy, numeracy, and digital skills of adults with low skill levels. The sample consisted of 14 participants who were between the ages of 18 and 40 and who had graduated from secondary school at most. In the study, the user-based method, in which user experiences are employed, was used, and according to the effectiveness, efficiency and satisfaction criteria determined by ISO 9241-11, the usability of the online skill assessment tool was evaluated. According to the data obtained from the screen recordings, effectiveness was considered as the completion rate of the tasks, and efficiency was considered as the completion duration of the tasks. Satisfaction was also considered as the values obtained from the questionnaire data. The findings indicated that the participants were able to complete most of the tasks and were generally satisfied with the use of the system. Some usability problems were identified as a result of the observations and the detailed examination of the screen recordings. These problems have been a guide for updates to render the system more usable before the system is used by end users.
Anahtar Kelime:

Online Beceri Değerlendirme Aracının Kullanılabilirliğinin Değerlendirilmesi

Öz:
Bu çalışmada, düşük beceri seviyesine sahip yetişkinlerin okuryazarlık, sayısal ve dijital becerilerinin değerlendirilebilmesi için geliştirilen online beceri değerlendirme aracının kullanılabilirliğinin değerlendirilmesi amaçlanmıştır. Bu amaç doğrultusunda, çalışma grubunu, 18-40 yaşları arasında ve en fazla ortaokul mezunu olan 14 katılımcı oluşturmaktadır. Çalışmada, kullanıcı deneyimlerinin işe koşulduğu kullanıcı tabanlı yöntem kullanılmış ve online beceri değerlendirme aracının kullanılabilirliğinin değerlendirilmesinde, ISO 9241-11 tarafından belirlenen etkililik, verimlilik ve memnuniyet ölçütleri temel alınmıştır. Bu bağlamda, görevlerin tamamlanma oranı olarak kabul edilen etkililik ile görevlerin tamamlanma süresi olarak kabul edilen verimlilik ekran kayıtlarından elde edilen verilere göre belirlenmiştir. Memnuniyet ise araştırmacılar tarafından geliştirilen anketten elde edilen veriler doğrultusunda belirlenmiştir. Elde edilen bulgular, katılımcıların, görevlerin çoğunluğunu tamamlayabildiklerini ve sistemin kullanımı ile ilgili genel olarak memnun olduklarını göstermiştir. Bunun yanı sıra, kullanılabilirlik testi sırasında yapılan gözlemler ve ekran kayıtlarının detaylı incelenmesi sonucunda bazı kullanılabilirlik problemleri tespit edilmiştir. Bu problemler, sistemin son kullanıcılara açılmadan önce, daha kullanılabilir olması adına sistemin tasarımında yapılacak güncellemeler için yol gösterici olmuştur.
Anahtar Kelime:

Belge Türü: Makale Makale Türü: Araştırma Makalesi Erişim Türü: Erişime Açık
  • Akgül, Y. (2020). Accessibility, usability, quality performance, and readability evaluation of university websites of Turkey: a comparative study of state and private universities. Universal Access in the Information Society, 1-14. doi:10.1007/s10209-020-00715-w.
  • Althobaiti, M. M., & Mayhew, P. (2015, September). Assessing the usability of learning management system: user experience study. In Second International Conference on E-Learning, E-Education, and Online Training (pp. 9-18).
  • Anam, H., Sadiq, M., & Jamil, H. (2020). Development of System Usability Scale (SUS) for the Urdu Language. International Journal of Computer Science and Information Security, 18(6), 73-78.
  • Battal, A., & Çağıltay, K. (2015). Investigation of usage frequency and department on usability ıssues with the Online Examination System (OSSI). Mersin University Journal of the Faculty of Education, 11(3), 752-763.
  • Bayrak, T., & Akcam, B. (2015). Exploring benefits of a web based testing and training tool. Procedia-Social and Behavioral Sciences, 195, 1032-1041. Doi:10.1016/j.sbspro.2015.06.146.
  • Cengiz, E. (2016). Hacettepe Üniversitesi Kütüphaneleri web sitesinin kullanılabilirliğinin eğitimle desteklenmesi ve değerlendirilmesi: Hacettepe Üniversitesi Bilgi ve Belge Yönetimi Bölümü öğrencileri üzerine bir çalışma. Türk Kütüphaneciliği, 30(1), 84-95.
  • Chang, C., & Almaghalsah, H. (2020). Usability evaluation of e-government websites: A case study from Taiwan. International Journal of Data and Network Science, 4(2), 127-138.
  • Chao, J. Y., Chao, S. J., Yao, L. Y., & Liu, C. H. (2016). A case study of design and usability evaluation of the collaborative problem solving instructional platform system. Eurasia Journal of Mathematics, Science and Technology Education, 12(10), 2647-2655. Doi:10.12973/eurasia.2016.1278a.
  • Cheon, J., & Grant, M. (2009). Active listening: Web-based assessment tool for communication and active listening skill development. TechTrends, 53(6), 24-34. Doi:10.1007/s11528-009-0340-4.
  • Çağıltay, K. (2016). İnsan bilgisayar etkileşimi ve eğitim teknolojileri. In K. Çağıltay and Y. Göktaş (Eds.) Öğretim Teknolojilerinin Temelleri: Teoriler Araştırmalar Eğilimler (pp. 297-314). Pegem Akademi, Ankara.
  • de Castro Lozano C. et al. (2011). Usable interface design for everyone. In Cipolla Ficarra FV, de Castro Lozano C, Nicol E, Kratky A, Cipolla-Ficarra M (Eds) Human- Computer Interaction, Tourism and Cultural Heritage (pp. 157-172). Springer, Berlin, Heidelberg. Doi:10.1007/978-3-642-18348-5_15.
  • Dillon, A. (2001). The evaluation of software usability. In Karwowski W (Ed) Encyclopedia of Human Factors and Ergonomics. Taylor and Francis, London. https://repository.arizona.edu/bitstream/handle/10150/105344/Adusability.pdf?sequence=1&isAllowed=y. Accessed 15 October 2020.
  • Dix, A., Finlay, J., Abowd, G., & Beale, R. (2004). Human-computer interaction. Prentice Hall, England.
  • Dumit, N. Y. (2012). Diagnostic/formative/summative assessment. https://vb.ckfu.org/attachments/%E3%E1%CA%DE%EC-%C7%E1%CE%D1%ED% CC%ED%E4/303900d1479934616-%E3%D1%CC%DA-%D4%C7%E3%E1-%E1%DF%DD%C7%ED%C7%CA-%C7%E1%DE%D3%E3- % C 7 % E1% DA % C 7 % E 3 - % E 6 % D F % D D % C 7 % E D % C 7 % C A - % C 7 % E1% C A % C E % D 5 % D 5 - % C 7 % E1% E1% D B % C 9 - %C7%E1%C7%E4%CC%E1%ED%D2%ED%C9-diagnostic-formative-summative-asst.pdf. Accessed 20 October 2020.
  • Ersoy, H. (2004). Bir çevrimiçi öğrenim destek sisteminin kullanılabilirlik testi: Planlama, uygulama, değerlendirme. The Turkish Online Journal of Educational Technology, 3(1), 75-82.
  • Gonçalves, R., Rocha, T., Martins, J., Branco, F., & Au-Yong-Oliveira, M. (2018). Evaluation of e-commerce websites accessibility and usability: An e- commerce platform analysis with the inclusion of blind users. Universal Access in the Information Society, 17(3), 567-583. Doi:10.1007/ s10209-017-0557-5.
  • Granić, A., & Ćukušić, M. (2011). Usability testing and expert inspections complemented by educational evaluation: A case study of an e-learning platform. Journal of Educational Technology & Society, 14(2), 107-123.
  • Gülbahar, Y., Kalelioğlu, F., & Madran, O. (2008). Usability evaluation of “web macerası” as an instructional and evaluation method. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 41(2), 209-236.
  • Güven, Ç. (1987) Okulda öğrenme-öğretme sürecinin denetimi. Ondokuz Mayıs Üniversitesi Eğitim Fakültesi Dergisi, 2(1), 271-285.
  • Harrati, N., Bouchrika, I., Tari, A., & Ladjailia, A. (2016). Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis. Computers in Human Behavior, 61, 463-471. Doi:10.1016/j.chb.2016.03.051.
  • Hasan, L. (2014). The website of the University of Jordan: Usability evaluation. International Arab Journal of e-Technology, 3(4), 258-269.
  • Hasan, L., Morris, A., & Probets, S. (2012). A comparison of usability evaluation methods for evaluating e-commerce websites. Behaviour & Information Technology, 31(7), 707-737. Doi:10.1080/0144929X.2011.596996.
  • Huang, Z., & Benyoucef, M. (2014). Usability and credibility of e-government websites. Government Information Quarterly, 31(4), 584-595. Doi:10.1016/j. giq.2014.07.002.
  • Ismail, A., & Kuppusamy, K. S. (2018). Accessibility of Indian universities’ homepages: An exploratory study. Journal of King Saud University-Computer and Information Sciences, 30(2), 268-278.
  • ISO 9241-11 (1998). Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability. https://pdfs. semanticscholar.org/17e5/3f6fa806ea573e4ee0904fbb12c3a527d5f0.pdf. Accessed 10 October 2020.
  • Kakasevski, G., Mihajlov, M., Arsenovski, S., & Chungurski, S. (2008, June). Evaluating usability in learning management system Moodle. In ITI 2008 - 30th International Conference on Information Technology Interfaces (pp. 613-618).
  • KAMIS (2018). Kullanılabilirlik ilkeleri. https://kamis.gov.tr/wp-content/uploads/2018/03/Bolum3.pdf. Accessed 12 October 2020.
  • Karahoca, A., Karahoca, D., & Günoğlu, S. (2009). Web tabanlı sınav otomasyon sisteminin kullanılabilirlik analizi. In 4. Ulusal Yazılım Mühendisliği Sempozyumu (pp 183-190).
  • Lai, T. L., Chen, P. Y., & Chou, C. Y. (2017, May). A user experience study of a web-based formative assessment system. In 2017 International Conference on Applied System Innovation (pp. 899-902).
  • Lee, S., & Koubek, R. J. (2010). The effects of usability and web design attributes on user preference for e-commerce web sites. Computers in Industry, 61(4), 329-341. Doi:10.1016/j.compind.2009.12.004.
  • Li, F., & Li, Y. (2011). Usability evaluation of e-commerce on B2C websites in China. Procedia Engineering, 15, 5299-5304. Doi:10.1016/j.proeng.2011.08.982.
  • Lin, H. C. K., Hsieh, M. C., Wang, C. H., Sie, Z. Y., & Chang, S. H. (2011). Establishment and usability evaluation of an interactive AR learning system on conservation of fish. Turkish Online Journal of Educational Technology, 10(4), 181-187.
  • Lyzara, R., Purwandari, B., Zulfikar, M. F., Santoso, H. B., & Solichah, I. (2019, January). E-government usability evaluation: Insights from a systematic literature review. In Proceedings of the 2nd International Conference on Software Engineering and Information Management (pp 249-253).
  • Majzub, R. M. (2009). The development of a web based ecological assessment of school readiness (WEBEASR). Procedia-Social and Behavioral Sciences, 1(1), 2568-2572. Doi:10.1016/j.sbspro.2009.01.453.
  • Mentes, S. A., & Turan, A. H. (2012). Assessing the usability of university websites: An empirical study on Namik Kemal University. Turkish Online Journal of Educational Technology, 11(3), 61-69.
  • Newell, A. F., & Gregor, P. (2002). Design for older and disabled people–where do we go from here?. Universal Access in the Information Society, 2(1), 3-7. Doi:10.1007/s10209-002-0031-9.
  • Nielsen, J. (1994). Usability engineering. Morgan Kaufmann. Ogolla, J. A. (2011). Usability evaluation: Tasks susceptible to concurrent think-aloud protocol. Dissertation, Linköping University.
  • Pant, A. (2015). Usability evaluation of an academic library website: Experience with the Central Science Library, University of Delhi. The Electronic Library, 33(5), 896-915. Doi:10.1108/EL-04-2014-0067.
  • Paul, S., & Das, S. (2019). Accessibility and usability analysis of Indian e-government websites. Universal Access in the Information Society, 19, 949–957. Doi:10.1007/s10209-019-00704-8.
  • Pensabe-Rodriguez, A., Lopez-Dominguez, E., Hernandez-Velazquez, Y., Dominguez-Isidro, S., & De-la-Calleja, J. (2020). Context-aware mobile learning system: Usability assessment based on a field study. Telematics and Informatics, 48, 101346. Doi: 10.1016/j.tele.2020.101346.
  • Phongphaew, N., & Jiamsanguanwong, A. (2017, July). Usability evaluation on learning management system. In International Conference on Applied Human Factors and Ergonomics (pp 39-48).
  • Sagar, K., & Saha, A. (2017). A systematic review of software usability studies . International Journal of Information Technology. Doi:10.1007/ s41870-017-0048-1.
  • Seffah, A., Donyaee, M., Kline, R. B., & Padda, H. K. (2006). Usability measurement and metrics: A consolidated model. Software Quality Journal, 14(2), 159-178. Doi:10.1007/s11219-006-7600-8.
  • Senemoğlu, N. (2013) Gelişim, öğrenme ve öğretim. Kuramdan uygulamaya (23.baskı). Yargı Yayınevi, Ankara.
  • Shi, L., Al Qudah, D., Qaffas, A., & Cristea, A. I. (2013, June). Topolor: a social personalized adaptive e-learning system. In International Conference on User Modeling, Adaptation, and Personalization (pp 338-340).
  • Story, M. F., Mueller, J. L., & Mace, R. L. (1998). The universal design file: Designing for people of all ages and abilities. https://files.eric.ed.gov/fulltext/ ED460554.pdf. Accessed 15 October 2020.
  • Şahin, İ., Hebebci, M. T., & Çelik, İ. (2014, May). Çevrimiçi öğrenme ortamlarında kullanılabilirlik: Bir literatür tarama çalışması [Usability in the online learning environments: A literature review study]. In International Conference on Education in Mathematics, Science & Technology (pp 1354-1362).
  • TÜİK (2019). Sorgulama ekranı. https://biruni.tuik.gov.tr/medas/. Accessed 11 October 2020.
  • Vairamuthu, S., & Anouncia, S. M. (2016). Reconnoitering students’ satisfaction of an online based assessment system to improve usability using PSO: An examination into a problem solving and programming course. Engineering, Technology & Applied Science Research, 6(5), 1207-1211. Doi:10.48084/ etasr.800.
  • Van Den Haak, M., De Jong, M., & Jan Schellens, P. (2003). Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue. Behaviour & Information Technology, 22(5), 339-351. Doi: 10.1080/0044929031000.
  • Ventayen, R. J. M., Estira, K. L. A., De Guzman, M. J., Cabaluna, C. M., & Espinosa, N. N. (2018). Usability evaluation of google classroom: Basis for the adaptation of gsuite e-learning platform. Asia Pacific Journal of Education, Arts and Sciences, 5(1), 47-51.
  • Yıldırım, A. & Şimşek, H. (2013). Sosyal bilimlerde nitel araştırma yöntemleri (9.baskı). Seçkin Yayıncılık, Ankara.
APA YILDIZ M, BERIGEL M, Kalyoncu F, Ozgenc O (2022). Usability Evaluation of the Online Skill Assessment Tool. , 19 - 32. 10.26650/acin.1077400
Chicago YILDIZ MERVE,BERIGEL MUHAMMET,Kalyoncu Furkan,Ozgenc Ozlem Usability Evaluation of the Online Skill Assessment Tool. (2022): 19 - 32. 10.26650/acin.1077400
MLA YILDIZ MERVE,BERIGEL MUHAMMET,Kalyoncu Furkan,Ozgenc Ozlem Usability Evaluation of the Online Skill Assessment Tool. , 2022, ss.19 - 32. 10.26650/acin.1077400
AMA YILDIZ M,BERIGEL M,Kalyoncu F,Ozgenc O Usability Evaluation of the Online Skill Assessment Tool. . 2022; 19 - 32. 10.26650/acin.1077400
Vancouver YILDIZ M,BERIGEL M,Kalyoncu F,Ozgenc O Usability Evaluation of the Online Skill Assessment Tool. . 2022; 19 - 32. 10.26650/acin.1077400
IEEE YILDIZ M,BERIGEL M,Kalyoncu F,Ozgenc O "Usability Evaluation of the Online Skill Assessment Tool." , ss.19 - 32, 2022. 10.26650/acin.1077400
ISNAD YILDIZ, MERVE vd. "Usability Evaluation of the Online Skill Assessment Tool". (2022), 19-32. https://doi.org/10.26650/acin.1077400
APA YILDIZ M, BERIGEL M, Kalyoncu F, Ozgenc O (2022). Usability Evaluation of the Online Skill Assessment Tool. Acta Infologica, 6(1), 19 - 32. 10.26650/acin.1077400
Chicago YILDIZ MERVE,BERIGEL MUHAMMET,Kalyoncu Furkan,Ozgenc Ozlem Usability Evaluation of the Online Skill Assessment Tool. Acta Infologica 6, no.1 (2022): 19 - 32. 10.26650/acin.1077400
MLA YILDIZ MERVE,BERIGEL MUHAMMET,Kalyoncu Furkan,Ozgenc Ozlem Usability Evaluation of the Online Skill Assessment Tool. Acta Infologica, vol.6, no.1, 2022, ss.19 - 32. 10.26650/acin.1077400
AMA YILDIZ M,BERIGEL M,Kalyoncu F,Ozgenc O Usability Evaluation of the Online Skill Assessment Tool. Acta Infologica. 2022; 6(1): 19 - 32. 10.26650/acin.1077400
Vancouver YILDIZ M,BERIGEL M,Kalyoncu F,Ozgenc O Usability Evaluation of the Online Skill Assessment Tool. Acta Infologica. 2022; 6(1): 19 - 32. 10.26650/acin.1077400
IEEE YILDIZ M,BERIGEL M,Kalyoncu F,Ozgenc O "Usability Evaluation of the Online Skill Assessment Tool." Acta Infologica, 6, ss.19 - 32, 2022. 10.26650/acin.1077400
ISNAD YILDIZ, MERVE vd. "Usability Evaluation of the Online Skill Assessment Tool". Acta Infologica 6/1 (2022), 19-32. https://doi.org/10.26650/acin.1077400
Sayfa Oluşturulma Tarihi
24-04-2024 2:47

İLETİŞİM

TÜBİTAK ULAKBİM TR Dizin

Yüzüncüyıl, İşçi Blokları Mahallesi

Muhsin Yazıcıoğlu Caddesi No:51/C

06530 Çankaya / ANKARA +90 (312) 298 92 00

trdizin@tubitak.gov.tr

TÜBİTAK ULAKBİM Ulusal Akademik Ağ ve Bilgi Merkezi Cahit Arf Bilgi Merkezi © Tüm Hakları Saklıdır. Gizlilik Politikası Kullanım Şartları